AE Experiment Design and Integration
Turn a business objective into an evidence-backed experiment design, an implementation contract, and, when requested and supported, an AE experiment draft.
Hard boundaries
- Use for every AE/TE platform interaction. Do not substitute raw HTTP, browser automation, direct database queries, or application SDKs.
- Do not infer an SDK request from the fact that an experiment needs implementation. Load SDK references only when the user explicitly asks about an A/B experiment SDK or client SDK integration.
- Do not copy general tracking SDK documentation into this Skill. Route generic initialization, event reporting, , user identity, user properties, data upload, LogBus, and REST questions to
ae-data-integration-helper
- Use only the event-metric calculation contracts defined in
. Bind one confirmed primary event metric and do not add
unsupported metric roles.
- Never claim that a platform asset exists, was created, passed a check, or generated project code without a successful response.
- Never invent platform IDs, schemas, SDK APIs, versions, defaults, or behavior.
Progressive disclosure router
Read only the references needed for the current request:
| Request | Required references |
|---|
| Any AE/TE platform read or write | references/platform-operations.md
|
| Create or reuse an experiment draft | references/experiment-creation.md
references/platform-operations.md
|
| Metric selection, feasibility, or creation | references/metric-readiness.md
|
| Explicit A/B experiment SDK or client SDK integration request | references/sdk-integration.md
|
| Exact SDK version, dependency, class, or source lookup | |
| Cross-platform Feature, default, fetch, cache, or assignment behavior | references/experiment-sdk-contract.md
|
| Android, iOS, or JavaScript experiment SDK | references/client-experiment-sdk.md
|
| Server-side assignment or evaluation | references/server-experiment-sdk.md
|
| Server assignment with client rendering | references/hybrid-experiment-sdk.md
|
| Exposure design, deduplication, or metric join | references/exposure-contract.md
|
| SDK retrieval, default, identity, exposure, or debug issue | references/sdk-troubleshooting.md
|
References are a curated fast path, not the whole documentation set.
Workflow
1. Frame the decision
Extract:
- business goal and desired direction;
- experiment variable and user-visible change;
- target population and exclusions;
- decision that the result must support;
- success threshold.
Convert these into a falsifiable hypothesis. Clarify only missing facts that materially change the design. For a conversion goal, establish the population, denominator or exposure behavior, numerator behavior, attribution window, and analysis unit.
Do not silently invent a target population, conversion definition, or technical platform.
2. Resolve the project and evidence
Pass the project gate in
. With
, establish candidate exposure and outcome events, assignment identity and join path, timestamps, exact saved-metric definitions, and—when available—baseline and eligible traffic.
If the project is unavailable, accept user-provided schemas or definitions and label all platform-dependent conclusions as unverified.
3. Assess metric readiness
Apply
. Classify candidates as
,
,
, or
; recommend one primary event metric and confirm its
calculation code before planning sample size or duration.
4. Design Feature, assignment, and groups
Define the Feature key, type, typed default, ownership, stable assignment unit, one control group, treatment groups, group values, traffic, allocations, layer, targeting, and exclusions.
Require allocations totaling
, experiment traffic in
, type-correct values, a stable exposure-to-outcome identity join, and at least one primary metric. Resolve real Features and layers with
before reuse or creation.
5. Calculate sample size and duration
Follow this order: confirm the primary metric and calculation code → obtain its
baseline, MDE, and any required variance → calculate the sample target with
scripts/calculate_experiment_plan.py
→ derive duration from the sample target and effective eligible daily units.
Apply the preregistered planning policy in
. Use fixed
and two-sided testing, policy-default
, and Bonferroni
planning for multiple treatments. Require the MDE type and direction; never
default MDE to 5% or assume variance for a continuous metric.
When experiment traffic is already confirmed, calculate its duration. When it
is not confirmed, obtain verified layer capacity and let the script recommend
the smallest absolute traffic candidate that reaches the target within the
maximum runtime. Default to at least seven days and full-week alignment. Return
the actual infeasible duration instead of truncating it. Explain the baseline,
MDE, power source, allocations, multiplicity rule, traffic evidence, sample
targets, duration adjustment, and any native-report mismatch. Do not return a
definitive plan when required evidence is unavailable.
6. Materialize the design
For an explicit draft-creation request, follow
and
. Create only authorized draft assets, verify the saved result by reading it back, run supported readiness and conflict checks, and return the compact receipt and experiment link defined there.
Submitting, starting, changing live traffic, pausing, ending, or deleting requires separate explicit confirmation. Never turn draft creation into launch.
Output requirements
- Use the language explicitly requested by the user. Otherwise, use the language of the user's latest substantive message.
- Localize all user-visible prose, including headings, table headers, field labels, status names, recommendations, warnings, assumptions, and next actions.
- Keep code, commands, raw IDs, event/property/metric names, Feature keys, SDK/API names, and official enum values unchanged when translation would alter their technical meaning.
- Treat section names in this Skill as semantic guidance, not literal output text. Do not copy an English heading into a non-English response.
- For a design request, lead with the experiment recommendation. For a creation, validation, or conflict-check request, lead with the operation outcome.
- Include only the smallest set of relevant sections; do not reproduce every workflow stage.
- Separate observed platform evidence, verified documentation, deterministic calculations, design judgments, and unresolved assumptions.
- Before responding, check every heading, table header, label, and status for unintended mixed-language output.
Treat project resolution, metadata discovery, candidate-event searches, metric comparison, Feature and layer inventory, capability discovery, schema inspection, and command execution as internal working context.
- Do not narrate the execution sequence in the final answer. Omit phrases such as "first load the reference", "now query in parallel", "verified with ae-cli", or "the evidence collection is complete".
- Do not expose raw commands, capability IDs, request schemas, full candidate lists, or a platform-evidence dump unless the user explicitly asks for the evidence, audit trail, or debugging details.
- Surface platform evidence only when it changes the design, blocks the operation, reveals a material semantic mismatch, or requires user confirmation. Summarize it in at most three concise bullets by default.
- Do not repeat the full experiment design after a creation request unless the user explicitly asks for the complete design.
- Do not expose hidden reasoning. Give the conclusion, the user-relevant basis, and the action result.
Failure behavior
- Missing project or ambiguous host: show candidates and ask; do not guess.
- Missing metadata: return the required event, property, identity, and timestamp checklist.
- Experiment product unavailable:
- State that the project has not enabled the experiment product only when an
explicit platform entitlement result establishes that fact. A missing
capability alone means the experiment capability is unavailable, not that
the product was not purchased.
- For a design request, tell the user that experiment design can continue,
but Feature, layer, metric, and traffic details cannot be verified on the
platform. Continue with an offline design and request the baseline, MDE,
and eligible daily units when sample-size or duration planning needs them.
- For a draft-creation request, lead with the outcome that the experiment
draft was not created. Explain that Feature, layer, and draft creation are
blocked, preserve the proposed design, and say that platform creation and
readiness checks can continue after the product is enabled or the required
access is granted.
- HTTP 403 or equivalent permission denial: state that the current account
lacks the required experiment permission, stop dependent writes, and explain
that this result does not establish whether the project purchased the
experiment product. Ask the project administrator to check both product
availability and the user's project permissions.
- Capability gap without an explicit entitlement or permission result: report
that the current environment does not expose the required experiment
capability, continue with an offline design when useful, and do not bypass
.
- For SDK gaps or conflicts, follow ; do not invent exact code.
- Validation failure: correct documented input or ask for the missing value; do not retry unchanged input.
- Partial success: report created and failed assets separately and never imply atomic success.