ae-experiment-design

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

AE Experiment Design and Integration

AE实验设计与集成

Turn a business objective into an evidence-backed experiment design, an implementation contract, and, when requested and supported, an AE experiment draft.
将业务目标转化为基于证据的实验设计、实现契约,以及在用户要求且支持的情况下,生成AE实验草稿。

Hard boundaries

硬性边界

  • Use
    ae-cli
    for every AE/TE platform interaction. Do not substitute raw HTTP, browser automation, direct database queries, or application SDKs.
  • Do not infer an SDK request from the fact that an experiment needs implementation. Load SDK references only when the user explicitly asks about an A/B experiment SDK or client SDK integration.
  • Do not copy general tracking SDK documentation into this Skill. Route generic initialization, event reporting,
    track
    , user identity, user properties, data upload, LogBus, and REST questions to
    ae-data-integration-helper
    when that Skill is available.
  • Use only the event-metric calculation contracts defined in
    metric-readiness.md
    . Bind one confirmed primary event metric and do not add unsupported metric roles.
  • Never claim that a platform asset exists, was created, passed a check, or generated project code without a successful
    ae-cli
    response.
  • Never invent platform IDs, schemas, SDK APIs, versions, defaults, or behavior.
  • 所有AE/TE平台交互均使用
    ae-cli
    。请勿使用原始HTTP、浏览器自动化、直接数据库查询或应用SDK替代。
  • 请勿因实验需要实现就推断用户有SDK相关需求。仅当用户明确询问A/B实验SDK或客户端SDK集成时,才加载SDK相关参考内容。
  • 请勿将通用追踪SDK文档复制到本Skill中。当
    ae-data-integration-helper
    Skill可用时,将通用初始化、事件上报、
    track
    、用户身份、用户属性、数据上传、LogBus及REST相关问题转至该Skill处理。
  • 仅使用
    metric-readiness.md
    中定义的事件指标计算契约。绑定一个已确认的主事件指标,且不得添加不支持的指标角色。
  • 除非获得
    ae-cli
    的成功响应,否则不得声称平台资产已存在、已创建、通过检查或生成了项目代码。
  • 不得虚构平台ID、 schema、SDK API、版本、默认值或行为。

Progressive disclosure router

渐进式披露路由

Read only the references needed for the current request:
RequestRequired references
Any AE/TE platform read or write
references/platform-operations.md
Create or reuse an experiment draft
references/experiment-creation.md
and
references/platform-operations.md
Metric selection, feasibility, or creation
references/metric-readiness.md
Explicit A/B experiment SDK or client SDK integration request
references/sdk-integration.md
Exact SDK version, dependency, class, or source lookup
references/sdk-index.md
Cross-platform Feature, default, fetch, cache, or assignment behavior
references/experiment-sdk-contract.md
Android, iOS, or JavaScript experiment SDK
references/client-experiment-sdk.md
Server-side assignment or evaluation
references/server-experiment-sdk.md
Server assignment with client rendering
references/hybrid-experiment-sdk.md
Exposure design, deduplication, or metric join
references/exposure-contract.md
SDK retrieval, default, identity, exposure, or debug issue
references/sdk-troubleshooting.md
References are a curated fast path, not the whole documentation set.
仅读取当前请求所需的参考内容:
请求类型所需参考内容
任何AE/TE平台读写操作
references/platform-operations.md
创建或复用实验草稿
references/experiment-creation.md
references/platform-operations.md
指标选择、可行性分析或创建
references/metric-readiness.md
明确的A/B实验SDK或客户端SDK集成请求
references/sdk-integration.md
精确的SDK版本、依赖、类或源码查询
references/sdk-index.md
跨平台Feature、默认值、获取、缓存或分配行为
references/experiment-sdk-contract.md
Android、iOS或JavaScript实验SDK
references/client-experiment-sdk.md
服务端分配或评估
references/server-experiment-sdk.md
服务端分配结合客户端渲染
references/hybrid-experiment-sdk.md
曝光设计、去重或指标关联
references/exposure-contract.md
SDK检索、默认值、身份、曝光或调试问题
references/sdk-troubleshooting.md
参考内容是经过筛选的快速路径,而非完整文档集。

Workflow

工作流程

1. Frame the decision

1. 明确决策框架

Extract:
  • business goal and desired direction;
  • experiment variable and user-visible change;
  • target population and exclusions;
  • decision that the result must support;
  • success threshold.
Convert these into a falsifiable hypothesis. Clarify only missing facts that materially change the design. For a conversion goal, establish the population, denominator or exposure behavior, numerator behavior, attribution window, and analysis unit.
Do not silently invent a target population, conversion definition, or technical platform.
提取以下信息:
  • 业务目标与预期方向;
  • 实验变量与用户可见变更;
  • 目标人群与排除规则;
  • 实验结果需支持的决策;
  • 成功阈值。
将上述信息转化为可证伪的假设。仅对会实质性改变设计的缺失信息进行澄清。对于转化目标,需确定人群范围、分母或曝光行为、分子行为、归因窗口及分析单元。
请勿擅自虚构目标人群、转化定义或技术平台。

2. Resolve the project and evidence

2. 确认项目与依据

Pass the project gate in
platform-operations.md
. With
ae-cli
, establish candidate exposure and outcome events, assignment identity and join path, timestamps, exact saved-metric definitions, and—when available—baseline and eligible traffic.
If the project is unavailable, accept user-provided schemas or definitions and label all platform-dependent conclusions as unverified.
通过
platform-operations.md
中的项目准入检查。使用
ae-cli
确定候选曝光与结果事件、分配身份与关联路径、时间戳、精确的已保存指标定义,以及(若可用)基准数据与合格流量。
若项目不可用,则接受用户提供的schema或定义,并将所有依赖平台的结论标记为未验证。

3. Assess metric readiness

3. 评估指标就绪性

Apply
metric-readiness.md
. Classify candidates as
recommended
,
available
,
blocked
, or
unverified
; recommend one primary event metric and confirm its calculation code before planning sample size or duration.
应用
metric-readiness.md
中的规则。将候选指标分类为
recommended
(推荐)、
available
(可用)、
blocked
(受阻)或
unverified
(未验证);推荐一个主事件指标,并在规划样本量或时长前确认其计算代码。

4. Design Feature, assignment, and groups

4. 设计Feature、分配规则与分组

Define the Feature key, type, typed default, ownership, stable assignment unit, one control group, treatment groups, group values, traffic, allocations, layer, targeting, and exclusions.
Require allocations totaling
1.0
, experiment traffic in
(0, 1]
, type-correct values, a stable exposure-to-outcome identity join, and at least one primary metric. Resolve real Features and layers with
ae-cli
before reuse or creation.
定义Feature键、类型、带类型的默认值、归属方、稳定分配单元、一个对照组、处理组、分组值、流量、分配比例、层级、定向规则及排除规则。
要求分配比例总和为
1.0
,实验流量范围为
(0, 1]
,值的类型正确,曝光与结果的身份关联稳定,且至少包含一个主指标。在复用或创建前,需通过
ae-cli
确认真实的Feature与层级。

5. Calculate sample size and duration

5. 计算样本量与时长

Follow this order: confirm the primary metric and calculation code → obtain its baseline, MDE, and any required variance → calculate the sample target with
scripts/calculate_experiment_plan.py
→ derive duration from the sample target and effective eligible daily units. Apply the preregistered planning policy in
metric-readiness.md
. Use fixed
alpha=0.05
and two-sided testing, policy-default
power=0.80
, and Bonferroni planning for multiple treatments. Require the MDE type and direction; never default MDE to 5% or assume variance for a continuous metric.
When experiment traffic is already confirmed, calculate its duration. When it is not confirmed, obtain verified layer capacity and let the script recommend the smallest absolute traffic candidate that reaches the target within the maximum runtime. Default to at least seven days and full-week alignment. Return the actual infeasible duration instead of truncating it. Explain the baseline, MDE, power source, allocations, multiplicity rule, traffic evidence, sample targets, duration adjustment, and any native-report mismatch. Do not return a definitive plan when required evidence is unavailable.
遵循以下顺序:确认主指标及其计算代码 → 获取基准值、MDE及所需方差 → 使用
scripts/calculate_experiment_plan.py
计算样本目标 → 根据样本目标与有效合格日活数推导时长。应用
metric-readiness.md
中预先注册的规划策略。使用固定的
alpha=0.05
与双侧检验,策略默认的
power=0.80
,以及针对多处理组的Bonferroni规划。要求明确MDE的类型与方向;不得将MDE默认设为5%,也不得假设连续指标的方差。
若实验流量已确认,则计算其所需时长。若未确认,则获取已验证的层级容量,让脚本推荐能在最长运行时间内达到目标的最小绝对流量候选值。默认时长至少为7天,并与完整周对齐。返回实际不可行的时长,而非截断时长。解释基准值、MDE、功效来源、分配比例、多重性规则、流量依据、样本目标、时长调整及任何原生报告不匹配的情况。当所需依据不可用时,不得返回确定的规划方案。

6. Materialize the design

6. 落地设计方案

For an explicit draft-creation request, follow
experiment-creation.md
and
platform-operations.md
. Create only authorized draft assets, verify the saved result by reading it back, run supported readiness and conflict checks, and return the compact receipt and experiment link defined there.
Submitting, starting, changing live traffic, pausing, ending, or deleting requires separate explicit confirmation. Never turn draft creation into launch.
对于明确的草稿创建请求,遵循
experiment-creation.md
platform-operations.md
的要求。仅创建经授权的草稿资产,通过回读验证已保存的结果,运行支持的就绪性与冲突检查,并返回其中定义的简洁回执与实验链接。
提交、启动、修改实时流量、暂停、结束或删除操作需单独的明确确认。请勿将草稿创建直接转化为启动操作。

Output requirements

输出要求

  • Use the language explicitly requested by the user. Otherwise, use the language of the user's latest substantive message.
  • Localize all user-visible prose, including headings, table headers, field labels, status names, recommendations, warnings, assumptions, and next actions.
  • Keep code, commands, raw IDs, event/property/metric names, Feature keys, SDK/API names, and official enum values unchanged when translation would alter their technical meaning.
  • Treat section names in this Skill as semantic guidance, not literal output text. Do not copy an English heading into a non-English response.
  • For a design request, lead with the experiment recommendation. For a creation, validation, or conflict-check request, lead with the operation outcome.
  • Include only the smallest set of relevant sections; do not reproduce every workflow stage.
  • Separate observed platform evidence, verified documentation, deterministic calculations, design judgments, and unresolved assumptions.
  • Before responding, check every heading, table header, label, and status for unintended mixed-language output.
Treat project resolution, metadata discovery, candidate-event searches, metric comparison, Feature and layer inventory, capability discovery, schema inspection, and command execution as internal working context.
  • Do not narrate the execution sequence in the final answer. Omit phrases such as "first load the reference", "now query in parallel", "verified with ae-cli", or "the evidence collection is complete".
  • Do not expose raw commands, capability IDs, request schemas, full candidate lists, or a platform-evidence dump unless the user explicitly asks for the evidence, audit trail, or debugging details.
  • Surface platform evidence only when it changes the design, blocks the operation, reveals a material semantic mismatch, or requires user confirmation. Summarize it in at most three concise bullets by default.
  • Do not repeat the full experiment design after a creation request unless the user explicitly asks for the complete design.
  • Do not expose hidden reasoning. Give the conclusion, the user-relevant basis, and the action result.
  • 使用用户明确要求的语言。否则,使用用户最新实质性消息的语言。
  • 对所有用户可见的文本进行本地化,包括标题、表头、字段标签、状态名称、建议、警告、假设及后续操作。
  • 若翻译会改变技术含义,则保留代码、命令、原始ID、事件/属性/指标名称、Feature键、SDK/API名称及官方枚举值不变。
  • 将本Skill中的章节名称视为语义指导,而非字面输出文本。请勿将英文标题复制到非英文响应中。
  • 对于设计请求,以实验建议开头。对于创建、验证或冲突检查请求,以操作结果开头。
  • 仅包含最小范围的相关章节;请勿复制每个工作流阶段。
  • 区分已观察到的平台依据、已验证的文档、确定性计算、设计判断及未解决的假设。
  • 响应前,检查所有标题、表头、标签及状态,避免出现意外的混合语言输出。
将项目确认、元数据发现、候选事件搜索、指标对比、Feature与层级清单、能力发现、schema检查及命令执行视为内部工作上下文。
  • 请勿在最终答案中叙述执行顺序。省略诸如“首先加载参考内容”“现在并行查询”“已通过ae-cli验证”或“证据收集完成”等表述。
  • 除非用户明确要求依据、审计轨迹或调试细节,否则不得暴露原始命令、能力ID、请求schema、完整候选列表或平台依据转储。
  • 仅当平台依据会改变设计、阻止操作、揭示重大语义不匹配或需要用户确认时,才予以展示。默认情况下,最多用三个简洁要点进行总结。
  • 创建请求完成后,除非用户明确要求完整设计,否则不得重复完整的实验设计。
  • 不得暴露隐藏的推理过程。给出结论、与用户相关的依据及操作结果。

Failure behavior

故障处理行为

  • Missing project or ambiguous host: show candidates and ask; do not guess.
  • Missing metadata: return the required event, property, identity, and timestamp checklist.
  • Experiment product unavailable:
    • State that the project has not enabled the experiment product only when an explicit platform entitlement result establishes that fact. A missing capability alone means the experiment capability is unavailable, not that the product was not purchased.
    • For a design request, tell the user that experiment design can continue, but Feature, layer, metric, and traffic details cannot be verified on the platform. Continue with an offline design and request the baseline, MDE, and eligible daily units when sample-size or duration planning needs them.
    • For a draft-creation request, lead with the outcome that the experiment draft was not created. Explain that Feature, layer, and draft creation are blocked, preserve the proposed design, and say that platform creation and readiness checks can continue after the product is enabled or the required access is granted.
  • HTTP 403 or equivalent permission denial: state that the current account lacks the required experiment permission, stop dependent writes, and explain that this result does not establish whether the project purchased the experiment product. Ask the project administrator to check both product availability and the user's project permissions.
  • Capability gap without an explicit entitlement or permission result: report that the current environment does not expose the required experiment capability, continue with an offline design when useful, and do not bypass
    ae-cli
    .
  • For SDK gaps or conflicts, follow
    sdk-integration.md
    ; do not invent exact code.
  • Validation failure: correct documented input or ask for the missing value; do not retry unchanged input.
  • Partial success: report created and failed assets separately and never imply atomic success.
  • 项目缺失或主机不明确:展示候选选项并询问;请勿猜测。
  • 元数据缺失:返回所需事件、属性、身份及时间戳清单。
  • 实验产品不可用:
    • 仅当明确的平台授权结果确认该事实时,才说明项目未启用实验产品。仅缺少能力仅表示实验能力不可用,而非未购买该产品。
    • 对于设计请求,告知用户实验设计可继续,但Feature、层级、指标及流量细节无法在平台上验证。继续进行离线设计,并在需要规划样本量或时长时,请求基准值、MDE及合格日活数。
    • 对于草稿创建请求,首先说明实验草稿未创建。解释Feature、层级及草稿创建已受阻,保留提议的设计方案,并说明在产品启用或获得所需权限后,可继续进行平台创建与就绪性检查。
  • HTTP 403或等效权限拒绝:说明当前账户缺少所需的实验权限,停止相关写入操作,并解释该结果无法确定项目是否已购买实验产品。请项目管理员检查产品可用性与用户的项目权限。
  • 无明确授权或权限结果的能力缺口:报告当前环境未暴露所需的实验能力,若有用则继续进行离线设计,且不得绕过
    ae-cli
  • 对于SDK缺口或冲突,遵循
    sdk-integration.md
    ;不得虚构精确代码。
  • 验证失败:修正已记录的输入或请求缺失值;不得重试未更改的输入。
  • 部分成功:分别报告已创建与创建失败的资产,不得暗示操作完全成功。