Back to Details

configuring-experiment-rollout

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Configuring experiment rollout

配置实验发布

This skill answers: Who sees what variant?
本技能解答：哪些用户会看到对应变体？

Recommended approach: equal split + adjust rollout percentage

推荐方案：均等拆分 + 调整发布百分比

In most cases, experiments work best with an equal split. If you want to limit exposure to the test variant, adjust the rollout percentage instead.
Why equal splits are better:
  • Equal splits maximize statistical power — each variant has the same sample size
  • Equal splits balance traffic and thus reach significance faster
  • Increasing user exposure throughout the experiment through increasing rollout is clean (changing split mid-experiment can cause users to switch variants, which is bad for user experience and data quality)
Always default to an equal split unless the user explicitly requests otherwise.
在大多数情况下，实验采用均等拆分效果最佳。如果想要限制测试变体的曝光范围，应调整整体发布百分比而非拆分比例。
为何均等拆分更优：
  • 均等拆分能最大化统计效力——每个变体拥有相同的样本量
  • 均等拆分平衡流量，从而更快达到统计显著性
  • 通过逐步提高发布百分比来增加用户曝光的方式更合理（实验中途修改拆分比例会导致用户切换变体，这对用户体验和数据质量都不利）
除非用户明确要求，否则始终默认使用均等拆分。

When an uneven split is required

何时需要使用不均等拆分

Uneven splits combined with the default "Exclude multivariate users" handling can introduce bias. If the experiment observes multi-variant users (users exposed to more than one variant) then those are dropped asymmetrically — the smaller variant loses a larger fraction of its assignments. If those users behave differently from the rest, the smaller variant's metrics will be skewed.
The right mitigation depends on experiment state:
  1. Pre-launch, or live but with few exposures so far — use an equal split and reduce the overall rollout. Achieves the same test-variant exposure without the bias and preserves statistical power. See the disambiguation question below.
  2. Live experiment with significant exposures — switch multivariate handling to "First seen variant". Changing the split mid-run reassigns users across variants (anti-pattern; see "Changing rollout on a running experiment" below). Switching handling instead keeps everyone in their original variant and avoids the asymmetric exclusion. See 
    configuring-experiment-analytics
    for how to set this. Note that "first seen" handling can introduce other biases, but it's preferable to mid-run reassignment.
不均等拆分结合默认的“排除多变体用户”处理方式可能会引入偏差。如果实验中出现多变体用户（即接触过多个变体的用户），这些用户会被不对称地剔除——较小的变体损失的分配用户比例更大。如果这些用户的行为与其他用户不同，那么较小变体的指标会出现偏差。
正确的缓解措施取决于实验状态：
  1. 预启动阶段，或已启动但曝光量极少——使用均等拆分并降低整体发布百分比。这种方式能达到相同的测试变体曝光量，同时避免偏差并保留统计效力。请参考下方的歧义澄清问题。
  2. 已启动且曝光量较大的实验——将多变体用户处理方式切换为“首次接触的变体”。实验中途修改拆分比例会导致用户在变体间重新分配（这是反模式；请参考下方“修改运行中实验的发布配置”）。切换处理方式则能让所有用户保留在最初分配的变体中，避免不对称剔除。请查看
    configuring-experiment-analytics
    了解设置方法。注意，“首次接触”处理方式可能会引入其他偏差，但比中途重新分配用户更可取。

The two rollout controls

两类发布控制项

There are two separate controls that determine who sees what. Both are set via 
parameters
.
有两个独立的控制项决定用户能看到什么内容，两者均通过
parameters
设置。

1. Variant split (
parameters.feature_flag_variants
)

1. 变体拆分（
parameters.feature_flag_variants
）

How users inside the experiment are distributed across variants.
  • Array of 
    {key, name, split_percent}
    — percentages must sum to 100
  • First variant must have key 
    "control"
    — this is the baseline
  • Minimum 2 variants, maximum 20
  • Default: control 50% / test 50%
If the user says "A/B/C test", map the baseline to 
"control"
and create additional variants for the others.
实验范围内的用户如何分配到不同变体中。
  • 由
    {key, name, split_percent}
    组成的数组——百分比总和必须为100
  • 第一个变体的key必须为
    "control"
    ——这是基准组
  • 最少2个变体，最多20个
  • 默认设置：control组50% / 测试组50%
如果用户提到“A/B/C测试”，将基准组映射为
"control"
，并为其他组创建额外变体。

2. Overall rollout (
parameters.rollout_percentage
)

2. 整体发布百分比（
parameters.rollout_percentage
）

What percentage of all users enter the experiment at all. Default: 100%.
Users not included are excluded entirely — they don't see any variant and are not part of the analysis.
所有用户中进入实验的比例。默认值：100%。
未被纳入的用户会被完全排除——他们看不到任何变体，且不参与分析。

How they interact

两者的交互逻辑

These two controls multiply:
Overall rolloutVariant split% seeing test% in analysis
100%50/5050%100%
100%75/25 control/test25%100%
50%50/5025%50%
25%50/5012.5%25%
这两个控制项的效果是相乘的：
整体发布百分比变体拆分比例看到测试变体的用户占比参与分析的用户占比
100%50/5050%100%
100%75/25 control/测试25%100%
50%50/5025%50%
25%50/5012.5%25%

The disambiguation question

歧义澄清问题

CRITICAL: If the user requests an uneven variant split (e.g. "60/40", "70/20/10") or mentions a specific percentage that could refer to either the split or the rollout (e.g. "roll out to 25%"), you MUST clarify before proceeding. This covers two cases:
关键提示：如果用户要求不均等的变体拆分比例（如“60/40”“70/20/10”），或提及可能同时指代拆分比例或发布百分比的具体数值（如“向25%用户发布”），你必须先澄清再继续操作。这涵盖两种情况：

Case 1: Single percentage ("25%", "roll out to 40%")

情况1：单一百分比（“25%”“向40%用户发布”）

The percentage is ambiguous — it could mean a variant split or a rollout change. Ask:
There are two ways to get 25% of users seeing the test variant:
  1. Reduced rollout with equal split (recommended): reduce the overall rollout and split variants equally. Only a subset of users enter the experiment, and of those, each variant gets the same share. Equal splits maximize statistical power and avoid bias.
  2. Asymmetric split: keep 100% rollout but give the test variant only 25%. All users enter the experiment, but the uneven split reduces power on the smaller variant and risks bias.
Which approach do you prefer?
Adjust the numbers to match whatever percentage the user requested.
该百分比存在歧义——它可能指变体拆分比例，也可能指发布百分比调整。请询问：
有两种方式可以让25%的用户看到测试变体：
  1. 降低发布百分比+均等拆分（推荐）：降低整体发布百分比，同时保持变体均等拆分。仅部分用户进入实验，且这些用户中每个变体获得相同的流量份额。 均等拆分能最大化统计效力并避免偏差。
  2. 不对称拆分：保持100%的发布百分比，但仅让25%的用户看到测试变体。 所有用户都进入实验，但不均等拆分会降低较小组的统计效力，且存在偏差风险。
你倾向于哪种方案？
根据用户提到的百分比调整上述内容中的数值。

Case 2: Uneven ratio ("60/40", "70/30", "80/20", etc.)

情况2：不均等比例（“60/40”“70/30”“80/20”等）

The ratio looks like an explicit variant split, but a reduced rollout with an equal split is almost always better. Explain the trade-off and recommend the alternative:
An uneven variant split works, but an equal split with reduced rollout is recommended:
  1. Equal split + reduced rollout (recommended): reduce the overall rollout so that the same fraction of users sees the test variant, but split variants equally within the experiment. Equal splits maximize statistical power and avoid bias from asymmetric multivariate exclusion.
  2. Uneven split. Achieves the same user-facing outcome, but reduces power on the smaller variant and risks bias.
Would you like the equal split approach, or do you have a specific reason for the uneven split?
Adjust the numbers to match the ratio. For experiments with more than two variants, "equal" means each variant gets the same share (e.g. 34/33/33 for three variants). If the user confirms they want the uneven split after seeing the trade-off, proceed — but DO NOT skip the next section.
该比例看似明确指向变体拆分，但降低发布百分比+均等拆分几乎总是更优的选择。请解释利弊并推荐替代方案：
不均等变体拆分是可行的，但更推荐采用均等拆分+降低发布百分比的方案：
  1. 均等拆分+降低发布百分比（推荐）：降低整体发布百分比，使看到测试变体的用户比例保持不变，但实验内的变体采用均等拆分。 均等拆分能最大化统计效力，避免因不对称剔除多变体用户而产生的偏差。
  2. 不均等拆分。 能达到相同的用户端效果，但会降低较小组的统计效力，且存在偏差风险。
你希望采用均等拆分方案，还是有特定理由需要使用不均等拆分？
根据用户提到的比例调整上述内容中的数值。对于超过两个变体的实验，“均等”指每个变体获得相同的份额（如三个变体为34/33/33）。如果用户在了解利弊后仍确认要使用不均等拆分，请继续操作——但不要跳过下一部分内容。

After the user picks the uneven split

用户选择不均等拆分后的操作

If the user proceeds with an uneven split (option 2 in either case above), you MUST surface the multivariate-handling implication BEFORE creating or updating the experiment. The user has chosen the riskier rollout path and needs to make an informed choice about how to mitigate.
Ask:
One more thing — with an uneven split, the default "Exclude multivariate users" handling drops users exposed to multiple variants asymmetrically. The smaller variant loses a larger fraction of its assignments, which can skew its metrics if those users behave differently from the rest.
Two options:
  1. Switch multivariate handling to "First seen variant" (recommended for uneven splits) — keeps all users in the analysis and avoids asymmetric exclusion. Has its own caveats (other biases can creep in) but is preferable to the default for uneven splits.
  2. Keep the default "Exclude" handling and accept the bias risk.
Which would you like?
See 
configuring-experiment-analytics
for how to set the multivariate handling. Apply the choice as part of the same operation (creation or update) — do not leave the user with an uneven split under default handling without an explicit, informed decision.
如果用户选择不均等拆分（上述任一情况中的选项2），你必须在创建或更新实验前，告知用户多变体用户处理方式的影响。用户选择了风险更高的发布路径，需要在充分知情的情况下做出缓解措施的选择。
请询问：
还有一点需要注意——使用不均等拆分时，默认的“排除多变体用户”处理方式会不对称地剔除接触过多个变体的用户。较小的变体损失的分配用户比例更大，如果这些用户的行为与其他用户不同，会导致该变体的指标出现偏差。
有两种选择：
  1. 将多变体用户处理方式切换为“首次接触的变体”（不均等拆分推荐方案）——保留所有用户参与分析，避免不对称剔除。这种方式也存在一些局限性（可能引入其他偏差），但比默认方式更适合不均等拆分。
  2. 保留默认的“排除”处理方式，接受偏差风险。
你选择哪种方式？
请查看
configuring-experiment-analytics
了解如何设置多变体用户处理方式。将用户的选择作为同一操作（创建或更新）的一部分应用——不要在未获得用户明确知情同意的情况下，让不均等拆分实验使用默认处理方式。

Persist flag across authentication steps

跨认证步骤保留标识

This option (
ensure_experience_continuity
on the feature flag) is only relevant when:
  • The feature flag is shown to both logged-out AND logged-in users
  • You need the same variant assignment before and after login
This is not compatible with all setups. Learn more: https://posthog.com/docs/feature-flags/creating-feature-flags#persisting-feature-flags-across-authentication-steps
Only mention this to the user if their use case involves pre/post-authentication experiences.
该选项（功能标识上的
ensure_experience_continuity
）仅在以下场景中相关：
  • 功能标识同时展示给未登录和已登录用户
  • 需要用户在登录前后获得相同的变体分配
该选项并非适用于所有设置。了解更多：https://posthog.com/docs/feature-flags/creating-feature-flags#persisting-feature-flags-across-authentication-steps
仅当用户的使用场景涉及登录前后的体验时，才向其提及该选项。

Resolving experiments

定位实验

Rollout changes require an experiment ID. If the user refers to an experiment by name or description (e.g. "change rollout on my signup test"), load the 
finding-experiments
skill to resolve it to a concrete ID before proceeding.
修改发布配置需要实验ID。如果用户通过名称或描述指代实验（如“修改我的注册测试的发布配置”），请调用
finding-experiments
技能将其解析为具体ID后再继续操作。

Changing rollout on a running experiment

修改运行中实验的发布配置

Any change to rollout or variant split on a running experiment affects both user experience and statistical validity. You MUST warn the user and get explicit confirmation before making the change.
Do NOT silently apply the change — even if the user asked for it directly. Present the warning covering both perspectives:
  1. Who sees what variant? — will users switch variants or lose a feature?
  2. Who is in my analysis? — how does this affect data quality?
Exception: Increasing rollout (without changing the split) is generally safe — no users switch variants, more users are added cleanly.
Mid-experiment fix for uneven-split bias: switching multivariate handling from "Exclude" to "First seen variant" is the recommended mitigation for already-launched experiments — no users switch variants and all collected data stays in the analysis. Changing the split to be even is an anti-pattern mid-run (typically requires resetting or ending the experiment) and is only preferred if the experiment hasn't been exposed to many users yet. See 
configuring-experiment-analytics
for how to change the handling.
See 
references/changing-distribution-after-launch.md
for detailed warnings, what to tell the user, and when to recommend alternatives.
**对运行中实验的发布百分比或变体拆分比例进行任何修改，都会影响用户体验和统计有效性。**在做出修改前，你必须向用户发出警告并获得明确确认。
不要静默执行修改——即使用户直接提出修改请求。请从以下两个角度向用户发出警告：
  1. 用户会看到什么变体？——用户是否会切换变体或失去某项功能？
  2. 哪些用户会参与分析？——这会如何影响数据质量？
例外情况：仅提高发布百分比（不修改拆分比例）通常是安全的——不会导致用户切换变体，只是干净地增加参与实验的用户数量。
实验中途不均等拆分偏差的修复方案：对于已启动的实验，将多变体用户处理方式从“排除”切换为“首次接触的变体”是推荐的缓解措施——不会导致用户切换变体，所有已收集的数据都会保留在分析中。实验中途将拆分比例改为均等是反模式（通常需要重置或结束实验），仅在实验曝光量极少时才推荐这样做。请查看
configuring-experiment-analytics
了解如何修改处理方式。
请查看
references/changing-distribution-after-launch.md
获取详细警告内容、告知用户的话术，以及推荐替代方案的时机。