Back to Details

configuring-experiment-rollout

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Configuring experiment rollout

配置实验发布

This skill answers: Who sees what variant?
本技能解答：哪些用户会看到哪个变体？

Recommended approach: equal split + adjust rollout percentage

推荐方案：均等拆分 + 调整发布百分比

In most cases, experiments work best with an equal split. If you want to limit exposure to the test variant, adjust the rollout percentage instead.
Why equal splits are better:
  • Equal splits maximize statistical power — each variant has the same sample size
  • Equal splits balance traffic and thus reach significance faster
  • Increasing user exposure throughout the experiment through increasing rollout is clean (changing split mid-experiment can cause users to switch variants, which is bad for user experience and data quality)
Always default to an equal split unless the user explicitly requests otherwise.
在大多数情况下，实验采用均等拆分效果最佳。如果想要限制测试变体的曝光范围，请调整整体发布百分比，而非更改拆分比例。
均等拆分更优的原因：
  • 均等拆分能最大化统计效力——每个变体拥有相同的样本量
  • 均等拆分平衡流量，从而更快达到显著性
  • 通过逐步提高发布百分比来增加用户曝光的方式更规范（实验中途更改拆分比例可能导致用户切换变体，这对用户体验和数据质量都不利）
除非用户明确要求，否则默认采用均等拆分。

When an uneven split is required

非均等拆分的适用场景

Uneven splits combined with the default "Exclude multivariate users" handling can introduce bias. If the experiment observes multi-variant users (users exposed to more than one variant) then those are dropped asymmetrically — the smaller variant loses a larger fraction of its assignments. If those users behave differently from the rest, the smaller variant's metrics will be skewed.
The right mitigation depends on experiment state:
  1. Pre-launch, or live but with few exposures so far — use an equal split and reduce the overall rollout. Achieves the same test-variant exposure without the bias and preserves statistical power. See the disambiguation question below.
  2. Live experiment with significant exposures — switch multivariate handling to "First seen variant". Changing the split mid-run reassigns users across variants (anti-pattern; see "Changing rollout on a running experiment" below). Switching handling instead keeps everyone in their original variant and avoids the asymmetric exclusion. See 
    configuring-experiment-analytics
    for how to set this. Note that "first seen" handling can introduce other biases, but it's preferable to mid-run reassignment.
非均等拆分结合默认的「排除多变体用户」处理方式可能会引入偏差。 如果实验中出现多变体用户（接触过多个变体的用户），这些用户会被不对称地剔除——较小的变体损失的分配占比更大。如果这些用户的行为与其他用户不同，较小变体的指标就会出现偏差。
正确的缓解措施取决于实验状态：
  1. 未启动，或已启动但曝光量极少——采用均等拆分并降低整体发布百分比。这种方式能达到相同的测试变体曝光量，同时避免偏差并保留统计效力。请参考下方的歧义消除问题。
  2. 已启动且曝光量较大——将多变体用户处理方式切换为「首次看到的变体」。实验中途更改拆分比例会重新为用户分配变体（这是反模式；请参阅下方「更改运行中实验的发布配置」）。切换处理方式则能让所有用户保持原变体分配，避免不对称剔除。有关设置方法，请查看
    configuring-experiment-analytics
    。请注意，「首次看到的变体」处理方式可能会引入其他偏差，但比中途重新分配变体更可取。

The two rollout controls

两类发布控制项

There are two separate controls that determine who sees what. Both are set via 
parameters
.
有两个独立的控制项决定用户能看到什么内容，两者均通过
parameters
设置。

1. Variant split (
parameters.feature_flag_variants
)

1. 变体拆分（
parameters.feature_flag_variants
）

How users inside the experiment are distributed across variants.
  • Array of 
    {key, name, split_percent}
    — percentages must sum to 100
  • First variant must have key 
    "control"
    — this is the baseline
  • Minimum 2 variants, maximum 20
  • Default: control 50% / test 50%
If the user says "A/B/C test", map the baseline to 
"control"
and create additional variants for the others.
实验范围内的用户如何分配到各个变体中。
  • 为
    {key, name, split_percent}
    数组——百分比总和必须为100
  • 第一个变体的key必须为
    "control"
    ——这是基准组
  • 最少2个变体，最多20个
  • 默认设置：对照组50% / 测试组50%
如果用户提及「A/B/C测试」，将基准映射为
"control"
，并为其他组创建额外变体。

2. Overall rollout (
parameters.rollout_percentage
)

2. 整体发布百分比（
parameters.rollout_percentage
）

What percentage of all users enter the experiment at all. Default: 100%.
Users not included are excluded entirely — they don't see any variant and are not part of the analysis.
所有用户中，有多少比例会进入实验。默认值：100%。
未被纳入的用户会被完全排除——他们看不到任何变体，且不参与分析。

How they interact

两者的交互方式

These two controls multiply:
Overall rolloutVariant split% seeing test% in analysis
100%50/5050%100%
100%75/25 control/test25%100%
50%50/5025%50%
25%50/5012.5%25%
这两个控制项的效果是相乘的：
整体发布百分比变体拆分比例看到测试变体的用户占比参与分析的用户占比
100%50/5050%100%
100%对照组75%/测试组25%25%100%
50%50/5025%50%
25%50/5012.5%25%

The disambiguation question

歧义消除问题

CRITICAL: If the user requests an uneven variant split (e.g. "60/40", "70/20/10") or mentions a specific percentage that could refer to either the split or the rollout (e.g. "roll out to 25%"), you MUST clarify before proceeding. This covers two cases:
关键注意事项：如果用户要求非均等变体拆分（例如「60/40」「70/20/10」），或提及可能指代拆分比例或发布百分比的特定百分比（例如「向25%的用户发布」），你必须先澄清再继续操作。这涵盖两种情况：

Case 1: Single percentage ("25%", "roll out to 40%")

情况1：单一百分比（「25%」「向40%的用户发布」）

The percentage is ambiguous — it could mean a variant split or a rollout change. Ask:
There are two ways to get 25% of users seeing the test variant:
  1. Reduced rollout with equal split (recommended): reduce the overall rollout and split variants equally. Only a subset of users enter the experiment, and of those, each variant gets the same share. Equal splits maximize statistical power and avoid bias.
  2. Asymmetric split: keep 100% rollout but give the test variant only 25%. All users enter the experiment, but the uneven split reduces power on the smaller variant and risks bias.
Which approach do you prefer?
Adjust the numbers to match whatever percentage the user requested.
该百分比存在歧义——它可能指变体拆分比例，也可能指发布百分比变更。请询问：
有两种方式可以让25%的用户看到测试变体：
  1. 降低发布百分比 + 均等拆分（推荐）：降低整体发布百分比，同时保持变体均等拆分。仅部分用户进入实验，且这些用户中每个变体获得相同的流量份额。 均等拆分能最大化统计效力并避免偏差。
  2. 非均等拆分：保持100%的发布百分比，但仅让25%的用户看到测试变体。 所有用户都进入实验，但非均等拆分会降低较小组的统计效力，并存在偏差风险。
你倾向于哪种方案？
根据用户提及的百分比调整数字。

Case 2: Uneven ratio ("60/40", "70/30", "80/20", etc.)

情况2：非均等比例（「60/40」「70/30」「80/20」等）

The ratio looks like an explicit variant split, but a reduced rollout with an equal split is almost always better. Explain the trade-off and recommend the alternative:
An uneven variant split works, but an equal split with reduced rollout is recommended:
  1. Equal split + reduced rollout (recommended): reduce the overall rollout so that the same fraction of users sees the test variant, but split variants equally within the experiment. Equal splits maximize statistical power and avoid bias from asymmetric multivariate exclusion.
  2. Uneven split. Achieves the same user-facing outcome, but reduces power on the smaller variant and risks bias.
Would you like the equal split approach, or do you have a specific reason for the uneven split?
Adjust the numbers to match the ratio. For experiments with more than two variants, "equal" means each variant gets the same share (e.g. 34/33/33 for three variants). If the user confirms they want the uneven split after seeing the trade-off, proceed — but DO NOT skip the next section.
该比例看似明确要求变体拆分，但降低发布百分比并采用均等拆分几乎总是更好的选择。请解释利弊并推荐替代方案：
非均等变体拆分是可行的，但更推荐采用均等拆分并降低发布百分比：
  1. 均等拆分 + 降低发布百分比（推荐）：降低整体发布百分比，使看到测试变体的用户比例相同，但在实验内保持变体均等拆分。 均等拆分能最大化统计效力，并避免因不对称剔除多变体用户而产生的偏差。
  2. 非均等拆分。 能达到相同的用户体验效果，但会降低较小组的统计效力，并存在偏差风险。
你希望采用均等拆分方案，还是有特定理由需要非均等拆分？
根据用户提及的比例调整数字。对于超过两个变体的实验，「均等」指每个变体获得相同的份额（例如三个变体为34/33/33）。如果用户了解利弊后仍确认要采用非均等拆分，请继续操作——但不要跳过下一部分。

After the user picks the uneven split

用户选择非均等拆分后

If the user proceeds with an uneven split (option 2 in either case above), you MUST surface the multivariate-handling implication BEFORE creating or updating the experiment. The user has chosen the riskier rollout path and needs to make an informed choice about how to mitigate.
Ask:
One more thing — with an uneven split, the default "Exclude multivariate users" handling drops users exposed to multiple variants asymmetrically. The smaller variant loses a larger fraction of its assignments, which can skew its metrics if those users behave differently from the rest.
Two options:
  1. Switch multivariate handling to "First seen variant" (recommended for uneven splits) — keeps all users in the analysis and avoids asymmetric exclusion. Has its own caveats (other biases can creep in) but is preferable to the default for uneven splits.
  2. Keep the default "Exclude" handling and accept the bias risk.
Which would you like?
See 
configuring-experiment-analytics
for how to set the multivariate handling. Apply the choice as part of the same operation (creation or update) — do not leave the user with an uneven split under default handling without an explicit, informed decision.
如果用户选择非均等拆分（上述任一情况中的选项2），你必须在创建或更新实验前告知其多变体用户处理方式的影响。用户选择了风险更高的发布路径，需要做出知情选择来缓解风险。
请询问：
还有一点需要注意——采用非均等拆分时，默认的「排除多变体用户」处理方式会不对称地剔除接触过多个变体的用户。较小的变体损失的分配占比更大，如果这些用户的行为与其他用户不同，该变体的指标可能会出现偏差。
有两种选项：
  1. 将多变体用户处理方式切换为「首次看到的变体」（非均等拆分推荐方案）——保留所有用户参与分析，避免不对称剔除。这种方式也有自身的局限性（可能引入其他偏差），但比默认方式更适合非均等拆分。
  2. 保持默认的「排除」处理方式，接受偏差风险。
你选择哪种方式？
有关设置多变体用户处理方式的方法，请查看
configuring-experiment-analytics
。将用户的选择作为同一操作（创建或更新）的一部分应用——不要让用户在未做出明确知情决策的情况下，使用默认处理方式运行非均等拆分实验。

Persist flag across authentication steps

跨认证步骤保留标记

This option (
ensure_experience_continuity
on the feature flag) is only relevant when:
  • The feature flag is shown to both logged-out AND logged-in users
  • You need the same variant assignment before and after login
This is not compatible with all setups. Learn more: https://posthog.com/docs/feature-flags/creating-feature-flags#persisting-feature-flags-across-authentication-steps
Only mention this to the user if their use case involves pre/post-authentication experiences.
该选项（功能标记上的
ensure_experience_continuity
）仅在以下场景中相关：
  • 功能标记同时展示给未登录和已登录用户
  • 需要用户在登录前后看到相同的变体分配
该选项并非适用于所有设置。了解更多：https://posthog.com/docs/feature-flags/creating-feature-flags#persisting-feature-flags-across-authentication-steps
仅当用户的使用场景涉及认证前后的体验时，才向其提及该选项。

Resolving experiments

解析实验

Rollout changes require an experiment ID. If the user refers to an experiment by name or description (e.g. "change rollout on my signup test"), load the 
finding-experiments
skill to resolve it to a concrete ID before proceeding.
更改发布配置需要实验ID。如果用户通过名称或描述指代实验（例如「更改我的注册测试的发布配置」），请调用
finding-experiments
技能将其解析为具体ID后再继续操作。

Changing rollout on a running experiment

更改运行中实验的发布配置

Any change to rollout or variant split on a running experiment affects both user experience and statistical validity. You MUST warn the user and get explicit confirmation before making the change.
Do NOT silently apply the change — even if the user asked for it directly. Present the warning covering both perspectives:
  1. Who sees what variant? — will users switch variants or lose a feature?
  2. Who is in my analysis? — how does this affect data quality?
Exception: Increasing rollout (without changing the split) is generally safe — no users switch variants, more users are added cleanly.
Mid-experiment fix for uneven-split bias: switching multivariate handling from "Exclude" to "First seen variant" is the recommended mitigation for already-launched experiments — no users switch variants and all collected data stays in the analysis. Changing the split to be even is an anti-pattern mid-run (typically requires resetting or ending the experiment) and is only preferred if the experiment hasn't been exposed to many users yet. See 
configuring-experiment-analytics
for how to change the handling.
See 
references/changing-distribution-after-launch.md
for detailed warnings, what to tell the user, and when to recommend alternatives.
任何对运行中实验的发布百分比或变体拆分的更改都会影响用户体验和统计有效性。 在做出更改前，你必须警告用户并获得明确确认。
不要静默应用更改——即使用户直接提出要求。请从以下两个角度呈现警告：
  1. 哪些用户会看到哪个变体？——用户是否会切换变体或失去某个功能？
  2. 哪些用户会参与我的分析？——这会如何影响数据质量？
例外情况：仅提高发布百分比（不更改拆分比例）通常是安全的——不会有用户切换变体，新增用户的方式更规范。
非均等拆分偏差的实验中途修复方案：将多变体用户处理方式从「排除」切换为「首次看到的变体」是已启动实验的推荐缓解措施——不会有用户切换变体，所有收集的数据都能保留在分析中。实验中途将拆分比例改为均等是反模式（通常需要重置或结束实验），仅在实验曝光量极少时更可取。有关更改处理方式的方法，请查看
configuring-experiment-analytics
。
有关详细警告、需告知用户的内容以及何时推荐替代方案，请参阅
references/changing-distribution-after-launch.md
。