Back to Details

experiment-analysis

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese
To analyze an experiment, follow the following steps. Make sure to execute all steps in this order, do not skip any. More details on the step below. If you already have partial information (eg. the experiment ID), you may skip that step only. Continue following all of the other steps.
  1. Get the experiment ID
  2. Get experiment details
  3. Get supporting chart data [DO NOT SKIP]
  4. Get experiment results
  5. Interpret results
  6. Report your overall findings
要分析实验，请遵循以下步骤。确保按此顺序执行所有步骤，不要跳过任何步骤。以下是各步骤的详细说明。如果你已掌握部分信息（例如实验ID），则仅可跳过对应步骤，其余所有步骤仍需遵循。
  1. 获取实验ID
  2. 获取实验详情
  3. 获取辅助图表数据【请勿跳过】
  4. 获取实验结果
  5. 解读结果
  6. 汇报整体发现

Step 1: Get the experiment ID

步骤1：获取实验ID

If you don't have the experiment ID yet, find it using the 
list-experiments
RevenueCat tool. It receives a single parameter: 
project_id
, which requires the 
proj...
Project ID. You can also pass a 
status
filter (
draft
, 
running
, 
paused
, 
stopped
).
如果你还没有实验ID，请使用
list-experiments
RevenueCat工具查找。该工具接收一个参数：
project_id
，即项目ID（格式为
proj...
）。你还可以传入
status
筛选条件（可选值：
draft
、
running
、
paused
、
stopped
）。

Step 2: Get experiment details

步骤2：获取实验详情

Step 2a: Experiment setup

步骤2a：实验配置

Get the experiment setup / metadata by using the 
get-experiment
RevenueCat tool. Parameters: 
project_id
(as above) and 
experiment_id
(from step 1). Pass the following values to the 
expand
parameter: 
offering.package.product.indicative_price
and 
offering.paywall
.
Relevant information to extract:
  • Offerings (offering_a, offering_b, ...): these define the variants of the experiment. More sophisticated setups might also include a 
    placements
    object which defines different offerings per placement (eg. onboarding, feature gate, ...). Offerings include details on the products offered. Products include an 
    indicative_price
    . Note that only USD prices for the US are returned to provide an understanding of overall pricing levels. Any introductory price is not returned. For price localization tests, tests scoped explicitly outside of the US, or introductory prices, this will not provide the full picture, you might have to resort to the 
    get-product-store-state
    tool instead (see the 
    revenuecat-store-state
    skill). Offerings may also include a 
    paywall_id
    , if the offering uses a RevenueCat Paywall. If 
    paywall_id
    is 
    null
    , that means the app is using a custom paywall.
  • notes: Any notes that were provided about the experiment
  • display_name: Name of the experiment
  • targeting_conditions: the experiment only applies to customers meeting these conditions
  • enrollment_mode: defines whether only new customers are enrolled (default) or whether this experiment also applies to existing customers
  • experiment_type: what kind of experiment this is (user selected out of a predefined list)
  • primary_metric, secondary_metrics: primary and secondary success metrics as set up when creating the experiment.
  • Status: 
    • draft
      means the experiment has not yet started, and there are no results yet.
    • running
      means the experiment is still actively enrolling new customers.
    • paused
      means the experiment is no longer enrolling customers, but already-enrolled customers are still assigned to their variant, and the experiment is continuing to collect data for up to 400 days. The experiment can be resumed to continue enrolling new customers.
    • stopped
      means the experiment is no longer enrolling customers, and already-enrolled customers have gone back to being served their default offering. Results continue to refresh for up to 400 days. The experiment can no longer be started or resumed.
使用
get-experiment
RevenueCat工具获取实验配置/元数据。参数包括：
project_id
（同上）和
experiment_id
（来自步骤1）。向
expand
参数传入以下值：
offering.package.product.indicative_price
和
offering.paywall
。
需要提取的相关信息：
  • Offerings（offering_a、offering_b等）：定义实验的不同变体。更复杂的配置可能还包含
    placements
    对象，用于定义不同场景（例如新手引导、功能权限等）下的不同Offerings。Offerings包含所提供产品的详细信息，产品包含
    indicative_price
    （指示价格）。请注意，仅返回美国地区的美元价格，用于帮助了解整体定价水平，不会返回任何入门价格。对于价格本地化测试、明确针对美国以外地区的测试或涉及入门价格的测试，此信息无法提供完整情况，你可能需要改用
    get-product-store-state
    工具（详见
    revenuecat-store-state
    技能）。如果Offering使用了RevenueCat Paywall，则会包含
    paywall_id
    ；若
    paywall_id
    为
    null
    ，则表示应用使用的是自定义付费墙。
  • notes：关于实验的备注信息
  • display_name：实验名称
  • targeting_conditions：实验仅适用于符合这些条件的用户
  • enrollment_mode：定义是否仅招募新用户（默认设置），还是该实验也适用于现有用户
  • experiment_type：实验类型（用户从预定义列表中选择）
  • primary_metric、secondary_metrics：创建实验时设定的主要和次要成功指标
  • 状态： 
    • draft
      ：实验尚未启动，暂无结果
    • running
      ：实验仍在积极招募新用户
    • paused
      ：实验不再招募新用户，但已参与的用户仍保留其分配的变体，实验将继续收集数据长达400天。实验可恢复以继续招募新用户
    • stopped
      ：实验不再招募新用户，已参与的用户将恢复使用默认Offering。结果将持续更新长达400天，且实验无法再启动或恢复

Step 2b: Paywalls

步骤2b：付费墙

For any paywall_id in any offering of the experiment, use the 
render-paywall-screenshot
tool (parameters 
project_id
, 
paywall_id
) to look at the paywall and understand the differences.
对于实验中任意Offering的
paywall_id
，使用
render-paywall-screenshot
工具（参数为
project_id
和
paywall_id
）查看付费墙，了解不同变体间的差异。

Step 3: Get supporting chart data

步骤3：获取辅助图表数据

Pull recent chart data scoped to the experiment's runtime, using the 
get-chart-data
RevenueCat tool. Do not skip this step, it provides valuable context to how the app was performing overall in the time frame.
Parameters:
  • project_id: {project_id}
  • chart_name: revenue
  • start_date: {experiment_start_date}
  • end_date: {experiment_end_date}, or today if still running
  • resolution: week (or day if < 2 weeks running)
Also pull data for chart_name :
  • conversion_to_paying — overall conversion trend during experiment
  • trials_new — trial volume trend
  • trial_conversion_rate — trial-to-paid during experiment
  • initial_conversion – Initial conversion rate (conversion from install to trial, upfront subscription, or one-time purchase)
使用
get-chart-data
RevenueCat工具提取实验运行期间的近期图表数据。请勿跳过此步骤，它能提供该时间段内应用整体表现的重要背景信息。
参数：
  • project_id: {project_id}
  • chart_name: revenue（收入）
  • start_date: {experiment_start_date}（实验开始日期）
  • end_date: {experiment_end_date}（实验结束日期），若实验仍在运行则为今日
  • resolution: week（周），若运行时长不足2周则为day（日）
同时提取以下chart_name的数据：
  • conversion_to_paying — 实验期间的整体付费转化趋势
  • trials_new — 试用用户量趋势
  • trial_conversion_rate — 实验期间的试用转付费率
  • initial_conversion – 初始转化率（从安装到试用、直接订阅或一次性购买的转化）

Step 4: Get experiment results

步骤4：获取实验结果

Get the experiment results (so far) using the 
get-experiment-results
RevenueCat tool. Parameters:
  • project_id
  • experiment_id
  • platform (optional), filter results by platform, eg. 
    ios
  • country (optional), filter results by country, eg. 
    us
  • exposure_status (optional), filter experiment results by exposure status. One of 
    enrolled
    , 
    exposed
    , 
    not_exposed
    . Only applicable for experiments for existing customers. Defaults to "enrolled" (all enrolled customers) when not provided.
使用
get-experiment-results
RevenueCat工具获取当前的实验结果。 参数：
  • project_id
  • experiment_id
  • platform（可选）：按平台筛选结果，例如
    ios
  • country（可选）：按国家/地区筛选结果，例如
    us
  • exposure_status（可选）：按曝光状态筛选实验结果，可选值为
    enrolled
    、
    exposed
    、
    not_exposed
    。仅适用于针对现有用户的实验，未提供时默认值为"enrolled"（所有参与实验的用户）

Step 5: Interpret results

步骤5：解读结果

Understanding what changes were made in the experiment, now interpret the results. Use your best judgment.
在了解实验中所做的变更后，解读实验结果。请运用你的最佳判断。