• Expected output: a test design (hypothesis, variant matrix, primary/secondary/guardrail metrics, sample-size + duration + power plan) and/or a read-out (effect estimate, interval, statistical flag, practical-effect flag, guardrails, and either an owner-governed recommendation or

  decision: UNDECIDED

  ).
• Reads: what the user wants to test, the ROAS profile (

  direct-response|prospecting|incremental-profit

  ), baseline CVR/CTR and traffic volume; for a read-out, the user's own exported results CSV (variant, sessions/impressions, conversions/clicks).
• Writes: a user-facing test-design or read-out doc plus a

  ### Handoff Summary

  .
• Promotes: the chosen hypothesis, design parameters, calculated read-out, and any explicitly owner-approved action (ask before writing memory).
• Done when: a falsifiable hypothesis is stated; the matrix isolates one variable per variant; baseline, MDE, alpha, power, multiplicity/sequential policy, duration, and guardrails are declared; and a read-out reports effect/interval/statistical/practical flags with

  Calculated

  provenance. Without a precommitted action rule and owner, return

  decision: UNDECIDED

  .
• Primary next skill: ad-creative-builder (to produce the winning direction) or paid-measurement-loop.

Emit the standard shape from skill-contract.md §Handoff Summary Format.

See CONNECTORS.md for tool category placeholders. Every input is the user's own data, manually exported. Keyed ad-platform APIs (Google Ads SDK, Meta Marketing API) are an optional Tier-2/3 MCP convenience — never required to design a test or read one out.

Statistical facts (keyless):

python3 "${CLAUDE_PLUGIN_ROOT}/scripts/connectors/experiment.py" proportion --control <conv> <n> --variant <conv> <n> --alpha <alpha> --min-lift <relative-bar>

returns rates, effect size, intervals, p-value, and separate statistical/practical flags. Revenue/AOV-style samples use

continuous

; prospective sizing uses

samplesize

. Every derived value is

Calculated

; the helper deliberately returns no winner, promote, rollback, or kill action.

~~ad platform

~~ad platform

~~web analytics

~~web analytics

~~ecommerce

1. Pick the mode. Design (plan a new test) or read-out (call a finished one). If neither a baseline+lift target nor a results CSV is present, stop and return NEEDS_INPUT naming the missing input.
2. Hypothesis. Write it falsifiable: Because [observation], we believe [one change] will [raise primary metric] by [X%] for [audience]; we'll know when [metric] moves past the design threshold. One change per hypothesis.
3. Variant matrix. One variable per variant (headline, hook, hero, CTA, LP). A/B for one change; A/B/n for ≤ 4 variants; isolate so a winner is attributable. Keep a holdout/control. See references/test-design-guide.md for the matrix template and a creative/LP/incrementality structure.
4. Metrics. Name a primary metric tied to value (CVR or CPA), secondary metrics for context, and guardrails that must not get worse (spend, refund rate, bounce).
5. Sample size, duration, power. Precommit baseline, MDE, alpha, power, comparison count, read date, and any sequential rule. Use the user's policy when supplied; otherwise disclose

   alpha=.05

   and

   power=.80

   as conventional design assumptions, not universal truth. Convert required samples to duration and cover a full business cycle. Use

   experiment.py samplesize

   when available; the static table is only the

   .05/.80

   reference case.
6. Significance read (keyless compute or documented math). Name the method and apply the gate:
   • Two-proportion z-test for precommitted CVR/CTR rate comparisons, evaluated at the declared alpha.
   • Mann-Whitney U for non-normal continuous metrics (revenue per user, time on page).
   • Bootstrap confidence interval when you want a CI on the lift instead of only a p-value.
   • Report the declared-alpha statistical flag and the precommitted practical-effect flag separately. Adjust for multiple cells or repeated looks according to the design; do not retrofit thresholds after seeing results.
7. Apply decision ownership. First report facts: direction, effect/interval, statistical flag, practical flag, sample completion, and every guardrail. Then identify the decision owner and precommitted rule. Apply that rule only if both exist; otherwise emit

   decision: UNDECIDED

   and the exact missing approval. A guardrail stop can be mandatory only when that stop rule was declared before the read.
8. Label provenance. Raw export counts are

   User-provided

   (or

   Measured

   only when directly instrumented under the repository convention); p-values, intervals, power, and effect estimates are

   Calculated

   ; assumptions are

   Estimated

   . Reference measurement-protocol.md and roas-benchmark.md.

• test-design-guide.md — variant matrix, reference sizing table, statistical procedures, and decision-ownership matrix
• measurement-protocol.md — preregistration, multiplicity/sequential controls, practical effects, provenance, and decision ownership
• ROAS Benchmark — the O (Offer) and S (Spend-efficiency / CTR / CVR) levers this test informs
• CONNECTORS.md —

  ~~ad platform

  ,

  ~~web analytics

  ,

  ~~ecommerce

  own-data export recipes
• SECURITY.md — untrusted-data boundary for exported results