Claude Code A/B Test Setup Skill for a Single Page Change
Turn a page change into a hypothesis, control, and treatment. Define the primary metric, sampling assumptions, and stopping rule before collecting results.
中文Install in Claude Code
Use Claude Code with the pinned skill directory. Bring baseline conversion, available traffic, a minimum meaningful change, and event definitions.
- Get the complete `cli-tool/components/skills/business-marketing/ab-test-setup` directory from the pinned GitHub checkout.
- Copy it under the target project's `.claude/skills/`. Confirm `.claude/skills/ab-test-setup/SKILL.md`, open Claude Code at the project root, and call `/ab-test-setup`.
Example task
Compare two button labels for a fictional course download page. Baseline completion is 8 percent and the smallest meaningful improvement is two percentage points. Change only the button wording.What to check
The plan defines visitor-level assignment, download completion as the primary metric, and sample size assumptions. Include a guardrail such as error rate and a predeclared stopping rule.
Original SKILL.md on GitHub · Updated
Frequently asked question
Why not stop as soon as the treatment is ahead?
Early differences can reflect noise. Choose an analysis method, sample size, and stopping rule in advance and evaluate using that rule.