Library · Traffic · How to run this
Pre-commit n before you peek
n per arm is sessions each landing needs before you call a winner — not clicks in an afternoon. Type base convert, the relative lift you would actually ship for, and power. Write the number. Do not peek.
Pair: n for a paid landing test on Traffic.
How to read it
Type the three fields. Read sessions per arm. Log it next to the hypothesis. Kill/keep after n still uses campaign kill if the metric is CAC.
What not to do
Do not stop early because one arm “looks ahead.” Do not change the metric mid-test. Do not treat this as a PhD stats suite.
Related: Traffic, CAPI coverage.
Procedure
- Put base convert and the lift you would actually ship for.
- Read n per arm.
- Write it in the decision log.
- Do not stop early because one arm ‘looks ahead.’
Failure modes
- Calling a winner at 200 visits.
- Changing the metric mid-test.
- Peeking every hour.
Worked example
None on this page yet. When one lands it will be a real case: what broke, the number, and what changed. Not a composite.
Glossary
- n / arm
- Sessions each landing needs before you call a winner. Not clicks in an afternoon.
Work with us — We run this job on your accounts. Services
FAQ
- Can I call a winner at 200 visits?
- No. Pre-commit n per arm. Peeking is how you ship noise.
- Is this a full stats package?
- No. Simple two-arm n. Good enough to stop afternoon “winners.”
- What metric do I use?
- Settled convert or settled CAC — pick one before you start. Do not change mid-test.