Before every A/B test
Check that the test can reach significance within a realistic timeframe.
Estimate how many visitors you need to detect a given relative difference (Minimum Detectable Effect) between control and variant. Two-sided z-test on two proportions.
Current control rate. E.g. 4% = 4.0.
Smallest relative difference you want to detect. 10% = going from 4% to 4.4%.
Sum of both groups. Used to estimate duration.
Recommended: alpha 5%, power 80%. Industry standard for B2B A/B tests.
Formula: per-variant sample size for a two-sided z-test on two proportions, with power 1−β and significance α. The normal approximation is valid when np ≥ 5 and n(1−p) ≥ 5.
The calculator applies the classic sample size formula for two independent proportions, used by every serious statistics tool (Optimizely, AB Tasty, Evan Miller, etc.).
The control’s current conversion rate (e.g. 4%).
The smallest relative difference you want to detect (e.g. 10%).
Standard: alpha 5% (95% confidence), power 80%.
Enter daily traffic to get the number of test days.
n = (Zα/2 · √(2 · p̄ · (1−p̄)) + Zβ · √(p₁(1−p₁) + p₂(1−p₂)))² / (p₂ − p₁)²
Where p₁ = baseline, p₂ = baseline · (1 + relative MDE), p̄ = (p₁ + p₂) / 2. Zα/2 and Zβ come from the standard normal distribution.
Check that the test can reach significance within a realistic timeframe.
Prioritise feasible tests vs. tests that would need months of traffic.
Align Product, Growth and Data on duration and uplift expectations.
Justify that a given uplift requires a given volume — or the other way around.
We audit your A/B stack, your tracking and your roadmap. 5 days, €0 for B2B scale-ups.