A/B test sample size calculator
Enter your conversion rate and the smallest lift worth detecting. Get the visitors each variant needs and how long the test will take.
Detects a change from 3% to 3.6%.
Two-sided test. The split applies to A/B tests; 3 or more variants are split evenly.
- Control
- 13,914
- Total visitors
- 27,828
- Conversions per variant (about)
- 417 → 501
- Days at that traffic
- 56
- Plan for
- 8 full weeks
At 500 visitors a day the test needs about 56 days. Plan for 8 full weeks, and don't call it before then.
The sample size formula for A/B tests
Four numbers set the sample size. Three of them are choices you make before launch.
Conversion rate
Your current conversion rate. The lower it is, the more visitors each sale costs the test.
Lift to detect
The minimum detectable effect. Halve it and you need about four times the visitors.
Confidence and power
95% confidence limits false winners to 5%. 80% power means a real lift of that size is found 4 times in 5.
The calculator uses the standard formula for comparing two conversion rates with a two-proportion z-test, the same test ElasticFunnels uses to report statistical significance on a split test. For a 50/50 split the visitors per variant are:
n = (z1−α/2 · √(2·p̄·(1−p̄)) + z1−β · √(p₁(1−p₁) + p₂(1−p₂)))² ÷ (p₂ − p₁)²
where p₁ is the control's rate, p₂ the rate you want to detect, p̄ their average, α the significance level and 1−β the power. Uneven splits and extra variants are handled as described in the FAQ below.
How many visitors a split test needs
At 95% confidence and 80% power, 50/50. Small lifts on low conversion rates take the most traffic.
If a page gets 1,000 visitors a week, a 2% to 2.4% test would run for most of a year. Test bigger changes: the offer, the headline, the page length.
| Conversion rate | Lift to detect | Visitors per variant |
|---|---|---|
| 2% | to 3% (+50%) | 3,826 |
| 2% | to 2.5% (+25%) | 13,809 |
| 2% | to 2.4% (+20%) | 21,109 |
| 3.3% | to 4.05% (+23%) | 9,878 |
| 5% | to 6% (+20%) | 8,158 |
A minimum sample is a floor, not a target
Many tools flag results below a minimum sample. Reaching it does not mean the test can detect the lift you care about.
In ElasticFunnels split testing that floor is Min Sample Size per Variant in the Split Test settings: below it, results are marked "interpret with caution". Set it to the number this calculator gives you, run the test under one URL so your ads never change, and judge the result at that sample. The A/B test tutorial shows the setup click by click.
- Significance level (α)
- Chance of a false winner
- Power (1−β)
- Chance of catching a real lift
- MDE
- Smallest lift worth detecting
- p-value
- How surprising the result is if nothing changed
- Sample size
- Visitors per variant, fixed before launch
A/B test sample size FAQ
How do you calculate the sample size for an A/B test?
You need four inputs: your current conversion rate, the smallest lift you want to detect, the significance level (usually 5%, for 95% confidence) and the statistical power (usually 80%). The calculator puts them into the standard formula for comparing two conversion rates and returns the visitors each variant needs.
How many visitors does an A/B test need?
It depends on the conversion rate and the lift. At a 2% conversion rate, detecting a jump to 3% takes about 3,800 visitors per variant; detecting 2% to 2.4% takes about 21,100. Small lifts on low conversion rates need the most traffic.
How many samples do I need for 95% confidence?
95% confidence alone does not fix a number. It sets how strict the test is about false winners. The sample size also depends on your conversion rate, the lift you want to detect and the power, which is why the calculator asks for all four.
What is the minimum detectable effect?
The smallest lift the test is designed to detect reliably. A 20% relative lift on a 3% conversion rate means detecting 3% to 3.6%. Halving the lift you want to detect roughly quadruples the sample size.
What happens if my sample size is too small?
The test has low power: a real improvement often comes out as not significant, and the winners it does find tend to overstate the lift. Either run longer, test a bigger change, or accept that the test can only detect large differences.
Does a 70/30 split need more visitors than 50/50?
Yes. The 30% arm collects visitors at 3/5 the pace of a 50/50 arm, so reaching the same number per variant takes 1.67 times as long. The larger control arm makes up part of it: for the same power, a 70/30 test needs about 1.2 times the total traffic of a 50/50 test.
Why do more variants need more visitors?
Each extra variant is another comparison with the control, and every comparison is another chance of a false winner. When you run 3 or more variants, the calculator applies a Bonferroni correction, dividing the significance level by the number of comparisons, so each variant needs more visitors.
How long should an A/B test run?
Until each variant reaches the sample size you calculated before launch, and for at least one full week so every weekday is in every variant. Divide the total visitors by your daily traffic and round up to whole weeks. Do not stop the first time the p-value dips below 0.05.
Is 1,000 visitors per variant enough?
Only for big lifts. A minimum such as 1,000 per variant (Min Sample Size per Variant in ElasticFunnels) is the point below which a result is flagged as low sample, not a target. At a 4% conversion rate, 1,000 visitors per variant can only reliably detect a lift of about 70%, from 4% to 6.8%.