A/B Test Sample Size Calculator
Run a test too small and the result is noise; the time to find out is before you launch, not after. Enter your baseline rate and the lift you care about to get the visitors per variation you need — and how long that will take.
Decide the effect you care about first
The single biggest driver of sample size is the minimum detectable effect — and it's the input people fudge. Halving the MDE roughly quadruples the visitors you need, because the formula scales with the inverse square of the effect. So be honest: is a 2% relative lift even worth shipping? If the smallest change that would change your decision is a 10% lift, set the MDE there and don't ask the test to chase smaller ghosts it would need months to confirm. Low-traffic sites should test bold changes with big expected effects, not button-colour tweaks.
Lock the number, then resist the urge to peek
This calculator's whole purpose is to let you commit to a stopping point before emotion enters the room. Once a test is live, watching it hourly and stopping the moment it crosses significance feels rigorous but quietly wrecks your results: every extra peek is another chance for random noise to cross the line, so your real false-positive rate climbs well above the 5% you signed up for. Run to the planned sample, then judge it once. If you must monitor early, use a sequential-testing method built for it — not the fixed-horizon math here.
Promoto.io
Stop hand-tuning ad campaigns. Promoto writes, launches, and manages your Google, Meta & LinkedIn ads on autopilot.
Frequently asked questions
How do I calculate A/B test sample size?
Sample size depends on four things: your baseline conversion rate, the minimum effect you want to detect, your significance level (usually 95%), and your statistical power (usually 80%). The standard two-proportion formula combines them into a required number of visitors per variation. Smaller effects and higher confidence demand far larger samples.
What is minimum detectable effect (MDE)?
The minimum detectable effect is the smallest improvement you want the test to be able to spot reliably. A relative MDE of 10% on a 4% baseline means detecting a lift to 4.4%. The smaller the effect you want to catch, the more traffic you need.
Why shouldn't I stop a test early?
Calculate the required sample first, then wait until you reach it. Repeatedly checking and stopping the moment a result looks significant ('peeking') dramatically inflates your false-positive rate, so you declare winners that aren't real.