A/B Test Significance Calculator
Before you call a winner, check the math. Enter the visitors and conversions for each variant to see the conversion rates, the uplift, and whether the difference is statistically significant — or just noise.
What the calculator does
It runs a two-tailed z-test for the difference between two proportions. It compares each variant's conversion rate, computes the pooled standard error, and converts the z-score into a p-value — the chance you'd see a gap this big if the variants were truly identical. Confidence is simply 1 − p.
A significant result isn't the whole story: make sure the test ran long enough to cover normal day-to-day variation, and that the lift is big enough to matter for your business.
Promoto.io
Stop hand-tuning ad campaigns. Promoto writes, launches, and manages your Google, Meta & LinkedIn ads on autopilot.
Frequently asked questions
What does statistical significance mean in an A/B test?
It means the difference between your two variants is unlikely to be due to random chance. A result is usually called significant at 95% confidence, i.e. a p-value below 0.05.
What is a p-value?
The p-value is the probability of seeing a difference at least this large if the two variants actually performed the same. A p-value of 0.03 means there's a 3% chance the result is a fluke.
How many visitors do I need for an A/B test?
It depends on your baseline conversion rate and the size of the lift you want to detect — smaller lifts need far more traffic. As a rule of thumb, run the test until each variant has at least a few hundred conversions, not just visitors, before trusting the result.