A/B Test Calculator

Plan how many visitors you need before launching a test — and check whether the test you already ran is statistically significant.

Plan a test

Inputs

Your current control conversion rate.

Smallest lift you care about. 10–20% is typical for landing pages.

For a duration estimate. Combined across both variants.

Sample size needed

24,190visitors per variant

Total of 48,380 visitors across both groups. At 5,000 daily visitors that is roughly 10 days.

Two-tailed test for two proportions. Lower MDE = more visitors needed (relationship is quadratic). Run for full week-cycles to avoid weekday bias.

INTERACTIVE DEMO Want to test how this tool works?
Save these results as a PDF report A clean, branded copy to share with your team or client.
Free PDF report

Where should we put your name on the report?

Your PDF downloads straight away. Your tool data never leaves your browser.

Jobair Mahmud, Web Analytics & Tracking Architect
Tool Creator

Jobair Mahmud

Founder & Tracking Architect

100% Top Rated
★★★★★
5.0 / 5.0

Want to optimize your conversion funnels?

I configure GA4 custom event tracking, micro-conversions, and Looker Studio dashboards to measure test results accurately.

Set up A/B test tracking →

How to use the Statistical Significance A/B Test Calculator

Making business decisions on under-powered A/B tests can damage revenue. This calculator uses two-tailed statistical hypothesis testing to compute confidence intervals, p-values, and statistical power to verify whether a conversion rate lift is genuine or random noise.

Step by step

  1. Enter total visitors and conversions for Variation A (Control).
  2. Enter total visitors and conversions for Variation B (Challenger).
  3. Set your required statistical confidence level (standard is 95%).
  4. Check whether the test reached statistical significance and review the required additional sample size if needed.

How to read your result

Statistically Significant (p < 0.05)

95%+ confidence that the observed difference is real and not due to random sampling chance. Safe to declare a winner.

Inconclusive / More Data Needed

Observed lift does not meet the 95% confidence threshold. Continue running the test to reach the required sample size.

Common mistakes

  • Calling tests too early during the first few days before capturing weekday/weekend cyclical variance.
  • Testing multiple overlapping changes simultaneously without multivariate test architecture.
  • Stopping tests immediately after a brief positive spike caused by novelty effect.

Questions

What sample size do I need for an A/B test?

Required sample size depends on baseline conversion rate and Minimum Detectable Effect (MDE). Smaller uplifts require significantly larger sample sizes to prove statistically.

How long should an A/B test run?

A test should run for at least two full business cycles (typically 2 to 4 weeks) to account for day-of-week seasonality, even if statistical significance is reached early.

Want to optimize your conversion funnels?

I configure GA4 custom event tracking, micro-conversions, and Looker Studio dashboards to measure test results accurately.