DataDad.
Join the closed betaJoin beta
DataDad / Free tools

A/B test planner and calculator

Find the traffic you need before changing a paywall, onboarding step, or landing page. Then check what the results support.

Two versions. One conversion goal. Random 50/50 assignment. All calculations stay in this browser.

Your test

What would make a difference?

Confidence sets the false-positive threshold. Power is the chance of detecting the improvement you chose, if it exists.

Traffic and timing

Count new, eligible people once. Included traffic is split equally between A and B. The 14-day default covers two weekly cycles; adjust it to your product.

Visitors per version

—

Total visitors
—
Estimated time to final results
—
Visitors included per day
—
—Gather visitors
—Wait for conversions
CompareRead the final result

Set the stopping point before you start.

Choose one conversion goal, the smallest useful improvement, and a stopping rule before the test starts. Repeated checks followed by early stopping increase false positives. This calculator does not support sequential testing, multiple variants, or several goals at once.

Randomly assign each person to one version and keep that assignment. Repeated sessions, overlapping groups, and revenue averages need different models. Let every visitor finish the same conversion window.

How we estimate sample size

We use a two-sided normal approximation for two independent proportions with equal group sizes. The null variance uses the pooled target rate; the alternative uses each planned rate. We round up and require at least 10 expected conversions and non-conversions per version. This is an estimate, not a guarantee of an exact power level. Sample-size method.

Enrollment time uses the larger of the traffic estimate and your minimum days. We then add the conversion window. Traffic changes, tracking loss, and delayed conversions can extend the test.

How we compare results

The result checker uses a pooled, two-sided two-proportion z-test. The chart shows a Wilson confidence interval for each rate. Their overlap is not the significance test. A p-value is not the probability that a version wins. NIST proportion test and Wilson intervals.

We withhold a final comparison when either version has fewer than 10 conversions or non-conversions, the entered sample target is unmet, or you have not confirmed completion. We also flag a 50/50 allocation mismatch at p < 0.001. Check assignment and missing events before trusting that result.

No clear difference does not prove the versions are equal. A statistically significant result can still be too small to matter to your business.

Example: a completed conversion test

Version A records 500 conversions from 10,000 people. Version B records 600 from 10,000. The conversion rates are 5% and 6%.

The difference is 1 percentage point, or 20% relative lift. The two-sided p-value is approximately 0.0019.

With a completed planned sample and conversion window, the calculator finds a statistically significant difference at 95% confidence. The p-value is not the probability that B wins. Decide separately whether the improvement is large enough to matter.

Put the result to work

Read the Retention guide for definitions and examples. Explore DataDad funnels to connect this work to your reports.