A/B test significance calculator

What the calculation means

Conversion rate is conversions divided by visitors. Absolute change is B − A in percentage points; relative uplift is (B − A) ÷ A and is undefined when A has zero conversions. The editable example gives 10% versus 13%: a 3 percentage point difference and 30% relative uplift. These are teaching inputs, not a forecast.

The test pools conversions across both groups under the null hypothesis that their true rates are equal. It divides B − A by √[p × (1 − p) × (1/nA + 1/nB)], then calculates the two-sided normal tail. No continuity correction is applied. This tool requires all four expected converted/nonconverted counts under that null to be at least 5; otherwise it withholds the p-value.

The 95% intervals use the Wilson score method for each individual group. They are approximate confidence intervals across repeated samples, not a 95% chance that the observed effect will recur. Very small p-values are shown in scientific notation; values below 1e−300 are reported as a bound, never as zero.

Check the experiment before choosing a variant

• Assign visitors randomly, keep groups mutually exclusive and check that exposure and outcome tracking are consistent. Repeat visits, clustered accounts and paired observations need a different model.
• Repeatedly checking results and stopping when p drops below 0.05 invalidates the fixed-horizon interpretation. Use a planned stopping rule or an appropriate sequential method.
• Testing many variants, metrics or segments increases false positives. This page shows one unadjusted comparison; it does not apply a multiple-testing correction.
• A small p-value does not establish a useful effect, profit or causality on its own. Review effect size, costs, guardrail metrics and experiment execution. Observational before/after counts do not become a randomized experiment here.