Percentage points vs. relative lift in conversion tests
Report conversion rates with raw counts, percentage-point change and relative lift, then separate the effect estimate from uncertainty and business value.
State both units for the same change
If a conversion rate moves from 4% to 5%, it rises by one percentage point. Relative to the original 4%, that one-point increase is 25%. Both descriptions are correct, but they answer different questions. Saying conversion rose by 1% would leave the reader guessing which unit you mean.
Use percentage points for subtraction between rates and relative lift for the difference divided by the baseline rate. Write the baseline beside the relative result. A 25% lift sounds much larger when its starting point is missing.
A reproducible two-group example
This fictional experiment assigns 2,000 eligible independent units to each group and records one yes/no conversion outcome per unit within the same complete observation window. The figures illustrate reporting arithmetic; they are not campaign results.
| Measure | Control A | Variant B |
|---|---|---|
| Exposed units | 2,000 | 2,000 |
| Converting units | 80 | 100 |
| Nonconverting units | 1,920 | 1,900 |
| Conversion rate | 4.0% | 5.0% |
Control rate = 80 / 2,000 = 0.04
Variant rate = 100 / 2,000 = 0.05
Absolute difference = 0.05 − 0.04 = 0.01 = 1 percentage point
Relative lift = (0.05 − 0.04) / 0.04 = 0.25 = 25%At 2,000 exposures, the observed gap corresponds to 20 additional converting units. A comparable future 10,000-exposure plan would imply 100 additional conversions if the one-point effect transferred exactly. That is a scenario, not a measured future result.
Put uncertainty beside the estimate
NIST distinguishes large-sample and exact methods for comparing independent binary proportions. The appropriate method depends on the data and assumptions, so the conversion counts matter as much as the displayed percentages. [NIST: Comparing two proportions]
A large-looking relative lift can still be uncertain. Under a basic independent-binomial approximation, the example’s standard error for the difference is about 0.66 percentage points. A rough unadjusted 95% interval is therefore about −0.28 to +2.28 percentage points. This illustrative interval includes zero and both commercially weak and useful outcomes; it is not a declaration that the variant has no effect.
Use the site’s significance calculator only when the independent binary setup and its large-sample method fit your data. Keep repeated-user clustering, sparse outcomes, multiple comparisons and any sequential analysis in the review. A polished percentage format cannot fix an invalid denominator.
Translate the absolute change into an operating decision
Suppose each accepted conversion contributes an illustrative USD 12 before the cost of the change. Twenty additional conversions would represent USD 240 at the observed group size if they had equivalent value. If implementation and added service work cost USD 700, the point estimate alone does not justify the rollout. Include the relevant time horizon and uncertainty rather than using relative lift as a business case.
The conversion definition should match that value. A form attempt, a saved inquiry and an accepted opportunity are different outcomes. If the variant makes more people submit but reduces acceptance, the raw conversion lift can overstate the commercial change.
- Report control and variant counts, rates and complete observation windows.
- Label subtraction as percentage points and the baseline-scaled change as relative lift.
- State the uncertainty method and whether the comparison was planned.
- Include value, operating costs and guardrails before choosing a rollout.
A report line that can be checked
A concise version is: “Variant B recorded 100 conversions from 2,000 exposed units, compared with 80 from 2,000 in A: 5% versus 4%, an observed increase of one percentage point or 25% relative. The uncertainty interval still includes outcomes below the launch requirement.” Replace the final sentence with the result from the method actually used.
When the baseline has zero conversions, relative lift is undefined. Report raw counts and the absolute rate difference, then use a method suitable for sparse outcomes. Do not label the result an infinite improvement.
Sources
AI-assisted writing and an AI-generated editorial cover. Source links and checked dates are listed below; illustrative examples are labeled in the text.
- NIST: Comparing two proportions — checked 2026-10-18