When aggregate conversion rates reverse the result
Use a two-device ledger to see how a changing traffic mix can reverse an aggregate conversion comparison, then standardize the mix transparently.
The total can point in the opposite direction
When traffic mix changes, the overall conversion rate can move differently from the rates inside each segment. This is the pattern usually called Simpson’s paradox. It is especially important in period or campaign comparisons where the share of desktop, mobile, country or customer type has shifted.
The total is a weighted ratio: all converting units divided by all eligible exposed units. A larger share of high-converting traffic can raise that ratio even when the segment-specific experience performs worse. Start with the counts to see which effect the dashboard is combining.
A two-device example with a complete reversal
These fictional periods use the same conversion definition and complete observation window. They are descriptive periods, not a claim about a valid randomized experiment. The intentionally large mix shift makes the arithmetic easy to inspect.
| Period and device | Conversions | Exposures | Conversion rate |
|---|---|---|---|
| A: desktop | 90 | 100 | 90% |
| A: mobile | 80 | 800 | 10% |
| A: total | 170 | 900 | 18.89% |
| B: desktop | 680 | 800 | 85% |
| B: mobile | 5 | 100 | 5% |
| B: total | 685 | 900 | 76.11% |
Desktop falls from 90% to 85%, and mobile falls from 10% to 5%. Yet the aggregate rises from 18.89% to 76.11%, because period B contains far more desktop exposure. Reporting only the total would turn two within-device declines into a broad improvement story.
The 90% and 10% figures are teaching inputs, not expected conversion benchmarks. The pattern can also occur with much smaller rates and less dramatic shifts. What matters is that segment performance and segment weights change together.
Calculate a common-mix view
Choose one explicit reference distribution. For a simple equal desktop/mobile mix, period A becomes 0.5 × 90% + 0.5 × 10% = 50%. Period B becomes 0.5 × 85% + 0.5 × 5% = 45%. Under that declared standard, the within-segment deterioration is visible.
Microsoft documents SUMPRODUCT as adding corresponding products, which can implement rate × reference-weight aggregation. The choice of reference weights is your analytical policy; the function does not make equal weights the right business mix. [Microsoft: SUMPRODUCT function]
A common-mix rate is a standardized comparison, not the actual historical conversion total. Keep the actual and standardized views labeled separately. A fixed business-relevant mix, such as the planned future exposure distribution, may be more useful than 50:50. State it before deciding which period looks better.
Find what actually changed
- Check segment definitions and coverage: device classification and missing values must be comparable.
- Inspect counts, not percentages alone, especially in a small segment.
- Show both the exposure share and conversion rate for each important segment.
- Keep unknown segments in the reconciliation instead of removing them from a preferred view.
- Separate traffic-acquisition changes from changes to the page or offer.
Segmentation does not prove causation. A device bucket may still contain a different country, offer or customer mix in the second period. Use it to identify the comparison that needs investigation. Avoid slicing until a favorable story appears; record the planned dimensions and any exploratory ones.
If this scale of mix shift appears in a supposedly balanced randomized experiment, investigate eligibility, assignment, exposure logging and filtering. Do not explain away a broken experiment as a benign traffic-mix effect.
Make the dashboard preserve both explanations
I would show the actual aggregate, the segment ledger and one declared standardized view together. The headline can then say that overall conversion rose because the traffic composition shifted while the observed device-specific rates fell. That is a more useful budget discussion than arguing over which single percentage is the true one.
Sources
AI-assisted writing and an AI-generated editorial cover. Source links and checked dates are listed below; illustrative examples are labeled in the text.
- Microsoft: SUMPRODUCT function — checked 2026-10-18