All tags
Tag · 3 articles

#Experiment QA

Articles about Experiment QA: practical guides, analysis and case studies with their market context clearly identified.

Use an A/A test to check the experiment pipeline
PerformanceGlobal frameworkOctober 18, 20263 min

Use an A/A test to check the experiment pipeline

Design an A/A dry run that checks assignment, exposure, event delivery and reporting joins without treating one nonsignificant result as certification.

Keep repeat visitors in one A/B test group
PerformanceGlobal frameworkOctober 18, 20263 min

Keep repeat visitors in one A/B test group

Specify a stable assignment unit for repeat visits, cookie loss and login transitions, with a concrete exposure contract and QA matrix.

Check sample ratio mismatch before reading a test
PerformanceGlobal frameworkOctober 18, 20263 min

Check sample ratio mismatch before reading a test

Compare expected and observed assignment counts, reproduce a sample-ratio check and investigate a suspicious split before interpreting conversion lift.