A Systematic Review of Variance Reduction Techniques in Online Controlled Experiments

Variance reduction in online controlled experiments encompasses changes to assignment, estimation, and the outcome being measured. This systematic review compares these mechanisms while retaining their estimands, information requirements, and inferential conditions. Exhaustively enumerated OpenAlex and arXiv searches to 25 September 2026, one backward citation pass, and bounded additional identification yielded 120 direct methodological works and 46 contextual sources, including three statistical foundations. Full-text extraction and retrospective descriptive coding mapped every main work to a primary method family, additional mechanisms, and forms of evidence. Covariate and prediction adjustment was the largest primary family, with 25 works; ratio, robust, distributional, and shrinkage estimation accounted for 19, and network or cluster design for 17. Analytical development appeared in 116 works, simulation or constructed experiments in 93, observed randomised contrasts in 56, and other empirical inputs in 51; these evidence categories overlap. The synthesis distinguishes exact fixed-coefficient identities from fitted-estimator asymptotics, target-preserving adjustment from metric construction, and variance from mean-squared error and detection frequency. Gains depend on assignment, treatment-effect heterogeneity, information timing, and uncertainty estimation. Incompatible baselines and targets preclude a pooled percentage benefit. Search restrictions, access barriers, and proprietary data limit the coverage and interpretation of the evidence.