Improving Effect Estimates by Limiting the Variability in Inverse Propensity Score Weights

This study describes a novel method to reweight a comparison group used for causal inference, so the group is similar to a treatment group on observable characteristics yet avoids highly variable weights that would limit statistical power. The proposed method generalizes the covariate-balancing propensity score (CBPS) methodology developed by Imai and Ratkovic (Imai, K., and Ratkovic, M., 2014), ”Covariate Balancing Propensity Score,” Journal of the Royal Statistical Society, Series B, 76, 243–263, to enable researchers to effectively prespecify the variance (or higher-order moments) of the matching weight distribution. This lets researchers choose among alternative sets of matching weights, some of which produce better balance and others of which yield higher statistical power. We demonstrate using simulations that our penalized CBPS approach can improve effect estimates over those from other established propensity score estimation approaches, producing lower mean squared error. We discuss applications where the method or extensions of it are especially likely to improve effect estimates and we provide an empirical example from the evaluation of Comprehensive Primary Care Plus, a U.S. health care model that aims to strengthen primary care across roughly 3000 practices. Programming code is available to implement the method in Stata.