(170) Minimizing Confounding in Comparative Observational Studies with Time-To-Event Outcomes: An Extensive Comparison of Covariate Balancing Methods Using Monte Carlo Simulation

Background: Observational studies are frequently used in medicine to estimate the effects of treatments or exposures on outcomes. To minimize the potential for confounding when estimating treatment effects, covariate balancing methods are frequently implemented.

Objectives: This study evaluated, using extensive Monte Carlo simulation, many methods of covariate balancing and two methods for propensity score estimation, for estimating the average treatment effect on the treated (ATT) using a hazard ratio from a Cox proportional hazards model.

Methods: We focus on estimators of the average treatment effect on the treated (ATT) in the context of time-to-event outcomes. The overall design of the simulation is a 7 (Conditional Treatment Effect HR: 0.8, 1.0, 1.1, 1.25, 1.5, 1.75, 2) X 4 (Treatment Prevalence: 0.05, 0.10, 0.25, 0.40) X 2 (Sample Size: 1,000, 5,000), but also evaluate the effects of propensity score model misspecification with sample size of 1,000. The metrics for assessing performance were covariate balance, bias and accuracy (mean square error) of the estimated treatment effect, accuracy of the standard error of the estimated treatment effect, and effective sample size. The methods evaluated include: greedy pair matching, ATT weighting, optimal matching, full optimal matching, fine stratification, cardinality matching and stable balancing weights.

Results: With respect to minimizing bias and maximizing accuracy (as measured by the mean square error) of the treatment effect, ATT weighting, fine stratification and optimal full matching with a conventional logistic regression model for the propensity score performed best across all simulated conditions. Other methods did perform well in more selective circumstances, such as pair matching when sample sizes are large (n=5,000) and the proportion treated is < 0.25. There was also decreasing effective sample size with increasing number of strata, therefore for stratification-based weighting methods it may be important to consider fewer strata.

Conclusions: Generally, we recommend methods that performed well in our simulations as candidate methods and selecting the one method for evaluating the treatment-outcome relationship that optimizes some combination of balance and sample size/effective sample size.