Performance of Entropy Balancing & Covariate Balancing Propensity Score Methods for Controlling Confounding under Challenging Real-World Conditions in Observational Studies: Simulation Study in the CorEvitas Psoriasis Registry

Background: As an approach to control confounding in observational studies, inverse probability of treatment weighting (IPTW), using weights derived from propensity scores (PS) estimated by logistic regression, balances treatment groups insufficiently under certain conditions. Contributing factors may include moderate overall sample size, group size imbalance, degree of baseline confounder imbalance, and number of confounders to balance.

Objectives: 1) To compare the performance of IPTW using entropy balancing (IPTW-EB), covariate balancing propensity score (IPTW-CBPS), and logistic regression (IPTW-LR) across a range of real-world conditions for balancing observed confounders between treatment groups; and 2) to assess accuracy and precision of treatment effect estimation using IPTW-EB and IPTW-CBPS.

Methods: Using Monte Carlo techniques, we evaluated performance of IPTW-EB, IPTW-CBPS (over- and just-identified) and IPTW-LR methods on simulated data informed by real-world data from the CorEvitas Psoriasis Registry. 27 scenarios were considered, representing a hypothetical comparative drug persistence study conducted under a range of challenging but realistic conditions: total sample size N=600; treatment-to-control group ratios of 1:1, 2:1, 5:1; 10 confounders representing varying degrees of overall baseline covariate imbalance (low, moderate, high); and true population average treatment effects (PATE) corresponding to true conditional hazard ratios of 1.0, 0.8, 0.5.

Results: IPTW-LR often failed to balance all observed confounders in scenarios with a 5:1 treatment-to-control ratio (500 treated, 100 controls) and/or a high degree of baseline confounder imbalance. Both IPTW-EB and IPTW-CBPS (just-identified) balanced all confounders in all simulated data sets, including scenarios where IPTW-LR failed. IPTW-EB and IPTW-CBPS demonstrated low bias for estimation of the true PATE (for both, bias ranged from -0.02 to 0.01 across 27 scenarios and Monte Carlo error ranged from 0.10 to 0.23), with near-nominal Type I error rates (respective ranges 0.03-0.07 and 0.04-0.06). Effective sample size for IPTW-EB ranged from 73%-96% of the overall sample, and 66%-96% for IPTW-CBPS. Power was ≥0.84 for all scenarios with a large effect size (PATE 0.5).

Conclusions: In this study, IPTW-EB and IPTW-CBPS showed promise as alternatives to IPTW-LR for controlling confounding in observational studies under challenging real-world conditions such as moderate overall sample size with unbalanced groups. Further study of these approaches should include evaluation under other circumstances commonly encountered in practice – e.g., presence of unmeasured confounders.