(189) Improving the Role of External Control Arms in Early-Phase Single Arm Trials

Background: External control arms (ECAs) are increasingly used as comparators for single-arm trials (SATs), and in early phases to guide later-phase development prior to regulatory and payer considerations becoming primary concerns. Imperative to assess implications and enhance use of ECAs.

Objectives: To determine the validity of ECAs as comparators for early-phase SATs and facilitate implementation in guiding decision-making processes.

Methods: Design
Assess the reliability of results from hypothetical real-world data comparing SATs with ECAs via generalizable simulations.

Setting
A simulated cohort of 10,000 observations to represent a hypothetical target population. Patient characteristics included simulated baseline value (x), treatment group (SAT/ECA), age, sex, severity scale (0-5), and binary outcome variable (Ybin). Treatments assigned to force confounding. Simulation scenarios (replicated 100 times each) included varying treatment group sample size sampling combinations.

Exposures
Exposure was identified by the treatment group variable (0=SAT, 1=ECA).

Main Outcome Measures
A simulated binary response; function of all other simulated covariates.

Statistical analysis
Three matching approaches were employed to achieve a 1:1 sample size ratio between SATs and ECAs: 1) Random sampling; 2) Propensity score (PS) Greedy matching (k=1) using x, sex (exact matching), and age; 3) same as 2) including severity scale, unmeasured confounder in 1 and 2); 4) same PS model as 3) with inverse probability treatment weighting (IPTW) implementation. Logistic models were performed. Results compared known and observed treatment group effects in terms of odds ratios (ORs).

Results: When n is low (n=15 or n=30) and the true treatment effect is null (OR=1), the fully adjusted model (3) returned a misleading OR ( < 0.5, >2.0) in 31% and 18% of the simulations, respectively. With an increased sample size (n=50 or n=100), 4% and 0.5%, respectively, of the simulations produced incorrect results. When the true treatment effect is positive (OR≈1.4, OR≈2.5), the chances of obtaining counterintuitive results (OR < 1) were 27% and 8%, respectively, for n=15. When n is low, IPTW returned parallel results to 3) with advantages when the true treatment effect is null 28% (n=15) and 8% (n=30).

Conclusions: Limited statistical power in small SATs necessitates reliance on point estimates for advancing drug. This simulation study demonstrates high risks of misleading estimates in very small SATs, regardless of ECA size. While advocating ECA use, fine-tuning covariates and ECA selection remain critical. To appropriately contextualize results from small SAT-ECA studies for publication or internal decisions, we recommend re-sampling techniques to assess the robustness of the observed results.