Developing a de-novo mortality model for relapsed and refractory multiple myeloma: emulation of subjective life expectancy criteria from a clinical trial in real-world data

Background: Subjectively assessed life expectancy (LE) beyond a minimum threshold is an inclusion criterion in some oncology trials to ensure patient fitness. In clinical practice, LE is challenging to estimate and rarely documented and is thus unavailable in real world (RW) data. The absence of LE data is a key challenge in using RW external control arms (ECAs) to emulate the design of single arm trials.

Objectives: To develop a life expectancy model for 6-month mortality in patients with relapsed/refractory multiple myeloma (RRMM) with ≥3 prior lines of therapy (LOTs) who were previously treated with a proteasome inhibitor, immunomodulatory drug, and anti-CD38 antibody, or were triple-class refractory (TCR) to these treatments (3L+TCE/TCR).

Methods: We identified relevant patients using electronic health records from COTA. Eleven Cox proportional hazards models were fit with up to 21 key prognostic variables that were identified via systematic literature review and ranked by multiple myeloma experts, including cytogenic risk, age, refractory status, disease stage, performance status, and other clinical and demographic variables. Pre-specified minimum levels of predictive performance were defined as: F1 score ≥0.80, concordance index ≥0.70, calibration slope of 0.9 to 1.1, and pseudo R2 ≥0.20. Due to the presence of missing covariates, model performance was pooled over the 5 multiply imputed datasets. Bootstrap model validation was used to correct for optimism. A preliminary threshold of ≥50% predicted probability of mortality was used to classify events for sensitivity and specificity estimation.

Results: The cohort included 1,112 patients and 2,626 LOTs. Across all LOTs, the 6-month mortality risk was 26%. Adjustment for the top 20 prognostic factors resulted in a bootstrap-adjusted concordance index of 0.77, integrated calibration index of 0.01, calibration slope of 1.10, F1 index of 0.47, and pseudo R2 of 0.23. The sensitivity, specificity, positive predictive value, and negative predictive value of this model were 0.37, 0.93, 0.65, 0.81, respectively. The model met all pre-specified performance criteria, except for the F1 index.

Conclusions: Using a RW data source and expert-identified prognostic factors, it was possible to fit a model for LE of RRMM patients that met prespecified thresholds for concordance, calibration, and pseudo R2. However, the F1 index and sensitivity of the model at the preliminary 50% threshold were low, and consideration of the desired model properties is needed to choose an ideal classification threshold. Further validation using clinical trial data based on physician assessed life expectancy as an outcome, rather than observed mortality, may be useful to ensure that the model can be used to mimic the trial inclusion criteria.