High-Dimensional Multiple Imputation (HDMI) For Partially Observed Confounders Including Natural Language Processing-Derived Auxiliary Covariates

Tuesday, August 27, 2024

8:45 AM – 9:00 AM CEST

Location: Conv Hall I D

Presenting Author(s)

Janick Weberpals, RPh, PhD (he/him/his)

Instructor in Medicine
Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA
Boston, United States

Slides Handouts

Co-Author(s)

PS

Pamela A. Shaw

Biostatistics Division, Kaiser Permanente Washington Health Research Institute, Seattle, WA, United States
Kueiyu Joshua Lin, MD, ScD

Assistant Professor
Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School
Boston, United States
Richard Wyss, PhD

Assistant Professor
Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA
Boston, United States
JP

Joseph M. Plasek

Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, United States
LZ

Li Zhou

Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, United States
KN

Kerry Ngan

Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, United States
TD

Thomas DeRamus

Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA, United States
Sudha R. R. Raman, PhD (she/her/hers)

Assistant Professor
Department of Population Health Sciences, Duke University School of Medicine, Durham, NC
Durham, United States
Bradley G. Hammill, DrPH (he/him/his)

Associate Professor
Department of Population Health Sciences, Duke University School of Medicine, Durham, NC
Durham, United States
HL

Hana Lee, PhD

Office of Biostatistics, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD, United States
DT

Darren Toh, ScD

Department of Population Medicine, Harvard Pilgrim Health Care Institute, Boston, Massachusetts, USA, United States
JC

John G. Connolly

Department of Population Medicine, Harvard Pilgrim Health Care Institute, Boston, MA, United States
KD

Kimberly J. Dandreo

Department of Population Medicine, Harvard Pilgrim Health Care Institute, Boston, MA, United States
FT

Fang Tian

Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, US Food and Drug Administration, Silver Spring, MD
Silver Spring, United States
WL

Wei Liu, PhD, MSc

Office of Surveillance and Epidemiology, US Food and Drug Administration
Silver Spring, United States
Jie J. Li, PhD (she/her/hers)

Associate Director RWE
Office of Surveillance and Epidemiology, CDER, US FDA
Silver Spring, United States
José J. Hernández-Muñoz, RPh, MPH, MSc, PhD

Pharmacoepidemiologist
Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, U.S. Food and Drug Administration, Silver Spring, USA
The Woodlands, United States
Sebastian Schneeweiss, MD, ScD (he/him/his)

Professor of Medicine
Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital, Harvard Medical School
Boston, United States
Rishi J. Desai, PhD

Associate Professor
Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women’s Hospital and Harvard Medical School, Boston, Massachusetts
Boston, United States

Background: The performance of multiple imputation (MI) models may be enhanced using auxiliary covariates (AC). Data-adaptive approaches to identify ACs for MI models are not well understood.

Objectives: To develop and compare high-dimensional MI (HDMI) approaches that leverage structured and natural language processing-derived ACs and investigate whether HDMI can increase efficiency and decrease bias in real-world evidence studies with partially observed confounders.

Methods: We designed a plasmode simulation study using a cohort of opioid vs NSAID drug initiators from Medicare claims linked to Mass General Brigham electronic health records in Boston. A complete cohort of patients with observed serum creatinine (sCr) labs was identified and time to acute kidney injury was defined as the outcome of interest. We simulated 100 cohorts with an investigator-defined null treatment effect (hazard ratio = 1) using sCr and 13 pre-defined confounders in the outcome-generating model. We imposed missingness on 50% of the sCr labs using a set of investigator-defined predictors (inert variables) that were not confounders but were associated with sCr missingness in the empirical cohort. To mimic scenarios where the missingness predictors are unmeasured, we subsequently left the inert covariates out of the set of candidate ACs for the HDMI models. We compared different sets of candidate ACs for imputing sCr, all measured in the 365 days pre-treatment: (a) Variables of the outcome-generating model only (baseline; included in all subsequent models), (b) structured claims indicating presence of a medical code, (c) unigrams indicating presence of a qualifying word from unstructured charts, (d) sentence embeddings as numeric representations of the semantic meaning from unstructured charts, (e) b + c, and (f) b + d. ACs in HDMI models (b)-(f) were included in the imputation model if they were selected as predictors for sCr missingness by LASSO regression. We also performed a complete case (CC) analysis and compared models in their ability to recover the defined effect.

Results: Baseline model (a) led to a 93% confidence interval coverage, root mean squared error (RMSE) of 0.175 (0.151-0.197) and % bias of 0.074 (0.043-0.106). We observed improvements using HDMI model (e) as being the most efficient with a 95% coverage and showing the lowest RMSE (0.169 [0.144-0.191]) and % bias (0.067 [0.036-0.098]). CC showed the worst performance (RMSE 0.315 [0.235-0.378]).

Conclusions: HMDI performed well and may be considered in scenarios where missingness is suspected to depend on not obviously observable factors. Considering the computational effort, the improvement was modest and scenarios need to be identified where HDMI may show a clearer benefit.