(BSPOT23) Evaluating performance and methodologies of machine learning and conventional pharmacoepidemiologic modeling to identify pregnant individuals at risk of giving birth to children with major congenital malformations

Background: With over 70% of pregnancies exposed to at least one medication and half of pregnancies unplanned, exposure to fetotoxic drugs may occur in the first trimester, which might be associated with increased risk of major congenital malformations (MCM).

Objectives: To identify profiles of pregnant individuals at risk of giving birth to children with MCM and to evaluate and compare the performance of machine learning (ML) and conventional pharmacoepidemiological modeling (PE) at predicting MCM and organ-specific malformations.

Methods: We conducted a retrospective cohort study in the Québec Pregnancy Cohort (QPC), including all pregnancies covered by Québec’s public prescription drug insurance and their children from 01/1998 to 12/2015. Risk factors for MCM identified from the literature and women’s characteristics 12 months before the date of conception and during the 1st trimester were considered alongside the 100 most used medications in the first trimester empirically identified from QPC. Robust poisson regression models were used to obtain adjusted risk ratios (aRR) and 95% confidence intervals. ML models (Logistic Regression, K-Nearest Neighbors, Support Vector Machine, Random Forest, XGBoost, Multilayer Perceptron) were built to predict pregnant individuals’ profiles at risk of MCM. Random oversampling was used and all models were trained using 5-fold cross-validation. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), receiver operating characteristic (ROC-AUC) and sensitivity-PPV (PR-AUC) curves were used to compare different performances.

Results: We analyzed 207,565 pregnancies. The prevalence of MCM was 9.95%, consistent with the Founders effect.
After accounting for potential confounders in PE, chronic hypertension (aRR: 1.11 [1.01 – 1.21]), pregestational diabetes (aRR: 1.26 [1.15 – 1.39]), depression (aRR: 1.08 [1.03 – 1.12]), asthma (aRR: 1.09 [1.05 – 1.13]), and autoimmune disease (aRR: 1.06 [1.03 – 1.10]) were associated with MCM. Some medication classes: antihypertensives (aRR: 1.23 [1.06 – 1.42]), antidiabetics (aRR: 1.27 [1.11 – 1.46]), antidepressive (aRR: 1.12 [1.03 – 1.21]) were also associated with MCM. Risk profiles varied according to organ systems.
Of all ML models, logistic regression performed the best and had balanced metrics. Average ML sensitivity, specificity, PPV, NPV, ROC-AUC, and PR-AUC were 0.25, 0.77, 0.51, 0.58, 0.51, and 0.69, respectively. ML models had a poor performance and could not distinguish between pregnancies with or without MCM. PE ROC-AUC was 0.53, showing a similar performance as ML.

Conclusions: Unlike ML, PE were able to estimate the strength of association between specific exposures and MCM. Using ML models showed no advantage over PE modeling.