Comparing the performance of probabilistic and rule-based phenotype algorithms for angioedema to the experience observed in randomized clinical trials.

Background: Rule-based phenotype algorithms (PAs) are the standard for identifying outcomes in observational data. However, the performance characteristics of the PAs, such as sensitivity and positive predictive value, are estimated to be low for many phenotypes. Probabilistic PAs, e.g., PAs based on logistic regression models, offer an alternative to the rule-based method.

Objectives: To develop probabilistic PAs for angioedema and compare the incidence of angioedema in exposed populations produced by rule-based and probabilistic PAs with incidence reported in randomized clinical trials (RCTs).

Methods: This retrospective cohort study included data from 1 electronic health record and 5 administrative claims databases from Merative, Optum, and IQVIA, all standardized to the OMOP Common Data Model. We characterized incidence of angioedema in 9 new user cohorts from 2010 to 2023 for captopril, enalapril, lisinopril, ramipril, losartan, valsartan, sacubitril/valsartan, saxagliptin, and sitagliptin. For each drug, we extracted incidence estimate and 95% confidence interval from RCTs as a basis for comparison. The rule-based algorithm used the first occurrence of a single diagnosis code for angioedema. Probabilistic PAs for angioedema were developed using the Observational Health Data Sciences and Informatics (OHDSI) software tool Patient Level Prediction. The supervised-learning model was informed by defined cases (i.e., subjects with 2 or more diagnosis codes for angioedema on the same day) and defined non-cases (i.e., subjects with no codes for angioedema in their record). We examined the results using probability cut-points of 0.5, 0.6, 0.67, 0.7, 0.75, 0.8, and 0.9.

Results: The mean incidence proportion or incidence rate for angioedema from the rule-based algorithm fell within the 95% CI in 23% of the RCTs. For example, a clinical trial for sacubitril/valsartan found 0.45% (95% CI 0.27, 0.70%) of subjects with angioedema whereas the rule-based algorithm estimated the proportion to be 0.12%. Using a probability cut-point of 0.67 for the probabilistic algorithm, we found that 77% of the results fell within the 95% CI of the RCTs. For example, in the same sacubitril/valsartan RCT, the probabilistic model estimated the proportion as 0.42%. The 0.67 cut-point performed better than the other cut-points, and each performed better than the rule-based algorithm.

Conclusions: We found that probabilistic models for angioedema estimated incidence closer to the results from RCTs than rule-based PAs. Further assessment of measurement error should help determine if phenotypes based on probabilistic PAs offer an improved alternative to rule-based PAs.