(168) Outcome Validation to Assess Misclassification Bias in Studies Using Electronic Healthcare Claims

Background: Outcome validation studies often report a positive predictive value (PPV) that is uninformative about the presence or extent of outcome misclassification bias. Instead, estimates of PPV and sensitivity (Se) in each treatment group are needed to fully assess outcome misclassification bias.

Objectives: Describe an outcome validation strategy to assess bias due to misclassification and illustrate this strategy in an example study of the risk of acute myocardial infarction (AMI) among new users of two medications.

Methods: New users of two medications were identified in the Healthcare Integrated Research Database. The study algorithm required an inpatient facility claim with a primary diagnosis of AMI (ICD-10-CM I21). A second screening algorithm was developed to capture all cases of AMI and required a diagnosis of AMI or angina pectoris (ICD-10-CM I20) from any setting and in any position. Both algorithms were applied to the study population to identify AMI cases. Plausible misclassification rates as might be obtained from an outcome validation study were applied to the cases to enable estimation of the PPV and Se of the study algorithm among users of both treatments separately. The PPV of the study algorithm was calculated as the number of true positive cases identified by the study algorithm divided by the number of all cases identified by the study algorithm. The Se of the study algorithm was calculated as the number of true positive cases identified by the study algorithm divided by the number of true positive cases identified by the screening algorithm. The incidence rate ratio (IRR) and 95% confidence interval (CI) were calculated to compare the rate of AMI in the two cohorts. The treatment-specific estimates of PPV and Se of the study algorithm were used to estimate a misclassification bias-adjusted IRR.

Results: A total of 5,578 and 14,217 new users of Drug A and Drug B were identified. The study algorithm identified 21 and 18 possible AMI cases among Drug A users and Drug B users. The unadjusted IRR of AMI comparing Drug A to Drug B was 3.15 (95% CI: 1.68, 5.92). The screening algorithm identified 77 and 108 possible AMI cases among Drug A users and Drug B users. After applying hypothetical misclassification rates, the PPV of the study algorithm was 0.95 (95% CI: 0.76, 1.00) and 0.83 (95% CI: 0.59, 0.96) among Drug A users and Drug B users, and the Se was 0.95 (95% CI: 0.76, 1.00) and 0.65 (95% CI: 0.43, 0.84). After correction for outcome misclassification bias, the bias-adjusted IRR was 2.46 (95% CI: 1.15, 5.15).

Conclusions: Studies using an algorithm with a high PPV are still susceptible to meaningful outcome misclassification bias. Multiple validated algorithms, including a high sensitivity screening algorithm, can be used to assess misclassification bias in comparative studies.