Performance of the Self-Controlled Case Series with active comparators for signal detection in the Clinical Practice Research Datalink (CPRD): a case study on antibiotics

Background: Many methods have been evaluated for drug safety signal detection in Real World Data, but performance reporting remains heterogeneous. The Self Controlled Case Series (SCCS) is one of the more promising methods. Active comparators have been recently implemented to address time varying confounding by indication in SCCS but this innovation has not been evaluated in a signal detection context.

Objectives: The aim of this study is to evaluate the performance of the SCCS with and without active comparators for signal detection in the Clinical Practice Research Datalink (CPRD) Aurum.

Methods: We used the SCCS, with and without active comparators, applied to macrolides and fluoroquinolone antibiotics. In total 7 drugs, and 30 outcomes from all organ classes from the drug labels were selected. We developed a specific reference set using a taxonomy framework to ensure the selected drug outcome pairs are theoretically well suited to the SCCS design. The reference set contains 104 positive controls (drug-outcome pairs) and 58 negative controls in total. 2 active comparators (amoxicillin and cefalexin) with similar indications to macrolides and fluoroquinolones were chosen. All adults with at least one prescription of a drug of interest or active comparator and one outcome of interest during one of the two observation periods (01/01/1997-31/12/1998 and 01/01/2017-31/12/2018) were included in the study, with a 30 days risk-window after each prescription. We used simple ratio and effect modifier methods for active comparator analyses. Diagnostic performance was measured using sensitivity and specificity with respect to the product labels.

Results: The sensitivity and specificity of the SCCS without active comparator in the 2017 – 2018 period were 55.8% and 77.3% when limited to pairs with satisfactory power. Specificity increased up to 89.3% with active comparators due to better controlling for confounding by indication; however sensitivity decreased to 18.2% seemingly mainly due to the loss of power. Sensitivity and specificity without active comparators were similar in the 1997 – 1998 period (55.6% and 68.8% respectively).

Conclusions: The SCCS performed moderately for signal detection in CPRD. Active comparators, although they effectively reduced confounding by indication, also reduced the number of correctly identified positive controls. Possible reasons for not highlighting positive controls included variable levels of evidence of a causal association for certain drug outcome pairs despite inclusion in the drug label and broad outcomes leading to unspecific codelists.
Power was one of the main limitations of this work. We will look to replicate the findings in two larger databases: the US claims databases MarketScan and the French National Healthcare database.