A tree-based scan statistic for database studies with time-to-event outcomes

Background: Tree-based scan statistics (TBSS) are popular methods to conduct disproportionality analyses in database studies. They simultaneously scan for thousands of hierarchically related outcomes to detect potential signals of harm from drugs and vaccines while controlling type 1 error. The outcomes are related using a tree that expresses how they can be aggregated meaningfully, for example, using the International Classification of Diseases or Multi-level Clinical Classification system. Current implementations of TBSS do not allow comparative safety evaluation with time-to-event outcomes. In most database studies, the time-to-event is available. Using this information can potentially improve power to detect signals.

Objectives: We implement a novel TBSS method that allows scanning thousands of hierarchically related outcomes with time-to-event data. The proposed approach extends the current implementation of TBSS test statistics, which use as input the number of events within a study period without accounting for person-time.

Methods: Our survival-TBSS tests whether the hazard rates computed at each node of the tree differ between two exposures of interest. The maximum observed Hazard Ratio (HR) among the tree nodes is used as a test statistic. Its null distribution is computed using Monte Carlo (MC) simulations that use a semi-parametric bootstrap to generate data replicates under the global null hypothesis that there are no outcome nodes for which there is an elevated HR for exposure. A global p-value is computed using the rank of the observed test statistic relative to the MC simulated null distribution. Node-specific p-values are defined via the rank of node-specific HR in the MC-generated null distribution for the global null. As in the standard TBSS, these p-values can be used to generate alerts that will be refined and evaluated with tailored protocol-based studies and chart reviews.

Results: In extensive simulation studies, we found that survival TBSS improves signal detection's power without compromising the type-I error controls compared with the unconditional Bernoulli TBSS. Using the same data replicates for the two models, the difference between the power of survival-TBSS and unconditional Bernoulli TBSS ranged from 8%, in settings close to the null, to 96%, depending on the true magnitude HR.

Conclusions: Our results show that when time-to-event data are available, their use in TBSS can largely improve the power of signal detection without compromising the type-I error control.