Background: Assessment of causality is challenging in observational studies because of lack of excheangeability.
Objectives: To propose a semi-quantitative method combining Hill’s criteria and counterfactual framework to assess the level of likelihood of causality in observational studies. We evaluated this method in three observational vaccine studies of which a scientific panel previously performed qualitative assessment on the level of causality [Gvozdenović et al. (2021) BMC medical research methodology.]
Methods: In this previous assessment, the authors combined Hill’s criteria and counterfactual criteria to assess causality. As a next step, we proposed a rating algorithm assigning a weight to each criterion and deriving a global score, which is translated into four categories of causality: very unlikely, unlikely, likely and very likely. The method works as follows: first, the Hill’s temporality criterion is considered as mandatory to meet causality. If this criterion is not met, causality is rated as 'very unlikely'. Otherwise, we proceed with the other criteria. For each of the Hill's criteria (strength, consistency, specificity, biological gradient, plausibility, coherence, experiment and analogy) the following score is applied: not met -1, unknown/not applicable 0, met 1. For the counterfactual criteria we use the three components (excheangeability, positivity and consistency) applying the same score. In the next step, we apply different weights (1 or 2) based on the importance of the criteria. The total score is the sum of the individual criterion scores multiplied by the weight. The total score ranges from -15 to +15 and and is divided in four levels of likelihood of causality: very unlikely (-15 to -7), unlikely (-6 to 0), likely (1-7) and very likely (8-15). This proposed method was tested in the three observational studies mentioned above: a Burden Of Disease (BOD) study, Post Authorization Safety Study (PASS) and a Vaccine Effectiveness (VE) study.
Results: Based on the method we obtained a score of -5 to the BOD study, -7 to the PASS study, and + 13 to the VE study, translating to the following levels of causality: unlikely, very unlikely and very likely, respectively.
Conclusions: The proposed semi-quantitative method tested in three observational studies showed a consistent assessment of the level of causality with the previous scientific panel assessment.
