(364) The Importance of Standardization in Observed-to-Expected Analysis: Using COVID-19 Vaccine Adverse Event as an Example.

Background: Observed-to-expected analysis has been widely used in the safety surveillance of medicines and vaccines. Traditionally, the observed number of cases is obtained from a spontaneous reporting system and compared with the expected number based on background incidence rates.

Objectives: To contextualize the impact of standardization in observed-to-expected analysis for adverse events after vaccination again COVID-19.

Methods: A population-level cohort study using electronic health records from the UK (CPRD GOLD) and the Netherlands (IPCI) was conducted. We constructed the background population cohort using data from 2017-2019. We identified individuals who received at least one dose of COVID-19 vaccines, and created cohorts for each vaccine dose by brand. The study outcomes were venous and arterial thromboembolic events (VTE, ATE), as composite outcomes as well as individual outcomes. Individuals with the study outcome during the 183 days prior to vaccination were excluded. We identified incident VTE and ATE events after vaccination, and compared this to the expected cases estimated from the pre-pandemic background population. We calculated the crude incidence rate ratios (IRR) and indirect age-sex standardized incidence ratios (SIR) within 30-, 60-, 90-, and 180-days after vaccination within each database.

Results: A total of 3,030,773 individuals were included in the analysis, with 2.9 million people in the first-dose cohort, 2.7 million in the second-dose cohort, 2 million in the third-dose cohort, and 0.8 million in the fourth-dose cohort.
Compared with the background population, the crude IRR showed higher than expected cases of both VTE and ATE up to 180 days after the 1st and 2nd doses of the ChAdOx1 vaccine in both the UK and the Netherlands. Higher than expected rates of VTE were also observed among 3rd-dose mRNA-1273 in IPCI and 3rd-dose BNT162b2 in CPRD. In CPRD, cohorts of all 3rd vaccines showed higher incidence rate of ATE comparing to the background population.
After age-sex standardization, an increased SIR of ATE was only seen during the 60 days after the first (SIR: 1.12 [95%CI: 1.00-1.26] in CPRD) and second (1.14 [1.02-1.28] in CPRD, 1.31 [1.06-1.61] in IPCI) dose of ChAdOx1. No increased SIR was observed for the mRNA vaccine cohorts.

Conclusions: While observed-to-expected analysis can provide timely information on detecting safety signals, this method is limited to high type 1 errors and could overestimate the risk. Standardized by age and sex should be applied whenever possible.