(310) Retrospective Chart Review Data Collection to Generate External Comparator Arm Populations: Considerations and Suggestions for Reducing Key Biases

Background: External comparator arm (ECA) studies play a crucial role in comparing outcomes between real-world (RW) patients and single-arm trials, especially in rare diseases; multiple regulatory agencies have guidelines for ECA design. Retrospective chart review is a potential efficient method for building ECAs, and have been performed primarily in non-regulatory contexts. Chart review methodology has unique limitations to consider for ECAs.

Objectives: To describe the considerations and suitability of employing retrospective chart review methodology for ECA studies.

Methods: Based on experience (>25 chart review studies, including multiple ECAs), we discuss key considerations for designing historical cohort ECA studies, with a focus on potential biases/limitations of methodology. Practical suggestions to mitigate biases are provided.

Results: When existing databases are non-viable for comparison, chart review may be an option for data collection in ECA historical cohort studies. Aspects to address include: accurately identifying the population of interest, collecting the necessary variables and outcomes (including timing of assessments), and ensuring data quality.
Strategies should involve prioritizing key eligibility criteria and outcomes found in medical records, focusing on sites of highest relevance, and utilizing therapeutic area-specific expertise. Site feasibility can explore the availability of patients within a contemporaneous eligibility period, and accessibility of data. Any country-specific need for patient consent may bias which patients are included.
Disease diagnosis, key patient characteristics and outcomes like survival, treatment decisions are generally captured in charts across therapeutic areas. Therapeutic area-specific expertise is valuable in understanding how clinical outcomes are assessed, as outcomes are sometimes captured differently in charts (e.g., physician notes or clinical assessment) than trials (e.g., imaging or procedures). If possible, focusing on sites that assess outcomes more similarly to the trial, or at a relevant timing, may decrease heterogeneity.
Data abstraction accuracy can be improved with data verification, such as remote data review (review without the source) or source verification. Under-reporting of data in charts may include safety events (particularly non-serious events). Focusing on serious events or events of special interest can reduce this bias, as they may be more consistently reported in charts.

Conclusions: Retrospective chart review studies have limitations and biases within the framework of ECA studies. By addressing biases, such studies can provide valuable data and serve as viable options for ECAs in non-regulatory contexts.