(003) Building transparency and reproducibility into the practice of pharmacoepidemiology and outcomes research

Background: Real-world evidence (RWE) studies are increasingly used to inform policy and clinical decisions. However, there remain concerns about the credibility and reproducibility of RWE studies. While there is universal agreement on the critical importance of transparent and reproducible science, the building blocks of open science practice that are common across many disciplines have not yet been built into routine workflows for pharmacoepidemiology and outcomes researchers.

Objectives: To propose a framework for an explicit transparency statement that declares the level of transparency a given RWE study has achieved.

Methods: The proposed transparency statement for researchers is based on the Transparency and Openness Promotion (TOP) guidelines developed a decade ago for journals to encourage more transparent and reproducible scientific practices. We provide examples and guidance on how to explicitly state the level of transparency that a given RWE study has achieved across five key domains: 1) protocol, 2) pre-registration, 3) data, 4) code sharing, and 5) reporting checklists. This framework puts it in the hands of researchers to declare and display the levels to which they have built transparent and reproducible research practices into each study.

Results: Within each of the 5 domains, there are clearly defined “levels” of transparency that can be declared. If investigators meet the criteria for higher levels of transparency, their transparency statement can be accompanied by one or more open science badges that were developed by the Open Science Foundation and are currently used by many non-medical journals.

Conclusions: The transparency statement outlined in the present paper can be used by research teams to proudly display which open science practices were used to generate evidence designed to inform public health policy and practice. Display of such badges may prompt discussion and lead to cultural shift by inspiring others to earn similar recognition. While transparency does not guarantee validity, such a statement signals confidence from the research team in the scientific choices that were made. Akin to the transformation among trialists two decades ago, the power to realize a future where real-world evidence is recognized as consistently transparent, reproducible, and fit-for-decision-making lies in the hands of current and future researchers.