From Theory to Practice: The Impact of the NICE Real-World Evidence Framework in Health Technology Assessment

Background: On 23 June 2022, England's National Institute for Health and Care Excellence (NICE) published its Real-World Evidence (RWE) framework to optimise the use of real-world data (RWD) to support decision-making. It is unknown, however, the extent to which the framework is being explicitly invoked in appraisals.

Objectives: To identify and characterise references to the RWE framework in NICE single technology appraisals (STAs).

Methods: A structured review of NICE STAs published between 23 June 2022 and 31 January 2024 was conducted. All STA guidance documents published during this time were reviewed by 2 independent screeners for explicit mention of the RWE framework. Furthermore, using a published automated algorithm, all accompanying documentation for each STA was screened for relevant keywords. Multiple mentions of the framework in the same section or relating to the same discussion point were counted as one unique mention, as were those reproduced verbatim across different documents or sections.

Results: During the study period, 115 STAs were published, excluding those withdrawn, terminated, or replaced. Of the 115 STAs, 8 made at least one reference to the framework, with 5 mentioning it multiple times for a total of 15 mentions. Six of these appraisals were for cancer therapies, 1 for type 1 diabetes, and 1 for Covid-19. All 8 STAs included RWD to estimate treatment effectiveness: 4 as external control arms, 1 as an indirect treatment comparison, and 3 providing real-world effectiveness of the technology under consideration. Of the 15 mentions made, 8 were made by the manufacturer, largely to demonstrate adherence to the framework. The remaining 7 mentions were made by external stakeholders, NICE committees, or the External Assessment Group to highlight the value of RWE or critique the RWE submitted. Of the 8 STAs mentioning the framework, 4 therapies were given a positive reimbursement recommendation. A subgroup analysis revealed an approximately 33% increase in the number of appraisals including RWE from the end of June 2022 through 2023 compared to the year prior to the framework’s publication.

Conclusions: The NICE RWE framework appears to have spurred a greater inclusion of RWE in STAs, although the framework itself has been sparingly mentioned in published documentation. When mentioned it seems to be when RWE is being considered for treatment effectiveness, an area that generally is the most challenging in terms of stakeholder acceptance. Although limited direct references does not necessarily indicate the framework’s under-utilisation, there is opportunity for further transparency in its use to provide learnings for all stakeholders involved in HTA.