Revolutionising Regulatory Science with Natural Language Processing: a case study on Paediatric Investigation Plans

Background: Regulatory science heavily relies on the analysis of vast amounts of regulatory documents, including drug approval dossiers, adverse event reports, and safety data. However, manually reviewing and extracting relevant information from these documents is a time-consuming and labor-intensive task.

Objectives: In this article we aimed at developing a new framework for extraction of data from regulatory documents using natural language processing (NLP) and we present a case study focusing on the application of our framework to automate the extraction of critical information from Paediatric Investigation Plans (PIPs) documents.

Methods: The method involved identifying regulatory documents, implementing a web crawler to access the Public Register of PIPs, and processing data through various steps including text and metadata extraction. Descriptive analysis was conducted on retrieved PIPs, assessing completeness and correctness, validated through manual inspection. Additional analyses included evaluating median values for waiver decision support studies and visualization of temporal trends using line charts and bar plots. Comprehensive tabulation documented observed changes and corresponding variables.

Results: The analysis encompassed 6495 (98%) PDF documents from the PIP Registers, comprising multiple versions reflecting temporal changes. None of these documents contained watermarks, notifications, or summaries; however, 83 documents included embedded images. Median completeness of generated variables stood at 99%, with instances of lower completeness attributed to information availability rather than procedural errors. Validation on 120 randomly selected documents yielded a median correctness of 100%. Notably, all PIPs received their initial decisions within 60 days, with only 12.5% receiving decisions before 50 days. Common decisions for waiver included "No therapeutic benefit over existing products" (median: 59.8%), "Disease absence in pediatric subset" (median: 26.5%), and "Likely unsafe" (median: 10%). Waivers were primarily applied to the 10-18 age group (60% of documents).

Conclusions: In conclusion, our study introduces an innovative framework employing NLP for automated extraction of critical information which was applicable to PIPs documents. We achieved high completeness and correctness rates in extracted variables. Notably, common waiver decisions predominantly centered on therapeutic benefit, disease absence in pediatric subsets, and safety concerns. These findings highlight the potential of our framework in enhancing efficiency and insights in regulatory document analysis.