Summary of the Session Exploring the Role of AI in Enhancing Cardiovascular Event Adjudication in Clinical Trials
Hiya Banerjee (Eli Lilly)
Introduction
Big data and artificial intelligence (AI) are transforming clinical drug development, revolutionizing various aspects of the healthcare and pharmaceutical industry. AI is being increasingly leveraged in clinical trials to enhance efficiency, expedite processes, reduce costs, and improve decision-making. By identifying eligible participants through diverse criteria, AI optimizes patient recruitment, streamlines trial timelines, and ensures efficient resource allocation. Additionally, natural language processing (NLP) facilitates the extraction of valuable insights from unstructured clinical data, such as electronic health records, medical literature, and patient narratives. AI also plays a crucial role in safety monitoring during trials, enabling real-time detection of adverse events and ensuring a proactive approach to participant well-being.
Despite the lack of clear regulatory guidelines, agencies like the FDA acknowledge the growing incorporation of AI and machine learning (ML) across the drug development lifecycle. The FDA has reported a significant rise in submissions involving AI and ML components, with over 100 applications in 2021 alone. These submissions span drug discovery, clinical research, post-market safety surveillance, and advanced pharmaceutical manufacturing. At the Regulatory-Industry Statistics Workshop (RISW), a session featuring industry and regulatory speakers highlighted the implementation of AI methods in drug development. Here, we present a highlights from the session.
AI in Clinical Event Adjudication
The first presentation focused on the application of NLP to automate the adjudication process for clinical events—a gold standard traditionally performed by a physician-led Clinical Events Committee (CEC). This process demands considerable time and expertise but is essential for reducing biases in investigator judgments and improving the reliability of safety and efficacy assessments in clinical trials.
Adjudication is particularly critical in cardiovascular (CV) outcome studies. When potential CV events such as strokes or heart failures occur, CECs independently review patient records. Discrepancies among adjudicators often require a third-party review or panel discussion. To streamline this process, AI models such as NLP and large language models (LLMs) were tested on clinical trial data, including patient demographics, doctor notes, ECG plots, and clinical event reports—amounting to up to 1,000 pages per patient. Optical character recognition (OCR) converted PDFs into text, while LLMs summarized the data for analysis.
Two AI models were compared: GPT-4 Turbo (LLM) and Longformer (a traditional NLP model). The results showed a concordance of 80% between the LLM model and CEC-confirmed outcomes for CV death, compared to 72% for investigator-assessed results and 62% for Longformer. LLMs proved faster and easier to integrate but lacked output probabilities and occasionally produced inconsistent results.
AI in Multi-Omics Drug Development
The second speaker delved into employing a multi-omics approach in drug discovery and development. AI and ML have enabled the automated generation of biomarkers for disease stage and progression, based on multi-dimensional data such as imaging and proteomics. Examples included proteomic scores for cardiovascular diseases and mortality, derived from plasma proteomics data in the context of lipid-lowering drug development. Another application involved proteomic risk scores and abdominal magnetic resonance imaging to evaluate liver fat fractions for non-invasive assessment of metabolic dysfunction-associated steatotic liver disease.
These biomarkers facilitate personalized strategies for clinical trial inclusion, targeting individuals based on predicted disease burden and treatment benefits. While substantial work is needed to validate these biomarkers as primary or secondary endpoints in clinical trials, they hold immediate promise for analyzing phase II trials and de-risking phase III trials.
Regulatory Perspectives on AI/ML in Drug Development
The third speaker, from the FDA, presented a comprehensive overview of AI/ML’s role in medical product development. Topics included a landscape analysis of FDA applications, a CDER discussion paper, and the Good Machine Learning Practices promoted by CDRH. Use cases illustrated the technology’s transformative potential, and the presentation highlighted the President’s executive order on AI/ML. Industry engagement programs with the FDA on AI/ML topics were also discussed, emphasizing collaboration to advance these technologies responsibly.
Conclusion
Throughout this session, all speakers emphasized that AI is reshaping clinical drug development by improving efficiency, accuracy, and personalization. From automating adjudication processes to enabling multi-omics biomarker generation, AI holds immense potential to revolutionize clinical trials. Regulatory support and collaboration between industry and oversight bodies will be crucial for the responsible integration of these technologies, ensuring their benefits are fully realized while maintaining safety and efficacy standards.