ML/AI Innovation in Regulatory: Interview with Andrea Manfrin
Di Zhang (Teva), Maria Kudela (Pfizer)
Andrea has over 30 years of international experience as an academic, consultant, and entrepreneur in the healthcare sector. He joined the Medicines and Healthcare products Regulatory Agency (MHRA) in June 2023 as Deputy Director of Clinical Investigations and Trials. His ambition is to facilitate regulatory change, making the UK one of the best places for sponsors and patients to conduct clinical research. Before working at the MHRA, Andrea was at the University of Central Lancashire (UCLan, UK) as the Faculty Director of Research and Innovation in the Faculty of Clinical and Biomedical Sciences and Chair Professor of Pharmacy Practice, where he led the conceptualization and development of clinical trials in medicine, dentistry, pharmacy, and health services research. He continues to hold a visiting professor position at UCLan.
We were grateful to have the opportunity to sit down and talk about the exciting developments at MHRA with Andrea.
What are some of the AI/ML related initiatives in MHRA?
We are now planning to test and deploy AI/LLM in three different areas: 1) support assessors during the clinical trial review, 2) support training of new assessors, 3) support sponsors by providing AI-driven assistance for common regulatory queries via the MHRA Web system.
We manage a significant volume of clinical investigation and trial applications. Last year alone, we processed more than 5300 applications, including 83 initial clinical investigations and 290 amendments, and 761 initial clinical trials, and over 4200 amendments to existing applications. These documents are often extensive and detailed, requiring a considerable amount of time for our assessors to review and extract critical information. To address this challenge, we focused on clinical trials and spent the summer of 2024 exploring various digital tools to enhance our efficiency. We tested several off-the-shelf AI tools but found none that met our specific needs. Consequently, we partnered with a company through our digital and technology team to develop a custom AI tool from scratch. This tool was designed to help assessors quickly locate necessary information, significantly reducing the time spent on this task from hours to just 34.5 seconds. This frees up assessors to focus on more complex and knowledge-intensive activities, such as evaluating the safety and benefit-risk profiles of the trials.
We also developed a training tool to help new assessors onboard more efficiently to not only help assessors get up to speed but alleviate the pressure on senior staff who were previously responsible for mentoring new employees.
We are also creating a tools for sponsors to pre-check regulatory concerns, which helps to strengthen their submissions.
Our AI tools will continuously improve, using some sources of information held in our data bank such as a dataset including 110,000 grounds for non-acceptance (GNAs, questions raised to the sponsors) that we use to train the system. This dataset helps the AI identify potential issues and improve its accuracy over time. The evidence from the literature shows that AI and natural language processing, save up to 70% of the time required to find information. By adopting this approach, our assessors should have more time to engage in more valuable activities, such as providing scientific advice to sponsors.
On April 22nd, following two successful proof of concept studied conducted between October 2024 and March 2025, we deployed the first two AI/LLM tools: 1) General Manufacturing Practice (GMP) compliance checker, 2) The knowledge hub using the searchable data base for past GNAs and assessment reports. Overall, our efforts should lead to significant improvements in efficiency and effectiveness in managing clinical trial applications, benefiting both our team and the sponsors we work with.
It's important to understand that while AI can provide valuable insights, we to ensure that humans, our assessors, are ultimately responsible for making the final decisions. For instance, consider a medical doctor conducting a brain or body scan on a patient. The machine performs the scan and generates the information, but the doctor interprets the data and makes the final decision, determining whether treatment is needed or if the scan is clear. We're using AI to support our assessment activities, not as a replacement. Only the assessors will make the final decision.
What unique challenges do you face when implementing AI/ML in a regulatory environment?
I must say we ventured into uncharted territory, as we've never done this before in our division and in my career. It was entirely new for us, and to the best of my knowledge ge we're among the few divisions in the MHRA testing this kind of activity, especially while supporting the assessment of submissions. Given our funding constraints and workload pressures, we knew we needed to find innovative ways to support our work. This led us to explore new avenues.
While I've used machine learning before, for example artificial neural networks, I had never applied it at this level. Fortunately, my colleagues, who are open-minded and passionate about AI, helped us start looking for solutions. The collaboration with the external partner was fundamental. The biggest challenge was realising we needed to create a solution from scratch, as there wasn't an off-the-shelf option available.
Through my colleagues in the digital team, we connected with an external , which has a great multidisciplinary with diverse skills. Statistics and machine learning were the common themes that united us, as we aimed to develop something truly innovative and new in our field. We followed the law of diffusion of innovation, starting with early innovators who think outside the box, and gradually building interest across the agency. Now, many people are involved in this programme, and it has been a great experience.
What trends do you see shaping the future of AI/ML in regulatory settings?
Currently, there are numerous AI projects underway across the MHRA, although I'm not familiar with the specifics of each programme. So far, we’ve focused on controlled pilots to ensure the tools are consistent and reliable. Now, we’re branching out - into areas like assessment support, training, , with more branches to come. These “branches” are all part of the same tree: a regulatory system supported by AI.
We see potential in using AI for various purposes, including finding solutions for clinical investigations and improving team efficiency. For instance, one of our teams, the Clinical Investigation and Trials Operation team, is already using basic AI tools like Copilot to manage workload and data analytics handling vast amounts of information and data. They are involved in many areas, such as developing clinical trial guidance for the new clinical trials regulations supporting point-of-care manufacturing, .
Our AI programme aims to create evidence, develop new regulatory pathways in life sciences, address gaps and challenges, and ensure long-term sustainability – all while co-creating tools and systems with others. The above examples show how we started to use AI , machine learning, and related technologies.
One exciting example is our involvement in the creation of Centre of Excellence for Regulatory Science Innovation (CERSI). Within this, we (Clinical Investigations and Trials at the MHRA) supports the CERSI on in-silico trials, - virtual simulations of how medical products and devices might behave. In-silico trials could help us move faster from preclinical to clinical testing, and address issues like recruitment delays. This CERSI initiative involves 85 organisations, including regulators, universities, and pharmaceutical companies. It's amazing how these entities have come together to develop new approaches for clinical investigations and trials testing.
While I can only speak for my division, I believe AI has significant potential within the regulatory environment. This includes leveraging AI to enhance our activities and supporting sponsors using AI for trials, data collection, and device usage. Another specialized team within our group, focuses on these aspects.
How do you foster collaboration and communication among team members from different disciplines?
Within our Clinical Investigations and Trials unit, we have three teams: the Clinical Trials team, the Clinical Investigations team, and the Clinical Investigations and Trials Operation team (CIT OPS). We’ve created a culture that is very flat, with no hierarchy, allowing everyone to express their ideas and concerns easily. Information flows quickly, which is essential since we are legally required to complete (for example) the initial clinical trial review within 30 days. This short timeframe necessitates agility and rapid information exchange.
We share information through various methods, including face-to-face meetings and data exchanges. For instance, the CIT OPS team developed new tools last year that provide real-time data analysis, which we didn't have before. This advancement allows the heads of different disciplines such as clinical non-clinical and pharmaceutical to view workflows immediately and plan work allocation more efficiently.
Although we are still under pressure, these tools give us a better overview of the situation and help us identify triggers for action more quickly.
Outside of our internal collaborations, we are also involved in the Access Consortium. It's a collaboration of five regulators from Australia, Singapore, Canada, Switzerland and us, representing around 150 million people. The diverse population helps achieve generalizable data. Since summer 2024, we have worked with colleagues across these jurisdictions aiming to safely enhance clinical trial delivery. We aim to start this program in autumn 2025, benefiting both us and patients. The first challenge was the time zone differences, but we've adapted. Working in a consortium requires a lot of open communication and collaboration to achieve our goals and address various country-specific regulations.
What advice would you give to someone aspiring to lead interdisciplinary teams in AI and machine learning?
Leading in AI and machine learning doesn't necessarily require a completely different skill set than leading in other areas. Leadership isn't solely about technical knowledge; it's a more holistic approach. I'd like to provide three examples of leadership.
Firstly, anyone leading AI and machine learning programs needs to have an open mind and shouldn't be intimidated by technology. They should work with people who can explain the technology simply, which has been invaluable to me. My colleagues have helped me understand complex concepts in a digestible way.
In terms of leadership overall, even when focusing on AI and machine learning, I believe leadership can be summarised through three individuals. The first is Simon Sinek, a cultural anthropologist and TED speaker, who famously said, "Managers eat first, leaders eat last." To be a good leader, you need to listen more and talk less, paying close attention to your team. You don’t have to know everything - and pretending you do won't get you far.
Secondly, I’m inspired by Harvard Business School professor Linda Hill, who suggests moving from a vision-based leadership to shaping the culture. This modern approach is necessary because achieving complex goals requires collaboration with diverse teams and the key element is co-creation. Hill outlines three functions of leadership called the ABC of leadership:
A, the architect builds the company’s culture and capabilities for innovations, leveraging different people’s skills and capabilities (“Collective Genius”). For example, we created something that did not exist before, the AI/LLM tools.
B, the bridger, knows that its company lacks all the talent and tools it needs to innovate quickly and efficiently. This is why we wanted to work with an external company that has a great and talented team of people with a completely different skillset to enable us to innovate and create the AI/LLM
C, the catalyst, accelerates the co-creation across the entire eco-system. A good example is the collaboration with the CERSI for in-silico trials, which could improve patient safety, reduce the time and costs for device and drug development and therefore, bring new devices and medications to patients faster. This activity could reshape the entire clinical trials eco-system
Creating something new is often a bumpy ride, requiring multiple iterations, but maintaining momentum and positivity is key to eventually achieving results.
Finally, Steve Jobs famously once said that it's pointless to hire smart people and then tell them what to do. Instead, it's better to hire them so they can tell us what to do. This means listening to a range of views, but ultimately making the decision and taking responsibility for it. A good leader takes responsibility for the outcomes: if the decision is right, the credit goes to the team; if it's wrong, the leader takes the blame. That’s what leadership is.
In essence, this captures my view of what makes a good leader. I don’t claim to be a perfect leader myself, but these are the qualities I believe are essential.
Where do you see the role of statistician within AI/ML related initiatives?
My Ph.D. focused on developing randomised control trials and the applications of advanced statistics. However, the statistics I've encountered in the past year have gone far beyond that. Interestingly, until a few years ago, statistics was often seen as a supporting discipline. Now, data science, machine learning, and AI have become the new currency and vocabulary, with other disciplines supporting AI and machine learning as the new cornerstone.
I believe AI and machine learning should be integrated into everyone's training, because our future work will increasingly involve these tools. That said, we’ll always need true experts in AI, as it will become crucial for exchanging information and analytics.
7. You will be publishing one of your articles in the summer issue. Could you tell us more about the upcoming article?
Thank you so much for this opportunity to share our work. The paper has several goals: sharing what we’ve learned, including our challenges, and to show that we brought together many skillsets, including software engineers, data scientists, assessors, and more. We will also describe our methodology and process, making AI and machine learning work more accessible and applicable to other regulators.
The paper will have a simple structure: an introduction, a straightforward method section, a narrative description with pictures of our journey, and a discussion of our experiences and future plans. We aim to demystify AI and machine learning and make them easier to understand and apply.
Our goal is to introduce applied evidence-based regulatory science, creating evidence that informs regulations and helps sponsors avoid potential harm. Ultimately, the agency's role is to support good research to benefit patients and the community.
Editor’s note:
We thank Andrea for a motivating interview and for sharing with us the latest AI/ML developments within MHRA. It is clear that the integration of AI technologies into regulatory processes is not only enhancing efficiency but also paving the way for innovative solutions in the way we conduct and assess clinical trials. Andrea's insights and experiences highlight the transformative potential of these technologies in the drug development sector. We look forward to seeing the continued advancements and positive impacts of AI/ML initiatives at MHRA. Thank you, Andrea, for your valuable contributions and for leading the way in this exciting field.