P301: Transforming Medical Affairs: The Role of GenAI Vision Models in Streamlined Reference Verification for Promotional Review
Poster Presenter
Sandeep Kumar Singh
Associate Director, Global Medical Information
Genmab United States
Objectives
We developed an automated reference-checking (ARC) tool using GenAI and Vision Models to streamline promotional review. The primary objectives were to reduce medical reviewers' time on fact-checking, ensure content accuracy, and assess data extraction capabilities of the ARC tool.
Method
User acceptance testing (UAT) assessed reference verification accuracy, and operational efficiency gains. The objective was to achieve an 80% accuracy rate in reference verification, and a 25% reduction in time of manual review for a single material.
Co-Author: Jane Oreper
Results
This tool leverages multimodal large language models (LLMs) to extract and validate complex scientific content, such as figures, charts, graphs and contextual data.
Preliminary testing during the proof-of-concept phase of development for the ARC tool met UAT objectives as a trend towards a reduction in time spent verifying references and confirming medical accuracy was observed. Thorough assessment of the ARC tool and its impact on process efficiencies is underway via a time trial analysis.
Regarding operational efficiencies, the time to review references in a single material trended down and was initially reduced by 25% using the ARC tool during the preliminary testing in the proof-of-concept stage. The tool achieved an accuracy rate above 80% in identifying correct medical references and flagging inconsistencies. Additionally, preliminary testing of the ARC tool during the UAT demonstrated data extraction ratings above 90%.
Upon completion of the time trial analysis, results will be presented. These results will aim to validate the initial findings.
Conclusion
The development of this AI-powered tool represents a pivotal advancement in Medical Affairs, particularly in enhancing the efficiency of the promotional material review process. By reducing the time and effort required for reference verification, the tool may allow medical reviewers to focus on other strategic activities.
The ARC tool’s built-in audit trail supports quality by providing traceable documentation of reference checks, enabling transparency, traceability, and faster reconciliation of medical claims in promotional materials during document reviews. By generating an audit trail of reference validation, the ARC tool reinforces the quality and consistency of promotional materials. The tool can also aid in the streamlining of the preparation of submissions to the Food and Drug Administration’s (FDA) Office of Prescription Drug Promotion (OPDP) and potentially reduce the risk of delays or rework due to citation-related discrepancies.
This initiative aligns with the pharmaceutical industry-wide trend of leveraging AI-driven solutions to optimize workflows in Medical Affairs. Future directions for the ARC tool include additional data integrations and expanding its capabilities to handle multilingual content and complex reference hierarchies. Additionally, as this tool evolves, based on refinement of LLM models, a 40% decrease in time spent on manual review and a 90% accuracy rate of references for a single material may be possible.
The development of this AI-powered tool represents a significant step forward in innovating and modernizing Medical Affairs workflows. However, while it streamlines workflows, it does not replace the critical role of medical reviewers, whose expertise remains essential in ensuring accuracy and scientific integrity.
