P239: Leveraging AI Agents for Clinical Development Knowledge Management and Utilization
Poster Presenter
Takayuki Yasuda
Statistical Programmer
Chugai Pharmaceutical Co., Ltd. Japan
Objectives
To effectively leverage knowledge accumulated across internal clinical development functions, we developed a chatbot-style AI agent and evaluated its performance and utility.
Method
Based on requirements aligned with expected outputs, we used Microsoft 365 Copilot to build a ServiceNow knowledge search agent and evaluated search accuracy and the quality and usefulness of generated answers through interviews with test users.
Results
As part of the technical performance evaluation, we assessed the data source formats referenced by the AI agent and compared different agent development approaches. The results showed that converting individual knowledge items into Word documents and providing them to the agent achieved the highest accuracy. In addition, a comparison between agents built using Microsoft 365 Copilot Agent Builder and Copilot Studio demonstrated greater operational stability with Agent Builder; therefore, this approach was adopted.
Subsequently, interviews were conducted with knowledge managers responsible for various processes within the clinical function to evaluate the developed agent. As a result, all respondents indicated that the agent was useful.
Conclusion
In this initiative, we developed a knowledge search AI agent to promote effective utilization of knowledge accumulated within the clinical function and evaluated its usefulness. Based on user evaluations, the agent was judged to be useful and has already been deployed for internal use. We plan to continue agile improvements to further enhance its accuracy. In addition, we aim to leverage the agent to extract cross-knowledge insights and explore more effective strategies for knowledge management.
