Menu Back to Poster-Presentations-Details

PP01-07: Conversational Interfaces for Data Mining in Pharmacovigilance

Poster Presenter

Sahith Doppalapudi

Sr.Manager, PV Analytics Center of Excellence
Bristol-Myers Squibb Company
United States

Objectives

To a) investigate the feasibility of integrating conversational agents and Natural Language Querying into PV activities like case prioritization and volume, in order to improve visibility of information and reduce time for decision making. and b) Explore the value of democratizing these data

Method

The experiment involved: 1. Identification of key questions for volume and productivity within PV case management 2. Training and validation of the Microsoft® LUIS Language Understanding chatbot on the questions identified above 3. Integration of Chatbot with business intelligence tools

Results

Adverse Event (AE) Case management data from 500,000 cases was included in this experiment. The data was used to do supervised training of LUIS by PV subject matter experts (SMEs). The Chatbot was trained and tested on several conversations comprising a variety of case parameters such as validity, case classification, seriousness, seriousness criteria, case country, case type, workflow step, user data, processing region and processing dates in various combinations to satisfy volumetric and productivity inquiries from the case management functions globally. These parameters were selected based on data on 1500 queries from SMEs over the past 2 years. The experiment also included exploring data through question and answer with the bot in an interactive manner. The bot was able to drill down on data iteratively across all of the parameters tested. Results were consistent and correct to the level of three drill downs for each question based on the SQL output. Below depicts a sample of questions, bot response and SQL results. 1) How many serious cases are currently in the workflow step of Medical Review (Chatbot Result # 44; SQL Query Result# 44) 2) How many cases were received from France on 03Apr2018 (Chatbot Result #19; SQL Query Result #19) 3) How many Death or Life-threatening cases are currently in Assessment workflow step (Chatbot Result # 119; SQL Query Result # 119) 4) How many cases were processed by User X on 22-Feb-2018 (Chatbot Result #34; SQL Query Result # 34) The chat response time for all the questions was between 5-10 seconds for a data set of 0.5M cases Interviews with key SMEs established that there is value in the democratization of volumetric and productivity data within PV. Enabling case processing managers and leadership to have this data on demand measurably reduced the demand on PV Analytics Professionals, while increasing efficiency of case assignment and prioritization

Conclusion

The past several years has seen an emergence of conversational agents in our everyday lives with Alexa, Google Assistant, Siri and others [1]. Beyond day to day use, conversational agents have demonstrated value for health-related purposes [2]. However, the utility of these agents within the field of PV have previously not been demonstrated. Although PV is a highly data driven field, the community is constrained by data mining tools and techniques which rely on specialized individuals to construct queries and reports in order to evaluate data. There is an opportunity to empower the PV professionals in data analysis by making data available on fingertips via conversational agents. As PV moves further towards multidimensional and multimodal data, it is important to simplify and unify interfaces that provide relevant information seamlessly. Conversational interfaces provide a valuable channel in the PV ecosystem for on demand operational data inquiry. Currently within our PV organization, 25%-30% of data requests received relate to metrics on volume and productivity with a turn-around time of 1-2 days; instead, these can be addressed by the bot in seconds. Based on the results observed in this study with the operational and volumetric data requests, chatbots have the potential to be the hub for data requests across the PV organization instead of relying on numerous dashboards, visualizations or reports built by specialized individuals using multiple tools. The results from this study also suggest that further investigation of conversation agents may support more complex PV concepts such as medical evaluation and signaling. References 1.McTear M, Callejas Z, Griol D.The Conversational Interface:Talking to Smart Devices. Springer; 2016 2.Laranjo L, Dunn AG,Tong HL, et al. Conversational agents in healthcare: a systematic review. J Am Med Inform Assoc. ;25(9):1248–1258. doi:10.1093/jamia/ocy072McTear MF.