T-03: Development of an AI Approach for Identifying Adverse Events

Poster Presenter

Danielle Abatemarco

Associate Director, PV Strategy, Planning and Execution
BeiGene USA, Inc.
United States

Objectives

The objective of this study was to determine if a novel approach utilizing deep learning could accurately identify adverse events in a methodology requiring no feature engineering and that is a scalable, reliable approach.

Method

A neural network was trained from data received between 2015-2016. The neural network was compared to a dictionary-based annotator’s performance regarding precision, sensitivity, and F1 score for identifying the adverse events in an annotated ground truth of spontaneous or solicit cases.

Results

Our neural network yielded a precision, recall, and F1 score of 76.4, 74.9, and 75.6, respectively, for spontaneous report cases and 76.1 recall for solicited cases. These results represent a 49.8 point increase in F1 performance over the dictionary-based method.

Conclusion

Our methodology demonstrates that a neural network approach can achieve accurate adverse event identification in diverse content sources. Further work is required to measure the benefits of a deployed instance of the system and assess its ability to incorporate organization-specific considerations. Neural networks provide a potentially scalable state-of-the-art solution for accurately identifying adverse events in various source documents and with additional data can be further trained to increase performance and reflect organization-specific adverse event detection rules.