P206: Evaluation of Different Search Strategies to Identify Cases of Anaphylaxis in the FDA Adverse Event Reporting System
Poster Presenter
Anran Liu
ORISE Postdoctoral Fellow
FDA United States
Objectives
To evaluate the performance characteristics of various search strategies utilizing different Medical Dictionary for Regulatory Activities (MedDRA) terms to retrieve cases of anaphylaxis with drug and therapeutic biologic products in the FDA Adverse Event Reporting System (FAERS).
Method
We analyzed completed FDA postmarket reviews of anaphylaxis from 2012-2024 containing FAERS searches using different levels of MedDRA terms (Preferred Term [PT], Standardized MedDRA Query [SMQ]). Performance metrics (e.g., positive predictive value (PPV), sensitivity, specificity) were calculated.
Results
Our analysis included 36 reviews of anaphylaxis with different products across 16 therapeutic areas. All reviews used the Anaphylactic reaction narrow SMQ (narrow SMQ), and 14/36 reviews also used the Algorithmic Anaphylactic reaction SMQ (algorithmic SMQ) in the searches. The narrow SMQ contains 12 PTs indicative of anaphylaxis including the PT Anaphylactic reaction. The algorithmic SMQ contains all narrow SMQ PTs and 87 PTs representing signs and symptoms of anaphylaxis organized into 3 categories; 1 PT from 2 of 3 categories must be present to be retrieved.
Overall, the searches retrieved 913 reports (1-222 per review), of which 231 (1-64 per review) were included in the final case series (i.e., event met anaphylaxis case definition and deemed plausibly related to the product of interest by FDA reviewers). On average, narrow SMQ searches retrieved 11 more reports, of which 2 were included in the final case series compared to the PT Anaphylactic reaction; the algorithmic SMQ retrieved 13 more reports, of which 5 were included in the final case series compared to narrow SMQ.
For the cohort of 36 reviews, the PPVs were: PT Anaphylactic reaction 30.9% (95% confidence interval [CI]: 26%, 36.2%), narrow SMQ 22.2% (95% CI: 19.1%, 25.3%).
For the cohort of 14 reviews that included the algorithmic SMQ, we used a weighted approach to accommodate the heterogeneity observed across different therapeutic areas. The metrics for each therapeutic area were computed separately and then combined using the number of reports retrieved by algorithmic SMQ as weights. In this subset, the PPVs were: PT Anaphylactic reaction 44.5% (95% CI: 28.1%, 59.5%), narrow SMQ 46.4% (95% CI: 34.5%, 57.6%), and algorithmic SMQ 40.9% (95% CI: 35%, 47.1%). For the PT Anaphylactic reaction, the sensitivity was 33.5% (95% CI: 23%, 46.5%) and specificity was 81.1% (95% CI: 72.2%, 88.8%). For the narrow SMQ, the sensitivity was 46.8% (95% CI: 36.1%, 60%) and specificity was 69.3% (95% CI: 59.9%, 78.1%).
Conclusion
FDA safety reviewers often search the FAERS database with the broadest MedDRA search terms available, without knowing whether the broad strategy actually retrieves more true cases or simply captures more noise compared to narrower strategies. The results of this analysis highlight variability in the performance of different FAERS search strategies using various levels of MedDRA terms to retrieve cases of anaphylaxis.
Among the different evaluated MedDRA search strategies, PT-based searches demonstrated a higher specificity but lower sensitivity. Narrow SMQ searches achieved a balance between sensitivity and specificity, whereas algorithmic SMQ searches captured the most cases of anaphylaxis but with reduced PPV and a higher burden of irrelevant reports.
These findings provide actionable insights for tailoring FAERS search strategies for anaphylaxis to the specific objective of the pharmacovigilance evaluation. PT-based and narrow SMQ searches for anaphylaxis may be better suited for targeted case identification where specificity and manageable workload are critical, such as when trying to confirm the presence of a safety signal. The algorithmic SMQ does retrieve more cases, but at the expense of many more false positives and may be suited for when all information is needed for decision making purposes.
Further research is needed to validate these findings, and we plan on prospectively evaluating the various search strategies on a larger dataset of FAERS reports.
Disclaimer: This abstract reflects the views of the authors and should not be construed to represent FDA’s views or policies.
