PP03-34: Monitor What Matters: Validation of a Key Risk Indicator Selection Method That Predicts Study Outcome

Poster Presenter

Heather Romero

Clinical Scientist
WCG – Analgesic Solutions
United States

Objectives

There are no defined methods for selecting key risk indicators (KRIs) for central statistical monitoring (CSM). The objective of this study was to develop and validate a method for identifying KRIs for CSM that predict the outcome of the clinical trial and therefore merit monitoring.

Method

Developing and validating KRIs consisted of the following steps: (1) Generation of a candidate variables list based on expert input; (2) validation of the variables ability to predict study outcome; (3) cross validation of the variable selection method in a pooled data sample.

Results

Step one of variable selection through an expert panel resulted in the selection of 47 variables, 3 of which were selected for further validation based on feasibility and qualitative rankings. These variables were: daily diary non-compliance, discordance between similar measures, and missed doses of study medication. In step two, we found that diary non-compliance and missed doses statistically significantly predicted the treatment difference between drug and placebo in the first clinical trial. In step three the variable selection method resulted in 6 variables thought to be associated with study outcome. We found that the average number of signals (aberrant values for 6 selected KRIs based on statistical thresholds) was 3.28 (SD=0.66) per subject across the 31 clinical trial sites. The number of aberrant site variables significantly predicted the effect of drug vs. placebo (R2 = 0.34, p < 0.001). Sites with more than 4.5 signals per subject (indicative of poor data quality) had near zero or negative effect sizes.

Conclusion

We developed and generated initial validation results for a method for selecting KRIs for central statistical monitoring that includes qualitative and quantitative methods and demonstrated the validity of this method for predicting the drug vs. placebo differences across clinical research sites. KRI selection does not need to be arbitrary or based solely on judgment: quantitative methods are available to validate variable selection, which, like other elements of clinical trial conduct, should be an evidence-based process. Authors: Heather Romero, Kathryn Evans, Larissa Miropolsky, Igal Seagal, Arturo Morales, Nathaniel Katz