W-10: How Evaluating Human Emotions Could Provide Valuable Evidence to Support Clinical Trial Endpoints
Poster Presenter
Rinah Yamamoto
Principal Scientist
Clinical ink United States
Objectives
This poster will review a pilot project to evaluate the available types of emotional data that can be passively collected during clinical trials without any additional effort required by the subject, and investigate how emotional intelligence technology can supplement traditional endpoint data.
ORAL PRESENTATION: 1:20PM
Method
The 11-person study used the electronic Columbia Suicide Severity Scale (eC-SSRS) with an ERT/Affectiva emotional intelligence app running in the background to capture emotional and facial expression metrics and determine if the data could provide a more comprehensive understanding of the patient.
Results
Participant experience: Participants were instructed to complete the “lifetime” version of the questionnaire, while responding with the actual responses they experienced within the last 24 hours. Participants were overwhelmingly able to answer the eC-SSRS questions honestly i.e. they were not distracted by the camera reading their facial muscle movements. 100% of the respondents indicated that they answered honestly - 73% of respondents said it was “very easy” to complete the items honestly, 9% responded it was “not difficult at all”, and 18% responded that it was “slightly difficult” to complete the items honestly.
Participant engagement: A measure of each subjects’ engagement will be provided, indicative of the intensity of subject’s facial expressiveness via facial muscle activation. Results for the first 200 data points in this pilot study demonstrated that subjects had an average of 36.1(± 30.0) out of 200 data points above the threshold of 30. There was considerable variability among the subjects with regard to the quantity of engagement data collected. The average engagement intensity among the subjects was 72.57 (± 22.6).
Participant attention: An important indicator of feasibility is whether subjects were paying attention during the assessment. We found that on average subjects were attentive for 171 (± 24.3) out of the first 200 data points, with average intensity of attention at 93.7 (± 8.1).
Participant emotions: Within the app, while the questionnaire is being completed emotions are predicted using computer vision algorithms to identify key landmarks on the face and deep learning algorithms to analyze pixels in those regions to classify facial expressions. Combinations of these facial expressions are then mapped to emotions. Results were plotted on a valance scale to indicate the intensity of 4 key emotions – engagement, fear, joy and sadness.
Conclusion
Overall conclusion: Acquiring information on emotions, engagement and attention while a subject completes a self-report assessment such as the eC-SSRS is feasible and does not sway the subject’s answers.
Technical feasibility: The emotional intelligence software library was easy to use, however the library requires significant CPU power for image processing. Running this software on handheld devices in the future would require a medium to high powered device.
Impact on clinical research: Acquiring information on affect and attention while a subject completes a self-report assessment such as the eC-SSRS is feasible. Determining the meaning of the results will depend on the nature and goals of the specific assessment the software is used in conjunction with. In the case of the eC-SSRS, for example, it is important to consider the results in terms of individual responses. If a subject’s emotional responses were to be discordant with their report of suicidality, the data could be used to as rationale for further face to face interviewing of the subject. Under different circumstances and with a different assessment, group results could be very interesting. One example for this technology might be in the study of psychiatric disorders such as attention deficit hyperactivity disorder or borderline personality where emotional dysregulation is a primary component. Affectiva in conjunction with psychophysiological stimuli designed to induce affect, such as the International Affective Picture System (IAPS), might provide important insight into the abilities of pharmaceutical drugs to impact mood regulation.
Next Steps: The next step will be external validation and additional user acceptance testing. This would also need to include a controlled trial in order to ensure accuracy. Proof of usability would need to include the ability to deploy and acquire useful information in additional therapeutic areas and other eClinical assessments.
