Sochiatrist: Automatically Predicting Emotion from Social Messaging Data
Abstract
The social lives of people have a profound influence, both positive and negative, on their emotional state and mental health, but this combination of social, emotional, and mental health has been difficult to credibly capture. Specifically, social data is not possible to self-report due to the volume, and even if people estimate the amount of time spent socializing, the content of the social messages is lost. The PIÕs research group has developed procedures to easily extract social messages (from platforms such as SMS texts, Google Voice, Facebook Messenger, Twitter direct messages, Kik, Snapchat, Instagram) from Android and iOS mobile devices. The extracted data allows the researchers to precisely track what was said, when, and with whom. The extraction is done on historical data and does not require any setup prior to the messages being sent, then the data is de-identified, merged, and encrypted into common format. Social messaging features can then be used to infer emotional states reported in the moment, using a computational approach that does not require any message to be read by a human. The software to be developed computes features based on message frequencies, sentiment, response timings, and network features. Emotion data collected as a basis for ground truth comprise both self-reported data from ecological momentary assessment and biomarkers of heart rate and galvanic skin response that serve as proxies for arousal and emotional regulation. Inference models comparing multiple supervised learning techniques will be assessed for effectiveness. The inference models are evaluated across two populations who are particularly vulnerable to mental health issues: college students at the university where the PI is located, and Veterans recruited from the Providence VA Hospital where the PI has an existing collaborative relationship. These evaluations will reveal how accurately can emotion, measured either through self-report or through biomarkers, be inferred purely from social messaging data, what aspects of social messaging affect emotion, and in what way do different social features such as frequency, content, timing, and recipients of messages influence affect. Additionally, a secondary analysis will be conducted to identify predictive patterns in emotion, using the sequential order of social and emotion events. This work will leverage social messaging data already collected in a separate NIH-funded study focused on populations with suicidal risk and trauma. These are adolescents admitted to the hospital, for whom the PI and collaborators have already collected social messaging data, and either self-reported emotion data or emotional responses from biomarker data. By using this dataset as a comparison, the proposed work can identify whether inferences are better or worse for different populations, particularly those who have been admitted for mental health concerns. Finally, the PIÕs research group will release the software that performs automated extraction of social messaging data and infers emotion. This allows social scientists to build on the proposed research or replicate the studies, or even use for other studies that incorporate social messaging data. Altogether, the research will enable a new software-based approach for determining a personÕs emotion based purely on their social messaging data that can be extracted and analyzed automatically. The techniques are privacy-preserving as the extraction and inference is done by software instead of a human reviewer.
Document Details
- Document Type
- DoD Grant Award
- Publication Date
- Jul 24, 2019
- Source ID
- W911NF1910368
Entities
People
- Jeff Huang
Organizations
- Army Contracting Command
- Brown University
- United States Army