DIPS: A Dyadic Impression Prediction System for Group Interaction Videos
Abstract
We consider the problem of predicting the impression that one subject has of another in a video clip showing a group of interacting people. Our novel Dyadic Impression Prediction System ( DIPS ) contains two major innovations. First, we develop a novel method to align the facial expressions of subjects p i and p j as well as account for the temporal delay that might be involved in p i reacting to p j ’s facial expressions. Second, we propose the concept of a multilayered stochastic network for impression prediction on top of which we build a novel Temporal Delayed Network graph neural network architecture. Our overall DIPS architecture predicts six dependent variables relating to the impression p i has of p j . Our experiments show that DIPS beats eight baselines from the literature, yielding statistically significant improvements of 19.9% to 30.8% in AUC and 12.6% to 47.2% in F1-score. We further conduct ablation studies showing that our novel features contribute to the overall quality of the predictions made by DIPS .
Document Details
- Document Type
- Pub Defense Publication
- Publication Date
- Jan 23, 2023
- Source ID
- 10.1145/3532865
Entities
People
- Chongyang Bai
- Maksim Bolonkin
- V. S. Subrahmanian
- Viney Regunath
Organizations
- Army Research Office
- Dartmouth College
- Northwestern University
- Office of Naval Research