The influence of individual traits and stress on expressions related to understanding and confusion

Introduction

As a fundamental component of communication, facial expressions can give information about the mental state of the listener  and can indicate important feedback on an individual’s understanding of an explanation. For example, facial expressions such as open eyes and raised eyebrows have been shown to be a sign of understanding,  while confusion is indicated by narrowed eyes, furrowed eyebrows, a wrinkled forehead, and tense lips.  Stress can influence expressions related to understanding and confusion, such as smiling and frowning.  These stress-induced changes in nonverbal signals can make it difficult to accurately interpret facial expressions in high-pressure scenarios.  The way people are influenced by stress can be modulated by differing personality traits, including resilience,  anxiety,  and depression.  Additionally, variation in the personality dimensions agreeableness, extraversion and openness can lead to different behaviors in facial expressions.  Findings exploring gender differences during tutoring suggest that women and men express mental effort or uncertainty differently. 

To investigate the dynamics of explanations and social signs of understanding, Paletschek et al.  developed an explanation paradigm, which provides a structured approach to analyze how stress influences facial expressions related to understanding in explanatory scenarios. Based on data collected using this paradigm, we contribute an analysis of facial expressions during stressful explanatory situations, examining how stress and individual traits affect facial expressions.

We address the following research questions.

• R1: To what extent do individual differences influence facial expressions related to understanding and confusion of the explainee in an explanatory situation?

• R2: How does acute psychological stress influence facial expressions related to understanding and confusion during explanatory interactions, considering the significant individual factors identified in R1?

Methods

We utilize data from a study using the paradigm of Paletschek et al.  In this paradigm, the rules of board games are explained to participants once in a stressful and once in a neutral condition. Stress is induced by combining time pressure and power imbalance, where a strict and formal researcher conducts the task. In addition, social evaluation is introduced by informing the participants that their performance will be recorded and compared with others. In the neutral condition, the induction of stress is mitigated by removing time pressure, having the explainer pose as a peer to minimize power imbalance, and removing performance comparisons to reduce social evaluative threat. Before the explanations, participants complete a series of questionnaires. In order to investigate personality dimensions, the short form of the Big-Five-Inventory  was used. To assess anxiety and depression, the Hopkins-Symptom-Checklist-11,  the Health-Questionnaire-9,  and the Liebowitz Social Anxiety Scale  were employed. In addition, the study included the Resilience Scale-13  and the ADHD Self-Report Scale. 

Participants were video recorded during the study procedure to capture their facial expressions. The recordings were processed using OpenFace software  to extract facial action units (AUs), with a focus on those related to understanding and confusion.  Participants utilized RankTrace, a continuous and unbounded affect annotation tool,  to recall and indicate their perceived understanding throughout the study.

We will compare the frequency of AUs during annotated understanding or confusion to their overall occurrence. To identify how individual traits and stress influence facial expressions, we will analyze the influence of individual character differences in both neutral and stress conditions.

Initial Results and Outlook

Initial results of 47 participants  show a significant increase in self-reported stress during the stress condition compared to the neutral condition (p < 0.03), with small-to-medium effect sizes (r = 0.225 - 0.452). This confirms the paradigm’s efficacy in modulating intra-individual stress responses. Furthermore, feedback from the participants indicated that the explanation scenario effectively induced both understanding and confusion. After further data cleaning and additional data collection, a dataset of 49 volunteers (24 men, 22 women, 1 diverse and 2 unknown) aged between 18 and 38 years (Median = 24) serves as a foundation for our analysis. We will present results on how stress influences facial expressions and how individual differences moderate these effects. Our findings offer insights into inter- and intra-individual differences in expressions related to understanding and confusion. These insights could inform future efforts to personalize explanations, taking into account individual traits and situational contexts.