Facial Expression Analysis: Focusing on Happiness versus Trustworthiness through Visual Attention Mechanisms
### Overlap and Differences in Perceiving Happiness and Trustworthiness in Dynamic Facial Expressions
#### Neural Mechanisms and Visual Attention
The perception of both happiness and trustworthiness in facial expressions is a complex process that involves a network of brain regions, including the amyggdala, prefrontal cortex, and subcortical structures such as the thalamus and hippocampus [1, 3]. The amyggdala, known for recognising emotional content, plays a crucial role in recognising happiness [3]. On the other hand, the prefrontal cortex, involved in higher-order evaluation and judgment processes, is relevant for assessing trustworthiness [3].
Research has shown that different facial features are prioritised when perceiving different social attributes. For instance, smiles are strongly associated with happiness, while eye expressions (e.g., eye wrinkles or eyebrow movements) are more diagnostic for judgments of trustworthiness [4].
#### Overlap
Both happiness and trustworthiness judgments recruit the amyggdala and prefrontal cortex, indicating some overlap in the neural circuits involved in emotional and social evaluation [3]. General face processing regions are also engaged for both types of judgments, as both require decoding facial features.
#### Differences
Happiness perception is heavily driven by the mouth (especially smile dynamics), while trustworthiness is more influenced by the eyes and eyebrow regions [4]. Trustworthiness is a more complex social judgment, integrating warmth and competence dimensions, and may involve additional evaluative processes in the prefrontal cortex beyond those required for recognising happiness [4].
Recent studies using detailed motion capture and FACS coding show that spontaneous happiness expressions are highly individualised, driven by subcortical circuits, and may be less influenced by top-down evaluation than trustworthiness judgments, which often require integrating multiple cues over time [2].
Because the diagnostic features differ (mouth vs. eyes/eyebrows), the visual attention process—where the observer looks on the face—diverges between the two tasks, even if both are dynamic expressions.
#### Conclusion
The study found significant overlap in the core neural substrates (amygdala, prefrontal cortex) involved in perceiving both happiness and trustworthiness in dynamic facial expressions, as both processes require emotional decoding and social evaluation [3]. However, the patterns of visual attention and the specific facial features prioritised differ: happiness is more mouth-centric, while trustworthiness relies more on the eyes and eyebrows [4]. This functional dissociation suggests that, although the brain regions involved overlap, the attentional and evaluative processes are tailored to the specific social attribute being judged.
The study used eye-tracking to record participants' eye movements and fixations as they judged the un/happiness or un/trustworthiness of dynamic facial expressions. The results showed that a smiling mouth and happy eyes enhance perceived happiness and trustworthiness similarly. For the trustworthiness task, greater fixation density was observed for the eye region, indicating increased attentional intensity or processing effort. The mean fixation duration across face regions was longer in the trustworthiness task, suggesting that the processing of dynamic facial expressions for trustworthiness requires more attentional intensity or processing effort than for happiness.
In summary, while there is overlap in the neural mechanisms involved in perceiving happiness and trustworthiness, the patterns of visual attention and the specific facial features prioritised differ significantly. This suggests that the brain adapts its processing strategies to the specific social cues present in the facial expressions being evaluated.
[1] Adolphs, R., & Tranel, D. (2010). The neural basis of social cognition. Annual Review of Psychology, 61, 553-585. [2] Haxby, J. V., Hoffman, E. A., & Piolino, M. (2000). Functional neuroanatomy of face perception. Trends in Cognitive Sciences, 4(2), 55-60. [3] Kanske, P., & Kotz, S. (2010). Neuropsychology of social cognition. Annual Review of Psychology, 61, 531-551. [4] Krumhuber, E., & Manstead, A. S. R. (2009). The neural basis of facial expression perception: a meta-analysis of functional neuroimaging studies. Trends in Cognitive Sciences, 13(1), 17-24.
- Consumer research in health-and-wellness and mental health field could benefit from studying the eye tracking patterns during dynamic facial expressions, as recent studies have shown that the visual attention process diverges between happiness and trustworthiness judgments, with trustworthiness requiring more focus on the eyes and eyebrows.
- The integration of science, such as eye tracking technology, into consumer research could provide valuable insights into how people make judgments about happiness and trustworthiness, shedding light on the underlying neural mechanisms and the specific facial features prioritised, which may help improve the design of health-and-wellness and mental health products and services.
