Multimodal coordination of facial action, head rotation, and eye motion during spontaneous smiles

Jeffrey F. Cohn · Lawrence Ian Reed · Jing Xiao · Karen L. Schmidt · Zara Ambadar

60 citations0/0 files storedInternational Conference on Automatic Face and Gesture Recognition

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Emotion and Mood Recognition Face recognition and analysis Face Recognition and Perception

Abstract

Both the configuration of facial features and the timing of facial actions are important to emotion and communication. Previous literature has focused on the former. We developed an automatic facial expression analysis system that quantifies the timing of facial actions as well as head and eye motion during spontaneous facial expression. To assess coherence among these modalities, we recorded and analyzed spontaneous smiles in 62 young women of varied ethnicity ranging in age from 18 to 35 years. Spontaneous smiles occurred following directed facial action tasks, a situation likely to elicit spontaneous smiles of embarrassment. Smiles (AU 12) were manually FACS coded by certified FACS coders. The 3D head motion was recovered using a cylindrical head model. The motion vectors for lip-corner displacement were measured using feature-point tracking. The eye closure and the horizontal and vertical eye motion (from which to infer direction of gaze or visual regard) were measured by the generative model fitting approach. The mean correlation within subjects between lip-corner displacement, head motion, and eye motion ranged from +/0.36 to 0.50, which suggests moderate coherence among these features. Lip-corner displacement and head pitch were negatively correlated, as predicted for smiles of embarrassment. These findings are consistent with recent research in psychology suggesting that facial actions are embedded within coordinated motor structures. They suggest that the direction of correlation among features may discriminate between facial actions with similar morphology but different communicative meaning, inform automatic facial expression recognition, and provide normative data for animating computer avatars.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 60 citations exist upstream.

References

1978Facial Action Coding System 2002Comprehensive database for facial expression analysis 1985Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage 1995Signs of appeasement: Evidence for the distinct displays of embarrassment, amusement, and shame.1979Social and emotional messages of smiling: An ethological approach.1999Measuring facial expressions by computer image analysis 2004THE TIMING OF FACIAL MOTION IN POSED AND SPONTANEOUS SMILES 2003Signal characteristics of spontaneous facial expressions: automatic movement in solitary and social smiles

Cited by

2015Bodily Expression for Automatic Affect Recognition 2010Eyes do not lie 2015Decoupling facial expressions and head motions in complex emotions

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.