Gesture–speech physics: The biomechanical basis for the emergence of gesture–speech synchrony.

Abstract

Hand gestures and speech move in a common rhythm, as exemplified by the synchrony between prosodic contrasts in gesture movement (e.g., peak velocity; maximum effort) and speech (e.g., peaks in fundamental frequency; F0). This joined rhythmic activity is hypothesized to have a variable set of functions, ranging from self-serving cognitive benefits for the gesturer, to communicational advantages that support listeners’ understanding. However, gesture-speech synchrony has been invariably understood as a “neural-cognitive” achievement; i.e., gesture and speech are coupled through neural-cognitive mediation. Yet, it is possible that gesture-speech synchrony emerges out of resonating forces that travel through a common physical medium – the body. In the current pre-registered study, we show that movements with relatively high physical impact affect phonation in a way that accommodates gesture-speech synchrony. Beating with one arm or two arms at a rhythmic pace led to acoustic peaks in the fundamental frequency (F0) and the amplitude envelope of phonation that were entrained with the rhythm of movement. Such effects were not found for upper limb movements with lower physical impetus (wrist movements), nor when participants were phonating without movement. We further provide evidence that postural stability is contributing to the effect of movement on phonation, as entrainment of movement and phonation was more pronounced when participants were standing as opposed to sitting. The current findings suggest that gesture-speech synchrony emerges from biomechanical constraints, potentially obviating the need for a cognitive predictive mechanism that ties gesture and speech in synchrony.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 69 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.