Ergotic Sounds: A New Way to Improve Playability, Believability and Presence of Virtual Musical Instruments

Abstract

We explore how an ‘ergotic gesture-sound situation’ is a genuine way to improve the playability, believability and presence of virtual musical instruments. An ergotic gesture-sound situation is a situation where the instrumental gesture and the sound produced are intimately energetically linked. We perform two sets of experiments: the first aims at identifying the respective role of haptic and auditory perceptions in recognition of a virtual object, here a virtual bowed string; the second aims at evaluating the role of haptic and auditory perceptions with respect to action, in the achievement of a difficult musical task. The musical goal consists in maintaining the continuity of sound when changing bow direction. The first experiments are subjective experiments. They show that the feeling of presence of the bowed string is strongly reinforced when sounds and gestural friction are intimately linked, as is the case when the vibrations of the string are returned to the fingers. The second are subjective and objective experiments. They show that achieving the goal is significantly improved by instrumental force feedback. Further, they show that the main property of subjects' performance with the virtual bowed string is fast adaptive dynamic learning due to the physical relation between action plus gesture perception and auditory perception, and that playing and embodiment are notably improved by a strong physical haptic resistance.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 34 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.