Weakly supervised pain localization using multiple instance learning

Karan Sikka · Abhinav Dhall · Marian Stewart Bartlett

62 citations0/0 files storedInternational Conference on Automatic Face and Gesture Recognition

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Emotion and Mood Recognition Face recognition and analysis Human Pose and Action Recognition

Abstract

Automatic pain recognition from videos is a vital clinical application and, owing to its spontaneous nature, poses interesting challenges to automatic facial expression recognition (AFER) research. Previous pain vs no-pain systems have highlighted two major challenges: (1) ground truth is provided for the sequence, but the presence or absence of the target expression for a given frame is unknown, and (2) the time point and the duration of the pain expression event(s) in each video are unknown. To address these issues we propose a novel framework (referred to as MS-MIL) where each sequence is represented as a bag containing multiple segments, and multiple instance learning (MIL) is employed to handle this weakly labeled data in the form of sequence level ground-truth. These segments are generated via multiple clustering of a sequence or running a multi-scale temporal scanning window, and are represented using a state-of-the-art Bag of Words (BoW) representation. This work extends the idea of detecting facial expressions through `concept frames' to `concept segments' and argues through extensive experiments that algorithms like MIL are needed to reap the benefits of such representation. The key advantages of our approach are: (1) joint detection and localization of painful frames using only sequence-level ground-truth, (2) incorporation of temporal dynamics by representing the data not as individual frames but as segments, and (3) extraction of multiple segments, which is well suited to signals with uncertain temporal location and duration in the video. Experiments on UNBC-McMaster Shoulder Pain dataset highlight the effectiveness of our approach by achieving promising results on the problem of pain detection in videos.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 62 citations exist upstream.

References

2010The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression 1994Handbook of pain assessment 2011Painful data: The UNBC-McMaster shoulder pain expression archive database 2006Automatic Recognition of Facial Actions in Spontaneous Expressions 2005Facial Expression Analysis 1996The science of emotion: research and tradition in the psychology of emotions

Cited by

2019Multiple Instance Learning for Emotion Recognition Using Physiological Signals 2016Video Emotion Recognition with Transferred Deep Feature Encodings 2017DeepFaceLIFT: Interpretable Personalized Models for Automatic Estimation of Self-Reported Pain 2021Computer aided pain detection and intensity estimation using compact CNN based fusion network 2015Multi-instance Hidden Markov Model for facial expression recognition

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.