Measuring meaningful human control in human–AI teaming: effects of team design in AI-assisted pandemic triage

Abstract

AI will increasingly be used to collaborate with humans on ethical tasks. This study contributes to the need for practical methods to assess whether a human–AI system is under Meaningful Human Control (MHC). We propose three measurable components of MHC: subjective-, normative-, and moral control. To empirically evaluate the qualities of the MHC measuring approach, we developed an experiment in which a human–AI team performs triage during a pandemic outbreak. Participants performed the role of physician. Moral pressure was induced by a rapid influx of patients and limited resources. Three designs of human–AI collaboration were tested as a repeated within-subjects factor: (A) agent provides information and decision advice; (B) human assigns some patients to agent for triage; (C) human instructs agent to autonomously conduct triage on all patients. The measures were sufficiently sensitive to detect effects of the three human–AI team design on MHC: When advised by an agent (A), or when issuing tasks to an agent (B), participants felt more engaged, were able to exercise more control, and were more compliant with ethical guidelines. When the agent performed triage autonomously (C), participants reported a lower moral load, and judged the collaboration as less believable. Subjective, normative, and moral control can serve as a practical approach for assessing MHC.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 10 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.