Measure what matters: Using an AI chatbot to evaluate the mental wellbeing impact of creative health programmes

Abstract

Rationale Creative health programmes can enhance mental wellbeing, yet their complexity presents challenges for robust evaluation. Traditional outcome measures may fail to capture the nuanced, multidimensional impacts of arts-based interventions. We investigated whether an AI-powered conversational agent (Kenji) could effectively capture wellbeing outcomes in community arts programmes to provide an alternative evaluation approach to conventional survey methods. Method We deployed Kenji, a domain-adapted AI chatbot, alongside its text-to-wellbeing scoring model across four arts programmes in London over nine weeks, engaging 73 participants through structured yet naturalistic chat interactions. Participants' wellbeing was assessed using both an AI-based scoring model and the Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS). We examined engagement patterns, correlations between AI-derived scores and WEMWBS, and the breadth of qualitative outcomes captured. Results Participants demonstrated significantly higher engagement with the chatbot compared to the WEMWBS (86% vs 50% completion rates) and preferred the interaction with the chatbot (89%). The overall AI-derived wellbeing scores showed positive correlation with WEMWBS scores (corr = 0.66, p < 0.05). The AI-based scoring model also captured broader qualitative outcomes, including enhanced sense of purpose (mentioned by 95% of participants), improved mood (95%), increased creativity (93%), and strengthened social connections (86%). Conclusions Our innovative study exemplifies the unexplored potential of AI-driven tools to enable scalable, person-centred evaluation of creative health interventions. This approach has the potential to address key methodological limitations in arts and health research and to offer new avenues for community organisations, funders, and healthcare providers seeking integrated, evidence-based approaches to social prescribing solutions.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 1 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.