A Novel Approach for Mixed‐Methods Research Using Large Language Models: A Report Using Patients’ Perspectives on Barriers to Arthroplasty

Abstract

OBJECTIVE: Mixed-methods research is valuable in health care to gain insights into patient perceptions. However, analyzing textual data from interviews can be time-consuming and require multiple analysts for investigator triangulation. This study aims to explore a novel approach to investigator triangulation in mixed-methods research by employing a large language model (LLM) for analyzing data from patient interviews. METHODS: This study compared the thematic analysis and survey generation performed by human investigators and ChatGPT-4, which uses GPT-4 as its backbone model, using data from an existing study that explored patient perceptions of barriers to arthroplasty. The human- and ChatGPT-4-generated themes and surveys were compared and evaluated based on their representation of salient themes from a predetermined topic guide. RESULTS: ChatGPT-4 generated analogous dominant themes and a comprehensive corresponding survey as the human investigators but in significantly less time. The survey questions generated by ChatGPT-4 were less precise than those developed by human investigators. The mixed-methods flowchart proposes integrating LLMs and human investigators as a supplementary tool for the preliminary thematic analysis of qualitative data and survey generation. CONCLUSION: By utilizing a combination of LLMs and human investigators through investigator triangulation, researchers may be able to conduct more efficient mixed-methods research to better understand patient perspectives. Ethical and qualitative implications of using LLMs should be considered.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 14 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.