Spelling correction with large language models to reduce measurement error in open-ended survey responses

Maxwell B. Allamong · Jongwoo Jeong · Paul M. Kellstedt

3 citations0/0 files storedResearch & Politics

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Computational and Text Analysis Methods Sentiment Analysis and Opinion Mining Survey Methodology and Nonresponse

Abstract

Open-ended survey questions have a long history in public opinion research and are seeing a renewed interest as computing power and tools of text analysis proliferate. A major challenge in performing text analyses on open-ended responses is that the documents—especially if transcribed or collected through web surveys—may contain measurement error in the form of misspellings which are not easily corrected in a reliable and systematic manner. This paper provides evidence that large language models (LLMs), specifically OpenAI’s GPT-4o, offer a flexible, dependable, and low-cost solution to correcting misspellings in open-ended responses. We demonstrate the efficacy of this approach with open-ended responses about the Democratic and Republican parties from the 1996–2020 American National Election Studies, where GPT is shown to correct 85%–90% of misspellings identified by human coders in a sample of responses. Following spelling correction on ∼50,000 responses, we document several consequential changes to the data. First, we show that spelling correction reduces the number of unique and single-use tokens while increasing the number of words matched to a sentiment dictionary. Then, to highlight the potential benefits and limitations of spelling correction we show improved out-of-sample prediction accuracy from a text-based machine learning classifier. Finally, we show a significantly larger degree of emotionality is captured in the spelling-corrected texts, though the size of this measure’s relationship with a known correlate in political interest remains relatively unchanged. Our findings point to LLMs as an effective tool for reducing measurement error by correcting misspellings in open-ended survey responses.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 3 citations exist upstream.

References

1992The Nature and Origins of Mass Opinion 1964Ideology and Discontent 2014Structural Topic Models for Open‐Ended Survey Responses 2023ChatGPT outperforms crowd workers for text-annotation tasks 2017Text Preprocessing For Unsupervised Learning: Why It Matters, When It Misleads, And What To Do About It 2023Text as Data: A New Framework for Machine Learning and the Social Sciences 2024Large language models as a substitute for human experts in annotating political text 2022Do AIs know what the most important issue is? Using language models to code open-text social survey responses at scale

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.