Reviewgreen2021

Concerns and recommendations for using Amazon MTurk for eating disorder research

Carolyn Blair Burnette · Jessica L Luzier · Brooke L. Bennett · Chantel M. Weisenmuller · Patrick L. Kerr · Shelby J. Martin +1 more

105 citations0/0 files storedInternational Journal of Eating Disorders

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Eating Disorders and Behaviors Survey Methodology and Nonresponse Mobile Crowdsensing and Crowdsourcing

Abstract

OBJECTIVE: Our original aim was to validate and norm common eating disorder (ED) symptom measures in a large, representative community sample of transgender adults in the United States. We recruited via Amazon Mechanical Turk (MTurk), a popular online recruitment and data collection platform both within and outside of the ED field. We present an overview of our experience using MTurk. METHOD: Recruitment began in Spring 2020; our original target N was 2,250 transgender adults stratified evenly across the United States. Measures included a demographics questionnaire, the Eating Disorder Examination-Questionnaire, and the Eating Attitudes Test-26. Consistent with current literature recommendations, we implemented a comprehensive set of attention and validity measures to reduce and identify bot responding, data farming, and participant misrepresentation. RESULTS: Recommended validity and attention checks failed to identify the majority of likely invalid responses. Our collection of two similar ED measures, thorough weight history assessment, and gender identity experiences allowed us to examine response concordance and identify impossible and improbable responses, which revealed glaring discrepancies and invalid data. Furthermore, qualitative data (e.g., emails received from MTurk workers) raised concerns about economic conditions facing MTurk workers that could compel misrepresentation. DISCUSSION: Our results strongly suggest most of our data were invalid, and call into question results of recently published MTurk studies. We assert that caution and rigor must be applied when using MTurk as a recruitment tool for ED research, and offer several suggestions for ED researchers to mitigate and identify invalid data.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 105 citations exist upstream.

References

1982The Eating Attitudes Test: psychometric features and clinical correlates 1979The Eating Attitudes Test: an index of the symptoms of anorexia nervosa 2018Prevalence of eating disorders over the 2000–2018 period: a systematic literature review 2008Cognitive Behavior Therapy and Eating Disorders 2003Validity of the Eating Disorder Examination Questionnaire (EDE-Q) in screening for eating disorders in community samples 2020A Literature Review of the Economics of Covid-19 2000Development and validation of the Eating Disorder Diagnostic Scale: A brief self-report measure of anorexia, bulimia, and binge-eating disorder.2011Eating Disorder Symptoms Among College Students: Prevalence, Persistence, Correlates, and Treatment-Seeking

Cited by

2022COVID-19 vaccine behaviors and intentions among a national sample of United States adults ages 18–45 2023Creation of the entrepreneurial personality scale: Removing conceptual and empirical barriers from the study of personality and entrepreneurship

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.