Articlehybrid2022

Statistical robustness of randomized controlled trials in high-impact journals has improved but was low across medical specialties

Jasper M. Kampman · Oren Turgman · Nicolaas H. Sperna Weiland · Markus Werner Hollmann · Sjoerd Repping · Jeroen Hermanides

14 citations0/0 files storedJournal of Clinical Epidemiology

No readable copy stored yet — this record is metadata only.

No open-access location is known for this record yet.

Literature note Cite in a project Citation map

Meta-analysis and systematic reviews Statistical Methods in Clinical Trials Statistical Methods in Epidemiology

Abstract

OBJECTIVES: To determine whether the statistical fragility of randomized controlled trials (RCTs) in high-impact journals has improved in the last decade and to perform an umbrella review of all published data on the Fragility Index (FI) across medical specialties. STUDY DESIGN AND SETTING: The FI was calculated for all eligible RCTs published from 2014-2021 in the New England Journal of Medicine, The Lancet, the Journal of the American Medical Association, the British Medical Journal, and the Annals of Internal Medicine. Trials reporting dichotomous, statistically significant, superiority results were eligible. All previously published systematic reviews on the FI were included in the umbrella review and analyzed by medical (sub) specialty. RESULTS: Of 2,544 screened RCTs, 643 were eligible for the FI analysis. These had a median sample size of 625 (interquartile range [IQR]: 265-2,056), a median FI of 12 (IQR: 3-28), and a median Fragility Quotient of 0.015 (IQR: 0.004-0.045). This is an improvement compared with the median FI of 8 (IQR: 3-18) of RCTs published a decade earlier in the same five journals (P < 0.001). The umbrella review included 57 publications across 15 different medical specialties, with a total of between 10 and 692 RCTs for each specialty. The median FI ranged between two and four for all disciplines. CONCLUSION: In the last decade, the median statistical robustness of RCTs published in high-impact journals has improved, yet the unchanged lower bound of the interquartile range reveals that statistical significance in 25% of trials is still dependent on three or less events. The umbrella review revealed that statistical fragility is prevalent across all medical specialties. The FI is an easy-to-understand metric that can be used to supplement reported P values and help readers look beyond merely reaching statistical significance.

Paper brief

Loading…

Cite this work

Citation unavailable for this work.

BibTeX RIS CSL-JSON

No public copy is stored for this record, so only the metadata is cited.

Citations in this library

Citation map →

Stored only when both papers are in the corpus — 14 citations exist upstream.

References

2009Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement 2005Why Most Published Research Findings Are False 2019The statistical significance of meta-analyses is frequently fragile: definition of a fragility index for meta-analyses 2020The weakness of fragility index exposed in an analysis of the traumatic brain injury management guidelines: A meta-epidemiological and simulation study

Cited by

2023Randomized Controlled Trials in ICU in the Four Highest-Impact General Medicine Journals 2023An analysis of randomized controlled trials on anal fistula conducted between 2000 and 2020 based on the Fragility Index and Reverse Fragility Index

Explore the map →

Papers that cite the same references — the relational fallback while embeddings are still backfilling.

Imported from OpenAlex (CC0) — nothing here is generated.