Reporting of Randomized Controlled Trials With Statistically Nonsignificant Primary Outcomes Published in High-impact Surgical Journals

Abstract

OBJECTIVE: To determine the nature and frequency of distorted presentation or "spin" (ie, specific reporting strategies which highlight that the experimental treatment is beneficial, despite a statistically nonsignificant difference for the primary outcome, or distract the reader from statistically nonsignificant results) in published reports of randomized controlled trials (RCTs) with statistically nonsignificant results for primary outcomes in surgical journals. BACKGROUND: Multiple reports have suggested that interpretation of RCT results in medical journals can be distorted by authors of published reports. METHODS: Using a defined search strategy, RCTs with clearly nonsignificant results for the primary outcome (P > 0.05) form 10 high-impact factor surgical journals (Annals of Surgery, Journal of Neurology, Neurosurgery and Psychiatry, Journal of Heart and Lung Transplantation, American Journal of Transplantation, British Journal of Surgery, Journal of Bone and Joint Surgery, Journal of the American College of Surgeons, Endoscopy, Archives of Surgery, and Liver transplantation), published between July 2013 to July 2015, were identified. Two reviewers independently appraised each selected article using a validated, standardized data abstraction form. RESULTS: In all, 110 eligible RCTs with nonsignificant primary outcomes were appraised. The title was reported with spin in 8 (7%) articles. Forty-four (40%) included abstracts and 39 (35%) main texts were classified as having spin in at least 1 section. The level of spin was high in 16 (14%) abstract and 19 (19%) main-text "Conclusions" sections. Twenty-five articles (23%) recommended the intervention of interest despite a nonsignificant primary outcome. There was no relationship between trial funding source, use of statistician and article section, and the presence of spin. CONCLUSIONS: In RCTs with statistically nonsignificant primary outcomes published in surgical journals, the reporting and interpretation of findings was frequently inconsistent with the results.