Reporting of sample‐size calculations for randomized trials in dermatology: comparison of publications with registries

Abstract

Dear Editor, Sample‐size calculations play an important role in the design and interpretation of clinical trials, but they have historically been poorly reported in publications.1,2 We systematically reviewed the reporting of sample‐size calculations in the primary reports of randomized, parallel‐group trials published in six dermatology journals between 1 January 2013 and 31 December 2015. We included the top five general dermatology journals and one dermatological surgery journal with the highest impact factors (Table 1). Brief reports or letters were excluded. Two reviewers independently recorded the study characteristics and components of sample‐size calculations defined by the CONSORT 2010 and SPIRIT 2013 reporting guidelines.3,4 We also extracted the target sample size listed in any associated prospective trial registry records (as identified by the article itself or by searching the World Health Organization trial registry search portal and Google) to evaluate discrepancies between the publication and registry. Reporting of sample‐size determinations for randomized trials published in top dermatology journals during 2013–2015 Data are given as n (%) or n/N (%). BJD, British Journal of Dermatology; JAAD, Journal of the American Academy of Dermatology; JID, Journal of Investigative Dermatology. aAmong trials reporting a formal calculation. bFor trials reporting sample‐size calculations using binary outcome measures. cFor trials reporting sample‐size calculations using continuous outcome measures. dData on comparison with registry available on request. Reporting of sample‐size determinations for randomized trials published in top dermatology journals during 2013–2015 Data are given as n (%) or n/N (%). BJD, British Journal of Dermatology; JAAD, Journal of the American Academy of Dermatology; JID, Journal of Investigative Dermatology. aAmong trials reporting a formal calculation. bFor trials reporting sample‐size calculations using binary outcome measures. cFor trials reporting sample‐size calculations using continuous outcome measures. dData on comparison with registry available on request. Among 147 trials, 79 (54%) reported that a formal sample‐size calculation had been performed; only 27 of these 79 (34%) reported all components necessary to replicate the calculation (Table 1). Four of 147 trials (3%) reported a nonstatistical justification for the sample size. The remainder (64 of 147, 44%) provided no explanation for the sample size. In a multivariable logistic regression model, prospectively registered trials were more likely than unregistered trials to report any sample‐size justification (adjusted odds ratio 5·32, 95% confidence interval 1·85–15·3). Trials published in Dermatologic Surgery were significantly less likely to report a justification than those published in the other journals (adjusted odds ratio 0·12, 95% confidence interval 0·03–0·50). No significant association was observed for industry sponsorship (vs. nonindustry or unreported), intervention type (drug/biologic vs. other), single vs. multicentre status, or achieved sample size. Among 42 registered trials reporting a sample‐size calculation in their publication, half (45%, n = 19) reported a different target sample size in the registry compared with the publication (Table 1). The publications for five (12%) of the 42 trials reported a target sample size that was ≥ 10% larger than that in the registry. One‐third (33%, n = 14) reported a target sample size that was smaller than that in the registry, with 12 of these trials differing by ≥ 10%. Of these 14 trials, 11 recruited fewer participants than the registered target sample size. None of the trial publications explained the discrepancies. Poor reporting of statistical and nonstatistical sample‐size determinations was common in our cohort of trials published in top dermatology journals. Incomplete or nonreporting impairs critical appraisal and transparency by making it difficult to assess whether a trial was appropriately powered at the design stage. Complete reporting of a priori sample‐size calculations in publications provides valuable insight into how the trial was planned, as well as the parameters upon which conclusions of superiority, noninferiority or equivalence of interventions are based. We found that the completeness of reporting differed significantly by journal and trial registration status. Endorsement of reporting guidelines by journals has been associated with improved reporting.5,Dermatologic Surgery is the only journal in our cohort that does not have an editorial policy requiring authors to adhere to the CONSORT Statement,3 which may explain why sample‐size justifications were reported less often for its trials compared with the other journals. To improve transparency and reduce selective reporting, four of the six journals in our cohort (all except Dermatologic Surgery and Acta Dermato‐Venereologica) also require that trials be publicly registered prior to participant enrolment to be considered for publication.6 We found that half of the prospectively registered trials in our cohort published a different target sample size than in the registry. Without explanations provided for these discrepancies, readers cannot evaluate the potential for biased practices such as changing the primary outcome variable, excluding randomized patients, or stopping the trial early to emphasize favourable results.7 Trials that do not achieve their target sample size may also report a revised calculation with a smaller‐size estimate in order to give the impression that the target sample size had been achieved.8 Accordingly, in our sample, 11 of 14 trials that reported a smaller target sample size in the publication had also failed to achieve their originally registered target. Our findings support the need to improve reporting of sample‐size determinations in published articles through adherence to the CONSORT guidelines,4 prospective registration of all clinical trials6 and public availability of trial protocols that adhere to the SPIRIT guidance.3 Complete reporting will help ensure that trial results can be properly evaluated and applied to patient care. E‐mail: tessaweinberg1@gmail.com Funding sources: none. Conflicts of interest: none to declare.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 6 citations exist upstream.