From Sample Size to Effect-Size: Small Study Effect Investigation (SSEi)

Abstract

A small-sized study may report over or under estimated effects of the investigated treatment in randomized controlled trials, which is called small-study effect (SSE). It is commonly related to publication bias. Notwithstanding, the intrinsic, probabilistic, component of SSE has never been assessed yet, which is the purpose of this study. A stochastic model, simulating the results of a given controlled trial with an increasing number of subjects in each group (from 1 to 200) was used. Predefined sets of input data (expected means and standard deviations) covering a full range of analytical situations were entered in a pseudo-random generator to reflect the variability of the individual's responses in each group. For each set of data, the process was repeated 200 times to take variability into account and therefore to investigate the validity of the model. Effect-size and its standard normal deviate were notably computed for each sample-size in order to determine a threshold for SSE. The median (25%; 75%; 90%) sample above which SSE was absent was 16.5 (8;30;50) subjects per group and was nonsignificantly impacted by the different input data sets. A sample size of 50 subjects in each group allowed for no small-study effect in 90% of the simulations. This study pointed out the fact that SSE is not only linked to selective publication, as well as the rationale to take it into account in addition to power calculation in the design of a RCT as well as in publication bias analysis.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 20 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.