Shifting effects in randomised controlled trials of complex interventions: a new kind of performance bias?

Abstract

OBJECTIVE: Randomised controlled trials (RCTs) aim to provide unbiased estimates of treatment effects. However, the process of implementing trial procedures may have an impact on the performance of complex interventions that rely strongly on the intuition and confidence of therapists. We aimed to examine whether shifting effects over the recruitment period can be observed that might indicate such impact. METHOD: Three RCTs investigating music therapy vs. standard care were included. The intervention was performed by experienced therapists and based on established methods. We examined outcomes of participants graphically, analysed cumulative effects and tested for differences between first vs. later participants. We tested for potential confounding population shifts through multiple regression models. RESULTS: Cumulative differences suggested trends over the recruitment period. Effect sizes tended to be less favourable among the first participants than later participants. In one study, effects even changed direction. Age, gender and baseline severity did not account for these shifting effects. CONCLUSION: Some trials of complex interventions have shifting effects over the recruitment period that cannot be explained by therapist experience or shifting demographics. Replication and further research should aim to find out which interventions and trial designs are most vulnerable to this new kind of performance bias.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 13 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.