No open-access location is known for this record yet.
No open-access location is known for this record yet.
Evidence-based practice (EBP) dictates that decisions about health care are based on patient values, clinical expertise, and the best available, relevant evidence.1 But what actually constitutes “best” in rehabilitation interventional research evidence? The fundamental view of the EBP movement is that not all evidence offers the same degree of support to make causal claims about a clinical intervention.2 To identify the best evidence from the huge volume and diversity of available evidence, evidence hierarchies are used. An evidence hierarchy is a heuristic used to rank the relative strength of results obtained from scientific research. One of the best-known evidence hierarchies is the evidence pyramid. This pyramid typically holds systematic reviews and meta-analyses of randomized controlled trials (RCTs) at the top, single RCTs near the top, nonrandomized cohort and case–control studies lower, and fundamental studies and case reports near or at the bottom. The general idea behind the evidence pyramid is that the best available evidence is the evidence closest to the top. In this editorial, we argue that the application of evidence hierarchies to distinguish bad and good evidence might actually be harmful to advancing EBP. But, before we get to that, we want to introduce you to Procrustes. According to ancient Greek mythology, Procrustes was a rather remarkable innkeeper. Procrustes, whose name translates into “he who stretches,” managed an inn on the side of the road. In this inn, he invited passing strangers to enjoy a copious meal and a night’s rest in his very special bed. He told his guests that his bed had a unique property: the length of the bed would always exactly match the person who slept in it. What Procrustes failed to mention, however, was how this bed was tailored to a guest’s length. Rather than adjusting the bed to the length of his guest, Procrustes would adjust the length of his guests to his bed. If guests were too short, Procrustes would stretch them on the rack. If guests were too long, Procrustes would cut off their legs. None of Procrustes’s guests survived the night, and he would rob his victims afterwards. According to the Encyclopedia Britannica,3 the Procrustean bed “has become proverbial for arbitrarily—and perhaps ruthlessly—forcing someone or something to fit into an unnatural scheme or pattern.” The evidence pyramid, which nowadays comes in many shapes and forms, might be an example of a Procrustean bed.4–6 On the one hand, the idea behind these hierarchies is appealing; they are intuitive and easy to understand, and, above all, they help to easily “weed out” the bad from the good intervention research evidence. On the other hand, the overtly strict and inflexible nature of these hierarchies might actually harm the thing we want to nurture. Over the years, several philosophers and researchers have questioned the utility of using evidence hierarchies for a wide range of reasons.7–10 We highlight 3 arguments against the use of hierarchies. Hierarchies are inherently simplistic, as they categorize quality of research evidence primarily on the basis of a study’s design. Ask yourself, does a poorly designed and conducted RCT produce “better evidence” than a well-designed and performed case–control study? There are scenarios where this isn’t the case. To adequately determine the quality of evidence from a study, the conduct and detailed description of the performed research should be weighed against the specific research question posed as well as the potential for its significance and impact. We believe that such a quality assessment should prevail over judging the quality of evidence based on the general name of the study design, especially knowing that researchers are notoriously poor at adequately naming their study designs.11 Hierarchies over-appreciate internal validity of a study’s design and largely neglect its external validity. An internally valid study confers a high probability of truth on the result of the study, whereas external validity reveals the degree to which the results established in the study will be true elsewhere. Alas, these 2 concepts are not independent and cannot be jointly maximized in a single study.9,12 Due to hierarchical thinking and subsequent study selection, external validity can suffer at the expense of internal validity. For example, a guideline panel ignores, due to hierarchical thinking, a large cohort study that contradicts a well-conducted but very selective trial. The latter might reduce the applicability of the research for the whole target population or even, to put it in Procrustes’s vocabulary, maim the potential effectiveness of treatment as a result of overstandardizing the studied intervention.13,14 Although it is never a good idea to use a single study to inform decision making in practice,7 systematically reviewing multiple studies won’t necessarily solve the problems of reduced external validity,15 especially when we cling to hierarchical thinking—which brings us to the third point. The use of hierarchies can lead us to discard valuable, complementary evidence that might be useful for decision making. Perhaps the most important question that health care research aims to address is: “Is this intervention safe and efficacious for this person?” However, there is not one single research design that can answer all relevant research questions about a rehabilitation intervention. Different types of evidence are required to answer different types of research questions, such as an intervention’s timeliness, equity, cost-effectiveness, or patient centeredness.16 When we systematically review evidence, we typically just try to answer one research question: “Is this intervention efficacious somewhere?” (ie, in a controlled experimental setting). Evidence hierarchies lack a methodological inclusiveness to review all the evidence relevant to answer that overall research question. The notion that one research question is “better,” “more valuable,” or “stronger” than the other is misleading, as is the notion that the answers to more deductive questions produce “better” or “stronger” evidence. Different research simply produces different answers and diverse knowledge.9 Regardless, many of the published systematic reviews still answer very narrow questions related solely to the efficacy of interventions. One might argue, “Aren’t you being unnecessarily harsh about a practical tool?” or “Shouldn’t we perceive these hierarchies more like guides rather than tools?”17 This seems logical—except for the fact that nowadays nearly every systematic review or guideline in the medical sciences uses some form of evidence hierarchy, consciously or unconsciously. The one-size-fits-all approach of evidence hierarchies makes us disqualify useful evidence, thereby robbing from us our most precious ambition, which is to determine whether a rehabilitation intervention is valuable for that new client in front of us. We need to start tailoring the “bed” to meet our needs, rather than the other way around. In the ancient Greek myth, Procrustes was brought to justice by Theseus, son of Poseidon. We realize PTJ plays an important role in bringing our own Procrustes to justice. The editors and Editorial Board, therefore, welcome a wide range of research designs related to our rehabilitation interventions. In addition to clinical trials, clinical trial protocols, and systematic reviews of clinical trials, PTJ publishes a range of research using cohort, case‐control, and cross-sectional designs; single-subject designs; diagnostic studies; implementation investigations; feasibility pilot studies; qualitative studies; and measurement psychometric investigations. We believe that all designs contribute to the growing evidentiary base of physical therapy and rehabilitation clinical interventions. Dr Jette and Dr Hoogeboom thank Sijmen Hacquebord for introducing them to the story of Procrustes.
Loading…
Stored only when both papers are in the corpus — 8 citations exist upstream.
Papers that cite the same references — the relational fallback while embeddings are still backfilling.
Imported from OpenAlex (CC0) — nothing here is generated.