Commentary: Relative treatment effects are consistent across the spectrum of underlying risks … usually

Abstract

To practice evidence-based therapeutics, clinicians have to integrate measures of efficacy and safety from the literature with their patient's unique risks and values.1 To do this requires two assumptions. First, we assume that we can accurately estimate our patient's underlying baseline risk (or the ‘expected event rate’ referred to by Furukawa and colleagues).2 This is no simple matter: while clinicians appear to be reasonably accurate in estimating the relative risks of different patients, even experienced clinicians perform poorly when estimating any one individual's absolute risk.3 Methods for estimating the expected event rate for a particular patient have recently been reviewed and will not be considered further in this commentary.1 The second common assumption in extrapolating from trials or meta-analyses to individual patients (who typically are at different risks from the ‘average patient’ in these studies), is that the relative effects of therapy are similar for patients at different risks. The study by Furukawa and colleagues2 in this issue of the International Journal of Epidemiology adds to the emerging evidence supporting the validity of this assumption. In discussing underlying or baseline risks in this commentary, I am not referring simply to the control event rate, but rather to patient characteristics (such as age, gender, disease aetiology, concomitant conditions, or disease status) present at baseline and known to impact prognosis for that particular disease. It is well recognized that any analyses relating treatment effects to control event rates in the same dataset will demonstrate a relation even if none exists (since the control event rate factors into both the expression for baseline risk and the expression for treatment effect).4 The best evidence for deciding whether treatment responsiveness differs across a spectrum of underlying risks would arise from individual patient data meta-analyses where the relative treatment effects in subgroups with widely varying risks can be directly compared. Although some such analyses have been conducted, they are few and far between and we are left to consider the second best approach to address this question: comparison of the relative treatment effects in different trials testing the same intervention (assuming that patients in different trials with the same condition will have different risk profiles). The problem is how to find these groups of related trials—the easiest solution (employed by Furukawa and colleagues) is to review the reference lists of the rigorously conducted systematic reviews included in the Cochrane Collaboration database. I have described this approach as second best as selection bias may very well distort the findings (since a group of trials reporting widely divergent treatment effects are unlikely to be pooled to provide a single summary measure due to excessive heterogeneity). However, at this time it is the best we have. Schmid et al.5 provided the first such systematic approach by examining the relationship between effect measures and baseline risk in 115 meta-analyses (using a hierarchical model to account for the functional correlation between observed rates discussed earlier and random error in measurement of the control rate). They found that while the risk difference was significantly related to underlying risk in 31% of cases, in 87% of cases the relative risk (RR) (and in 86% of cases the odds ratio [OR]) did not vary significantly with the control event rate. Fukurawa and colleagues extend this work in a separate dataset by demonstrating high rates of concordance for both the OR and the RR in 1843 comparisons between individual trials and the summary effect measures derived from pooling all other trials in that topic area.2 In particular, they found that the concordance rates were high for both OR and RR even when control event rates differed substantially (up to threefold) between trials. They found a qualitative discrepancy (where an individual trial reported an RR [or OR] in the opposite direction from the summary RR [or OR] for the other trials) in only 4% of cases but could not discern any features of these trials that suggested why such a discrepancy may occur. Thus, while both studies suggest that relative effect measures are constant across the usual spectrum of underlying risks in the vast majority of cases, neither study has advanced our understanding of when this assumption is unlikely to hold. Sackett has hypothesized that relative treatment effects will be constant over the usual range of underlying risks for ‘risk factor’ interventions designed to slow the progress of disease, but that they will rise with increasing baseline risk for interventions designed to reverse the consequences of a disease process.6 This latter situation would seem to apply particularly when the intervention has both positive and negative effects on the outcome of interest (for example, surgical procedures to prevent certain outcomes in the long-term usually expose patients to an increased risk of these same outcomes in the immediate peri-operative period). Thus, we would expect the relative risk reduction (RRR) associated with treatments such as angiotensin converting enzyme inhibitors in heart failure, beta-blockers in myocardial infarction, or thiazides for hypertension to be similar in patients with different underlying risks—indeed this is exactly what is seen.7–9 On the other hand, we would expect the RRR associated with interventions such as carotid endarterectomy or coronary artery bypass grafting to be higher in patients at higher risk—again, exactly what is seen.10,11 However, a word of caution: the validity of this hypothesis depends on the outcomes examined and is unlikely to hold for combined endpoints where the risk factor intervention impacts on only one of these endpoints.12 For example, consider the example of cholesterol lowering agents. Although these drugs produce a consistent RRR in ‘coronary events' and ‘cardiac mortality’ across different risk strata, their RRR for the combined endpoint of ‘all-cause mortality’ will vary and indeed be greater in ‘high-risk’ patients (such as those with established coronary disease) in whom a greater proportion of all deaths will be cardiac.13 Nevertheless, on the basis of the studies by Drs Schmid and Furukawa, it now seems reasonable to accept the assumption that relative treatment effects are consistent across the spectrum of underlying risks … usually. FM is a Population Health Investigator of the Alberta Heritage Foundation for Medical Research. The author thanks D Sackett, W Taylor, R Roberts, B Haynes, PJ Devereaux, A Laupacis, S Walter, S Yusuf, J Sinclair, S Connolly, T Louis, and B Djulbegovic for their active participation and thoughtful insights in an e-mail discussion group on this issue and M Egger for helpful editorial comments.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 53 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.