We are witnessing exponential growth in a heavily capitalized digital health industry which promises to transform behavioral and mental health care1, 2. Consequently, it is critical that there is no ambiguity about the evidence standards necessary for the safe and effective treatment of psychiatric disorders through digital approaches. In our opinion, these standards should be essentially the same as for any other form of treatment, or even arguably higher, given the intrinsic likelihood of placebo effects in software products that are specifically designed for user engagement3. These standards are all the more necessary as user engagement is often far less than reported, in clinical studies but especially in the real world, where it hovers at less than 5% after two weeks4, 5. The nascent digital industry, therefore, must resist marketing and investor pressures in favor of sound clinical governance when developing “software as a medical device” (SaMD) where the declared purpose is “treatment or alleviation of disease”6; a modality increasingly referred to as “digital therapeutics” (DTx)2. If the industry unequivocally adopts evidence-based gold standards, there is opportunity for great good, and this need not be a complex undertaking, because the treatment evaluation template is well established. The randomized controlled trial (RCT), despite its focus on internal validity and inherent limitations to generalizability, has been the mainstay of evidence-based medicine for many decades, and the solution to the crucial matter of external validity may be found in real-world data (RWD), which is best regarded as complementary, rather than alternative, to clinical trials data7. We are concerned, however, that it may be tempting to utilize the near-hand available RWD associated with SaMD to supplant the need for robust clinical trials. We do not believe that this is the US Food and Drug Administration (FDA)’s intention when they require RWD as part of their Pre-Cert model, but rather that the combination of RCT and RWD offers a compelling safety and effectiveness argument. This combination is necessary as, while an RCT can establish efficacy, the very nature of DTx, as compared to pharmaceuticals, requires their clinical effectiveness to be studied. With mounting data that real-world longitudinal engagement with many of these apps is minimal, the need for this clinical “pipeline” of studies has become critical5. The risks of ignoring this rigorous pathway are substantial. There is already a parallel concern relating to neurotechnology devices being marketed to consumers as aids to cognitive and mental health without sufficient oversight8. We are mindful of a history of what has been termed “stealth research”9 in the digital sector, which has already caused reputational damage, and are wary of reliance on an “evidence-informed” company rhetoric that is not consistent with evidence-based standards. Although we absolutely do recognize that clinical trials research should be combined with other inputs to ensure evidence-informed decision-making in clinical practice, our point is that it is spurious to regard evidence-informed as a substitute for evidence-based. The requirement to generate clinically meaningful evidence on a DTx should be related to the product itself being evidence-based. However, one becomes familiar with an unhelpful form of “inductive reasoning” along the lines of: “1. X treats Y effectively; 2. This new product contains X; 3. Therefore, this new product treats Y effectively”. No novel selective serotonin reuptake inhibitor (SSRI) would ever be approved or offered to patients without testing just because other SSRIs have established effects. Of course, DTx are likely to contain behavioral elements, but the fallacious argument equally applies. Indeed, 14 out of the 25 FDA-cleared DTx products utilize cognitive-behavioral therapy (CBT) to treat the conditions they target1. Simply having content that is drawn from an evidence-based field, or endorsed by subject experts, does not demonstrate clinical efficacy of a novel DTx. Our argument is that any candidate DTx product itself, with its integrated content fields and software algorithms, needs to be subjected to rigorous evaluation in a clinical trial program as well as in real-world use cases, in order to be regarded as a safe and evidence-based treatment. Consequently, a candidate DTx should not be made available to treat a medical condition until it has proven benefits, because the intention to become a therapeutic does not make any intervention a therapeutic. The most obvious danger of treating evidence-informed as evidence-based in DTx is the potential for adverse effects, reckless inefficacy, and devaluation of the entire space. Along with this, however, there is an additional danger stemming from any perceived equivalence of evidence-informed and evidence-based. Specifically, treating evidence-informed DTx as though they are evidence-based creates an environment in which actual evidence-based interventions (e.g., in-person CBT) could be easily replaced by DTx, which claim to have the same evidence as those existing interventions, but in reality could lack the efficacy of those genuinely evidence-based approaches. Thus, in a worst-case scenario, blind substitutions of evidence-based care with evidence-informed DTx could deprive patients of effective interventions, while providing them with a time-wasting or even adverse alternative. Although our analogies to drugs and to in-person therapeutics may be imperfect, we strongly urge that it is prudent to apply the same standards, if not even higher, to the DTx clinical research pipeline. Some may say that the emergence of a novel, disruptive approach like DTx presents the opportunity to “break the mould”. However, surely the counterbalance to that is that the greater the novelty, the greater the need for caution. There is much at stake where treatment of psychiatric conditions is concerned, and the duty to be evidence-based should not be taken lightly. The popular phrase attributed to astronomer Carl Sagan, “Extraordinary claims require extraordinary evidence”, is a fitting end to this piece, and a beginning to the journey towards a higher standard for evidence.
Stored only when both papers are in the corpus — 28 citations exist upstream.