Mind the Psychedelic Hype: Characterizing the Risks and Benefits of Psychedelics for Depression

Abstract

Rationale: Psychedelic research re-emerged from a period of suppression into the so-called psychedelic renaissance. In parallel, most media reporting has shifted from the overstatement of the risks of psychedelics to overly positive hype. As the empirical evidence is more equivocal than frequently portrayed, the conclusions about the effectiveness of psychedelics should be considered preliminary. Poor science communication about psychedelics’ therapeutic potential may lead potential participants or patients to feel misled and policy decisions to be misinformed. An evidence-informed characterization of their risks and benefits is needed. Objectives: This article assesses the state of psychedelic research for treating depression and the effect sizes of psychedelics on therapeutic outcomes, the risk of bias, and the prevalence of adverse effects. We review research on the risks and benefits of psychedelics and discuss how the following depression treatments have shown decreasing effect sizes over time: (1) cognitive behavioral therapy, (2) mindfulness interventions, (3) selective serotonin reuptake inhibitors, and (4) ketamine. We speculate that a similar trend may occur for psychedelic treatments. Results and conclusions: It is likely that larger and better-controlled psychedelic trials will demonstrate smaller effect sizes that are more comparable to other conventional and emerging treatments for mood disorders. Clear science communication is critical for setting public expectations and psychedelic policy. With this evidence-based assessment, we aim to cut through the misinformation about the benefits, risks, and future prospects of psychedelic treatments.

1. Introduction

In recent years, the therapeutic effects of psychedelics have been investigated for multiple mental health conditions, and public interest in the potential benefits of psychedelics has grown. While these results are promising, more research is required to more precisely identify the factors that foster positive therapeutic outcomes from psychedelic-assisted therapy [1,2,3]. Despite the preliminary nature of the research quantifying the risks and benefits of psychedelics, this field has attracted a substantial amount of hype or attention, coinciding with inflated expectations, echoing the hype around mindfulness-based interventions that occurred a few years ago (i.e., in an article called “Mind the Hype” [4], which we directly acknowledge in the title of the present article). The hype surrounding psychedelics may lead to expectancy effects and selection bias. Given the growing number of psychedelic clinical trials and highly publicized results, it is time to critically assess the current hype around psychedelics.

Psychedelics are a group of psychoactive substances that can temporarily alter perception, emotions, cognition, and the sense of self [5,6,7,8,9]. Classic psychedelics (or serotonergic psychedelics) are marked by their characteristic 5-HT2a partial agonism [10] and include psilocybin, lysergic acid diethylamide (LSD), mescaline, and N,N-dimethyltryptamine (DMT). These substances are considered physiologically safe and largely non-addictive [10]. After decades of prohibition and suppressed research on psychedelics, a new resurgence of psychedelic research is occurring [11]. Over the past few years, the number of publications on psychedelics has increased dramatically (Figure 1), including a growing number of clinical studies demonstrating the promising therapeutic potential of psychedelics for treating a broad range of conditions [2,12,13,14,15], such as major depressive disorder [5,16,17,18,19,20], anxiety or depression due to a life-threatening diagnosis [21,22,23], and substance use disorders [24].

The representation of psychedelics in public media has moved from alarmism about their risks (for the past several decades) to hyping their potential benefits (in just the past few years). The exaggeration of potential harms due to psychedelics led to overly negative expectations of psychedelic drugs’ effects, including the perception that psychedelics routinely triggered psychotic episodes and dangerous behavior, or even blatant scare-mongering related to long-term DNA damage (for which there is no evidence). This alarmism was driven by anecdotes and misinformation [25] and resulted in punitive public policies (e.g., the “War on Drugs”) and restrictions on research [26,27,28].

In contrast, exaggeratedly positive expectations have become common in major media outlets during the recent resurgence of psychedelic research. The potential benefits of emphasizing positive drug effects include the provision of a corrective to decades of overly negative propaganda [29]; however, some representations inflate the evidence of their potential benefits [30]. Dubbed the “Michael Pollan Effect”, such pervasive positive messaging about psychedelics has led to heightened expectations regarding the efficacy of psychedelics [25]. Participants may enter a trial with expectations regarding both their experience of the treatment as well as the treatment outcomes, and these expectations likely interact with both the treatment process and the therapeutic response [25].

These two forms of exaggeration regarding psychedelic drug effects, overly positive and overly negative, can both have negative epistemic consequences regarding the public perception of their risks and benefits that are disconnected from empirical evidence. Overly negative perceptions of psychedelics delayed research progress for decades, and overly optimistic representations of psychedelics may compromise the trust in psychedelic research when outcomes do not match inflated expectations. An evidence-based recalibration of these expectations of the potential risks and benefits of psychedelics is needed to inform the public about psychedelics. As part of this recalibration, the risks and benefits of psychedelics should be quantified and ideally compared in various ways to other relevant psychiatric treatments and psychological interventions. This narrative review examines the reported benefits and risks of psychedelics for the treatment of depression, including an analysis of the published effect sizes and a Cochrane risk of bias analysis. It then argues that bigger and better trials may well show decreased effect sizes, as has been found over time with other treatments and interventions: (a) cognitive behavioral therapy (CBT), (b) mindfulness interventions, (c) selective serotonin reuptake inhibitors (SSRIs), and (d) ketamine. Additionally, this manuscript will discuss the methodological shortcomings of the research, including placebo effects, bias in the conduct and reporting of trials, the breaking-blind problem, and publication bias. Finally, recommendations are provided regarding standardization for future trials and the science communication related to psychedelic-assisted therapy.

2. Methods

Narrative Review Strategy

The peer-reviewed literature was searched for major meta-analyses of randomized controlled trials (RCTs) of psychedelics, CBT, mindfulness, and SSRIs on depressive symptoms. The key results from these papers are briefly summarized. From these meta-analyses, the following data were extracted from each individual trial: the year of publication, compound, dose, number of sessions, primary outcome measure, and comparison group. Between-group effect sizes were imputed using the Webplot Digitizer version 4.7 tool to extract data, if not reported directly in the original manuscript or available in the Haikazian et al. meta-analysis [31]. Effect sizes, which indicate the magnitude of the difference between the groups being compared, are reported as either Cohen’s d or Hedge’s g [32]. These two measures of effect sizes are similar and comparable, though, due to differences in their calculation methods, Hedge’s g is less subject to bias with small sample sizes [32]. Psychedelic clinical trials’ data were described to show the magnitude of the benefits observed so far (through the effect size of primary and secondary outcomes) and risks (through the incidences of adverse events), by year of publication. A qualitative comparison of these risks and benefits relative to other treatments was conducted to show how the risk/benefit profiles of other treatments have generally become worse over time and, therefore, might be an expected trajectory for psychedelics. We also conducted a quantitative Cochrane risk of bias assessment to analyze the domains of bias for the psychedelic trials. The specific measures and analyses of interest in this review are (1) between-subject studies comparing psychedelics with control groups, (2) open-label and within-subject studies assessing the pre–post changes in measures of depression due to psychedelics, and (3) assessments of the adverse effects of psychedelics.

3. Results

3.1. Overview of the Benefits and Risks of Psychedelics

The effects sizes of psychedelic treatments, as well as the prevalence of adverse events, vary over different conditions and disorders. A recent systematic review that only included double-blind placebo-controlled clinical trials found that a treatment with a classic psychedelic compared to a placebo resulted in lasting improvements 1 day, 1 week, and 3–5 weeks following the intervention, with between-group standardized mean differences (SMDs) ranging from 1.36 to 3.12 [33].

The risks of psychedelics in controlled therapeutic settings are primarily driven by drug-induced changes in perception, cognition, and emotion, with the side effects of short-term mood instability and possible sensory disturbances [34,35]. Reviews of the psychedelic literature show that when administered in a regulated or medical context, the medical risks of psychedelics are low and adverse events requiring medical intervention are rare [25,36,37]. A recent systematic review and meta-analysis found low rates of serious adverse events (SAEs) overall, with no SAEs after psychedelic administration in healthy participants and with around 4% of participants with pre-existing neuropsychiatric conditions experiencing SAEs [37]. According to Carbonaro et al. in 2016, after more than 380 sessions of psilocybin administrations (doses of 20 mg/70 kg or higher) with about 250 research participants at Johns Hopkins, only three participants (0.9%) showed disorientation during a session to a level that might have put them or staff members at risk without adequate supervision [38]. In the research, no completed suicides, hallucinogen persisting perception disorders, or persistent psychotic symptoms have been reported following the administration of moderate to high doses of classic psychedelics [39].

In what follows, we provide a more focused view on the benefits and risks of psychedelic treatments.

3.2. Overview of Individual Psychedelic Trials for Depression

The treatment of depression is an important field of application for psychedelic-assisted therapy. Nine controlled trials of psychedelics for treatment-resistant depression or major depressive disorder (MDD), published between 2014 and 2023, have been summarized in Table 1 and in Figure 2, some of which have shown large effect sizes (typically considered ≥0.8). Note that while the primary outcome in Gasser et al. was anxiety, depression was highly comorbid in the study sample and the results were reported using the Hospital Anxiety and Depression Scale. The sample size of these trials ranged from 12 to 233 participants and follow-up times ranged from two weeks to twelve months (Table 1). All nine trials reported between-group effect sizes (Table 1). Seven trials used psilocybin as the primary intervention (with varying dosages) and the remaining trials used LSD or ayahuasca as the psychedelic intervention (Table 1). The placebo condition varied; four studies used inactive placebos, four studies used active placebos, and one study used a delayed treatment group comparison (Table 1). Eight different scales were used to measure depression across all eight trials; versions of the Hamilton Depression Rating Scale (HAM-D) were the most common (used in three trials). The between-group effect sizes at the final follow-up ranged from a Cohen’s d of 0.43 on the Quick Inventory of Depressive Symptomatology (Self-Report) scale to a Cohen’s d of 5.2, also on the Quick Inventory of Depressive Symptomatology (Self-Report) scale (Figure 2). As a function of the year of publication and the time of follow-up, the sample size for psychedelic trials tends to grow over time, while there currently is no quantitative trend of a decline in effect sizes (Figure 2).

Table 1. Clinical randomized controlled studies on the effects of psychedelics on depression and disease-related depression and anxiety.

Study	Compound and Dose	Primary Outcome Measure	Follow-Up Time	Sample Size	Comparison Groups	Standardized Mean Difference a
von Rotz et al., 2022 [20]	Psilocybin, 0.215 mg/kg (1 session)	MADRS, BDI	14 days	52 (26 psilocybin, 26 placebo)	Randomized, double-blind, placebo-controlled clinical trial: psilocybin vs. placebo	MADRS: 0.92 (Day 14)
Goodwin et al., 2022 [40]	Psilocybin, 0.215 mg/kg (1 session)	MADRS	Week 3	233 (79 receive 25 mg, 75 received 10 mg, 79 received 1 mg)	Randomized double-blind, controlled trial: single dose for each group (1 mg vs. 10 mg vs. 25 mg)	MADRS: 0.61 (Week 3)
Davis et al., 2021 [18]	Psilocybin, 0.29, 0.43 mg/kg (2 sessions)	GRID-HAMD, QIDS-SR	Weeks 1 and 4	27 (15 immediate treatment, 12 waiting list control)	Randomized, waiting-list-controlled clinical trial: treatment condition group vs. delayed treatment condition group	GRID-HAMD: 2.21 (Week 4)
Griffiths et al., 2016 [22]	Psilocybin, High dose 22 or 30 mg/70 kg, low dose 1 or 3 mg/kg (2 sessions)	GRID-HAMD HAM-A	Week 5 and month 6	51	Randomized double-blind, cross-over trial: comparison of low versus high psilocybin dose	GRID-HAMD: 1.25 (Week 5)
Ross et al., 2016 [23]	Psilocybin, 0.3 mg/kg (1 session)	BDI STAI-T	7 weeks	29	Randomized, double-blind, placebo-controlled, crossover trial: psilocybin vs. placebo	BDI: 0.87 (Week 7)
Gasser et al., 2014 [21] b	LSD, 200 µg (2 sessions)	STAI-S STAI-T	2 months and 12 months	12	Randomized, double-blind, active placebo-controlled pilot study	HADS-D: 2.7 (Week 8)
Palhano-Fontes et al., 2019 [41] b	Ayahuasca, 1.0 mL/kg (0.36 mg/mL of DMT, 1.86 mg/mL harmine) (1 session)	HAM-D	1 week	29 (15 placebo, 15 ayahuasca)	Randomized placebo-controlled trial: treatment vs. placebo	HAM-D: 1.46 (Day 7)
Raison et al., 2023 [17]	Psilocybin, 25 mg (1 session)	MADRS	43 days	104 (53 placebo, 51 psilocybin)	Randomized Phase II double-blinded active placebo-controlled trial	MADRS: 0.92 (Day 43)
Carhart-Harris et al., 2021 [16]	Psilocybin, 25 mg (2 sessions)	QIDS-SR	6 weeks	59 (29 escitalopram and 1 mg psilocybin placebo, 30 psilocybin)	Randomized Phase II double-blinded placebo-controlled trial	QIDS-SR: 0.36 (Week 6)

Note: This selection of studies includes randomized, controlled studies researching the effects of psychedelics on depression or disease-related depression and anxiety in comparison to placebo, waitlists, or other controls. Follow-up time is recorded as primary endpoint. ^a All values extracted from Figure 2 in Haikazian et al. [31] unless otherwise noted. ^b These values were calculated using the Webplot Digitizer tool and Excel. Calculations available in the Supplementary Material.

Table 1. Clinical randomized controlled studies on the effects of psychedelics on depression and disease-related depression and anxiety.

Study	Compound and Dose	Primary Outcome Measure	Follow-Up Time	Sample Size	Comparison Groups	Standardized Mean Difference a
von Rotz et al., 2022 [20]	Psilocybin, 0.215 mg/kg (1 session)	MADRS, BDI	14 days	52 (26 psilocybin, 26 placebo)	Randomized, double-blind, placebo-controlled clinical trial: psilocybin vs. placebo	MADRS: 0.92 (Day 14)
Goodwin et al., 2022 [40]	Psilocybin, 0.215 mg/kg (1 session)	MADRS	Week 3	233 (79 receive 25 mg, 75 received 10 mg, 79 received 1 mg)	Randomized double-blind, controlled trial: single dose for each group (1 mg vs. 10 mg vs. 25 mg)	MADRS: 0.61 (Week 3)
Davis et al., 2021 [18]	Psilocybin, 0.29, 0.43 mg/kg (2 sessions)	GRID-HAMD, QIDS-SR	Weeks 1 and 4	27 (15 immediate treatment, 12 waiting list control)	Randomized, waiting-list-controlled clinical trial: treatment condition group vs. delayed treatment condition group	GRID-HAMD: 2.21 (Week 4)
Griffiths et al., 2016 [22]	Psilocybin, High dose 22 or 30 mg/70 kg, low dose 1 or 3 mg/kg (2 sessions)	GRID-HAMD HAM-A	Week 5 and month 6	51	Randomized double-blind, cross-over trial: comparison of low versus high psilocybin dose	GRID-HAMD: 1.25 (Week 5)
Ross et al., 2016 [23]	Psilocybin, 0.3 mg/kg (1 session)	BDI STAI-T	7 weeks	29	Randomized, double-blind, placebo-controlled, crossover trial: psilocybin vs. placebo	BDI: 0.87 (Week 7)
Gasser et al., 2014 [21] b	LSD, 200 µg (2 sessions)	STAI-S STAI-T	2 months and 12 months	12	Randomized, double-blind, active placebo-controlled pilot study	HADS-D: 2.7 (Week 8)
Palhano-Fontes et al., 2019 [41] b	Ayahuasca, 1.0 mL/kg (0.36 mg/mL of DMT, 1.86 mg/mL harmine) (1 session)	HAM-D	1 week	29 (15 placebo, 15 ayahuasca)	Randomized placebo-controlled trial: treatment vs. placebo	HAM-D: 1.46 (Day 7)
Raison et al., 2023 [17]	Psilocybin, 25 mg (1 session)	MADRS	43 days	104 (53 placebo, 51 psilocybin)	Randomized Phase II double-blinded active placebo-controlled trial	MADRS: 0.92 (Day 43)
Carhart-Harris et al., 2021 [16]	Psilocybin, 25 mg (2 sessions)	QIDS-SR	6 weeks	59 (29 escitalopram and 1 mg psilocybin placebo, 30 psilocybin)	Randomized Phase II double-blinded placebo-controlled trial	QIDS-SR: 0.36 (Week 6)

Note: This selection of studies includes randomized, controlled studies researching the effects of psychedelics on depression or disease-related depression and anxiety in comparison to placebo, waitlists, or other controls. Follow-up time is recorded as primary endpoint. ^a All values extracted from Figure 2 in Haikazian et al. [31] unless otherwise noted. ^b These values were calculated using the Webplot Digitizer tool and Excel. Calculations available in the Supplementary Material.

Figure 2. Overview of the effect sizes of psychedelic-assisted psychotherapy at different follow-up measurement points. The vertical axis represents the effect sizes. The horizontal axis represents the year of publication. The different follow-up measurement points are represented in differently-shaded circles: a darker shading indicates a later measurement time point. The sample size of each study is represented by the size of the circle: a larger circle indicates a larger sample size [16,17,18,20,21,22,23,40,41].

Figure 2. Overview of the effect sizes of psychedelic-assisted psychotherapy at different follow-up measurement points. The vertical axis represents the effect sizes. The horizontal axis represents the year of publication. The different follow-up measurement points are represented in differently-shaded circles: a darker shading indicates a later measurement time point. The sample size of each study is represented by the size of the circle: a larger circle indicates a larger sample size [16,17,18,20,21,22,23,40,41].

A Cochrane risk of bias 2.0 analysis was conducted for all eight of the psychedelic trials by two authors (M.G. and R.E.) and any coding differences were resolved through group discussion (Supplementary Table S1). The domains of bias evaluated included potential biases due to the randomization process, deviations from intended interventions, missing outcome data, the measurement of the outcome, the selection of the reported result, and the overall risk of bias. The risk of bias was estimated for each trial and for each domain across all trials. Any study with a high risk of bias in any individual domain was recorded as generating some concerns about bias in the overall assessment of that study. Any study with a high risk of bias in two or more individual domains was recorded as having high risk of bias in the overall assessment of that study. The risk of bias calculation for each bias domain was weighted by study sample size. Risk of bias plots were created using the robvis package in R Studio Version 2023.06.2+561 [42].

Three quarters of all psychedelic trials were at a low to moderate overall risk of bias, with the remaining 25% were graded as having a likely high overall risk of bias (Figure 3). Four trials had at least one domain in which there was a high risk of bias, and half of all trials had only one domain of some concern while all other domains were judged as having low risk of bias (Figure 3). All trials were considered to have a low risk of bias in the selection of their reported results, and no trials were at high risk of bias in the missing data domain. The deviations from the intended intervention domain had the highest number of trials with a high risk for bias, which was due to the likelihood that blinding may have failed for the participant and/or assessor due to the chosen control condition.

Figure 4 displays the proportion of the risk of bias categories across all trials at low risk, unclear risk, and high risk of bias, weighted based on the sample size of each study. Across all trials, the domain with the lowest risk of bias was the selection of the reported result, and the domain with the highest risk of bias was bias due to deviation from the intended intervention (Figure 4). Other bias domains display a predominantly low risk of bias (Figure 4). The overall risk of bias is low, although moderate or high risks of bias remain present in the published psychedelic trials for depression (Figure 4).

3.3. Overview of Meta-Analyses for Psychedelic Trials for Depression

Recent meta-analyses are summarized here to provide a wider and more systematic overview of the existing literature on psychedelic trials. In a meta-analysis of 14 studies assessing the effects of psilocybin (11 studies), ayahuasca (2 studies), and LSD (1 study) on depressive symptoms, standardized mean differences (SMDs) were calculated. Large between-group effect sizes were reported as follows: SMD = −1.36 (95% CI [−2.50, −0.22], k = 4 studies) at day 1 post psychedelic administration, SMD = −1.37 (95% CI [−2.41, −0.34], k = 3 studies) at week 1, SMD = −3.12 (95% CI [−6.19, −0.04], k = 3 studies) at weeks 3–5, and SMD = −1.52 (95% CI [−3.55, −0.51], k = 3 studies) at weeks 6–8 [33]. This meta-analysis demonstrated a statistically significant reduction in depressive symptoms with clinically meaningful implications at all timepoints after psychedelic administration, with the exception of weeks 6–8 [33]. A 2023 systematic review and meta-analysis of psilocybin-assisted therapy for depression had a pooled sample of nine trials with a total of 596 participants (n = 340 in the psilocybin group and n = 256 in the control conditions) [31]. This study reported a large overall effect size favoring psilocybin over the placebo for decreasing depressive symptoms (SMD = −0.78, p < 0.001), though this is lower than the effect sizes found by Ko et al. [31]. Additionally, this study calculated the risk ratios for response and remission, finding an overall risk ratio of 2.63 (95% CI: 1.84, 3.77) for the response to treatment and an overall risk ratio of 3.13 (95% CI: 2.22, 4.41) for remission [31]. Thus, Haikazian et al. found that those receiving psilocybin were more than twice as likely to have a treatment response and more than three times as likely to remit than those in the control group.

3.4. Adverse Effects of Psychedelics

The risks of psychedelic treatments for patients with depressive symptoms are relatively low when evaluated in terms of the prevalence of adverse events (AE) and serious adverse events (SAE) after the administration of psychedelic substances during clinical trials. Transient AEs have been reported frequently in several clinical studies on psychedelic treatments of depression. Among the AEs, the most common psychiatric adverse event was transient anxiety [16,22]. The most common medical adverse events included non-clinically significant elevations in blood pressure and heart rate [18,23], headaches/migraine, and nausea [5], as well as vomiting, specifically for ayahuasca [41,43]. All reported adverse events were transient and rarely lasted longer than the acute effects of the psychedelic substance (or longer than the following day in the case of headaches).

Goodwin et al. recently reported 14 treatment-emergent serious adverse events (TESAEs) across nine participants within their large randomized controlled study of psilocybin for treatment-resistant depression. These TESAEs included suicidal behavior, intentional self-injury, and suicidal ideation, which are frequently observed in this patient population [19]. We note, however, that in the published literature there appears to be a lack of consistency and transparency in the reporting of adverse and serious adverse events in response to psychedelic therapy, including how adverse events are defined [36].

3.5. Comparing Psychedelics to Other Treatments and Interventions

In addition to examining the benefits and risks of psychedelic treatments in the extant literature, it is useful to consider the likely trend of this research. For this, we briefly describe the research trends of other evidence-based depression treatments that also received substantial hype in the past decades, such as (a) cognitive behavioral therapy, (b) mindfulness-based interventions, (c) SSRIs, and (d) ketamine. Given the heterogeneity of the literature in terms of study designs, control groups, and outcome assessments, our intended formal quantitative comparison of the risks and benefits was not possible, thus a narrative comparison was conducted instead. This heterogeneity is an important ongoing limitation for addressing the population-level risks and benefits of psychedelic treatments. Nevertheless, this qualitative comparison serves as a baseline and reference for future research as the field of psychedelic treatments grows. If psychedelic treatments follow the trend of other psychiatric interventions in studies with larger sample sizes, as discussed below, their effect sizes will decrease over time and adverse event reporting will likely increase.

3.6. Cognitive Behavioral Therapy

CBT is the most researched form of psychotherapy for adult depression [44] and is considered by some as the current gold standard of psychotherapy [45]. CBT is a directed, skills-based treatment modality aimed at modulating maladaptive emotional responses by changing thoughts and/or behaviors [46]. CBT is typically a time-limited treatment, wherein a complete course of CBT may last between 5 and 20 h long sessions has demonstrated efficacy in treating post-traumatic stress disorder, generalized anxiety disorder, and major depressive disorder [46,47].

Two meta-analyses of CBT for depression were reviewed, Cuijpers et al. in 2013 [44] and Johnsen and Friborg in 2015 [48]. Cuijpers et al. conducted a meta-analysis of 94 comparisons from 75 controlled studies on the short-term efficacy of CBT for adult depression. Studies were eligible for inclusion if they evaluated CBT in which cognitive restructuring was the core element of the treatment, and for which at least two of the following other components were mentioned: behavioral activation, social skills training, relaxation training, or coping skills training [44]. The included studies were published between the years 1977 and 2010, the primary target group across all studies was the general adult population, and most studies conducted between 8 and 16 therapy sessions. In this analysis, CBT, in addition to pharmacotherapy, was found to be more effective than pharmacotherapy alone [44]. However, CBT alone was not found to be more or less effective than other forms of therapy or pharmacotherapy individually [44]. The average effect size for CBT was a Hedges g = 0.71, which was reduced to 0.53 after adjusting for publication bias and study quality (with lower quality studies demonstrating higher effect sizes) [44]. Subgroup analyses demonstrated that effect sizes tended to be larger in studies run in the United States, and that effect sizes were small in studies using clinical samples and for the general adult population rather than a specific subgroup (e.g., postpartum women) [44].

Some have argued that the CBT literature has shown a decline in efficacy over time [48], although others have disputed this [45]. Johnsen et al. conducted a meta-analysis of 70 studies (both randomized controlled and uncontrolled) on the effects of traditional CBT techniques on unipolar depressive disorders. CBT effect sizes were reported for both patients (BDI) and therapists (HRSD). The included studies were published between 1977 and 2014, had an average of 34.6 patients per study (range: 7–217, standard deviation = 34.1), and the majority of participants were female (69.1%). Evidence of publication bias indicated that studies with smaller sample sizes testing BDI changes had larger effect sizes than studies with larger samples, although this was not seen for studies testing HRSD changes. Overall, this meta-analysis showed large average effect sizes of a Hedges’ g = 1.58 for the BDI and Hedges’ g = 1.69 for the HRSD [48]. Crucially, based on metaregression, Johnsen and Friborg found that the efficacy of CBT has declined over time when measured based on patient reporting, clinician reporting, and rates of remission [48].

Notably, the large difference in effect size between the meta-analyses of Cuijpers et al. and Johnsen et al. may be due to the inclusion of within-group comparisons in the Johnsen et al. analysis, which are expected to inflate effect sizes in comparison to the between-subject comparisons focused on in Cuijpers et al. [44] As with psychedelic studies, blinding is a likely cause of bias in CBT trials. Even when outcome assessors are blind to the intervention, if a participant guessed their treatment allocation, they may have responded differently during their outcome assessment. There are additional similarities with the bias observed for psychedelic trials, e.g., related to creating an active control for a therapy-based intervention, as any other form of therapy offered may have its own therapeutic effects [49].

In yet another re-analysis by Ljótsson et al. [50], the authors conclude that the effect sizes fell for a couple of decades before leveling off. Thus, there appears to be evidence that, over time, we are gaining a more accurate estimate of the efficacy of CBT, which is moderately effective for the treatment of depression. Despite evidence of its efficacy for treating depression, there is a risk of deterioration and adverse events with CBT. Deterioration is broadly defined as a case in which patients or control groups score higher on symptom severity after treatment than they did at baseline [51]. A meta-analysis of the deterioration rates from 11 studies of CBT for adult depression compared with control groups [51] showed a pooled risk ratio of 0.39 for deterioration in CBT vs. control groups, leading to a risk difference of 0.05 (p = 0.98). However, only 6% of psychotherapy trials report deterioration rates. Similarly, the older literature on CBT rarely reported rates of adverse events, although adverse event reporting is increasing and becoming standard research practice [52]. However, what constitutes an adverse event in psychotherapy is still ill-defined [53,54]. A recent study provided evidence to suggest that the most common therapy-related adverse events were stigmatization and interpersonal problems [55].

3.7. Mindfulness Interventions

Mindfulness meditation is a practice that has moved from a fringe practice to a household word, including in psychotherapy contexts [4]. While there is no universally accepted definition of “mindfulness”, in modern Western interpretations it is most frequently defined as a deliberate practice involving the cultivation of moment-to-moment awareness with an attitude of nonreactivity and nonjudgment [56]. Mindfulness meditation-based interventions are commonly based on the Mindfulness-Based Stress Reduction course, which typically consists of a 9-week program of one multi-hour session per week in a small-group setting [57]. Mindfulness meditation has been shown to reduce depressive symptoms and substance use compared to active control interventions [57].

Both a systematic review and a meta-analysis investigating mindfulness interventions are reviewed here. The first, a recent systematic review of 44 meta-analyses of randomized controlled studies on mindfulness-based interventions, reports 160 effect sizes, summarizing 336 randomized controlled trials (RCTs) published between 2010 and 2019 comprising 30,483 participants [58]. This meta-analysis demonstrated that mindfulness-based interventions were superior to active controls and had a moderate effect size on depression (d = 0.54, 95% CI [0.36, 0.73], k = 16). Evidence suggests that mindfulness interventions performed better than other evidence-based modalities for preventing depressive relapses [58]. Potential sources of bias across the literature included a moderate to high risk of heterogeneity and publication bias, both of which may have inflated the effect sizes of mindfulness interventions [58]. Larger effect sizes for mindfulness interventions compared to active controls have been associated with a higher risk of bias [59]. Thus, in meta-analyses, studies without active control conditions may inflate their overall effect size and, as better-controlled studies emerge, the effect sizes of mindfulness interventions may decline. This potential trajectory may emerge for psychedelic studies as well, for similar reasons related to improving the design of controlled conditions, as the science advances.

Multiple meta-analyses demonstrate the robust, moderate effect sizes of mindfulness interventions on depression. These effects were found in the meta-analyses of pre–post studies (Hedges’ g = 0.69, 95% CI [0.52, 0.86]) [60], waitlist-control studies (Hedges’ g = 0.53, 95% CI [0.32, 0.73]) [60], and non-specific control conditions (Cohen’s d = 0.71, 95% CI [0.47, 0.96]) [58]. The meta-analysis with the smallest effect sizes evaluated improvements in depression at 8 weeks (Cohen’s d = 0.30 (95% CI [0.00, 0.59]) and at 3–6 months (Cohen’s d = 0.23 (95% CI [0.05, 0.42]) [61].

The potential for harm in mindfulness interventions appears more difficult to assess due to the small number of studies that have systematically investigated the adverse events related to mindfulness interventions [62]. Within these constraints, one meta-analysis identified 36 randomized controlled studies (25 studies using MBSR and 11 using MBCT), which found that no study reported any serious adverse events and only three studies reported six (0.49%) intervention-related adverse events from among 1231 participants.

In contrast to general adverse events, another recent study has systematically assessed the prevalence of meditation-related side effects (MRSEs), i.e., meditation effects with a negative valence and/or negative impacts [63]. While distinct from adverse events, MRSEs may be important to capture the full spectrum of risks related to mindfulness interventions. In Britton et al. (2021), 83% of the mindfulness-based program sample reported at least one MRSE. In the sample, meditation-related adverse effects occurred with negative valences (58% of the time) or a negative impact on participants’ functioning (37% of the time). These effects endured in 6% to 14% of the sample and were associated with signs of dysregulated arousal (hyperarousal and dissociation) [63].

3.8. SSRIs

SSRIs are a class of medications often used as first-line pharmacotherapy for depression [64]. SSRIs exert their effects through inhibiting the reuptake of serotonin, thereby increasing serotonin availability [64]. Typically, these medications are taken daily and may take six weeks or more to exert their effects [64]. SSRIs are indicated for major depressive disorder, generalized anxiety disorder, and post-traumatic stress disorder, among other conditions.

Three meta-analyses of SSRI pharmacotherapies for depression, conducted over the last decade, were reviewed. The primary outcome in Leucht et al. [65] was relapse between 7 and 12 months, and 65 trials were analyzed (6493 total patients). Leucht et al. [65] found a Cohen’s d = 0.32 for major depressive disorder when comparing SSRIs to a placebo [65]. In a meta-analysis of SSRIs (in 552 participants) compared to a placebo (562 participants) for the treatment of depression in a primary care setting, the chance of improvement was 1.37 with a weighted mean difference of −3.68 [66]. While not only including SSRIs but antidepressants in general, a more recent meta-analysis found an overall SMD for antidepressants versus placebo of 0.29, 95% CI [0.27, 0.31], over 390 comparisons, k = 253 studies [67]. These meta-analysis results, with moderate to strong effect sizes, indicate the efficacy of SSRIs for treating depression. However, early studies of SSRI treatments were commonly sponsored by the pharmaceutical industry, and such studies have a five times higher likelihood of finding significant effects than non-sponsored studies [68]. A re-analysis of existing datasets suggests that some of the therapeutic efficacy of antidepressants may be driven by the placebo effect and the breaking-blind problem, where the researcher and/or participant have guessed whether the participant received an active drug [69]. This analysis also indicated that antidepressants are primarily effective for the severely depressed. A systematic review and meta-analysis determined that all 131 randomized placebo-controlled trials evaluated had a high risk of bias, and none of the trials used either an active placebo or an intervention as a control [70]. This is in contrast to psychedelic studies for depression, where the majority of trials were assessed as having lower than a high risk of bias. Furthermore, this review found limited reporting of long-term outcomes, including the long-term data on adverse effects. While longer term (>1 year) data are emerging for psychedelic studies, most of the data currently available do not track participants for a year or more. Thus, despite the effect sizes seen for SSRIs and depression, bias may play a significant role in driving the results.

There is robust evidence to suggest that SSRIs cause side effects that may affect quality of life [71]. Among the most frequently reported side effects of SSRIs are gastrointestinal disturbances (>15%), anxiety (10.1% to 15.0% for Fluoxetine), and agitation (5.1% to 10.0% for Sertraline) [71]. The most troubling adverse effects during long-term SSRI therapy are sexual dysfunction (55% of patients), weight gain (mean weight gains of 15 lb./6.75 kg for sertraline, 21 lb./9.45 kg for fluoxetine, and 24 lb./10.80 kg for paroxetine after 6 to 12 months of therapy), and sleep disturbance [71]. The prevalence of these side effects may be underestimated but may often lead to a discontinuation of the medication [72]. Furthermore, the Food and Drug Administration issued a black box warning for SSRIs in 2004 due to a potential risk of increased suicidality in populations up to the age of 25 [64]. Other serious but rare side effects include cardiac complications such as coagulopathy and arrhythmia [72].

3.9. Ketamine

Ketamine is a glutamate receptor antagonist with dissociative and anesthetic properties. Ketamine has strongly mind-altering effects on consciousness while being subjectively distinct from classic psychedelics [73]. Recent research has found that ketamine is efficacious in treating depression, and, based on consistent results, the U.S. Food and Drug Administration (FDA) has approved the clinical administration of intranasal esketamine (the S-(+) enantiomer of ketamine) for treatment-resistant depression (U.S. Food and Drug Administration, 2019).

In a review of nine studies on the effects of ketamine on depressive symptoms (including 192 patients with major depressive disorder and 34 patients with bipolar depression), depression scores were significantly decreased in the ketamine groups compared to those in the control groups (SMD = −0.99, 95% CI [−1.23, −0.75]) [74]. A systematic review by Coyle and Laws of 21 studies, including 437 participants receiving ketamine, showed large and significant overall pooled effect sizes for various time points, with a Hedges’ g = −1.67 (95% CI [−2.85, −0.49]) 12–14 days after ketamine infusions [75]. A systematic review of 49 randomized controlled trials of ketamine for major depression found that most studies had a low or unclear risk of bias [76], which is similar to the risk of bias analysis in this manuscript. The effect sizes of ketamine may have been inflated due to publication bias and underpowered trials [76]. Small sample sizes for both ketamine and psychedelic studies may lead to smaller effect sizes over time when larger trials are conducted.

The adverse effects of ketamine comprise several side effects that are commonly transient and disappear completely within 60 min after the end of administration. Among them are general psychiatric symptoms (e.g., thought disorder, anxiety–depression, hostility–suspiciousness), psychotomimetic symptoms, dissociative symptoms, perceptual disturbances, and transient physical adverse effects (such as light-headedness, headache, nausea, diplopia, drowsiness, and dizziness) [77]. An analysis of the pooled data from three clinical trials showed that 4 of 205 infusions (1.95%) were discontinued due to adverse events. There is also significant evidence that long-term ketamine use can be addictive, causing craving and difficulty quitting after increased tolerance [78,79].

3.10. Summary of the Comparisons between Treatment Modalities

All treatment modalities evaluated demonstrated their efficacy in treating depression, with a range of effect sizes. The largest effect sizes found for psychedelic treatments, as reported by Ko et al., ranged from SMD = 1.36 to SMD = 3.12 for 1-day, 1-week, and 3–5-week follow-up treatments [33]. These are followed by cognitive behavioral therapy (Hedges’ g = 1.37) [48,50] and ketamine (Hedges’ g = −1.67 after 12–14 days) [75]. Mindfulness interventions showed an effect size of Hedges’ g = 0.53 at the end of the intervention [60]. The smallest effect sizes were found for SSRIs, with a Cohen’s d = 0.32 at the end of treatment [65]. However, we reiterate that a clean quantitative comparison is not possible between these various treatments, so we urge caution in interpretating these effect sizes as the control conditions varied a great deal between the studies on different treatments.

A trend towards smaller effect sizes is seen as time progresses and larger studies are conducted, in each of these research areas, on treatments. Again, due to the significant heterogeneity within the scientific literature, it was not possible to make formal comparisons across different treatments. Given the importance of making direct comparisons, this is an area for improvement and a future direction for the field. Evaluating between-treatment comparisons may be especially important for psychedelics as an emerging yet potentially controversial treatment modality. This informal comparison of effect sizes for the treatment of depression between psychedelic treatments, CBT, mindfulness interventions, SSRIs, and ketamine is summarized in

Table 2. Overview of some recent meta-analyses and the effect sizes of various forms of treatments for depression.

Psychedelic Treatments	Cognitive-Behavioral Therapy	Mindfulness Interventions	SSRIs	Ketamine
Cohen’s d = 1.46 at day 7 [31]	Hedges’ g = 0.53 at the end of the treatment [44]	Cohen’s d = 0.59 [58]	Cohen’s d = 0.32 at the end of the treatment [65]	Hedges’ g = 1.29 (after 4 h) [75]
Cohen’s d = 0.92 at day 14 [31]	Hedges’ g = 1.37 at the end of the treatment [48]	Hedges’ g = 0.53 at the end of the intervention [60]	Cohen’s d = 0.29 [67] a	Hedges’ g = 1.24 (after 24 h) [75]
Cohen’s d = 2.21 at week 4 [31]		Cohen’s d = 0.30 at 8 weeks [61]		Hedges’ g = 1.06 (after 7 days) [75]
Cohen’s d = 2.7 at week 8 [31]		Cohen’s d = 0.23 at 3–6 months [61]		Hedges’ g = 1.67 (after 12–14 days) [75]

^a Munkholm et al. [67] included in their meta-analysis studies the testing of a variety of antidepressants including, but not limited to, SSRIs.

.

The type, frequency, and severity of adverse events varies significantly by treatment modality. The quality of adverse events differs significantly, where some adverse events are more psychological (e.g., anxiety and psychedelics) and some of a more physical nature (e.g., weight gain and SSRIs). The nature of the therapy influences the nature of the adverse events/side effects, and some modalities may carry a social stigma that negatively influences the participant. Adverse effects from mindfulness interventions differ from those arising during psychedelic treatments, e.g., 6–14% experienced mindfulness-related transient dysregulated arousal [63] vs. a range of 26–100% experiencing psilocybin-related transient anxiety [16,22]. The duration of the adverse event also varies, from under a day for psychedelics and ketamine (which are administered only once or several times) to over a year for SSRIs (which are taken daily). Over time, the frequency and type of adverse events seen for each modality are progressively better described and becoming more commonly known.

4. Discussion

To better balance the expectations around psychedelic treatments with the available evidence and compare it to existing treatments, we have provided a review of the evidence of the efficacy and risks, as well as analyzed the extent of bias, in a subset of psychedelic studies. In general, at present, psychedelics demonstrate superior effect sizes and a relatively low incidence of adverse events compared to other evidence-based treatments. Most psychedelic trials have a low overall risk of bias, but there are still notable biases across trials in terms of outcome reporting and deviations from intended interventions (primarily due to blinding concerns). However, psychedelics are the newest of these treatments; many psychedelic trials are small, have unmasked control conditions, and use highly screened study samples (for a critical review, see van Elk and Fried, 2023) [80]. The early state of this research calls for caution about predicting the long-term reproducibility and generalizability of psychedelic treatments for depression. Based on our review of other interventions that received a great deal of hype, as psychedelic research progresses, its effect sizes may also decrease while its adverse events will likely increase due to improvements in study design, generalizability, larger sample sizes, and more reporting. However, based on the current available empirical evidence no, such trend can be observed at this point (see Figure 2). As the literature on psychedelics is still emerging, its trends over time may change the status of the comparison between psychedelic effects and adverse events relative to other treatment modalities. Balanced science communication is critical for promoting public understanding of the risks and benefits of psychedelics and avoiding overly pessimistic or overly optimistic predictions that are not grounded in evidence.

Why Might Effect Sizes Decrease over Time?

There is a general trend of effect sizes being higher in earlier studies with small sample sizes [81,82]. The effect sizes of all psychiatric treatments tend to decrease over time: for CBT, meta-regressions on 70 studies from 1977 to 2014 showed temporal trends indicating that its effect sizes have declined linearly, as assessed through patients’ and clinicians’ ratings and remission rates [45,48,50]. Multiple factors may lead to declining effect sizes, likely including regression to the mean, the tendency to conduct larger studies over time, publication bias, different study designs, and evolving research standards. Blinding, expectancy bias, a lack of long-term follow-up, and the placebo effect are discussed in greater detail below.

5. Study Design Considerations

5.1. Blinding and Expectancy Confounders in Psychedelic RCTs

Blinding is a particularly important reason to be cautious about the interpretation of psychedelic trial effect sizes. While psychedelic RCTs have generally shown promising results, with large effect sizes reported, Muthukumaraswamy et al. argue that the treatment effect sizes in psychedelic RCTs are likely overestimated due to the de-blinding of participants and high levels of response expectancy [83]. This effect is especially increased in participants with previous psychedelic drug experience, rendering selection bias especially relevant in psychedelic clinical trials [25]. These concerns are not unique to psychedelics but also occur in research with CBT, mindfulness, SSRIs, and ketamine. As the field of psychedelic research develops a set of standardized procedures for dosing and control conditions, including more effective blinding procedures, the effect sizes of psychedelic treatments will likely further decrease. It is important to emphasize to the public that the field is still developing and experimenting with treatment modalities, given the likelihood that increased harmonization across the field may impact these results over time.

5.2. Lack of Long-Term Follow-Up Measurements

The lack of long-term follow-up measurements in most of the studies cited is another concern. It is estimated that around 53% of prevalent cases of untreated depression remit spontaneously within one year [84]. Without longer term follow-up measurements, it remains an open question whether the effects of psychedelic-assisted psychotherapy and other treatments are only short-term or persistent. Follow-up measurements of over a year are required to test the long-term treatment effects of psychedelics on depressive symptoms.

5.3. Placebo Effect

Another concern regarding the efficacy of psychedelics relates to the placebo effect. A meta-analysis of 96 antidepressant trials, encompassing 9566 patients under a placebo condition, demonstrated an effect size of d = 1.69 (95% CI [1.54, 1.85]) at the primary outcome follow-up time point [85]. These base rates of placebo effect sizes are a useful context for understanding within-group effect size estimates. For example, in Palhano-Fontes et al. [41], the placebo effect size is d = 0.46, 95% CI [−0.27, 1.18]; lower than is typical in other treatment-resistant depression trials. This substantial difference in placebo effect sizes between this psychedelic study and other treatments (d = 0.46 vs. g = 1.05) may reveal a nocebo effect in the control group participants in their psychedelic trial (i.e., patients being disappointed they did not receive the ‘active’ treatment). This might be another indicator of the inadequacy of control conditions in psychedelic depression trials.

6. Communicating Psychedelic Research Results

6.1. Overestimations in Published Effect Sizes Due to Publication Bias

Published estimates of effect sizes may be too large due to publication bias [86,87,88]. To address this issue of exaggerated effect sizes, Gelman and Carlin have provided a method to formally assess publication bias through a set of statistical calculations that estimate what could happen under hypothetical replications of study designs based on external information [89]. Their probability model is based on assessing the probability of a replicated effect size estimate being in the wrong direction (Type S error) and of the factor to which an effect might be exaggerated (Type M error or exaggeration ratio) [89]. With the current state of psychedelic science, there is too little data to formally assess these estimates of publication bias. However, with the rapid increase in the number of publications in the field, the calculation of Type S errors and Type M errors may soon be possible and able to provide informative insights for interpreting the effect sizes of current psychedelic trials.

6.2. Science Communication about Benefits and Risks

Research integrity is enhanced through responsible science communication [90]. Responsible science communication promotes public trust in research and promotes evidence-based science policy. Science communication plays an essential role in characterizing public expectations regarding the relative risks and benefits and therapeutic uses of psychedelic-assisted treatments. This expectancy bias may directly affect who is most likely to participate in psychedelic research and influence trial results [91]. For decades, overly negative communication has dominated, with a bias towards the potential risks of psychedelics, resulting in anti-psychedelic policies. With the recent resurgence of psychedelic research, this tendency has shifted towards more positive communication, with a bias towards the potential therapeutic uses of psychedelics [1,30]. Critical claims regarding the efficacy of psychedelic treatments are gaining ground, both in the popular press (e.g., blog posts such as Jesse Singal’s July 2023 article ‘So it looks like psychedelics research is a big mess’) and in the research literature (e.g., the Van Elk and Fried ‘History repeating: A roadmap to address common problems in psychedelic science’). However, currently, the majority of mainstream outlets remain on the positive hype spectrum, whereas new negative articles are still limited to the cultural vanguard. A swing back to the other end of the spectrum, with extremely negative and fearful media messaging, as seen in decades past, is also possible. For reasonable public understanding and informing policy and decision making, as well as for the safe potential integration of psychedelics into therapeutic contexts, both positive and negative extremes are best avoided to the extent that they are out of keeping with the empirical evidence. Therefore, it is important to provide a transparent and balanced view of the potential benefits and the potential risks to reduce the harm of psychedelics and to communicate about and continue to investigate their potential benefits. Recalibrating public expectations is fundamentally the responsibility of the scientists involved in conducting these studies, but also extends to other stakeholders involved in science communication.

7. Conclusions

Depression is a debilitating disease that causes a massive amount of suffering and disability around the world, and new interventions are urgently needed. Psychedelics may provide a novel treatment or at least provide an avenue for productive research. Over the past few decades, psychedelics have attracted both overly negative alarmist statements about their risks and overly positive statements about their benefits, a dynamic that continues with frequent overly extreme statements coming in both directions. When compared to other treatments of mood disorders, in terms of their risks and benefits, psychedelics compare favorably. Yet, larger clinical trials with more heterogeneous samples and active controls are needed. Psychedelics may not be exceptional in their efficacy when considering that early unmasked studies with small sample sizes tend to have larger effect sizes. These factors will likely result in smaller effect sizes for the psychedelic studies in more rigorous trials in the future. While it is yet to be seen whether psychedelic treatments are superior to existing treatments, the effect sizes and risks reported so far are a cause for cautious optimism and further curiosity about the therapeutic potential of psychedelics.