Comparative Efficacy of Seven Psychotherapeutic Interventions for Patients with Depression: A Network Meta-Analysis

Summary

Background:
Previous meta-analyses comparing the efficacy of psychotherapeutic interventions for depression were clouded by a limited number of within-study treatment comparisons. This study used network meta-analysis, a novel methodological approach that integrates direct and indirect evidence from randomised controlled studies, to re-examine the comparative efficacy of seven psychotherapeutic interventions for adult depression.

Methods and Findings:
We conducted systematic literature searches in PubMed, PsycINFO, and Embase up to November 2012, and identified additional studies through earlier meta-analyses and the references of included studies. We identified 198 studies, including 15,118 adult patients with depression, and coded moderator variables. Each of the seven psychotherapeutic interventions was superior to a waitlist control condition with moderate to large effects (range d = −0.62 to d = −0.92). Relative effects of different psychotherapeutic interventions on depressive symptoms were absent to small (range d = 0.01 to d = −0.30). Interpersonal therapy was significantly more effective than supportive therapy (d = −0.30, 95% credibility interval [CrI] [−0.54 to −0.05]). Moderator analysis showed that patient characteristics had no influence on treatment effects, but identified aspects of study quality and sample size as effect modifiers. Smaller effects were found in studies of at least moderate (Δd = 0.29 [−0.01 to 0.58]; p = 0.063) and large size (Δd = 0.33 [0.08 to 0.61]; p = 0.012) and those that had adequate outcome assessment (Δd = 0.38 [−0.06 to 0.87]; p = 0.100). Stepwise restriction of analyses by sample size showed robust effects for cognitive-behavioural therapy, interpersonal therapy, and problem-solving therapy (all d>0.46) compared to waitlist. Empirical evidence from large studies was unavailable or limited for other psychotherapeutic interventions.

Conclusions:
Overall our results are consistent with the notion that different psychotherapeutic interventions for depression have comparable benefits. However, the robustness of the evidence varies considerably between different psychotherapeutic treatments.

Please see later in the article for the Editors' Summary

Introduction

Depressive disorders are very common; about one-fifth of the population will be affected in their lifetime in high-income countries [1],[2]. Major depression ranks fourth on the list of disorders with the highest burden of disease worldwide, and it is expected to be ranked first in high-income countries by 2030 [3]. Depressive disorders can lower the ability to function in work and daily life [4].

Various psychotherapeutic interventions have been developed to treat depression, including cognitive-behavioural, interpersonal, humanistic, and psychodynamic approaches. There is substantial evidence that many types of psychotherapeutic interventions have a moderate to large effect (d = −0.66, 95% CI [−0.73 to −0.60] [5]), when compared to control conditions. Substantial effects compared to control conditions have been documented specifically for behavioural activation [6],[7], cognitive-behavioural therapy [8],[9], interpersonal psychotherapy [10],[11], problem-solving therapy [12], and short term psychodynamic therapy [13],[14]. Different modalities and forms of psychotherapeutic intervention have similar benefits and no difference has yet been found between individual or group treatment formats [15],[16]. Similarly, it seems to make little difference whether psychotherapeutic interventions on depression are provided face-to-face, via telephone, or on the Internet [17].

While there is broad consensus that psychotherapeutic interventions are beneficial for depressed patients, there is an ongoing debate about the comparative efficacy of different psychotherapeutic interventions. Meta-analyses of within-study comparisons of different types of psychotherapeutic interventions for depression are varied in their conclusions. Two meta-analyses found cognitive-behavioural therapy to be more effective than non-cognitive-behavioural interventions [9],[18] in direct comparisons. In contrast, a meta-analytic comparison of cognitive-behavioural therapy and short-term psychodynamic therapy for depression found no significant differences in efficacy between these interventions [14].

In the most comprehensive investigation of the relative efficacy of psychotherapeutic interventions for depression to date, Cuijpers and colleagues [19] synthesized 53 studies containing within-study comparisons of different psychotherapeutic interventions. The effects of cognitive-behavioural therapy, psychodynamic therapy, behavioural activation treatment, problem-solving therapy, and social skills training did not differ significantly from other psychotherapeutic interventions. Interpersonal therapy was found to be somewhat more effective (d = −0.21, CI 95% [−0.42 to −0.01]), and supportive counselling was found to be somewhat less effective than other psychotherapeutic interventions (d = 0.17, CI 95% [−0.32 to −0.03]). However, in this meta-analysis, the relative effects of psychotherapeutic interventions were established by pooling all studies that compared the respective treatment to any other intervention. This method complicates the interpretation of the results because the pooled comparator interventions compile different psychotherapeutic interventions. However, this approach had to be used since the number of available within-study comparisons of two specific interventions (e.g., interpersonal psychotherapy versus psychodynamic therapy) was limited.

We employed network meta-analysis—a methodological approach that allows for the comparison of a variety of psychotherapeutic interventions head to head or with a control condition [20]—to overcome the restrictions of limited available comparisons and the problem of lumping together different psychotherapeutic interventions. In network meta-analyses, the information available from within-study comparisons of treatment A and treatment B is combined with indirect comparisons of A and B derived from studies that compare either of the two treatments with a common comparator C (either a third psychotherapeutic intervention or a control condition). Network meta-analysis has already been used to investigate pharmacological treatments for depression [21] and mania [22], but has not yet been used to our knowledge in psychotherapy research.

The quality of primary studies potentially threatens the validity of meta-analyses [23]–[25]. Inadequate concealment of allocation, exclusion of patients from the analysis (i.e., if analysis is not intention-to-treat), and lack of blinding of outcome assessors [26]–[28] are known to have a biasing effect. Furthermore, evidence suggests that treatment efficacy is overestimated in studies with small sample size [29].

We aimed to re-examine the comparative efficacy of different psychotherapeutic interventions for adult depression by using network meta-analysis to integrate all available information from randomized controlled studies. We also wanted to assess the influence of study quality, sample size, and clinical characteristics on effect estimates (for a protocol of the network meta-analysis see [30]).

Methods

Study Selection and Inclusion Criteria

We used a database of randomized controlled trials on the efficacy of psychotherapeutic interventions of adult depression (www.evidencebasedpsychotherapies.org), which has been described in detail elsewhere [5],[31]. The database was developed through a comprehensive literature search (from 1966 to November 1, 2012) in PubMed, PsycINFO, Embase, and the Cochrane Central Register of Controlled Trials. Controlled vocabulary and text words related to different psychotherapeutic interventions and depression were used in the search. In the WHO Afro Library we used the text word depression for our search. We obtained all primary studies from 42 meta-analyses of psychotherapeutic interventions for depression [32] and checked the references of the included studies.

We included in this meta-analysis only studies with a randomised design. Studies of adults with a depressive disorder, or with elevated levels of depressive symptoms were required to compare the effects of a psychotherapeutic intervention to a control condition (i.e., waitlist, usual care, or placebo), or to another psychotherapeutic treatment. Psychotherapeutic interventions were defined as interventions with a primary focus on language-based communication between a patient and a therapist, or as bibliotherapy supported by a therapist. We included psychotherapeutic interventions from seven pre-specified categories (see below), defined for a previous meta-analysis [19]. No restrictions were made based on format (individual or group) or treatment setting (face to face, telephone, or internet). Combinations of psychotherapeutic interventions with pharmacotherapy or other non-psychotherapeutic interventions (e.g., managed care interventions and disease management programmes) were excluded. Eligible control conditions were waitlist, usual care, and (psychological or pill) placebo. Comparisons of a psychotherapeutic intervention with pharmacotherapy or other non-psychotherapeutic interventions were excluded. We also excluded studies on maintenance treatment and relapse prevention, and studies that included participants who were anxious but not depressed at the time-point of inclusion. Studies were eligible irrespective of the inclusion of patients with comorbid general medical or psychiatric disorders. No language restrictions and restrictions on publication type were applied.

Psychotherapeutic Interventions

Psychotherapeutic interventions were coded according to type of intervention and treatment format. Based on an expert taxonomy of psychotherapy for depression [19], we classified psychotherapy into seven different types: interpersonal therapy, behavioural activation, cognitive-behavioural therapy, problem solving therapy, social skills training, psychodynamic therapy, and supportive counselling. A description of each type of intervention is presented in Table 1. We coded further, on the basis of whether a psychotherapeutic intervention was delivered individually, face-to-face, or in a different setting (e.g., group psychotherapeutic intervention, internet based individual treatment, bibliotherapy). The number of sessions was used to rate the treatment dose as low (six or fewer sessions) or high (more than six sessions).

Study Characteristics

We coded inclusion in the study of depressed adults in general, as opposed to specific populations (elderly people, women with postpartum depression, patients with somatic illness, or student populations). We used the procedure for diagnosing depressive symptoms to distinguish trials of patients with a formal diagnosis (e.g., according to DSM) from trials of patients with a probable diagnosis of depression (e.g., by using screenings).

Concealment of allocation, outcome assessment, and type of analysis were coded as components of study quality. Concealment of allocation was considered adequate if the investigators responsible for the selection of patients could not foresee a patient's allocation (e.g., by using external randomisation, or sealed, opaque, and sequentially numbered assignment envelopes). Any procedures based on predictable generation of allocation sequences (e.g., alternation), or potentially transparent attempts to conceal allocation, such as non-opaque envelopes, were considered inadequate. Outcome assessment was considered adequate if self-report measures or outcome assessors were blinded towards patients' treatment condition. Outcome assessment was regarded as inadequate if clinical raters were not blind. Because blinding therapists and patients is not possible in psychotherapy trials, we did not assess blinding towards treatment delivery. Type of analysis was considered adequate if all randomised patients were included (i.e., an intention-to-treat approach was used), and inadequate if some randomised patients were excluded from the analysis, or if the analysis was based only on those who completed treatment. Concealment of allocation, outcome assessment, and type of analysis were regarded as inadequate in cases where specific information was unavailable.

Studies were classified in three groups according to average number of patients per condition. We distinguished studies with <25 patients per group (small), from those with 25 to <50 patient per group (moderate), and ≥50 patients per group (large). All coding was done in duplicate by two independent raters. Disagreements were resolved by consensus.

Statistical Analysis

A Cohen's d effect size with Hedges's correction for small sample bias was calculated for all comparisons contained in the studies [33]. A d = 0.20 represented a small, 0.50 a moderate, and 0.80 a large difference between interventions [34]. If means (Ms) and standard deviations (SDs) were not provided, we calculated them from standard errors, confidence intervals, or other statistical indices as described elsewhere [35],[36]. Depressive symptoms at post treatment were used as outcome. When the results of post treatment measurement were not reported, we extracted the results of the earliest follow-up measurement. Results from intention-to-treat analysis were preferred over results from completer analyses. All standardised self-report and observer-rated instruments measuring depressive symptoms were extracted. If results for more than one instrument were reported, we calculated the mean of the effect sizes, so that each comparison of conditions contributed only one effect size to the analyses.

To allow comparisons of conventional meta-analyses with results from subsequent network meta-analyses, we first calculated pair-wise meta-analyses of all within-study comparisons available for each contrast, using a Bayesian random effects model based on minimally informative prior distributions [37]. For the network meta-analysis, we used an extension of this model to compare various interventions [20],[38]. The model allows comparison of all conditions evaluated in a connected network of studies, and accounts for multiple comparisons from one study. The code is available in Text S1 in Supporting Information S1.

Relative effect sizes between different psychotherapeutic interventions and between psychotherapeutic interventions and control groups were estimated from the median of the posterior distribution. Corresponding 95% credibility intervals (CrI) were estimated from the 2.5th and 97.5th percentiles of the posterior distribution. In the presence of minimally informative priors, CrIs can be interpreted similarly to confidence intervals, and conventional levels of statistical significance at a two-sided p<0.05 can be assumed if 95% CrIs do not include 0.

The variance estimate τ² as a measure of between-study heterogeneity was derived from the median observed in the posterior distribution. Tau (square root of τ²) represents the standard deviation of the underlying distribution from which the included trials are assumed to be a random sample from. On the basis of our definition of small, moderate, and large differences between interventions we interpreted τ² as follows: τ² = (0.2/2)² = 0.01 was considered to represent low heterogeneity, τ² = 0.0625 [(0.5/2)²] moderate heterogeneity, and τ² = 0.16 [(0.8/2)²] high heterogeneity between studies. Differences between direct estimates (e.g., based on all available within-study comparisons) and indirect estimates (e.g., via shared comparators) were calculated to estimate inconsistency as previously described [39].

To determine if the results were affected by treatment format, target group, diagnosis, treatment dose, concealment of allocation, type of analysis, outcome assessment, and study size, we performed network meta-analyses stratified by these study characteristics. Two different stratifications were made for sample size based on the cut-offs defined above. The first stratification compared results from small studies to results from moderate and large studies. The second stratification compared the results from small and moderate studies to the results of large studies to analyse the effect of restricting studies according to sample size. To determine whether sample size was related to study quality, we used sample size as an ordinal variable (small, moderate, and large) and study quality indicators as dichotomous items and calculated Somer's D—a correlation coefficient for a dichotomous and an ordinal variable. We quantified the size of the interaction for all stratified analyses and provided corresponding p-values. Given the scarcity of the data, we were only able to run a model that assumed that the interaction effect across all comparisons (mean bias) was the same size.

For all Bayesian analyses, Markov-chain-Monte-Carlo methods were used. Initially we had run analyses with three chains to determine the burn-in.

Convergence of Markov chains was considered achieved when plots of the Gelman-Rubin statistics indicated that widths of pooled runs and individual runs stabilised around the same value and their ratio was around one [40]. Finally, all analyses were run using only one chain. We carried out 100,000 iterations. The first 50,000 were discarded after the burn-in period and estimates were based on the subsequent 50,000. We used Stata release 11 (StataCorp) and WinBUGS version 1.4 (MRC Biostatistics Unit 2007) for all analyses.

Results

We analysed 198 studies including 433 conditions (psychotherapeutic interventions or control) and 15,118 patients (see the flow chart in Figure 1; references of included studies are available in the Text S2 in Supporting Information S1). Sixty-three studies contained at least one comparison between two psychological interventions, and 162 studies contained comparisons of psychotherapeutic interventions with a control condition. Overall, 9,314 patients were randomised to psychotherapeutic interventions (cognitive-behavioural therapy n = 5,378, supportive counselling n = 1,125, interpersonal therapy n = 992, problem solving therapy n = 852, psychodynamic therapy n = 440, behavioural activation n = 431, social skills training n = 96). Another 5,805 patients were randomised to control conditions (waitlist, usual care, or placebo). The median number of patients included per treatment condition was 23 (range 5 to 418). The median publication year of studies was 2003 (range 1975 to 2012). Further descriptive information about the included studies is given in Table 2. Ninety-four studies (47%) investigated adults with depression, while 104 studies (53%) investigated depression in more specific patient populations. Of all psychotherapeutic interventions, cognitive-behavioural therapy was the intervention that was most often investigated (139 studies, 70%), while social skills training was investigated least often (seven studies, 4%). The most common control condition was waitlist (75 studies, 38%). More than half of the studies investigated psychotherapeutic interventions in an individual, face-to-face setting. Most studies were conducted in the United States (115 studies, 58%). Four studies were published in German and two studies in Spanish. Descriptive information and the coding of variables for each of the 198 included studies are provided in Tables S1 and S2 in Supporting Information S1.

Meta-Analysis of Within-Study Comparisons

The results of conventional meta-analyses (i.e., based on available within-study comparisons) for each pair of conditions are shown in the upper triangle in Table 3. Given that we had ten conditions (seven psychotherapeutic interventions and three control conditions), 45 contrasts were possible. Of 21 possible contrasts of specific psychotherapeutic interventions, there was no within-study comparison available for six of the contrasts. There was little evidence for superiority or inferiority of any of the psychotherapeutic interventions in the remaining 15 contrasts (all 95% CrIs included 0). Of 21 possible contrasts between psychotherapeutic interventions and control conditions, no within-study comparison was available for five contrasts. Supportive counselling, social skills training, problem solving, cognitive behavioural therapy, and behavioural activation were more effective than waitlist. Effect sizes were moderate to large (range d = −0.56 to d = −1.23) and significant (the 95% CrIs did not include 0). Large but non-significant effects were found for psychodynamic therapy and interpersonal therapy vis-à-vis waitlist (the 95% CrIs included 0).

Network Meta-Analysis

All available within-study comparisons were then synthesized with network meta-analysis (see lower triangle in Table 3). Most relative effects of psychotherapeutic interventions were absent to small (range d = 0.01 to d = −0.30) and all but one failed to reach statistical significance (the 95% CrIs did include 0). Interpersonal therapy was significantly superior to supportive therapy (d = −0.30, 95% CrI [−0.54 to −0.05]). All seven psychotherapeutic interventions were more beneficial than waitlist, with effect sizes between d = −0.62 and d = −0.92. Compared to usual care, all psychotherapeutic interventions except for social skills training were more beneficial, with effect sizes between d = −0.29 and d = −0.59. Similar results were found in comparison to placebo. Heterogeneity between effect sizes was low (τ² = 0.010) and suggested good interpretability of the results. There was no evidence that direct and indirect effects were inconsistent (95% CrIs of differences between direct and indirect estimates included 0, see Figure S1 in Supporting Information S1).

Moderator Analyses

We further explored the influence of several potential moderator variables. Table 4 presents the results of network-meta-analyses stratified by different study characteristics. There was no evidence to suggest effect modification for patient population and intervention format (Δd = 0.08, 95% CrI [−0.34 to 0.18]; p = 0.54 and Δd = −0.07, 95% CrI [−0.37 to 0.22]; p = 0.65, respectively). Thus, psychotherapeutic interventions appeared comparably effective in different populations of depressed patients, and when provided in a face-to-face and individual setting, compared to other settings. The effectiveness of psychotherapeutic interventions was the same in studies in which patients were formally diagnosed with depression and studies in which patients were probably depressed (Δd = −0.11, 95% CrI [−0.39 to 0.18]; p = 0.45). However, the effectiveness of psychotherapy was affected by the treatment dose. Unexpectedly, low-dose treatments were more effective in a stratified analysis (Δd = 0.33, 95% CrI [−0.01 to 0.65]; p = 0.06). However, a linear regression analysis did not confirm this initial finding (p = 0.63).

Of the three study quality indicators, one was related to treatment effects in trend. Treatment effects were smaller in studies where outcome was assessed with self-report measures or blinded observers, compared to non-blind observers (Δd = 0.38, 95% CrI [−0.06 to 0.87]; p = 0.10). Also, there was a hint towards smaller effects in studies with adequately concealed randomisation sequences, compared to studies with inadequate or unclear concealment (Δd = 0.19, 95% CrI [−0.08 to 0.47]; p = 0.16). No significant difference was found between studies in which analysis was to intention-to-treat, compared to studies with more complete analysis (Δd = 0.13, 95% CrI [−0.14 to 0.39]; p = 0.35).

Treatment effects were lower in at least moderately sized studies (Δd = 0.29, 95% CrI [−0.01 to 0.58]; p = 0.06) and large studies (Δd = 0.33, 95% CrI [0.08 to 0.61]; p = 0.01). Consistent with this result, we found a significant funnel plot asymmetry (Egger's test p<0.001; see Figure S2 in Supporting Information S1), suggesting a linear relation between the standard error as a proxy for study size and effect size (i.e., small studies showed larger effects). Because sample size is discussed as a proxy for study quality, we also investigated associations between the two variables. We found that all three aspects of study quality were significantly more often fulfilled in larger studies (all p-values <0.001, see Table S3 in Supporting Information S1). No clear difference between study effects was found when we contrasted studies published before 2000 and studies published in 2000 or after (Δd = 0.14, 95% CrI [−0.15 to 0.42]; p = 0.36).

Stepwise Restriction of Network Meta-Analyses According to Sample Size

The moderator analyses strongly suggested that the studies included in the present meta-analysis are prone to bias. In order to reduce overestimation of effects and test the robustness of the findings, we conducted further analyses restricted to studies with moderate sample size and studies with large sample size. Since there was an association between sample size and study quality, restricting the analysis to larger studies might reduce bias owing to low study quality. Figures 2 and 3 present networks of evidence and forest plots for all (A1 and A2, cf. lower triangle in Table 2), at least moderately sized (B1 and B2), and large (C1 and C2) trials.

The networks of evidence (A1, B1, and C1) reflect the available within-study comparisons for each contrast between conditions and the number of patients investigated for each condition (Figure 2). Restricting analysis to studies with moderate and large sample sizes reduced the number of psychotherapeutic interventions that could be adequately represented in the network. In the network of studies with at least moderate sample size, no studies were available for social skills training, and only two studies were available for behavioural activation. In the network based on large studies, no studies were available for behavioural activation and social skills training, and only one study was available for psychodynamic therapy. Thus, the influence of sample size on estimated treatment efficacy could not be determined for social skills training, and the evidence base for behavioural activation and psychodynamic therapy was limited.

The forest plots in Figure 3 (A2, B2, and C2) show the relative effect of psychotherapeutic interventions compared with waitlist. In the network meta-analysis restricted to at least moderately sized trials, significant moderate to large effects against waitlist were found for supportive counselling, psychodynamic therapy, problem solving therapy, cognitive behavioural therapy, behavioural activation and interpersonal therapy (all 95% CrIs did not include 0). The effects of all these psychotherapeutic interventions were reduced in comparison to the network meta-analysis based on all studies (A2; range of differences between estimates for treatments Δd = 0.07 to Δd = 0.17). The only significant difference between treatments was that interpersonal therapy appeared to be more effective than supportive counselling (Δd = −0.34, 95% CrI [−0.63 to −0.03]). As mentioned, the effect estimate for behavioural activation was based only on two studies and thus should be interpreted with caution. All relative effects found in the network of at least moderately sized studies are shown in the upper triangle of Table S4 in Supporting Information S1.

In the network meta-analysis restricted to large studies problem solving therapy, cognitive behavioural therapy, and interpersonal therapy showed moderate effects compared to waitlist (95% CrIs did not include 0). Supportive counselling and psychodynamic therapy were not significantly more effective than waitlist (95% CrIs included 0). The effects of all treatments in the network were again reduced, compared to the network of at least moderately sized studies (range of differences between estimates for treatments Δd = 0.11 to Δd = 0.48). The effect size difference between interpersonal therapy and supportive counselling was no longer significant in this analysis (Δd = −0.45, 95% CrI [−0.89 to 0.02]). As mentioned above, the effect of psychodynamic therapy is based on only one study and thus should be interpreted with caution. All relative effects found in the network large studies are shown in the lower triangle in Table S4 in Supporting Information S1.

Heterogeneity between effect sizes in the network meta-analyses restricted to ≥25 and ≥50 patients per condition was low (τ² = 0.013 and τ² = 0.025, respectively) and suggested good interpretability of the results. Again in neither network meta-analyses did we find evidence for inconsistency between direct and indirect estimates (95% CrIs of differences between direct and indirect estimates included 0, see Figures S3 and S4 in Supporting Information S1).

Discussion

After synthesizing 198 randomized controlled trials, we found evidence that most of the seven psychotherapeutic interventions under investigation have comparable effects on depressive symptoms and achieve moderate to large effects vis-à-vis waitlist. The only significant difference was that interpersonal therapy was somewhat more beneficial than supportive counselling. All seven psychotherapeutic interventions achieved a small to moderate effect compared to usual care. These effects were statistically significant for six psychotherapeutic interventions, and insignificant for social skills training.

Moderator analyses suggested that psychotherapeutic interventions work similarly well in different populations of depressed patients and in different settings. We found evidence that study quality influences treatment effects. Studies using non-blind outcome assessors found significantly larger effects than studies that used self-report measures or blinded observers. Also, there was a trend towards larger effects in studies with inadequately concealed randomisation sequence compared to studies with adequate concealment of allocation. This network meta-analysis also found a decrease in treatment effects in studies with a larger sample size. Treatment effects were about a moderate effect size lower in studies that had 50 or more patients per condition.

To adjust for small study effects, we conducted additional analyses restricted to at least moderately sized and large studies. Adjusting for small study effects resulted in divergent conclusions about the robustness of different psychotherapeutic approaches. Although somewhat reduced in magnitude, problem solving therapy, cognitive behavioural therapy, and interpersonal therapy showed significant moderate effects vis-à-vis waitlist in all restricted analyses. For supportive counselling and psychodynamic therapy, significant effects against waitlist were found in the network of at least moderately sized trials, but not in the network of large trials. Behavioural activation was more effective than waitlist in the network of at least moderately sized studies; however no information from large studies was available. Social skills training was only investigated in small studies.

The diminished effects found for psychodynamic therapy and supportive therapy in the network of large studies need further discussion. Only one large study was available for psychodynamic therapy, in which five sessions of psychodynamic therapy were compared to usual care in patients with chronic depression [41]. From a clinical perspective, a 3-fold treatment dose might be the lower limit for such patients. This was shown in the study on the cognitive behavioural-analysis system of psychotherapy (CBASP), which uses about 18 sessions [42]. The evidence base for supportive counselling was broader and our results suggest that supportive counselling might be a less adequate treatment for depression. However, there have been criticisms of the implementation of supportive counselling in psychotherapy outcome research. Conceptual restrictions might limit its effectiveness (i.e., in many cases supportive counselling was not intended to be therapeutic [43],[44]). After controlling for researcher allegiance, a recent meta-analysis found no difference between supportive counselling and other treatments [45]. Taking these limitations into account, we believe that dismissing psychodynamic therapy and supportive therapy as suboptimal treatments for depression is unjustified.

This research has some limitations. Like standard meta-analysis, network meta-analysis assumes the included studies are drawn from the same population (i.e., homogeneity). But network meta-analysis makes an additional assumption to come to consistent results (i.e., no inconsistency of estimates) from direct and indirect estimates of relative effects. Yet, both assumptions hold for our results because heterogeneity was low in all our analyses, and we found no evidence of inconsistency between direct and indirect estimates.

Network meta-analysis also assumes that particular treatments are similar in rationale and procedure (i.e., the specific ingredients responsible for change), allowing us to group them together as one knot in the network according to a classification system. Grouping treatments that have important differences in rationale and procedures might obscure differences between treatments and cause us to underestimate the relative efficacy of intervention strategies. For the present study, an established classification system of psychotherapeutic interventions for depression [19] differentiated seven clearly defined treatment strategies and did not include a category summarising “other psychotherapeutic interventions.” But the category of usual care may have merged treatments of different intensity, which might have returned biased results. Nor did our study control for researcher allegiance bias [46]–[48], which may have introduced bias in effect estimates. Because data on a comparison level like allegiance cannot be considered in network meta-analysis, it is likely that researcher preferences influence the treatment effects found in this study to some extent.

Our approach has some additional minor limitations that should be taken into account when interpreting our findings. Clinical ratings lead to larger effects than self-report measures [49]. Our study summarizes both types of outcomes in a single aggregate measure in order to make all trials available for the network meta-analysis, and this might overestimate the treatment effects. Our results might have limited generalizability, because studies were mostly conducted in Western countries. It is not possible to come to conclusions about long-term effects because our effectiveness data were collected at the end of treatment. Finally, interaction effects and corresponding p-values were derived from a model that assumes the same interaction effect across comparisons. A more flexible modelling approach would allow for different interaction effects across comparisons, but data was too scarce to allow this.

Given the availability of effective treatment options and the severity of the disorder, the use of wait list controls should be considered as unethical. It should only be used in trials, where no adequate treatment is available. In the future, large trials that compare psychotherapeutic approaches with robust evidence of effectiveness may also prove the effectiveness of supportive counselling, psychodynamic therapy, and behavioural activation. To control for allegiance bias, we suggest future trials be carried out by collaborative research teams, representing allegiances to each intervention. For the dissemination of study results into practice, the availability of treatments in the health care system should also guide selection of psychotherapeutic approaches for clinical studies.

Conclusions

Small study effects affect the results of randomized controlled trials of psychotherapeutic interventions and should receive more attention in further meta-analyses. In larger trials, we found robust effects for cognitive-behavioural therapy, interpersonal therapy, and problem-solving therapy, while effects were less robust for psychodynamic therapy, supportive counselling, and behavioural activation. However, effect differences between these six psychotherapeutic interventions were rather small. Overall, we found that different psychotherapeutic interventions for depression have comparable, moderate-to-large effects.

Supporting Information

Attachment 1