Risks and Benefits of Nalmefene in the Treatment of Adult Alcohol Dependence: A Systematic Literature Review and Meta-Analysis of Published and Unpublished Double-Blind Randomized Controlled Trials

Summary

In a systematic review and meta-analysis, Florian Naudet and colleagues assess whether medication with the opioid antagonist nalmefene can reduce consumption and other outcomes of alcohol addiction.

Introduction

The net effect of alcohol consumption on health is detrimental in terms of global mortality and global disability [1]. Harm reduction by controlled drinking is a concept that has been developed as an alternative to abstinence. While this approach still raises questions [2,3], nalmefene, an opioid antagonist, was recently approved by the European Medicines Agency (EMA) [4] for the treatment of alcohol dependence in order to help reduce alcohol consumption in adults who consume more than 60 g of alcohol per day (for men) or more than 40 g per day (for women). The UK National Institute for Health and Care Excellence (NICE) initially recommended it as a “possible” treatment for alcohol dependence [5], but, subsequently, the NICE evidence review group distanced itself from its earlier advice [6]. The French National Authority for Health Transparency Committee was also cautious and considered that nalmefene provided a minor improvement in actual benefit (ranking IV on a scale from I to V, with V = no improvement) compared to psychosocial support alone in the treatment of alcohol dependence. The Transparency Committee recommended that prescribing authority be restricted to addiction and alcohol specialists during the first year [7] (a recommendation that was not subsequently followed by the French Ministry of Social Affairs and Health). In contrast, the German Institute for Quality and Efficiency in Health Care concluded that studies of nalmefene showed no additional benefits for alcohol dependence compared to naltrexone, another opioid antagonist that was an earlier and less costly comparator therapy [8]. Furthermore, the Swedish Dental and Pharmaceutical Benefits Agency concluded that the lack of advantage of nalmefene compared to existing treatments suggested that it did not warrant recommendation for reimbursement [9]. Researchers’ opinions are more divided, with some arguing that nalmefene is a “paradigm shift” [10,11] and others claiming that it is a perfect example of “bad medicine” [12,13].

This state of affairs raises a fundamental problem. While phase III clinical testing should lead to objective conclusions concerning treatment effect, in the case of nalmefene, it has led to divergent and, indeed, antagonistic opinions. There are at present doubts about the clinical significance of differences in alcohol consumption outcomes reported in the different studies. In addition, nalmefene studies have not directly explored morbidity/mortality outcomes. These outcomes were recently proposed as relevant in the clinical evaluation of alcohol dependence treatments [14] and are indeed expected benefits of a harm reduction approach.

We therefore planned a meta-analysis of aggregated data to enable an objective reappraisal of the efficacy of nalmefene for relevant health outcomes, not solely restricted to alcohol consumption endpoints.

Methods

We developed and followed a standard meta-analysis protocol (systematic review registration—PROSPERO 2014:CRD42014014853).

Eligibility Criteria

Types of participants

We reviewed studies involving adults (aged 18 y and over) with a diagnosis of non-abstinent alcohol dependence.

Types of interventions

Studies were eligible if they focused on the comparison of oral nalmefene with either placebo or another active comparator. For multiple dose studies, only the dose closest to 20 mg per day (recommended dose in the EMA authorization) was considered. The EMA recommendation was based on data from dose–response studies that suggested that doses below this dosage were less efficacious [4]. However, no clear dose–response relationship could be established in terms of the endpoint monthly number of heavy drinking days (HDDs). We initially planned to include studies lasting 6 mo (±1 mo) and/or 1 y (±1 mo), as in the pivotal studies, because the expected benefits of nalmefene on health outcomes, and indeed in terms of public health, were not expected over shorter durations. Nonetheless, it appeared during the review process that there were a substantial number of trials over shorter periods. These studies were included in a post hoc analysis.

Types of outcomes

We considered four health outcomes as primary outcomes: (1) mortality, (2) accidents (including motor vehicle crashes) or injuries, (3) quality of life or functioning, and (4) somatic complications of alcoholism. These health outcomes were used earlier in a systematic review [14] and describe the expected clinical benefits of treatment of alcohol dependence.

Secondary outcomes were in three categories: alcohol consumption outcomes, biological outcomes, and treatment safety outcomes. The alcohol consumption outcomes were (1) monthly number of HDDs, defined as days with alcohol consumption of 60 g or more for males and 40 g or more for females, (2) total alcohol consumption, (3) response (responders are patients decreasing their consumption to low-risk levels or no consumption), (4) complete abstinence, (5) total Drinker Inventory of Consequences (DrInC) score, (6) Clinical Global Impression–Severity score, and (7) Alcohol Dependence Scale score. The biological outcomes were (1) γ-glutamyltransferase, (2) alanine-aminotransferase, (3) mean corpuscular volume, and (4) carbohydrate-deficient transferrin. The treatment safety outcomes were (1) treatment-emergent adverse events, (2) serious adverse events, (3) withdrawal from the study, and (4) withdrawal for safety reasons.

All outcomes were collected at both 6 mo (±1 mo) and 1 y (±1 mo). Studies reporting outcomes with follow-up of under 5 mo were considered only in a post hoc sensitivity analysis.

Types of studies

In this review, only randomized controlled trials (RCTs) were included. Studies were searched for without limitation in terms of date. Only study reports in English, French, or Spanish language were considered.

Search Strategy

Eligible studies were identified from PubMed/Medline, the Cochrane Library and Embase, including conference abstracts. On PubMed, the keyword used was “nalmefene” with the filter “clinical trial.” Searches in other bases are detailed in S1 Supporting Information.

Unpublished studies were also searched for by communication with key organizations such as the US Food and Drug Administration and the “access to documents” service at the EMA, which granted us access to study reports of both published and unpublished clinical studies. The pharmaceutical companies Biotie Therapies and Lundbeck were contacted to provide information concerning their studies. A search on ClinicalTrials.gov and Current Controlled Trials was performed. When needed, the authors of abstracts were contacted for further information and were asked for the references of the studies. If no response was obtained to a first solicitation, they were contacted a second time.

Study Selection

The eligibility assessment was performed independently in a blind standardized manner by two reviewers (C. P. and F. N.). Disagreements were resolved by consensus or in consultation with a third reviewer (B. L.). A comparison across studies, checking author names, treatment comparisons, sample sizes, and outcomes, was performed to avoid duplicates and compilations of data from several reports of the same study.

Assessment of Methodological Quality

Each paper was then assessed for methodological quality prior to inclusion in the review using the Cochrane Collaboration tool for assessing risk of bias.

Data Collection

A data extraction sheet based on the Cochrane Handbook for Systematic Reviews of Interventions guidelines [15] was used. Two review authors (C. P. and F. N.) extracted the data from the studies included. Disagreements were resolved by consensus or in consultation with a third reviewer (B. L.). For each study included, information was extracted on (1) characteristics of the study (year, country, type of comparator, number of arms, funding), (2) characteristics of trial participants (age, gender, number of patients included in the analysis, population analyzed), (3) type of intervention (treatment, duration), and (4) outcome measures as detailed above.

Data Analysis

General strategy

The efficacy index used for binary outcomes was the risk ratio (RR), and the efficacy index used for continuous outcomes was the mean difference (MD), or standardized mean difference (SMD) when appropriate. These efficacy indices are reported with 95% confidence intervals. As a substantial number of patients lost to follow-up was expected, sensitivity analyses were planned using the baseline observation carried forward (BOCF) approach for continuous outcomes; for binary outcomes, loss to follow-up was considered as failure (i.e., non-assessed patients were recorded as not having responded in both groups).

Risk of bias across studies

We used visual inspection of the forest plots, the I2 index, and the Q statistic to investigate the possibility of statistical heterogeneity. In the absence of heterogeneity, we performed direct meta-analyses by synthesizing studies that compared the same interventions using a fixed effect model. When there was heterogeneity, we performed a direct meta-analysis with a random effect model.

Publication bias was not investigated graphically (funnel plots) nor by testing (the rank correlation test) because there was a small number of studies, which limits the usefulness of these methods [16].

Analyses were performed using R [17] with meta [18] and rmeta [19] libraries. Results are presented according to PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) format [20].

Minor changes to the initial protocol

During our searches, it appeared that some RCTs of nalmefene in the treatment of alcohol dependence also included patients with difficulty in controlling drinking, rather than alcohol dependence. These studies were considered if alcohol dependence was not an explicit exclusion criterion. For mortality, which is a rare outcome, a sensitivity analysis was performed using the gmeta library [21], which enables exact meta-analysis methods to be run. These methods can use all available information without artificial continuity correction on zero-event studies [22].

During the peer review process, the issue was raised of exploring a possible dose–response relationship between nalmefene and the outcomes. We therefore performed a networked meta-analysis on monthly number of HDDs, looking at different doses (particularly higher doses) and different administration regimens for nalmefene. This network meta-analysis was performed using the frequentist approach, which was implemented in R in the netmeta library [23].

Results

Study Selection

After adjusting for duplicates, the searches provided a total of 169 citations. Of these, 129 studies were discarded (on the basis of title and abstract) because they did not meet the selection criteria. No study was discarded for language. After examination of the full text of the remaining 40 articles, 31 additional references were discarded. Nine studies were included in the quantitative review and five in the main analysis (studies with at least 6 mo of follow-up). Concerning unpublished data (missing outcomes in the publication or unpublished studies), clinical study reports were provided by the access to documents service at the EMA, and no supplemental information was retrieved by communication with study authors. A flowchart detailing the study selection process is given in Fig 1.

Study Characteristics and Risk of Bias within Studies

The main analysis involved 2,567 randomized participants in five RCTs. All five studies provided data at 6 mo, and only one provided a 1-y follow-up. Four additional RCTs versus placebo (which did not meet the duration criterion) involving 353 additional randomized participants were also considered in the sensitivity analysis. All of these studies involved adults with alcohol dependence, but none was performed in the specific population defined by the EMA approval (i.e., adults with alcohol dependence consuming more than 60 g of alcohol per day for men or more than 40 g per day for women). In two of the studies included in our main analysis and one of the four studies added in the sensitivity analysis, patients were eligible based on alcohol dependence but also based on difficulty in controlling drinking, defined as (1) alcohol often taken in larger amounts or over a longer period than was intended or (2) a persistent desire or unsuccessful efforts to cut down or control drinking. No RCTs compared nalmefene with another active comparator.

The main characteristics of the different studies are presented in Table 1. The quality assessment of these studies is presented in Table 2. All of these studies were classified as presenting a high risk of incomplete outcome data because of large numbers of patients lost to follow-up.

Results from Individual Studies and Synthesis of Results

Main analysis

Individual study results, direct meta-analyses (both fixed effect and random effect meta-analyses), and Q statistics are presented in detail in S1 Supporting Information for all outcomes. Concerning health outcomes (Fig 2), no difference was found across groups concerning mortality, and no results were extractable for accidents/injuries or for somatic alcoholism complications because these outcomes were not assessed. Quality of life was reported in three studies at 6 mo (but not at 1 y). Whereas one study showed superiority of nalmefene treatment over placebo on the SF-36 physical component summary score and the SF−36 mental component summary score, the other two studies did not replicate this finding—the summary measures for the two components did not provide evidence for any superiority (Fig 2; Table 3). Results concerning clinical, biological, and safety outcomes are reported in Table 3. As regards biological outcomes, only one study reported useable data concerning γ-glutamyltransferase, alanine-aminotransferase, and mean corpuscular volume.

Additional analyses

The sensitivity analyses (Table 4) using BOCF analysis were possible for a small number of efficacy outcomes (monthly number of HDDs, total alcohol consumption) and in only the three studies that reported these data. These analyses found no difference between nalmefene and placebo. Sensitivity analyses considering loss to follow-up as failure did not change the results concerning safety outcomes. Sensitivity analyses involving studies lasting less than 6 mo showed the robustness of our estimations, and are presented in S1 Supporting Information. For death, which was a rare outcome, we ran exact meta-analysis methods, which yielded the same results as the main analysis (i.e., no difference found between nalmefene and placebo).

Dose–response analysis

The main analysis was based on a set of very similar studies. Three of these used 20 mg nalmefene as needed, and two of them used adjustable dosages (20 mg, which could be adjusted to 10 mg or 40 mg if needed).

The network meta-analysis on monthly number of HDDs found no evidence for a dose–response relationship, although the possibility remains that the 10-mg daily dose could have been less efficacious than the 20 mg nalmefene as needed dosing. Network geometry and quantitative results are presented in detail in S2 Supporting Information.

Discussion

Summary of Evidence

Compared to placebo, there was no evidence for the efficacy of nalmefene on health outcomes. Nalmefene was shown to be slightly superior to placebo in reducing the monthly number of HDDs, total alcohol consumption, DrInC score, Clinical Global Impression–Severity score, and Alcohol Dependence Scale score. However, these findings were not robust and disappeared when a conservative approach to managing withdrawals was used.

Currently, there is a wide debate on whether abstinence or reduced drinking should be the aim of alcohol dependence treatment. Epidemiological studies have suggested that drinking-related outcomes were shown to determine unspecific health outcomes like quality of life in adolescents [31], but there is no high-quality randomized evidence concerning the efficacy of managed alcohol programs on their own on these health outcomes [32]. Without entering further into this debate [2], our results support the view that nalmefene RCTs give little evidence of a reduction in alcohol consumption and no evidence of “harm reduction” with nalmefene use. Additionally, no difference between nalmefene and placebo was found concerning serious adverse events. At best, evidence of “harm reduction” is provided by simulation studies [33,34].

In addition, we found no RCTs of nalmefene versus placebo conducted in the specific population defined by the EMA approval of nalmefene (i.e., adult patients with alcohol dependence who consume more than 60 g of alcohol per day for men or more than 40 g per day for women). In this population, the only available data are pooled subgroup analyses in two or three of the RCTs we identified [34,35]. However, the credibility of subgroup analyses, even when claims are strong, is usually low [36], and such analyses should be considered as exploratory rather than confirmatory. On the one hand, regarding efficacy, some studies have found lower efficacy of opioid antagonists in hazardous/harmful drinkers compared to more severely dependent drinkers [37,38], and this latter target population could be of interest. On the other hand, subgroup analyses raise important methodological issues, especially when they are not prespecified [39,40]. In the case of the nalmefene RCTs, the subgroup analyses were not defined a priori.

Finally, we found no RCTs comparing nalmefene with another medication as an active comparator. The current state of knowledge and treatment evidence suggest that there is little or no difference in efficacy in reducing heavy drinking between nalmefene and naltrexone [41]. Although naltrexone is not authorized for reducing drinking, as opposed to promoting abstinence, this is probably its main effect [42].

Strengths and Limitations

Individual study results were very much influenced by the studies’ management of missing data. Most of the studies included presented a high rate of withdrawals. Whereas a loss to follow-up of 5% or lower is usually of little concern, a loss of 20% or more prevents good-quality intention-to-treat analysis, can cause biased estimates of the treatment effect [43], and restricts the scope for generalizing results [44]. There were more withdrawals, including more withdrawals for safety reasons, in the nalmefene group than in the placebo group in the included studies. In addition, extractable results were mainly derived from observed case analyses. These two characteristics expose studies to an attrition bias. Indeed, when patients with side effects are not included in the analysis, it can make a treatment appear to be effective when it is not. For this reason, we used a conservative approach to managing withdrawals based on a “worst” hypothesis. The BOCF approach could have nonetheless been overly conservative, as suggested recently [45], and its results, although informative, should be weighed accordingly.

Concerning our literature search, one might suspect a “Tower of Babel bias” [46] because only studies in English, French, and Spanish were considered. In fact, no study was excluded for this reason. Also, we did not use “Selincro,” the commercial name of nalmefene, as a keyword. In response to a reviewer’s comment before the publication of this paper, we conducted a post hoc search on PubMed with “nalmefene OR selincro,” and it provided the same number of references. All in all, the major strength of our analysis is that we included what we believe to be all available relevant data from completed studies of nalmefene meeting our inclusion criteria. Indeed, the EMA granted us permission to access all study reports. This also limits the risk of selective outcome reporting, despite the fact that in the identified publications, not all outcomes were reported.

It can be noted that we chose four primary outcomes (the four health outcomes) and that our analysis was not adjusted for multiple comparisons, with a possible inflation of type I error. We chose this approach to put equal emphasis on these health outcomes, which are the expected outcomes in terms of public health. While this is a limitation for our study, it also enables the issue of multiple comparisons to be tackled. This was problematic in the three pivotal studies on nalmefene for alcohol dependence [24,25,27], which used two co-primary outcomes without any specific adjustment for multiple comparisons.

In addition, none of the published reports presented all four health outcomes that were considered in this meta-analysis. In the published reports, data on mortality were accessible, but data concerning accidents/injuries, quality of life/functioning, and somatic alcoholism complications were not (or only partially) presented. One could argue that we could have extracted data on accidents/injuries and somatic alcoholism complications from serious adverse events described in the study reports, but post hoc reconstructions of the criteria in this way could have been biased. It can be said that this is a limitation of our meta-analysis, but it is rather one of the main results: the outcomes that are clinically relevant (i.e., outcomes that are not surrogate outcomes) are lacking in the primary evidence that led to the EMA’s nalmefene approval.

Conclusion

Implications for research.This review calls into question the decisions of some of the regulatory and advisory bodies that have approved nalmefene on the basis of this evidence. Given our results, certain conditions should be set by health authorities for the maintenance of nalmefene market approval. In our opinion, RCTs against placebo and naltrexone proving the superiority of nalmefene in the approved indication are needed. In these studies, health outcomes should be assessed. These studies may be unrealistic from a methodological point of view because of the need for long-term follow-up and probably a large number of participants. There is thus a need for independent and well-designed post-marketing studies. In addition, a network meta-analysis is needed to assess the effectiveness of nalmefene in reducing alcohol consumption compared to the other active treatments available. We are currently performing this meta-analysis (see PROSPERO 2015:CRD42015019841 for more details).

Clinical implications. Clinicians must be aware that the value of nalmefene for the treatment of alcohol addiction is not established. At best, nalmefene has limited efficacy in reducing alcohol consumption.

Supporting Information

Attachment 1

Attachment 2

Attachment 3

Attachment 4