Why Most Clinical Research Is Not Useful

1. Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, et al. Biomedical research: increasing value, reducing waste. Lancet. 2014;383(9912):101–4. doi: 10.1016/S0140-6736(13)62329-6 24411643

2. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):e124. 16060722

3. Ioannidis JP. How to make more published research true. PLoS Med. 2014;11(10):e1001747. doi: 10.1371/journal.pmed.1001747 25334033

4. Moynihan R, Doran E, Henry D. Disease mongering is now part of the global health debate. PLoS Med. 2008;5(5):e106. doi: 10.1371/journal.pmed.0050106 18507498

5. Evans JA, Shim JM, Ioannidis JP. Attention to local health burden and the global disparity of health research. PLoS ONE. 2014;9(4):e90147. doi: 10.1371/journal.pone.0090147 eCollection 2014. 24691431

6. Viergever RF, Terry RF, Karam G. Use of data from registered clinical trials to identify gaps in health research and development. Bull World Health Organ. 2013;91(6):416–425C. doi: 10.2471/BLT.12.114454 24052678

7. Moynihan R, Heath I, Henry D. Selling sickness: the pharmaceutical industry and disease mongering. BMJ 2002;324(7342):886–91. 11950740

8. Frances A. Saving normal: an insider's revolt against out-of-control psychiatric diagnosis, DSM-5, big pharma, and the medicalization of ordinary life. HarperCollins, New York, 2013.

9. Evangelou E, Siontis KC, Pfeiffer T, Ioannidis JP. Perceived information gain from randomized trials correlates with publication in high-impact factor journals. J Clin Epidemiol. 2012; 65(12):1274–81. doi: 10.1016/j.jclinepi.2012.06.009 22959593

10. Clarke M, Chalmers I. Discussion sections in reports of controlled trials published in general medical journals: islands in search of continents? JAMA. 1998;280(3):280–2. 9676682

11. Clarke M1, Hopewell S, Chalmers I. Reports of clinical trials should begin and end with up-to-date systematic reviews of other relevant evidence: a status report. J R Soc Med. 2007;100(4):187–90. 17404342

12. Hopewell S, Dutton S, Yu LM, Chan AW, Altman DG. The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed. BMJ. 2010;340:c723. doi: 10.1136/bmj.c723 20332510

13. Fleming TR, DeMets DL. Surrogate end points in clinical trials: are we being misled? Ann Intern Med. 1996;125(7):605–13. 8815760

14. Ferreira-González I, Busse JW, Heels-Ansdell D, Montori VM, Akl EA, et al. Problems with use of composite end points in cardiovascular trials: systematic review of randomised controlled trials. BMJ. 2007;334(7597):786. 17403713

15. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003;290(12):1624–32. 14506122

16. Buesching DP, Luce BR, Berger ML. The role of private industry in pragmatic comparative effectiveness trials. J Comp Eff Res. 2012;1(2):147–56. doi: 10.2217/cer.12.9 24237375

17. Prasad V, Jorgenson J, Ioannidis JP, Cifu A. Observational studies often make clinical practice recommendations: an empirical evaluation of authors' attitudes. J Clin Epidemiol. 2013;66(4):361–366.e4. doi: 10.1016/j.jclinepi.2012.11.005 23384591

18. Hemkens LG, Contopoulos-Ioannidis DG, Ioannidis JP. Routinely collected data and comparative effectiveness evidence: promises and limitations. CMAJ. 2016 May 17;188(8):E158–64. doi: 10.1503/cmaj.150653

19. Hemkens LG, Contopoulos-Ioannidis DG, Ioannidis JP. Agreement of treatment effects for mortality from routinely collected data and subsequent randomized trials: meta-epidemiological survey. BMJ. 2016;352:i493. doi: 10.1136/bmj.i493 26858277

20. Mullins CD, Vandigo J, Zheng Z, Wicks P. Patient-centeredness in the design of clinical trials. Value Health. 2014;17(4):471–5. doi: 10.1016/j.jval.2014.02.012 24969009

21. Selby JV, Lipstein SH. PCORI at 3 years–progress, lessons, and plans. N Engl J Med. 2014;370(7):592–5. doi: 10.1056/NEJMp1313061 24521104

22. Meltzer DO, Hoomans T, Chung JW, Basu A. Minimal modeling approaches to value of information analysis for health research. Med Decis Making. 2011;31(6):E1–E22. doi: 10.1177/0272989X11412975 21712493

23. Detsky AS. Are clinical trials a cost-effective investment? JAMA 1989;262(13);1795–1800. 2506366

24. Soares MO, Welton NJ, Harrison DA, Peura P, Shankar- Hari M, et al. An evaluation of the feasibility, cost and value of information of a multicentre randomised controlled trial of intravenous immunoglobulin for sepsis (severe sepsis and septic shock): incorporating a systematic review, meta-analysis and value of information analysis. Health Technol Assess. 2012;16(7):1–186. doi: 10.3310/hta16070 22361003

25. Reith C, Landray M, Devereaux PJ, Bosch J, Granger CB, et al. Randomized clinical trials—removing unnecessary obstacles. N Engl J Med. 2013;369(11):1061–5. doi: 10.1056/NEJMsb1300760 24024845

26. Minelli C, Baio G. Value of information: a tool to improve research prioritization and reduce waste. PLoS Med. 2015;12(9):e1001882. doi: 10.1371/journal.pmed.1001882 26418866

27. Eisenstein EL, Collins R, Cracknell BS, Podesta O, Reid ED, et al. Sensible approaches for reducing clinical trial costs. Clin Trials. 2008;5(1):75–84. doi: 10.1177/1740774507087551 18283084

28. Kasenda B, von Elm E, You J, Blümle A, Tomonaga Y, et al. Prevalence, characteristics, and publication of discontinued randomized trials. JAMA. 2014;311(10):1045–51. doi: 10.1001/jama.2014.1361 24618966

29. Chapman SJ, Shelton B, Mahmood H, Fitzgerald JE, Harrison EM, et al. Discontinuation and non-publication of surgical randomised controlled trials: observational study. BMJ. 2014;349:g6870. doi: 10.1136/bmj.g6870 25491195

30. Van de Wetering FT, Scholten RJ, Haring T, Clarke M, Hooft L. Trial registration numbers are underreported in biomedical publications. PLoS ONE. 2012;7:e49599. doi: 10.1371/journal.pone.0049599 23166724

31. Dal-Ré R, Bracken MB, Ioannidis JP. Call to improve transparency of trials of non-regulated interventions. BMJ. 2015;350:h1323. doi: 10.1136/bmj.h1323 25820265

32. Milette K, Roseman M, Thombs BD. Transparency of outcome reporting and trial registration of randomized controlled trials in top psychosomatic and behavioral health journals: a systematic review. J Psychosom Res. 2011;70:205–17. doi: 10.1016/j.jpsychores.2010.09.015 21334491

33. Babu AS, Veluswamy SK, Rao PT, Maiya AG. Clinical trial registration in physical therapy journals: a cross-sectional study. Phys Ther. 2014;94:83–90. doi: 10.2522/ptj.20120531 24009345

34. Wager E, Williams P, for the OPEN Project. “Hardly worth the effort”? Medical journals’ policies and their editors’ and publishers’ views on trial registration and publication bias: quantitative and qualitative study. BMJ 2013;347:f5248. doi: 10.1136/bmj.f5248 24014339

35. Doshi P, Goodman SN, Ioannidis JP. Raw data from clinical trials: within reach? Trends Pharmacol Sci. 2013 Dec;34(12):645–7. doi: 10.1016/j.tips.2013.10.006 24295825

36. Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med. 2009;361(20):1963–71. doi: 10.1056/NEJMsa0906126 19907043

37. Jefferson T, Doshi P. Multisystem failure: the story of anti-influenza drugs. BMJ. 2014;348:g2263. doi: 10.1136/bmj.g2263 24721793

38. Abramson JD, Rosenberg HG, Jewell N, Wright JM. Should people at low risk of cardiovascular disease take a statin? BMJ. 2013;347:f6123. Erratum in: BMJ. 2014;348:g3329. doi: 10.1136/bmj.f6123 24149819

39. Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet. 2014;383(9912):166–75. doi: 10.1016/S0140-6736(13)62227-8 24411645

40. Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267–76. doi: 10.1016/S0140-6736(13)62228-X 24411647

41. Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383(9913):257–66. doi: 10.1016/S0140-6736(13)62296-5 24411650

42. Ioannidis JP, Cappelleri JC, Lau J, Skolnik PR, Melville B, et al. Early or deferred zidovudine therapy in HIV-infected patients without an AIDS-defining illness. Ann Intern Med. 1995;122(11):856–66. 7741372

43. Jørgensen AW, Hilden J, Gøtzsche PC. Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. BMJ. 2006;333(7572):782. 17028106

44. Bell CM, Urbach DR, Ray JG, Bayoumi A, Rosen AB, Greenberg D, Neumann PJ. Bias in published cost effectiveness studies: systematic review. BMJ. 2006;332(7543):699–703. 16495332

45. Lenzer J, Hoffman JR, Furberg CD, Ioannidis JP; Guideline Panel Review Working Group. Ensuring the integrity of clinical practice guidelines: a tool for protecting patients. BMJ. 2013;347:f5535. doi: 10.1136/bmj.f5535 24046286

46. Altman DG. ISIS and the emergence of large, simple trials. Lancet. 2015;386 (9994):636–7. doi: 10.1016/S0140-6736(15)61450-7 26334142

47. Siontis KC, Evangelou E, Ioannidis JP. Magnitude of effects in clinical trials published in high-impact general medical journals. Int J Epidemiol. 2011;40(5):1280–91. doi: 10.1093/ije/dyr095 22039194

48. Ioannidis JP, Boyack KW, Klavans R. Estimates of the continuously publishing core in the scientific workforce. PLoS ONE. 2014;9(7):e101698. doi: 10.1371/journal.pone.0101698 eCollection 2014. 25007173

49. Emanuel EJ. Reinventing American health care. PublicAffairs, New York, 2014.

50. Fleurence RL, Curtis LH, Califf RM, Platt R, Selby JV, et al. Launching PCORnet, a national patient-centered clinical research network. J Am Med Inform Assoc. 2014;21(4):578–82. doi: 10.1136/amiajnl-2014-002747 24821743

51. Sugarman J, Califf RM. Ethics and regulatory complexities for pragmatic clinical trials. JAMA. 2014;311(23):2381–2. doi: 10.1001/jama.2014.4164 24810723

52. Ioannidis JP. Mega-trials for blockbusters. JAMA. 2013;309(3):239–40. doi: 10.1001/jama.2012.168095 23321760

53. Schwitzer G. A guide to reading health care news stories. JAMA Intern Med. 2014;174(7):1183–6. doi: 10.1001/jamainternmed.2014.1359 24796314