Threats to Validity in the Design and Conduct of Preclinical Efficacy Studies: A Systematic Review of Guidelines for In Vivo Animal Experiments

English version České info

Background:
The vast majority of medical interventions introduced into clinical development prove unsafe or ineffective. One prominent explanation for the dismal success rate is flawed preclinical research. We conducted a systematic review of preclinical research guidelines and organized recommendations according to the type of validity threat (internal, construct, or external) or programmatic research activity they primarily address.

Methods and Findings:
We searched MEDLINE, Google Scholar, Google, and the EQUATOR Network website for all preclinical guideline documents published up to April 9, 2013 that addressed the design and conduct of in vivo animal experiments aimed at supporting clinical translation. To be eligible, documents had to provide guidance on the design or execution of preclinical animal experiments and represent the aggregated consensus of four or more investigators. Data from included guidelines were independently extracted by two individuals for discrete recommendations on the design and implementation of preclinical efficacy studies. These recommendations were then organized according to the type of validity threat they addressed. A total of 2,029 citations were identified through our search strategy. From these, we identified 26 guidelines that met our eligibility criteria—most of which were directed at neurological or cerebrovascular drug development. Together, these guidelines offered 55 different recommendations. Some of the most common recommendations included performance of a power calculation to determine sample size, randomized treatment allocation, and characterization of disease phenotype in the animal model prior to experimentation.

Conclusions:
By identifying the most recurrent recommendations among preclinical guidelines, we provide a starting point for developing preclinical guidelines in other disease domains. We also provide a basis for the study and evaluation of preclinical research practice.

Please see later in the article for the Editors' Summary

Vyšlo v časopise: Threats to Validity in the Design and Conduct of Preclinical Efficacy Studies: A Systematic Review of Guidelines for In Vivo Animal Experiments. PLoS Med 10(7): e32767. doi:10.1371/journal.pmed.1001489
Kategorie: Research Article
prolekare.web.journal.doi_sk: https://doi.org/10.1371/journal.pmed.1001489

Souhrn

Zdroje

1. KolaI, LandisJ (2004) Can the pharmaceutical industry reduce attrition rates? Nat Rev Drug Discov 3: 711–715.

2. Contopoulos-IoannidisDG, NtzaniE, IoannidisJP (2003) Translation of highly promising basic science research into clinical applications. Am J Med 114: 477–484.

3. LondonAJ, KimmelmanJ, EmborgME (2010) Research ethics. Beyond access vs. protection in trials of innovative therapies. Science 328: 829–830.

4. KimmelmanJ, AndersonJA (2012) Should preclinical studies be registered? Nat Biotechnol 30: 488–489.

5. BegleyCG, EllisLM (2012) Drug development: raise standards for preclinical cancer research. Nature 483: 531–533.

6. PrinzF, SchlangeT, AsadullahK (2011) Believe it or not: how much can we rely on published data on potential drug targets? Nat Rev Drug Discov 10: 712.

7. SenaES, van der WorpHB, BathPM, HowellsDW, MacleodMR (2010) Publication bias in reports of animal stroke studies leads to major overstatement of efficacy. PLoS Biol 8: e1000344 doi:10.1371/journal.pbio.1000344

8. van der WorpHB, HowellsDW, SenaES, PorrittMJ, RewellS, et al. (2010) Can animal models of disease reliably inform human studies? PLoS Med 7: e1000245 doi:10.1371/journal.pmed.1000245

9. Shadish WR, Cook TD, Campbell DT (2002) Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.

10. FisherM, FeuersteinG, HowellsDW, HurnPD, KentTA, et al. (2009) Update of the stroke therapy academic industry roundtable preclinical recommendations. Stroke 40: 2244–2250.

11. AltmanDG, SimeraI, HoeyJ, MoherD, SchulzK (2008) EQUATOR: reporting guidelines for health research. Lancet 371: 1149–1150.

12. Recommendations for standards regarding preclinical neuroprotective and restorative drug development. Stroke 30: 2752–2758.

13. KilkennyC, BrowneW, CuthillIC, EmersonM, AltmanDG (2010) Animal research: reporting in vivo experiments: the ARRIVE guidelines. Br J Pharmacol 160: 1577–1579.

14. KilkennyC, BrowneW, CuthillIC, EmersonM, AltmanDG (2011) Animal research: reporting in vivo experiments—the ARRIVE guidelines. J Cereb Blood Flow Metab 31: 991–993.

15. KilkennyC, BrowneWJ, CuthillIC, EmersonM, AltmanDG (2010) Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. PLoS Biol 8: e1000412 doi:10.1371/journal.pbio.1000412

16. BrouwersMC, KhoME, BrowmanGP, BurgersJS, CluzeauF, et al. (2010) AGREE II: advancing guideline development, reporting, and evaluation in health care. Prev Med 51: 421–424.

17. Cronbach LJ, Shapiro K (1982) Designing evaluations of educational and social programs. Hoboken (New Jersey): Jossey-Bass. 374 p.

18. LamontagneF, BrielM, DuffettM, Fox-RobichaudA, CookDJ, et al. (2010) Systematic review of reviews including animal studies addressing therapeutic interventions for sepsis. Crit Care Med 38: 2401–2408.

19. PetersJL, SuttonAJ, JonesDR, RushtonL, AbramsKR (2006) A systematic review of systematic reviews and meta-analyses of animal experiments with guidelines for reporting. J Environ Sci Health B 41: 1245–1258.

20. BriggsK, CasesM, HeardDJ, PastorM, PognanF, et al. (2012) Inroads to predict in vivo toxicology—an introduction to the eTOX Project. Int J Mol Sci 13: 3820–3846.

21. TaylorCF, FieldD, SansoneSA, AertsJ, ApweilerR, et al. (2008) Promoting coherent minimum reporting guidelines for biological and biomedical investigations: the MIBBI project. Nat Biotechnol 26: 889–896.

22. SmithB, AshburnerM, RosseC, BardJ, BugW, et al. (2007) The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nat Biotechnol 25: 1251–1255.

23. KilkennyC, ParsonsP, KadyszewskiE, FestingMF, CuthillIC, et al. (2010) Survey of the quality of experimental design, statistical analysis and reporting of research using animals. PLoS One 4: e7824 doi:10.1371/journal.pone.0007824

24. ReynoldsJC, RittenbergerJC, MenegazziJJ (2007) Drug administration in animal studies of cardiac arrest does not reflect human clinical experience. Resuscitation 74: 13–26.

25. ScottS, KranzJE, ColeJ, LincecumJM, ThompsonK, et al. (2008) Design, power, and interpretation of studies in the standard murine model of ALS. Amyotroph Lateral Scler 9: 4–15.

26. O'CollinsVE, MacleodMR, DonnanGA, HorkyLL, van der WorpBH, et al. (2006) 1,026 experimental treatments in acute stroke. Ann Neurol 59: 467–477.

27. NoseworthyJH, EbersGC, VandervoortMK, FarquharRE, YetisirE, et al. (1994) The impact of blinding on the results of a randomized, placebo-controlled multiple sclerosis clinical trial. Neurology 44: 16–20.

28. WoodL, EggerM, GluudLL, SchulzKF, JuniP, et al. (2008) Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ 336: 601–605.

29. CrossleyNA, SenaE, GoehlerJ, HornJ, van der WorpB, et al. (2008) Empirical evidence of bias in the design of experimental stroke studies: a metaepidemiologic approach. Stroke 39: 929–934.

30. RookeED, VesterinenHM, SenaES, EganKJ, MacleodMR (2011) Dopamine agonists in animal models of Parkinson's disease: a systematic review and meta-analysis. Parkinsonism Relat Disord 17: 313–320.

31. BathPM, GrayLJ, BathAJ, BuchanA, MiyataT, et al. (2009) Effects of NXY-059 in experimental stroke: an individual animal meta-analysis. Br J Pharmacol 157: 1157–1171.

32. HackamDG, RedelmeierDA (2006) Translation of research evidence from animals to humans. JAMA 296: 1731–1732.

33. Odgaard-JensenJ, VistGE, TimmerA, KunzR, AklEA, et al. (2011) Randomisation to protect against selection bias in healthcare trials. Cochrane Database Syst Rev 2011: MR000012.

34. WoodcockJ, WoosleyR (2008) The FDA critical path initiative and its influence on new drug development. Annu Rev Med 59: 1–12.

35. Gauthier C, Koëter H, Griffin G, Hendriksen C, Kavlock R, et al.. (2011) Montréal declaration on the synthesis of evidence to advance the 3Rs principles in science. Eighth World Congress on Alternatives and Animal Use in the Life Sciences; 21–25 August 2011; Montréal, Canada.

36. MarguliesS, HicksR (2009) Combination therapies for traumatic brain injury: prospective considerations. J Neurotrauma 26: 925–939.

37. LudolphAC, BendottiC, BlaugrundE, ChioA, GreensmithL, et al. (2010) Guidelines for preclinical animal research in ALS/MND: a consensus meeting. Amyotroph Lateral Scler 11: 38–45.

38. RiceAS, Cimino-BrownD, EisenachJC, KontinenVK, Lacroix-FralishML, et al. (2008) Animal models and the prediction of efficacy in clinical trials of analgesic drugs: a critical appraisal and call for uniform reporting standards. Pain 139: 243–247.

39. GroundsMD, RadleyHG, LynchGS, NagarajuK, De LucaA (2008) Towards developing standard operating procedures for pre-clinical testing in the mdx mouse model of Duchenne muscular dystrophy. Neurobiol Dis 31: 1–19.

40. WillmannR, LucaAD, BenatarM, GroundsM, DubachJ, et al. (2012) Enhancing translation: guidelines for standard pre-clinical experiments in mdx mice. Neuromuscul Disord 22: 43–49.

41. EcclesM, ClappZ, GrimshawJ, AdamsPC, HigginsB, et al. (1996) North of England evidence based guidelines development project: methods of guideline development. BMJ 312: 760–762.

42. Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E, editors (2011) Clinical practice guidelines we can trust. Washington (District of Columbia): The National Academies Press.

43. Announcement: Reducing our irreproducibility. Nature 496: 398.

44. SchwartzRS, EdelmanE, VirmaniR, CarterA, GranadaJF, et al. (2008) Drug-eluting stents in preclinical studies: updated consensus recommendations for preclinical evaluation. Circ Cardiovasc Interv 1: 143–153.

45. VerhagenH, AruomaOI, van DelftJH, DragstedLO, FergusonLR, et al. (2003) The 10 basic requirements for a scientific paper reporting antioxidant, antimutagenic or anticarcinogenic potential of test substances in in vitro experiments and animal studies in vivo. Food Chem Toxicol 41: 603–610.

46. García-BonillaL, RosellA, TorregrosaG, SalomJB, AlborchE, et al. (2011) Recommendations guide for experimental animal models in stroke research. Neurologia 26: 105–110.

47. Kelloff GJ, Johnson JR, Crowell JA, Boone CW, DeGeorge JJ, et al.. (1994) Guidance for development of chemopreventive agents. J Cell Biochem (Suppl 20): 25–31.

48. KamathAT, FruthU, BrennanMJ, DobbelaerR, HubrechtsP, et al. (2005) New live mycobacterial vaccines: the Geneva consensus on essential steps towards clinical development. Vaccine 23: 3753–3761.

49. BellomoR, RoncoC, KellumJA, MehtaRL, PalevskyP (2004) Acute renal failure—definition, outcome measures, animal models, fluid therapy and information technology needs: the Second International Consensus Conference of the Acute Dialysis Quality Initiative (ADQI) Group. Crit Care 8: R204–R212.

50. MorenoB, EspejoC, MestreL, SuardiazM, ClementeD, et al. (2012) [Guidelines on the appropriate use of animal models for developing therapies in multiple sclerosis.]. Rev Neurol 54: 114–124.

51. WalkerMJ, CurtisMJ, HearseDJ, CampbellRW, JanseMJ, et al. (1988) The Lambeth Conventions: guidelines for the study of arrhythmias in ischaemia infarction, and reperfusion. Cardiovasc Res 22: 447–455.

52. CurtisM, HancoxJ, FarkasA, WainwrightC, StablesC, et al. (2013) The Lambeth Conventions (II): guidelines for the study of animal and human ventricular and supraventricular arrhythmias. . Pharmacol Ther E-pub ahead of print. doi: 10.1016/j.pharmthera.2013.04.008

53. PiperRD, CookDJ, BoneRC, SibbaldWJ (1996) Introducing critical appraisal to studies of animal models investigating novel therapies in sepsis. Crit Care Med 24: 2059–2070.

54. LiuS, ZhenG, MeloniBP, CampbellK, WinnHR (2009) Rodent stroke model guidelines for preclinical stroke trials (1st edition). J Exp Stroke Transl Med 2: 2–27.

55. LandisSC, AmaraSG, AsadullahK, AustinCP, BlumensteinR, et al. (2012) A call for transparent reporting to optimize the predictive value of preclinical research. Nature 490: 187–191.

56. BolonB, StolinaM, KingC, MiddletonS, GasserJ, et al. (2011) Rodent preclinical models for developing novel antiarthritic molecules: comparative biology and preferred methods for evaluating efficacy. J Biomed Biotechnol 2011: 569068.

57. MacleodMR, FisherM, O'CollinsV, SenaES, DirnaglU, et al. (2009) Good laboratory practice: preventing introduction of bias at the bench. Stroke 40: e50–e52.

58. US National Institutes of Health National Institute of Neurological Disorders and Stroke (2011) Improving the quality of NINDS-supported preclinical and clinical research through rigorous study design and transparent reporting. Bethesda (Maryland): US National Institutes of Health National Institute of Neurological Disorders and Stroke.

59. PullenN, BirchCL, DouglasGJ, HussainQ, Pruimboom-BreesI, et al. (2011) The translational challenge in the development of new and effective therapies for endometriosis: a review of confidence from published preclinical efficacy studies. Hum Reprod Update 17: 791–802.

60. ShinemanDW, BasiGS, BizonJL, ColtonCA, GreenbergBD, et al. (2011) Accelerating drug discovery for Alzheimer's disease: best practices for preclinical animal studies. Alzheimers Res Ther 3: 28.

61. BolliR, BeckerL, GrossG, MentzerRJr, BalshawD, et al. (2004) Myocardial protection at a crossroads: the need for translation into clinical therapy. Circ Res 95: 125–134.

62. Stem Cell Therapies as an Emerging Paradigm in Stroke Participants (2009) Stem Cell Therapies as an Emerging Paradigm in Stroke (STEPS): bridging basic and clinical science for cellular and neurogenic factor therapy in treating stroke. Stroke 40: 510–515.

63. SavitzSI, ChoppM, DeansR, CarmichaelST, PhinneyD, et al. (2011) Stem Cell Therapy as an Emerging Paradigm for Stroke (STEPS) II. Stroke 42: 825–829.

64. KatzDM, Berger-SweeneyJE, EubanksJH, JusticeMJ, NeulJL, et al. (2012) Preclinical research in Rett syndrome: setting the foundation for translational success. Dis Model Mech 5: 733–745.