Trial registration as a safeguard against outcome reporting bias and spin? A case study of randomized controlled trials of acupuncture
Autoři:
Jiyoon Won aff001; Seoyeon Kim aff001; Inhu Bae aff003; Hyangsook Lee aff001
Působiště autorů:
Department of Korean Medical Science, Graduate School, Kyung Hee University, Dongdaemun-gu, Seoul, Republic of Korea
aff001; Acupuncture & Meridian Science Research Centre, College of Korean Medicine, Kyung Hee University, Dongdaemun-gu, Seoul, Republic of Korea
aff002; Kyung Hee University Korean Medicine Hospital, Dongdaemun-gu, Seoul, Republic of Korea
aff003
Vyšlo v časopise:
PLoS ONE 14(10)
Kategorie:
Research Article
prolekare.web.journal.doi_sk:
https://doi.org/10.1371/journal.pone.0223305
Souhrn
Background and objective
Trial registration is widely endorsed as it is considered not only to enhance transparency and quality of reporting but also to help safeguard against outcome reporting bias and probably spin, known as specific reporting that could distort the interpretation of results thus mislead readers. We planned to investigate the current registration status of recently published randomized controlled trials (RCTs) of acupuncture, outcome reporting bias in the prospectively registered trials, and the association between trial registration and presence of spin and methodological factors in acupuncture RCTs.
Methods
Acupuncture RCTs published in English in recent 5 years (January 2013 to December 2017) were searched in PubMed, Cochrane Central Register of Controlled Trials, and EMBASE. Trial registration records identified in the publications and trial registries were classified into prospectively registered, retrospectively registered, or unregistered. Primary outcomes were identified and the direction of the results was judged as statistically significant (positive) or statistically nonsignificant (negative). We compared registered and published primary outcomes to assess outcome reporting bias and assessed whether discrepancies favored statistically significant outcomes. Frequency and strategies of spin in published reports with statistically nonsignificant results for primary outcomes were then identified. We also analyzed whether the trial registration status was associated with spin and quality of methodological factors.
Results
Of the 322 included RCTs, 41.9% (n = 135) were prospectively registered. Among 64 studies that were prospectively registered and specified primary outcomes, 25 trials had the discrepancies between the registered and published primary outcomes and 60% of them (15 trials) favored the statistically significant findings. Among 169 studies that specified primary outcomes, trial registration status was not associated with the direction of results, i.e., statistically significant or not. Spin was identified in 56.4% out of 78 studies with statistically nonsignificant primary outcomes and claiming efficacy with no consideration of statistically nonsignificant primary outcomes was the most common strategy for spin. Trial registration status was not statistically different between studies with and without spin.
Conclusion
While trial registration seemed to have improved over time, primary outcomes in registered records and publications were often inconsistent, tending to favor statistically significant findings and spin was common in studies with statistically nonsignificant primary outcomes. Journal editors and researchers in this field should be alerted to still prevalent reporting bias and spin.
Klíčová slova:
Drug therapy – Clinical trials – Database searching – Asia – Scientific publishing – Randomized controlled trials – Sleep – Acupuncture
Zdroje
1. Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869. doi: 10.1136/bmj.c869 20332511
2. Chan A, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457–2465. doi: 10.1001/jama.291.20.2457 15161896
3. Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan A, Cronin E, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS one. 2008;3(8):e3081. doi: 10.1371/journal.pone.0003081 18769481
4. Dwan K, Gamble C, Williamson PR, Kirkham JJ. Systematic review of the empirical evidence of study publication bias and outcome reporting bias—an updated review. PLoS one. 2013;8(7):e66844. doi: 10.1371/journal.pone.0066844 23861749
5. Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ. 2005;330(7494):753. doi: 10.1136/bmj.38356.424606.8F 15681569
6. Higgins J, Altman D, Sterne C. Chapter 8; Assessing the risk of bias in included studies. In: Higgins JPT, Churchill R, Chandler J, Cumpston MS (editors), Cochrane Handbook for Systematic Reviews of Interventions version 5.2.0 (updated June 2017), Cochrane, 2017. www.training.cochrane.org/handbook.
7. Horton R. The rhetoric of research. BMJ. 1995;310(6985):985–987. doi: 10.1136/bmj.310.6985.985 7728037
8. Fletcher RH, Black B. Spin in scientific writing: scientific mischief and legal jeopardy. Med Law. 2007;26(3):511–525. 17970249
9. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303(20):2058–2064. doi: 10.1001/jama.2010.651 20501928
10. Boutron I, Altman DG, Hopewell S, Vera-Badillo F, Tannock I, Ravaud P. Impact of spin in the abstracts of articles reporting results of randomized controlled trials in the field of cancer: the SPIIN randomized controlled trial. J Clin Oncol. 2014;32(36):4120–4126. doi: 10.1200/JCO.2014.56.7503 25403215
11. Song F, Parekh S, Hooper L, Loke YK, Ryder J, Sutton AJ, et al. Dissemination and publication of research findings: an updated review of related biases. Health Technol Assess. 2010;14(8):iii, ix–xi, 1–193. doi: 10.3310/hta14080 20181324
12. Boutron I, Ravaud P. Misrepresentation and distortion of research in biomedical literature. Proc Natl Acad Sci U S A. 2018;115(11):2613–2619. doi: 10.1073/pnas.1710755115 29531025
13. Laine C, Horton R, DeAngelis CD, Drazen JM, Frizelle FA, Godlee F, et al. Clinical trial registration—looking back and moving ahead. N Z Med J. 2007;120(1256):U2586. 17589554
14. Pilot LR, Waldmann DR. Food and Drug Administration Modernization Act of 1997: medical device provisions. Food Drug Law J. 1998;53(2):267–295. 10346685
15. De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R, et al. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. Croat Med J. 2004;45(5):531–532. 15906482
16. Zarin DA, Tse T, Sheehan J. The proposed rule for US clinical trial registration and results submission. N Engl J Med 2015;372(2):174–180. doi: 10.1056/NEJMsr1414226 25539444
17. Viergever RF, Li K. Trends in global clinical trial registration: an analysis of numbers of registered clinical trials in different parts of the world from 2004 to 2013. BMJ Open. 2015;5(9):e008932. doi: 10.1136/bmjopen-2015-008932 26408831
18. Zarin DA, Tse T, Williams RJ, Rajakannan T. Update on Trial Registration 11 Years after the ICMJE Policy Was Established. N Engl J Med. 2017;376(4):383–391. doi: 10.1056/NEJMsr1601330 28121511
19. US National Institutes of Health. [cited 12 Oct 2018] ClinicalTrials.gov. http://clinicaltrials.gov.
20. Harriman SL, Patel J. When are clinical trials registered? An analysis of prospective versus retrospective registration. Trials. 2016;17(1):187.
21. Dechartres A, Ravaud P, Atal I, Riveros C, Boutron I. Association between trial registration and treatment effect estimates: a meta-epidemiological study. BMC Med. 2016;14(1):100. doi: 10.1186/s12916-016-0639-x 27377062
22. Huić M, Marušić M, Marušić A. Completeness and changes in registered data and reporting bias of randomized controlled trials in ICMJE journals after trial registration policy. PLoS One 2011;6(9):e25258. doi: 10.1371/journal.pone.0025258 21957485
23. Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. 2009;302(9):977–984. doi: 10.1001/jama.2009.1242 19724045
24. Emdin C, Odutayo A, Hsiao A, Shakir M, Hopewell S, Rahimi K, et al. Association of cardiovascular trial registration with positive study findings: Epidemiological Study of Randomized Trials (ESORT). JAMA Intern Med. 2015;175(2):304–307. doi: 10.1001/jamainternmed.2014.6924 25545611
25. You B, Gan HK, Pond G, Chen EX. Consistency in the analysis and reporting of primary end points in oncology randomized controlled trials from registration to publication: a systematic review. J Clin Oncol. 2012;30(2):210–216. doi: 10.1200/JCO.2011.37.0890 22162583
26. Nankervis H, Baibergenova A, Williams HC, Thomas KS. Prospective registration and outcome-reporting bias in randomized controlled trials of eczema treatments: a systematic review. J Invest Dermatol. 2012;132(12):2727–2734. doi: 10.1038/jid.2012.231 22832489
27. Burke A, Upchurch DM, Dye C, Chyu L. Acupuncture use in the United States: Findings from the national health interview survey. J Altern Complement Med. 2006;12(7):639–648. doi: 10.1089/acm.2006.12.639 16970534
28. Hunt KJ, Coelho HF, Wider B, Perry R, Hung S, Terry R, et al. Complementary and alternative medicine use in England: results from a national survey. Int J Clin Pract. 2010;64(11):1496–1502. doi: 10.1111/j.1742-1241.2010.02484.x 20698902
29. Han J, Ho Y. Global trends and performances of acupuncture research. Neurosci Biobehav Rev. 2011;35(3):680–687. doi: 10.1016/j.neubiorev.2010.08.006 20800613
30. Liu JP, Han M, Li XX, Mu YJ, Lewith G, Wang YY, et al. Prospective registration, bias risk and outcome-reporting bias in randomised clinical trials of traditional Chinese medicine: an empirical methodological study. BMJ Open. 2013;3(7):e002968. doi: 10.1136/bmjopen-2013-002968 23864210
31. Jones CW, Keil LG, Holland WC, Caughey MC, Platts-Mills TF. Comparison of registered and published outcomes in randomized controlled trials: a systematic review. BMC Med. 2015;13(1):282.
32. Spieth PM, Kubasch AS, Penzlin AI, Illigens BM, Barlinn K, Siepmann T. Randomized controlled trials. Neuropsychiatr Dis Treat. 2016;12:1341–1349. doi: 10.2147/NDT.S101938 27354804
33. Juszczak E, Altman DG, Hopewell S, Schulz K. Reporting of multi-arm parallel group randomized trials: extensions of the CONSORT 2010 statement. JAMA. 2019;321(16):1610–1620. doi: 10.1001/jama.2019.3087 31012939
34. World Health Organization. Acupuncture: review and analysis of reports on controlled clinical trials. World Health Organization; 2002.
35. World Health Organization. [cited 24 Oct 2018] International clinical trial registry platform search portal. 2018; http://apps.who.int/trialsearch/.
36. Killeen S, Sourallous P, Hunter IA, Hartley JE, Grady HL. Registration rates, adequacy of registration, and a comparison of registered and published primary outcomes in randomized controlled trials published in surgery journals. Ann Surg. 2014;259(1):193–196. doi: 10.1097/SLA.0b013e318299d00b 23732270
37. International Committee of Medical Journal Editors. [cited 24 Oct 2018] Clinical trials registration. http://www.icmje.org/about-icmje/faqs/clinical-trials-registration/.
38. International Committee of Medical Journal Editors. [cited 24 Oct 2018] FDAAA 801 and the Final Rule. https://clinicaltrials.gov/ct2/manage-recs/fdaaa.
39. Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials. 1998;19(2):159. 9551280
40. Roest AM, De Jonge P, Williams CD, De Vries YA, Schoevers RA, Turner EH. Reporting bias in clinical trials investigating the efficacy of second-generation antidepressants in the treatment of anxiety disorders: a report of 2 meta-analyses. Jama Psychiatry. 2015;72(5):500–510. doi: 10.1001/jamapsychiatry.2015.15 25806940
41. Yavchitz A, Ravaud P, Altman DG, Moher D, Hróbjartsson A, Lasserson T, et al. A new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity. J Clin Epidemiol. 2016;75:56–65. doi: 10.1016/j.jclinepi.2016.01.020 26845744
42. Su C, Han M, Ren J, Li W, Yue S, Hao Y, et al. Empirical evidence for outcome reporting bias in randomized clinical trials of acupuncture: comparison of registered records and subsequent publications. Trials. 2015;16(1):28.
43. Turrentine M. It’s all how you “spin” it: interpretive bias in research findings in the obstetrics and gynecology literature. Obstet Gynecol. 2017;129(2):239–242. doi: 10.1097/AOG.0000000000001818 28079777
44. van de Wetering Fleur T, Scholten RJ, Haring T, Clarke M, Hooft L. Trial registration numbers are underreported in biomedical publications. PLoS One 2012;7(11):e49599. doi: 10.1371/journal.pone.0049599 23166724
45. Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340:c365. doi: 10.1136/bmj.c365 20156912
46. Papageorgiou SN, Xavier GM, Cobourne MT, Eliades T. Registered trials report less beneficial treatment effects than unregistered ones: a meta-epidemiological study in orthodontics. J Clin Epidemiol. 2018;100:44–52. doi: 10.1016/j.jclinepi.2018.04.017 29705094
47. Odutayo A, Emdin CA, Hsiao AJ, Shakir M, Copsey B, Dutton S, et al. Association between trial registration and positive study findings: cross sectional study (Epidemiological Study of Randomized Trials-ESORT). BMJ. 2017;356:j917. doi: 10.1136/bmj.j917 28292744
48. Ma B, Chen Z, Xu J, Wang Y, Chen K, Ke F, et al. Do the CONSORT and STRICTA checklists improve the reporting quality of acupuncture and moxibustion randomized controlled trials published in chinese journals? a systematic review and analysis of trends. PloS one. 2016;11(1):e0147244. doi: 10.1371/journal.pone.0147244 26808550
49. Zhao H, Zhang J, Yang F, Tan L. Improve the ethical review of clinical trials on traditional medicine: a cross-sectional study of clinical trials registration, ethical review, and informed consent in clinical trials of Traditional Chinese Medicine. Medicine. 2018;97(47):e13062. doi: 10.1097/MD.0000000000013062 30461608
50. Wager E, Williams P, Project Overcome failure to Publish nEgative fiNdings Consortium. "Hardly worth the effort"? Medical journals’ policies and their editors’ and publishers’ views on trial registration and publication bias: quantitative and qualitative study. BMJ. 2013;347:f5248. doi: 10.1136/bmj.f5248 24014339
51. Kaptchuk TJ. Effect of interpretive bias on research evidence. BMJ. 2003;326(7404):1453–1455. doi: 10.1136/bmj.326.7404.1453 12829562
52. Chiu K, Grundy Q, Bero L. ‘Spin’ in published biomedical literature: A methodological systematic review. PLoS biol. 2017;15(9):e2002173. doi: 10.1371/journal.pbio.2002173 28892482
53. Anand V, Scales DC, Parshuram CS, Kavanagh BP. Registration and design alterations of clinical trials in critical care: a cross-sectional observational study. Intensive Care Med. 2014;40(5):700–722. doi: 10.1007/s00134-014-3250-7 24737256
54. Mayo-Wilson E, Heyward J, Keyes A, Reynolds J, White S, Atri N, et al. Clinical trial registration and reporting: a survey of academic organizations in the United States. BMC Med. 2018;16(1):60. doi: 10.1186/s12916-018-1042-6 29716585
55. Lazarus C, Haneef R, Ravaud P, Hopewell S, Altman DG, Boutron I. Peer reviewers identified spin in manuscripts of nonrandomized studies assessing therapeutic interventions, but their impact on spin in abstract conclusions was limited. J Clin Epidemiol. 2016;77:44–51. doi: 10.1016/j.jclinepi.2016.04.012 27164274
56. Chauvin A, Moher D, Altman D, Schriger DL, Alam S, Hopewell S, et al. A protocol of a cross-sectional study evaluating an online tool for early career peer reviewers assessing reports of randomised controlled trials. BMJ Open. 2017;7(9):e017462. doi: 10.1136/bmjopen-2017-017462 28918414
Článok vyšiel v časopise
PLOS One
2019 Číslo 10
- Metamizol jako analgetikum první volby: kdy, pro koho, jak a proč?
- Nejasný stín na plicích – kazuistika
- Masturbační chování žen v ČR − dotazníková studie
- Těžké menstruační krvácení může značit poruchu krevní srážlivosti. Jaký management vyšetření a léčby je v takovém případě vhodný?
- Fixní kombinace paracetamol/kodein nabízí synergické analgetické účinky
Najčítanejšie v tomto čísle
- Correction: Low dose naltrexone: Effects on medication in rheumatoid and seropositive arthritis. A nationwide register-based controlled quasi-experimental before-after study
- Combining CDK4/6 inhibitors ribociclib and palbociclib with cytotoxic agents does not enhance cytotoxicity
- Experimentally validated simulation of coronary stents considering different dogboning ratios and asymmetric stent positioning
- Risk factors associated with IgA vasculitis with nephritis (Henoch–Schönlein purpura nephritis) progressing to unfavorable outcomes: A meta-analysis