A usability study to improve a clinical decision support system for the prescription of antibiotic drugs

English version

Autoři: H. Akhloufi ^aff001; S. J. C. Verhaegh ^aff001; M. W. M. Jaspers ^aff003; D. C. Melles ^aff001; H. van der Sijs ^aff004; A. Verbon ^aff001
Působiště autorů: Department of Medical Microbiology and Infectious Diseases, Erasmus University Medical Center, Rotterdam, the Netherlands ^aff001; Department of Internal Medicine, Division of Infectious Diseases, Erasmus University Medical Center, Rotterdam, the Netherlands ^aff002; Department of Medical Informatics, Center for Human Factors Engineering of Health Information Technology (HIT-Lab), Academic Medical Center, Amsterdam, the Netherlands ^aff003; Department of Hospital Pharmacy, Erasmus University Medical Center, Rotterdam, the Netherlands ^aff004
Vyšlo v časopise: PLoS ONE 14(9)
Kategorie: Research Article
prolekare.web.journal.doi_sk: https://doi.org/10.1371/journal.pone.0223073

Souhrn

Objective

A clinical decision support system (CDSS) for empirical antibiotic treatment has the potential to increase appropriate antibiotic use. Before using such a system on a broad scale, it needs to be tailored to the users preferred way of working. We have developed a CDSS for empirical antibiotic treatment in hospitalized adult patients. Here we determined in a usability study if the developed CDSS needed changes.

Methods

Four prespecified patient cases, based on real life clinical scenarios, were evaluated by 8 medical residents in the study. The “think-aloud” method was used, and sessions were recorded and analyzed afterwards. Usability was assessed by 3 evaluators using an augmented classification scheme, which combines the User Action Framework with severity rating of the usability problems and the assessment of the potential impact of these problems on the final task outcomes.

Results

In total 51 usability problems were identified, which could be grouped into 29 different categories. Most (n = 17/29) of the usability problems were cosmetic problems or minor problems. Eighteen (out of 29) of the usability categories could have an ordering error as a result. Classification of the problems showed that some of the problems would get a low priority based on their severity rating, but got a high priority for their impact on the task outcome. This effectively provided information to prioritize system redesign efforts.

Conclusion

Usability studies improve lay-out and functionality of a CDSS for empirical antibiotic treatment, even after development by a multidisciplinary system.

Klíčová slova:

Physicians – Antimicrobials – Antibiotics – Antibiotic resistance – Pneumonia – Treatment guidelines – Clinical decision support systems – Neutropenia

Zdroje

1. O'Neill J. Review on Antimicrobial Resistance. Antimicrobial Resistance: Tackling a Crisis for the Health and Wealth of Nations. 2014 [cited 2018 February 2, 2018]. https://amr-review.org/sites/default/files/AMR%20Review%20Paper%20-%20Tackling%20a%20crisis%20for%20the%20health%20and%20wealth%20of%20nations_1.pdf.

2. Camins BC, King MD, Wells JB, Googe HL, Patel M, Kourbatova EV, et al. Impact of an antimicrobial utilization program on antimicrobial use at a large teaching hospital: a randomized controlled trial. Infect Control Hosp Epidemiol. 2009;30(10):931–8. doi: 10.1086/605924 19712032.

3. Ingram PR, Seet JM, Budgeon CA, Murray R. Point-prevalence study of inappropriate antibiotic use at a tertiary Australian hospital. Intern Med J. 2012;42(6):719–21. doi: 10.1111/j.1445-5994.2012.02809.x 22697156.

4. Akhloufi H, Streefkerk RH, Melles DC, de Steenwinkel JE, Schurink CA, Verkooijen RP, et al. Point prevalence of appropriate antimicrobial therapy in a Dutch university hospital. Eur J Clin Microbiol Infect Dis. 2015;34(8):1631–7. doi: 10.1007/s10096-015-2398-6 26017664.

5. Kerremans JJ, Verbrugh HA, Vos MC. Frequency of microbiologically correct antibiotic therapy increased by infectious disease consultations and microbiological results. J Clin Microbiol. 2012;50(6):2066–8. doi: 10.1128/JCM.06051-11 22422850.

6. Willemsen I, Groenhuijzen A, Bogaers D, Stuurman A, van Keulen P, Kluytmans J. Appropriateness of antimicrobial therapy measured by repeated prevalence surveys. Antimicrob Agents Chemother. 2007;51(3):864–7. doi: 10.1128/AAC.00994-06 17210766.

7. Kerremans JJ, Verboom P, Stijnen T, Hakkaart-van Roijen L, Goessens W, Verbrugh HA, et al. Rapid identification and antimicrobial susceptibility testing reduce antibiotic use and accelerate pathogen-directed antibiotic use. J Antimicrob Chemother. 2008;61(2):428–35. doi: 10.1093/jac/dkm497 18156278.

8. Schuts EC, Hulscher ME, Mouton JW, Verduin CM, Stuart JW, Overdiek HW, et al. Current evidence on hospital antimicrobial stewardship objectives: a systematic review and meta-analysis. The Lancet infectious diseases. 2016;16(7):847–56. Epub 2016/03/08. doi: 10.1016/S1473-3099(16)00065-7 26947617.

9. Kallen MC, Prins JM. A Systematic Review of Quality Indicators for Appropriate Antibiotic Use in Hospitalized Adult Patients. Infect Dis Rep. 2017;9(1):6821. doi: 10.4081/idr.2017.6821 28458795.

10. Leibovici L, Gitelman V, Yehezkelli Y, Poznanski O, Milo G, Paul M, et al. Improving empirical antibiotic treatment: prospective, nonintervention testing of a decision support system. J Intern Med. 1997;242(5):395–400. doi: 10.1046/j.1365-2796.1997.00232.x 9408069.

11. Evans RS, Classen DC, Pestotnik SL, Lundsgaarde HP, Burke JP. Improving empiric antibiotic selection using computer decision support. Arch Intern Med. 1994;154(8):878–84. 8154950.

12. Warner H Jr., Reimer L, Suvinier D, Li L, Nelson M. Modeling empiric antibiotic therapy evaluation of QID. Proc AMIA Symp. 1999:440–4. 10566397.

13. Mullett CJ, Thomas JG, Smith CL, Sarwari AR, Khakoo RA. Computerized antimicrobial decision support: an offline evaluation of a database-driven empiric antimicrobial guidance program in hospitalized patients with a bloodstream infection. Int J Med Inform. 2004;73(5):455–60. doi: 10.1016/j.ijmedinf.2004.04.002 15171986.

14. Paul M, Andreassen S, Tacconelli E, Nielsen AD, Almanasreh N, Frank U, et al. Improving empirical antibiotic treatment using TREAT, a computerized decision support system: cluster randomized trial. J Antimicrob Chemother. 2006;58(6):1238–45. doi: 10.1093/jac/dkl372 16998208.

15. Rawson TM, Moore LSP, Hernandez B, Charani E, Castro-Sanchez E, Herrero P, et al. A systematic review of clinical decision support systems for antimicrobial management: are we failing to investigate these interventions appropriately? Clin Microbiol Infect. 2017;23(8):524–32. doi: 10.1016/j.cmi.2017.02.028 28268133.

16. James BC. Making it easy to do it right. N Engl J Med. 2001;345(13):991–3. doi: 10.1056/NEJM200109273451311 11575294.

17. Vincent WR, Martin CA, Winstead PS, Smith KM, Gatz J, Lewis DA. Effects of a pharmacist-to-dose computerized request on promptness of antimicrobial therapy. J Am Med Inform Assoc. 2009;16(1):47–53. doi: 10.1197/jamia.M2559 18952947.

18. Phillips IE, Nelsen C, Peterson J, Sullivan TM, Waitman LR. Improving aminoglycoside dosing through computerized clinical decision support and pharmacy therapeutic monitoring systems. AMIA Annu Symp Proc. 2008:1093. 18999099.

19. Diasinos N, Baysari M, Kumar S, Day RO. Does the availability of therapeutic drug monitoring, computerised dose recommendation and prescribing decision support services promote compliance with national gentamicin prescribing guidelines? Intern Med J. 2015;45(1):55–62. doi: 10.1111/imj.12627 25371347.

20. Beaulieu J, Fortin R, Palmisciano L, Elsaid K, Collins C. Enhancing clinical decision support to improve appropriate antimicrobial use. Am J Health Syst Pharm. 2013;70(13):1103–4, 13. doi: 10.2146/ajhp120589 23784157.

21. Schulz L, Osterby K, Fox B. The use of best practice alerts with the development of an antimicrobial stewardship navigator to promote antibiotic de-escalation in the electronic medical record. Infect Control Hosp Epidemiol. 2013;34(12):1259–65. doi: 10.1086/673977 24225610.

22. Rodrigues JF, Casado A, Palos C, Santos C, Duarte A, Fernandez-Llimos F. A computer-assisted prescription system to improve antibacterial surgical prophylaxis. Infect Control Hosp Epidemiol. 2012;33(4):435–7. doi: 10.1086/664923 22418648.

23. Filice GA, Drekonja DM, Thurn JR, Rector TS, Hamann GM, Masoud BT, et al. Use of a computer decision support system and antimicrobial therapy appropriateness. Infect Control Hosp Epidemiol. 2013;34(6):558–65. doi: 10.1086/670627 23651885.

24. Tsopra R, Sedki K, Courtine M, Falcoff H, De Beco A, Madar R, et al. Helping GPs to extrapolate guideline recommendations to patients for whom there are no explicit recommendations, through the visualization of drug properties. The example of AntibioHelp(R) in bacterial diseases. J Am Med Inform Assoc. 2019. doi: 10.1093/jamia/ocz057 31077275.

25. Mullett CJ, Thomas JG. Database-driven computerized antibiotic decision support: novel use of expert antibiotic susceptibility rules embedded in a pathogen-antibiotic logic matrix. AMIA Annu Symp Proc. 2003:480–3. 14728219.

26. Khairat S, Marc D, Crosby W, Al Sanousi A. Reasons For Physicians Not Adopting Clinical Decision Support Systems: Critical Analysis. JMIR Med Inform. 2018;6(2):e24. doi: 10.2196/medinform.8912 29669706.

27. Khajouei R, Peute LW, Hasman A, Jaspers MW. Classification and prioritization of usability problems using an augmented classification scheme. J Biomed Inform. 2011;44(6):948–57. doi: 10.1016/j.jbi.2011.07.002 21782036.

28. Yen PY, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc. 2012;19(3):413–22. doi: 10.1136/amiajnl-2010-000020 21828224.

29. Jordan P.W. T B, Weerdmeester B.A., McClelland I.L. Usability Evaluation in Industry. London: Taylor & Francis; 1996.

30. Sauro J. L JR. Quantifying the User Experience: Practical Statistics for User Research Amsterdam: Elsevier/Morgan Kaufmann; 2012.

31. Bangor A, Kortum PT, Miller JT. An Empirical Evaluation of the System Usability Scale. International Journal of Human–Computer Interaction. 2008;24(6):574–94. doi: 10.1080/10447310802205776

32. Tsopra R, Jais JP, Venot A, Duclos C. Comparison of two kinds of interface, based on guided navigation or usability principles, for improving the adoption of computerized decision support systems: application to the prescription of antibiotics. J Am Med Inform Assoc. 2014;21(e1):e107–16. doi: 10.1136/amiajnl-2013-002042 24008427.

33. Jaspers MW. A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Inform. 2009;78(5):340–53. doi: 10.1016/j.ijmedinf.2008.10.002 19046928.

34. Nielsen J. Heuristic Evaluation. New York: Wiley; 1994 1994.

35. Nielsen J. Usability Engineering: Morgan Kaufmann Publishers Inc.; 1993.

36. Nielsen J. How Many Test Users in a Usability Study? 2012 https://www.nngroup.com/articles/how-many-test-users/.

37. Virzi RA. Refining the Test Phase of Usability Evaluation: How Many Subjects Is Enough? Human Factors. 1992;34(4):457–68. doi: 10.1177/001872089203400407

38. Macefield R. How To Specify the Participant Group Size for Usability Studies: A Practitioner's Guide. J Usability Stud. 2009;5.

39. McCambridge J, Witton J, Elbourne DR. Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. Journal of clinical epidemiology. 2014;67(3):267–77. doi: 10.1016/j.jclinepi.2013.08.015 24275499.

40. R J. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests New York, USA: John Wiley & Sons; 1994.

41. Litvin CB, Ornstein SM, Wessell AM, Nemeth LS, Nietert PJ. Adoption of a clinical decision support system to promote judicious use of antibiotics for acute respiratory infections in primary care. International Journal of Medical Informatics. 2012;81(8):521–6. https://doi.org/10.1016/j.ijmedinf.2012.03.002. 22483528