The predictive performance of SAPS 2 and SAPS 3 in an intermediate care unit for internal medicine at a German university transplant center; A retrospective analysis

English version

Autoři: Michael Jahn ^aff001; Jan Rekowski ^aff002; Guido Gerken ^aff003; Andreas Kribben ^aff001; Ali Canbay ^aff004; Antonios Katsounas ^aff004
Působiště autorů: Department of Nephrology, University Hospital Essen, University Duisburg-Essen, Essen, Germany ^aff001; Institute for Medical Informatics, Biometry and Epidemiology, University Hospital Essen, University Duisburg-Essen, Essen, Germany ^aff002; Department of Gastroenterology and Hepatology, University Hospital Essen, University Duisburg-Essen, Essen, Germany ^aff003; Department of Gastroenterology, Hepatology and Infectious Diseases, University Hospital Magdeburg, Otto-von-Guericke-University Magdeburg, Magdeburg, Germany ^aff004
Vyšlo v časopise: PLoS ONE 14(9)
Kategorie: Research Article
prolekare.web.journal.doi_sk: https://doi.org/10.1371/journal.pone.0222164

Souhrn

Objective

To analyze and compare the performance of the Simplified-Acute-Physiology-Score (SAPS) 2 and SAPS 3 among intermediate care patients with internal disorders.

Materials and methods

We conducted a retrospective single-center analysis in patients (n = 305) admitted to an intermediate-care-unit (ImCU) for internal medicine at the University Hospital Essen, Germany. We employed and compared the SAPS 2 vs. the SAPS 3 scoring system for the assessment of disease severity and prediction of mortality rates among patients admitted to the ImCU within an 18-month period. Both scores, which utilize parameters recorded at admission to the intensive-care-unit (ICU), represent the most widely applied scoring systems in European intensive care medicine. The area-under-the-receiver-operating-characteristic-curve (AUROC) was used to evaluate the SAPS 2 and SAPS 3 discrimination performance. Ultimately, standardized-mortality-ratios (SMRs) were calculated alongside their respective 95%-confidence-intervals (95% CI) in order to determine the observed-to-expected death ratio and calibration belt plots were generated to evaluate the SAPS 2 and SAPS 3 calibration performance.

Results

Both scores provided acceptable discrimination performance, i.e., the AUROC was 0.71 (95% CI, 0.65–0.77) for SAPS 2 and 0.77 (95% CI, 0.72–0.82) for SAPS 3. Against the observed in-hospital mortality of 30.2%, SAPS 2 showed a weak performance with a predicted mortality of 17.4% and a SMR of 1.74 (95% CI, 1.38–2.09), especially in association with liver diseases and/or sepsis. SAPS 3 performed accurately, resulting in a predicted mortality of 29.9% and a SMR of 1.01 (95% CI, 0.8–1.21). Based on Calibration belt plots, SAPS 2 showed a poor calibration-performance especially in patients with low mortality risk (P<0.001), while SAPS 3 exhibited a highly accurate calibration performance (P = 0.906) across all risk levels.

Conclusions

In our study, the SAPS 3 exhibited high accuracy in prediction of mortality in ImCU patients with internal disorders. In contrast, the SAPS 2 underestimated mortality particularly in patients with liver diseases and sepsis.

Klíčová slova:

Death rates – Cirrhosis – Kidneys – Chronic kidney disease – Intensive care units – Liver transplantation – Sepsis – Trauma surgery

Zdroje

1. Arzeno NM, Lawson KA, Duzinski SV, Vikalo H. Designing optimal mortality risk prediction scores that preserve clinical knowledge. J Biomed Inform. 2015;56:145–56. doi: 10.1016/j.jbi.2015.05.021 26056073

2. Metnitz B, Schaden E, Moreno R, Le Gall JR, Bauer P, Metnitz PG. Austrian validation and customization of the SAPS 3 Admission Score. Intensive Care Med. 2009;35(4):616–22. doi: 10.1007/s00134-008-1286-2 18846365

3. Poole D, Rossi C, Anghileri A, Giardino M, Latronico N, Radrizzani D, et al. External validation of the Simplified Acute Physiology Score (SAPS) 3 in a cohort of 28,357 patients from 147 Italian intensive care units. Intensive Care Med. 2009;35(11):1916–24. doi: 10.1007/s00134-009-1615-0 19685038

4. Vincent JL, Moreno R. Clinical review: scoring systems in the critically ill. Crit Care. 2010;14(2):207. doi: 10.1186/cc8204 20392287

5. Martinez-Urbistondo D, Alegre F, Carmona-Torre F, Huerta A, Fernandez-Ros N, Landecho MF, et al. Mortality Prediction in Patients Undergoing Non-Invasive Ventilation in Intermediate Care. PLoS One. 2015;10(10):e0139702. doi: 10.1371/journal.pone.0139702 26436420

6. Moreno R, Morais P. Outcome prediction in intensive care: results of a prospective, multicentre, Portuguese study. Intensive Care Med. 1997;23(2):177–86. doi: 10.1007/s001340050313 9069003

7. Strand K, Soreide E, Aardal S, Flaatten H. A comparison of SAPS II and SAPS 3 in a Norwegian intensive care unit population. Acta Anaesthesiol Scand. 2009;53(5):595–600. doi: 10.1111/j.1399-6576.2009.01948.x 19419352

8. Richardson DB, Keil AP, Tchetgen Tchetgen E, Cooper G. Negative Control Outcomes and the Analysis of Standardized Mortality Ratios. Epidemiology. 2015;26(5):727–32. doi: 10.1097/EDE.0000000000000353 26172862

9. Cooper LM, Linde-Zwirble WT. Medicare intensive care unit use: analysis of incidence, cost, and payment. Crit Care Med. 2004;32(11):2247–53. doi: 10.1097/01.ccm.0000146301.47334.bd 15640637

10. Nassar AP, Malbouisson LM, Moreno R. Evaluation of Simplified Acute Physiology Score 3 performance: a systematic review of external validation studies. Crit Care. 2014;18(3):R117. doi: 10.1186/cc13911 24906651

11. Metnitz PG, Valentin A, Vesely H, Alberti C, Lang T, Lenz K, et al. Prognostic performance and customization of the SAPS II: results of a multicenter Austrian study. Simplified Acute Physiology Score. Intensive Care Med. 1999;25(2):192–7. doi: 10.1007/s001340050815 10193547

12. Sakr Y, Krauss C, Amaral AC, Rea-Neto A, Specht M, Reinhart K, et al. Comparison of the performance of SAPS II, SAPS 3, APACHE II, and their customized prognostic models in a surgical intensive care unit. Br J Anaesth. 2008;101(6):798–803. doi: 10.1093/bja/aen291 18845649

13. Keegan MT, Gajic O, Afessa B. Comparison of APACHE III, APACHE IV, SAPS 3, and MPM0III and influence of resuscitation status on model performance. Chest. 2012;142(4):851–8. doi: 10.1378/chest.11-2164 22499827

14. Katsounas A, Kamacharova I, Tyczynski B, Eggebrecht H, Erbel R, Canbay A, et al. The predictive performance of the SAPS II and SAPS 3 scoring systems: A retrospective analysis. J Crit Care. 2016;33:180–5. doi: 10.1016/j.jcrc.2016.01.013 26883275

15. Capuzzo M, Scaramuzza A, Vaccarini B, Gilli G, Zannoli S, Farabegoli L, et al. Validation of SAPS 3 Admission Score and comparison with SAPS II. Acta Anaesthesiol Scand. 2009;53(5):589–94. doi: 10.1111/j.1399-6576.2009.01929.x 19419351

16. Poole D, Rossi C, Latronico N, Rossi G, Finazzi S, Bertolini G. Comparison between SAPS II and SAPS 3 in predicting hospital mortality in a cohort of 103 Italian ICUs. Is new always better? Intensive Care Med. 2012;38(8):1280–8. doi: 10.1007/s00134-012-2578-0 22584793

17. Metnitz PG, Lang T, Vesely H, Valentin A, Le Gall JR. Ratios of observed to expected mortality are affected by differences in case mix and quality of care. Intensive Care Med. 2000;26(10):1466–72. doi: 10.1007/s001340000638 11126258

18. Lucena JF, Alegre F, Martinez-Urbistondo D, Landecho MF, Huerta A, Garcia-Mouriz A, et al. Performance of SAPS II and SAPS 3 in intermediate care. PLoS One. 2013;8(10):e77229. doi: 10.1371/journal.pone.0077229 24130860

19. Lucena JF, Alegre F, Rodil R, Landecho MF, Garcia-Mouriz A, Marques M, et al. Results of a retrospective observational study of intermediate care staffed by hospitalists: impact on mortality, co-management, and teaching. J Hosp Med. 2012;7(5):411–5. doi: 10.1002/jhm.1905 22271454

20. Auriant I, Vinatier I, Thaler F, Tourneur M, Loirat P. Simplified acute physiology score II for measuring severity of illness in intermediate care units. Crit Care Med. 1998;26(8):1368–71. doi: 10.1097/00003246-199808000-00023 9710096

21. Dupont B, Delvincourt M, Kone M, du Cheyron D, Ollivier-Hourmand I, Piquet MA, et al. Retrospective evaluation of prognostic score performances in cirrhotic patients admitted to an intermediate care unit. Dig Liver Dis. 2015;47(8):675–81. doi: 10.1016/j.dld.2015.04.001 25937626

22. Capuzzo M, Volta C, Tassinati T, Moreno R, Valentin A, Guidet B, et al. Hospital mortality of adults admitted to Intensive Care Units in hospitals with and without Intermediate Care Units: a multicentre European cohort study. Crit Care. 2014;18(5):551. doi: 10.1186/s13054-014-0551-8 25664865

23. Hamsen U, Lefering R, Fisahn C, Schildhauer TA, Waydhas C. Workload and severity of illness of patients on intensive care units with available intermediate care units: a multicenter cohort study. Minerva Anestesiol. 2018;84(8):938–45. doi: 10.23736/S0375-9393.18.12516-8 29469547

24. Plate JDJ, Leenen LPH, Houwert M, Hietbrink F. Utilisation of Intermediate Care Units: A Systematic Review. Crit Care Res Pract. 2017;2017:8038460. doi: 10.1155/2017/8038460 28775898

25. Armstrong E, de Waard MC, de Grooth HJ, Heymans MW, Reis Miranda D, Girbes AR, et al. Using Nursing Activities Score to Assess Nursing Workload on a Medium Care Unit. Anesth Analg. 2015;121(5):1274–80. doi: 10.1213/ANE.0000000000000968 26484461

26. Moreno RP, Metnitz PG, Almeida E, Jordan B, Bauer P, Campos RA, et al. SAPS 3—From evaluation of the patient to evaluation of the intensive care unit. Part 2: Development of a prognostic model for hospital mortality at ICU admission. Intensive Care Med. 2005;31(10):1345–55. doi: 10.1007/s00134-005-2763-5 16132892

27. Le Gall JR, Lemeshow S, Saulnier F. A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA. 1993;270(24):2957–63 8254858

28. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36. doi: 10.1148/radiology.143.1.7063747 7063747

29. Finazzi S, Poole D, Luciani D, Cogo PE, Bertolini G. Calibration belt for quality-of-care assessment based on dichotomous outcomes. PLoS One. 2011;6(2):e16110. doi: 10.1371/journal.pone.0016110 21373178

30. Nattino G, Finazzi S, Bertolini G. A new test and graphical tool to assess the goodness of fit of logistic regression models. Statistics in Medicine. 2016;35(5):709–20. doi: 10.1002/sim.6744 26439593

31. Chapter 1: Definition and classification of CKD. Kidney International Supplements. 2013;3(1):19–62. doi: 10.1038/kisup.2012.64 25018975

32. Section 2: AKI Definition. Kidney International Supplements. 2012;2(1):19–36. doi: 10.1038/kisup.2011.32 25018918

33. Levy MM, Fink MP, Marshall JC, Abraham E, Angus D, Cook D, et al. 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference. Intensive Care Med. 2003;29(4):530–8. doi: 10.1007/s00134-003-1662-x 12664219

34. Alegre F, Huerta A, Landecho MF, Lucena JF. Comment on "Retrospective evaluation of prognostic score performances in cirrhotic patients admitted to an intermediate care unit" by Benoit Dupont et al. [Digestive and Liver Disease 2015;47:675–81]. Dig Liver Dis. 2016;48(2):209–10. doi: 10.1016/j.dld.2015.10.002 26853043

35. Lemeshow S, Hosmer DW Jr. A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol. 1982;115(1):92–106. doi: 10.1093/oxfordjournals.aje.a113284 7055134

36. Kramer AA, Zimmerman JE. Assessing the calibration of mortality benchmarks in critical care: The Hosmer-Lemeshow test revisited. Crit Care Med. 2007;35(9):2052–6. doi: 10.1097/01.CCM.0000275267.64078.B0 17568333

37. Bertolini G, D’Amico R, Nardi D, Tinazzi A, Apolone G. One model, several results: the paradox of the Hosmer-Lemeshow goodness-of-fit test for the logistic regression model. J Epidemiol Biostat. 2000;5(4):251–3. 11055275

38. Xu W, Zhu L. A modified Hosmer–Lemeshow test for large data sets AU—Yu, Wei. Communications in Statistics—Theory and Methods. 2017;46(23):11813–25.