Introduction of quality management in a National Reference Laboratory in Germany

Summary

Background

High quality diagnostic services are crucial for tuberculosis (TB) diagnosis, treatment and control. A strong laboratory quality management system (QMS) is critical to ensuring the quality of testing and results. Recent initiatives to improve TB laboratory quality have focused on low and middle-income countries, but similar issues also apply to high-income countries.

Methods and findings

Using a multipronged approach reviews of facilities, equipment, processes (purchasing, pre-analytic, analytic and post-analytic), staff, health and safety, documentation, information management and organization based on the ISO 15189 and the twelve quality system essentials were conducted between October 2015 and January 2016 at the National TB Reference Laboratory in Germany. Outcome assessment included proportion of smear positive slides, proportion of contaminated liquid cultures and DNA contamination rates before and after implementation of QMS. The odds ratio for these outcomes was calculated using a before/after comparison. Reviews highlighted deficiencies across all twelve quality system essentials and were addressed in order of priority and urgency. Actions aimed at improving analytical quality, health and safety and information management were prioritised for initial implementation in parallel with each other. The odds ratio for a sample to be tested as microscopically positive increased by 2.08 (95%CI 1.41–3.06) comparing the time before with the time after implementation of quality managed fluorescence microscopy. Liquid culture contamination rates decreased from 23.6- 7.6% in April-July 2016 to <10% in November 2017-March 2018. The proportion of negative controls showing evidence of DNA contamination decreased from 38.2% in 2013 to 8.1% in 2017, the corresponding odds ratio was 0.14 (95%CI 0.07–0.29).

Conclusion

This study showed marked improvement on quality indicators after implementation of a QMS in a National TB Reference Laboratory. The challenges and lessons learned in this study are valuable not just for high-income settings, but are equally generalizable to other laboratories.

Keywords:

equipment – Research laboratories – Biological laboratories – Clinical laboratory sciences – Government laboratories – Company laboratories – Safety equipment – Fire safety

Introduction

High-quality laboratory services are an essential component for tuberculosis care and control.[1] New diagnostic methods and increasing awareness about medical errors and their consequences emphasizes the great importance of quality in health.[2] A strong laboratory quality management system (QMS) is critical to ensuring the quality of testing.

Over the last six decades laboratory quality management has experienced ongoing development.[3] A ten-fold reduction in the analytical error rate has been achieved over the past decade as a result of improved reliability and standardization of analytical techniques, reagents, and instrumentation especially in clinical chemistry.[4–6] In addition, advances in information technology, quality control and quality assurance methods have also led to error reduction.

Laboratory quality management is generally accepted as state of the art practise and is legally prescribed in many countries such as Germany and the USA.[7, 8] Integral parts of a QMS, often implemented in the early stages of an improvement initiative, include quality control (QC), external quality assessment (EQA), standard operating procedures (SOP) and competency assessment (CA). Several studies have shown that QMS implementation results in a measurable improvement in the quality of services and increased patient safety due to a reduction in laboratory errors.[4, 9–11]

Germany is a low tuberculosis incident country with a total of 5915 tuberculosis patients notified in 2016 amounting to an annual tuberculosis notification rate of 7.2 cases per 100 000 population.[12] Microbiological confirmation was obtained for 71.5% of all cases. The laboratory network in Germany comprises stand-alone microbiology laboratories run by district, town or university hospitals, private laboratories and private laboratory companies operating either regionally or nationally. Approximately 150 and 50 laboratories perform mycobacterial culture and tuberculosis drug susceptibility testing (DST) respectively. Germany has an insurance-based health care system with fixed prices for diagnostic tests. Different prices are applied for in- and outpatients and for patient insured by standard or private insurance. Laboratories are at liberty to offer discount packages. The National Mycobacterium Reference Laboratory (NRL) was appointed by the Ministry of Health through consultation with the Robert Koch Institute. The NRL receives some federal funding and generates revenue through diagnostic procedures.

The NRL has been implementing a comprehensive and data-driven approach to improving quality management systems at the laboratory, starting in 2015 with a detailed laboratory review process. This review led to adoption of a phased implementation approach starting with introduction of QM and health and safety policies, and building renovations. In 2017, a laboratory information system (LIS) with audit trails, automated quality statistics and barcodes was introduced. Here we present our experiences of implementing a QMS, challenges and lessons learned and the impact of the interventions based on measurement of trends in key quality indicators before, during and after implementation of various aspects of the QMS.

Methods

National Reference Laboratory

The NRL receives 12 000–15 000 samples every year. The samples are referred by the regional referral hospital specializing in respiratory diseases (including tuberculosis and specifically drug resistant tuberculosis) (30%), hospitals and private practitioners regionally (15%) and other laboratories nationally (40%). The remaining 15% of samples are sent to the NRL in its capacity as a WHO (World Health Organisation) supranational reference laboratory from partner countries internationally. These samples are referred for DST as part of quality assurance for national drug resistance surveys or for second line DST for rifampicin resistant isolates from countries without local DST capacity. The NRL in partnership with INSTAND (Gesellschaft zur Förderung der Qualitätssicherung in medizinischen Laboratorien e.V) runs the twice-yearly mycobacterial EQAs for Germany and through a European Center for Disease Prevention and Control (ECDC) funded scheme for countries in the European Union.

The NRL employs a director, two scientists, seven full-time and two part-time technicians, two part-time laboratory assistants and two administrators.

Diagnostic procedures at the NRL include smear microscopy, solid and liquid mycobacterial culture, direct molecular tests to detect non-tuberculosis mycobacterial (NTM) DNA, Mycobacterium tuberculosis complex DNA and resistance conferring mutations, species identification, genotypic and phenotypic DST. The laboratory infrastructure includes biosafety level II and III laboratories (BSL II and III).[13]

Laboratory review

A series of laboratory reviews was conducted between October 2015 and January 2016, prompted by a change in laboratory leadership. Using a multipronged approach reviews of facilities, equipment, processes (purchasing, pre-analytic, analytic and post-analytic), staff, health and safety, documentation, information management and organization based on the ISO 15189 and the twelve quality system essentials were conducted.[14]

The newly appointed clinical laboratory director together with an external quality management consultant reviewed all available documents, directly observed sample processing and analytic procedures, interviewed staff working in the laboratory and at core facilities (human resources and central procurement). The results of the review were summarized in a narrative report and a list of priorities areas for actions was drawn up. This was presented to the institutional senior management resulting in additional resources for process optimization, infrastructure, staff and laboratory information and data management.

In addition, the laboratory manager (previous quality manager) of the English National Mycobacterial Reference Laboratory was invited for a 3-day visit to identify areas for improvement mainly focused on analytic processes and health and safety of employees. Findings were summarized in a supervisory report, presented both to senior management and laboratory staff. The institutional health and safety officer, together with the head of technical services and the clinical laboratory director undertook a detailed risk assessment aligned with the German biosafety regulations.[15] In addition, a ventilation engineer was asked to review the ventilation system and measure pressure differences and air exchanges.

Actions

Following the reviews, a quality manager and a quality management technician were employed. One of the technicians was promoted to acting laboratory manager and funded to undertake a two-year part-time course in laboratory management. As a result of the risk assessment and the ventilation engineer report, a laboratory planner and architect was commissioned to assess the feasibility of refurbishing existing facilities in order to comply with the 2013-biosafety regulations.

Because the baseline reviews had identified deficiencies across all aspects of the twelve quality management essentials, the decision was taken to address deficiencies in order of priority and urgency. Actions aimed at improving analytical quality, health and safety and information management were prioritised for initial implementation in parallel with each other.

Outcomes

Impact was measured using quality indicators across three analytic methods before and after implementing improvement activities: i) smear microscopy ii) liquid mycobacterial culture and iii) molecular DST. Patient population, referring hospitals and sample numbers were comparable across different time periods. The prevalence of positive cultures for mycobacteria (M. tuberculosis complex or non-tuberculous mycobacteria) was not significantly different before and after the intervention (p = 0.10).

The proportion of smear positive slides from respiratory samples and the proportion of false positive samples were calculated before, during and after implementing improvement activities. Differences in proportions were compared using χ² test. Odds ratios for smear positivity were calculated using the pre-implementation period as the baseline. Only the first respiratory sample of a patient was included in the analysis. A false positive smear was defined as a smear categorized as positive in treatment naïve patients without detection of mycobacterial DNA in the primary sample, no mycobacterial growth in liquid media after 6 weeks and solid culture after 8 weeks.

As part of the baseline review the proportions of contaminated liquid cultures were determined retrospectively over three consecutive months (February-May) and years (2013–2015) by extracting data manually from paper records In line with routine laboratory procedures positive liquid cultures were investigated using the standard Ziehl-Neelsen stain and light microscopy, sub-culture on Columbia blood agar and Loewenstein-Jensen slants and molecular diagnostics aimed at detecting non-tuberculosis mycobacterial or M. tuberculosis complex DNA. Our limited data collection allowed investigation into possible monthly and yearly variation while limiting the staff time required for the data extraction. Routine prospective collection of contamination rates as quality indicators was implemented in April 2016. According to German microbiology standards contamination rates were calculated for sputum samples only and restricted to treatment naïve patients, not known to have cystic fibrosis. [16] The proportion of contaminated liquid cultures was determined over time.

For each Genotype MTBDRplus (HAIN Lifescience, Nehren, Germany) run performed between 2013 and 2017 the number of samples tested and the result of the negative control (positive or negative) were extracted from laboratory worksheets. The proportion of “positive”negative controls was calculated for each year and compared using χ² test. Logistic regression was used to determine the odds ratios for PCR runs with evidence of DNA contamination (“positive” negative control) with 2013 serving as the baseline comparison.

All statistical analysis was performed using Stata version 14.2 (Texas, USA).

Results

Baseline reviews, actions and results

Table 1 summarizes the findings of the baseline reviews and the resulting actions across the twelve quality management essentials.

Review of facilities and safety resulted in several infrastructural changes of the existing building including additional emergency exits, smoke detectors and fire alarms. Some infrastructural deficiencies were addressed by optimising and standardising processes such as sample flow and delineating laboratory activities. To improve safety a waste management plan was introduced and appropriate equipment (autoclave boxes) and consumables were purchased, a detailed cleaning plan was developed and staff were required to wear gloves in the BSL III. The feasibility study revealed that refurbishment of the existing building would not suffice to comply with the 2013 biosafety regulations. As a result, senior management sourced and secured financing for a new BSL III. Building permission was granted in September 2018 and completion is planned for 2020.

Documentation of equipment, purchasing and inventory was limited. Processes and documentation were agreed and an equipment and baseline stock inventory completed. Lack of sample rejection criteria meant that all samples were processed regardless of suitability. Minimal requirements for quality of samples were established, users informed and scientists and technicians trained. Standardised operating procedures and documentation for validation and verification of new methods were introduced. In total five new CE-marked methods (CE: Conformité Européenne [European conformity] with health, safety, and environmental protection standards for products sold within the European Economic Area) have been verified up to December 2017, and one method (DST for bedaquiline, delamanid and clofazimine) has been validated. Up to 2017 the NRL participated in the national EQA scheme led by the NRL itself. From 2018 the NRL took successfully part in the United States College of American Pathologists (US CAP) EQA for microscopy, NAT, culture, identification and phenotypic DST.

The majority of staff had been trained at the NRL and never worked at or visited other laboratories. The concept of quality management was unfamiliar to technical, scientific and administrative staff. Staff were encouraged to participate in training and visit other laboratories with the aim to create an environment of continuous learning, promote critical thinking and motivate for sustained change. A total of 25 laboratory visits by individual staff members took place over a period of two years. External training days totalled 48 (excluding the acting laboratory manager’s training days). In addition, diagnostic companies were invited to conduct on-the-job trainings in small groups. A one-hour internal fortnightly teaching session was implemented to introduce quality management concepts such as corrective and preventive actions, discuss SOPs under development and inform staff about future plans and changes. The implementation of the LIS was accompanied by intensive theoretical and practical training (three group sessions, and three individual training sessions for each staff member).

Components of the QMS were developed using a bottom-up approach wherever possible.

Quality indicators were not established and given the largely paper-based documentation of laboratory results were difficult to collect and analyse in real time. Initially a limited set of quality indicators (culture contamination rates, positive microscopy and NAT rates, discordances in microscopy, NAT and culture results) was identified to introduce the concept of quality indicator monitoring and continuous process improvement. Once the LIS was implemented in October 2017 trends of quality indicators were analysed on a monthly basis.

SOPs for analytical processes did not exist. Staff were using product inserts to perform diagnostics procedures. Because none of the technicians had ever drafted a SOP, the quality manager, clinical laboratory director and quality management technician did the actual writing of SOPs and validated the content with other staff members. The development of SOPs followed an iterative process: observation of the analytic process by the quality management team, drafting of the SOP, actively seeking feedback from the technicians, amendments and edits, multiple feedback rounds followed by the approval of the final SOP. Alongside the introduction of standardised processes and improved documentation a document control system was set up. All laboratory staff and the members of the workers council (mandatory elected body comprising elected employees with mandate to protect working conditions, health and safety and workers’ rights) were trained on document control. Laboratory documents were made available to the workers’ council on request to check compliance with working directives. Thus members of the workers’ council had to understand and adhere to document control procedures.

In mid 2015, an in-house access based (Microsoft 2013) LIS was introduced replacing a completely paper-based system, where results were entered on hard-copy sample sheets (one sheet per sample) and reports typed and signed. The access based LIS had several shortcomings and did not comply with national regulations. Features such as technical verification, medical authorization, audit trails, direct import of laboratory results from analysers, automated registration of samples via an optical character recognition (OCR) scanner and automated invoicing were not available. Technicians continued to use paper-based sample worksheets duplicating reporting. A new LIS (MLab, Dorner, Muehlhausen, Germany), OCR scanners for sample registration and automated result import from analysers was introduced in October 2017. Despite limited computer literacy of technical staff, the BSL III was able to move to near paperless result reporting. Registration of samples became less time consuming and error prone. Automated and standardised invoicing resulted in an increase in revenue and reduction of administrative staff time freeing 0.3 full time equivalents of administrator time.

Smear microscopy

In 2015, smears were stained using a staining automate according to the manufactore´s instruction (Kreienbaum, Germany) and the Kinyoun staining method.[17] Briefly, the Kinyoun method refers to an acid-fast stain used to detect any species of the genus Mycobacterium. It involves the application of a primary stain (Carbol-Fuchsin), a decolorizer (acid-alcohol), and a counterstain (Armand Solution). In contrast to the classical Ziehl–Neelsen staining, the Kinyoun method does not require a heating step.

A review of the existing method revealed poor quality of smears and staining which included a high proportion of staining artefacts. As a result of the review it was decided to i) introduce fluorescence microscopy recommended by WHO [18] with a higher sensitivity [19] accompanied by changes such as the use of frosted slides and water filters to avoid contamination with environmental mycobacteria from tap water ii) remove the staining automate iii) develop SOPs iv) introduce and document batch controls of staining solution as well as positive and negative controls v) train technicians in smear preparation, staining and reading vi) assess competencies vii) routinely double-check any positive slide and viii) perform duplicate reading of ten randomly selected negative slides per week. These changes and the completion of all documents related to quality management of smear microscopy were implemented over a period of 3 months.

Before implementation of quality managed fluorescence microscopy the proportion of respiratory samples classified as smear positive was 3.3% (Fig 1). During the implementation and post-implementation period the proportion of respiratory samples testing smear positive increased to 6.2% and 6.6% respectively (p-value <0.01). The odds ratio for a sample to be tested as microscopically positive increased by 1.94 (95% CI 1.16–3.26, p = 0.012) and 2.08 (95%CI 1.41–3.06, p <0.001) comparing the time before the interventions with the scale-up phase and the time when quality managed fluorescence microscopy was fully implemented. The proportion of false positives was 0.02%, 0.01%, 0.02% before, during and post-implementation of the interventions respectively.

Culture contamination rates

Baseline reviews showed that contamination rates in previous years (2013–2015) exceeded the target of <10% in liquid media set by the national German microbiology standards (Fig 2).[16] This resulted in the implementation of several measures aimed at improving quality. First the results of the retrospective assessment were presented and discussed at one of the forth-nightly laboratory meetings. Contamination rates were selected as one of the first quality indicators to be monitored and data were extracted manually from paper records from April 2016 onwards. However, because the process was not automated contamination rates were not available in real-time.

In August 2016, a study comparing different decontamination methods (three commercially available methods and the in-house method) was conducted over a period of 10 weeks.[20] For each method short SOPs were developed, technicians were trained and following competency assessments they were observed performing the decontamination procedures. This led to a temporary reduction of contamination rates among routinely processed samples during the study period (Fig 2). Finally, in March 2017 the decision was taken to implement a CE-marked commercially available method based on N-Acetyl-L-Cysteine-Sodium Hydroxide (NALC-NAOH, MycoDDRTM, IMMY, Norman, USA). Following the implementation of a new LIS in August-September 2017, quality indicators and statistics were generated automatically by Structured Query Language (SQL) queries. Weekly meetings were held to review contamination rates with all staff members involved in processing primary samples. Together with the technicians performing the decontamination procedures prompts and aids were developed to ensure correct incubation times and sufficient vortexing. Graphs plotting contamination rates over time were displayed in the staff room. Over a period of two weeks, decontamination procedures were directly observed on five separate days by the acting laboratory manager. Contamination rates decreased from 23.6–27.6% in April-July 2016 to 10.7–17.4% in the same period in 2017 and fell under the 10% mark in November 2017-March 2018.

DNA contamination

In 2015, the Xpert MTB/Rif (Cepheid, USA) was the main molecular diagnostic used for detection of resistance referring mutations at the NRL. In contrast, the number of samples investigated using the Genotype MTBDRplus was limited, because DNA contamination was a frequently observed event. Strict separation of pre-PCR, PCR and post-PCR processes was difficult due to suboptimal building design and sample flow. Further challenges included non-directional airflow, inadequate staff training on how to prevent DNA contamination and a culture of austerity aimed at saving consumables and time. Over a period of several months, additional bench and storage space was built to allow separation of pre-PCR, PCR and post-PCR processes. In addition, room access with regards to whom (staff) and when (time of the day) and cleaning procedures were discussed with scientific and technical staff and agreed upon. Any PCR run with a “positive” negative control was brought to the attention of senior management and the run was repeated. If still positive on the repeat run, intensive cleaning was conducted, new reagents and pipettes were sourced.

Between 2013 and 2017 a total of 610 individual Genotype MDRplus diagnostic runs were performed investigating a total of 1780 samples. The median number of samples tested per run increased from one in 2013 and 2014 to two, three and five in 2015, 2016 and 2017 respectively (Table 2), while the proportion of negative controls showing evidence of DNA contamination decreased from 38.2% in 2013 to 8.1% in 2017 (p value <0.01). The odds ratio for a run with evidence of DNA contamination was 0.91 (95%CI 0.53–1.57) for 2014 compared to 2013, but decreased for the subsequent years to 0.42 (95%CI 0.23–0.75), 0.28 (95%CI 0.15–0.54) and 0.14 (95%CI 0.07–0.29).

Of the 11 runs with “positive” negative controls in 2017 seven occurred in July and August (Fig 3). These triggered corrective and preventive actions including repeat testing of the clinical specimens, replacement of pipettes and mastermix and intensive cleaning. These measures decreased DNA contamination immediately (4.1%; 95%CI 0.49–13.98). Excluding the months of July and August DNA contamination was evident in 3.6% (95%CI 0.98–8.89) runs in 2017.

Discussion

This study demonstrates the impact of quality management in a National TB reference laboratory in a high-income country across different methods: smear microscopy, culture and molecular diagnostics. Implementation of quality managed fluorescence microscopy doubled the yield in respiratory samples, while contamination rates in liquid media decreased significantly from over 20% at the beginning of this study to less than 10% during the most recent follow-up. Retrospective data collection showed evidence of DNA contamination in more than a third of PCR runs in 2013 and 2014. The odds of DNA contamination decreased by 85% comparing 2013 data with 2017 data.

While QMS implementation has not been completed and the laboratory is not yet accredited, the results show significant improvements due to introduction of quality management processes. In parallel with the implementation of a QMS and LIS finances have been secured to build a new BSL III. The decision has been taken to defer accreditation until the new laboratory has been completed. This is currently planned for 2020. Publication of our experience and these findings at this time prior to reaching accreditation serves two purposes. First our data-driven approach by implementing measurement of the impact of QMS using trend analysis of quality indicators has already demonstrated substantial benefit. The analysis has been a key enabler to motivate for the additional investment needed. Secondly, although accreditation is our eventual aim, laboratories considering improvement of QMS can follow our approach with or without accreditation as the final goal.

Success in attaining sustained improvement cannot be attributed to a single measure. A combination of committed senior leadership, significant financial resources and a bottom-up approach was crucial for these achievements. In addition, a non-blame culture of continuous improvement has resulted in enhanced staff competency and fostered their ability of critical appraisal, improved operational consistency and reliability and teamwork. The baseline review included interviews with all staff members and revealed that most of them were afraid to make any mistakes and even more of being caught. A two-day team building exercise in February 2016 showed that lack of skill and understanding of analytic procedures impeded effective teamwork and communication. Therefore, a special emphasis was placed on training, exposure to other laboratories and communication through in-house seminars and daily staff meetings.

Unfortunately, no formal staff feedback was sought. However, a study conducted in three laboratories in the United Kingdom reporting that the majority of laboratory technicians thought the documentation required for accreditation increased the workload without improving quality of test results.[21] In contrast, laboratory managers and clinicians in the United Kingdom and South Korea felt that accreditation resulted in better laboratory performance with more documentation, better health and safety and training procedures and improved infrastructure.[22, 23]

There seems widespread agreement that laboratory quality management is important for patient safety and impacts on quality of clinical care.[4, 11] However, studies reporting on the impact of full-scale implementation of QMS and/or accreditation on laboratory performance are relatively few.[9, 10, 24–26] We have recently conducted a systematic review showing that quality improvement measures had indeed a measurable impact in TB laboratories in low- and lower-middle income countries.[27] Only one of the studies included in the review reported on the effect of implementing a comprehensive QMS followed by accreditation.[9] All other studies described the effect of various activities aimed at improving quality such as EQAs, supervisory visits and staff training.

One of the main issues of measuring the effect of QMS is the measurement of quality before implementing QMS. Data collection on quality indicators requires partial QMS implementation. This in turn makes it difficult to assess the full impact of QMS on quality indicators.

In this study, pre-intervention data were collected retrospectively. Hence data collection of quality indicators itself could not have impacted on performance. Outcomes in this study were analyzed using an uncontrolled before-and-after comparison, which is always vulnerable to coincidental time trends. However, impact was measured across three different analytic tests over a prolonged time period. Thus changes due to seasonal variations or differences in patient population are unlikely to explain the significant differences detected over time.

Unfortunately, we did not collect any data on cost and hence were unable to calculate cost-effectiveness or report on any cost-savings.

Over the past decade the need to improve laboratory quality in low and middle-income countries has been widely acknowledged.[28–30] The Strengthening Laboratory Management Toward Accreditation (SLMTA) initiative launched in 2009 has had a substantive impact on provision of quality laboratory services and patient care.[31–34] However, this study shows that quality management is effective regardless of low or high resource settings. It serves as a stark reminder that while quality management is worth our while it is cost and resource intensive and requires full commitment from senior management.