==== Front Am J Clin Pathol Am. J. Clin. Pathol ajcp American Journal of Clinical Pathology 0002-9173 1943-7722 Oxford University Press US 29931081 10.1093/ajcp/aqy047 aqy047 Original Articles Featured Clinical Referral Laboratories in Rwanda The Status of Quality Improvement After 7 Years of the SLMTA ProgramRusanganwa Vincent MD, MPH124 Gahutu Jean Bosco MD, PhD1 Nzabahimana Innocent MSc5 Ngendakabaniga Jean Marie Vianney MSc6 Hurtig Anna-Karin MD, PhD3 http://orcid.org/0000-0001-6034-4807Evander Magnus PhD2 1 College of Medicine and Health Sciences, University of Rwanda, Kigali, Rwanda 2 Department of Clinical Microbiology, Virology, Umeå University, Umeå, Sweden 3 Department of Public Health and Clinical Medicine, Epidemiology and Global Health, Umeå University, Umeå, Sweden 4 Ministry of Health, Kigali, Rwanda 5 National Referral Laboratory, Kigali, Rwanda 6 Butare University Teaching Hospital, Huye, Rwanda Corresponding author: Magnus Evander, PhD, Dept of Clinical Microbiology, Virology, Umeå University, SE-901 85 Umeå, Sweden; magnus.evander@umu.se 7 2018 19 6 2018 19 6 2018 150 3 240 245 © American Society for Clinical Pathology, 2018.2018This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is properly cited. For commercial re-use, please contact journals.permissions@oup.comAbstract Objectives We investigated the quality system performance in Rwandan referral laboratories to determine their progress toward accreditation. Methods We conducted audits across five laboratories in 2017, using the Stepwise Laboratory Quality Improvement Process Towards Accreditation checklist. Laboratories were scored based on the World Health Organization grading scale (0-5 stars scale) and compared with earlier audits. Results Between 2012 and 2017, only one laboratory progressed (from four to five stars). Four of the five laboratories decreased to one (three laboratories) and zero (one laboratory) stars from four and three stars. Management reviews, evaluation, audits, documents, records, and identification of nonconformities showed a low performance. Conclusions Four of five laboratories are not moving toward accreditation. However, this target is still achievable by energizing responsibilities of stakeholders and monitoring and evaluation. This would be possible because of the ability that laboratories showed in earlier audits, coupled with existing health policy that enables sustainable quality health care in Rwanda. LaboratoryAccreditationRwandaSLMTAHealth systemQuality health careSwedish International Development Cooperation Agency10.13039/10000444151160027-0451160059-10 ==== Body Clinical laboratories are vital in health systems and play a critical role in the quality of health care.1-3 Accurate and reliable laboratory test results are key factors in guiding decision making and evidence-based actions for effective prevention, control, and surveillance of diseases as well as patient management.1,2,4-8 Laboratories in low-income countries, especially in sub-Saharan Africa, face a challenge of insufficient resources for quality improvement, despite their crucial role in supporting the health system.9-11 In addition, the region has the lowest rate of international accredited laboratories and the highest burden of disease.11-14 This situation creates an imbalance in terms of demand and provision of reliable laboratory services.15 The compliance with standards of the quality of health care is verified through the international accreditation of health facilities, including clinical laboratories.5,16,17 To respond to the need of quality laboratory services as well as working toward international accreditation in low-income countries, a World Health Organization (WHO)–Africa region stepwise approach has been adopted as an effective strategy to enhance health care quality.8,18,19 The approach is formed by the Strengthening Laboratory Management Towards Accreditation (SLMTA) program. The program is based on a series of trainings and mentorship of laboratory staff in laboratory quality management system improvement. The progress in quality improvement is measured using a Stepwise Laboratory Quality Improvement Process Towards Accreditation (SLIPTA), a checklist tailored on International Organization for Standardization (ISO) 15189.19-22 Since December 2017, 49 countries, including Rwanda, have implemented the SLMTA program,23 and by 2014, 617 laboratories from 47 countries were enrolled in the program.24 Published data of baseline and exit audits from these countries are encouraging.24-26 Here, the baseline and exit audits are performed before and after (respectively) a series of training in SLMTA and mentorship in quality improvement projects implementation, whereas follow-up audits are performed after this period. In addition to the quality improvement registered between baseline and exit audits, laboratories are recommended to continuously apply laboratory quality systems to achieve international accreditation.27 To be eligible for international accreditation, a laboratory in the SLMTA program is required to score 95% or more out of 275 points on the SLIPTA checklist audit.22,28 This score is equivalent to five stars on the WHO grading scale of 0 (<55%) to five stars (≥95%), and the awarded score in this WHO system is valid for 2 years.28 The Ministry of Health of Rwanda started a process of quality improvement for its health facilities to achieve their accreditation.29,30 On the front line of the process, there were four national referral hospitals alongside their respective laboratories, as well as the national reference laboratory, with the mandate of supporting their subordinate ones. Therefore, the quality performance of these hospitals and laboratories will influence those under their supervision. In Rwanda, the SLMTA program started in January 2010 with a baseline audit of these five clinical referral laboratories as a first cohort.25 This was followed by exit and follow-up audits, respectively, in 2011 and 2012.25 Then, progressively, other laboratories were enrolled with the ultimate goal of covering all hospital laboratories in Rwanda.25,31 These five clinical referral laboratories demonstrated quality system improvement between baseline and exit as well as in the follow-up audits.25 However, no available data after 2012 evaluate quality improvement toward accreditation. Such information is needed to evaluate the progress and guide new strategies for sustainability and further improvement. The aim of our study, 7 years after the start of the program and 5 years after the follow-up audit, was to assess the level of quality performance of the five clinical referral laboratories in Rwanda. The study would determine whether they are progressing toward international accreditation or if they are declining. The findings of the study will inform policy and practice on strengths and weaknesses to work on toward laboratory accreditation and quality of health care improvement. Materials and Methods Study Design To determine the quality performance of laboratories, we performed a cross-sectional systemic audit using the SLIPTA checklist (version 2015)22 across all five clinical referral laboratories, and their performances were compared with previous audits held in 2010, 2011, and 2012. Settings The five clinical referral laboratories (laboratories 1-5) are composed of four laboratories belonging to national referral and teaching hospitals and one national reference laboratory (NRL). The NRL supervises the four and performs quality control of laboratory tests from district laboratories (especially human immunodeficiency virus, malaria, and tuberculosis tests), as well as disease surveillance and outbreak investigation. Data Collection Audits were conducted in April and August 2017 by authors qualified and certified by the WHO, US Centers for Disease Control and Prevention (CDC), and African Society of Laboratory Medicine (ASLM) in SLMTA and use of the SLIPTA checklist. We conducted the audits in teams of three persons for each laboratory, but no one audited his own institution. Prior to the audit, the team met for 1 day to review the tool and agree on the approach of assessment. We visited laboratories and discussed with laboratory leadership and staff to inform them on the research. We assessed each laboratory over 3 days through observation of processes and operations as well as working environment through reviewing documents and records, following specimens through the laboratory, interviewing relevant staff for additional clarifications, and other procedures as described in the tool.22 Analysis We scored each laboratory with the WHO grading scale of zero to five stars for a percentage of performance (<55%, 55% to <65%, ≥65% to <75%, ≥75% to <85%, ≥85% to <95%, and ≥95%, respectively). The overall performance of each laboratory was calculated by summing the performance of 12 different SLIPTA sections, which are based on 12 quality system essentials, and the percentage was calculated for comparison. The performance in the 12 sections was compared within each laboratory and between all laboratories. The trend of performance since 2010 was made using the 2017 score as well as the scores of 2010, 2011, and 2012. Charts were made with GraphPad Prism 7 (GraphPad Software, La Jolla, CA) and Microsoft Excel, version 2016 (Microsoft, Redmond, WA). Ethical Considerations The research proposal was presented, ethical clearance (No. 0059/RNEC/2017) was obtained from the Rwanda National Ethics Committee, and approval was acquired from the Ministry of Health and Ministry of Education prior to its implementation. Additional permissions were requested from leadership of institutions to which the evaluated laboratories belonged. The research project was presented and discussed in study sites prior to data collection, and feedback was provided at the end of the assessment. Results The findings are displayed in three sections: the overall performance of the laboratories in 2017, the quality system performance in 12 quality system essentials, and the evolution of quality performance of laboratories from 2010 to 2017. Overall Laboratory Performance in 2017 All five laboratories were enrolled in the SLMTA program as the first cohort in Rwanda. The cohort was audited with the SLIPTA tool in 2010 (baseline), 2011 (exit), and 2012 (follow-up), and we performed our audit in 2017 using the SLIPTA tool. The findings of the 2017 evaluation showed that the laboratory that performed best scored 95.3%, which corresponds to five stars on the WHO grading scale. The laboratory that showed the lowest performance scored 52%, which is equivalent to zero stars. The remaining three laboratories reached the one-star level Figure 1. Figure 1 Overall quality performance of referral laboratories in Rwanda assessed in 2017 using the Stepwise Laboratory Quality Improvement Process Towards Accreditation checklist. Performance in 12 Quality System Essentials For quality system essentials, our assessment found that certain determinants influenced the overall result. Management reviews, evaluations and audits, documents and records, and identification of nonconformities showed a low performance in the laboratory quality system in four laboratories. The facility and biosafety, purchasing and inventory, client management, and customer services as well as occurrence and incident management had a positive trend in almost all of the laboratories. One laboratory showed good performance in all quality system essentials Figure 2. Figure 2 Quality performance in 12 quality system essentials of referral laboratories in Rwanda assessed using the Stepwise Laboratory Quality Improvement Process Towards Accreditation checklist in 2017. Evolution of Quality Performance From 2010 to 2017 When comparing the performance of the latest follow-up in 2012 and our evaluation in 2017, the current findings indicated progress of only one of the laboratories, which scored four stars in 2012 and currently scored five stars. The remaining four laboratories declined in their performance, as three of them scored one star and one scored zero stars, while they scored four and three stars in 2012, respectively Figure 3. Figure 3 Evaluation of quality performance of referral laboratories in Rwanda, 2010 to 2017. Discussion The Health Sector Policy and the Health Sector Strategic Plan of Rwanda emphasize the quality of health care as their major priority.29,30 The accreditation of health facilities, including laboratories, is highlighted as the strategy to achieve the desired quality health care status.29,30 Therefore, the Ministry of Health enrolled in the first cohort of SLMTA in 2010 its five clinical referral laboratories as a process toward accreditation and quality improvement. This study showed the quality system performance of the laboratories in 2017, 7 years since 2010, when the program started, and 5 years since the last audit in 2012. The 2017 audits showed that one laboratory scored zero stars and three others one star, whereas one laboratory scored five stars. When comparing with the previous audits, we showed that 5 years after the latest stepwise approach toward accreditation, most of the Rwandan clinical referral laboratories (four of five) had decreased in quality performance. While the general performance decreased for the four laboratories, some common patterns like management review, document and records, identification of nonconformities, and evaluation and audits critically affected the performance of the laboratories.26 These patterns are cornerstones of the entire laboratory quality improvement system. The good or poor performance in these cornerstones affects positively or negatively the quality system, respectively. The fact that most of these laboratories were affected systematically in the same quality system essentials suggests system problems that need to be identified and addressed. The 2011 exit audit, which marked the end of SLMTA trainings, had shown improvement in quality performance in all laboratories compared with the baseline of 2010.25 The follow-up audit performed in 2012 also showed continuous improvement in all five laboratories.25 The audit of 2012 for five laboratories was planned as a follow-up of the 2011 one and after 12 months of addressing identified gaps. Here, we showed that this dynamic coordinated action of identifying gaps and planning how to address them led to positive results. The continuous quality improvement score of the high-performing laboratory could be explained by the fact that the laboratory is enrolled in the East Africa Public Health Laboratory Networking Project, which provides financial support and regular audits that helped the laboratory to identify and address gaps.31 The four other laboratories had each evolved on their own since 2012. Two laboratories (laboratory 2 and laboratory 3) had only occasional SLMTA/SLIPTA activities, whereas the two remaining laboratories (laboratory 1 and laboratory 4) had no SLMTA/SLIPTA-related activity in the same period. Reevaluating and applying lessons learned in the circle of planning as well as effective monitoring are the gold standards to achieve the desired quality of health care. Our findings corroborated with this principle by the fact that most of the assessed laboratories were declining in their quality performance, and insufficient follow-up was one of the contributing factors. Further research exploring factors regulating the observed quality performance status of clinical referral laboratories in Rwanda may find additional factors. Rwanda is well known to be effective and efficient in implementing sociodevelopment programs, especially health-related reforms to improve population health outcomes.32-35 Therefore, it should be possible for these laboratories to achieve and sustain international accreditation and then mentor lower-level laboratories. This possibility is based on laboratory ability demonstrated at the beginning of SLMTA program and the political commitment of the country. It requires a redefinition of responsibilities among national stakeholders and establishment of regular audits as well as an effective monitoring and evaluation framework. The clinical referral laboratories of Rwanda may not be an isolated case among more than 617 laboratories enrolled in SLMTA in different countries.24 While a national coordination and effective monitoring and evaluation mechanisms are recommended at the country level, such regional monitoring needs to be streamlined for effective synergy toward international accreditation and quality of health care as a target of the third sustainable development goal, with “no one left behind.”36,37 Even if the findings from this study can provide some insight for other countries with a similar context, the results cannot be generalized. Furthermore, readers of this article could also keep in mind that the audit of 2017 used the 2015 SLIPTA version based on ISO 15189:2012, while previous audits used the version of SLIPTA aligned to ISO 15189:2007. However, the difference in these two versions is too minor (17 points), which cannot explain the observed difference in quality performance while principles remained the same. Conclusions This study demonstrated that almost all assessed laboratories decreased in quality performance and, therefore, did not achieve the level of accreditation. Our findings highlighted that management reviews, evaluation and audits, documents and records, and identification of nonconformities constituted the bottlenecks that severely affected the quality system of the laboratories. In addition, the study showed the need for regular follow-up, periodic retraining, and external audits to support laboratory quality improvement in low-income countries. Based on the political will of Rwanda and the ability demonstrated by the assessed laboratories in earlier audits, their international accreditation is possible by energizing the responsibilities of national stakeholders and setting up a monitoring and evaluation mechanism. Acknowledgments We are grateful to Olivia Wesula Lwande for valuable insights on the study and reviewing of the manuscript. This work was supported by the Swedish International Development Cooperation Agency (grants 51160027-04 and 51160059-10). ==== Refs References 1. Kubono K Quality management system in the medical laboratory—ISO15189 and laboratory accreditation . Rinsho Byori . 2004 ;52 :274 -278 .15137330 2. Peter TF , Rotz PD , Blair DH , et al Impact of laboratory accreditation on patient care and the health system . Am J Clin Pathol . 2010 ;134 :550 -555 .20855635 3. Guzel O , Guner EI ISO 15189 accreditation: requirements for quality and competence of medical laboratories, experience of a laboratory I . Clin Biochem . 2009 ;42 :274 -278 .19863920 4. Skeels MR Toward a national laboratory system for public health . Emerg Infect Dis . 2001 ;7 :531 -532 .11485648 5. Plebani M , Sciacovelli L ISO 15189 accreditation: navigation between quality management and patient safety . J Med Biochem . 2017 ;36 :225 -230 .30564060 6. Da Rin G Pre-analytical workstations: a tool for reducing laboratory errors . Clin Chim Acta . 2009 ;404 :68 -74 .19302988 7. Forsman RW Why is the laboratory an afterthought for managed care organizations ?Clin Chem . 1996 ;42 :813 -816 .8653920 8. Alemnji GA , Zeh C , Yao K , et al Strengthening national health laboratories in sub-Saharan Africa: a decade of remarkable progress . Trop Med Int Health . 2014 ;19 :450 -458 .24506521 9. Nkengsong J , Boeras DI , Abimiku A , et al Assuring the quality of diagnostic testing: the future is now . Afr J Lab Med . 2016 ;5 :558 .28879134 10. Nkengasong JN. A Shifting Paradigm in Strengthening Laboratory Health Systems for Global Health: Acting Now, Acting Collectively, but Acting Differently . Oxford, UK : Oxford University Press ; 2010 . 11. Petti CA , Polage CR , Quinn TC , et al Laboratory medicine in Africa: a barrier to effective health care . Clin Infect Dis . 2006 ;42 :377 -382 .16392084 12. Murray CJ , Lopez AD ; World Health Organization The Global Burden of Disease: A Comprehensive Assessment of Mortality and Disability From Diseases, Injuries, and Risk Factors in 1990 and Projected to 2020: Summary . Boston, MA: Harvard School of Public Health; 1996 . 13. Barbé B , Yansouni CP , Affolabi D , et al Implementation of quality management for clinical bacteriology in low-resource settings . Clin Microbiol Infect . 2017 ;23 :426 -433 .28506781 14. Schroeder LF , Amukele T Medical laboratories in sub-Saharan Africa that meet international quality standards . Am J Clin Pathol . 2014 ;141 :791 -795 .24838322 15. Davies J , Abimiku A , Alobo M , et al Sustainable clinical laboratory capacity for health in Africa . Lancet Glob Health . 2017 ;5 :e248 -e249 .28108137 16. Zima T Accreditation of medical laboratories—system, process, benefits for labs [published online July 14, 2017] . J Med Biochem . 17. Pouliakis A , Athanasiadi E , Karakitsou E , et al ISO 15189: 2012 management requirements for cytopathology laboratory information systems . Int J Reliable Qual E-Healthcare . 2014 ;3 :37 -57 . 18. Yao K , Luman ET , Nkengasong JN , et al The SLMTA programme: transforming the laboratory landscape in developing countries: lessons from the field . Afr J Lab Med . 2016 ;3 :1 -8 . 19. Gershy-Damet GM , Rotz P , Cross D , et al The World Health Organization African region laboratory accreditation process: improving the quality of laboratory systems in the African region . Am J Clin Pathol . 2010 ;134 :393 -400 .20716795 20. International Organization for Standardization (ISO) . 15189: 2012 Medical laboratories—Requirements for Quality and Competence . Geneva, Switzerland : ISO ; 2012 . 21. Yao K , McKinney B , Murphy A , et al Improving quality management systems of laboratories in developing countries: an innovative training approach to accelerate laboratory accreditation . Am J Clin Pathol . 2010 ;134 :401 -409 .20716796 22. World Health Organization–Regional Office for Africa . Stepwise Laboratory Quality Improvement Process Toward Accreditation . 2 nd ed. Brazaville, Congo : World Health Organization ; 2015 . 23. Newsletter: 2017 updates SLMTA 2017 www.slmta.org. Accessed March 20, 2018. 24. Yao K , Luman ET Evidence from 617 laboratories in 47 countries for SLMTA-driven improvement in quality management systems . Afr J Lab Med . 2016 ;3 :1 -11 . 25. Nzabahimana I , Sebasirimu S , Ruzindana E , et al Innovative strategies for a successful SLMTA country programme: the Rwanda story . Afr J Lab Med . 2016 ;3 :1 -6 . 26. Luman ET , Yao K , Nkengasong JN A comprehensive review of the SLMTA literature part 1: content analysis and future priorities . Afr J Lab Med . 2016 ;3 :1 -11 . 27. Ndihokubwayo JB , Maruta T , Ndlovu N , et al Implementation of the World Health Organization Regional Office for Africa stepwise laboratory quality improvement process towards accreditation . Afr J Lab Med . 2016 ;5 :280 .28879103 28. World Health Organization–Regional Office for Africa . WHO Guide for the Stepwise Laboratory Improvement Process Towards Accreditation in the African Region . Brazzaville, Congo : WHO ; 2015 . 29. Rwanda Ministry of Health . Health Sector Policy . 2 nd ed. Kigali, Rwanda : Ministry of Health ; 2015 . 30. Rwanda Ministry of Health . Third Health Sector Strategic Plan . Kigali, Rwanda : Ministry of Health of Rwanda ; 2012 . 31. Kumar M , Lehmann J , Rucogoza A , et al East Africa Public Health Laboratory Networking Project: Evaluation of Performance-based Financing for Public Health Laboratories in Rwanda . Washington, DC : The World Bank ; 2016 . 32. Binagwaho A , Farmer PE , Nsanzimana S , et al Rwanda 20 years on: investing in life . Lancet . 2014 ;384 :371 -375 .24703831 33. Abbott P. Millennium Development Goals Progress Report: Rwanda Country Report 2010 . Kigali, Rwanda: Institute of Policy Analysis and Research (IPAR); 2011 . 34. Farmer PE , Nutt CT , Wagner CM , et al Reduced premature mortality in Rwanda: lessons from success . BMJ . 2013 ;346 :f65 .23335479 35. National Institute of Statistics of Rwanda . Rwanda Demographic and Health Survey , 2014-2015 Edition. Kigali, Rwanda : National Institute of Statistics of Rwanda ; 2016 . 36. World Health Organization . World Health Statistics 2016: Monitoring Health for the SDGs, Sustainable Development Goals . Geneva, Switzerland : World Health Organization ; 2016 . 37. Murray CJL Choosing indicators for the health-related SDG targets . Lancet . 2015 ;386 :1314 -1317 .26460761