Received 24 March 2016; accepted 24 April 2016; published 27 April 2016

ABSTRACT

Background: The incidence of in-hospital adverse events is about 10%, with a majority of these related to surgery, and nearly half considered preventable events. In attempts to improve patient safety, the World Health Organization (WHO) developed a checklist to be used at critical perioperative moments. This meta-analysis examines the impact of the WHO surgical safety checklist (SSC) on various patient outcomes. Methods: A comprehensive search of all published studies assessing the use of the WHO SSC in patients undergoing surgery was conducted. Studies using the WHO SSC in any surgical setting, with pre-implementation and post-implementation outcome data were included. The incidence of patient outcomes (total complications, surgical site infections, unplanned return to the operating room (OR) within 30 days, and overall mortality) and adherence to safety measures were analyzed. Results: 10 studies involving 51,125 patients (27,490 prior to implementation and 23,635 after implementation of the WHO SSC) were analyzed. The implementation of the WHO SSC significantly reduced the risk of total complications by 37.9%, surgical site infections by 45.5%, unplanned return to OR by 32.1%, and mortality by 15.3%. Increased adherence to safety measures including airway evaluation, use of pulse oximetry, prophylactic antibiotics when necessary, confirmation of patient name and surgical site, and sponge count was also observed. Conclusions: The use of the WHO SSC is associated with a significant reduction in post-operative complication rates and mortality. The WHO SSC is a valuable tool that should be universally implemented in all surgical centers and utilized in all surgical patients.

Keywords:

World Health Organization, Surgical Checklist, Safety Checklist, Patient Safety

1. Introduction

Approximately 321 million surgical procedures are performed annually throughout the world, or approximately one operation annually for every 25 people [1] - [3] . The incidence of in-hospital adverse events is about 10%, with a majority of these related to surgery, and nearly half of these considered preventable events [3] . Serious, preventable events, termed “never events” continue to occur, and it is estimated that 500 wrong site surgeries and 5,000 retained surgical items occur in the United States (US) annually [4] .

In attempts to improve overall patient safety, the World Health Organization (WHO) developed a checklist, consisting of 22-items to be used at critical perioperative moments: induction, incision, and before leaving the operating room (OR) [3] . Prior to induction of anesthesia, the “sign-in” checklist confirms the patient’s identity, surgical site and procedure to be performed, ensures the anesthesia machine is functioning and medication is correct, ensures the pulse oximeter is on the patient, and assesses the patients allergies, risk for difficult airway and aspiration, and risk for extensive blood loss [3] . Prior to skin incision, “time-out” items include introducing team members by name and role, confirming the patients name and surgical procedure, ensuring prophylactic antibiotics have been administered within the last hour if applicable, ensuring essential imaging is displayed, and a discussion on anticipated length of surgery and estimated blood loss [3] . Before the patient leaves the OR, “sign-out” items consist of a complete instrument, sponge, and needle count, ensures specimens are properly labelled, and addresses any key concerns for patient management and recovery [3] .

Haynes et al. (2009) published the first study evaluating the effectiveness of the WHO surgical safety checklist [5] . In a multicenter study involving 8 worldwide medical centers and a total of 7,688 patients (3733 pre-im- plementation and 3955 post-implementation), significant reductions in major postoperative complications (11.0% vs. 7.0%, p < 0.001), surgical site infections (SSI) (6.2% vs. 3.4%, p < 0.001), and mortality (1.5% vs. 0.8%, p = 0.003) were observed [5] .

Despite the benefits reported, compliance with the safety checklist has been low [6] [7] . Fourcade et al. reported a checklist completion rate of only 61%, while other studies have reported completion rates as low as 21% [8] - [10] . Levy et al. observed compliance rates among 142 pediatric surgical cases over a 7-week period, and reported that none of the cases completed the entire checklist, which was significantly lower than the hospital reported 100% compliance rate [6] . Time-out was completed in 97% of cases and confirmation of patient name and case was completed in 96% of cases; however, the patients wristband was checked in only 4% of cases and incision site was only confirmed in 32% of cases [6] .

Bergs et al. conducted a meta-analysis including 6 studies involving 40,711 patients (17,920 patients pre- implementation and 22,791 patients post-implementation) and demonstrated significant overall reductions in overall complications (RR = 0.59; 95% CI, 0.47 - 0.74; p < 0.001), SSIs (RR = 0.57; 95% CI, 0.41 - 0.79; p < 0.001), and overall mortality (RR = 0.77; 95% CI, 0.60 - 0.98; p = 0.035) following the implementation of a surgical safety checklist [11] .

Several more recent studies not included in the meta-analysis by Bergs et al. have subsequently been published with conflicting results. Biskup et al. conducted a study in the United States involving 4,476 patients undergoing plastic surgery (2166 patients before implementation and 2,310 patients after implementation) and reported no significant reductions in postoperative complications (5.8% vs. 6.0%, p = 0.830) or mortality (0.04% vs. 0.05%, p = 0.549) with the use of the WHO surgical safety checklist in plastic surgery [12] .

This meta-analysis provides an updated comprehensive perspective on the impact of the WHO surgical safety checklist on the incidences of overall complications, SSIs, unplanned return to the OR within 30 days, and overall mortality.

2. Materials and Methods

2.1. Study Selection

A comprehensive search of all published studies evaluating the use of the WHO surgical safety checklist in patients undergoing surgery was conducted using PubMed, Cochrane Central Registry of Controlled Trials, and Google Scholar from the time the WHO surgical safety checklist was introduced to the current time (2008-2016) (Figure 1). Additional citations were searched, using the references of the articles retrieved from prior publications. The last search was conducted on January 18, 2016, and only articles written in English were considered. Keywords used in the search included combinations of “World Health Organization”, “WHO”, “surgical checklist”, and “safety checklist”. Inclusion criteria included the use of the WHO surgical safety checklist in its original form (without modifications) in any surgical setting, with pre-implementation and post-implementation outcome data. In case of duplicate publications, only the most recent and updated report of the clinical trial was included. This study was conducted according to Preferred Reporting Items for Systematic Reviews and Meta- Analyses (PRISMA) guidelines.

2.2. Data Extraction

Articles retrieved from the searches were assessed for eligibility, and data pertaining to patients, intervention, control groups, outcomes, and methodology, were abstracted.

Primary clinical outcomes of interest included the incidences of various patient outcomes-total complications, SSIs, unplanned return to the OR within 30 days, and overall mortality. Total complications were defined as complications occurring prior to hospital discharge or the first 30 days of hospital stay according to the American College of Surgeons’ National Surgical Quality Improvement Program: acute renal failure, bleeding requiring the transfusion of four or more units of red cells within the first 72 hours after surgery, cardiac arrest requiring cardiopulmonary resuscitation, coma of 24 hours or longer, deep vein thrombosis, myocardial infarction, unplanned intubation, ventilator use of 48 hours or more, pneumonia, pulmonary embolism, stroke, major disruption of wound, SSI, sepsis, septic shock, systemic inflammatory response syndrome, unplanned return to the OR, vascular graft failure, and death [13] .

Adherence to safety measures (airway evaluation, use of pulse oximeter, presence of catheter lines, prophylactic antibiotics, confirmation of patient and surgical site, and sponge count) were also analyzed.

2.3. Statistical Analysis

For each trial, relative risk (RR) with a 95% confidence interval (95% CI) for incidence of total complications, SSIs, unplanned return to the OR within 30 days, and overall mortality were calculated. RR and 95% CI for

adherence to each safety measure (airway evaluation, use of pulse oximeter, presence of intravenous line, prophylactic antibiotics, confirmation of patient and surgical site, and sponge count) were also calculated. Meta- analysis of the pooled data was performed using the Comprehensive Meta-Analysis software Version 3 (Biostat, Englewood, NJ, USA). For studies reporting zero events in any group, a continuity correction factor of 0.5 was adopted to calculate the RR and variance. In the event of zero events in both groups, the RR was not calculable and the study was excluded from the meta-analysis. Both the fixed-effects model and random-effects model were considered, depending on the heterogeneity of the included studies. To assess the heterogeneity between studies, both Cochrane’s Q statistic and I² statistic was used. Heterogeneity was considered statistically significant when p < 0.05 or I² > 50. If heterogeneity was observed, data was analyzed using a random-effects model. Conversely, in the absence of heterogeneity, a fixed-effects model was assumed. Risk of bias among the included studies was assessed using the Cochrane Collaboration Risk of Bias tool. Publication bias regarding the primary outcome (overall mortality) was first visually evaluated by a funnel plot, and further evaluated using Egger’s and Begg’s tests. A two-tailed p-value of < 0.05 was considered statistically significant. Subgroup analysis was performed based on country economic income status?upper economic income countries (with a gross national income (GNI) per capita of over $12,736 USD) versus lower/middle economic income countries (GNI per capita of $1046 - $4125 USD), as defined by country economic income according to the World Bank [14] .

3. Results

3.1. Demographic Characteristics of the Studies

A total of 10 studies meeting the inclusion criteria were identified. The 10 studies involved a total of 51,125 patients (Table 1). 27,490 of the patients were enrolled prior to the implementation of the WHO surgical safety checklist and 23,635 patients were enrolled following the implementation of the WHO surgical safety checklist.

3.2. Effect of the WHO Surgical Safety Checklist on Total Complications

Data on the incidence of total complications were reported in 8 trials, involving 25,450 patients (13,039 pre-im- plementation and 12,411 post-implementation). There were fewer complications following the implementation of the WHO surgical safety checklist (1,078/12,411 [8.7%] vs. 2,047/13,039 [15.7%]). There was significant heterogeneity between trials (p < 0.001, I² = 77.325), and a random-effects model was assumed. Meta-analysis showed a significant reduction in the risk of complications by 37.9% (RR = 0.621; 95% CI, 0.519 - 0.742; p < 0.001) (Figure 2).

There was a significant reduction in the risk of complications following the implementation of the WHO surgical safety checklist in both upper (RR = 0.718; 95% CI, 0.600 - 0.860; p < 0.001) and lower/middle economic income country hospitals (RR = 0.539; 95% CI, 0.406 - 0.715; p < 0.001). Subgroup analysis identified a slightly greater reduction in the risk of complications with the use of the WHO surgical safety checklist among lower/middle economic income country hospitals, compared to upper economic income country hospitals, although this was not statistically significant (p = 0.092).

3.3. Effect of the WHO Surgical Safety Checklist on Surgical Site Infections

Data on the incidence of SSIs were reported in 8 trials, involving 21,076 patients (10,902 pre-implementation and 10,174 post-implementation). There were fewer SSIs following the implementation of the WHO surgical safety checklist (332/10,174 [3.3%] vs. 819/10,902 [7.5%]). There was significant heterogeneity between trials (p < 0.001, I² = 70.841), and a random-effects model was assumed. Meta-analysis showed a significant reduction in the risk of SSIs by 45.5% (RR = 0.545; 95% CI, 0.416 - 0.714; p < 0.001) (Figure 3).

There was a significant reduction in the risk of SSIs following the implementation of the WHO surgical safety checklist in both upper (RR = 0.705; 95% CI, 0.560 - 0.888; p = 0.003) and lower/middle economic income country hospitals (RR = 0.440; 95% CI, 0.300 - 0.645; p < 0.001). Subgroup analysis identified a significantly greater risk reduction in SSIs with the use of the WHO surgical safety checklist among lower/middle economic income country hospitals, compared to upper economic income country hospitals (p = 0.039).

3.4. Effect of the WHO Surgical Safety Checklist on Number of Unplanned Return to the OR

Data on the number of unplanned returns to the OR were reported in 5 trials, involving 18,209 patients (8507

Abbreviations: NR = not reported, OR = operating room, UK = United Kingdom, USA = United States of America.

pre-implementation and 9,702 post-implementation). There were fewer returns to the OR following the implementation of the WHO surgical safety checklist (128/9,702 [1.3%] vs. 186/8,507 [2.2%]). There was significant heterogeneity between trials (p = 0.044, I² = 45.237), and a random-effects model was assumed. Meta-analysis showed a significant reduction in the number of returns to the OR by 32.1% (RR = 0.679; 95% CI, 0.484 - 0.952; p = 0.025) (Figure 4).

The risk of unplanned returns to the OR was significantly reduced in both upper economic income country hospitals (RR = 0.540; 95% CI, 0.373 - 0.781; p = 0.001) and lower/middle economic income country hospitals (RR = 0.939; 95% CI, 0.557 - 1.582; p = 0.813), however, the difference between these groups was not statistically significant (p = 0.089).

3.5. Effect of the WHO Surgical Safety Checklist on Overall Mortality

Mortality data was reported in 7 trials, involving 48,517 patients (25,178 pre-implementation and 23,339 post-implementation). There were fewer deaths following the implementation of the WHO surgical safety

checklist (462/23,339 [2.0%] vs. 637/25,178 [2.5%]). There was no significant heterogeneity between trials (p = 0.228, I² = 20.269), and a fixed-effects model was assumed. Meta-analysis showed a significant reduction in the risk of mortality by 15.3% (RR = 0.847; 95% CI, 0.752 - 0.954; p = 0.006) (Figure 5).

There was a greater reduction in the risk of mortality following the implementation of the WHO surgical safety checklist in lower/middle economic income country hospitals (RR = 0.722; 95% CI, 0.551 - 0.946; p =

0.018) compared to upper economic income country hospitals (RR = 0.880; 95% CI, 0.771 - 1.005; p = 0.058), however, the difference was not statistically significant (p = 0.319).

3.6. Effect of the WHO Surgical Safety Checklist on Adherence to Intraoperative Safety Measures

Data on the adherence to various intraoperative safety measures were reported in 4 trials, involving 12,820 patients (6254 pre-implementation and 6,566 post-implementation). Meta-analysis showed a significant increase in the adherence to most intraoperative safety measures, including the use of a pulse oximeter (RR = 1.016; 95% CI, 1.006 - 1.027; p < 0.001), use of prophylactic antibiotics when necessary (RR = 1.099; 95% CI, 1.010 - 1.195; p = 0.028), and verbally confirming the patient’s identity and site of surgery (RR = 2.716; 95% CI, 1.919 - 3.843; p < 0.001). There was also an increase in airway evaluation (RR = 1.021; 95% CI, 0.984 - 1.060; p = 0.273) and completion of a sponge count (RR = 1.009; 95% CI, 0.995 - 1.023; p = 0.207) however, this did not reach statistical significance. There was a significant decrease in ensuring the patient had adequate intravenous access (RR = 0.865; 95% CI, 0.778 - 0.963; p = 0.008) following the implementation of the WHO surgical safety checklist.

Subgroup analysis identified a significantly greater increase in adherence to various safety measures among lower/middle economic income country hospitals compared to upper economic income country hospitals. The difference was statistically significant for airway evaluation (p < 0.001), use of pulse oximeter (p < 0.001), use of prophylactic antibiotics (p = 0.024), and completion of a sponge count (p = 0.037).

3.7. Risk of Bias of Included Studies

All studies had moderate risk of bias and were susceptible to bias inherent to the methodology of included studies. The included studies reported on pre-implementation and post-implementation data, which does not allow for allocation concealment or blinding of participants, personnel, and outcome assessors. In many of the studies, adherence to various safety measures was determined by having an observer present during the surgery, which could lead to bias.

3.8. Publication Bias

A funnel plot was used to qualitatively assess for publication bias, and Egger’s and Begg’s tests were done to calculate publication bias. There was no obvious evidence of asymmetry on the funnel plot (Figure 6). Furthermore, there was no evidence of publication bias for the primary end point (relative risk of mortality following implementation of the WHO surgical safety checklist) by either the Egger’s (p = 0.198) or Begg’s test (p = 0.072).

4. Discussion

With the rising number of surgical procedures performed and the high risk of morbidity and mortality associated with surgery, considerable attention has been placed on the prevention of adverse events and improving patient outcomes. The World Health Organization surgical safety checklist was developed in 2008 to ensure a

standardized approach to patient care with a defined set of safety standards to reinforce established safety practices and to ensure specific perioperative steps are completed in a timely manner [3] .

The current meta-analysis found that the WHO surgical safety checklist was associated with significant risk reductions in postoperative complications, SSIs, number of unplanned returns to the OR, and overall mortality. There were also significant increases in adherence to intraoperative safety measures, including the use of a pulse oximeter, use of prophylactic antibiotics when necessary, and confirming the patient’s identity and surgical procedure and site.

The WHO surgical safety checklist proved beneficial in both upper and lower economic income country hospitals, however, there was a greater impact on lower/middle economic income hospitals as observed by greater reductions in overall complications, SSIs, and overall mortality. It has been suggested that upper economic income hospitals were already utilizing components of the checklist prior to the implementation of a formal checklist [15] . In this study, high economic income countries had higher pre-implementation adherence rates to all six intraoperative items compared to lower economic income countries. Similarly, Vohra et al. conducted an online survey with 6,269 respondents from 69 countries and reported that respondents from high economic income centers were more likely to routinely use the WHO surgical safety checklist, compared to lower income countries (83.5% vs. 43.5%, p < 0.001) [15] . The authors also reported that university teaching hospitals routinely used the checklists more often than non-university teaching hospitals (61.4% vs. 53.7%, p < 0.001) [15] .

Substantial evidence now exists documenting the benefits of the WHO surgical safety checklist, however, the mechanism for improved patient safety is less clear and is most likely multifactorial, including improved communication among staff and the repetitive reminders to complete key perioperative steps [5] . Completion of the sign-in, time-out, and sign-out checklists require a formal pause in patient care with all team members present and communicating. Preoperative team introductions and briefings have been shown to also significantly improve patient safety, team morale, and specific clinical outcomes [16] - [18] . Ali et al. conducted a questionnaire study among 37 surgical staff members and reported that 89% of staff believed the WHO safety checklist improved communication and 97% thought the pause highlighted potential patient problems [19] . Helmio et al. conducted a retrospective insurance claim study of all patient injuries in otolaryngology over a 10-year study period in Finland, and reported that 80.6% of all claims were associated with operative care. Of these, 75.5% occurred in the operating room, 9.6% of errors corresponded with a WHO surgical safety checklist item, and 4.8% could have been prevented with adherence to the checklist [20] .

Despite the significant improvements in patient safety associated with the implementation and use of the WHO surgical safety checklist, its use remains low. Vohra et al. conducted a survey of 6269 medical professionals and only 57.5% of respondents routinely used the WHO surgical safety checklist [15] . The authors also found that female gender (61.3% vs. 56.4%, p = 0.001), age >46 (OR = 1.94; p < 0.001), and attending/consul- ting physicians (OR = 1.54; p < 0.001), were associated with increased use of the checklist [15] . Healthcare workers who believe the checklist is useful and beneficial are significantly more likely to use the checklist, and adherence to certain safety measures has been shown to be associated with increased checklist effectiveness [15] . Bergs et al. reported a significant correlation between adequate adherence to safety measures and reductions in postoperative complications (p = 0.042) [11] .

Several barriers to full implementation and utilization of the WHO surgical safety checklist have been recognized [15] . Russ et al. conducted a longitudinal interview study in England with 119 operating room personnel and reported that 29% of workers thought the checklist would cause unnecessary delays and 25% thought it was repetitive and failed to add anything new to their current system [7] . The same authors conducted a multi-center observational cohort study and reported the average length of time required to complete the “time-out” and “sign-out” checklist items were 68.0 ± 37.5 and 29.0 ± 15.5 seconds, respectively [21] . Ali et al. also reported no significant difference in operating start times with the use of the WHO surgical safety checklists (30.7 minutes vs. 23.5 minutes; p = 0.1) [19] . Fear of provoking anxiety among the patients when questions are asked has also hindered implementation and utilization of the checklist, however, the opposite has been observed in published studies [7] [22] - [24] . Kawano et al. questioned 15 women who underwent a Cesarean section and were fully aware the checklist took place and 12 of the patients felt less anxious, 13 reported being less fearful, and 12 patients reported being less tense [22] . Russ et al. reported that 74% (N = 104) of patients felt safer with the use of the checklist [23] .

Although the results of this meta-analysis are significant, there are limitations to this study due to the variation and heterogeneity of the RCTs. The patient demographics and medical comorbidities, as well as the surgery that the patients underwent also differed between studies. Most of the studies included all surgical procedures in their study, and only one study examined each specific surgical subspecialty such as neurosurgery, plastic surgery, and orthopedic surgery, which limits the ability to perform a subgroup analysis based on the type of surgery. Biskup et al. reported no significant reduction in complications or mortality with the use of the WHO surgical safety checklist in plastic surgery, and stated that the checklist may not be as applicable to the ambulatory setting where most plastic surgery cases tend to be are performed [12] . The included studies compared post-implementation data with pre-implementation data which inherently results in the cohorts being studied at different times. During the years that elapsed, other safety and technological advances may have also been implemented to improve patient safety. In many of the studies, adherence to various safety measures was determined by having an observer present during the surgery, which could lead to bias. The improvement in performance due to the subjects’ knowledge of being observed, known as the Hawthorne effect, may also have contributed to the increase in adherence to safety measures.

Despite these limitations, this study clearly demonstrates that the WHO surgical safety checklist is an effective and valuable tool for improving patient surgical outcomes. Given the number of surgical procedures performed, the risk of morbidity and mortality associated with surgery, and the significant reductions in postoperative complications and overall mortality, the WHO surgical safety checklist should be universally implemented in all surgical centers and in all surgical patients.

Conflict of Interest

All authors listed declare that there are no conflicts of interest, and have not accepted financial sponsorship in producing and presenting this article. The manuscript has been seen and approved by all members and has not been (and will not be) submitted to any other journal while it is under consideration by Surgical Science.

Cite this paper

Christine S. M. Lau,Ronald S. Chamberlain,1 1,1 1, (2016) The World Health Organization Surgical Safety Checklist Improves Post-Operative Outcomes: A Meta-Analysis and Systematic Review. Surgical Science,07,206-217. doi: 10.4236/ss.2016.74029

References

NOTES

^*Corresponding author.