Promise & limitations of surgical checklists: How can we effectively use them to improve the quality of surgical care? Shawn J. Rangel, MD, MSCE & J. William Sparks, MD Northeast Regional Patient Safety & Quality Improvement Conference February 5th, 2011 WHO checklist Reality check- IHI map Even the stars are using it! So, will the WHO checklist save mankind? Reality check: one size does not fit all… Outline of today’s discussion • Review evidence supporting surgical safety checklists • Attitudes toward the safety checklist at (CHB survey) • Strategies for improving checklist utilization & relevance • Next steps: IT, custom checklists & beyond… WHO study 8 Evaluation Sites PAHO I Toronto, Canada EURO EMRO London, UK Amman, Jordan WPRO I Manila, Philippines PAHO II Seattle, USA WPRO II AFRO Ifakara, Tanzania SEARO New Delhi, India Auckland, NZ Methods • 1 to 4 operating rooms targeted at each site • 18-item checklist implemented (sign-in, time-out, sign-off) • Pre-post intervention study design (general surgery cases) • Primary outcome measure: aggregate 30-day major complication rate (NSQIP* defined) *National Surgical Quality Improvement Project Results: impact on morbidity & mortality (3 month comparison periods) Baseline Checklist P value Cases 3,733 3,955 - Death 1.5% 0.8% 0.003 Any complication 11.0% 7.0% <0.001 Surgical site infection 6.2% 3.4% <0.001 Unplanned Reoperation 2.4% 1.8% 0.047 Limitations of the study • Unknown influence of the Hawthorne effect • Unable to prove causality (non-randomized design) • Effect size may be exaggerated (developing nations) • Only one of the eight centers was in the U.S. • Pediatric patients not included in analysis SURPASS Intervention The comprehensive “SURgical PAtient Safety System”: Methods •SURPASS implemented at 6 tertiary-care hospitals •Pre-post intervention study design (3 month periods) •12 adverse event categories audited •Outcomes compared with five “control” hospitals Results: impact on morbidity & mortality (3 month comparison periods) Baseline Checklist (n=3,760) (n=3,820) Mortality 1.5% 0.8% - Any complication 15.4% 10.6% <0.001 Complications/100 cases 27.3% 16.7% <0.001 P value Use of the checklist was associated with a significant reduction in complication rates for 10 of the 12 (82%) event categories in the study But, kids are not small adults! Pediatric Safe Surgery Collaborative • Children’s Hospital Boston Shawn J. Rangel (Study PI) Beth K. Norton (Co-study PI) Jessica Baxter • Texas children’s Hospital Thomas Luerssen (site PI) Carrie Smith-Bruce • Riley Children’s Hospital Fred Rescorla (Co-site PI) Charles Leys (Co-site PI) Margo Regas • Denver Children’s Hospital Tammy Woolley (Site PI) Jenae Nieman • Children’s Healthcare Atlanta Kurt Heiss (Site PI) Kawana Mitchell • Children’s National Med Center Kurt Newman (Co-site PI) Rahul Shah (Co-site PI) Andrea Ewing-Thomas • Children’s Hospital of Philadelphia Peter Mattei (Site PI) Lisa Czyzewski Collaborative process • Identification & recruitment of checklist champions • Development of site-specific checklist • Plan for piloting on small scale • Obtaining buy-in from hospital leadership & peers • Full implementation OR-wide • Develop internal plan for auditing compliance General study design • Seven hospitals included as “checklist” implementers • All inpatient procedures from 7 surgical services included Cardiac, General, Neuro, Ortho, Plastics, Oto & Urology • Pre-post intervention comparison design (9 months) • Primary endpoint: Aggregate 30-day adverse event rate • PHIS database used to identify events • Results compared against seven control hospitals Patient characteristics (demographics) Checklist hospitals (n=7) Control Hospitals (n=7) Pre-intervention Post-intervention Pre-intervention Post-intervention (n=19,867) (n=18,850) (n=15,616) (n=15,298) Age (mean years) 6.9 7.1 7.2 7.4 Sex (% male) 55.3 54.2 56.5 56.6 Insurance status : Government (%) Private (%) Other (%) 62.5 24.8 12.7 63.2 24.6 12.2 50.2 39 10.8 53.9 39.7 7.3 Race/ethnicity: African American(%) Hispanic (%) 20.2 16.9 18.5 17.5 10.6 23.3 10.8 26 Case Mix Index 6.1 6.1 5.5 5.5 Acuity of procedure (% emergent) 15.6 18.9 18.8 19.6 Rate of any adverse event P=0.364 Incidence (%) P=0.064 MORTALITY RR of death with checklist utilization: 0.73 (95%CI: 0.57-0.93) p=0.018 Incidence (%) p=0.758 MORTALITY (High-risk specialties) p=0.332 Incidence (%) p=0.064 p=0.080 MORTALITY (emergent procedures) RR of death with checklist utilization: 0.58 (95%CI: 0.36-0.95) p=0.029 Incidence (%) p=0.724 MORTALITY (ICU admissions) RR of death with checklist utilization: 0.57 (95%CI: 0.40-0.82) Incidence (%) p=0.002 p=0.731 Limitations of the study • Unknown influence of the Hawthorne effect • Unable to prove causality (non-randomized design) • Reliance on administrative data for outcomes analysis • Variation/degree of checklist compliance unknown What can be concluded from the available data? USE OF A SURGICAL SAFETY CHECKLIST CAN SAVE LIVES !!!!!! So then, how do people feel about using a surgical safety checklist? --CHB Checklist Survey-• Multidisciplinary targeting (3-headed monster!) • Assess attitudes towards the checklist • Gain insight on CHB’s current safety culture • Obtain feedback for improving checklist utility Checklist survey: Responses by specialty (n=177) Has the checklist improved safety? (response = yes) Proportion of responders (%) Chi2, p=0.948 Proportion of responders (%) How has the checklist improved safety? Have you witnessed an error or complication prevented by the checklist? Proportion of responders (%) (response=yes) Chi2, p=0.048 Has the checklist improved efficiency? Proportion of responders (%) (response=yes) Chi2, p=0.110 Would I want the checklist used for my child? (response=yes) Proportion of responders (%) Chi2, p=0.122 Proportion of responders (%) Content adequacy of current checklist? Chi2, p=0.987 Chi2, p=0.987 Chi2, p=0.987 So, is everyone in love with the checklist?? “This checklist is bullsh&! and just reinforces the Betty Crocker approach to medicine !!” “This is probably the most important surgical safety intervention we could ever implement !!” Root causes of “checklist fatigue” “This is stupid- the checklist does not apply to my cases” • Blood products available? • Imaging reviewed? • Special equipment available? • DVT prophylaxis considered? • IV access adequate? “This is stupid- we do this all the time anyway” So then, how can we improve the effectiveness of our checklist? • Implementation of forcing cues into work flow • Incorporation of a more effective auditing system • Transition to a “quality”-centered checklist paradigm • Development of customized checklists Change in the checklist paradigm: transitioning from “safety” to “quality” Safe Value-based Surgical Quality Efficient Effective Variation in the use of surgical antibiotic prophylaxis for common pediatric procedures How can we accomplish these goals? How can we accomplish these goals? Take home lessons…. • EFFECTIVE use of surgical checklists CAN SAVE LIVES! • Checklist MUST be team-based and emphasize communication! • Checklists HAVE to be developed with input from ALL stakeholders • Leadership ABSOLUTELY has to be on board! And finally…. YOU ARE THE FUTURE OF SAFETY CULTURE!!!