Emergency Medicine Milestones April 2, 2012 2012 CORD Academic Assembly Based on Core Competencies Patient Care- 14 Medical Knowledge-1 Professionalism- 2 Interpersonal Communication Skills- 2 Practice-based Learning and Improvement- 2 Systems-based Practice- 3 A total of 24 Milestones Developed and completed Milestones in 5 months Based on Milestone progress, EM invited into NAS trial rollout July, 2013 Milestones are truly along a continuum of end of medical school to certification standards Approved by ABEM BOD 1/2012, and by RRC-EM 2/2012 Only specialty to take ABMS certification standards and apply to Milestones Milestones are based on extensive survey data related to ABEM certification standards Only specialty allowed to make revisions in program requirements Only specialty to integrate the Milestones into proposed program requirement changes Obtain outcome measures (i.e. milestones of competency development) to use as evidence of programs’ educational effectiveness 6 Case Logs Milestone Reporting (semiannually) 7 Resident and Faculty Opinions Continuous Oversight & Improvement Emphasis Program & Institutional Information ACGME Residency Programs • Accreditation – continuous monitoring of programs; lengthening of site visit cycles • Public Accountability – report at a national level on competency outcomes • Community of practice for evaluation and research, with focus on continuous improvement • Guide curriculum development • More explicit expectations of residents • Support better assessment • Enhanced opportunities for early identification of under-performers Milestones Certification Boards Residents • Potential use – ascertain whether individuals have demonstrated qualifications needed to sit for Board exams • Increased transparency of performance requirements • Encourage resident self-assessment and self-directed learning • Better feedback to residents 8 Feasibility Quality Reporting • Balance costs with benefits • Manageable number of milestones • Improvement over current approaches • Meaningful & substantive • “Measurable” • Specialtywide use of 5level template for milestones reporting • Central data repository 9 Sponsorship Who When/How • ACGME • Certification Boards • Working Group • 3 – 4 meetings • Interim work and regular communication • Milestones developed in 12 months (or less!) • PDs, Residents, Board, RRC, Specialty Organization, ACGME • Advisory Group 10 Who Who • Expert Panel • Assessment Group • Developing milestones for ICS, Prof, PBLI, SBP • For adaption or adoption 11 • Identifying assessment tools • Developing implementation guidelines 1999 - Outcome Project Begins 2001- Quadrads (Board, PD, RRC, Res) Convened 2002-2008 – Implementation of 6 Competency Domains 2009 – 2012 Milestone Development • All specialties to be completed by 12/2012 • Pilot testing ongoing 12 2013 & Beyond • Large scale implementation of milestones for testing • New accreditation system launch ~ staggered approach (e.g. 45 specialties at first) Development of Milestones Almost as if there was a plan… Chair: Michael Beeson (Vice Chair, RRCEM) Ted Christopher, M.D. (AACEM) Kevin Rodgers, M.D. (AAEM) Jamie Jones, M.D. (ABEM) Mary Jo Wagner, M.D. (ACEP) Philip Shayne, M.D. (CORD) Jonathan Heidt, M.D. (EMRA) Susan Promes, M.D. (SAEM) The Model of the Clinical Practice of Emergency Medicine Most are familiar with the “Listing of Conditions and Components” There is another aspect: Physician Task Definitions Prehospital Care Emergency Stabilization Performance of Focused History and Physical Exam Modifying Factors Professional and Legal Issues Diagnostic Studies Diagnosis Therapeutic Interventions Pharmacotherapy Observation & Reassessment Consultation and disposition Prevention & Education Documentation Multi-tasking and Team Management Made up of ABEM Board members Tasked with looking at entire initial certification process What are the standards? Have they changed? Evaluated changes in physician practice Recommended changes to the content and methods of administration of ABEM’s examinations to assure relevancy to EM practice The result was additive and claritive to physician task definitions Pre-hospital Care Emergency Stabilization Performance of Focused History and Physical Exam Modifying Factors Professional and Legal Issues Diagnostic Studies Diagnosis Therapeutic Interventions Pharmacotherapy Observation & Reassessment Consultation Disposition Prevention & Education Documentation Multi-tasking (Task-switching) Team Management General Approach to Procedures Procedures Contract Principles Financial Issues Operations Clinical Informatics Knowledge Translation Performance Improvement Systems-based Management Disaster Management Communication and Interpersonal Skills Teaching Research With physician task definitions in place and the REPP report’s addition, ABEM was ready to write the Knowledge, Skills, and Abilities (KSAs) that should make up an Initial Certification Exam An Advisory Panel was created in which KSAs were written that defined expectations of an individual pursuing initial certification in EM Based upon using Physician Task Definitions as starting point Each KSA was then developed into hierarchical scales of performance competency Accepted level of performance for the ABEM Diplomate ABEM sent a survey to EM Diplomates and had over 7000 responses The survey queried importance and frequency for each of the identified KSAs and Model Content The result is EM is a specialty in which our practitioners have defined the frequency and importance of expert panel defined KSAs A smooth transition? Identified the Physician Task Definitions essential to the defined needs for Milestones Pre-hospital Care Emergency Stabilization Performance of Focused History and Physical Exam Modifying Factors Professional and Legal Issues Diagnostic Studies Diagnosis Therapeutic Interventions Pharmacotherapy Observation & Reassessment Consultation Disposition Prevention & Education Documentation Multi-tasking (Task-switching) Team Management General Approach to Procedures Procedures Contract Principles Financial Issues Operations Clinical Informatics Knowledge Translation Performance Improvement Systems-based Management Disaster Management Communication and Interpersonal Skills Teaching Research Identified the Physician Task Definitions essential to the defined needs for Milestones Narrowed the list to 17 from 29 Then looked at ABEM’s additional work on hierarchical scales of performance Identified Milestones for each core competency Five levels Level 1- entry level for a medical school graduate Level 4- The ABEM certification standard By definition where an individual should be at time of graduation Level 5- Attained after practice experience Based on Core Competencies Patient Care- 14 Medical Knowledge-1 Professionalism- 2 Interpersonal Communication Skills- 2 Practice-based Learning and Improvement- 2 Systems-based Practice- 3 A total of 24 Milestones PC1- Emergency Stabilization PC2- Performance of Focused History and Physical Examination PC3- Diagnostic Studies PC4- Diagnosis PC5- Pharmacotherapy PC6- Observation and Reassessment PC7- Disposition PC8- Multi-tasking (Task-switching) PC9- General Approach to Procedures PC10- Airway Management PC11- Anesthesia and Acute Pain Management PC12- Other Diagnostic and Therapeutic Procedures: Ultrasound (Diagnostic / Procedural) PC13- Other Diagnostic and Therapeutic Procedures: Wounds Management PC14- Other Diagnostic and Therapeutic Procedures: Vascular Access MK- Medical Knowledge PROF1- Professional values PROF2- Accountability ICS1- Patient Centered Communication ICS2- Team Management PBLI1- Teaching PBLI2- Practice Based Performance Improvement SBP1- Patient Safety SBP2- Systems-based Management SBP3- Technology Working Group Meeting March 10, 2012 CORD, CDEM invited Evaluated potential assessment methods Made recommendations for assessment End of shift, direct observation, and simulation were determined to likely be best assessment methods Validity studies ABEM undertaking at end of April, 2012 Avoid use as a simple subjective Likert scale Base marker scoring on objective measures EM will be used as pilot specialty in NAS Integration of Milestones into EM Program Requirements Begins July, 2013 A first! Development of assessment methodology Specialty-wide implementation of assessment methods? Partner with CORD Can CORD develop tools that are accepted by most programs, that can be applied to end of shift, direct observation, or simulation methods? Can CORD develop validity and reliability studies for developed tools? Example is SDOT as a tool used with direct observation as an assessment method Inter-rater reliability Can CORD develop faculty development programs that teach use of developed tools to ensure reliable application?