Portfolio Use to Develop Teaching Skills and Meet Program Goals Brian Mavis, PhD Objectives • What is a portfolio? • How can it be used to document accomplishments related to teaching? • What resources are available to guide faculty and RPT committees? What is a portfolio? • Living document • Extension of your CV • Personal annual report (annual review) • Necessary part of many promotion and tenure packages What is a portfolio? • Originally conceptualized like those used with artists or architects • Demonstrates quality of your work • Records breadth of your work • Illustrates professional development What is a portfolio? “…a method of encouraging adult and reflective learning … based on developing a collection of evidence that learning has taken place” Snadden and Thomas 1998, p. 192 What is a portfolio? Personal reflection • Central to successful portfolio • Explains – What is included? – Why it is included? – How it is organized? – How it relates to program or institutional goals? • Tell your story – Where have you been? – What have you done? – What have you learned? – Where are you going? What is a portfolio? • How to build a portfolio? – Find a place to store your work – Keep everything – Ask for it in writing – Be organized – Paper vs electronic What is a portfolio? Documenting Competence Miller GE. The Assessment of Clinical Skills/Competence/Performance; Acad Med 1990 65(9):63-67. Adapted by Drs R. Mehay & R. Burns, UK (Jan 2009). Why a portfolio? Why a portfolio? How can portfolios be used to document accomplishments related to teaching? Documenting Accomplishments • AAMC Taskforce on Educator Evaluation: 2010 – 2012 • The Charge: To provide resources that will aid decision-makers in developing clear, consistent and efficient evaluation processes for faculty with a career focus in education Task Force Members Maryellen Gusic Indiana University Chair of the Task Force Jonathan Amiel Columbia University Brian Mavis Michigan State University Suzanne Rose University of Connecticut Constance Baldwin University of Rochester Kathe Nelson University of Alabama Deborah Simpson Medical College of Wisconsin Latha Chandran SUNY Stony Brook Lois Nora The Commonwealth Medical College Henry Strobel University of Texas Medical School at Houston Ruth-Marie E. Fincher GHSU/Medical College of Georgia Jamie Padmore MedStar Health Craig Timm University of New Mexico Nancy Lowitt University of Maryland Pat O’Sullivan UCSF Tom Viggiano Mayo Medical School What do educators do? • • • • • Teaching Learner Assessment Curriculum Development Mentoring and Advising Educational Leadership and Administration Simpson et al, 2007 Evaluating the work of educators? Four typical indicators of competence: • Quantity • Quality • Scholarly approach • Scholarship Contributions in Teaching • Quantity – Duration, number, scope of teaching activities • Quality – Teaching effective and well-received • Scholarly approach – Incorporates best practices • Scholarship – Workshops, peer-reviewed presentations – Adoption by others Scholarship Reconsidered • Scholarship involves: – Discovery of new knowledge – Application of knowledge – Integration of knowledge – Dissemination of knowledge Boyer, 1990 Glassick’s Criteria 1. 2. 3. 4. 5. 6. Clear goals Adequate preparation Appropriate methods Significant results Effective presentation Reflective critique Glassick, 2000 Let’s Focus on Teaching 1. Clear Goals • Learning objectives for teaching session/curriculum are: – Clearly stated – At level appropriate for learners – Specified to measure learner’s performance 1. Clear Goals • Learning objectives are: – Based on documented needs – SMART (specific, measurable, achievable, realistic and timely) – Address multiple domains (e.g., knowledge, skills and/or attitudes) 2. Adequate Preparation • Congruence/integration with other curricular components • Use of best practices • Necessary resource planning 2. Adequate Preparation • Best Practices – Content is up-to-date and evidence-based – Content is logically integrated with other curricular components – Content to be covered appropriate for time available – Content depth and breadth matched to learners’ needs 2. Adequate Preparation • Resource Planning – Specific needed resources are specified – Needed resources are available – Adequate preparation for use of technology 3. Appropriate Methods • Teaching methods aligned with learning objectives • Methods are feasible, practical and ethical • Innovative teaching methods used to achieve learning objectives 3. Appropriate Methods • Chooses teaching strategies that incorporate a variety of approaches • Variety of approaches is evidencebased • Uses interactive approaches and promotes self-directed learning • Includes strategies for monitoring learner progress • Provides evidence of innovation 4. Significant Results • Satisfaction/reaction of learners • Learning: Measures knowledge, skills, attitudes and/or behaviors • Application: desired performance demonstrated in other settings • Impact: educational programs and processes here or elsewhere Kirkpatrick & Kirkpatrick, 2006 4. Significant Results • Satisfaction/Reaction – Teaching ratings by learners or peers/experts – Compare learner ratings across teachers • Learning – Measurable changes in knowledge, skills, etc. – Comparison to benchmarks or prior data 4. Significant Results • Application – Demonstration of knowledge, skills, etc. in subsequent settings or curricular components • Impact – Evaluation by knowledgeable peers, educational leaders, etc. – Internal or external awards or recognition 5. Effective Presentation • Recognized as valuable (internally or externally) through: – Peer review – Dissemination – Use by others 5. Effective Presentation • Invitations to conduct faculty development, workshops, presentations • Peer review of other teachers • Dissemination and adoption of teaching materials or methods 6. Reflective Critique • Ongoing improvement – Personal reflection – Learner performance data – Evaluation results – Peer review 6. Reflective Critique • Critical analysis of teaching activities using information from others and self-reflection • Evidence of continuous quality improvement of teaching activities Glassick’s Criteria 1. 2. 3. 4. 5. 6. Clear goals Adequate preparation Appropriate methods Significant results Effective presentation Reflective critique Glassick, 2000 Toolkit Criteria Teaching Assessm’t Curric Developmt Mentoring/ Advising Leadership /Admin Clear goals √ √ √ √ √ Adequate Preparation √ √ √ √ √ Appropriate Methods √ √ √ √ √ Significant Results √ √ √ √ √ Effective Presentation √ √ √ √ √ Reflective Critique √ √ √ √ √ Where to Find It AAMC Toolbox for Evaluating Educators • Available through MedEdPortal: www.mededportal.org/publication/9313 References • Boyer EL. Scholarship reconsidered: priorities of the professoriate. San Francisco, CA. Jossey-Bass Publishers; 1990. • Glassick CE. Boyer’s expanded definition of scholarship, the standards for assessing scholarship and the elusiveness of the scholarship of teaching. Acad Med. 2000; 75:877-880. • Kirkpatrick DL and Kirkpatrick JD. Evaluating Training Programs: The Four Levels (3rd Ed). San Francisco, CA: Berrett-Koehler Publishers, 2006. • Miller GE. The Assessment of Clinical Skills/Competence/Performance; Acad Med 1990 65(9):63-67. • Simpson D, Fincher RM, Hafler JP, Irby DM, Richards BF, Rosenfeld GC, Viggiano TR. Advancing educators and education by defining the components and evidence associated with educational scholarship. Med Educ. 2007;41:1002-1009. • Snadden D. & Thomas ML. The use of portfolio learning in medical education. Med Teach. 1998; 20: 192-199. Additional Resources • Baldwin C, Chandran L, Gusic M. Guidelines for evaluating the educational performance of medical school faculty: priming a national conversation. Teach Learn Med. 2011; 23(3):285-97. • Hutchings, P. and Shulman, L.S. (1999). The scholarship of teaching: new elaborations and developments. Change, 31(5), 10-5. • Van Tartwijk, J. & Driessen, EW. Portfolios for assessment and learning: AMEE Guide No. 45. Med Teach. 2009; 31: 790-801.