Assessment of High Impact Practices: Student Learning Portfolios and Other Tools Monica A. Devanas [email protected] http://CTAAR.rutgers.edu AAC&U, LEAP, HIPs, VALUE Rubrics American Association of Colleges and Universities Liberal Education and America’s Promise – 2005 21st Century Liberal Education Essential Learning Outcomes High Impact Practices Authentic Assessment - Valid Assessment of Learning in Undergraduate Education = VALUE Rubrics High Impact Practices What? techniques and curricular designs for teaching and learning Why? proven to be beneficial for student engagement and successful learning students from many backgrounds How? intentional program design and pedagogy Why? enhance student learning narrow gaps in achievement across populations High Impact Practices • • • • • • • • • • First-Year Seminars and Experiences Common Intellectual Experiences Learning Communities Writing-Intensive Courses Collaborative Assignments and Projects Undergraduate Research Service Learning, Community-Based Learning Diversity/Global Learning Internships Capstone Courses and Projects List Summarize Solve Analyze Design Evaluate Name Explain Illustrate Organize Hypothesize Choose Identify Interpret Calculate Deduce Support Estimate Show Describe Use Contrast Schematize Judge Define Compare Interpret Compare Write Defend Recognize Paraphrase Relate Distinguish Report Criticize Recall Differentiate Manipulate Discuss Justify State Demonstrate Apply Plan, Visualize Classify Modify Devise Assessment of Learning in HIP What is the goal of High Impact Practice? What is the context of student learning? What assignments will be used to grade? How will these assessments give feedback on progress of student? … of the program? Assessment Vocabulary • Formative – ongoing feedback to improve learning, areas for improvement “low stakes” few points, but frequent Ex: first draft, questions on reading, concept map • Summative – evaluate learning for final grade compare to standards or benchmark “high stakes” high point value Ex: midterm, final, final project, paper, performance Assessment Vocabulary Qualitative - data fits interpretive criteria, not easily analyzed by quantitative methods Quantitative – numerical data that can be analyzed with statistical methods Assessment Vocabulary Direct – students knowledge, skills, attitudes, data measures achievement of expected outcomes Indirect – students on knowledge, skills, attitudes, give their opinion, easy to administer, focus on attitudes, values, beliefs Direct Methods of Assessment Pre and posttests Course-embed assessment Portfolio evaluation Case studies (homework assignment, essay, Reflective journals locally developed tests) Capstone projects Comprehensive exams Juried performance, exhibits National Major Field Internship, clinical evaluation Achievement Tests External examiners GRE subject exams Peer review Certification & licensure Senior thesis exams Major project Grading criteria or rubrics (QUALITATIVE) (QUANTITATIVE) Direct Methods of Assessment Indirect Methods of Assessment • • • • • • • • • • Departmental survey Student satisfaction surveys Self-reported gains Exit interviews Alumni survey Employer survey Focus groups Job placement statistics Percentage of students who study abroad Graduation and retention rates High Impact Practices Documentation: assignments, reports, essays Collaboration: instructors, peers, advisors Reflection: can student see “impact” on learning, on skill development, on beliefs, attitudes and values? Components of Learning Portfolio Learning Portfolio: focus process of learning Documentation = products for evaluation Collaboration = mentor to understand process of reflection and documentation Reflection = think about process of learning, metacognition, thinking about thinking Components of Learning Portfolio Learning Portfolio: focus process of learning Zubezarreta’s “order” effect of planning Reflection = think about process of learning, metacognition, thinking about thinking Documentation = products for evaluation Collaboration = mentor to understand process of reflection and documentation Documentation • Organization, selectivity driven by purpose – Course, activity, program, curriculum • Concise reflective narrative on documents – Relevance, impact on learning, applications • Selected “evidence” described in narrative and link in appendix – OpenLab, web-based, paper-based Mentor is Important • “Collaborative” Mentor: to keep focus, manageable, current, accurate, organized, relevant, time on task, deadlines, coach • Need some objective review – Peer mentor – Advisor – Draft reader, editor – Portfolio “coach” Size, Scope of Learning Portfolio Represent one set of assignments and activities from one course OR experiential learning activity or over a longer period of time, core courses courses in major undergraduate curriculum Student Portfolio = Learning Portfolio without reflection Even process of collecting artifacts has value Students see evidence of significant learning • connections between learning experiences, i.e. in courses, in co-curricular activities. • Valuable as summation of work in learning in some context as a product Useful for assessment of course, program Flavors of Portfolios Student Portfolio: Product – collection of student work in some context Learning Portfolio: Product AND PROCESS reflective learning = process of reflection On Learning Portfolios Zubizarreta… “organized documentation of growth and achievement that provides tangible evidence of the attainment of professional knowledge, skills and disposition. Each portfolios is goaldriven, original, and reflective. The Learning Portfolio value lies in student reflection, construct the notion of learning as a coherent, unified, developmental process, that it is active and lifelong.” Students ask, "Do e-Portfolios Have To Be "E"? Fundamental First Steps For Successful Learning Portfolios. http://eportfolioca.org/index.php?option=com_content&view=article&id=141&Itemid=159 The E-Portfolio Forum: Deepening High Impact Learning Sessions January 29, 2011 - San Francisco, CA Assessment of High Impact Practices Assortment of Assessments Methods Formative – Summative Qualitative – Quantitative Direct – Indirect Balance and Variety Assessment of High Impact Practices Assortment of Assessments Methods Balance and Variety Difficult to Assess Activities Diversity/Global Learning Internships Capstone Courses and Projects Student Portfolios – Learning Portfolios Portfolios at City Tech Student Portfolios - Reflection with Portfolios – Utilized work from open learning environment – Link to past to represent what you have developed through the college years – Binding evidence from past, alternately reflect back on how past work informed current work, has past work driven interest – Is there maturity over time / enrichment – Reflection what student learned through guidance Karen Goodlad, notes from 12/2/13 High Impact Practices Goals important for assessment Kinds of assessment Student Portfolios, Learning Portfolios useful for student learning and assessment ~~Wrap UP~~ Closing Ideas from Participants Questions??