Open Pathways: Testing the Degree Profile Activity 3 Marshall University October 5 2012 – February 1, 2013 Open Pathways Project Deadlines • December 15, 2012 o Complete and submit assessment rubrics for remaining outcomes in program or major. • February 1, 2013 o Submit fall 2012 Assessment report using Activity 2 program outcome rubrics. • May 17, 2013 o Submit spring 2013 assessment report using new program outcome rubrics. • June 30, 2013 o Quality Initiative: Testing the Lumina Foundation’s Degree Qualifications Profile Learning Outcomes Final Report sent to Higher Learning Commission and Lumina Foundation. Activity 3 (in a nutshell) Step 1: Review Activity 2 Rubrics. Step 2: Set benchmarks for each assessment point. Step 3: Review courses and assessments identified for each assessment point. Step 4: Use Activity 2 rubrics to assess artifacts at each assessment point. Step 5: Collect data Step 6: Analyze data. Identify areas in need of improvement. Step 7: Based on data analysis, determine an action plan to improve student learning in program. Step 8: Report data using data reporting submission form available at www.marshall.edu/hlcopenpathways Activity 3 Step 1: Review Activity 2 rubrics Step 2: Set benchmarks. Review Activity 2 rubrics and set benchmarks: Why? • To revise Performance level names and set benchmarks as follows: o Introductory Benchmark = benchmark expected during first year of study for Baccalaureate students (benchmark for first assessment point for Associate’s Degrees) o Milestone Benchmark = first assessment point benchmark for Baccalaureate Degrees (exiting benchmark for Associate’s Degrees) o Capstone Benchmark = exiting benchmark for Baccalaureate Degrees and first assessment point for Master’s Degrees o Advanced Benchmark = exiting benchmark for Master’s Degrees Activity 3 Step 3: Review courses and assessments identified for each assessment point. Review courses and assessments selected for • • • • Introductory benchmark assessments Milestone benchmark assessments Capstone benchmark assessments Advanced benchmark assessments Considerations during review • Have courses and embedded assessments been identified for each assessment point? • Are courses and assessments appropriate o Correct level of instruction for performance benchmark level? o Controlled by your program/department? • Have all courses used for assessment points been mapped to program and DQP outcomes? If not • Map course outcomes to program outcomes and to the broad areas of learning and degree appropriate outcomes in the DQP. o Use Activity 1 form to do this. Activity 3, Step 4: Use Activity 2 rubrics to assess artifacts at each assessment point. Use Activity 2 rubrics to • • Assess course embedded (or other appropriate) artifacts Assess at a minimum of two assessment points o For Associate’s Programs: Introductory and Milestone o For Bachelor’s Programs: Milestone and Capstone o For Master’s Programs: Capstone and Advanced Activity 3 Step 5: Collect data • Collect data for each outcome o Collect data separately for each outcome o Enter data into Excel, SPSS, or other statistical package. • SPSS Example o On bottom of data entry sheet, select “variable view” tab. o Enter the following variable names in the “name” column o Student o Each rubric trait in a separate row (One word – no spaces – if you want to describe in more detail, add information under the “label” column o Set performance level value labels for each rubric trait Activity 3, Step 5 Continued • SPSS Example Continued o On bottom of data entry sheet, select “data view” tab. o The variable names that you entered earlier will appear at the top of each column. o Enter data for each variable in the appropriate column o Student – can use ID, code or name (If you student name, must make variable “String” rather than “Numeric” o Enter number that corresponds to rating for each rubric trait for each student o Run frequency analysis o o o o At top of sheet, select “analyze,” then “frequencies” Move trait variables to the variables box Click on “OK” to run frequency analysis Results will be reported in frequency tables Activity 3 Step 6: Analyze data. Step 7: Determine an action plan. Step 8: Report data. • Step 6 o Analyze data. Identify areas in need of improvement (i.e. relative strengths and weaknesses). • Step 7 o Determine an action plan to address weaknesses or make other modifications to improve student learning. • Step 8 o Report all information using reporting form found at o www.marshall.edu/hlcopenpathways Benchmark = Milestone Performance Level Areas in need of improvement are method of inquiry and data analysis and conclusions 0 100% 90% 0 10 20 35 80% 60 70% 60% Advanced Capstone 50% 40% 90 Milestone 80 Introductory 65 30% 40 20% 10% 0% 0 Problem 0 Research Method Analysis Due Date Reminders • December 15, 2012 o Complete mapping of any course used for program level assessment during fall 2012 that was not previously mapped in Activity 1. Use Activity 1 submission process. o Revise Activity 2 rubrics as appropriate. o Complete and submit assessment rubrics for remaining outcomes in program or major. Submission form will be available at www.marshall.edu/hlcopenpathways no later than November 1. • February 1, 2013 o Submit fall 2012 Assessment reports using Activity 2 program outcome rubrics. Form for this process will be available no later than November 1. Course mapping form Preview of Activity 4 Spring 2013 • Use program outcome rubrics completed in Activity 3 to assess remaining program outcomes. • Submit report of spring 2013 assessment by May 17, 2013, using reporting template used for Activity 3. Final Goal of Project • To provide feedback to the HLC and to the Lumina Foundation for the purpose of improving the DQP. • To develop a degree profile unique to Marshall University. o Watch for proposal for a Marshall University Baccalaureate Degree Profile soon!