Closing the Loop on Student Learning Assessment Mohua Bose, Ph.D Director of Outcomes Assessment Mary Beth Hanner, Ph.D, RN, ANEF Provost and Chief Academic Officer Excelsior College, Albany, NY Presentation at MSCHE, December 12, 2011 Purpose Describe a framework that addresses assessment of student learning in a nontraditional degree completion institution. Discuss improvements based on assessment results at the program and institutional level. Describe the process of facilitating an assessment culture. Background Excelsior College founded in 1971 (previously Regents College). 1998 Chartered as a private, independent college. Large student body (approximately 33,000 students). Distance education, flexible learning format. Philosophy: “What you know is more important that where or how you learned it” Background Valuing prior learning is a central premise for adult learners (Green, 1998). Acceptance of a variety of credit sources: Examples: Regionally accredited institutions ACE/PONSI approved courses Exams – CLEP, DANTE,EC EC portfolio assessment for prior learning Online courses at Excelsior and preferred providers Residency requirement (presently minimal, historically none). Background Accreditation Middle States (all programs) NLNAC (specialized accreditation - Nursing) ABET (specialized accreditation - Technology) IACBE (specialized accreditation - Business) Challenge: Who is a typical student? Career oriented working adult College 2 College 1 Average age: 39 year College 3 At a distance Excelsior College College 5 College 4 No first time, full time students. Credits transferred from other approved sources ranges from 25% to 75% of total credits required for the degree program. Accountability and Continuous Improvement OAIR Steering Committee School of Business & Tech School of Liberal Arts OAIR Office School of Health Sciences School of Nursing Institutional Assessment Plan for Student Learning (IAPSL) Institutional Assessment Plan for Student Learning (IAPSL) Provides a framework for assessment of learning outcomes at course, program and institutional level. Addresses the key question “How effectively do we deliver on our mission and make a distinctive impact relative to our resources?” (Collins, 2011). IAPSL link: http://www.excelsior.edu/web/excelsior-theinstitution Assessment Process at Schools Establish clear learning outcomes Provide learning opportunities Use the Results Assess Student Learning Step 1: Establish Learning Outcomes Ultimate responsibility to determine learning outcomes for a program rests with the faculty. Instructional/Adjunct faculty receive an online faculty training that orients them to EC. The orientation includes an introduction to learning outcomes assessment. Development of online courses requires completion of a structured template that involves program, course and module outcomes. Step 1: Establish Learning Outcomes (continued) A raised awareness of the importance of learning outcomes assessment at the College due to OAIRSC and OAIR offices. Senior level personnel significantly involved (eg. Deans). In addition to OAIR, an assessment director within the schools work with the faculty. Continuous and evolving process in defining learning outcomes. Step 2: Create Learning Opportunities Monitoring the learning opportunities provided to students: Course revisions based on: Centralized student course evaluations Faculty evaluation of courses Faculty curriculum committee Curriculum reviews Example: Curriculum Review Committee of 3 reviewers (some members are completely external to the college). 2-3 days including two teleconferences. Reviewers examine curricular structure, topical areas covered, course descriptions. Provide feedback on curriculum rigor, curriculum currency, and program outcomes attainability and currency. Feedback from the external curriculum review is taken back to program faculty. Step 3: Assessing Student Learning Direct assessments of program outcome measures for all programs using: o Capstones, Portfolios, or Exams (theory or performance) o State licensing exams (e.g. NCLEX) Direct assessments of general education outcomes using: o ETS Proficiency Profile & VALUE Rubrics Step 3: Assessing Student Learning Also have indirect assessments: Alumni surveys Supervisor surveys Centralized for all degree programs Discipline Specific Example on Closing the Loop Associate Degree in Nursing NCLEX pass rates Consistently at or above national average over a decade. Dipped below for one quarter in national and state pass rates by approximately 1%. Conducted analysis of subscales on the NCLEX exam. Examined the CPNE exam. Associate Degree in Nursing Analysis of the data revealed that students scored lower on o Knowledge of adolescent growth and development o Care of the ill adolescent After faculty review – new curriculum requirements. Growth and Development across the Lifespan. Associate Degree in Nursing After approximately 2 years • Scores exceeded both state and national levels • NCLEX scores in overall exam and subscales Remained up for a number of years. Continuing to monitor. General Education Example General Education Assess Writing Outcome: Write with grammatical competence, clarity, and coherence Administered standardized assessment: ETS Proficiency Profile Reading Writing Mathematics Critical thinking General Education ETS Proficiency Profile administered annually 2009, 2010, and 2011. Results pertaining to Writing show that Excelsior students at the Associate’s and Bachelor’s level have performed at or above the ETS norms for writing. General Education College decided to use multiple measures to assess writing. Use of VALUE rubrics in writing Capstone papers from Bachelor’s degree programs across the College used. Independent panel of reviewers. Data collection completed, shows variations in student performance. Closing the loop takes time. Key Steps in Implementing Outcomes Assessment Institutional support and commitment. Creation of a college-wide oversight committee to set a relevant and strategic agenda for assessment of student learning and institutional performance. A visionary and guiding document for implementation of outcomes assessment College-wide. Key Steps in Implementing Outcomes Assessment Training and implementation of an assessment management tool/database. Centralized assessment model supported by a distributed model of assessment staff. Showcase assessment results - Quarterly Dean’s presentations at Steering Committees. Bi-annual assessment audits. Continuous and evolving process!