CS 22: Using e-Portfolios for General Education Assessment at

Using ePortfolios for General
Education Assessment
AAC&U General Education and Assessment
February 2014
Martha VanCleave
Professor of Mathematics, Interim Director of Division of Continuing Education
Linfield College
Linfield College
 Private, comprehensive, undergraduate institution in the Pacific
 47 majors through three programs:
 McMinnville - traditional liberal arts and pre-professional programs
(1700 students)
 A Portland Campus - Linfield-Good Samaritan School of Nursing
(450 students)
 The Adult Degree Program - a pioneer in adult education
primarily online (805, 410 FTE students).
Linfield College
The Linfield Curriculum(LC)
Linfield’s general education curriculum was revised in 2007
The Linfield Curriculum Structure
 Four major components:
 Inquiry Seminar(INQS) – writing intensive first year
 Six Modes of Inquiry
 Diversity Studies
 Major Writing-Intensive (MWI).
Linfield College
The Six Modes of Inquiry
 Creative Studies (CS)
 Individuals, Systems, and Societies (IS)
 Natural World (NW)
 Quantitative Reasoning (QR)
 Ultimate Questions (UQ)
 Vital Past (VP)
Linfield College
Upper-Division Mode of Inquiry
 An upper-division course (300 or above)
 In one of the six modes of inquiry
 CS, IS, NW, QR, UQ or VP
 From outside the major department
Linfield College
Diversity Studies
 Global Pluralisms (GP)
 U.S. Pluralisms (US)
Linfield College
Linfield Curriculum Course
Requirements - “LC’s”
Students are required to complete:
 Inquiry Seminar (INQS) – 1 course during first or
second year
 Modes of Inquiry – 6 courses,1 from each area
 Diversity Studies - 2 courses,1 from each area
 Upper-Division Mode of Inquiry - 1 course
 Major Writing Intensive Course - 1 course
The INQS is 4 semester credits, all other courses are a
minimum of 3 credits
Total of 11 courses, minimum of 34 credits
Learning Outcomes
 Unique for each Mode of Inquiry and Diversity Studies area
 Developed by a separate group of faculty (working group) for
each Mode of Inquiry and Diversity Studies area
 Consistent with LEAP Essential Learning Outcomes/VALUE
 Requests for courses to fulfill requirements reviewed and
approved by working group
 Learning outcomes utilized when reviewing courses
 Learning Outcomes must be included in syllabus
 Syllabus must describe how learning outcomes will be met
Linfield Curriculum Assessment
Plan (LCAT)
adopted 2010
 Students must demonstrate meeting the learning
objectives of each mode of inquiry and area of diversity
studies through:
 Direct evidence
 Choosing an exemplar or exemplars
 Posting the work in an online repository
 Indirect Evidence
 Supporting the choice of these exemplars with a reflection on how they
meet the learning outcomes
Linfield College
 Faculty Working group for each LC designation
 Develop rubric for assessing evidence of student learning
 Assess random sample of student submissions
Linfield College
 Assessment of LC submissions on a rotating schedule
 2 designations per year
 First assessment summer 2011 - QR and VP
 2012 – US and NW
 2013 – GP and UQ
 2014 – CS and IS
 Cycle repeats
Linfield College
First Round of Assessment
QR and VP assessed
QR and VP assessment
 Exemplars submitted in LMS
 Limited ability to provide instructions for students
 No tool for collecting demographic data
 Submissions downloaded for evaluation
 Record of evaluation compiled manually
Linfield College
Assessing QR
Assessing QR
 Calibrate use of assessment rubric
 Random selection of submissions scored by 2 readers,
then consensus
 Results
1. Pose questions involving
quantitative relationships . . .
35 (39.8%)
24 (27.3%)
25 (28.4%)
4 (4.5%)
2. Analyze problems by
discussing models …
17 (19.3%)
29 (33.0%)
38 (43.2%)
4 (4.5%)
3. Understand uses and
constraints …
7 (8%)
17 (19.3%)
59 (67.0%)
5 (5.7%)
10 (11.4%)
20 (22.7%)
52 (59.1%)
6 (6.8%)
4. Communicate and
critique quantitative
Linfield College
Assessing QR
 Observations and recommendations
 Rated satisfactory or above
 #1 67.1%
#2 52.3%
#3 27.3%
#4 34.1%
 Difficulty submitting materials from online courses
 Students need guidance
 Revision of learning outcomes
 Need benchmarks
Linfield College
Lessons Learned
 Students need direction and structure
 Faculty should make connections between coursework
and LC learning outcomes
Linfield College
Implementing an ePortfolio system
Selecting the ePortfolio Platform
 Working group
 Faculty
 Dean of Student Affairs
 Registrar
 Associate Dean of Faculty
 Chief Technology Officer
 Director of Educational Technologies
Linfield College
Selecting the ePortfolio Platform
 Criteria
 Tiered access
 Matrices
 Evaluation
 Areas for student reflection
 Ability to aggregate data
 Aesthetics and usability
 Support from provider
Linfield College
TaskStream and Linfield
 Create a Directed Response Folio (DRF)
 Work done by team of faculty
 Supported by TaskStream Mentoring Services
Linfield College
Linfield College
The DRF form
 Form for each area
 Course Location
 Course Subject Code
 Course Number
 Course Name
 Course Delivery (option that best describes how this course
was taught)
 Year Course Was Taken
 Class Standing
 Term Course Was Taken
Linfield College
The DRF form
 LC Outcomes
 Please select all of the QR learning outcomes covered in this course. For courses
taken at Linfield, this information can be found in your course syllabus. For
transfer courses, check all that apply. Note that for courses taken at Linfield, all
four learning outcomes should be selected. In addition, if this is a transfer course,
please select "Transfer Course."
 Description of Assignment(s) for Submission
 Supporting Description of Quantitative Reasoning
Linfield College
TaskStream and Linfield
 Enroll faculty and students
 Create single sign-on using CatNET Username and Password
 https://www.linfield.edu/taskstream/index.php
 Batch process enrollment via secure FTP site
Linfield College
Assessment Using Taskstream
Developing Rubrics with Taskstream
Beneficial Features of TaskStream
 DRF provides structure for students
 Form developed to provide explicit instructions
 Online evaluation using the rubric
 Data storage
 Data reporting
Linfield College
Assessing NW
Assessing NW
 Calibrate use of assessment rubric
 Random selection of submissions scored by readers
 Results
1. Use scientific method. . .
2.Think critically…
3. Historical development of
4. Appreciate how
knowledge affects...
13 (17%)
6 (8%)
56 (75%)
27 (36%)
29 (39%)
19 (25%)
12 (16%)
5 (7%)
58 (77%)
12 (16%)
26 (35%)
37 (49%)
Linfield College
Assessing NW
 Observations and recommendations
 Rated satisfactory or above
 #1 25%
#2 75%
#3 23%
#4 51%
 Only 6 submission (8%) received at least a satisfactory for all
four learning outcomes
 13 submissions (17%) received no satisfactory scores
 Most students submitted a single document
 Lack of understanding of what constitutes a goo exemplar
 Revision of learning outcomes
Linfield College
New NW Learning Outcomes
Courses with the NW designation are intended to help students
understand the scientific method. In particular, students must
1) an understanding of the theoretical and/or experimental
background of a particular topic or model, sufficient to form a
2) an ability to critically analyze results of scientific inquiry in light of
3) an understanding of how scientific results can be extended to
more general situations in contemporary society
Courses with NW designation address all of the above learning
Lessons Learned
 More attention needs to be paid to developing
students’understanding of what makes a good exemplar
Linfield College
Advantages of Using Taskstream
 Streamlines mechanics of evaluation process
 Additional reports available (summary statistics)
 Provides for more meaningful faculty discussions
 Between pairs of faculty evaluators
 Among members of working group (4 faculty)
 With faculty teaching courses in each area through sharing of
assessment results
 Student understanding of the LC assessment
 Individual student data is not assessed
 Problematic to ensure students take the task seriously
 Need to further develop reflective habit of mind
 Faculty development
 Use of ePortfolio
 Consistent articulation of relationship between LC learning
outcomes and course content
Linfield College
 Linfield College
 Charles Dunn - Curriculum Committee Chair, director of LC
Assessment, co-author of DRF
 Susan Agre-Kippenhan –Vice President of Academic Affairs, Dean of
 TaskStream
 Ben Coulter – Manager, Systems Implementation
 Jose Romero – Mentoring Services
 Lauren McCarty – Associate Director of Marketing
 Dara Wexler – Education Solutions Specialist
 Trudy Milburn - Education Solutions Specialist
Linfield College
 Questions – contact Martha VanCleave
 [email protected]
Linfield College

similar documents