Workshop Handout Slides - Learner Analytics Summit

Report
Analytics: Moving From Great Promise to
Tangible Success Workbook
University of
Maryland University
College
July 21-23, 2014
Dr. Linda L. Baer
Dr. Donald Norris
I. First Workshop:–9:00 – 10:30 am, July 22, 2014
Leadership, Culture and Readiness for Analytics
• Where are you?
• How ready are you?
• What’s your analytics maturity index?
Where are you?
Strategic Intelligence for Higher Education
What’s the best that can happen?
What will happen next?
What if these trends continue?
Why is this happening?
What actions are needed?
Where exactly is the problem?
How many, how often, where?
Key Questions Addressed by Analytics
Past
What happened?
Information
Insight
(Reporting)
Present
What’s happening
Now?
(Alerts)
How and why
did it happen?
What’s the next
best action?
(Modeling,
Experimental
design)
(Recommendation)
Future
What will happen?
(Extrapolation)
What’s the best/worst
that can
happen?
(Prediction,
Optimization,
Simulation)
Analytics at Work. Davenport, Harris and Morison.2010
ANALYTICS
STRATEGIC QUESTION
DATA
ANALYSIS AND
PREDICTION
INSIGHT AND
ACTION
http://www.educause.edu/library/resources/2012-ecar-study-analytics-highereducation
Defining Your Current Problem
Briefly explain your current problem:
Give it a code name (just for fun):
What about this problem lends itself to an analytics solution process?
Who would benefit if the problem
were solved?
What is your role in the
problem/solution?
What resources ($) may be available
to you and others?
Who has tried to solve this before?
Were they successful? Why or why
not?
DESIGN Worksheet
Write the Problem Statement:
What data do you have?
What data do you need? Gap between
what you have and what you need?
Who can get the data?
What campus team will need to be
involved?
What research, assessment, and
analysis will you do?
List criteria for completion/success:
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
DESIGN Worksheet
Describe current leadership and culture environment for evidence-based decision making supporting student success:
What role is leadership currently
playing?
What is the investment plan?
How are the stakeholders involved?
What campus team will need to be
involved?
How would you describe your culture
for data driven analytics to support
student success?
List criteria for completion/success:
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
ECAR MATURITY OVERALL MATURITY INDEX
Analytics Maturity Index
http://www.educause.edu/ecar/research-publications/ecar-analytics-maturity-index-highereducation
DESIGN Worksheet
Campus Readiness – see ECAR Analytics Maturity Index
Culture/Process
List criteria for completion/success:
•
•
•
•
•
•
•
•
•
•
Investment
Governance and
Infrastructure
•
•
•
•
•
•
•
•
•
Data, Reporting, Tools
Expertise
•
•
•
•
•
•
•
•
•
Second Workshop: 3:15-4:15 pm, July 22, 2014
Applying Analytics to Optimizing Student Success
• Baer Norris Ways to Use Analytics in Optimizing Student Success
• Stages of Student Success
• Case Study Examples
Stages of Student Success Analytics
• Static Reporting
• Dynamic Analysis and Intervention
• Optimization
Stages of Student Success Analytics
Stage I
Getting Started with
Analytics
Stage II
Accelerating the Progress
of Analytics
Stage III
Transforming/Optimizing Leveraging
Analytics
Technology,
Processes and
Practices
Basic ERP, LMS, Report
Writing Tools
First Ventures into
Analytics Tools and
Applications, Pilot
Projects /Dept Solutions
Combination of ERP, LMS,
Analytics Applications,
CRM, Advising)
First-Generation
Predictive Analytics
Sophisticated Data Warehouses with
Analytics, Embedded Predictive
Analytics, Leading to More Advanced
Practices
Skills and Values
Power Users Provide
Reports
Analytics Spread More
Broadly – Concept of
Analytics for the Masses
Pervasive Data, Information, and
Analytics Engages Everyone from
Front-Line Knowledge Worker to
Executives
Leadership
Leadership Focuses on
Data and Reporting
Leadership to Encourage
Evidence-Based Decision
Making
Strong, Consistent Leadership,
Analytics is a Strategic Imperative
Culture
Culture of Reporting
Culture of Evidence-Based
Decision Making
Reflected in Behaviors
Culture of Performance
Measurement and Improvement
Number of
Institutions in this
Category
3,000+
Approximately 800-900
Approximately 40-60 Mostly ForProfits and Online Universities
UMBC, UMUC are Among These
Characteristics
What is Your Stage of Development of
Student Success Analytics
Characteristics
Technology,
Processes and
Practices
Skills and Values
Leadership
Culture
Stage I
Getting Started with
Analytics
Stage II
Accelerating the Progress
of Analytics
Stage III
Transforming/Optimizing Leveraging
Analytics
The market is immature, but multiple solutions are emerging in each application
category
Education
Planning
Counseling and coaching
Risk Targeting and
Intervention
Transfer and
Articulation
Legacy ERP/SIS/LMS
Vendor point solutions
Homegrown point solutions
Sinclair’s MAP
Valencia’s
LifeMap
Austin Peay’s Degree
Compass
Central Piedmont’s Online
Student Profile
WICHE’s Predictive Analytics
Reporting
Direct-to-student
© 2010 Bill & Melinda Gates Foundation
|
19
Higher Education Analytics Landscape
Institutional Analytics (HR, Finance, Enrollment)
(Blackboard, COGNOS)
Student Success Analytics
“Retention CRM”
Analytics
(Starfish EARLY ALERT and
Starfish CONNECT, Hobsons)
At-Risk Student
Predictive Analytics
(Noel-Levitz, Mapworks,
EAB,
J. Gardner, Campus Labs,
CIVITAS Learning)
Instructional Analytics
Program Effectiveness
Analytics
(Knewton, MyLabs, ALEKS)
(Starfish INSIGHT v.1)
Starfish INSIGHT v.2-v.3
Adapted from Presentation by David Yaskin, President Starfish at IPAS Conference 2014
Integrate or Replace?
Learning Management Systems
(LMS) Module
Course clicks predict risk e.g. Blackboard
Retention Center, Blackboard Analytics,
Ellucian Signals, Desire2Learn Student
Success System
Student Information Systems
(SIS) Module
Instructors flag students and/or
demographics predicts risk e.g.
Jenzabar Retention, Ellucian Retention
Alert, Banner Retention Mgr
integrate/replace
Client Relationship
Management (CRM) Module
Instructors flag students + email
students to fix the problem e.g.
Hobsons Retain, Campus
Management Talisma
replace
Degree Audit and Planning
Academic Plans prevent mis-steps e.g.
Ellucian Degreeworks, Hobsons
AgileGrad, CollegeSource
u.Achieve/u.Direct
replace
integrate/replace
Starfish®
Enterprise Success Platform
integrate/replace
Tracking / Scheduling
Systems
Track meetings, schedule
meetings e.g. AdvisorTrac, SARS,
TutorTrac, AccuTrac
replace
Early Alert Systems
replace
Case Management Systems
integrate
Predictive Analytics
integrate
Student Survey Systems
(standalone)
Survey instructors + schedule
meetings e.g. GradesFirst,
Pharos, Symplicity
InsightAdvising
Walk a student through the steps
for success e.g. Student Success
Plan
Anticipate problems based on
data from systems e.g. Civitas
Learning, Education Advisory
Board
Anticipate problems based on
student’s self reporting e.g. Noel
Levitz CSI, EBI Map-works,
Campus Labs Beacon
Making Student Success An Institutional
Priority
Worksheet – Making Student Success an Institutional Priority
Establish a Sense of Urgency for Taking a Systemic and Systematic Approach to Student Success:
Form a Powerful Guiding Coalition:
Create a Vision/Strategy and
Communicate the Vision/Strategy:
Empower Others to Act on That
Vision/Eliminate Barriers:
Institutionalize New Approaches and
Behaviors, Focus on Changing Culture
and Behaviors, Demonstrate Successes:
Consolidate Improvements, Produce
Still More Change, Follow an
Expeditionary Strategy:
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Plan for an Leverage Short-Term
Wins/Leverage Existing Pilots and
Analytics Capabilities/Demonstrate
Improvements:
•
•
•
•
•
•
•
•
•
Third Workshop: 1:30-2:30 July 23, 2014
Putting the Pieces Together
• Getting Started from Where you Are
• Future of Student Success Solutions
• Build, Buy, Buddy, Borrow
• 3-year planning cycle
• Emerging now are a set of rapidly developing technology tools
to support students as they progress through a course and
program.
• Depending on the individual student’s understanding of specific
ideas, concepts, and operations, these tools have the capacity
to
• personalize the learning experience for every student
• chart individual pathways through course materials
STUDENT SUCCESS SOLUTIONS
Education
Planning
Counseling and
Coaching
Risk Targeting and
Intervention
Legacy
ERP/SIS/LMS
Vendor point
solutions
Homegrown point
solutions
Direct-to-student
Sinclair’s
MAP
Valencia’s
LifeMap
Austin Peay’s
Degree Compass
Central Piedmont’s
Online Student Profile
WICHE’s Predictive
Analytics Reporting
Transfer and
Articulation
Higher Education Analytics Landscape
Institutional Analytics (HR, Finance, Enrollment)
(Blackboard, COGNOS)
Student Success Analytics
“Retention CRM”
Analytics
(Starfish EARLY ALERT and
Starfish CONNECT, Hobsons)
At-Risk Student
Predictive Analytics
(Noel-Levitz, Mapworks,
EAB,
J. Gardner, Campus Labs,
CIVITAS Learning)
Instructional Analytics
Program Effectiveness
Analytics
(Knewton, MyLabs, ALEKS)
(Starfish INSIGHT v.1)
Starfish INSIGHT v.2-v.3
Adapted from Presentation by David Yaskin, President Starfish at IPAS Conference 2014
Integrate or Replace?
Learning Management Systems
(LMS) Module
Course clicks predict risk e.g. Blackboard
Retention Center, Blackboard Analytics,
Ellucian Signals, Desire2Learn Student
Success System
Student Information Systems
(SIS) Module
Instructors flag students and/or
demographics predicts risk e.g.
Jenzabar Retention, Ellucian Retention
Alert, Banner Retention Mgr
integrate/replace
Client Relationship
Management (CRM) Module
Instructors flag students + email
students to fix the problem e.g.
Hobsons Retain, Campus
Management Talisma
replace
Degree Audit and Planning
Academic Plans prevent mis-steps e.g.
Ellucian Degreeworks, Hobsons
AgileGrad, CollegeSource
u.Achieve/u.Direct
replace
integrate/replace
Starfish®
Enterprise Success Platform
integrate/replace
Tracking / Scheduling
Systems
Track meetings, schedule
meetings e.g. AdvisorTrac, SARS,
TutorTrac, AccuTrac
replace
Early Alert Systems
replace
Case Management Systems
integrate
Predictive Analytics
integrate
Student Survey Systems
(standalone)
Survey instructors + schedule
meetings e.g. GradesFirst,
Pharos, Symplicity
InsightAdvising
Walk a student through the steps
for success e.g. Student Success
Plan
Anticipate problems based on
data from systems e.g. Civitas
Learning, Education Advisory
Board
Anticipate problems based on
student’s self reporting e.g. Noel
Levitz CSI, EBI Map-works,
Campus Labs Beacon
Inventory of Your Student SolutionTool Sets
Educational
Planning
Legacy
ERP/LMS
Vendor Point
Solutions
Home Grown
Solutions
Direct to
Students
Counseling and
Coaching
At Risk and
Early Alert
Transfer and
Articulation
Predictive
Analytics
Build, Buy, Buddy, Borrow Solution Tools
• Build – homegrown systems, applications, processes, and
competences
• Buy - utilize solution providers to acquire systems, applications,
processes and competences
• Buddy – partner collaborate and share
• Borrow
• Mash-up Strategy combining all four
QUESTIONS
Build, Buy, Buddy, Borrow?
•
•
•
•
•
What are you seeking?
What do you already have?
Where do you want to go next?
What options work best?
What is the short term and long term strategy and plan?
DESIGN Worksheet
Describe current student solution tool decision making environment
What factors are shaping the BBBB
decision?
What are you seeking?
What is the short term and long term
strategy and plan?
What options work best in BBBB?
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
What do you already have?
Where do you want to go next?
•
•
•
•
•
•
•
•
•
•
IMPLEMENTATION Worksheet
Summarize the Implementation Strategy for year 1, 2, and 3.
What are the key elements and features?
Who does this involve?
What resources ($) are necessary?
What does success look like?
List specific outcomes and metrics:
•
•
•
•
•
Target 1: Student Solution Sets
Target 2: Build, Buy, Buddy, Borrow
Target 3: Roadmap for 3-Year Plan
Specific actions that are necessary?
•
•
•
•
•
•
Specific actions that are necessary?
•
•
•
•
•
•
Specific actions that are necessary?
•
•
•
•
•
•
How does this serve as a gateway to
analytics that optimize student
success?
SOAR Template and Analytics Readiness
Strengths
Leadership and Culture
Data Readiness and
Organizational Structure
Technology and Enterprise
Systems
Opportunities
Aspirations
Results
Roadmap: 3-Year Plan Evolution
2011
DESIGN
Ideation
PILOTS
Test & Learn
SCALE
Implement
OPERATIONAL
Measure, Monitor,
Optimize
2012
2013
2014
2015
Analytics: Moving From Great Promise to
Tangible Success Workbook
Dr. Linda L. Baer
[email protected]
Dr. Donald Norris
[email protected]

similar documents