How best to evaluate a VLE

How best to evaluate
the effectiveness of a
Chris Rowell
Technology Enhanced Learning Advisor – Regent’s
College, London.
Session outcomes:
Identify ways of evaluating technology
enhanced learning
Understand how to evaluate a VLE using
qualitative data
Identify ways in which effective academic
practice in VLE’s can be supported and
The context
Regent’s College:
Central London
7 Schools
Fees: £13, 000 to 21,000
4000 students
350 academics
My role:
learning technologist
How do you evaluate your VLE?
Spend 10 mins discussing the following:
Does your institution evaluate its VLE?
How is your VLE evaluated?
Who does it?
Why do they do it?
Baume (2010) paraphrasing Kirkpatrick (1992):
 Did people like it?
 What have they learnt from it?
 Have they applied what they learnt to their
 Have the results improved?
My research project at Regent’s
Quantitative data:
My research:
1. Online questionnaire
2. Interviews
3. Analysis/report
Interview questions
Your task:
Arrange the interview question in order of
Interview questions for lecturers
Starting Bb
1.a) How easy was it for you to start adding content to your module area?
b) What were the main difficulties you faced when creating a new module area in Bb (if any)
2. Have you experienced any technical difficulties using Bb?
Staff/Student interaction
3. Do you think the students have an increased expectation of your availability since Bb was
4. Do you believe that Bb can replace a part of your face-to-face contact with your students?
5. Do you think that Bb affects the attendance in your lessons?
6. In your opinion, has Bb changed the quality of interaction with your students?
7. Has the introduction of Bb affected your relationship with students?
8. Do you think that Bb enables you to become more or less student-centred in your teaching?
9. Has Bb affected (or could affect) the way you teach in the classroom?
10. How effective is Bb at supporting collaborative activities on your course?
11. How has Bb affected the way you assess students work?
12. Have you changed the amount of feedback to students since the introduction of Bb?
13. Do you use Bb to evaluate how much work the students are doing on your module?
14. Do you use Bb to identify any students who might be “at risk” on your module?
15. Has the introduction of Bb changed your overall workload?
Future needs
16. Would you like more training or on-to-one support to develop your Bb module? If so, what areas of
Bb would you like to develop?
17. Would you like to make any other comments about Bb?
Revisiting our evaluation criteria
 Did people like it?
 What have they learnt from it?
 Have they applied what they learnt to their practice?
 Have the results improved?
Who gets the report?
Why bother?
What I have got out of it:
 Justification of what I am already doing
 Influenced future direction of the VLE
 Ticks SM’s boxes
 Unintended consequences
Any questions?
Contact me:
[email protected]

similar documents