Research on formative assessment practices 2013 AERA Classroom Assessment SIG Symposium: Advancing Research in Classroom Assessment Dylan Wiliam Section 3: Formative assessment 2 Four chapters Formative and summative aspects of assessment: Theoretical and research foundations in the context of pedagogy (Black) Gathering evidence of student understanding (Heritage) Feedback and instructional correctives (Wiliam) Examining formative feedback in the classroom context: New research perspectives (Ruiz-Primo & Li) Formative and summative aspects of assessment: Theoretical and research foundations in the context of pedagogy Paul Black Formative and summative assessment 4 A simple model of instruction aims Planning activities Interaction Review of the learning Summing up Regulation of learning processes Clear Proactive Interactive Retroactive Tensions arise between summative and formative functions, especially in terms of instruments, methods, and agents The role of teachers 5 Teachers’ exclusion from summative functions of assessment weakens the kinds of inferences that can be made But, inclusion of teachers brings its own issues Poor construct definition makes assessment design more of a discretionary process than it should be Teachers model their assessments on tests “Summative drives out formative” Alignment of curriculum, assessment and instruction Case studies (Australia, England, Scotland, Sweden) Gathering evidence of student understanding Margaret Heritage Eliciting evidence 7 Two fundamental purposes of assessment provide information on current achievement inform future instruction Sources and quality of evidence 8 Sources of evidence What learners say, write, make, or do (Griffin 2007) Interactions (questions, discussions) Tasks Directed (model-eliciting activities) Partially directed (activities) Undirected (observations) Quality of evidence Construct underrepresentation Construct-irrelevant variance systematic random (i.e., unreliability) Learning progressions 9 “What gets better when someone gets better” Two key aspects 1. progressions lay out in successive steps, increasingly more sophisticated understandings of core concepts and principles in a domain, and 2. progressions describe typical development over an extended period of time. Two approaches to their development Top-down (theoretical) Bottom-up (empirical) Feedback and instructional correctives Dylan Wiliam Reviews of research on feedback 11 Fuchs & Fuchs (1986) Nyquist (2003) Natriello (1987) Brookhart (2004) Crooks (1988) Allal & Lopez (2005) Bangert-Drowns et al. (1991) Köller (2005) Dempster (1991, 1992) Brookhart (2007) Elshout-Mohr (1994) Wiliam (2007) Kluger & DeNisi (1996) Hattie & Timperley (2007) Black & Wiliam (1998) Shute (2008) Feedback: an evolving concept (Brookhart, 2007) 12 Conceptualization Information about the learning process… … that teachers can use for instructional decisions… …and students can use to improve performance… …which motivates students Source(s) Scriven (1967) Bloom, Hastings and Madaus (1971) Sadler (1983; 1989) Natriello (1987); Crooks (1988); Black and Wiliam (1998); Brookhart (1997) Back to the future 13 “These considerations of utility and alternative interventions suggest that even [a feedback intervention (FI)] with demonstrated positive effects on performance should not be administered whenever possible. Rather, additional development of [feedback intervention theory] is needed to establish the circumstance under which positive FI effects on performance are also lasting and efficient and when these effects are transient and have questionable utility. This research must focus on the processes induced by FIs and not on the general question of whether FIs improve performance—look at how little progress 90 years of attempts to answer the latter question have yielded.” (Kluger & DeNisi, 1996 p. 278) Dual pathway theory (Boekaerts) 14 “It is assumed that students who are invited to participate in a learning activity use three sources of information to form a mental representation of the task-in-context and to appraise it: (1) current perceptions of the task and the physical, social, and instructional context within which it is embedded; (2) activated domain-specific knowledge and (meta)cognitive strategies related to the task; and (3) motivational beliefs, including domain-specific capacity, interest and effort beliefs.” (Boekaerts, 2006 p. 349) 15 As a result of the appraisal, the student activates energy and attention along one of two pathways the growth pathway (increasing competence) the well- being pathway (prevent harm, threat or loss) Integration of other theories Mindset (Dweck, 2000) Mastery and performance goals (Dweck, 2000) Interest (Hidi & Harackiewicz, 2000) Self-regulated learning (Deci & Ryan, 1994) Examining formative feedback in the classroom context: New research perspectives Maria Araceli Ruiz-Primo and Min Li Formative assessment in the classroom 17 Updated review of research on feedback in classroom learning Vast No majority of studies are poorly designed control groups Feedback effects evaluated in the instructional session No evidence of pedagogical orientation (e.g., orientations to the purpose of feedback, or how it could be used) A new, expanded, notion of feedback 18 Feedback should: Be seen as a process guided by the learning goals towards which the teacher and students work Actively involve students in the process Be considered as an instructional scaffold that goes beyond written or oral comments. Be specifically intended to improve learning outcomes Ensure its usefulness by making feedback accessible and practical. Consider different sources of information Demonstrate, over time, alignment with a learning trajectory Future priorities 19 Defining feedback Understanding variability in feedback practices Understanding feedback in use Reviewing feedback research from new theoretical understandings Major unresolved questions 20 Can teachers be involved in the summative assessment of their students without compromising the potential of formative assessment to improve achievement? How can learning progressions be developed? How can we better theorize the impact of feedback interventions?