Instructional Sensitivity on assessments is?

Report
Yellow Belt: Lesson #4
Instructionally Sensitive
Assessment: Embedding Daily
Classroom Assessment within
Instruction for Common Core
Skill Building
School Certification
A Process of Discovery, Support and Mastery
THE NATIONAL PATHWAY
CONDITIONS FOR SUCCESSFUL
IMPLEMENTATION
Curriculum Mapping Implementation
Vision
+
Skills
+
Incentives
+
Resources
+
Plan
Action Plan
=
Sustainable
Change
Skills
+
Incentives
+
Resources
+
Plan
Action Plan
=
Confusion
Incentives
+
Resources
+
Plan
Action
Plan
=
Anxiety
Resources
+
Plan
Action
Plan
= Resistance
Plan
Action
Plan
=
Frustration
=
Treadmill
Vision
+
Vision
+
Skills
+
Vision
+
Skills
+
Incentives
+
Vision
+
Skills
+
Incentives
+
Resources
Key
Questions:
Plan: Provides the direction to
Vision:
The “Why are we doing this?” to combat confusion.
Resources -- "Do we have tools, time, and training to map effectively?"
Vision
-"Why
are
we
doing
this?"
eliminate the treadmill effect.
Skills: The skill sets needed to combat anxiety.
Skills -- "How do we build effective maps?" Action Plan -- "Over the next three years, do we have attainable
Incentives: Reasons, perks, advantages totimelines
combat resistance
and goals? Who will be the responsible parties for
Incentives -- "How will mapping improve
Resources:
Tools
and
time
needed
to
combat
frustration.
implementations,
monitoring, and feedback?"
teaching and learning?"
Knoster, T., Villa, R., & Thousand, J. (2000)
Learner Objectives
• Define Instructionally Sensitive Assessments
• Differentiate between factors of instructionally
sensitive and insensitive assessments
• Discuss evaluative dimensions for identifying
instructionally sensitive assessments
• Explain the classroom implications for designing and
utilizing instructionally sensitive assessments
• Identify strategies for embedding daily classroom
assessment within instruction for Common Core skill
building
Pre-Lesson Reflection
• Thinking back to what was covered in previous
lessons about assessments and the various types,
formative, summative and performance-based, what
things do you see that can impact student
performance on assessments?
Overview of Lesson 4
• Part 1—A Close Look at Instructionally Sensitive
Assessment
o Origins and background information on instructional sensitivity
• What is instructionally sensitive assessment?
• What is instructional insensitivity?
• How can instructional sensitivity be measured?
o Framework for identifying instructional sensitivity
o Implications for Teachers
• Evaluative dimensions for identifying instructional sensitivity
• Shifts in Taxonomy
o Cognitive Rigor Matrix—Karen Hess
Overview of Lesson 4
• Part 2—Stepping Into The Classroom
o Suggestions for Utilizing Instructionally Sensitive
Assessments
• Classroom Assessment Techniques
• Daily Formative Assessments
• Examples of Assessment Items
Part I:
A Close Look at
Instructionally Sensitive
Assessment
Origins and Background Information about
Instructionally Sensitive Assessment
• What is Instructionally Sensitive
Assessment?
o Instructional Sensitivity—the extent to which
student performance on a test or item reflects
the instruction received. (Polikoff 2010)
o Instructional Sensitivity—the degree to which
students’ performances on a test accurately
reflect the quality of instruction specifically
provided to promote students’ mastery of
knowledge and skills being assessed.
(Popham 2007)
Origins and Background Information about
Instructionally Sensitive Assessment
• What is Instructionally Sensitive Assessment?
o Also known as “Instructional Validity”
o Research base—1960s and 70s with the start of criterionreferenced assessments
• A test that provides a basis for determining a candidate’s
level of knowledge and skills in relation to a welldefined domain of content.
o As criterion-referenced assessment gained in popularity
over the last 30 years, instructional sensitivity became
assumed rather than studied. (Polikoff 2010)
Origins and Background Information about
Instructionally Sensitive Assessment
• What in Instructional Insensitivity?
o Assessment that are incapable of distinguishing between
effective and ineffective instruction. (Popham 2007)
o Instructional insensitivity due to:
• Socio-economic status
• Inherited academic aptitude
• Prior knowledge
• Misalignment of tested and taught content
• Flaws in test design or construction
Origins and Background Information about
Instructionally Sensitive Assessment
• How can Instructional Sensitivity be
measured?
o Statistical methods based upon test scores
o Instruction-focused methods based on combination of test
scores and teacher-reported instruction
o Methods based upon expert judgment
Reflection…
• Considering the origins and background information
about Instructional Sensitivity, how do you see this
impacting your students on assessments?
Proposed Framework for Identifying
Instructional Sensitivity
• Framework designed for supplying accurate and
credible answer to how instructionally sensitive is a
given accountability assessment . (Popham 2007)
o Identified through well-formed rubrics
• Permit qualitative differentiations
• Two-directional
o Relevant evidence that contributes to determination
• Judgmental Evidence
o Actual test development
• Empirical Evidence
o Data gathered from actual student assessment performances
Proposed Framework for Identifying
Instructional Sensitivity
Reflection…
• Looking at the proposed framework for identifying
instructional sensitivity, what challenges do you see
in trying to evaluate your assessments?
Implications of Instructionally
Sensitive Assessments for Teachers
• Designing your classroom assessments
o Consider the four evaluative dimensions:
• Number of curricular aims assessed
• Clarity of assessment targets
• Items per assessed curricular area
• Item sensitivity—SES/Aptitude Contamination
o Important to evaluate assessment development,
especially for use with standards’-based
instruction and reform
Implications of Instructionally
Sensitive Assessments for Teachers
• Power of assessments
o The power of assessments, regardless of whether they take the form of a
classroom quiz, a standardized test, or a statewide assessment battery,
resides in their close connection to objectives and instruction. (Airasian,
Miranda 2002)
• To align objectives and instruction with Common
Core Standards:
o Instructional shifts must occur!
o Shifts in taxonomy
• Cognitive Rigor Matrix--Hess
Implications of Instructionally
Sensitive Assessments for Teachers
Reflection…
• One of the biggest areas for change in our classrooms
that results from considering instructional sensitivity
is insuring that our assessments align with our
instruction. What are some other areas for change?
Part II: Stepping into the
Classroom using
Instructionally Sensitive
Assessments
Suggestions for Using Instructional
Sensitive Assessment
• Check student learning as you go through:
o Classroom Assessment Techniques focus on
aligning assessments more closely with the
instructional strategies actually used with
children
o Meaningful assessment involved examining the
learner’s entire conceptual network, not just
focusing on discreet facts and principles.
o Using a wide variety of assessment tools allows
teachers to determine which instructional
strategies are effective and which need to be
modified.
(Badders, 2000)
Format
ASSESSMENT FORMATS
Nature/Purpose
Stage
Oral and written responses based on individual experience
Baseline Assessments
Baseline
Assess prior knowledge
Paper and Pencil Tests
Multiple choice, short answer, essay, constructed response,
written reports
Formative
Assess students acquisition of knowledge and concepts
Embedded
Assessments
Assess an aspect of student learning in the context of the learning
Formative
experience
Oral Reports
Require communication by the student that demonstrates
scientific understanding
Formative
Interviews
Assess individual and group performance before, during, and
after a science experience
Formative
Performance Tasks
Require students to create or take an action related to a problem, Formative and
issue, or scientific concept
Summative
Checklists
Monitor and record anecdotal information
Investigative Projects
Require students to explore a problem or concern stated either by
Summative
the teacher or the students
Extended or Unit
Projects
Require the application of knowledge and skills in an openended setting
Portfolios
Assist students in the process of developing and reflecting on a
Formative and
Summative
Summative
Formative and
(Graph
from
Badders,
Suggestions for Using Instructional
Sensitive Assessment
• Verify that your classroom instruction is meeting the
needs of your students through:
o Daily Formative Assessment to evaluate student learning and
guide/change/modify future instruction
• Observations
• Discussion
• Graphic Organizers
• Learning Logs
• Self-Assessments
• Individual Whiteboards
• Think-Pair-Share
• Exit/Admin Slips
o TeachingChannel Examples
Daily Formative Assessments to
Guide Instruction
• TeachingChannel.org Examples
Suggestions for Using Instructional
Sensitive Assessment
• Evaluating Assessment Items
o Math
o ELA/Literacy
ELA Example
MA State
Test
ELA Example
PARCC Grade 6 Technology-Enhanced
Selected-Response Item
Part A
Choose one word that describes Miyax based on evidence from the text. There is more than one
correct choice listed below.
A.
reckless
B.
lively
C.
imaginative*
D.
observant*
E.
impatient
F.
confident
Part B
Find a sentence in the passage with details that support your response to Part A. Click on that
sentence and drag and drop it into the box below.
Part C
Find a second sentence in the passage with details that support your response to Part A. Click on
that sentence and drag and drop it into the box below.
Math Example
Math ExamplePARCC 7th Grade Mathematics Question
Sample PARCC Assessment
Learner Objectives
• Define Instructionally Sensitive Assessments
• Differentiate between factors of instructionally sensitive and
insensitive assessments
• Discuss evaluative dimensions for identifying instructionally
sensitive assessments
• Explain the classroom implications for designing and utilizing
instructionally sensitive assessments
• Identify strategies for embedding daily classroom assessment
within instruction for Common Core skill building
Next Steps
• Evaluate the types of assessments, both formative
and summative, that are used in your school and
classroom.
• Begin having discussions among your colleagues
about if the assessments truly reflect the instruction
that students receive.
• As a group, work to develop and change assessments
to align with current instruction and Common Core
Standards.
Knowledge Check
Question: Instructional Sensitivity on
assessments is?
a) The emotional impact an assessment has on
learners.
b) The degree of difficulty in assessments.
c) The extent to which student performance on an
assessment reflects the instruction received.
Knowledge Check
Question: Instructionally insensitive assessments are
generally due to:
a) Socio-economic differences of learners
b) Flaws in test design or construction
c) Inherited academic aptitude
d) Prior knowledge
Homework Assignments
Homework Assignment
• FIRST
o Select an assessment that you currently use for one of your units of
instruction
o Prepare to evaluate it through the lens of instructional sensitivity
• How well does your assessment indicate the extent to which student
performance on a test or item reflects the instruction received?
• NEXT
o Use the 4 Evaluative Dimensions and Framework Diagram to identify if
your assessment is instructionally sensitive or insensitive
• FINALLY
o Use Hess’ Cognitive Rigor Matrix to enhance/modify your assessment to
align more closely with your instruction and student learning gains
Submit your assignment to [email protected]
THANK YOU!
Important for Attendance: Go to next screen and access the URL
to complete the survey to be credited for attendance.
Attendance & Evaluation
Access the URL below and complete the short survey to
record your attendance and provide feedback on this
session.
https://www.surveymonkey.com/s/YBLesson4
Resources
• Airasian, Peter, and Helena Miranda. "The Role of Assessment
in the Revised Taxonomy." Theory Into Practice. Volume 41,
Number 4 ed. Columbus, OH: Ohio State University, 2002. 249254. Print.
• Badders, William. "Methods of Assessment, William Badders."
Education Place®. N.p., n.d. Web. 4 Apr. 2013.
<http://www.eduplace.com/science/profdev/articles/badders.ht
ml>.
• D'Agostino, J. V., Welsh, M. E., & Corson, N. M. (2007).
Instructional sensitivity of a state standards-based assessment.
Educational Measurement, 12(1), 1-22.
• "Examples of Formative Assessment." West Virginia
Department of Education. N.p., n.d. Web. 4 Apr. 2013.
<http://wvde.state.wv.us/teach21/ExamplesofFormativeAssessm
ent.html>.
Resources
• Polikoff, M.S. (Winter 2010). Instructional sensitivity as a
psychometric property of assessments. Educational
Measurement: Issues and Practice, 29(4), 3-14.
• Popham, J. W. (2007). Instructional insensitivity of tests:
Accountability's dire drawback. Phi Delta Kappan, 89(2), 146155.
• Popham,W. J., & Yalow, E. (1982). Determining the instructional
validity of the SSAT-II. Culver City, CA: IOX Assessment
Associates.
• Popham, J.W. (2007). Determining the instructional sensitivity
of accountability tests. A Paper delivered at A presentation at
the annual Large-Scale Assessment Conference, Council of
Chief State School Officers, San Francisco, California, June 2528, 2006
Resources
• http://schoolleader.typepad.com/schoolleader/2012/03/encouraging-signs-from-new-teachersurvey-on-standardized-assessment.html
• http://www.education.com/reference/article/criterion
-referenced-tests1/
• http://www.cde.ca.gov/ta/tg/sr/documents/rtqgr6mat
h.pdf
• http://www.doe.mass.edu/mcas/2008/release/g5ela.p
df

similar documents