Report

Assessment in mathematics education: The interface between the multiplicative conceptual field and the Rasch measurement model Caroline Long, CEA, University of Pretoria, [email protected] Presentation outline • Challenges for mathematics education • Theoretical tools • Theory of conceptual fields (Vergnaud, 1983) • Threshold concepts (Meyer & Land, 2005) • Zone of proximal development (Vygotsky, 1962, 1978) • Research study • Rasch measurement theory • Reflections on the outcomes • Educational implications • Application of the Rasch model in mathematics education settings • Further developments Challenges for mathematics teaching • The transition from working with whole number to rational numbers (and real numbers), is noted as a critical mathematical development in the transition years, Grades 6 to 10 (Usiskin, 2005), which in large part determines the mathematical future for learners. • Fractions, ratio and related topics like percent, are taught procedurally without regard for their complex interconnections and their applications in everyday situations (Kieren, 1976; Parker & Leinhardt, 1995; Lamon, 2007). • It is the assimilation and accommodation of the component subconstructs comprising rational number that are required for synthesis in the predicative form (Kieren, 1976; Vergnaud, 1983; Lamon, 2007). • The danger is that “tricks” are introduced to avoid real engagement with the concept (Davis, 2010). Theory of conceptual fields (Vergnaud, 1983 - ) • Mathematics is developed through encountering concepts embedded in problem contexts • No one-to-one mapping from mathematical concept to problem situation. In fact all such mappings are many-to-many • Concepts are never learned singly but in relation to other concepts • The multiplicative conceptual field include • • • at base multiplication and division, but also the problem situations which are modelled by multiplication and division and the various representations that may be required. • The purpose of mathematics education is to transform extant implicit and local conceptions of individual learners encountered in single problem situations into generalised mathematics concepts and theorems that may be applied to a class of related problems – from operational to predicative form Threshold concepts (Meyer & Land, 2005) • An historical perspective • • • increasingly abstract systems, radical conceptual shifts (Dantzig, 2007) parallel conceptual reorganisation in the classroom (Sfard, 1991) resonates with the notion of “epistemological obstacles” (Brousseau, 1983) • A threshold concept is a critical concept which provides the gateway (to higher mathematics), and which by corollary inhibits mathematical progress where learners have not gained mastery. • These conceptual gateways may be • • • • • transformative – “occasioning a significant shift in perception of a subject” “troublesome” knowledge - defines “critical moments of irreversible conceptual transformation” irreversible – “unlikely to be forgotten”, “unlearned only through considerable effort”, and integrative – “exposing the previously hidden interrelatedness of concepts” The danger of substitution by naïve concepts which confine progress Zone of proximal development (Vygotsky,1962) Theory of conceptual fields builds on the work of Piaget and Vygotsky • “(t)he discrepancy between a child’s actual mental age and the level he reaches in solving problems with assistance indicates the zone of his proximal development” (Vygotsky,1962, p. 102). • Current interpretation - a conceptual and cognitive space comprising the distance or extent from current proficiency to a higher level, over which learning with assistance from teachers or peers may occur . There may be hierarchies of concepts that build on one another and that some concepts may be within the learner’s current “zone of proximal development”. More abstract concepts may require prior mastery or at least partial mastery. For example an understanding of ratio may precede an understanding of time distance and speed. Assessment instruments The contextual features discussed lead us to assert that educational experiences in mathematics are ordered somewhat hierarchically and aligned with learners current proficiency levels. How do we identify for individual learners, or groups of learners a suitable current conceptual space, an educational locale (or zone of proximal development) which is neither too tedious because the associated problem solutions are obvious, nor too difficult for the learner and hence frustrating or demotivating? Can assessment instruments be designed that make explicit a conceptual and cognitive pathway that provides insights for the teacher, in the preparation of learning sequences and subsequent engagement with individual learners? And in this process, can instruments help identify threshold concepts? Research design Instrument construction, 36 items (18 MC and 12 polytomous items) • TIMSS 2003 released items, adapted slightly in some cases • Matrix design (12 common items and a further 24 distributed across 4 groups) Learners at two schools, across three grades, 7, 8 and 9, 16 classes, and 330 learners Reasonably well-functioning schools, each covering a range of South African demographics Follow-up interviews on 4 items of graded difficulty with 6 learner groups (21 learners in all, located at specified proficiency levels as determined from a Rasch analysis). Person-Item Map PERSONS [locations=estimated proficiency] 7.0 6.0 Persons and items on the same scale, according to probabilistic estimates. Selected ratio, rate and proportion items are described on the right. Levels* aligned with logits are demarcated by bands from (- ,-2) [-2, -1), [-1,0), [0, 1), [1,2), [3, + ) 5.0 4.0 3.0 2.0 1.0 0.0 -1.0 -2.0 -3.0 o = 2 Persons ITEMS [locations=estimated difficulty] o | | | 34 | | | | | 30. Find the average speed | 26 given distance, 160 km, and | time, 2.5 hours. | | o | o | | | | 27 | o | | 10. 2.4 litres for 30 hours, o | how many litres for 100 o | hours? oo | 35 | 31 30 ooo | o | oo | oo | o | 28 5. Identifies proportional ooooo | 7 ooooo | 32 share of amount 45 000 oooooooooo | 29 10 divided in the ratio 2:3:4. ooooooo | ooooo | 18 19 ooooooooooooo | 9 8 15 oooooooooooo | 33 16 oooooooooooo | 14 25 5 20. Selects fraction oooooooooo | representing part to whole ooooooooooooooooooo | 3 21 ratio. ooooooooooo | 23 oooooo | 4 24 ooooooooo | 20 6 12 11 ooooooooo | ooooooooo | 22 oooo | ooooo | 2 17 1. Given a proportional o | 13 1 problem, able to apply ooooo | multiplicative reasoning, | o | 1:3:: 9:x. | A note on terminology “Levels” • overused, a place holder, • familiar (Griffin, 2006; Pisa 2009; TIMSS) • denotes a defined position on a unidimensional line “Locale” • an area, a domain, or range on the 1D line • a difficulty locale (for a subset of problem situations) • a proficiency locale (for a subset of learners currently exhibiting a similar proficiency) Criterion zone (Wilson, 2005) Item analysis by level and analytic category Mathematical situation Notation Range Mathematical structure Proportional sharing Natural language, whole numbers Less than 30 20. Selects fraction representing part to whole ratio. Part-whole comparison Inclusive ratio Fraction notation 30 16 of total (16 + 14) Level 6 [1,L 2) Level 4 [0,1) Level 3 [-1, 0) 1. Given a Collecting proportional bottles problem, apply multiplicative reasoning, 1: 3 :: 9: ? Level 1 [- ∞, -2) Context Level 2 [-2,-1) Description Birthday M1 1 9 16 16 14 5. 45 000 zeds Family shared in sharing unequal money proportions, of 2, 3 and 4. Proportional sharing 10. 2.4 litres to Capacity30 hours, how time many for 100 hours? Proportion Multiplicative comparison 30. Average speed given distance and time. Rate calculation Car rally, Distance, time, speed Natural numbers in thousands, fraction and ratio notation Decimal notation Decimal notation 45000 (1000) 100 M1 (time) first half total 16 30 2:3:4 4 of 45 000 M1 4 9 9 f (x) = 2 .4 30 160 160 km = 2 .5 h 64km/h d t s Response process M2 3 x Identify multiplicative relationship Selects part-whole M2 (children) relationship 16 30 M2 x 45 Identifies proportional relationship Identifies M1 M2 (capacity) (hours) proportional 2,4 30 relationship and x 100 multiplicative operator Identifies M1 M2 (hours) (km) proportional 2.5 160 relationship and 1 x calculates unit rate Distractor analysis Item 1 (n=330) Quartile Low Q ML Q MH Q High Q Item analysis Selected items (Item 1, Item 20, Item 5, Item 10, Item 30) at graded levels of difficulty, Mean % 58 84 93 98 Simple ratio Item 20 (n=83) Quartile Low Q ML Q MH Q High Q Mean % 28 70 81 86 Part-whole comparison levels of proficiency: means of 4 quartile groups (LQ, MLQ, MHQ, HQ). The key mathematical concepts for success on the item noted below. Item 5 (n=330) Quartile Mean % 24 35 55 85 Multiplicative relationship, unequal proportions Item 10 (n=330) Quartile Mean % Multiple choice distractor plots: percentage of quartile group selecting each false choice Low Q ML Q MH Q High Q Low Q ML Q MH Q High Q 8 19 28 65 Finding the unit rate Item 30 (n =330) Quartile Low Q ML Q MH Q High Q Mean % 0 0 5 41 Understanding the relationship of distance speed and time MCF concepts embedded in the problem situations at each level. Associated errors by level • Part-whole of discrete and continuous quantities • Both fraction and ratio meaning of fraction notation Level 2 [-2-1) Errors • Confusion between fraction measure and ratio meaning and with fraction notation LQ MLQ MHQ (-1.9) (-1.1) (0) HQ (1.1) 53% 88% 93% 94% 37% 61% 79% 91% 16% 35% 57% 78% 14% 22% 28% 58% 0% 3% 9% 29% • Multiplicative relationship between sets of ratios • Fraction equivalence • Part-part and part-whole ratios • Percent concept and notation • Connect probability with fraction measure • • • • Natural number confusion Just add the percent sign % Language difficulty Ignoring part of the problem • Covariant relationships • Comparative relationships • Rational number, operator subconstruct • Identification of ratio Level 3 [-1,0) Summary information Level 1[-3,2) MCF concepts • Multiplicative comparison (operator construct) • Confusion with operator construct • Confusion of additive and multiplicative relationship • Ignoring part of the problem • Percentage increase (operator subconstruct) • Applying ratio operator construct to find the sample • Fraction measures, addition (subtraction), multiplication (division) • Ratio and rate concepts • Multiplication (division) of decimals • Consider only the numerator • Additive reasoning • Lack of fluency with multiplication and division • Confusion with terminology • Probability and statistics concepts, “sample”, “random” • Rate and ratio Level 5 to 7 ( > 1) Mean percentages correct for items by level and by quartile Level 4 [0, 1) • Multiplicative comparison • Percent, identifying referents • Covariant relationships • Reasoning with unknowns • 2 step problems • Confusion with percent language and referents Interviews: Item level by learner proficiency Item 5: Three brothers Thabo, Samuel and Dan, receive a gift of 45 000 zeds from their father. The money is shared between the brothers in proportion to the number of children each one has. Thabo has two children, Samuel has three children, and Dan has 4 children. How many zeds does Dan get? Carola, (0.18 logits), Interview data Shiluba, (0.26 logits) I didn’t really completely understand but I got a part answer. My logic behind it … I divided 3 into the R45 000 … And this gave me R15 000. … I think I then timesed by two over one which gave me 30, which I then divided by three which then gave me R10 000. Linda, (0.12 logits) I realise I did it wrong … really bad … I worked out all of them together. All the children. And I divided them by the zeds. Then I got 5 000. Then I think you would divide the 5 000 by 4. … I did that wrong, I don’t know why. Kate,(0.12 logits) I said 45 000 divide by 3 and I got 9 000. And then I don’t know what I did and I got 5 000 Interviews with the middle high proficiency group, on Item 5. Written responses on the left, explanations of thinking about the problem on the right. I divided all of them 4 500 divide by 2 … … For Sam, for Thabo, and I did the same for Samuel (divide by 3) and Dan (divide by 4). … I don’t know why I did that (laughs, and group laughs with her). I never worked out how much Dan alone gets. If I had divided 5 000 by 4, I would have got the answer. You mean multiply by 4. I have no idea (and laughs) Interview transcript Speaker Transcript Interviewer OK, so has anybody got a picture of what is happening here? Shiluba We were supposed to add up the children and divide it into 45 000. And then we would get 5 000. And what does that 5 000 mean. 5 000 for … Dan? No, 5 000 for each of the children For each of the children … (Murmurs of realisation from the group) You get 15 000. (The group corrects her.) You get 20 000. Interviewer OK. Did you get that? (Confirmation and chuckling from the group) So where was the problem? Was it in the reading? Shiluba In the “How many zeds does Dan get?” Because I didn’t really, … I kind of shut off … Which is that part? The middle part? Ja, “The money is shared between the brothers in proportion to the number of children each one has”. OK, So I disregarded the children. Item 5, middle high quartile group transcript Discussion occurred after individual explanations of reasoning about the problem. Common errors are noted in brackets, Low (L1, 2) Fluency with addition and subtraction (Mistakes, use of addition where multiplication more efficient) (Use of addition where multiplication more efficient) Fluency with multiplication and division, proportional sharing (Confusing multiplication and division in proportional shares) Some fluency with multiplication and division, sharing (Lack of fluency with multiplication and division; Abandon sense making Applying incorrect algorithm) Some fluency with multiplication and division, sharing (Confusing multiplication and division; Incorrect conception of proportional sharing) Some fluency with multiplication and division (Misreading of the problem) Conversions, operations with decimal fractions Conversions of measurement units Conversions from fraction to percent Multiplying decimal fractions Fluency of operations with decimal fractions Conversions from percent to decimals Percent change percent increase Percent - fraction type Converting ratio relationship to a percentage Finding percent increase (finding percent decrease) Percent, fraction type (Mixing objects, adding percents and quantities) (Confusing percent with amount, rather than ratio. Difficulty changing referent when converting ratio to percent) Using ratio understanding to find proportions, finding unit rate Reasoning with ratio and rates (Lack of fluency, multiplication and division) Identifying variables and relationships Identifying scalar and function operators Identifying variables and relationships (Confuses relationships incorrect use of algorithms) (Incorrect identification of relationships, working back from the answer) (Incorrect identification of relationships; working backwards from the answer) Fluency with algorithms (Applying algorithms in conceptually clumsy ways, using inappropriate algorithms Misuse of equals sign) (Incorrect use of algorithms, Mistakes with cancelling, when applying algorithms) (Rote use of algorithm, sometimes correct) (Rote use of algorithms) Multiply and divide Fluency with addition and subtraction Fractions, decimals, operation On the vertical axis, the key concepts identified are listed from lesser difficulty to greater difficulty. Middle low (L3) Percent, concept, operations On the horizontal axis the proficiency levels are arranged from high to low. Middle high (L4) Using ratio and rate concepts Composite information from summaries of item analyses at levels of difficulty and summaries of interview data at levels of proficiency. High ( L 5,6,7) Identify Algorithms, symbolic variables and notation relationships Composite summary Add and subtract Levels Description of concepts and skills in bold, (errors enclosed in brackets) Reflections: theory of conceptual fields and Rasch measurement • The insights gleaned from such a study have application in their local context, for example for the population targeted in the study, Grades 7, 8 and 9 in South African schools. – While some learners have yet to achieve fluency with multiplication and division, others need extension at the higher end of the spectrum of multiplicative structures, for example, rate and double proportion problems – The selection of proximate items not yet correctly mastered by learners is easily executed given the outcomes of the Rasch analysis, in conjunction with the theoretical analysis of the multiplicative conceptual field. – The method advanced in this study follows that advocated Wilson (2005) for the most part, though further cycles of refinement may be implemented for any context. The interactive relationship of theory and measurement • This development and validation may elicit joint orderings of sets of items within a conceptual field, and respondents, the distances between which have a meaningful interpretation and permit the specification of a ZPD. • We may infer from these relationships, positing educationally pertinent interventions. • We claim that, given theoretical analysis, we may infer a degree of hierarchical ordering that, with further cycles replicated in these grades, may prove to be a helpful instrument in the planning and support of teaching. • Provided that the instrument has been carefully constructed to include potential threshold concepts, these thresholds may be hypothesised and perhaps identified. Concluding remarks The theory of conceptual fields raises some questions about current articulations of the intended curriculum From relationships of item difficulty and learner ability, educationally pertinent interventions can be targeted at groups of similar learners, as a means of reaching all learners efficiently – notion of ZPD The identification of threshold concepts - somewhat more complicated • From an historical perspective, the topics which caused particular consternation, for example the critical concepts that led to the construction of new number systems, the zero, the integer, the rational number etc., may be the starting point. • The theoretical insights (see for example, Dantzig, 2007) into the topics which appear particularly troublesome and which require attention Critical for threshold concepts to be hypothesised and then included in assessment research Further developments: A model of assessment (Bennett & Gitomer, 2009) Accountability component Professional development component Formative component Accessible and powerful assessment Accountability • A process of teacher involvement at the various phases of the item construction • Accountability becomes intrinsic to the teacher Professional development component • Deep domain specific knowledge • Expert input on identified problem areas Formative assessment • A classroom component where teachers develop, experiment and reflect on the suitability of current assessment in relation to the knowledge domain • Professional learning communities (PLCs) Critical questions for future research How may the theoretical notions of conceptual fields (threshold concepts and zones of proximal development) inform educational assessment and measurement? And reciprocally, how may measurement theory highlight and extend insights into conceptual fields, threshold concepts and zones of proximal development as they manifest in the progressive development of learning? Thank you for your attention [email protected]