A Cross-Battery Approach to Assessment of Psychological Processing Disorders Milton J. Dehn, Ed.D. Winter 2011 Schoolhouse Educational Services [email protected] Copyright Notice; Copyright, 2010 The content of this PowerPoint presentation and accompanying handouts is copyrighted by Schoolhouse Educational Services, LLC and Milton J. Dehn. It is not to be reprinted, copied, or electronically disseminated without permission. To obtain permission, contact Milton J. Dehn at [email protected] Workshop Topics 1. 2. 3. 4. 5. 6. 7. Processing overview and MN’s criteria Processing and learning Organizing a processing assessment Cross-battery testing and analysis Interpreting results Determining eligibility Evidence-based interventions overview What is psychological processing? 1. Brain’s processes, operations, functions 2. Any time mental contents are operated on 3. When information is perceived, transformed, manipulated, stored, retrieved, expressed 4. Whenever we think, reason, problem-solve 5. Can’t learn and perform without processing 6. Doesn’t include knowledge base, skills, or what is stored in long-term memory 7. Topic limited academic learning processes Problems with Past Processing Assessment 1. 2. 3. 4. 5. 6. Not enough known about processing Not sure how to test processing Assessment too informal (grad. example) Over emphasis on low level processes Memory seldom assessed Some ineffective interventions Now Is the Time 1. Better understanding of psych processes and their relations with specific academics 2. Neuroscience and neuropsychological evidence on brain and learning 3. Neuropsychological influence on assessment 4. Research on processing, memory, academics 5. More assessment of memory 6. New research on interventions 7. Better assessment options Why Processing Assessment is a Good Idea 1. Process. deficits underlie (“cause”) SLD 2. Identifying a processing deficit differentiates SLD & slow learning 3. Understanding of the learner 4. Interventions for the processing deficits 5. Identifying processing deficits provides direction for academic interventions 6. Assessment benefits the learner Processing and RTI Compatibility 1. RTI purists are not current; too behavioral 2. Processing model consistent with problemsolving and early intervention 3. Evidence-based processing interventions 4. Why do “blind” general academic interventions 5. Different causes of disabilities 6. Processing and academics are compatible 7. Which academic interventions succeed? 1. Those that address processing problems “Resistance” to Intervention • Successful achievement depends on adequate psychological processes • Process deficits impede learning • A processing deficit can cause an academic intervention to fail---example – When severe; When not addressed – When intervention options limited • Processing testing can determine whether failure to respond to RTI is due to a processing deficit, adding more support for LD placement Processes and IQ 1. Processes underlie intelligence 2. Processing versus content (e.g. verbal) 3. Global IQ versus specific processes 4. Processing problems can pull down IQ IQ-Achievement Discrepancy Model and Processing • • • • Discrepancy model misses some SLD Deficient processing “pulls down” IQ Poor achievement “pulls down” IQ With processing, use a consistency approach, not a discrepancy approach – Low process + low academic skill = SLD – NOT high process + low academic skill Psychological Processing Model 1. Cognitive psychology theory of learning 2. Input – output memory model of processing Link 3. More parallel, complex, and integrated than can be represented 4. Neuroscience supports model: these are brain-based processes 5. Memory functioning is the core Psychological Processing, General Characteristics • • • • There’s a flow of information Mostly automated Parallel, complex, and integrated Executive functions exert control; affected by effortful, conscious control • Some input is internal from memory • Neuroscience supports cognitive model Link What the Brain Needs for Learning • Attention • Organized information to input • Both visual and auditory • Metacognition and strategies • Repetition • Sleep, glucose, oxygen Brain Lobes and Processes • Frontal Lobes: Attention, Exec. Functions, Fluid Reasoning, Working Memory • Temporal Lobes: Long-Term Memory, Auditory, Short-Term Memory • Parietal Lobes: Fine motor, Working Memory, Auditory, Phonological, • Occipital Lobes: Visual • The Hippocampus, absolutely essential for learning MN SLD Definition “Specific learning disability means a disorder in one or more of the basic psychological processes involved in understanding or in using language, spoken or written, that may manifest itself in the imperfect ability to listen, speak, read, write, spell, or do mathematical calculations, including conditions such as perceptual disabilities, brain injury, minimal brain dysfunction, dyslexia, and developmental aphasia” MN Processing Criteria “The child has a disorder in one or more of the basic psychological processes which includes an information processing condition that is manifested in a variety of settings by behaviors such as inadequate …. (next slide) MN Processing Criteria ….acquisition of information; organization; planning and sequencing; working memory, including verbal, visual, or spatial; visual and auditory processing; speed of processing; verbal and nonverbal expression; transfer of information; and motor control for written tasks.” (list is not exhaustive; essentially covers all major aspects of information processing) See Handout Minnesota SLD Criteria • A basic psychological processing deficit must be documented across multiple settings using a variety of sources under the IQ severe discrepancy option or the inadequate rate of progress (RTI) option • Two prior interventions required • Observation required Dehn’s Recommended Processes for SLD Assessment 1. Attention 2. Auditory Processing 3. Executive Functions 4. Fine motor Control 5. Fluid Reasoning 6. Language Processes 7. Long-Term Memory Recall 8. Phonological Processing 9. Processing Speed 10. Visual-Spatial Processing 11. Working Memory (See Handout) Attention • • • • • • • • • • Frontal lobes Cognitive levels of attention; not arousal Types: Selective, focused, divided, sustained Necessary for acquisition The problem is attentional control Lack of inhibition Under executive control Interest of material is a factor Attention weakness often part of SLD Learning: A necessary prerequisite for all learning and memory Auditory Processing • Ability to perceive, analyze, synthesize, and discriminate auditory stimuli, e.g. speech • Phonological processing may be a subtype • Learning: Perceiving and comprehending instruction; being able to understand words with background noise Executive Functions • More to do with performance than skills • Involves management of cognitive functions and psychological processes and also skills • Complex; difficult to assess as a whole • Includes planning (easiest to assess) • Organization depends on executive functions • Effectiveness depends on self-monitoring, selfregulation, and metacognition • Related to use of learning strategies • Learning: Knowing how much you need to study material Executive Control Processes • Self awareness (metacognition) – Awareness of strengths and weaknesses – Awareness of the task requirements – Awareness of which strategies work • Self regulation (metacognition) – Monitoring; evaluating; planning – Detecting errors, problems – Deciding which strategy to use – Fixing the problem Fine Motor Control • Is it perceptual or motor? • Seems related to attention • Does it interfere with learning or just written performance? • Interventions for it do not improve learning but may improve performance • Learning: Be able to copy letters accurately Fluid Reasoning • A higher level process (close to IQ) • Inductive and deductive reasoning • Learning: problem-solving, math reasoning, comprehension Language Processes • Not the content (vocabulary) or the written output but the oral processes • May acquire knowledge but have difficulty expressing it • Retrieval may be part of the problem • Learning: Oral expression of thoughts, ideas Long-Term Memory • Not specifically part of MN criteria but important part of info. processing & learning • Some students learn well initially but can’t retain; others learn poorly but retain well • Difficult to measure – Not known what has been learned – Intervals in tests not long enough • Deficits due to hippocampal damage Rapid Automatic Naming (RAN) • Naming facility for commonly known objects • Highly predictive of early reading achievement • May be associated with slow long-term retrieval speed or slow processing speed Phonological Processing • Processing of phonemes, e.g. blending • Phonemic awareness is part of • Related to language and literacy development • Related to phonological short-term memory • Learning: Recognizing rhyming words, segmenting words into phonemes Processing Speed • How quickly information flows through the processing system; a matter of efficiency • Too slow: info. lost, process not completed • Speed increases significantly until adolescence – Adolescents 2x faster than 8 year olds, 3x than 4 years • Related to retrieval speed, RAN • Learning: Rapidly computing and retrieving the answer to simple math calculation, reading fluency • Case example Visual-Spatial Processing • Ability to perceive, analyze, synthesize, manipulate & think with visual patterns • A strength in most LD cases • Visual and spatial can be differentiated • Not that correlated with academic learning in general • Orthographic is stronger • Learning: Distinguishing between shapes Orthographic Processing • A subtype of visual processing • Has to do with processing of letters, symbols, and words • Should be assessed – For example, PALS Working Memory • Includes short-term memory • Processing while retaining information • Four units of information at a time • Very important for learning & performance • About half of LD students have a deficit • Involves executive functions • Needed for language development (ESL?) See monkey video Working Memory • • • • • Includes auditory and visual processing Includes sequencing Related with processing speed Is the interface between input and storage Some information in working memory is retrieved from long term memory • The most common processing weakness • Automaticity increases capacity Types of Working Memory • Verbal Link • Visuospatial • Executive • Episodic Activated long-term memory items Learning: Being able to take notes while listening Long-Term Memory Systems • Episodic: memory for events, Link autobiographical, context, details – Learning is initially episodic, then semantic • Semantic: facts, knowledge, vocabulary – Abstracted from episodes • When tested, divided into visual and verbal (both episodic during standardized memory testing) • Implicit: unconscious, includes procedural Long-Term Memory Processes • • • • • Encoding Consolidation Storage Retrieval---most directly measureable Learning: Knows a fact but can’t recall when needed without prompting Forgetting • Fast at first, then slows down • 50% forgotten within 24 hours • Some retain well within 30 minutes or 1st day but then have very poor retention afterwards • Is it lost, or not retrievable at moment? • Interference: proactive and retroactive • Recognition versus free recall Consolidation 1. 2. 3. 4. 5. Often ignored Evidence from TBI and amnesia Takes time Unconscious mostly Movement from medial temporal lobe/hippocampus to cortical areas 6. Episodic to semantic 7. Much of it occurs during sleep Strategy or Processing Deficit 1. Effective strategies enhance processing 2. LD students are deficient in use of strategies 3. Difficult to assess and differentiate 4. Learning and using strategies requires more processing until they become automated (WM Catch 22) 5. LD due more to processing deficits than lack of strategies Development of Processes • Early gradual development followed by long-term plateau: auditory processing, visual processing, long-term memory • Rapid early development followed by gradual decline after age 25: processing speed, short-term memory • Basic functions develop earlier; higher level, such as executive (frontal lobes), later Processing the Workshop Info. • Explain to a parent what psychological processing is, in general • Then explain one of the 10 processes to a parent Processes and Academic Learning • See Chart • Relationships established through research • During assessment look at processes related to area of academic concern • Discuss these evidence-based links in your reports to justify placement Highest Influence of Processes on Academics by Grade • Early elementary: Phonological, visual, auditory, rapid automatic naming, STM, sequencing, fine motor control • Late elementary and beyond: Executive, working memory, long-term memory, planning • All grades: Processing speed, attention, verbal expression SLD by Processing Subtypes • Visual-Spatial Deficits: Math calculation and math problem solving • Processing Speed Deficits: Reading comprehension, written expression • Working Memory Deficit: Math calculation, Written expression • Attention: Written expression • Source: Hain, Hale, Kendorski WISC-IV Profile for Reading Disabled Sample Perceptual Reasoning Processing Speed Verbal Comprehension Working Memory 94.4 92.5 91.9 87.0 WISC-IV Profile for Math Disabled Sample • • • • Verbal Comprehension Working Memory Processing Speed Perceptual Reasoning 93.2 92.9 90.6 87.7 What is a Processing Disorder? Disorder---a group of symptoms involving abnormal behaviors or a disruption in physiological functioning Not many official processing disorders E.g. CAPD, aphasia, amnesia, dyspraxia A disorder involves deficits Deficit---a lack or shortage of something • Weakness---lacking normal ability Evidence of a Processing Deficit • It’s not specific to one environment • Both an intra-individual weakness and a normative weakness (deficit per Dehn) • A normative weakness is a score below 90 • Intra-individual: score is 12 or more points lower than IQ or mean of assessed processes • A low score in a process is not necessarily a deficit indicative of LD, unless it’s also an intraindividual weakness – E.g., very low IQ have inherent processing problems Deficits Indicate Disabilities 1. 2. 3. 4. Deficits (double weakness) are rare They have an underlying neurological basis They impair learning Students with below average IQ are less likely to have 5. Deficits more predictive of need for LD than an ipsative weakness alone. 6. Below average processing (normative weakness) is still a processing weakness that impacts learning and benefits from intervention When Does a Processing Deficit Support an SLD Diagnosis? • When it’s a double weakness deficit • When the process is one that is related to the academic skill deficiency • When a process is significantly higher than related academics, it’s not SLD but some other cause of poor skills (example) • When both the academic skill and process are similarly low (consistency approach) Consistency Approach • With processing, use a consistency approach, not a discrepancy approach – Low process + low academic skill = SLD – NOT high process + low academic skill Normative Weaknesses Only • Normative weaknesses that are not also ipsative deficits may still be indicative of processing deficits that are impeding academic learning and causing a learning disability, as long as there is a pattern of strengths and weaknesses • Normative weaknesses that are not ipsative weaknesses should targeted for intervention Discussion: Processing the Workshop Information 1. How would you explain to a parent what psychological processes have to do with learning 1. For example, what is working memory and why is it important for learning 2. Pick 2 or 3 processes that you consider the most important for academic learning, in general, and explain why Processing Assessment Challenges 1. 2. 3. 4. 5. 6. 7. Connecting to academic concerns Interrelated processes Informal methods lack validity Not found in one convenient battery Doing it efficiently Having enough expertise Connecting with interventions Records Review 1. Any medical or health conditions 2. Developmental delays 3. Look for indications of same behaviors that could be observed 1. Difficulty memorizing arithmetic facts could be a working memory deficit 2. Difficulty blending could be phonological proc. 3. Low fluency could be processing speed problem Interviews 1. From middle school up, do interviews 1. E.g., “Do you often forget what was just said?” 2. Teachers: Ask about observable behaviors, then ask their hypothess 3. With parents, use home environment examples 1. “Does your child have difficulty rhyming?” See examples Observations 1. Some processes more observable than others, e.g. memory is more observable 2. There’s no one-to-one correspondence between a behavior and a process 3. Each behavior depends on multiple processes; try to narrow it down 1. E.g., poor organization could be due several (organization is not a specific brain process) See examples Observations of Working Memory • • • • • • Abandonment of activity Reserved in group activities Short attention spans Poor monitoring of work quality Loses place in complicated tasks Examples of evidence-based behaviors associated with WM deficiencies Link • See also LTM observations Observations During Testing • Corroborates scores • Examples: – Speed – Retrieval – Needing directions repeated – Attention – Fine motor Cross-Battery Method 1. Administer specific subtests/factors only 2. Mix tests/batteries to cover all the processes (at least those of concern) 1. Limit to 2 or 3 batteries 2. Should be normed about the same time 3. Avoid redundancies 4. Ideally, 2 subtests per process 5. Analyze results together by computing a cross-battery mean Cross-Battery Criticisms 1. Only some subtests are administered 2. Tests may have been normed at different times 3. Different subtests with same names measure different tasks & vice versa 4. There are no norms for the cross-battery mean 5. It relies mainly on subtest analysis Dehn’s Approach to Cross Battery versus Flanagan’s 1. Not limited to CHC factors 1. CHC factors are too broad; some processes need to be differentiated, e.g., phonological from auditory and visuospatial short-term memory from visuospatial 2. Not Quantitative reasoning, and crystallized 2. Analyze scores at the factor (twosubtest) level whenever possible 3. Use a hand computation analysis sheet 4. A more neuropsychological approach Hypothesis Testing Approach 1. Collect preliminary data 1. Records review, observation, interview 2. Identify referral concerns 3. Generate processing hypotheses (suspected processing problems) 4. Select batteries/tests/subtests 1. Cover all hypotheses 2. Avoid redundancies Selective Testing 1. Not necessary to administer entire battery 2. Focus most on hypothesized deficits 3. Know the factors/subtests that measure processes 4. Apply a cross-battery approach 5. See selective testing table for cog. & ach. Completing the Processing Assessment Planner 1. Enter academic learning referral concerns 2. Look at Table 1: Processes and learning 3. Enter hypothesized, suspected processes 4. Select assessment method 5. Look at selective testing tables 6. Select battery 7. Select specific subtest or factor Planner Case Study: Hannah • Concerns: – Struggles in math, reading comprehension, and written expression – Does poorly on classroom exams – Difficulty learning new concepts • Background: adopted; slow speech dev.; retained in K; learning problems since K • Strengths: Oral expression, vocabulary • Completed Planner Example Cross-Battery Analysis of Scores 1. See Processing Analysis Worksheet 2. Get factor scores from test manual when possible 3. Exclude non-processing factors and subtests 4. Compute clinical factor scores by averaging 1. First convert scaled scores to 100/15 metric 5. 6. 7. 8. Compute processing mean or use IQ Calculate difference scores Determine weaknesses and deficits Do pairwise comparisons Examples of Logical Processing Pairs • • • • • • • • Visual vs auditory STM vs LTR Phonological vs STM Attention vs Executive Auditory vs verbal expression Verbal expression vs fine motor control WM vs STM WM vs LTR Case Study Hannah’s Results • • • • Full Scale IQ = 89 Basic Reading Skills = 95 Reading Comprehension = 81 Mathematics Reasoning = 82 Case Study: Hannah 1. 2. 3. 4. 5. (finish example) Identify normative weaknesses/strengths Identify ipsative weaknesses/strengths Identify deficits and assets Which processing hypotheses are supported and which are not? 1. Look back at planning sheet Case Study: Hannah 1. Use Hannah’s data 2. What are her processing deficits? 3. Do they link with her learning and achievement problems? 4. Does her processing profile explain her academic learning problems? 5. Does she qualify for the processing component of LD? 6. Does she qualify for LD overall? Assessing Processing with Wechslers • Processes assessed by Wechslers – Processing speed – Visual processing---compute clinical factor – Working memory (not WPPSI-III) • Need to supplement with other scales • Processing interpretation will be based on processing factors only; exclude Verbal Comprehension Index Wechsler GAI Score • General Ability Index • A traditional measure of IQ based on 6 core subtests • Recommended for IQ-Ach discrepancy when processing speed or working memory are pulling down FSIQ • However, do not use for determining processing strengths and weaknesses WISC-IV Integrated 1. 2. 3. 4. 16 additional process subtests Requires regular WISC-IV administration Based on Kaplan neuropsych approach To parse out the cognitive processes that underlie subtest performance 5. Select subtests based on hypotheses 6. Emphasizes subtest and intrasubtest interpretation WISC-IV Integrated • Best for in-depth working memory testing • Processes measured by process subtests: – Visual processing – Planning/executive processing – Long-term retrieval – Short-term memory (registration) – Working memory (manipulation) • Visual • Auditory WISC-IV Integrated Interpretation 1. Compute clinical factor scores 1. 2. 3. 4. 5. 6. Short-Term Memory Visual Short-Term Memory Auditory Short-Term Memory Working Memory Visual Working Memory Auditory Working Memory Link 2. Use processing analysis worksheet WJ III Cog. • Ideal: measures several processes Link • Works well as CB supplement to other tests • Has at least 2 subtests for most processes • Processing analysis worksheet needed less • Co-normed with WJ III achievement WJ III COG Interpretation Options 1. Use Compuscore Intra-Cog. Discrepancy 1. Use 1 SD as ipsative significance indicator 2. Use Predicted Achievement/Achievement Discrepancy 1. Predicted achievement is based on the processes that are most related to the skill 2. The process scores are weighted 3. Look for consistency or even lower achievement 3. Use processing analysis worksheet if other tests 1. Omit Comprehension-Knowledge Assessing Memory • Should be a priority with LD referrals • Intellectual/cognitive scales not enough • Many different types of long-term memory – Semantic is primary for academic learning WRAML-2 • • • • • Authors: Adams and Sheslow Published by Wide Range, 2003 Ages 5-90 Has a new Working Memory Composite In 1990, WRAML was 1st well normed test battery for memory assessment in children • Normed on sample of 1200 • Assess memory subcomponents WRAML-2 Composites • • • • • • • • General Memory Verbal Memory Visual Memory Attention/Concentration Working Memory Verbal Recognition Visual Recognition General Recognition NEPSY-II • Neuropsychological test (SPs with training) • 2007, Pearson • Authors: Korkman, Kirk, and Kemp • Ages 3-16:11 • 34 subtests across 6 domains • Based on Lurian tradition of looking for subcomponents that may underlie impairment of system NEPSY-II Processes in 6 Domains 1. 2. 3. 4. 5. 6. Attention and executive functioning Language Memory and learning Sensorimotor Social perception Visuospatial processing Testing Processes with Achievement Tests • WJ III Achievement • KTEA-II • WIAT-III Link The BRIEF • Behavior Rating Inventory of Executive Function • Parent, Teacher, and Student Forms • Ages 5-18; 86 items • Normed in the state of Maryland only • Has a global executive composite score • Two Indexes: Behavioral Regulation and Metacognition The Brief’s Scales • • • • • • • • Inhibit Shift Emotional Control Initiate Working Memory (weak scale) Plan/Organize Organization of Materials Monitor Children’s Psychological Processes Scale (Dehn) • Available fall of 2011 • Scaled score for each of the 11 processes discussed today • Teacher rating scale, about 130 items • Entirely online, except manual • Will provide documentation of patterns of strengths and weaknesses • More MN subjects needed Other Tests • Includes rating scales, self-report scales, specialized process tests, etc. • Learning Efficiency Tests (LET-II): Memory • See List General Interpretative Guidelines for Processing Assessment 1. A deviation from a reference point 1. IQ 2. Cross-Battery mean 3. Related pairing (not best evidence of deficit) 2. Normative versus ipsative 1. With both, impairments more likely 3. Qualitative information 4. Corroborating information Interpretative Guidelines 5. Go beyond test labels; reclassify 6. Consider disability profiles 7. Link to achievement deficit 8. Consider influence of related processes 9. Consider strategy deficiencies 10. Okay to focus on subtests/subprocesses 11. Consider interventions 12. Consider other causes The Influence of Other Factors on Process Test Performance • Poor processing performance maybe due to lack of strategy development • May be due to lack of exposure and opportunities, e.g. phonemic awareness lower in lower SES • Temporary medical conditions, e.g. concussion • Brain process may be deteriorating Report Writing Recommendations 1. Do not report on a test-by-test basis; rather process by process 1. Integrate data from other sources 2. Explain how the processing results were analyzed, including processing mean 3. Distinguish between normative and ipsative weaknesses 4. Define each process See Example Case Study: “Jacob” • • • • • • • • Age 13; 7th grade Foster care; special ed placement 3 months premature; failure to thrive Early elementary IQ of 70; recent IQ of 95 ADHD diagnosis; poor organization Social skills problems Difficulty completing homework Moderately high test anxiety Case Study Continued • • • • • • • Likes to read Struggles with Math and Written Lang. Reading Composite – 106 Math Composite – 88 Wr. Lang. Composite – 73 Oral Lang Composite - 87 List your processing hypotheses to account for low achievement and background factors Case Study Continued • • • • Finish Jacob’s Proc. Analysis Worksheet What are his processing deficits? What are his processing strengths? Do his processing deficits link to his low achievement areas? • Does his processing profile explain his learning difficulties? • Does he qualify for SLD in processing? • What else should be tested? Determining Psych. Processing Deficits for MN’s SLD Criteria Use Jacob’s data See MN worksheet; go through steps What are his processing deficits? Do they link with his learning and achievement problems? 5. Complete other steps. 1. 2. 3. 4. Case Study Continued • Discuss how you will explain to Jacob’s parents and teachers how his processing deficits are impacting his academic learning • What strengths does Jacob have that might be utilized during interventions? • Which processing areas are most in need of intervention? Conclusions About Processing Assessment • It can be done well but there are challenges • Professional judgment is necessary • Selective, cross-battery testing necessary • Test the processes associated with the academic deficiency • Use the consistency approach • Explain the link Interventions • • • • • • • Should be brain-based; change the brain Strengthen weakness when possible Teach strategies Some are content specific Some are similar to study skills Metacognitive aspect is important Can be part of effective instru Group Discussion 1. Which assessment procedures do you plan to use? 2. What recommendations do you plan to take back to your district for processing assessment? 3. How do you view the processing assessment-intervention link?