John Breidert & James Hellrung GMA Haiku General Mental Tests One Concept With Many Parts Test the “g factor” Overview Introduction into GMA and Supporting Theories GMA on the job and in tests Introduction into GMA and Supporting Theories Introduction to General Mental Ability Spearman’s Two-factor Theory of Intelligence Vernon’s Hierarchical Theory of Intelligence Carroll’s Three-Stratum Factor Analytic Theory of Cognitive Abilities Introduction to General Mental Ability General Mental Ability is the sum of many parts of intelligence Building Example Spearman’s Two-factor Theory of Intelligence Spearman (1863- 1945) Proposed the theory in 1927 General Factor (g) in addition to one or more specific factors accounted for people’s performance on intelligence tests Spearman saw the g factor as a mental energy that was expended on different mental tasks Spearman saw the g factor as more of the inventive aspect of mental ability Spearman’s Two-factor Theory of Intelligence Vernon’s Hierarchical Theory of Intelligence Philip E. Vernon (1950) Hierarchical theory of intelligence g at highest level, must consider it in order to understand or measure intelligence At next level are the major group factors: Verbal-Educational Spatial-Mechanical Vernon’s Hierarchical Theory of Intelligence Next level is minor group factors: Lowest level contains specialized factors that are unique to specific tests Therefore, the lower on the hierarchy, the most specific the behavior Vernon’s theory is supported by numerous studies finding positive intercorrelations among different tests Vernon’s Hierarchical Theory of Intelligence Carroll’s Three-Stratum Factor Analytic Theory of Cognitive Abilities John B. Carroll (1993) proposed a three stratum factor analytic theory of cognitive abilities There are many distinct differences in cognitive ability Carroll’s Three-Stratum Factor Analytic Theory of Cognitive Abilities Narrow (stratum 1) 65 narrow abilities Level factors Speed factors Rate factors Broad (stratum 2) 8 broad factors General (stratum 3) Consists of only g Carroll’s Three-Stratum Factor Analytic Theory of Cognitive Abilities GMA on the Job and in Tests GMA and Occupational Level GMA and Job Performance GMA and Training Performance Other Traits and Variables Affecting Job Performance Group Differences for GMA General Reactions to GMA New Methods of Testing GMA GMA and Occupational Level Cross-sectional & Longitudinal Studies relate GMA to occupational level Cross-sectional Studies – mean GMA increases with occupational level Longitudinal Studies – GMA measured earlier in life predicts later occupational level. Job mobility predicted by congruence between peoples’ GMA scores and complexity of their job Childhood GMA predicts adult occupation level (r = .51) and income (r = .53) GMA predicts attained job level, but not which occupation within that level GMA and Job Performance GMA used for predicting Job Performance since WWI Situational Specificity theory says GMA predicts job performance sporadically Validity coefficients varied across studies Some statistically significant, some not Truth – variability in validity findings due to statistical and measurement artifacts. After correcting for effects of artifacts, there was little variability in validity, and GMA measures were predictive of job performance for all jobs. GMA and Job Performance Validity ranges .58 for most complex jobs .23 for least complex jobs Validity for job performance shown in many studies: Clerical jobs - .52 (Pearlman, Schmidt, & Hunter, 1980) Law Enforcement - .38 ( Hirsh, Northrup, & Schmidt (1986) Military “Core Technical Proficiency” - .63 (McHenry et al., 1990) Military “General Soldiering Proficiency” - .65 (McHenry et al., 1990) Air Force jobs – mean of .45 (Ree, Earles & Teachout, 1994) GMA and Training Performance Validity for training performance also: Meta-analysis of 90 studies - > .50 (Hunter & Hunter, 1984) Military meta-analysis of over 82,000 trainees - > .63 (Hunter, 1986) Air Force meta-analysis of over 77,958 trainees - > .60 (Ree & Earles, 1991) Clerical workers – mean of .71 (Pearlman et al., 1980) Law enforcement – mean value of .76 (Hirsh et al., 1986) Across meta analyses, unweighted average validity: .55 for job performance .63 for training performance Other Traits and Variables Affecting Job Performance Specific Aptitudes Cognitive abilities narrower than GMA Regression equations optimize prediction of job and training performance Disconfirmed - Causal analysis modeling failed to fit the data, but a hierarchical model fit well (Hunter, 1983b) Use of specific aptitudes may reduce group differences Job Experience More job experience, not GMA should predict job performance As experience increases, predictive validity of GMA does not decrease. Actually goes from .36 for 0-6 years to .44 for 6-12 years, up to .59 for more than 12 years. If anything, as experience increases, so does validity of GMA Other Traits and Variables Affecting Job Performance Personality Traits Predicted occupational level and income (Judge et al., 1999) Conscientiousness : .49 and .41 Openness to experience: .32 and .26 Neuroticism: -.26 and -.34 GMA: .51 and .53 When placed career success in regression equation: Multiple r = .63 Neuroticism: β = -.05 Openness: β = -.03 Conscientiousness: β = .27 GMA: β = .43 When only Conscientiousness and GMA in equation: Multiple r = .63 Conscientiousness is only personality trait contributing to career success Group Differences for GMA Specific aptitudes have smaller group differences May be due to unreliability and range restriction However GMA tests are more reliable than other predictors GMA produces racial differences 3-5 times more difference than produced by interviews, biodata, and work sample tests. Could be due to measurement error in the above Four-fifths rule Infers adverse impact when selection rate for the low-scoring group < 4/5 the selection rate for the high-scoring group Because job complexity increases the likelihood of adverse impact, Viswesvaran & Ones (2002) suggest a sliding adverse impact rule (e.g., .50 for complex jobs and .80 for simple ones) GMA is a best predictor of job performance, but also predictor with most adverse impact General Reactions to GMA Even students who are not aware of group differences have negative reactions In homogenous societies, there are also negative reactions to GMA (Viswesvaran & Ones, 2002) Past abuses of testing for GMA still haunt us Research on applicant reactions to GMA needs to continue, but still at its infancy Laypeople maybe convince that cognitive ability is not important in determining intelligent behavior. Although research suggests validity of GMA increases with increased job complexity, organizations are less likely to use GMA for high-level jobs than lower-level jobs. (Face validity?) New Methods of Testing GMA Low cost of paper & pencil Killed demand for other testing media To reduce group differences One strategy is to change test medium Computerized and video-based assessments Must be careful not to change construct being measured Format changes may induce differences in GMA and individual differences in responding to the new medium New Methods of Testing GMA In the future: May see tools based on physiological, biological, and genetic markers identified for GMA Whether they are accepted depends on societal views on privacy rights versus organizational needs Bottom line – If the use of different mediums reduces adverse impact without reducing validity for a criterion, then the new method is preferred Questions?