Report

Michaela Saisana Second Conference on Measuring Human Progress New York, 4-5 March 2013 “Reflections on the Human Development Index” (paper by J. Foster) Additional Considerations Michaela Saisana [email protected] European Commission Joint Research Centre Econometrics and Applied Statistics Unit 1 Second Conference on Measuring Human Progress New York, 4-5 March 2013 Michaela Saisana 250,000 21,300 ~5-fold increase since18,600 2000 20,000 20,000 21,300 200,000 18,600 17,800 16,200 15,000 15,000 12,600 115,000 10,000 7,730 7,730 5,000 1985 “Yet the dimensions of the HDI do not easily meld into one. And without a systematic method […prices…] the index could prove difficult to explain and defend” (J. Foster, 2013) 100,000 77,100 4,460 5,000 0 150,000 12,600 10,000 50,000 4,460 0 1985 180 1990 180 1990 1995 1995 2000 2000 2005 Year 2010 2005 Scholar Google hits on "Gross domestic product" Achievements The challenge The measure Popularity Scholar Google hits on "Human development index" • • • • Scholar Google hits on "Human development index" Introduction 25,00025,000 2010 0 2015 2015 Year It is exactly the “unobserved” nature of composite indicators that is their main limitation and their raison d'être. 2 Michaela Saisana Second Conference on Measuring Human Progress New York, 4-5 March 2013 Main points • • • • Calibration Goalposts Gaterories Cobb-Douglas HDI 3 Michaela Saisana Second Conference on Measuring Human Progress New York, 4-5 March 2013 Main points • • • • • Calibration Goalposts Categories Aggregation New HDI “Frequent recalibration gives the strong suggestion that HDI values are contingent and temporary and depend importantly on arbitrary constructs” Foster’s suggestion: 1) ~ 10 year recalibration (as for poverty) 2) Crossover between calibration periods: process outlined explicitly and transparently Source: Global Innovation Index 4 Michaela Saisana Second Conference on Measuring Human Progress New York, 4-5 March 2013 Source: Global Innovation Index 5 Second Conference on Measuring Human Progress New York, 4-5 March 2013 Michaela Saisana Main points • • • • “The HDI is typically cast and interpreted as a multidimensional measure of size and hence is seen to be an absolute measure. […] Yet in actual implementation, this is not necessarily the way the HDI behaves.” Calibration Goalposts Categories Cobb-Douglas HDI Source: Wikipedia Life expectancy at birth Bounds in the HDI After 2010: 20y – observed (83.2 y, JN) Before : 25y – 85y 6 Second Conference on Measuring Human Progress New York, 4-5 March 2013 Michaela Saisana Main points • • • • Calibration Goalposts Categories Cobb-Douglas HDI Life expectancy at birth Suggestion: Fixed bounds 30y (Early 20th Century) – 87 years Life expectancy at birth (years) Minimum and Maximum across 194 countries 90 85 80 75 70 65 60 55 50 45 40 35 30 1970 76.7 82.3 79.0 83.4 85.6 47.8 44.0 39.8 34.6 1980 32.8 1990 2000 2010 2020 Similarly for the other indicators 7 Second Conference on Measuring Human Progress New York, 4-5 March 2013 Michaela Saisana Categories of Human Development Main points • • • • Calibration Goalposts Categories Cobb-Douglas HDI Relative (since 2010) versus Absolute (before 2010) + progress against other countries, rather than arbitrary numerical cutoffs whose meaning may vary with each new calibration. - fuzzy incentives, less practical value for the country - many factors enter into the determination of progress (e.g. different calibrations, performance of other countries, policies of the country, or inclusion of new countries). - a country can not set a meaningful numerical target to achieve over time. Foster’s suggestion: 1) A staggered recalibration schedule & 2) Fixed numerical cutoffs for the four HD categories (e.g. WB grouping by income) 8 Second Conference on Measuring Human Progress New York, 4-5 March 2013 Michaela Saisana Further recommendation: Main points • • • • Calibration Goalposts Categories Cobb-Douglas HDI HDI To present the fixed cutoffs for the HDI with respect to the raw data (assuming an even performance) Life Mean Expected GNI per expectancy at years of years of capita birth (years) schooling schooling (PPP$) 0.6 58.2 7.9 10.8 6,487 0.8 … … 9 Second Conference on Measuring Human Progress New York, 4-5 March 2013 Michaela Saisana Main points “[…] attempt to view the HDI more as a social evaluation function that aggregates across dimensional variables directly” •Calibration •Goalposts •Categories • Cobb-Douglas HDI W L E 1/ 3 1/ 3 Y 1/ 3 L= life expectancy - 20 years E =1/2 (mean years of schooling + expected years of schooling) Y= ln (GNI per capita) – ln (100) H W /W * W*= target social evaluation level 10 Second Conference on Measuring Human Progress New York, 4-5 March 2013 Michaela Saisana More on the geometric mean in the case of the HDI… HDI HDI 2011 Liberia’s (arithmetic) (geometric) improvement Life Edu GNI stdev Mali .496 .270 .346 .115 .371 (176) .359 (175) Liberia .580 .439 .140 .225 .386 (175) .329 (182) Option A .680 .439 .140 .419 .347 5.5% Option B .580 .439 .240 .419 .394 19.8% Advantages of the geometric mean versus the arithmetic mean for the HDI 1) implies only partial compensability, i.e. poor performance in one HD dimension cannot be fully compensated by good performance in another, 2) rewards balance by penalizing uneven performance between dimensions, 3) encourages improvements in the weak dimensions, i.e. the lower the performance in a particular HD dimension, the more urgent it becomes to improve in that dimension. 11 Second Conference on Measuring Human Progress New York, 4-5 March 2013 Michaela Saisana More on the “quality” of the HDI… (Implicit Weights) We suggest to use as a measure of importance of a variable in an index what is known as: ‐ Pearson’s correlation ratio HDI Life Expectancy ‐ First order effect ‐ Top marginal variance - Main effect … (~ ) = () Using these points we can compute a statistics that tells us: How much (on average) would the variance of the HDI scores be reduced if one could fix “Life expectancy”? Source: Paruolo, Saisana, Saltelli, 2013, J.Royal Stat. Society A 12 Second Conference on Measuring Human Progress New York, 4-5 March 2013 Michaela Saisana More on the “quality” of the HDI… (Implicit Weights) (~ ) = () HDI HDI 2011 Life Expectancy Nominal Implicit Weights (wi) Weights (Si) Life expectancy .333 .83 [.81 .85] Education .333 .88 [.83 .87] GNI .333 .90 [.88 .91] We could reduce the variation of the HDI scores by 83% by fixing ‘Life expectancy”. Quality check: The HDI is balanced in its three underlying dimensions (Si values are very similar) 13 Second Conference on Measuring Human Progress New York, 4-5 March 2013 Michaela Saisana More on the “quality” of the HDI… (Marginal weights) Marginal Weights= Recommendation: To plot life expectancy instead to evidence that countries with low life expectancy are more encouraged to improve 14 Michaela Saisana Second Conference on Measuring Human Progress New York, 4-5 March 2013 Some recent criticism… Tradeoffs = marginal rate of substitution, i.e. how much of one dimension must be given up for an extra unit of another, keeping the index constant. Previous HDI The new HDI has devalued longevity, especially in poor countries. Source: M. Ravallion (2012) Troubling tradeoffs in the HDI, J. Dev. Economics, 99:201-209 15 Second Conference on Measuring Human Progress New York, 4-5 March 2013 Michaela Saisana Final considerations Simply take the log of GNI just once (now logged twice) Take the arithmetic average the two education indicators (now geometric) Use two indicators per dimension (now only in case of education) Use the generalized mean of the three dimensions (a compromise solution between arithmetic-geometric averaging) 1/ HDI ( L E Y ) 0.5 0, geometricmean 1, arithmeticmean 16 Michaela Saisana Second Conference on Measuring Human Progress New York, 4-5 March 2013 Assess any new calibration formula in terms of: Implicit weights (reduction in the HDI variance by fixing one dimension at a time) Marginal weights (impact on HDI of 1% increase in one of the dimensions) Marginal rate of substitution (how much of one component must be given up for an extra unit of another, keeping the index constant) More reading at: http://composite-indicators.jrc.ec.europa.eu (first Google hit on “composite indicators” over the last 10 years!) 17 Michaela Saisana Second Conference on Measuring Human Progress New York, 4-5 March 2013 References and Related Reading 1. Paruolo P., Saisana M., Saltelli A., 2013, Ratings and Rankings: voodoo or science?. J Royal Statistical Society A 176(2). 2. Saisana M., Saltelli A., 2012, JRC audit on the 2012 WJP Rule of Law Index, In Agrast, M., Botero, J., Martinez, J., Ponce, A., & Pratt, C. WJP Rule of Law Index® 2012. Washington, D.C.: The World Justice Project. 3. Saisana M., Philippas D., 2012, Sustainable Society Index (SSI): Taking societies’ pulse along social, environmental and economic issues, EUR 25578, Joint Research Centre, Publications Office of the European Union, Italy. 4. Saisana M., D’Hombres B., Saltelli A., 2011, Rickety Numbers: Volatility of university rankings and policy implications. Research Policy 40, 165–177. 5. Saisana M., Saltelli A., Tarantola S., 2005, Uncertainty and sensitivity analysis techniques as tools for the analysis and validation of composite indicators. J Royal Statistical Society A 168(2), 307-323. 6. OECD/JRC, 2008, Handbook on Constructing Composite Indicators. Methodology and user Guide, OECD Publishing, ISBN 978-92-64-04345-9. 18