Report

OLS & Logistic Regression Analysis – A Recap Cristina Penaloza & Eoin Maloney Health Economics Unit 1 Outline • What is regression analysis? • Relevance of regression analysis • Regression modelling process – OLS regression – Logistic regression • Exercise 2 What is Regression Analysis? “Regression analysis is concerned with the study of the dependence of one variable, the dependent variable, on one or more other variables, the explanatory variables, … with a view to estimating and/or predicting the (population) mean or average value of the dependent variable in terms of known or fixed (in repeated sampling) values of the explanatory variables.” Gujarati (1995: 16) 3 Terminology Dependent variable, explained variable, outcome variable, outcome, response variable, regressand, output variable, predicted value, predictand, endogenous Explanatory variable, Independent variable, predictor variable, predictor, regressor, stimulus/control variable, exogenous Disturbance (random error) term, residual, residual error 4 Causation / correlation • Regression vs causation – “A statistical relationship, however strong and however suggestive, can never establish causal connection: our ideas of causation must come from outside statistics” Gujarati (1995: 20) • Regression vs correlation – Correlation analysis: seeks to measure the strength of linear association between two variables – Regression analysis: seeks to estimate or predict the average value of one variable on the basis of fixed values of other variables 5 Why study regression? • Adjusting for baseline characteristics in Economic Evaluation (Nathwani et al. 2004; Manca et al. 2005; Hoch et al 2002) • Predicting/mapping utility-based outcome measures for use in Economic Evaluation (Gray et al. 2006; Kaambwa et al.2011; Sengupta et al 2004) • Predicting costs for use in Economic Evaluation (Smith et al. 2007; Bonizzato et al. 2000; Baumeister et al. 2009) • Constructing CEACs (Hoch et al. 2006) • Regression imputation for missing data (Billingham et al. 2002; Engels & Diehr, 2003; Blazer et al. 1995) • Explaining factors which cause variation in outcome and cost data (Barber &Thomspon, 2004; Kaambwa et al. 2008; Raine et al, 2010) 6 The regression modelling process 1. 2. 3. 4. 5. 6. 7. Statement of hypothesis (theory) Specification of the model Obtaining the data Estimation of the regression model Diagnostic analysis Hypothesis testing Prediction/forecasting 7 1. Statement of hypothesis Example: High Blood Pressure and older people “Amongst those over the age of 65, the incidence of high diastolic blood pressure (dipb) increases with age. Therefore, dipb is, in part, explained by age.” 8 2. Specification of the model In Functional form: Mean Diastolic High Blood Pressure, DIBP, is some function of age, A: DIBP = f (A) (1) 9 2. Specification of the model (cntd) In Mathematical (linear) form: Y = 1 + 2X where (2) Y = Mean DIBP and X = age 1 & 2 = parameters 10 Linear relationship E(Y|X) .. .. x1 .. .. .. . .. . x3 . . . . x4 . . . . . x6 X 11 2. Specification of the model (cntd) Econometric (Regression) model Y = 1 + 2X + u Where (3) Y = Mean DIBP - the dependent variable X = Age - explanatory variable u = Disturbance (random error) term 1 & 2 = parameters 12 The error term (u) • Omitted explanatory variables • Measurement error • Wrong functional form • Unavailability of data • Inherent randomness etc…. 13 3 & 4. Data / estimation of parameters • Obtaining the data – observed values of Y and X • Estimation of the parameters – Y and X are the variables (“known”) – 1, 2 and u are the parameters (“unknown”) 14 5. Diagnostic analysis • Is the model correctly specified? • Have all assumptions been met? • Are there any unusual observations or outliers that may unduly influence results? More of this later this morning… 15 6. Hypothesis Testing • Is estimate statistically close to a postulated value? Or are estimates in accord with expectations from theory? • Only after model has been shown to be adequate 16 7. Forecasting or Prediction • If hypothesis or theory being tested is confirmed, then future values of the dependent variable can be predicted or forecast • Policy recommendations 17 The practice of regression modelling Hypothesis / theory Model specification Data Estimation Specification testing and diagnostic testing Yes Is the model adequate? No Hypothesis testing Policy: prediction and forecasting 18 Sample regression • In practice we will never observe the population regression line. • Instead we take a random sample of observations in order to estimate the s. • We distinguish the sample regression from the population regression as follows: 19 Sample regression Mathematical Model Econometric Model Yˆi ˆ1 ˆ2 X i Yi ˆ1 ˆ2 X i uˆ i where Yˆ = estimator of E(Y/Xi) ˆ1 = estimator of 1 ˆ2 = estimator of 2 uˆ i = estimate of ui 20 Population regression Mathematical Model Yi 1 2 X i ui Yi 1 2 X i where Y Econometric Model = E(Y/Xi) 1 = constant/Y intercept 2 = coefficient for Xi ui = error term 21 .Y Y 4 . Yˆ3 . .Y . 2 Yˆ1 . Y. Yˆi ˆ1 ˆ2 X i Yˆ4 . Y 3 Yˆ2 1 X1 X2 X3 X4 X 22 .Y Y uˆ4 uˆ2 Yˆ1 . uˆ Y. 2 . Yˆi ˆ1 ˆ2 X i ˆi Yˆ4 Yˆ3 . .Y . 4 uˆ . Y 3 3 Yˆ2 1 1 X1 X2 X3 X4 X 23 : The Ordinary Least Squares (OLS) Model Dependent variable is modelled as a linear function of predictor or independent variables. The dependent variable is continuous e.g. Blood pressure, Cholesterol level or Weight . 24 OLS •What factors cause variation in an individual’s Diastolic blood pressure? •What variables explain movement in Men’s cholesterol level? •What variables are predictive of high birth weight in a population of mothers from Birmingham? Dependent variable can take on any numerical value within the limits of the range of that variable. 25 OLS The OLS method seeks to minimise the residual sum of squares: uˆ (Y i ˆ1 ˆ 2 X i) n i 1 n 2 i 2 n i 1 n uˆ (Y i Yˆ i) i 1 2 i 2 i 1 26 Minimising the residual… Y uˆ 4 uˆ 2 . {. uˆ 3 . uˆ1 } . X1 X2 X3 X4 X 27 Describing the overall fit of the estimated model Coefficient of determination, or R2, is a measure of the ‘goodness of fit’ of a regression i.e. the proportion of the variation in Yi which is explained by the regression 2 0< R <1 But focusing solely on maximising R2 is not a good idea! (other measures will be consider this afternoon…) 28 Models for Categorical Dependent Variables For use on dependent variables that are either dichotomous (individual has CVD or not), or polytomous (Low, Medium or High cholesterol level) which are quite common in Health-related datasets 29 Models for Categorical Dependent Variables Focus Binary response variable – independent variables are used to predict whether or not some event will occur: Based on certain described characteristics: Will an individual get cancer or not? Will a patient survive or die? will an individual develop CVD or not? 30 Coding of outcomes: Usually coded 1 if the attribute of interest is present and 0 otherwise. Approach to be used: Logistic regression - best for dichotomous dependent variable, and continuous and categorical independent variables. Other commonly used approaches: Probit & Nested Logit 31 Major difference from Ordinary Linear Regression • Uses link for relationship between dependent and independent variable • Substitute maximum likelihood estimation (MLE) of a link function of the dependent variable for regression's use of least squares estimation of the dependent variable itself. MLE - Method of estimating unknown parameters in such a way that the probability of observing a given dependent variable is as high (or maximum) as possible 32 Issues to consider… • Why are OLS models not suitable for dichotomous data? • Logit transformation – Link Function • Marginal & Conditional Odds and Probability 33 Suppose we want to model Yi = β0 + β1X1+ ε but 1 if the i-th individual has the attribute of interest – e.g. CVD yi = 0, otherwise and • β0 is the coefficient on the constant term, • β1 is the coefficient on the independent variable, • X1 is the independent variable – e.g. Age, and • ε is the error term. 34 Let Yi = 1 if the ith individual has CVD, and 0 otherwise. Let also Yi take the values 1 and 0 with probabilities pi and 1-pi, respectively. i.e. P(Y1=1) = P(CVD =1) = p1 P(Y1=0) = P(CVD =0) = 1- p1 35 Why not just use Simple Linear (OLS) regression? Consider a simple OLS regression model CVD = β0 + β1Age+ ε , Assumptions a) ε ~N(0, δ2) b) var (ε) is constant i.e. Homoscedasticity Binary outcome variables violate these assumptions… 36 Why not just use Simple Linear (OLS) regression? • CVD is binary as P takes on only two values. Consequently, ‘ε’ is also binary and therefore ‘normality of residuals’ assumption is violated. • The error terms are heteroscedastic, so regression assumption that the variance of the error term is constant is violated. • The predicted probabilities can be greater than 1 or less than 0 which can be a problem if the predicted values are used in a subsequent analysis! 37 Logit transformation 1. Move from probabilities to Odds Pi CVD exists Odds 1 Pi CVD doesn't exist 2. Take logs of both sides, to get log-odds or Logit Pi log (odds) logit ( Pi ) log βi Agei 1 Pi or equivalently, exp( β Age ) i i Pi (CVD exists) 1 exp( βi Agei ) 38 The Logit transformation removes the floor restriction 39 Logistic Regression Output Part of this output is in form of Odds, Odds ratios and probability. An understanding of these concepts (both marginal and conditional) is therefore cardinal to interpreting Logistic Regression output Key Question to be explored: What factors determine the probability that an individual will or will not develop CVD? 40 Marginal & Conditional odds. CVD No CVD Column Total Smokers 75 25 100 Non-Smokers 40 60 100 Row Total 115 85 200 • The odds of having CVD are 115/85 = 1.353. This is the marginal or unconditional odds of having CVD. The conditional odds of having CVD, given “smokers” is 75:25, or 3. A smoker is 3.0 times as likely to have CVD than he is not to have it The conditional odds of having CVD, given the category “Non-smokers" is 40:60, or 0.67. A non-smoker is 0.67 times as likely to have CVD than he is not to have it 41 Probability The probability of having CVD is 115/200 = 0.575 The probability of having CVD given that one is a smoker is 75/100 = 0.75 The probability of having CVD given that one is a non-smoker is 40/100 = 0.40 42 Odds Ratio The odds ratio of smokers (numerator) to non-smokers (denominator) for CVD, is 3/0.67= 4.478 (This means that the odds of smokers having CVD are 4.478 times as high as those of non-smokers having CVD) Odds ratio is cross-product ratio i.e. (60* 75) 4.478 (40* 25) When one moves from being a non-smoker to a smoker, the odds of having CVD increase by 347.8% (i.e. from 0.67 odds for non-smokers to 3 for smokers) 43 Alternative interpretation of Odds Ratio • Smokers are 4.478 times more likely to have CVD as nonsmokers • The risk of having CVD is 4.478 times greater for smokers than non-smokers • The odds of CVD for smokers are 347.8% higher than the odds of CVD for non-smokers (4.478 - 1.00) • The predicted odds for smokers are 4.478 times the odds for non-smokers. • A one unit change in the independent variable Smokers (smokers to non-smokers) increases the odds of having CVD by a factor of 4.478. 44 References • Altman D.G. 1991. Practical Statistics For Medical Research (London: Chapman & Hall/CRC) • Gujarati D.N. 1995. Basic Econometrics (New York: McGrawHill, Inc) • Johnston J. and J. DiNardo. 1997. Econometric Methods (London: The McGraw-Hill Companies, Inc) • Long J.S. 1997. Regression Models for Categorical and Limited Dependent. A Volume in the Sage Series for Advanced Quantitative Techniques (Thousand Oaks, CA: Sage Publications • Want, MinQi, James M. Eddy, Eugene C. Fitzhugh. 1995. "Application of Odds Ratio and Logistic Models in Epidemiology and Health Research," Health Values 19 : 59-62. 45 References • Nathwani et al. 2004. “An economic evaluation of a European cohort from a multinational trial of linezolid versus teicoplanin in serious Gram-positive bacterial infections: the importance of treatment setting in evaluating treatment effects” International Journal of Antimicrobial Agents 23: 315–324 • Manca A, Hawkins N, Sculpher M. 2005. “Estimating mean QALYs in trial-based cost-effectiveness analysis: the importance of controlling for baseline utility” Health Economics 14:487-496 • Hoch et al. 2002 “Something old, something new, something blue: a framework for the marriage of health econometrics and costeffectiveness analysis” Health Econ 11:415–430. • Gray et al. 2006, "Estimating the association between SF-12 responses and EQ-5D utility values by response mapping", Med Decis Making., vol. 26, no. 1, pp. 18-29. 46 References • Kaambwa et al. 2011, “Mapping utility scores from the Barthel index", Eur. Journal of Health Economics, DOI: 10.1007/s10198011-0364-5 • Sengupta et al. 2004, "Mapping the SF-12 to the HUI3 and VAS in a managed care population", Med Care.,42,9: 927-937. • Smith et al. 2007. Predicting Costs Of Care In Chronic Kidney Disease: The Role Of Comorbid Conditions. The Internet Journal of Nephrology 4, 1 • Bonizzato et al. 2000, “Community-based mental health care: to what extent are service costs associated with clinical, social and service history variables? Psychological Medicine, 30: 12051215. • Baumeister et al. 2009, “Predictive modeling of health care costs: do cardiovascular risk markers improve prediction? European 47 Journal of Cardiovascular Prevention & Rehabilitation References • Hoch et al. 2006, “Using the net benefit regression framework to construct cost-effectiveness acceptability curves: an example using data from a trial of external loop recorders versus Holter monitoring for ambulatory monitoring of "community acquired" syncope”, BMC Health Services Research, 6:68 • Billingham LJ et al. 2002. “Patterns, costs and cost-effectiveness of care in a trial of chemotherapy for advanced non-small cell lung cancer: evidence from a randomised trial” Lung Cancer 37:219-225 • Engels, J.M. & Diehr, P. 2003, “Imputation of missing longitudinal data: a comparison of methods”, Journal of Clinical Epidemiology 56: 968–976 • Blazer et al. 1995. “Health Services Access and Use among Older Adults in North Carolina:Urban vs Rural Residents” American Journal of Public Health, 85, 10:1384-1390 48 References • Barber, J. & Thomspon, S. 2004, “Multiple regression of cost data: use of generalised linear models”, J Health Serv Res Policy 9:197-204 • Kaambwa, B., Bryan, S., Barton, P., Parker, H., Martin, G., Hewitt, G., Parker, S., & Wilson, A. 2008, "Costs and health outcomes of intermediate care: results from five UK case study sites", Health Soc. Care Community 16: 573 - 581 • Raine et al. 2010, “Social variations in access to hospital care for patients with colorectal, breast, and lung cancer between 1999 and 2006: retrospective analysis of hospital episode statistics”, BMJ 340:b5479 49 Exercises • OLS regression • Logistic Regression 50