Report

Assumptions “Essentially, all models are wrong, but some are useful” Your model has to be wrong… … but that’s o.k. if it’s illuminating! George E.P. Box Linear Model Assumptions Absence of Collinearity No influential data points Normality of Errors Homoskedasticity of Errors Independence Linear Model Assumptions Absence of Collinearity No influential data points Normality of Errors Homoskedasticity of Errors Independence Absence of Collinearity Baayen (2008: 182) Absence of Collinearity Baayen (2008: 182) Where does collinearity come from? …most often, correlated predictor variables Demo What to do? Linear Model Assumptions Absence of Collinearity No influential data points Normality of Errors Homoskedasticity of Errors Independence Leverage Baayen (2008: 189-190) 0 -5 y 5 p = 0.37 -5 0 x 5 0 -5 y 5 p = 0.000529 -5 0 x 5 -1000 -500 0 500 1000 p = 1.18618272971752e-26 -1000 -500 0 500 1000 p = 0.0699135015186808 Leave-one-out Influence Diagnostics DFbeta (…and much more) Winter & Matlock (2013) Linear Model Assumptions Absence of Collinearity No influential data points Normality of Errors Homoskedasticity of Errors Independence Normality of Error The error (not the data!) is assumed to be normally distributed So, the residuals should be normally distributed Histogram of residuals(xmdl) 50 100 ✔ 0 Frequency 150 200 xmdl = lm(y ~ x) hist(residuals(xmdl)) -4 -2 0 residuals(xmdl) 2 qqnorm(residuals(xmdl)) qqline(residuals(xmdl)) 1 -1 0 ✔ -2 Sample Quantiles 2 Normal Q-Q Plot -2 -1 0 Theoretical Quantiles 1 2 qqnorm(residuals(xmdl)) qqline(residuals(xmdl)) ✗ Linear Model Assumptions Absence of Collinearity No influential data points Normality of Errors Homoskedasticity of Errors Independence Homoskedasticity of Error The error (not the data!) is assumed to have equal variance across the predicted values So, the residuals should have equal variance across the predicted values Noise 0 200 400 600 Reaction time 800 1000 -3 -2 -1 0 1 residuals(xmdl) ✔ -10 -5 0 fitted(xmdl) 5 2 3 4 -5 0 fitted(xmdl) 5 -0.5 0.0 0.5 1.0 residuals(xmdl) ✗ 1.5 2.0 2.5 150000 50000 residuals(xmdl) 0 -100000 -200000 1e+06 2e+06 3e+06 fitted(xmdl) 4e+06 5e+06 ✗ 6e+06 WHAT TO IF NORMALITY/HOMOSKEDASTI CITY IS VIOLATED? Either: nothing + report the violation Or: report the violation + transformations Two types of transformations Linear Transformations Nonlinear Transformations Leave shape of the distribution intact (centering, scaling) Do change the shape of the distribution 200 100 0 Frequency 300 400 Histogram of xdata$Pic.RT 0 1000 2000 xdata$Pic.RT 3000 4000 100 50 0 Frequency 150 Histogram of log(xdata$Pic.RT) 5.5 6.0 6.5 7.0 log(xdata$Pic.RT) 7.5 8.0 8.5 1000 500 0 -500 -1000 residuals(xmdl) 1500 2000 Before transformation 500 1000 fitted(xmdl) 1500 2000 0.5 0.0 -0.5 residuals(xmdl.log) 1.0 After transformation 6.0 6.5 fitted(xmdl.log) 7.0 Still bad…. 7.5 …. but better!! Assumptions Absence of Collinearity No influential data points Normality of Errors Homoskedasticity of Errors Independence Assumptions Normality of Errors Homoskedasticity of Errors (Histogram of Residuals) Residual Plot Q-Q plot of Residuals Assumptions Normality of Errors Absence of Collinearity Independence Homoskedasticity Errors Noofinfluential data points Assumptions Absence of Collinearity No influential data points Normality of Errors Homoskedasticity of Errors Independence What is independence? Common experimental data Subject Item #1 Rep 1 Rep 3 Rep 2 Item ... Item ... Common experimental data Subject Item #1 Rep 1 Rep 3 Rep 2 Item ... Pseudoreplication = Disregarding Dependencies Item ... Subject1 Subject1 Subject1 … Item1 Item2 Item3 … “pooling fallacy” Machlis et al. (1985) Subject2 Subject2 Subject3 …. Item1 Item2 Item3 … “pseudoreplication” Hurlbert (1984) Hierarchical data is everywhere • Typological data (e.g., Bell 1978, Dryer 1989, Perkins 1989; Jaeger et al., 2011) • Organizational data • Classroom data Finnish Norwegian Swedish English French Spanish Germa n Hungarian Romanian Italian Turkish Finnish Norwegian Swedish English French Spanish Germa n Hungarian Romanian Italian Turkish Hierarchical data is everywhere Class 1 Class 2 Hierarchical data is everywhere Class 1 Class 2 Hierarchical data is everywhere Class 1 Class 2 Hierarchical data is everywhere Hierarchical data is everywhere Intraclass Correlation (ICC) Simulation for 16 subjects pseudoreplication Type I error rate items analysis Interpretational Problem: What’s the population for inference? Violating the independence assumption makes the p-value… …meaningless S1 S2 S1 S2 That’s it (for now)