Report

Data Quality Sharp project 5 June 2010 Statistical Problems with Data Quality in EHR • Missing Data • Uncertain Diagnosis • Uneven/unequal precision / measurement error • Bias •… Missing Data: (Rage in Statistical Theory) • Common problem with observational/ retrospective data • Statistical approaches – Imputation – Multiple imputation (MI) (Statisticians have acronyms too) – Regression with residual error – draw from Posterior distribution Missing Data– Empirical approach • Regression on Y with Missing X-variables • “X is missing” is also information. • Analyze data set using – Imputation (mean?) – “missing” indicator – Empirical approach– let data tell you what to do Uncertain diagnosis • Universal problem with health data • No Gold standard • Disease/health is a spectrum, not a dichotomy • Probabilistic perspective – Probability (Peripheral Arterial Disease) – From {0,1} to [0-1] as phenotype – More realistic phenotype? Uncertain Diagnosis • Result is a probability • Probability is a posterior distribution of a 0/1 variable – Use p itself (certainty equivalent) • Analogous to single imputation – Use multiple imputation • “1” with probability p, “0” with probability 1-p Uncertain Diagnosis– PAD example (eMERGE) • Mayo Vascular Lab Database– n=18000 • Gold Standard— Ankle/Brachial Index (ABI) • Use of Diagnostic / procedural codes – ICD-9 / HICDA / CPT • Logistic regression of gold standard (PAD by ABI) on diagnostic codes • posterior probability of PAD Uncertain Diagnosis • Model for Pr(PAD)– 90% predictive value • Export model for Pr{PAD} to patients without gold standard ascertainment? • (Coding practices?) Uncertain Diagnosis • Use Pr{PAD} in analysis of – Incidence of PAD – Incidence trends – Surveillance – Analysis of etiology, risk factors Unequal Precision of continuous phenotype • eMERGE example: Red Blood Count • Use retrospective Laboratory Data • N=3000, K=20,000 – 1 measurement 100 measurements/subject • Account for differential precision • Components of variance • Weighted regression? • Posterior distribution– same model fits Sample from Posterior Distribution • Missing Data, uncertain diagnosis, unequal precision can all be represented by sampling from posterior distribution • They are all the “same problem” • Statistical / computational tools for this have been developed – Markov Chain Monte Carlo (MCMC) – Multiple Imputation Summary: Data Quality • ‘Data’ is not ‘a number’ but ‘a posterior distribution’ – Mean and variance – Posterior probability • Data quality – Don’t try to change it – Measure it – Allow for it-- propagation of error What is “Data”? • Data is whatever input goes into the next procedure. • (= output from previous procedure) • ‘Propagation of error’ • Output of NLP is also “Data” How Assess Data Quality? • What if there is no Gold Standard? • Use any external standard – E.g. outcome data • Stronger predictive relationship= better signal/noise ratio? • “Errors-in-variables” principle – Larger error in X –> Smaller beta for Y|X Summary: Help! • What are the important tasks in Data Quality? – Measurement? – Allowance for? • Important tasks for this Project? – Integrate with other projects