Module 14 Overview Qualitative Research Broad view of features Many traditions Contrasts with quantitative research Qualtitative method roadmap Data collection methods Standards of evaluation Mixed Methods • Approaches to mixing • Examples Research Methods In the social sciences, there are 3 generic types of research methods Quantitative methods Qualitative methods ‘Mixed methods’ blending quantitative and qualitative Qualitative Research Focus on making sense of or interpreting social or human phenomena: experiences, behaviors, organizational functioning, social movements, interactional relationships, descriptions of culture Phenomena are studied in their natural settings Investigators interpret phenomena in terms of the meanings people bring to them Denzin and Lincoln, 2000. Qualitative work: Common features The researcher is the primary instrument for data collection and analysis Involves fieldwork Attends to low inference descriptive data Qualitative Methods - Traditions Tradition Purpose Methods Allied Discipline Ethnography Cultural description Intensive fieldwork Anthropology Phenomenology Explore lived experience Analysis of cases to Philosophy formulate meaning Psychology Biography Meaning of experience as recorded through story Interviews and documents exploring the life of an individual Literature Literary Criticism Grounded Theory Theory development, provide an explanation Constant comparative method Sociology Qualitative Research Attends to description of how vs. how many Is an inductive and open process: the investigator builds concepts, hypotheses, and theories from analysis of the data. The data collection and analysis process is iterative. The investigator returns to the data with new questions and ideas to explore until there is a very deep understanding of the phenomenon / issue. Qualitative Assumptions Reality is socially constructed Primacy of subject *Variables are complex, interwoven and difficult to measure Relationships between variables are generally described as observed patterns or cases Quantitative Assumptions Social facts have an objective reality Primacy of method *Variables are measured with existing tools Relationships between variables can be assessed using standard statistics Qualitative Purpose Contextualization Interpretation Understanding peoples’ perspectives Researcher’s Role Personal involvement and partiality Emphatic understanding Design Flexible and emergent Quantitative Purpose Generalizable findings Prediction Causal explanations Researcher’s Role Detachment and impartiality Objective portrayal Design a priori and inflexible Qualitative Purpose Contextualization Interpretation Understanding peoples’ perspectives Researcher’s Role Personal involvement and partiality Emphatic understanding Design Flexible and emergent Quantitative Purpose Generalizable findings Prediction Causal explanations Researcher’s Role Detachment and impartiality Objective portrayal Design a priori and inflexible Qualitative Quantitative Approach Approach End with hypotheses and Begins with hypotheses and grounded theory Emergence and portrayal Naturalistic Inductive Searches for patterns Seeks pluralism, complexity Minor use of numerical indices Thick description through writing theories Manipulation and control Experimentation Deductive Component analysis Seeks consensus, the norm Reduces all data to numerical indices Precise technical language, numerical presentation Questions? The Circle of Inquiry Identify Knowledge Gap Search for Existing Information Disseminate & Implement Focus the Study Question Analyze & Interpret Results Collect Data Design the Study Adapted from: Nutting, PA, Stange, KC. Practice-based research: The opportunity to create a learning discipline. In: The Textbook of Family Practice, 6th Edition. Rakel RE (ed.), W. B. Saunders Company, 2001. Qualitative Data Collection Methods In-depth interviews Narratives, detailed responses to probing questions Direct observation Field notes containing descriptions of activities, behaviors, actions, interactions, and processes Document analysis Written documents: records, memoranda, correspondence, reports, diaries Qualitative Methods Road Map Sampling / Data collection Describing Organizing Connecting Corroborating / Legitimating Crabtree & Miller, Doing Qualitative Research 1999 2nd Ed. Qualitative Methods Road Map Sampling / Data collection Describing Organizing Connecting Representing the account Corroborating / Legitimating Qualitative Methods Road Map Sampling / Data collection Describing Organizing Connecting Actively reflecting, Specifically how what is happening is influencing & shaping the interpretive process. Determining next steps. Representing the account Corroborating / Legitimating Qualitative Methods Road Map Sampling / Data collection Describing Organizing Connecting 3 general strategies: - Template - Editing - Immersion /Crystalization Corroborating / Legitimating Qualitative Methods Road Map Sampling / Data collection Describing Organizing Connecting Representing the account Corroborating / Legitimating Qualitative Methods Road Map Gathering Process Sampling / Data collection Describing Organizing Connecting Representing the account Corroborating / Legitimating Qualitative Methods Road Map Sampling / Data collection Describing Organizing Connecting Corroborating / Legitimating Analysis Process Representing the account Qualitative Methods Road Map Interpretive Process Sampling / Data collection Describing Organizing Connecting Corroborating / Legitimating Representing the account Qualitative Methods Road Map Sampling / Data collection Describing Organizing Connecting Reflexivity Representing the account Corroborating / Legitimating Questions? Standards of Qualitative Research God is in the details. R. Preston, 1994, p. 298 The devil is in the details. Popular adage. Frankel, Standards of Qualitative Research. In Doing Qualitative Research, 2nd Ed. Standards for Evaluation Quantitative Internal Validity Qualitative External Validity (Transferability) Reliability Confirmability, dependability Objectivity Engagement, reflexivity Credibility / Trustworthiness Trustworthiness Ways to achieve trustworthiness: Member checks: recycling interpretation back to the key informants Searching for disconfirming evidence Triangulation: multiple data sources and multiple methods Thick description: a thorough description of the context of the study Confirmability Ways to address confirmability: Collection of data in ways that allow for audits Audio recordings Full transcripts of interviews Collection of low inference observational data Engaging a team approach Independent auditors Reflexivity Ways to address reflexivity: Document beliefs, framework, theories underlying approach to the problem before beginning the data collection. Actively journal reflections, possible biases/ limitations to ‘lens’ or ‘gaze’. Document how to overcome limitations to more fully examine phenomena. Engage other perspectives in team analysis. Report reflexivity in final product Summary: What’s Involved in Qualitative Inquiry Spend time in the field gathering data Engage in data analysis to reduce raw data to themes or categories; complex & time-consuming Write extensively to show multiple perspectives in order to substantiate findings Participate in a form of research that does not have firm guidelines or specific procedures, and is evolving and constantly changing Source: Creswell JW. 1998. Qualitative inquiry and research design: choosing among five traditions. Thousand Oaks, CA: Sage. Qualitative/Quantitative Both approaches are highly respected When done well, both contribute equally to the knowledge base When possible, researchers should blend the methods enriches the data to enable deeper understanding validates the conclusions from more than one perspective Summary of Quantitative and Qualitative features Feature Quantitative Qualitative Role of researcher Detached, objective Engaged Purpose Test hypothesis Describe; Develop theory Data collection methods A priori, structured Flexible, iterative Data analysis Deductive, statistical Inductive, iterative leading to more data collection and revision of question Type of data Numbers Product Status of hypothesis Rich description; Theory development; A book Predominately Words Mixed Methods General consensus that no longer a quantitative vs. qualitative methods – both are necessary. Mixed Methods Attack the research problem with an arsenal of methods that have non-overlapping weaknesses in addition to complementary strengths. J Brewer & A Hunter Foundations of multimethod research: Synthesizing styles. Sage 2006. page 4. Mixed Methods Scope of mixing methods – within study – within line of inquiry – within broad topic area Approaches to mixing methods Sequential studies Quantitative Qualitative Qualitative Quantitative Approaches to mixing methods Mixing methods in the same study Dominant/ Less Dominant Secondary method plays a small role Triangulation Equal weight, both Quantitative and Qualitative data collected & analyzed in a complementary manner. Examples of mixing Data transformation – convert data of one type to the other can be analyzed together. Typology development – one type of data used to develop a typology that is used to drive analysis with other type of data. Extreme case analysis – pursue data collection or analysis of data of the other type with the intent of refining the initial explanation for the extreme case. Example line of inquiry Observational Studies DOPC Intervention Studies STEP-UP Direct Observation of Primary Care (NCI, RWJF: 1994-97) Study To Enhance Prevention by Understanding Practice (NCI: 1997-2000) P&CD Prevention & Competing Demands in Primary Care (AHRQ: 1996-99) IMPACT Insights from Multimethod Practice Assessment of Change over Time (NCI: 2001-2004) ULTRA Using Learning Teams for Reflective Adaptation (NHLBI: 2002-07) EPOCHS Enhancing Practice Outcomes through Community and Healthcare Systems (2004-09) Direct Observation of Primary Care To accurately measure, using direct observation, the level of preventive services delivered to eligible patients, and to document the context of the competing demands of the primary care medical encounter To determine the optimal non-observational method… To use a multimethod approach to identify and explore which characteristics of the medical encounter, physicians, patients and the office environment are associated with delivery of preventive services to eligible patients Direct Observation of Primary Care Cross-sectional observation of 84 family practices & 4454 patient visits to 138 physicians in Ohio Direct Observation Davis Observation Code Checklists Medical Record Reviews Patient Exit questionnaire Billing Data Practice Environment Checklist Ethnographic Fieldnotes Prevention & Competing Demands PI – Crabtree Aim: Understand primary care practice from the ground up through intensive direct observation of the practice environment and patient care to better understand how these factors affect preventive service delivery Prevention & Competing Demands In-depth multimethod comparative case study of 18 family practices & 1,600 visits to 56 clinicians in Nebraska Longer direct observation of practice environment recorded in checklists and field notes (4-8 weeks of observation) Direct observation of 30 encounters/clinician recorded in checklists and field notes Chart audits of patients who were observed Interviews of all clinicians, most staff, some community members Study To Enhance Prevention by Understanding Practice (STEP-UP) Randomized clinical trial of 80 family practices in Ohio Multimethod assessment (MAP) of values, structures, and processes Patient survey and medical record review to assess preventive service delivery at 6 month intervals Practice-individualized intervention Doctor-patient communication study Specifically this study aims to: 1. Characterize the way a ‘teachable moment’ arises and is constructed within the patient-physician interaction. 2. Identify factors that enhance or impede the use of illness visits to provide health behavior advice. 3. Determine the effectiveness of a ‘teachable moment’ for increasing: patient recall of advice, motivation to modify behavior, and change in health behavior. Doctor-patient communication study Graphic representation of mixing Doctor-patient communication study Outputs Primary paper #1 Discovery of the teachable moment communication process. Primary paper #2 Quantitative testing the association of TM vs. other kinds of advice with outcomes Doctor-patient communication study Embedded example National Heart Lung & Blood guideline for assessment and treatment of obesity. - coding template of key guideline elements - audio recordings, apply template - findings from quantitative prompted qualitative examination. (Unplanned) Mixing methods Expertise/training in methods you want to mix - team members - consultants - cross training - learning by doing* Summary The Power of Qualitative Data Stories are powerful and memorable. have a concrete, vivid, meaningful flavor that often proves far more convincing and memorable to a reader – another researcher, a policymaker, a practitioner Reports include explanations of processes in context and derive explanations Enable investigators to address complex social issues where little is understood. Methods push beyond initial conceptions to generate or revise conceptual frameworks Source: Miles MB, Huberman AM. 1994. Qualitative data analysis: an expanded sourcebook. 2nd ed. Thousand Oaks, CA: Sage. Making Decisions about which Methods to Use What is the research question? What is the goal / purpose of the study? What kinds of information are needed to answer the Q? When is the information needed? What resources are available to conduct the study? Given answers to the preceding questions, what methods are appropriate? Source: Patton MQ. 1990. Qualitative evaluation and research methods, 2nd ed. Newbury Park, CA: Sage. p. 12 Questions? Sue Flocke [email protected] Qualitative Analysis Affixing codes to interview data and field notes from observations Noting reflections in the margins Sorting and sifting through these materials to identify similar phrases, relationships, patterns, themes, commonalities, & differences Isolating patterns, processes, commonalities, & differences and incorporating methods to further explore them into the next wave of data collection Gradually developing a small set of generalizations about what consistently appears in the data Confronting those generalizations with a formalized body of knowledge in the form of constructs or theories Themes of Qualitative Inquiry Theme Definition Naturalistic Inquiry Studying real-world situations as they unfold naturally; nonmanipulative; unobtrusive; lack of pre-determined constraints on outcomes Holistic perspective The whole phenomenon is understood as a complex system that is more than the sum of its parts; not reduced to a few discrete variables and cause and effect relationships Direct, close contact with people, situation, or phenomenon under Personal contact and study; researcher’s insights are important to inquiry and insight understanding Source: Patton MQ. 1990. Qualitative evaluation and research methods, 2nd ed. Newbury Park, CA: Sage. Themes of Qualitative Inquiry Theme Dynamic systems Unique case orientation Context sensitivity Empathic neutrality Design flexibility Definition Attention to process; assumes change is constant and ongoing Assumes each case is special and unique; captures details of individual cases; cross-case analysis follows from individual case studies Places findings in a social, historical, and temporal context; dubious of generalizations from other times and settings Overriding objective is to understand the subject of investigation in all its complexity rather than proving, advocating, or advancing personal agendas Open to adapting inquiry as understanding deepens and situations change; avoids getting locked into rigid designs; pursues new paths of discovery as they emerge Source: Patton MQ. 1990. Qualitative evaluation and research methods, 2nd ed. Newbury Park, CA: Sage.