Module 6 Slides Darcy Freedman

Module 6
Darcy Freedman, MPH, PhD
June 18, 2014
Assumptions in Scientific Research
 Nature is orderly and regular
 To some extent, events are consistent and predictable
 Events or conditions have one or more causes that can
be discovered
 This enables establishing cause and effect relationships
The Scientific Method
The scientific method has been defined as a systematic,
empirical, controlled and critical examination of
hypothetical propositions about the association among
natural phenomena.1,2
Kerlinger FN. Foundations of Behavioral Research. New York: Holt, Rinehart &
Winston, 1973.
Portney LG, Watkins MP. 2000. Foundations of Clinical Research: Applications to
Practice. 2nd Ed. Upper Saddle River, NJ: Prentice Hall Health.
Properties of scientific method
 Use of orderly procedures to ensure reliability
 Logical sequence is used from problem identification, through data
collection, analysis, & interpretation
 Documentation of objective data through direct observation (or other
systematic methods)
 Findings are grounded in the objective observation of phenomena
rather than the personal bias or subjective belief of the researcher
 In order to understand how one phenomenon relates to another,
factors are controlled that are not directly related to the variables in
 Investigators have confidence in their research outcomes to the extent
that they control extraneous influences
 Science is imperfect, especially when it is applied to
human behavior and performance
 Science is imperfect, especially when it is applied to
human behavior and performance
 Sources of uncertainty:
 Complexity and variability within nature
 The unique psychosocial and physiological capacities of
 Science is imperfect, especially when it is applied to
human behavior and performance
 Sources of uncertainty:
 Complexity and variability within nature
 The unique psychosocial and physiological capacities of
 Social science researchers must be acutely aware of
extraneous influences in order to interpret findings in
a meaningful way
Types of Research
Case study
Cross-sectional study
Qualitative study
Cohort study
Case control study
True experimental designs
Quasi-experimental designs
Descriptive Research
 Descriptive: investigator attempts to describe a group
of individuals on a set of variables or characteristics.
 Enables classification and understanding
 Methods: survey research, case study, qualitative,
developmental (natural history of something, patterns
of growth and change), normative, evaluation
Store Type
(33 stores)
Quality Composite
Score (mean)
Stores (70%)
Local Markets
Supermarkets (6%)
Composite score = sum of scores for access to fresh fruit,
fresh vegetables, lean meats, low-fat milk,
tobacco products, alcohol. Chronbach’s alpha = .76
Source: Freedman & Bell, 2009
Exploratory Research
 Investigator examines a phenomenon of interest and
explores its dimensions, including how it relates to
other factors.
 Proven relationships between the phenomenon and
other factors can lead to predictive models
 Correlational studies, cohort and case control,
secondary analysis, historical research
Freedman, Blake, & Liese, 2013
Figure 2. Simplified Path Analytic Model 1 of Environmental Influence on FV Intake
Source: Liese et al., 2013.
Freedman et al., under review
Experimental Research
 Provides a basis for comparing 2 or more conditions
 Controls or accounts for the effects of extraneous
factors, providing the highest degree of confidence in
the validity of outcomes
 Enables the researcher to draw meaningful
conclusions about observed differences
 Randomized controlled trials, single subject designs,
sequential clinical trials, evaluation research, quasiexperimental research, meta-analysis
Individual-level Change in Fruit and
Vegetable Consumption
 Design: Longitudinal; no comparison group
 Sample: 45 diabetic patients at FQHC
 Intervention: FQHC-based farmers’ market + financial
incentive (up to $50)
 Outcome measure: F/V consumption measured with
NCI screener
 Results:
 Dose-response relationship between improvement in
F/V consumption and use of market
 Improvers more likely to rely on financial incentive to
purchase foods at market
Source: Freedman et al., 2013
Describe populations
Find relationships
Cause and Effect
Continuum of Research
Case Study
randomized controlled
trial (RCT)
Quasi-experimental designs
----------------------------------------------Survey research-----------------------------------------------------Qualitative research---------------Correlational research
Sequential clinical trial
Single subject designs
Evaluation research
Evaluation research
------------------------Secondary analysis-----------------
-----Cohort/Case-Control Study----Historical research
Based on: Portney LG, Watkins MP. 2000. Foundations of Clinical Research: Applications to Practice. 2nd Ed. Upper
Saddle River, NJ: Prentice Hall Health., p. 13.
Community-engaged research
 Philosophy versus method
 Who are the “knowers” of phenomenon?
 Participatory processes during some or all stages of
 Knowledge for action/change
 Can be used with any research approach
 Example: Community Visions Photovoice Project (~9
 Quantitative research involves measurement of
outcomes using numerical data under standardized
 May be used along the continuum of research
 Qualitative research is concerned with narrative
information under less structured conditions that
often takes the research context into account
 Descriptive and exploratory research
 Purposes: describing conditions, exploring associations,
formulating theory, generating hypotheses
Choosing evaluation methods
Descriptive Research
 Case study
 Cross-sectional study
 Qualitative study
Case Study Design
 Often a description of a individual case’s condition or
response to an intervention
 can focus on a group, institution, school, community,
family, etc.
 data may be qualitative, quantitative, or both
 Case series: observations of several similar cases are
Case Study
 In 1848, young railroad worker, Phineas Gage, was forcing gun powder
into a rock with a long iron rod when the gun powder exploded. The
iron rod shot through his cheek and out the top of his head, resulting
in substantial damage to the frontal lobe of his brain. Incredibly, he
did not appear to be seriously injured. His memory and mental abilities
were intact, and he could speak and work. However, his personality was
markedly changed. Before the accident, he had been a kind and
friendly person, but afterward he became ill-tempered and dishonest.
 Phineas Gage’s injury served as a case study for the effects of frontal
lobe damage. He did not lose a specific mental ability, such as the
ability to speak or follow directions. However, his personality and
moral sense were altered. It is now known that parts of the cortex
(called the association areas) are involved in general mental processes,
and damage to those areas can greatly change a person’s personality.
Case Study Design
 Strengths
 Enables understanding of the totality of an individual’s (or
organization, community) experience
 The in-depth examination of a situation or ‘case’ can lead to
discovery of relationships that were not obvious before
 Useful for generating new hypotheses or for describing new
 Weaknesses
 No control group
 Prone to selection bias and confounding
The interaction of environmental and personal characteristics make
it weak in internal validity
 Limited generalizability
Cross-sectional Study
 Researcher studies a stratified group of subjects at one
point in time
 Draws conclusions by comparing the characteristics of
the stratified groups
 Well-suited to describing variables and their
distribution patterns
 Can be used for examining associations;
determination of which variables are predictors and
which are outcomes depends on the hypothesis
 eg. Does lead paint ingestion cause hyperactivity or does
hyperactivity lead to lead paint ingestion?
Cross-sectional Study
 Example:
What is the prevalence of chlamydia in women age 18-35
in Cleveland, and is it associated with the use of oral
Select a sample of 100 women attending an STD clinic in the
city of Cleveland
Measure the predictor and outcome variables by taking a
history of oral contraceptive use and sending a cervical swab
to the lab for chlamydia culture
 A questionnaire may be used to gather information abut oral
contraceptive history
Cross-sectional Study
 Strengths
 Fast and inexpensive
 No loss to follow-up (no follow-up)
 Ideal for studying prevalence
 Convenient for examining potential networks of causal links
e.g., in analysis, examine age as a predictor of oral contraceptive use, and
then examine oral contraceptive use as a predictor for chlamydia
 Weaknesses:
 Difficult to establish a causal relationship from data collected in a
cross-sectional time-frame (Lack of a temporal relationship
between predictor variables and outcome variables - Does not
establish sequence of events)
 Not practical for studying rare phenomena
Qualitative Study
 Seeks to describe how individuals perceive their own
experiences within a social context
 Emphasizes in-depth, nuanced understanding of
human experience and interactions
 Methods include in-depth interviews, direct
observations, examining documents, focus groups
 Data are often participants’ own words and narrative
summaries of observed behavior
Qualitative Study
A researcher wants to understand how provision of
healthcare to undocumented persons affects the people
and institutions involved
 In 3 communities, information is gathered from
undocumented patients, FQHC primary care clinicians,
specialists, and hospital administrators
 Methods: in-depth interviews, key informant
interviews, participant observations, case studies, focus
Qualitative Study
 Data based on the participants’ own categories of meaning
 Useful for studying a limited number of cases in depth or describing complex
 Provides understanding and description of people’s personal experiences of phenomena
 Can describe in rich detail phenomena as they are embedded in local contexts
 The researcher can study dynamic processes (i.e., document sequential patterns/change)
 Knowledge produced might not generalize to other people or other settings
 It is difficult to make quantitative predictions
 It might have lower credibility with some administrators and commissioners of programs
 Takes more time to collect and analyze the data when compared to quantitative research
 The results are more easily influenced by the researcher’s personal biases and
Exploratory Research
 Cohort study
 Case control study
Cohort Study
 A group of individuals who do not yet have the
outcome of interest are followed together over time to
see who develops the condition
 Participants are interviewed or observed to determine
the presence or absence of certain exposures, risks, or
 May be simply descriptive
 May identify risk by comparing the incidence of
specific outcomes in exposed and not exposed
Cohort Study
 Example
To determine whether exercise protects against coronary
heart disease (CHD).
1. Assemble the cohort: 16,936 Harvard alumni were
2. Measure predictor variables: Administer a
questionnaire about activity and other potential risk
factors , collected data from college records
3. 10 years later, sent a follow-up questionnaire about
CHD and collected data about CHD from death
Cohort Study
 Strengths
 Powerful strategy for defining incidence and
investigating potential causes of an outcome before it
 Time sequence strengthens inference that the factor
may cause the outcome
 Weaknesses
 Expensive – many subjects must be studied to observe
outcome of interest
 Potential confounders: eg, cigarette smoking might
confound the association between exercise and CHD
Case-Control Study
 Generally retrospective
 Identify groups with or without the condition
 Look backward in time to find differences in predictor
variables that may explain why the cases got the
condition and the controls did not
 Assumption is that differences in exposure histories
should explain why the cases have the condition
 Data collection via direct interview, mailed
questionnaire, chart review
Case-Control Study
 Strengths
 Useful for studying rare conditions
 Short duration & relatively inexpensive
 High yield of information from relatively few participants
 Useful for generating hypotheses
 Weaknesses
 Increased susceptibility to bias:
Separate sampling of cases and controls
Retrospective measurement of predictor variables
 No way to estimate the excess risk of exposure
 Only one outcome can be studied
Case-Control Study
 Example
Purpose: To determine whether there is an association
between the use of aspirin and the development of Reye’s
syndrome in children.
Draw the sample of cases – 30 patients who have had
Reye’s syndrome
Draw the sample of controls – 60 patients from the much
larger population who have had minor viral illnesses
without Reye’s syndrome
Measure the predictor variable: ask patients in both
groups about their use of aspirin
Experimental Research
 True experimental designs
 Quasi-experimental designs
Efficacy vs. Effectiveness
 Efficacy: the benefit of an intervention compared to a
control or standard program under controlled,
randomized conditions
 Randomized controlled trial (RCT) design often used
 Effectiveness: the benefit of an intervention under less
controlled ‘real world’ conditions
 Quasi-experimental design often used
Types of designs
1. One group posttest only design
P = Program or intervention
T2 = Posttest
Types of designs
2. Before and After Design
One group pretest-post-test design
T1 = Pretest (treatment group)
T2 = Posttest (treatment group)
P = Program or intervention
How much of the effect is due to the program?
Time (X)
Types of designs
2. Comparison Group Design
T1 = Pretest (treatment group)
T2 = Posttest (treatment group)
P = Program or intervention
C1 = Pretest (comparison group)
C2 = Posttest (comparison group)
Experimental Design
 True experimental design: Subjects are randomly
assigned to at least 2 comparison groups
 Purpose is to compare 2 or more groups that are
formed by random assignment
 The groups differ solely on the basis of what occurs
between measurements (ie, intervention)
 Changes from pretest to posttest can be reasonably
attributed to the intervention
 Most basic is the pretest-posttest control group design
(randomized controlled trial, RCT)
Experimental Design
 Researchers conducted an RCT to study the effect of
progressive resistance exercises in depressed elders. They
studied 35 volunteers who had depression.
 Participants were randomly assigned to an exercise group,
which met three times per week for 10 weeks, or a control
group which met 2 times per week for an interactive health
education program.
 The outcome variables were: level of depression, functional
status, and quality of life, using standardized instruments.
 Pretest and posttest measures were taken for both groups
and differences were compared.
Experimental Design
 Controls the influence of confounding variables, providing more
conclusive answers
 Randomization eliminates bias due to pre-randomization confounding
 Blinding the interventions eliminates bias due to unintended
 Costly in time and money
 Many research questions are not suitable for experimental designs
 Usually reserved for more mature research questions that have already
been examined by descriptive studies
 Experiments tend to restrict the scope and narrow the study question
Quasi-Experimental Design
 Quasi-Experimental designs do not use randomized
assignments for comparisons
Quasi-Experimental Design
 Example:
 A study was designed to examine the effect of electrical
stimulation on passive range of motion of wrist
extension in 16 patients who suffered a stroke.
 Outcomes: effects of treatment on sensation, range of
motion, & hand strength.
 Patients were given pretest and posttest measurements
before and after a 4-week intervention program.
 Note: No randomization, and no comparison group
Quasi-Experimental Design
 Q-E designs are a reasonable alternative to RCT
 Useful where pre-selection and randomization of groups is difficult
 Saves time and resources vs. experimental designs
 Nonequivalent groups may differ in many ways -- in addition to the
differences between treatment conditions, introducing bias
 Non-blinding allows the possibility of unintended interventions;
blinding can be used in some Q-E studies
 Must document participant characteristics extensively
 Potential biases of the sample must be acknowledged when reporting
 Causal inferences are weakened by the potential for biases vs.
experimental designs
Compared to what?
 Over time
 Pre to post
 Longitudinal
 Between groups
 Randomly composed
 Naturally occurring (waitlist,
other programs)
 National norms/standards
Low Ability to
Attribute Effect
Pre & Post
High Ability to
Attribute Effect
(matched groups,
Major sources
Portney LG, Watkins MP. 2000. Foundations of Clinical
Research: applications to practice. 2nd Ed. Upper Saddle
River, NJ: Prentice Hall Health
Hulley SB, Cummings SR. 1998. Designing Clinical
Research: an epidemiologic approach. Baltimore, MD:
Williams and Wilkins
Cook TD, Campbell DT. 1979. Quasi-Experimentation:
design & analysis issues for field settings. Boston, MA:
Houghton Mifflin Company

similar documents