Evaluating Physical Activity Intervention Programs. Thomas

Evaluating Physical Activity
Intervention Programs
Thomas Schmid, PhD
Physical Activity and Health Branch
Atlanta, Georgia, USA
Evaluation Definitions and
CDC Physical Activity
Evaluation Handbook – 6
Evaluation issues
The systematic examination and
assessment of features of a
program and its effects. Evaluation
is designed to produce information
that can be used by those who have
an interest in the improvement or
effectiveness of an initiative or
 To
assess the effects and value of a
◦ for accountability
◦ for progress
◦ to identify best practice
◦ as a planning tool
◦ for research
the program
target groups and needs
realistic achievable objectives
is the program ready for evaluation?
Identify needs
State goals and measurable objectives
Assess progress towards objectives
Intervention tested under ideal conditions
with optimal study design
Field based intervention testing whether the
intervention works as intended for a defined
population in the real world
Results in comparison to the efforts required
to obtain them – often cost or cost
Measures short term
Develop program
& resources
Measures activities,
program quality, reach
Measures long
term effects
What is likely to work
Best practices in health promotion
Idea or concept development
Developing and pilot testing materials
Focus groups
Focus groups and
interviews with
partners about their
Reviewing best
practices from
other programs
such as Agita Sao
Pilot testing in one
Process evaluation
◦ Program implemented as intended ?
◦ Quality of program components
◦ Attendance rates
◦ Audience awareness of message
◦ Appropriate message learned
◦ Participant satisfaction
Companies enrolled
Workshops held
Promoters trained
participating in
community events
Impact & Outcome
Short and long term effects of program
◦ Hierarchy of effects : what level of effects indicates
program success
◦ Assessing the impact upon those who attend or
◦ Community effects
Physical activity level
of employees
PA level in community
Disease rates or costs
Program Evaluation Framework
Physical Activity Evaluation Handbook
Program Evaluation Framework
Physical Activity Evaluation Handbook
1. Engage Stakeholders
Include people
with a vested interest in the program:
1. Engage
implementers, partners, participants, possible
opponents and decision makers.
2. Clearly Describe the Program
The problem being addressed, descriptions of the
planning, implementation and maintenance phases of
the program, and finally, defining and communicating
how all phases work together (eg. the "logic model").
Program Evaluation Framework
Physical Activity Evaluation Handbook
3. Clearly Focus
3. Clearly
a clear, straightforward question and
ensuring linkage to the problem being
addressed. Includes clarifying process and/or
outcome evaluations of interest
4. Develop and Implement Data Collection
Design of data collection efforts, selection of
appropriate control groups, designation of
appropriate baseline values and decisions about
how much time should be built in to adequately
measure "change”
Program Evaluation Framework
Physical Activity Evaluation Handbook
5. Develop Conclusions
5. Develop Conclusions
Data analyses and interpretation.
6. Communicate
Promotes the importance of tangible products from the
evaluation, dissemination of those products, and follow-up (as
All physical activity interventions should be
evaluated. Evaluation begins with program
There are many good programs, but very few
are well evaluated
Involve stakeholders
Community – University partnerships can be
very useful
Process as well as outcome evaluation
Quantitative and qualitative
Tailor evaluation to stakeholders and
Evaluation is often undervalued
compared with “hard science”
Adequate resources should be devoted
to evaluation. 10% is recommended
Evaluation results should be used to
adjust programs and inform policy
Share results with stakeholders
Academic and practical training are
The CDC Physical Activity Evaluation
Handbook provides a good framework
Gracias/Thank you

similar documents