Program Evaluation
Lecture Overview
Program evaluation and program
development
Logic of program evaluation (Program
theory)
Four-Step Model
Comprehensive Quality Programming
Empowerment Evaluation
Group Activity
Program Development
refers to the planning and implementation
of a program determining its purposes and
setting goals and outcomes, evaluating
program process, and evaluating attainment
of desired program outcomes and wider
impacts.
History of Program Evaluation
1930’s human service field grew (post depression)
WWII programs were launched to meet the needs
for urban development and housing, technological
and cultural education, occupation training, and
preventive health activities.
By the end of the 1950’s program evaluation was
commonplace
1960’s increased papers on evaluation research
1970’s, evaluation was a distinct field
1980’s benefits and costs of programs
1990’s new social problems and booming
economy
Program Evaluation
Refers to the collection of evidence to judge
the effectiveness of a community programs
or other community interventions (com
psych definition)
The use of social research procedures to
systematically investigate the effectiveness
of social intervention programs (research
definition).
Project Drug Abuse Resistance
Education D.A.R.E.
Most popular school based intervention
Little evidence on its effectiveness
Limited positive influence on psychological
factors (self-esteem)
How can this lack of measurable outcomes
in a popular intervention be explained?
Reasons Why Programs May Be
Unsuccessful
Theory Failure: concerns program theory
which is the rationale for why a particular
intervention is considered appropriate for a
particular problem with a specific target
population
Implementation Failure: concerns quality
of program implementation (ie: program
may not be generalizable)
Effects do not occur when…….
The underlying assumptions of program
theory are incorrect.
The program, even if implemented well,
doesn’t affect the variables specified by
program theory.
Or the activity or program is not
implemented adequately
Logic Model
Graphic representation of the program.
Shows the logical connections between the
conditions that contribute to the need for a
program in a community, the activities
aimed at addressing these conditions, and
the outcomes and impacts expected to result
from the activities.
Evaluation “Terminology”
Stakeholders
Fiduciary
Funding sources
Need Assessment
Evaluation Instruments
Four Step Model of Program
Evaluation (Lindsay & Wandersman, 1991)
Step 1: Identify Goals and Desired
Outcomes
Step 2: Process Evaluation
Step 3 Outcome Evaluation
Step 4: Impact Evaluation:
STEP 1) Identifying Goals and
Desired Outcomes
What is your project striving for?
Goals set a framework for specific
outcomes.
Program Developers MUST describe:
The primary goals of the program
Target population
Outcomes desired
STEP 2) Process Evaluation
Monitoring program activities helps organize
program efforts
Provides information to help manage the program
accountability that the program is conducting what
was done and who was reached
After an evaluation of outcomes and impacts , the
evaluation can provide information about why the
program worked or did not work
Can help decide whether or not you are ready to
assess the effects of your program
What were the intended and actual activities of the
program/ Did you learn anything.
STEP 3) Outcome/Impact
Evaluation
Assess the immediate effects of a program
“the bottom line”
Looks at the desired outcomes defined in
Step 1 and seeks evidence regarding the
extent to which those outcomes were
achieved
Should be closely linked to goals, but more
specific
STEP 4) Impact Evaluation
Concerned with the ultimate effects of the
program
Impacts are ultimate or longer-term effects
of the program
Comprehensive Quality
Programming: 9 Strategies
Why is the intervention or program needed?
What are the program’s goals, target populations and
desired outcomes?
How does your program use the scientific knowledge and
best practice of what works?
How will this new program fit in with existing programs
How will you carry out the program
How well was the program carried out
How did the program work?
What can you do to improve the program the next time you
do it?
If the program or parts of the program were effective, what
are you doing to continue institutionalize the program
Accountability Questions &
Corresponding CQP Strategies
Why is the intervention or program Needs and assets assessments
needed?
What are the program’s goals, target Identifying goals and desired outcomes
populations and desired outcomes?
How does your program use the Review research literature and best
scientific knowledge and best practices
practice of what works?
How will this new program fit in Community feedback and planning
with existing programs Planning, implementation
How will you carry out the program
Process evaluation
How well was the program carried
Outcome and impact evaluations
out
How did the program work?
Continuous program improvement
What can you do to improve the
program the next time you do it? Institutionalization strategies
If the program or parts of the
program were effective, what are you
doing to continue institutionalize the
program
Empowerment Evaluation
Program planners and developers learn the
basics of program evaluation in ongoing
program improvement.
Help program developers and staff to
achieve their program goals, by providing
the with the tools for assessing and
improving the planning, implementation,
and results of their program.
Limitations of Evaluation
Use of an outside evaluator
Evaluation feedback is provided at the end
of the program
Evaluation findings are perceived as being
too complex
Conclusions
Program evaluation is necessary to find out
whether a program is working
Evaluation can be used for program
planning and program implementation
As society becomes more concerned about
accountability and results , evaluation will
become more necessary.
Group Activity
Apply 4 step model in terms of your group
project.
Answer CQP questions