LESSON 6 REVIEWER | Training Evaluation
LESSON 6 REVIEWER
Training Evaluation
Learning Objectives
• Define training evaluation and explain the reasons for conducting the evaluation
• Discuss barriers to evaluation
• Describe the different types of evaluation
• Discuss the models of evaluation and their interrelationships
Part 1: Introduction – Key Concepts
Training Effectiveness: The benefits that the company and trainees receive from training. Benefits
may include new skills or behaviors, increased sales, and/or more satisfied customers.
Training Outcomes / Criteria: Measures that the trainer and the company use to evaluate training
programs. To determine effectiveness, an evaluation must occur.
Training Evaluation: The process of collecting the outcomes needed to determine whether training is
effective.
Evaluation Design: The collection of information — including what, when, how, and from whom — that
will be used to determine the effectiveness of the training program.
Part 2: Types of Evaluation
A. Based on Timing/Purpose of Evaluation
Type Description
Formative Evaluation Evaluation that takes place DURING program design and
development. Ensures the program is well-organized, runs
smoothly, and that trainees learn and are satisfied. Involves
PILOT TESTING.
Summative Evaluation Evaluation conducted AFTER training to determine the extent
to which trainees have changed as a result of the program.
Includes measuring monetary benefits (ROI).
Pilot Testing: The process of previewing the training program with potential trainees, managers, or
other customers before full delivery. Part of formative evaluation.
Page 1
LESSON 6 REVIEWER | Training Evaluation
B. Based on Purpose of Information Gathered
Type Description
Descriptive Evaluations Provide information DESCRIBING trainees once they have
completed the program. (What happened?)
Causal Evaluations Provide information to determine WHETHER training CAUSES
the post-training behavior. (Did training cause it?)
Part 3: Barriers to Training Evaluation
Three general reasons evaluations are often NOT conducted:
1. Sophisticated training evaluations are perceived as too complicated to implement, too time-
consuming, and/or too expensive.
2. Training managers resist evaluation because of the difficulty of isolating the unique impact of
training from other variables that affect employee effectiveness.
3. Top management may have no demand for evaluation, and others simply do not wish to know the
results.
Two Categories of Barriers
1. Pragmatic Barriers 2. Political Barriers
• Time and effort to gather and analyze • Some trainers avoid evaluation on
data may not be available ethical grounds
• Evaluations cost money • Conflict of interest
• Technical issues inhibit evaluation • Evaluation is threatening — it may reveal
• Mystification of evaluation (seen as ineffective training
overly complex) • Absence of management demand for
evaluation
Part 4: Models of Training Evaluation
Model 1: Kirkpatrick's Hierarchical Model (4 Levels)
Provides a structured way to measure training impact, ensuring it aligns with broader organizational
goals. Higher levels build upon lower levels.
Level Name Definition How Measured
Level 1 Reaction The degree to which participants Questionnaires, focus
find the training favorable, engaging, groups, interviews
and relevant to their jobs.
Page 2
LESSON 6 REVIEWER | Training Evaluation
Level 2 Learning The degree to which participants Declarative: multiple choice
acquire the intended knowledge, / true-false tests;
skills, attitude, confidence, and Procedural: situational and
commitment based on their mastery tests
participation.
Level 3 Behavior The degree to which participants Self-reports, supervisory
apply what they learned during reports, direct/indirect
training when they are back on the observations, production
job. records
Level 4 Results The degree to which targeted Organizational records,
outcomes occur as a result of the KPIs, ROI
training and the support and
accountability package.
⚠ Remember: Higher levels build upon lower levels. Reaction → Learning → Behavior → Results
Model 2: COMA Model
COMA Model: A training evaluation model designed for FORMATIVE evaluation. Measures Cognitive,
Organizational, Motivational, and Attitudinal variables.
Letter Variable What It Measures Evaluation Assesses
Whether Trainees...
C Cognitive Level of learning gained — Have mastered the skills
declarative knowledge (knowing
information) and procedural
knowledge (translating
knowledge into behavior)
O Organizational Variables generated by the work Accurately perceive the degree
environment that impact the to which the organizational
transfer of training environment will support and
help them apply the skills
M Motivational The desire to learn and to apply Are motivated to learn and
the learned skill on the job apply the skills on the job
A Attitudinal Individual's feelings and thinking Have developed attitudes and
process (e.g., self-efficacy, beliefs that allow them to feel
perceptions of control) capable of applying the newly
acquired skills on the job
Model 3: Decisions-Based Evaluation (DBE) Model
DBE Model: Specifies three key components of evaluation: the target, focus, and method of evaluation.
Component Definition
Target What do you want to find out from the evaluation?
Focus What are the variables we will measure?
Page 3
LESSON 6 REVIEWER | Training Evaluation
Method of Evaluation What appropriate method will be used to conduct the
evaluation?
Part 5: Training Evaluation Variables
The main variables measured in training evaluation:
Variable Definition How Measured
Reactions Trainee perceptions of the program Questionnaires, focus
and/or specific aspects of the course groups, interviews
Learning Trainee acquisition of program Declarative: multiple choice
material. Declarative = knowing info; or true-false tests;
Procedural = translating knowledge Procedural: situational and
into a behavioral sequence mastery tests
Behaviour On-the-job behavior display, objective Self-reports, supervisory
performance measures reports, direct and indirect
observations, production
records
Motivation Trainee desire to learn and/or transfer Questionnaires
skills
Self-Efficacy Trainee confidence in learning and/or Questionnaires
behavior display on the job
Perceived/Anticipated The assistance trainees obtain and/or Questionnaires
Support expect from the organization
Organizational How trainees perceive the Standardized questionnaires
Perceptions organization's culture and climate for
learning and transfer
Organizational Results The impact of training on organizational Organizational records
outcomes
Part 6: Approaches in Measuring Behavior
Behavior (in evaluation): The display of newly acquired skills or competencies on the job.
Approach Description
1. Self-Reports Participants provide information about their own behaviors,
thoughts, or feelings through interviews or surveys.
2. Direct Observation Involves watching and recording behaviors as they happen,
providing firsthand accounts.
3. Physiological Measures Involves biological signals to infer behavior or internal states, often
Page 4
LESSON 6 REVIEWER | Training Evaluation
using technology.
Quick Comparison: The 3 Evaluation Models
Feature Kirkpatrick COMA DBE
Purpose Measure training Formative evaluation of Guide decision-making
impact hierarchically internal trainee factors in evaluation design
Type of Evaluation Summative (and some Formative Both formative and
formative) summative
Structure 4 levels: Reaction, 4 variables: Cognitive, 3 components: Target,
Learning, Behavior, Organizational, Focus, Method
Results Motivational, Attitudinal
Key Focus Progression from Internal trainee What to evaluate, what
satisfaction to business readiness and to measure, how to
results capability measure
Relationship Hierarchical — higher Complementary — Flexible framework —
levels build on lower works alongside applicable to any
ones Kirkpatrick model
Key Terms Cheat Sheet
Term Definition
Training Effectiveness Benefits the company and trainees receive from training
Training Outcomes/Criteria Measures used to evaluate training programs
Training Evaluation Process of collecting outcomes to determine if training is
effective
Evaluation Design Plan for what, when, how, and from whom data will be
collected
Formative Evaluation Evaluation DURING program design/development; includes
pilot testing
Summative Evaluation Evaluation AFTER training to determine change and ROI in
trainees
Pilot Testing Previewing the training program with potential trainees or
managers
Descriptive Evaluation Describes what trainees are like after completing the program
Causal Evaluation Determines whether training caused the post-training behavior
Pragmatic Barriers Practical obstacles: time, cost, technical issues, mystification
Page 5
LESSON 6 REVIEWER | Training Evaluation
Political Barriers Ethical/motivational obstacles: conflict of interest, fear of bad
results, no management demand
Kirkpatrick Model 4-level hierarchical model: Reaction, Learning, Behavior,
Results
COMA Model Formative model measuring Cognitive, Organizational,
Motivational, Attitudinal variables
DBE Model Decisions-Based Evaluation: specifies Target, Focus, and
Method
Behavior (evaluation) Display of newly acquired skills or competencies on the job
Self-Reports Participants report their own behaviors through interviews or
surveys
Direct Observation Watching and recording behaviors as they happen firsthand
Physiological Measures Using biological signals to infer behavior or internal states
— End of Reviewer —
Page 6