Session 9

SAT in depth
Design

101
 • Objectives
Session 9 – After the session be able to describe:
• Purpose and main activities of the design phase
SAT in depth - • Key elements in developing training objectives
• Just-in-time training, initial and continuing training
Design programmes
• Training settings and approach
• Test items and constructing test
• Estimation of resource requirements
• Courses that apply to more than one job
• Outputs and lessons learned from the design phase

009_00 Describe the purpose of the design phase in the ADDIE cycle to determine
training objectives, test items to assess achievement of training objectives organized
into a training programme

102
 Design phase

Purpose:
Develop clear training objectives organized in a
logical, pedagogical sense for effective learning in
modules, courses and training programs

Inputs:
• Procedures that define the design process
• Entry level requirements
• Inputs from evaluations
• Tasks and competencies selected for training
• Conditions and standards of task
performance
• KSAs needed for job performance
• Data from existing relevant training
programmes
• Availability of data required for design

009_00 Describe the purpose of the design phase in the ADDIE cycle to determine
training objectives, test items to assess achievement of training objectives organized
into a training programme

Purpose:
Develop clear training objectives organized in a logical, pedagogical sense for
effective learning in modules, courses and training programs

Inputs:
Procedures that define the design process
Entry level requirements
Inputs from evaluations
Tasks and competencies selected for training
Conditions and standards of task performance
KSAs needed for job performance
Data from existing relevant training programmes
Availability of data required for design

103
Design phase

009_00 Describe the purpose of the design phase in the ADDIE cycle to determine
training objectives, test items to assess achievement of training objectives organized
into a training programme
009_01 Describe the purpose, essentials and inputs of/to the design phase

The main activities performed during the design phase are:

• Development of training objectives
• Selection of suitable training settings
• Organization of the training objectives into meaningful units/modules and
• Schedule these units/modules into training programme
• Development of test items
• Development of the training programme description for initial training,
and a framework for continuing training
• Development of a work plan including compiled evaluation activities and
estimation of resources required

104
 Training objectives

•what the
trainee will
•the basis for
questions and S • Specific
know and be tests
able to do
after training
M • Measurable/observable
Define Form
• Attainable for target
A audience within specified
time and conditions

Guides R • Relevant and results

oriented

•the training • Targeted to the learner and

program
content T to the desired level of
learning

009_00 Describe the purpose of the design phase in the ADDIE cycle to determine
training objectives, test items to assess achievement of training objectives organized
into a training programme
009_02 Describe how to develop training objectives

The first step within the design phase is to convert the selected training tasks
or competencies and associated KSAs into measurable training objectives.

Training objectives:
• Define what the trainee will know and be able to do after training
• Form the basis for questions and tests
• Guides the training program content

Generally, a training objective is a statement that specifies measurable

behaviour that a trainee should exhibit after instruction.
Training objectives should be:
• Specific
• Measurable/observable
• Attainable for target audience within scheduled time and specified
conditions
• Relevant and results oriented
• Targeted to the learner and to the desired level of learning.

105
Training objectives are
developed with:

Data from analisys phase: Three components of

• Need to know, do and understand to training objectives: (Also identified in ”Task analysis”)
performe? (KSA)
Conditions: under which the action will take place

Task Action: a statement of behaviour the trainee must exhibit

Examples of conditions:
• Measuring
radiation levels - During normal conditions Standards: of satisfactory
Examples of action statement: performance
with radiation - Given __ failure
meters Affective domain:
- During shift turnover - Prevent __; Examples of standards:
- Using the __ tools - Exhibit __; – Within 20 minutes;
Competences
• Analyze - Using __ procedure Cognitive domain: – In full accordance with the __
radiation
measures over - Using __ survey results procedure;
time and - State the purpose of the __ – Without error;
localization - From memory system
– Without entering a limiting
- Through observation - Predict the consequences of condition of operation;
__ component failure on the __ – __ components of __;
system
– Within 5 mm.
Psychomotor domain:
- Perform __;
- Assemble __;
KSA - Practice __.
• Knowledge about radiation types
• Able to use equipment to detect and
mesaure radiation
• Understanding of radiation hazards

009_00 Describe the purpose of the design phase in the ADDIE cycle to determine
training objectives, test items to assess achievement of training objectives organized
into a training programme
009_03 Explain components and classification of training objectives
009_04 Explain enabling and terminal objectives

Use given KSAs, conditions, actions and standards

Training objectives normally have three components:

• Conditions: under which the action will take place
• Action: a statement of behaviour the trainee must exhibit
• Standards: of satisfactory performance

106
Design phase

009_00 Describe the purpose of the design phase in the ADDIE cycle to determine
training objectives, test items to assess achievement of training objectives organized
into a training
009_05 Describe the design of initial and continuing training programmes

Training courses and their instructional units are sometimes referred to as

modules

A Training Programme is a planned and organized set of training activities,

devised to achieve all the specified training objectives

Estimate resources for development and implementation

• In Design phase, estimation of the necessary resources or valuation of
planned resources for development and implementation should be
carried out
• Resources is often estimated before design phase in the process for
planning

107
Design phase

Reference
documentation,
SMEs,
instructors,
instructional
specialists

Estimating
resource
requirments

Facilities,
Training equipment,
material software,
developers funds, time,
etc.

009_00 Describe the purpose of the design phase in the ADDIE cycle to determine
training objectives, test items to assess achievement of training objectives organized
into a training
009_10 Explain estimation of resource requirements

Before the training programme description is approved, the resources

required for the development and implementation of the training programme
should be estimated
It is only at this point in the SAT process that a detailed schedule for the
development of training can be finalized and specific resources identified and
committed for the development of training materials and tools.
Using estimates of resources, a work plan for the development and
implementation of the training programme should be developed and
approved by the facility senior manager. For a newly built nuclear facility and
a newly developed training system a comprehensive plan for the
establishment of the entire training system should be developed and
approved prior to embarking on the establishment of any individual training
programmes.

For the training facilities and tools, the estimates should include the
resources required for:
• Design and construction of a new training centre or new training
premises

108
• Classroom facilities
• Laboratories and workshop facilities
• Mock–ups
• Real facility equipment used in the training centre
• Simulators
• Digital learning technology/e–learning systems
• Audio–visual aids
• Tools and equipment
• Offices and furnishings

108
Design phase
Two perspectives on a training program

009_00 Describe the purpose of the design phase in the ADDIE cycle to determine
training objectives, test items to assess achievement of training objectives organized
into a training
009_05 Describe the design of initial and continuing training programmes

Some examples of training courses:

• Theory/fundamental courses
• Courses on design of the facility, systems, structures and components
• Normal operation
• Abnormal conditions
• Design basis accidents
• Beyond design basis accidents
• In–facility/on–site/on–job training courses (e.g., complex OJT in the field
or in a control room)

109
Design phase

Continuing training

009_00 Describe the purpose of the design phase in the ADDIE cycle to determine
training objectives, test items to assess achievement of training objectives organized
into a training
009_05 Describe the design of initial and continuing training programmes"

Continuing training is designed to maintain and enhance the competences of

the employees to support the safe and reliable operation of a nuclear facility
and personnel performance improvement. FIG. shows graphically how
continuing training relates to the competence development of nuclear facility
personnel.

110
Design phase
Just in time training

Operation Operation

Preparing for
testing and
Operation Outage
maintenance of
equipment

009_00 Describe the purpose of the design phase in the ADDIE cycle to determine
training objectives, test items to assess achievement of training objectives organized
into a training
009_06 Explain just-in-time training

Just–In–time (JIT) training is provided to facility individuals and teams to

prepare them for upcoming evolutions and maintenance activities

Examples of tasks and activities for which JIT training may be considered
appropriate and desirable for operations, engineering and maintenance staff
include:
• Reactor planned shutdown in preparation for a re-fuelling or maintenance
outage;
• Facility start–up following a re-fuelling outage;
• Initial start–up of new or modified equipment;
• Core physics testing;
• Preparing for testing and maintenance of equipment (e.g., safety relief
valves);
• Contingency actions for anticipated severe weather or other
environmental conditions.

111
Design phase

Factors affecting selection of training settings

009_00 Describe the purpose of the design phase in the ADDIE cycle to determine
training objectives, test items to assess achievement of training objectives organized
into a training
009_07 Describe how to select training settings and approach

The training setting is the environment or location in which training takes

place.

The training mode is the way in which learning, or training takes place, for
example in large groups, small groups or individually

The training method is a particular way of providing training instruction such

as:
• Lecture
• Demonstration
• Observation
• Practice
• Walkthrough
• Drill
• Discussion/facilitation
• Oral questioning
• Role playing
• E–learning

112
• Self–study

112
Design phase

Developing test items and constructing tests

Description of the Verb Examples of the Verbs Test type

Skill action verbs Start, shut down Performance test format

Knowledge action verbs Recall, One or more of the written

formats

Identify, List Completion/short–answer

format

Discriminate, Select Multiple choice format

Classify, Relate Matching format
Higher–order verbs Synthesize, Evaluate Essay format

009_00 Describe the purpose of the design phase in the ADDIE cycle to determine
training objectives, test items to assess achievement of training objectives organized
into a training
009_08 Describe the design of test items and constructing test

Example of verbs and test type in “Bloom Levels”

113
 TEST ITEMS EXAMPLE

Fill in the blank A positive displacement pump should be started with its suction valve ________ and its discharge valve ________

Multiple Choice *Select ONE correct answer

The atoms of a chemical element all have the same STEM
(a) Atomic number (Correct choice) ☐
(b) Relative atomic mass (Distractor) ☐
(c) Number of neutrons in the nucleus (Distractor) ☐
(d) Number of nucleons in the nucleus (Distractor) ☐

Matching *Match the alarm (letter in the left column) with its appropriate tone (number in the right column). Responses (numbers)
may be used more than once

Premises responses
(a) Fire (1) Pulse tone
(b) Reactor building evacuation (2) Wailing siren
(c) Site evacuation (3) Steady tone
(4) Bell ring 5 sec / 2 sec
Structured Essay A pressurized water reactor at full power has the following conditions:
☐ Core flow (100%)
☐ Tavg (308.9C)
☐ Core ΔT (15.6C)
☐ Steam Generator ΔT (Tavg–Tsteam) (6.7C)
The reactor is operating at steady–state power with four loops operating. One reactor coolant pump trips, resulting in a three–loop
operation. The rod control system is in automatic mode. The reactor has not tripped.

(a) Sketch graphs of RCS mass flow, reactor power, core ΔT, and steam generator pressure versus time if no operator action is
taken. Use the same time axis for all graphs.

(a) Calculate the final reactor power, Tavg, core ΔT, and steam generator ΔT.

009_00 Describe the purpose of the design phase in the ADDIE cycle to determine
training objectives, test items to assess achievement of training objectives organized
into a training
009_08 Describe the design of test items and constructing test

Four examples.

114
Design phase Training courses that apply to more
than one job

Examples of training courses that may be used

for various job positions are:

• Fundamentals, basic theory and facility

design;
• General employee training (GET);
• Management training;
• Training on regulations;
• Human factor and human performance
related competencies;
• Security issues;
• Administrative or financial issues;
• Advanced technologies used in the
nuclear industry;
• Training on the use of IT tools.

009_00 Describe the purpose of the design phase in the ADDIE cycle to determine
training objectives, test items to assess achievement of training objectives organized
into a training
009_11 Describe training courses that apply to more than one job

Examples of training courses that may be used for various job positions are:
• Fundamentals, basic theory and facility design
• General employee training (GET)
• Management training
• Training on regulations
• Human factor and human performance related competencies
• Security issues
• Administrative or financial issues
• Advanced technologies used in the nuclear industry
• Training on the use of IT tools

The overall approach to the design of training modules used for many job
positions is the development of an applicability matrix using data from the
analysis phase.

115
Design phase

Outputs Lessons learned

• Development of training
objectives and test items
– is one of the most
resource intensive SAT
phases from a
methodological point of
view

• Incorrect development of
training objectives and
test items may negatively
impact on the nuclear
facility safety and overall
performance including
performance of personnel

• The nuclear facility should

have a
strategy/procedure for
review and approval of
data from design phase.

009_00 Describe the purpose of the design phase in the ADDIE cycle to determine
training objectives, test items to assess achievement of training objectives organized
into a training
009_12 Describe outputs and lessons learned from the design phase

• The design phase – in particular, the development of training objectives

and test items – is one of the most resource intensive SAT phases from a
methodological point of view;
• Incorrect development of training objectives and test items may
negatively impact on the nuclear facility safety and overall performance
including performance of personnel;
• Training and assessment standards contain job–related data for measuring
task/ competency performance;
• Selection of training settings considers tasks/competencies, training
objectives and any constraints regarding the availability of instructional
and other resources;
• Training objectives are used to identify training content and satisfactory
trainee performance;
• Training objectives identify observable and measurable trainee action or
behaviour;
• Test items should be appropriate for the training objectives. There is no
one unique way to identify test items for particular training objectives.

116
 Guidelines and experience should be actively employed;
• Training objectives should be compatible with expected entry–level
requirements for the trainees, including expected entry–level skills and
knowledge;
• Training objectives should be organized (grouped and sequenced) to assist
trainees in effective learning, and making the transitions from one skill or
knowledge level to another, to finally master terminal training objectives;
• Pre–tests are often developed to determine trainees’ entry level and to
identify remedial training and exemption from training requirements as
applicable;
• Progress tests are developed to evaluate trainee performance and determine
whether there is a need for additional assistance;
• Post–tests are developed to measure trainees’ satisfactory completion of
training; and serve as one of the main tools for qualification and authorization
of personnel;
• The nuclear facility should have a strategy/procedure how to review and
approval of data from design phase.

116
 • Summary
• To develop clear training objectives organized in a logical,
pedagogical sense for effective learning in modules, courses and
training programs against desired results
• The availability of key personnel including instructor/training
specialist and SMEs (e.g., job incumbents, training staff), trained in
the process, and committed to be involved throughout the
Session 9 completion is essential
• Inputs to the design phase are
SAT in depth • Procedures that define the design process
• Entry level requirements
Design • List of tasks and/or competencies with associated KSAs,
selected for training
• Availability of data required for design, including data from
existing internal or external training programme
• Outputs from the design phase are
• Grouped and sequenced training objectives
• Test items
• Evaluation activities
• Training programme description

009_00 Describe the purpose of the design phase in the ADDIE cycle to determine
training objectives, test items to assess achievement of training objectives organized
into a training

117