Downloaded from [Link].

dk on: dez 30, 2022

The role of error in organizing behaviour

Rasmussen, Jens

Published in:
Quality and Safety in Health Care

Link to article, DOI:

10.1136/qhc.12.5.377

Publication date:
2003

Document Version
Publisher's PDF, also known as Version of record

Link back to DTU Orbit

Citation (APA):
Rasmussen, J. (2003). The role of error in organizing behaviour. Quality and Safety in Health Care, 12(5), 377-
383. [Link]

General rights
Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright
owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

 Users may download and print one copy of any publication from the public portal for the purpose of private study or research.
 You may not further distribute the material or use it for any profit-making activity or commercial gain
 You may freely distribute the URL identifying the publication in the public portal

If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately
and investigate your claim.
 377

CLASSIC PAPER

The role of error in organizing behaviour*

J Rasmussen
.............................................................................................................................

Qual Saf Health Care 2003;12:377–385

During recent years the significance of the concept of reliability analysis is focused on the less well
human error has changed considerably. The reason for structured and stable tasks of diagnosis and con-
tingency planning. Focus of error analysis is
this has partly been an increasing interest of moved back from overt acts to decision functions
psychological research in the analysis of complex real and further on to psychological mechanisms (fig
1).
life phenomena, and partly the changes of modern work It is a remarkable fact that, given a particular
conditions caused by advanced information technology. sequence of human acts, taxonomies of error
Consequently, the topic of the present contribution is not analysis resulting from detailed analysis of actual
cases of incidents and accidents and from
a definition of the concept or a proper taxonomy. psychological laboratory research show definitive
Instead, a review is given of two professional contexts convergent properties.2 When a particular task
sequence can be taken as reference (i.e. a
for which the concept of error is important. Three cases sequence which is functionally constrained by the
of analysis of human–system interaction are reviewed: equipment to be operated or firmly established by
(1) traditional task analysis and human reliability training), a failure-mode-and-effect analysis is a
very feasible approach to identify the hazards
estimation; (2) causal analysis of accidents after the presented by human error. It will be effective dur-
fact, and (3) design of reliable work conditions in ing design to ensure error tolerance, even if quan-
modern sociotechnical systems. It is concluded that titative reliability prediction may not be realistic.3
A necessary precondition is, however, that the
“errors” cannot be studied as a separate category of sequence in which the “error” is analysed can be
behaviour fragments; the object of study should be taken for granted. This is the case only when we
are involved in a local analysis focused on the
cognitive control of behaviour in complex environments. immediate human–machine interface: we then
.......................................................................... try to predict the risk involved in the operation of
some particular technological system of a known
TRADITIONAL TASK ANALYSIS AND design. The acceptable work procedure is identi-
fied from the functional requirements of equip-
HUMAN RELIABILITY ESTIMATION
ment, given a definite goal. This is, as noted, a
Human activities in traditional work environ-
reasonable assumption if the task is repetitive,
ments can be described in terms of repetitive which was the normal case in established
tasks, i.e. sequences of acts in control of some technology. In addition, we are dealing with a
equipment or tool. Manufacturing systems were human link in an extended chain of events; the
normally planned for effective and economic “error” is a link in the chain, in most cases not the
operation over long periods of time. Planned or origin of the course of events. This kind of analy-
normal work sequences had time to settle into sis and, consequently, definition of error is
stable patterns which could be identified during completely inadequate when we are dealing with
design by analysis of the task, to control tools and design or improvement of large scale sociotechni-
equipment, or afterwards by field studies. Since cal systems. In general, we do not have a simple
successful operation during production or a mis- causal trace deflected from its intended course
sion was of fundamental interest, technical and toward one goal. Actually, such a separate trace is
human reliability analysis became important the manifestation of the dynamic flow of events
design tools both for military and high hazard in a complex network involving several goals and
industrial operations. side effects and many side branches. Previous
In this situation, human errors can easily be flows of events along these branches serve to pre-
defined; normative sequences of proper acts are condition the “riverbed” in which the dynamic
available for reference and errors can be identified flow is found.
and recorded. As long as actors enter the proper
sequence at all, errors caused by lack of resources CAUSAL ANALYSIS OF ACCIDENTS AFTER
or proper intention are of minor importance and THE FACT
errors can be studied in terms of their overt In this case, we are analysing an accidental chain
effects (Swain’s Therp method).1 In modern of events upstream from an accident in order to
workplaces people are frequently moved to super- understand why it happened; to find somebody to
....................... visory tasks and decision making. In that case, blame, who did it; or to find out how to improve
the system. We are trying to describe a particular
Correspondence to: course of events and to identify the particular
J Rasmussen, Risø National ................................................. causal trace in which human error is embedded.
Laboratory, P O Box 49,
DK-4000 Roskilde, *This is a reprint of a paper that appeared in Ergonomics Accidents are normally analysed in terms of
Denmark 1990, Volume 33, pages 1185–99. Copyright © 1990 accidental chains of events, i.e. causal representa-
....................... Taylor & Francis Ltd. tions. Since no two accidents will be identical,

[Link]
378 Rasmussen

Behavior shaping
Task
Functional
Usual road Choice of Unusually Loss of
factors of
requirements
system
closed new route steep slope speed
work environment properties
control

External cause, Psychological Decision Overt Consequences

Lorry Loss of
antecedent mechanism function act for system
overloaded route
event activated involved affected operation
control
Figure 1 An illustration of the human involvement in a causal
sequence of events. The event of human error is decomposed to
identify the cognitive task element and the psychological mechanism Fault in Choice of
Collision
involved in the error. At this level of detail, an event in the work usual lorry replacement
context activates a particular psychological mechanism which Brakes
influences the immediate decision task required by the work. A faulty
decision error in turn introduces an error in the overt action
sequence, with unacceptable consequences for the work goal. Two Bad Injury
aspects are essential in the present context. One is that human maintenance
“error” very frequently will be a link in a sequence, not the origin.
Secondly, “errors” can be categorized at different states in the flow. Figure 2 An illustration of a causal explanation of a driving
This representation is well suited for failure-mode-and-effect analysis accident. The flow of behaviour is decomposed into chains of events.
in the interface of technical systems: the various psychological “error Note that only abnormal or unusual events together with violations of
mechanisms” are folded on to the cognitive task from which the rules are included. The normal activities conditioning the path are
effects, in turn, are folded on to functional system properties for not included. Furthermore, decomposition and causal backtracking
evaluation of the acceptability of consequences. stop at events which are taken to be “reasonable explanations”.
Adapted from Leplat and Rasmussen.6

accident analysis will depend on prototypical categories of

causes, events, and consequences.4 An explicit representation tendency to see what is expected; during one period technical
of elements in the physical world makes causal analysis a very faults were in focus as causes of accidents, then human errors
effective technique for identifying and representing accidental predominated, while in the future focus will probably move
conditions. It is, however, important to consider the implicit upstream to designers and managers.
frame of reference of a causal analysis. The perception of stop rules is very important in the control
The behaviour of the complex, real world is a continuous, of causal explanations. Every student knows the relief felt
dynamic flow which can only be explained in causal terms when finding a list of solutions to mathematical problems. Not
after decomposition into discrete events. The concept of a that it gave the path to solution to any great extent, but it gave
causal interaction of events and objects depends on a catego- a clear stop rule for the search for possible mistakes, overseen
rization of human observations and experiences. Perception of preconditions, and calculation errors. The result: hours saved
occurrences as events in causal connection does not depend on and peace of mind. A more professional example to the same
categories which are defined by lists of objective attributes, but point is given by Kuhn.7 He mentions the fact that chemical
on categories which are identified by typical examples, research was able to come up with whole number relations
prototypes.5 This is the case for objects as well as for events. between elements of chemical substances only after the
Everybody knows perfectly well what “a cup” is. To define it acceptance of John Dalton’s chemical atom theory. There had
objectively by a list of attributes that separates cups from jars, been no stop rule for efforts to refine the experimental
vases, and bowls is no trivial problem. It has, for instance, been technique until the acceptance of this theory.
faced in many attempts to design computer programs for pic- Stop rules are not usually formulated explicitly. The search
ture analysis. The problem is that the property of being “a cup” will typically be terminated pragmatically in one of the
is not a feature of an isolated object, but depends on the con- following ways: (a) an event will be accepted as a cause and
text of human needs and experience. The identification of the search terminated if the causal path can no longer be fol-
events in the same way depends on the relationship in which lowed because information is missing; (b) when a familiar,
they appear in a causal statement. An objective definition, abnormal event is found to be a reasonable explanation; or ©
therefore, will be circular. if a cure is available. The dependence of the stop rule upon
In the analysis of accidents, decomposition of the dynamic familiarity and the availability of a cure makes the judgement
flow of changes will normally terminate when a sequence is very dependent upon the role in which a judge finds himself.
found including events which match the prototypes familiar An operator, a supervisor, a designer, and a legal judge may
to the analysis. The resulting explanation will take for granted very likely reach different conclusions.
his or her frame of reference and, in general, only what he or To summarize: identification of accident causes is controlled
she finds to be unusual will be included: the less familiar the by pragmatic, subjective stop rules. These rules depend on the
context, the more detailed the decomposition. By means of the aim of the analysis, i.e. whether the aim is to explain the
analysis, a causal path is found upstream from the accidental course of events, to allocate responsibility and blame, or to
effect. This path will be prepared by resident conditions which identify possible system improvements in order to avoid future
are latent effects of earlier events or acts. Also, the resident accidents.
conditions can be explained by causal back tracking, in this
case branches in the path are found. To explain the accident, Analysis of explanation
these branches are also traced backward until all conditions In an analysis to explain an accident, the backtracking will be
are explained by abnormal, but familiar, events or acts. The continued until a cause is found which is familiar to the ana-
point is: How does the degree of decomposition of the causal lysts. If a technical component fails, a component fault will
explanation and the selection of the side branches depend on only be accepted as the prime cause if the failure of the
the circumstances of the analysis? Another question is: What particular type of component appears to be “as usual”. Further
is the stop rule applied for termination of the search for search will probably be made if the consequences of the fault
causes? Ambiguous and implicit stop rules will make the make the designer’s choice of component quality unreason-
results of analysis very sensitive to the topics discussed in the able, or if a reasonable operator could have terminated the
professional community at any given time (fig 2). There is a effect had he been more alert or been trained better. In such a

[Link]
Role of error in organizing behaviour 379

case, a design or manufacturing error, respectively, or an In conclusion, the choice of stop rules for the analysis of
operator error can be found. accidents is normally left to the subjective judgement of the
In most recent reviews of larger industrial accidents it has analyst, depending heavily on the aim of his analysis. Analyses
been found that human errors played an important role in the made for one purpose may therefore be misleading for other
course of events. Frequently, errors are attributed to operators purposes.
involved in the dynamic flow of events. This can be an effect of
the very nature of the causal explanation. Human error is, DESIGN OF RELIABLE WORK CONDITIONS AND
particularly at present, familiar to analysts: to err is human, SOCIOTECHNICAL SYSTEMS
and highly skilled people will frequently depart from norma- Modern work conditions
tive procedures, as we will see subsequently. A number of problems are met when attempts are made to
Analysis for allocation of responsibility improve the safety of sociotechnical systems from analyses
In order to allocate responsibility, the stop rule of the tied to particular paths of accidental events. This is due to the
backward tracing of events will be to identify a person who fact that each path is a particular token shaped by higher
has made an error and at the same time was “in control” of his order relational structures. If changes are introduced to
or her acts. The very nature of the causal explanation will remove the conditions for the existence of a particular link in
focus attention on people directly and dynamically involved in the chain, odds are that this particular situation will never
the flow of abnormal events. This is unfortunate because they occur again. We should be fighting types of accident causation,
may very well be in a situation where they do not have “con- not these individual tokens. Only in the immediate interface
trol”. Traditionally, a person is not considered responsible if with technical systems is human behaviour constrained in a
physically forced to act by another person or when subject to way that makes the chain of events reasonably predictable.
disorders such as epileptic attacks. In such cases, acts are The farther away from the technical core we are, the more
involuntary8 9 from a judgement based on physical or physio- degrees of freedom agents will have in their mode of
logical factors. It is, however, a question as to whether cogni- behaviour. Consequently, the less certain is the reference in
tive psychological factors also should be taken more into terms of normal or proper behaviour for judging “errors”. This
account when judging responsibility. Inadequate response of problem is becoming increasingly important as modern
operators to unfamiliar events depends very much on the con- manufacturing systems and organizations are forced to
ditioning taking place during normal work. This problem also respond to increasingly dynamic market requirements,
raises the question of the nature of human error. The technological innovations, and legal constraints.
behaviour of operators is conditioned by the conscious Given this situation, improvements of safety features of a
decisions made by work planners or managers. They will be sociotechnical system depend on a global analysis: no longer
more “responsible” than an operator in the dynamic flow of can we assume the time course of human behaviour to be pre-
events. However, their decisions may not be considered during dictable. Tasks will be formed for the occasion, and design for
a causal analysis after an accident because they are “normal improvements must be based on attempts to find means of
events” which are not usually represented in an accident control at higher levels than that of particular task procedures.
analysis. Furthermore, they can be missed in analysis because If, for instance, sociotechnical systems have features of adap-
they are to be found in a conditioning side branch of the causal tation and self-organization, changes which aim to improve
tree, not in the path involved in the dynamic flow. safety at the individual task level might well be compared with
Present technological development toward high hazard attempts to control the temperature in a room with a thermo-
systems requires a very careful consideration by designers of stat controlled heater by opening the window. In other words,
the effects of “human errors” which are commonplace in nor- it is not sensible to try to change the performance of a
mal daily activities, but unacceptable in large scale systems. feedback system by alterations inside the loop; you have to
There is considerable danger that systematic traps can be identify mechanisms that are sensitive, i.e. related to the con-
arranged for people in the dynamic course of events. The trol reference itself.
present concept of “responsibility” should be reconsidered Some basic high level features of “human error” in a flexible
from a cognitive point of view, as should the ambiguity of stop sociotechnical system are related to the dependence of human
rules in causal analysis. performance on features such as: (1) learning and adaptation;
(2) conflicts among cognitive control structures; (3) resource
Analysis for system improvements limitations; and finally, (4) stochastic variability. An attempt
Analysis for therapeutic purpose, i.e. for system improvement, to develop guidelines for the design of human-work interfaces
will require a different focus with respect to selection of the has been presented elsewhere.10
causal network and of the stop rule. The stop rule will now be
related to the question of whether an effective cure is known. Human adaptation
Frequently, cure will be associated with events perceived to be In all work situations constraints are found which must be
root causes. In general, however, the effects of accidental respected in order to perform satisfactorily. There are,
courses of events can be avoided by breaking or blocking any however, also many degrees of freedom which have to be
link in the causal tree or its conditioning branches. resolved at the worker’s discretion. In stable work conditions,
Explanatory descriptions of accidents are, as mentioned, know-how will develop which represents prior decisions and
focused on the unusual events. However, the path can also be choice and the perceived degrees of behavioural freedom will
broken by changing normal events and functions involved. ultimately be very limited, i.e. “normal ways” of doing this will
The decomposition of the flow of events, therefore, should not emerge, and the process of adaptation will no longer confuse
focus solely on unusual events, but should also include normal the concept of error. By contrast, in modern, flexible, and
activities. dynamic work conditions, the immediate degrees of freedom
The aim is to find conditions sensitive to improvements. will have to be continuously resolved. This implies that effec-
Improvements imply that some person in the system makes tive work performance includes continuous awareness of the
decisions differently in the future. How do we systematically available degrees of freedom together with effective strategies
identify persons and decisions in a (normal) situation where it for making choices, ahead of the task of controlling the chosen
would be psychologically feasible to ask for a change in path to a goal. This changes the concept of error in a very fun-
behaviour, when reports from accidents focus only on the flow damental way.
of unusual events? An approach to such an analysis for The behaviour in work of individuals (and, consequently,
improving safety has been discussed elsewhere.6 also of organizations) is, by definition, oriented towards the

[Link]
380 Rasmussen

Means-end relations
Boundary of individual
resource profile

Goals and values, Work, income, intellectual and

Boundary of acceptable constraints esthetic pleasure;
state of affairs social family relations

Space of possibilities; degrees of

Priority mearsures, Expenses, joy probability of
freedom to be resolved according
flow of money, material, success and delays
to subjective preferences
information

Boundary of available Work function: transport

means of work General functions
Family function: shopping

Physical processes Train operation, schedules, space for

Figure 3 Human behaviour is governed by constraints which must
in work and reading; taxi: time spent, traffic
be respected by the actors for the work performance to be
equipment jams, price; car: traffic, time, price
successful. Identification of such constraints will specify the “space”
in which the human can navigate freely. Violation of the constraints
will be considered human error or task violation in the usual sense. Material objects, Train, taxi, private car

For successful performance, humans have to navigate between two tools, buildings, etc. routes, distance, scenery;

boundaries of constraints. One boundary is given by the control topography shops and locations, etc.
requirements posed by the system. The other constraining boundary
is given by the human resource profile which depends on individual Figure 4 An example: the activities involved in going to work. The
characteristics such as competence, mental capacity, physical work given constraints are related to the location, the time of arrival,
strength, etc. Navigation within the envelope specified by these and the probability of delays. Constraints in means are defined by
boundaries will depend on subjective criteria for choice, such as aim the transport alternatives, i.e. to take the tube, a taxi, or to drive by
to save time, to spare memory load, to have fun, to explore new yourself. The subjective process criterion determining your choice
land, etc. depends on economy, your husband or wife’s request to bring some
grocery and, maybe, consideration of the time spent, the likelihood
of traffic jams. Given the decision to drive by yourself, the choice of
requirements of the work environment as perceived by the route depends on the secondary task of shopping, of your joy with a
individual. Work requirements, what should be done, will nor- particular scenery, and the traffic density. Finally, en route, the
mally be perceived in terms of control of the state of affairs in speed you choose depends on traffic given constraints, on formal
the work environment according to a goal, i.e. why it should be conditions such as speed limits or your husband or wife’s anxiety,
done. How these changes are made is, to a certain degree, at and ultimately “sporty” criteria related to your driving skill, i.e. to
the discretion of the agent. drive fast with smooth gear changes.
The alternative acceptable work activities, how to work, will
be shaped by the work environment which defines the such as time spent, cognitive strain, joy, cost of failure, etc. In
boundaries of the range of possibilities, i.e. acceptable work general, the freedom to choose work strategy is very important
strategies. This range of possibilities will be further bounded as a means to resolve resource-demand conflicts met during
by the resource profile of the particular agent in terms of tools performance.
available, knowledge (competence), information about state Modelling work activity from this point of view depends on
of affairs, and processing capacity. The presence of alternatives identification of the range of acceptable and possible work
for action depends on a many-to-many mapping between strategies (i.e. prototypical sets of behaviour sequences), the
means and ends present in the work situation as perceived by human resource profile, and the subjective criteria governing
the individual; in general, several functions can serve the the resolution of the remaining degrees of freedom in
individual goals and each of the functions can be imple- different work scenarios. Some work requirements are explicit
mented by different tools and physical processes. If this was and discrete, with specified limits of acceptance. Other
not the case, the work environment would be totally predeter- requirements are formulated as optimizing criteria serving to
mined and there would be no need for human choice or deci- resolve ambiguity in goal specification, such as the request to
sion (figs 3 and 4). reach a solution which is as cheap or as safe as possible. Such
Within the area of acceptable work performance, between product criteria, together with the subjective process criteria,
the boundaries defined by the work requirements on one side will necessarily lead to an adaptive behaviour seeking to opti-
and the individual resource profile on the other, considerable mize performance according to the criteria, along with evolu-
degrees of freedom are still left for the individual to choose tion of training and expertise (fig 5).
among strategies and to implement them in particular
sequences of behaviour. These degrees of freedom must be Adaptation, self-organization, and error
eliminated by the choice of an agent to finally enter a particu- It follows directly from this discussion that the structuring of
lar course of action. The different ways to accomplish work can work processes through on-the-job training by an individual
be categorized in terms of strategies, defined as types of will be a self-organizing, evolutionary process, simply because
behavioural sequences which are similar in some well defined an optimizing search is the only way in which the large
aspects, such as the physical process applied in work and the number of degrees of freedom in a complex situation can be
related tools, or, for mental strategies, the underlying kind of resolved. The basic synchronization to the work requirements
mental representation and the level of interpretation of can be based on procedures learned from an instructor or a
perceived information. In any particular situation-dependent more experienced colleague, or it can sometimes be planned
exemplar of actual performance, a token will emerge which is by the individual in a knowledge based mode of reasoning by
an implementation of the chosen strategy under the influence means of mental experiments. From here, the smoothness and
of the complexity of detail in the environment. The particular speed characterizing high professional skill, together with a
token of performance will be unique and impossible to large repertoire of heuristic know-how rules, will evolve
predict, whereas the strategy chosen will, in principle, be pre- through an adaptation process in which “errors” are unavoid-
dictable. This choice made by individual agents depends on able side effects of the exploration of the boundaries of
subjective performance criteria related to the process of work, acceptable performance. During this adaptation, performance

[Link]
Role of error in organizing behaviour 381

Cognitive strain depends on adaptation governed by subjective process criteria.

Opportunities for experiments are necessary to find shortcuts
Resource and to identify convenient and reliable cues for action without
requirements of
analytical diagnosis. In other words, effective, professional
diagnostic
strategies Functional performance depends on empirical correlation of cues to suc-
reasoning cessful acts. Humans typically seek the way of least effort.
Therefore, it can be expected that no more information will be
Search used than is necessary for discrimination among the perceived
alternative for action in any particular situation. This implies
Prior Time that the choice is “under-specified”11 outside that situation.
experience spent When situations change, e.g. due to disturbances or faults in
the system to be controlled, reliance on the usual cues which
are no longer valid will cause an error due to inappropriate
“expectations”. In this way, traps causing systematic mistakes
can be designed into the system. Two types of errors are related
to this kind of adaptation: firstly, the effect of the test of a
Basic knowledge
hypothesis of salient cues and action which turn out to be
Figure 5 An illustration of the different resource requirements of negative and, secondly, the effects of acts chosen from famil-
different mental strategies. This difference makes a shift in strategy iar and tested cues when a change in system conditions makes
when faced with difficulties in a task, an effective way to navigate the perceived set of alternatives unreliable.
along the path of least effort, a very popular strategy in skilled
performance to adapt behaviour to immediate work situation. An example in which local adaptation is in conflict with
delayed and conditional effects is working instructions which
take into consideration the possible presence of abnormal
will be optimized according to the individual’s subjective conditions that will make certain orders of actions unaccept-
process criteria, within the boundary of his individual able. The instruction to this effect prescribes a certain
resources. This complex adaptation of performance to work sequence. If this prescribed order is in conflict with the actor’s
requirements, eliminating the necessity of continuous choice, immediate process criteria, modification of the prescribed
will result in stereotype practices depending on the individual
procedure is very likely and will have no adverse effect in the
performance criteria of the agents. These criteria will be
daily routine. (If, for instance, an actor has to move back and
significantly influenced by the social norms and culture of the
forth between several distant locations because that sequence
group and organization.
is safer under certain infrequent risky conditions, his or her
Conflict will probably be found between global work goals
process criterion will rapidly teach him or her to group actions
and the effect of local adaptation according to subjective proc-
at the same location together, because this change in the pro-
ess criteria. Unfortunately, the perception of process quality
can be immediate and unconditional, while the effect on cedure will have no visible effect under normal circum-
product quality of the choice of actor can be considerably stances.)
delayed, obscure and frequently conditional with respect to Even within an established effective sequence of actions,
many other factors. adaptation of the patterns of movements will occur according
In a first encounter, when representation of work con- to subconscious perception of certain process qualities. In a
straints is not present in the form of instructions from an manual skill, fine-tuning depends upon a continuous updat-
experienced colleague or a teacher, and know-how from ing of automated patterns of movement to the temporal and
previous experiences is not available, the constraints of the spatial features of the task environment. If the optimization
work have to be explored in a knowledge based mode from criteria are speed and smoothness, adaptation can only be
explicit consideration of the actual goal and a functional constrained by the occasional experience gained when
understanding of the relational structure of the work content. crossing the tolerance limits, i.e. by the experience of errors or
For such initial exploration as well as for problem solving dur- near errors (speed–accuracy trade off). Some errors therefore
ing unusual task conditions, opportunities for tests of hypoth- have a function in maintaining a skill at its proper level, and
eses and trial-and-error learning are important. It is typically they cannot be considered a separable category of events in a
expected that qualified personnel such as process operators causal chain because they are integral parts of a feedback loop.
check their diagnostic hypotheses conceptually—by thought Another effect of increasing skill is the evolution of
experiments—before actual operations if acts are likely to be increasingly long and complex patterns of movements which
irreversible and risky. This appears, however, to be an unreal- could run off without conscious control. During such lengthy
istic assumption, since it may be tempting to test a hypothesis automated patterns, attention is directed towards reviews of
on the physical work environment itself in order to avoid the past experience or planning of future needs (fig 6) and
strain and uncertainty related to unsupported reasoning in a performance is sensitive to interference, i.e. capture from very
complex causal net. For such a task, a designer is supplied with familiar cues.
effective tools such as experimental set-ups, simulation When delayed or conditional global effects of behaviour are
programs, and computational aids, whereas the operator has possible, feedback correction and control of the local
only his or her head and the plant itself. In the actual adaptation is not possible, and adaptation is controlled by an
situation, no explicit stop rule exists to guide the termination evolutionary “survival of the fittest” work process. In order to
of conceptual analysis and the start of action. This means that compete effectively with the effect of the local process criteria,
the definition of error, as seen from the situation of a decision the perception of fitness of such stored procedures must be
maker, is very arbitrary. Acts which are quite rational and maintained in another way (e.g. by artificial reinforcement or,
important during the search for information and tests of preferably, by rearranging the environment to include the glo-
hypotheses may appear to be unacceptable mistakes with bal requirements in the local criteria). Otherwise, simple decay
hindsight, without access to the details of the situation. of memory of stored work rules (decay is, in effect, necessary
Even if a human actor is “synchronized” to the basic for adaptation to changing requirements from a work
requirements of work by effective procedures, there will be environment) will necessarily require a repeated experience of
ample opportunities for modification of such procedures. the conflict in order to maintain proper adaptation to charac-
Development of expert know-how and rules-of-thumb teristics of the environment.

[Link]
382 Rasmussen

KNOWLEDGE- Planning in terms of OFF-LINE EVALUATION

BASED DOMAIN functional reasoning by AND PLANNING

means of symbolic

model:

“As Can Be”

Achronic
RULE-BASED Planning in terms of Attention on cue

DOMAIN recall of past and classification and

rehearsal of future, choice of action

predicted scenarios: alternatives

“As Has Been

and May Be” Synchronic
Diachronic “As Is”
SKILL-BASED Synchronous “As Is”
DOMAIN

Data driven chaining of sub-routines with interrupt to

conscious, rule-based choice in case of ambiguity or ON-LINE, REAL

deviation from current state of the internal world model TIME OPERATION

Figure 6 An illustration of the complex interaction between the different levels of cognitive control. Tasks are frequently analysed in terms of
sequences of separate acts. In general, however, control of several acts takes place concurrently. At the level of skilled sensory motor control,
activity is like a continuous dynamic interaction with the environment. Attention, on the other hand, is scanning across time and activities in
order to analyse past performance, monitor current activity, and plan for foreseen future requirements. In this way, the internal dynamic world
model is being prepared for oncoming demands, and the related cues and rules are rehearsed and modified to match predicted requirements,
and symbolic reasoning is used to understand responses from the environment and to prepare rules for foreseen but unfamiliar situations.
Attention may not always be focused on current activities, and different levels may simultaneously be involved in the control of different tasks,
related to different time slots, in a time sharing or in a parallel processing mode.

The structure of cooperative work Coordination of cooperative work

So far, the discussion has been focused on the individual For concerted work activity, the different processes and func-
adaptation of work strategies to task requirements. In general, tions of work within the various levels of the mean-ends space
however, several people will be active in a work environment, of a work domain will be allocated among several individuals.
and the allocation or acceptance of the roles of individuals will Often, coordination will be allocated among other individuals
evolve in a self-organizing mode according to local criteria and than those directly performing the functions to be coordi-
within the constraints of externally imposed allocation struc- nated. This is the case in all hierarchic organizations. In effect,
tures. Such constraints on the evolutionary allocation can boundaries are found between roles at different levels in the
have their origin in work requirements as well as in human hierarchical control structure, as well as among roles within
resource limitations. these levels.

Role allocation
Some constraints on work allocation originate in the work COMMUNICATION CONVENTIONS
domain. Actions can, for instance, be required simultaneously FORM OF CO-CORD Values and intentions
in separate locations; or work can require competence which is controlled by E supplying criteria for
dependent on more than one profession. Such conditions will or by one or more system and individual
of Group ABCD
limit the extent to which allocation can be dynamically
adapted to the preference of the involved individuals. In some
LEVEL OF SOCIAL CONTROL
cases, however, constraints are rather lenient and will not be CONTENT OF CO-ORD Formal constraints
respected strictly during adaptation (e.g. the boundaries AGENTS:
B on coordination
C
between activities which have been assigned to members of A
D
Resource constraints
different unions by labour market agreements). In other cases, of individuals
constraints are effectively reinforced, as, for example, when LEVEL OF WORK CO-ORDINATION
CONTROL
performance is governed by strict quality control standards, as
is the case for manufacturing according to machining specifi-
Functional constraints
from work content
cations, or in financial operations with strict legal control. In
most cases, however, boundaries among the roles allocated the
individual actors are continuously adjusted according to the LEVEL OF WORK CONTROL & ALLOCATION
requirements of the immediate work situation.
As was the case for the choice among alternative work Figure 7 An illustration of the coordination of cooperative work. At
strategies, the dynamic shifting of boundaries among allo- the level of work, a dynamically changing allocation to individuals is
governed by criteria such as sharing load, minimizing
cated rules will be used to resolve resource demand conflicts communication, individual interest, etc. At the level of coordination,
and to match performance to individual preferences. The sub- the content of communication necessary for concerted action is
jective criteria active in this adaptation will be very situation- specified by the work content and the actual role allocation. In this
dependent and directly related to the particular work process, way, the work organization is dynamically shaped bottom up.
such as perception of differences in workload among Management practice and social values define rules of conduct, i.e.
colleagues, the amount of communication necessary among the form of the coordination, and therefore are shaping the social
organization top down. In addition, formal constraints such as laws,
agents for coordination, subjective preferences for certain regulations, and union agreements add constraints on allocation and
activities, etc. This adaptation of role allocation and coordina- coordination “side in”. Within the boundaries defined in this way,
tion to work requirements during normal conditions will there is plenty of room for adaptation guided by subjective criteria.
endanger functioning during exceptional situations. Adapted from Rasmussen.12

[Link]
Role of error in organizing behaviour 383

The basic structure of the allocation depends on the depends on the introduction of locally visible boundaries of
functional requirements of the work content, such as the acceptable adaptation and the introduction of related control
topographic location of work items, the workload related to mechanisms. What does this mean in terms of organizational
certain functions, the timing required between functions in structures? What kind of top down influence from “manage-
different places, and the time frames to consider in coordina- ment culture” and bottom up technological constraints can be
tion at the various levels. In other words, technology shapes used to guide and limit adaptation? How can we model and
organizations bottom up by imposing strict constraints on predict the evolution of organizational structure?
allocation of functions to groups and individuals. In many
domains, in particular in tightly coupled technical domains CONCLUSION
like manufacturing, process control, etc. strict control and Work in modern “high tech” societies calls for a reconsidera-
timing requirements can be explicitly formulated from an tion of the notion of human error: research should be focused
analysis of the work requirements (fig 7). on a general understanding of human behaviour and social
Within the allocation and coordination constraints imposed interaction in cognitive terms in complex, dynamic environ-
by the work content, there are many degrees of freedom to ments, not on fragments of behaviour called “error”. This
arrange the role allocation and to structure the way in which approach has similarities to the “risk homeostasis” theories of
coordination is brought about. Additional formal constraints traffic safety, with the reservation that the controlling mecha-
on allocation can originate in legal requirements (authoriza- nisms are adaptation in a wider sense than control governed
tion, etc.), agreements (union boundaries), regulations (qual- by criteria related to risk.
ity assurance standards) and rules of conduct (military).
REFERENCES
System reliability and safety 1 Swain AD, Guttmann HE. Handbook of human reliability analysis with
The dynamic adaptation to immediate work requirements, emphasis on nuclear power plant applications. NUREG/CR 1278.
both of individual performance and of the allocation between Albuquerque, NM: Sandia Laboratories, 1983.
2 Rasmussen J. Human error mechanisms in complex work environments.
individuals, will probably create a very high degree of reliabil- Reliability Engineering System Safety 1983;22:155–69.
ity as long as the interaction is transparent (i.e. critical aspects 3 Rasmussen J. Human errors: a taxonomy for describing human
are visible without excessive delay), and individual process malfunction in industrial installations. J Occup Accidents 1982;4:311–3.
criteria are not in conflict with, or are not overriding, critical 4 Rasmussen J. Coping safely with complex systems. Invited paper for
Annual Meeting of the American Association for Advancement of
product criteria. Science, Boston, February 1988.
Under certain conditions, however, self-organizing and 5 Rosch E. Human categorization. In: Warren N, ed. Advances in
adaptive features will necessarily lead to “catastrophic” cross-cultural psychology. New York: Halsted Press, 1975.
6 Leplat J, Rasmussen J. Analysis of human errors in industrial incidents
system behaviour unless certain organizational criteria are and accidents for the improvement of work safety. Accident Anal Prevent
met. Adaptation will normally be governed by local criteria, 1984;16:77–88.
related to an individual’s perception of process qualities in 7 Kuhn T. The structure of scientific revolutions. Chicago: University of
Chicago Press, 1962.
order to resolve the perceived degrees of freedom in the 8 Fitzgerald PJ. Voluntary and involuntary acts. In: Guest AC, ed. Oxford
immediate situation. Some critical product criteria (e.g. essays in jurisprudence. Oxford: Clarendon Press; reprinted in White AR,
safety) are conditionally related to higher level combination or ed. The philosophy of action. Oxford: Oxford University Press,
1961:120–44.
coincidence of effects of several activities, allocated among 9 Feinberg F. Action and responsibility. In: Black M, ed. Philosophy in
different agents and, probably, in different time slots. The vio- America. London: Allen & Unwin, 1965; reprinted in White AR, ed. The
lation of such high level, conditional criteria cannot be moni- philosophy of action. Oxford: Oxford University Press, 95–120.
10 Rasmussen J, Vicente K. Cognitive control of human activities and
tored and detected at the local criterion level, and monitoring errors: implications for ecological interface design. Invited paper for the
by their ultimate criterion effect will be unacceptably delayed. International Conference on Event Perception and Action, Trieste, Italy,
Catastrophic effects of adaptation can be avoided only if local August 1987.
activities are tightly monitored with reference to a prediction 11 Reason J. Cognitive under-specification: its varieties and consequences.
In: Baars B, ed. The psychology of error: a window on the mind. New
of their role in the ultimate conditional effect, i.e. the bounda- York: Plenum Press, 1986.
ries at the local activities are necessarily defined by normal 12 Rasmussen J. Modelling distributed decision-making. Position paper for
prescriptions, not active functional conditions. International Workshop on New Technology and Distributed
Decision-Making, Bad Homburg, May 1988.
This feature of adaptation to local work requirements prob- 13 Rasmussen J. Safety control and risk management: topics for
ably constitutes the fallacy of the defence-in-depth design cross-disciplinary research and development. Invited keynote presentation
principle normally applied in high risk industries.13 In systems for the International Conference on Preventing Major Chemical and
Related Accidents, in Institution of Chemical Engineers, Publication Series
designed according to this principle, an accident is dependent 110. New York: Hemisphere Publishing Corporation, 1988.
on simultaneous violation of several lines of defence: an
operational disturbance (technical fault or operator error)
must coincide with a latent faulty maintenance condition in
.................. COMMENTARY ..................
protective systems, with inadequacies in protective barriers,
with inadequate control of the location of people close to the
installation, etc. The activities threatening the various HUMAN ERROR
conditions normally belong to different branches of the
organization. The presence of a potentially catastrophic
combination of the effects of local adaptation to performance In the 1970s and 1980s there was great interest among applied
criteria can only be detected at a level in the organization with psychologists and systems reliability engineers in analysing
the proper overview. However, at this level in the control hier- accidents and “near miss” incidents in large scale systems
archy (organization), the required understanding of condi- where public safety was a primary concern. Efforts to define
tionally dangerous relations cannot be maintained in the long and develop taxonomies of human error were motivated by
term because the required functional and technical knowl- the meltdown at the Three Mile Island power plant near Har-
edge is foreign to the normal management tasks at this level. risbug, PA, by the nuclear plant accident at Chernobyl in the
The conclusion of this discussion is that catastrophic system Soviet Union, by the poison gas release at Bhopal, India, and
breakdown is a normal feature of systems which have by aviation’s most deadly crash of two 747 aircraft at Tenerife
self-organizing features and, at the same time, depend on pro- in the Canary Islands. Key to these efforts were the contribu-
tection against rare combinations of conditions which are tions of Professor Jens Rasmussen of the Riso Energy Labora-
individually affected by adaptation. Safety in such systems tory and the University of Copenhagen in Denmark.1–3 Riso

[Link]