Five Principles 1

To be published in: Clark, R. E. & Feldon, D. F. (In press for 2005). Five common but
questionable principles of multimedia learning. In Mayer, R. (Ed.) Cambridge Handbook
of Multimedia Learning. Cambridge: Cambridge University Press.

Five Common but Questionable Principles of Multimedia Learning1

Richard E. Clark
Rossier School of Education
University of Southern California

and

David F. Feldon
Graduate School of Education & Information Sciences
University of California at Los Angeles

1
Parts of the discussion in this chapter have been summarized from two previous
manuscripts including: Choi, S. and Clark, R. E. (April 2004) Five suggestions for the
design of experiments on the effects of animated pedagogical agents. Symposium paper
presented at the American Educational Research Association Annual Convention in San
Diego California; and Clark, R. E. (2003) Research on Web-based Learning: A half-full
glass. In Bruning, R. Horn, C. A., & PytlikZillig, L. M. (Eds.) Web-Based Learning:
What do we know? Where do we go? Greenwich, Conn: Information Age Publishers, 1-
22.
Five Principles 2

Principle: A basic generalization that is accepted as true and that can be used
as a basis for reasoning or conduct ([Link] Dictionary)

Abstract

This chapter describes five commonly held principles about multimedia learning that are
not supported by research and suggests alternative generalizations that are more firmly
based on existing studies. The questionable beliefs include the expectations that
multimedia instruction: 1) yields more learning than live instruction or older media; 2) is
more motivating than other instructional delivery options; 3) provides animated
pedagogical agents that aid learning; 4) accommodates different learning styles and so
maximizes learning for more students; and 5) facilitates student managed constructivist
and discovery approaches that are beneficial to learning.
Five Principles 3

Introduction
Multimedia instruction is one of the current examples of a new area of
instructional research and practice that has generated a considerable amount of
excitement. Like other new areas, its early advocates begin with a set of assumptions
about the learning and access problems it will solve and the opportunities it affords (see
for example a report by the American Society for Training and Development, 2001). The
goal of this chapter is to examine the early expectations about multimedia benefits that
seem so intuitively correct that advocates may not have carefully examined research
evidence for them. If these implicit assumptions are incorrect we may unintentionally be
using them as the basis for designing multimedia instruction that does not support
learning or enhance motivation. Even when easily available research findings contradict
widely shared beliefs about benefits, it is tempting to ignore the research by assuming,
without careful analysis, that the multimedia instruction has been poorly designed.
Definition of multimedia. So many different definitions of multimedia have been
offered (see for example Clark, 2001) that it is important at the start of this discussion to
clearly specify what is being discussed. “Instructional media” generally refers to any
vehicle for presenting or delivering instruction. Examples of these vehicles usually refer
to computers, books, television, radio, newspapers, and people. “Multimedia” usually
refers to the capacity of computers to provide real-time representations of nearly all
existing media and sensory modes of instruction. Sensory modes are distinguished from
media because they relate to the sensory format of information so that it is compatible
with one of the five senses. Visual and aural forms of information can be provided by a
variety of media whereas taste, smell, and texture representations in media are very
limited. Multimedia instruction is most often offered at a “distance” from live teachers
and so is occasionally referred to as “distance education”. One of the issues raised in this
chapter is that the impressive breadth of multimedia formats for instruction and learning
may invite a confounding of the specific factors that influence (or fail to influence)
learning and motivation for different people and different learning tasks.
Chapter goals. This chapter examines the research evidence for five of the
implicit assumptions one finds in much of the current literature on multimedia
instruction. Each of these assumptions seem to be so widely shared, that they have taken
on the mantle of “principles” that guide the design of instruction and research on
multimedia instruction. Yet each of these beliefs has been examined by a body of well-
designed research and found to either be incorrect or only to apply in a very limited set of
circumstances. The goal of this chapter is to provide a brief survey of some of the
research and related analysis that challenge each of the five mistaken principles. In each
case, the discussion will provide an alternative generalization that seems warranted, given
the current research. The discussion begins with the most dominant and perhaps the most
erroneous multimedia assumption, that learning benefits are greater from multimedia than
from other instructional media.
Principle #1: Multimedia Instruction Produces More Learning than “Live” Instruction or
Older Media.
There is no credible evidence of learning benefits from any medium or
combination of media that cannot be explained by other, non multimedia factors (Clark,
2001; Clark & Salomon, 1986; Mielke, 1968; Salomon, 1984; Schramm, 1977). Even the
critics of this conclusion, for example, Robert Kozma (1994), have acknowledged that no
Five Principles 4

evidence exists to support the argument that media has influenced learning in past
research. Critics of the “no learning from media” view who are familiar with the
research, hope that multimedia will provide unique forms of influence on learning in the
future. Yet it appears that this optimistic hope is swimming upstream against a
considerable body of evidence to the contrary extending back over 75 years (Mielke,
1968; Clark, 2001).
The capacity of multimedia is broad and inclusive. It even permits us to provide
presentations by human instructors that have been “recorded” on video and presented on
a computer screen as well as all instructional methods, including interactivity between
instruction and learner (for example, feedback to the learner on their progress or
answering questions posed by learners as they progress through instruction), the
providing of examples in the form of simulations or models, and other methods of
teaching that have been found to influence learning. In order to fully understand the
impact of multimedia on learning and motivation, it is important to separate it from the
instructional methods multimedia can present and the sensory modality (visual, aural,
olfactory, tactile and taste information) chosen to represent instructional methods.
Research and evaluation studies that provide evidence for more learning from multimedia
than from live instruction or other media have been challenged because of their failure to
separate media from method and sensory mode. A number of reviews have argued that
when experiments or evaluation studies report learning advantages for multimedia when
compared with other media, the learning benefits attributed to multimedia are more
plausibly due to the uncontrolled effects of instructional methods and/or sensory mode
influences (not media) and/or different test-relevant information being given to different
groups (Morrison, 2001; Clark & Salomon, 1986; Mielke, 1968; Salomon, 1984;
Schramm, 1977). Clark (2001) has argued that all instructional methods, sensory modes
and information components of instruction can be presented in a variety of media with
equal learning outcomes but with very different costs and access outcomes.
Method confounding. The most promising approach to learning is to assume that
it is influenced by instructional methods (Cronbach & Snow, 1977) that can be embedded
in instruction and presented by a variety of media and not only by multimedia per se.
Instructional methods are defined as “...any way to shape information that compensates
for or supplants the cognitive processes necessary for achievement or motivation. For
example, learners often need an example to connect new information in a learning task
with information in their prior experience. If students cannot (or will not) give themselves
an adequate example, an instructional (method) must provide it for them” (Clark, 2001, p.
208). Variations in interactivity can be provided to learners by a number of media,
including live instructors. If studies provide a necessary method of instruction in a
multimedia condition and do not provide an equivalent form of the method in a compared
instructional treatment, the results will appear to favor multimedia when in fact, the
method influenced the learning. The key issue here is whether any instructional method
can be presented in more than one medium. Clark (2001) has argued that all instructional
methods that are necessary for any kind of learning can be presented in a variety of
media. He claims therefore, that the benefits of media are economic or are to be found in
the increased access to instruction by disadvantaged groups in society, but that learning
benefits due to multimedia alone have not been found and cannot be claimed.
Five Principles 5

Sensory mode and learning. Multimedia instructional designers are tempted

toward instructional presentations that, besides agents, include very active animation,
motion video, colorful graphic displays, background sounds, music and other multi
sensory depictions of course concepts, “voice over” narration and other visually and
aurally exciting displays. While many learners seem to welcome the visual and aural
entertainment, the best evidence suggests that learners are often overloaded by seductive
but irrelevant distractions or the effort of processing redundant information so their
learning is reduced (Mayer, 2001; Moreno & Mayer, 2000; see also chapter 12 by Mayer
in this volume). Mayer (2001) has described a systematic program of research designed
to tease out the benefits of multimedia-supported integrations of visual and aural
depictions of processes that are being learned. He reports evidence that multimedia
lessons where both visual and text-based explanations of processes are spatially or
temporarily separated, and/or are heavily text laden seem to overload the working
memory of many students and decrease their learning. He also reports instances where
the presentation of spatially and temporally integrated visual and aural descriptions of the
same process can enhance learning (Mayer, 2001). He suggests that providing both a
visual and a narrative description of a process being learned will increase the amount of
time information about the process can be held and processed in working memory. This
finding suggests that formatting process information in two sensory modes results in
better learning than presenting the same information in either visual or auditory form
alone. Multimedia, computer-based instruction is a very efficient vehicle for presenting
integrated visual and auditory information yet other media (including live instructors
using silent motion films or television) could provide the same instruction. Because a
number of different media will present visual and aural sensory mode information, this
instructional method is not considered to be a potential learning benefit that is exclusive
to multimedia.
Meta-analytic studies of multimedia. The most recent summary of instructional
media research has been provided in an extensive meta-analysis conducted by Bernard, et
al. (in press) who examined over 650 empirical studies comparing live and multimedia
distance learning to locate 167 studies that met their criteria for design. Their
comprehensive analysis concluded that a very weak learning advantage for multimedia in
empirical studies was attributable to uncontrolled instructional methods. They also
reviewed four previous meta-analyses of earlier and different multimedia issues and
suggested that the evidence in all of them pointed to “no differences” as the most
reasonable conclusion.
A recent example of methods and multimedia, An interesting example of the
difference between multimedia and instructional method can be found in a series of
experiments by Corbett (2001) that focused on the impact of a variety of instructional
methods used to teach Lisp programming based on Anderson’s Lisp tutor (see Anderson
& Gluck, 2001) . Corbett describes an approximate 1.5 sigma effect size increase in
learning over standard mastery learning methods due instructional methods called “model
tracing” and “cognitive mastery”. The addition of scaffolding (providing more tracing
and cognitive mastery support for novice students then withdrawing it slowly as they gain
expertise) increased the effect size impact another .42 . This means that Corbett’s
methods produce a learning benefit of approximately sixty percent over instruction that
gave all necessary information to students but did not use the experimental methods.
Five Principles 6

Time tolLearn. Equally interesting is that when the new methods were compared
with mastery methods (Bloom, 1984) they produced a 40 percent increase in learning.
The methods described were derived not from a study of multimedia but instead from
recent research on cognitive architecture and its influence on the learning of complex
knowledge (Anderson & Gluck, 2001). All methods were delivered by a computer but all
could have been provided also by human tutors though with much less efficiency. The
latter point is emphasized by Corbett’s (2001) finding that the computer delivered version
of the powerful methods resulted in a forty percent decrease in the time required to learn
when compared with human tutors.
If not “learning benefits”, what are the advantages of multimedia
instruction?Clark (2000) has described a number of strategies for evaluating multimedia
instructional programs that separate the benefits of the media from the benefits of the
instructional methods used. Multimedia benefits, he suggests, are to be found in the cost
of instruction, including time savings for students and instructors (when the investment in
instructional design and development are amortized across increasing numbers of
students) and increased access to quality instruction by disadvantaged or rural groups of
students. Evidence for cost and time savings can be found in the work of Corbett (2001)
described above and cost-benefit and cost-effectiveness studies conducted by Levin and
his colleagues (Levin, Glass & Meister, 1987; Levin & McEwan, 2001). Evidence for
access benefits is more difficult to locate and it is possible that this is a less explored area.
One example is a government report of increased access to instruction by people whose
educational alternatives are severely limited by geography or other handicapping
conditions such as economic, physical or social barriers (Office of Technology
Assessment, 1988).
In addition to learning benefits, advocates often implicitly and explicitly (e.g.
Abrahamson, 1998) claim that multimedia results in increased motivation to learn when
compared with more traditional instructional media. The discussion turns next to this
issue.
Principle #2: Multimedia Instruction Is More Motivating than Traditional Instructional
Media or Live Instructors
Abrahamson (1998) may represent the majority of multimedia advocates when he
states that “a primary function of the use of television, computers, and
telecommunications in distance learning is to motivate students rather than just to provide
information to them” (p. 2). However, evidence for the motivational qualities of
multimedia instruction has been elusive at best. The best conclusion at this point is that
overall, multimedia courses may be more attractive to students and so they tend to choose
them when offered options, but student interest does not result in more learning and
overall it appears to actually result in significantly less learning than would have occurred
in “instructor led” courses (Bernard, et al., in press). In order to explain this ironic twist
in empirical research, the discussion turns first to a definition of motivation.
What is motivation? Pintrich and Schunk (2002) in their review of research on
motivation to learn suggest that the existing research focuses on one or more of three
“indexes” or outcomes of motivation: 1) active choice (actively starting to do something
that one formerly “intended” to do but had not started), 2) persistence (continuing to
work towards a goal, despite distractions or competing goals), and 3) mental effort,
Five Principles 7

defined by Salomon (1984) as “the number of non-automatic elaborations invested in

learning” (p. 647).
Each of these indices play a different role in the learning process and some may
not be related to learning. On one hand it is possible that active choice (e.g. the choice to
engage in multimedia learning by choosing to start a multimedia lesson or to select a
multimedia course alternative over a more traditional option ) may be facilitated by
attractive multimedia features such as ease of access, flexibility of scheduling and the
personal control students are often able to exercise when starting, pausing or moving
between different sections of a course of instruction (often called “navigation control”).
Yet initially attractive features of a multimedia course might work against students when
they engage in learning.
Do motivated students learn less in multimedia instruction? In their
comprehensive meta-analysis of 232 empirical studies reporting nearly 600 comparisons
conducted between 1985 and 2002, Bernard, et al. (in press) concluded that courses
reporting high levels of student interest also tended to report lower levels of achievement.
They also concluded that end of course measures of interest tended to be negatively
correlated with end of course achievement. Thus, as achievement increased in multimedia
distance studies, student interest and satisfaction decreased. They conclude that “interest
satisfaction may not indicate success but the opposite, since students may spend less
effort learning, especially when they choose between [multimedia distance education]
and regular courses for convenience purposes (i.e. happy to have a choice and satisfied
but because they wish to make less of an effort to learn…)” (words in brackets inserted to
replace acronym, p. 43). Salomon (1984) presented compelling evidence that may
explain the negative relationship between interest and satisfaction with multimedia
courses and significantly lower learning by students who express a preference for
multimedia. He hypothesized that student interest in newer media is based on an
expectation that it will be a less demanding way to learn. This expectation results in the
investment of lower levels of mental effort, and consequent lower achievement levels,
when compared to instructional conditions that are perceived as more demanding. He
presented compelling evidence to support his hypothesis. This finding has been
replicated a number of times with different media (see for example the discussion of
related studies in Clark, 2001). Salomon’s theory is the most compelling explanation for
Bernard, et al. (in press) meta-analytic finding of an inverse relationship between interest
and achievement.
Mental effort. Apart from the Salomon (1984) studies, not much is known about
the direct impact of multimedia instructional formats on mental effort but recent research
is not promising. Studies by John Sweller and others (for example, Mousavi, Low, &
Sweller, 1995; Sweller & Chandler, 1994) indicate that many instructional strategies and
complex screen displays risk overloading working memory and causing “automated”
cognitive defaults (Clark, 2001) where mental effort is both reduced and directed to non-
learning goals. Complicating this finding is strong evidence that learners are not aware
when they become overloaded and enter a default state (Gimino, 2000). Because all
methods used to measure mental effort involve self report (e.g. Bandura, 1997), this
finding is very distressing. Pintrich and Schunk (2002) suggest the use of various
measures for on going assessment of motivation including self-efficacy (Bandura, 1997),
value for learning goals (Eccles & Wigfield, 2000), mood or emotionality, and dual task
Five Principles 8

measures for mental effort (Gimino, 2000). In general, it seems that mental effort may be
influenced in large part by the amount of perceived difficulty in a multimedia course. It is
possible that when moderately challenging learning goals and tasks are presented, mental
effort increases. When learning tasks are too easy or impossibly difficult, mental effort
decreases radically. Students seem to be able to accurately report the amount of mental
effort they are investing in easy to moderately difficult tasks. Yet there is disturbing
evidence that they seem unaware when they stop investing mental effort as learning tasks
become extremely difficult or impossible. Designers must exercise caution not to
overwhelm multimedia students with extremely complex tasks or screen design features
that overload working memory. Meanwhile, researchers should continue to study how
specific tasks and design features impact mental effort.
Separating motivation to choose multimedia courses and motivation to learn.
Many of the currently measured motivation variables in multimedia studies seem to
reflect interest and enjoyment factors that influence access to instruction or choice of
instructional media rather than learning. Students appear to choose multimedia courses
based on expected flexibility and ease of learning, but those expectations may cause them
to reduce their effort and learn less. This is Bernard, et al.’s (in press) conclusion in their
review of empirical work. On the other hand, persistence and mental effort seem to be
very important learning-related motivation indexes for multimedia since the added
control computers allow students may make it more possible for them to become
distracted and avoid instruction. Imagine a scenario where students stop a multimedia
lesson when they are tired or bored, intending to restart soon, and yet become distracted
and allow a great deal of time pass before restarting the lesson. These gaps in time may
make recall of previously learned material more difficult and/or push students so close to
course or lesson completion deadlines that they must rush to finish on time. While
multimedia lessons do not have to permit this kind of flexibility because it is possible to
program required “milestone” completion schedules, the attractiveness of flexibility may
increase the probability that students will take multimedia courses if they have choices
and so increase access while at the same time placing considerable stress on motivational
processes that support persistence over time.
Persistence in multimedia coursesMultimedia courses may be chosen over other
forms of instruction because students expect that they will receive more individualized
instructor contact. While there appears to be no empirical work on this issue, in
Kennedy’s (2000) survey of a group of online students, 68 % of the 40 respondents said
they enrolled online rather than self-study because they wanted instructor feedback and
guidance through the course. Most students also believe that the heightened instructor
contact enhanced their learning in the course. The State University of New York students
who reported the highest levels of instructor interaction also reported the highest levels of
value for the course (Fredericksen, et al., 2000). The University of California at Los
Angeles has also reduced drop out with a system, in which course manager’s contact
“missing” students to prod them into persisting (Frankola, 2001). Thus, though it seems
clear that multimedia instruction can include (or be perceived to include) more instructor-
student contact, and that this increased interaction may enhance the value of the course
and student persistence, to the extent that such increased interaction is missing,
motivation to persist may be lacking as well. Additional studies concerning the factors
Five Principles 9

and strategies that would further enhance student persistence in multimedia courses
would be useful.
One issue that has been examined for a number of years is the extent to which
multimedia allows designers to accommodate learning styles and therefore enhance the
learning of a broader range of students.
Principle #3: Multimedia Shapes Instruction for Different Learning Styles
Quite understandably, individual differences between people that may impact the
efficacy of instruction have been a major focus of research for decades. If we were to
understand all of the factors that contributed to instructional outcome differences, it could
be expected that we would be able to optimally align pedagogical approaches with
learner profiles, thereby narrowing achievement gaps. While this goal is an important
one for the future of instructional research and multimedia design, to date researchers in
this area have found no evidence that tailoring multimedia instruction to different
learning styles results in learning benefits. This section briefly reviews the research on a
variety of individual differences investigated in the research literature (learning
preferences, cognitive styles, motivation, intelligence, and prior knowledge) and
identifies those that have consistently been found to be relevant factors in the success of
learning outcomes in most instructional environments, including multimedia courses, and
those that have not.
Cognitive styles and learning preferences. Cognitive styles and learning
preferences have been advocated by some researchers for a number of years as traits that
contribute to differential success in learning tasks on the basis of learners’ innate
approaches to learning or solving problems. By understanding these proclivities, it is
argued, multimedia instruction can be optimally matched to the learner in order to
maximize achievement. Unfortunately, these constructs have proven notoriously difficult
to validate for both the stable assessment of learner characteristics and the customization
of instruction to improve student outcomes (e.g. Duff & Duffy, 2002; Henson & Hwang,
2002; Kavale & Forness, 1987; Loo, 1997, Richardson, 2000, Stahl, 1999). In general,
cognitive style theories posit one or more linear scales on which learners can score closer
to one extreme or another. These descriptors typically have a global, integrative,
contextualized reasoning pattern at one extreme and a highly focused, isolative,
decontextualized pattern at the other (e.g. Cassidy, 2004). In similarly structured
dichotomies, some theories also include a visualizer/verbalizer differentiation for
sensory, rather than logical, cognition or other descriptive dimensions (e.g. Riding &
Cheema, 1991).
Classifying learners in these systems entails requiring each learner to complete a
self-report instrument that usually asks questions about their preferred learning modalities
and typical approaches to solving problems. However, in addition to persistent problems
achieving intraindividual score reliability over time and across domains, attempts to
validate these styles have also failed to yield consistent differentiation between cognitive
style and measures of intelligence (Richardson & Turner, 2000).
An additional problem with cognitive and learning styles lies in the self-report
method of identification wherein learners are asked to report their preferences for
approaches to learning and solving problems. Mayer and Massa (2003) tested 95
undergraduates using measures of visual and verbal reasoning ability and found no
significant relationship between subjects’ self-reported measures of style and their
Five Principles 10

performance on the reasoning tests. This finding is consistent with other investigations of
learners’ abilities to adequately select effective learning approaches. Clark (1982) found
in an extensive meta-analysis of studies that utilized learner preference or enjoyment for
particular instructional media or techniques that learner enjoyment was typically
uncorrelated or negatively correlated to performance outcomes. That is, subjects who
reported preferring a particular instructional technique typically did not derive any
instructional benefit from experiencing it. Salomon (1984) found similar results in an
experimental study of sixth grade learner preferences for learning from television or
print: The subjects who learned more of the material presented in instruction were those
who did not receive instruction via their preferred medium. More recently, these results
were replicated with adult distance learners by Li, O’Neil, & Feldon (in press).
Motivation/goal orientation. Another individual difference known to impact
achievement in instructional settings is goal orientation. Goal orientation refers to the
source of an individual’s motivation for learning. Those who are classified as having
mastery goal orientations pursue the acquisition of new knowledge for their own
satisfaction and are not motivated by the comparison of their performance to that of
others. In contrast, performance oriented learners invest effort in learning primarily for
the purpose of attaining public or comparative recognition for their accomplishments
(Pintrich & Schunk, 2002). Because mastery oriented students engage with the material
for the purpose of understanding, they have been consistently found to be more likely to
expend effort to learn the concepts presented and engage with the material more
strategically and at a deeper level. However, their internal focus may sometimes prove
maladaptive in the context of an evaluated course, because their focus may not have been
on the learning objectives on which they would be assessed (Barron & Harackiewicz,
2001). Likewise, performance oriented learners can manifest both adaptive and
maladaptive behaviors. Successful behaviors are referred to as “approach” strategies,
because they entail a proactive attempt to gain recognition for success by self-regulating
and scaffolding learning opportunities to ensure success. In contrast, “avoidance”
behaviors are those by which performance oriented learners seek to dissociate their
performance in the learning environment from negative evaluations of their abilities
through self-handicapping behaviors that prevent their best efforts from being
demonstrated (Eccles & Wigfield, 2002).
Intelligence. One of the first traits found to account for stable differences
between learners is intelligence. Fluid reasoning ability has been found to reliably
predict performance on novel problem solving tasks (Cattell, 1987). However, as
instruction familiarizes learners with a given set of skills and problems over time, such
advantages diminish when criterion-referenced performance is evaluated (e.g. Ackerman,
1987, 1988, 1990, 1992). Indeed, studies of experts in a variety of fields have found no
correlation between fluid ability and performance (e.g. Ceci & Liker, 1986; Doll & Mayr,
1987; Ericsson & Lehmann, 1996; Hulin, Henry, & Noon, 1990; Masunaga & Horn,
2001), precisely because the high levels of deliberate practice that are necessary to excel
in a domain entail the development of skills that are applied to problems whose qualities
are known. Although individuals can acquire new knowledge and problem solving
strategies and apply them to improve their performance within a particular domain, there
is not yet any evidence that such improvements can impact the general problem solving
skills associated with fluid ability (Perkins & Grotzer, 1997).
Five Principles 11

Prior knowledge. Learners’ acquired knowledge prior to participating in a course

can also account for significant individual differences in academic outcomes. Not only
do discrete pieces of knowledge relevant to the course material provide a relative
advantage to those learners who possess them, but having such knowledge can directly
affect the efficacy of certain pedagogical strategies. When novices acquire knowledge in
a domain, the learning process is slow and effortful. The requisite effort to process
relevant information decreases as schemas are constructed and skills are practiced. As a
result, learners with low levels of prior knowledge require more extensive instructional
support to minimize the level of unnecessary cognitive load imposed by the material
presented. By reducing the amount of effort required of novice learners, more attentional
capacity is available for the accurate encoding of material. If excessive or unstructured
information is presented to the novice learner, he will become overloaded and subsequent
performance will suffer (van Merriënboer, Kirschner, & Kester, 2003). Conversely,
learners with higher levels of prior knowledge in the domain benefit from less structured
instruction. Whereas the novice requires scaffolding to properly organize the information
presented without overwhelming limited working memory, for a more knowledgeable
learner, superfluous instructional support will likely interfere with existing schemas and
consequently impose unnecessary cognitive load, resulting in performance decrements
(Kalyuga, Ayres, Chandler, & Sweller, 2003).
Conclusions about accommodating learning styles. Whereas cognitive and
learning styles have not proven to be robust foundations on which to customize
instruction to accommodate individual differences, intelligence, motivational goal
orientations, and prior knowledge have demonstrated significant effects. Although there
seems to be little that can be done to modify intelligence and goal orientation, the
assessment of prior knowledge for the customization of multimedia instruction offers
great promise. Past studies have demonstrated significant relative improvements in
instruction when learner support was faded out as learners acquired more knowledge
(Kalyuga, Chandler, Tuovinen, & Sweller, 2001), and new research suggests that rapid
assessments of learners’ knowledge states can dynamically shape the course of computer-
based instruction to effectively improve overall achievement (Kalyuga & Sweller, in
press-a, in press-b).
The multimedia pedagogical support used to scaffold learning for less
experienced students is often provided by animated instructional figures or “agents”. The
chapter turns next to a review of the research on the learning impact of multimedia
agents.
Principle #4: Multimedia Instruction Can Provide Active Pedagogical Agents that
Increase Motivation and Aid Learning
Animated pedagogical agents (agents) are defined by Craig, Gholson and Driscoll
(2002) as “a computerized character (either humanlike or otherwise) designed to facilitate
learning” (p. 428). Many multimedia instructional programs directed to both children
and adults seem to provide instructional support in the form of animated agents.
Atkinson (2002) suggests that agents “ ... reside in the learning environment by appearing
as animated “humanlike” characters, which allows them to exploit ... communication
typically reserved for human-human interaction ... [and] can focus a learner’s attention by
moving around the screen, using gaze and gesture, providing ... feedback and conveying
emotions” (p. 416-417). Agents are a product of recent technological advances in
Five Principles 12

multimedia computer animation and user interface design. Advocates suggest that they
have great potential for aiding human learning (e.g., Sampson et. al., 2002). The use of
agents is a recent, welcome and visible attempt to insert pedagogical support into
multimedia instruction yet initial empirical studies suggest that they may distract and
interfere with learning more than aid it.
Agent research results are mixed. In some studies, agent-based instruction results
in more learning and/or more positive attitudes towards lessons (e.g. Bosseler & Massaro,
2003; Mitrovic & Suraweera, 2000; Moundridou & Virvou, 2002; Ryokai, Vaucelle, &
Cassell, 2003), whereas in others agents produce no learning or motivational benefits
(e.g. Andre et. al, 1999; Baylor, 2002; Craig, Driscoll, & Gholson, 2004; Mayer, Dow, &
Mayer, 2002). However, in other experiments, results are mixed and somewhat confusing
(e.g. Atkinson, 2002; Moreno et. al., 2001) and many studies that demonstrate learning
benefits from agents have been criticized for design errors (Choi & Clark, 2004). Our
review of these studies suggests that positive learning results most often come from
studies where the method being used by the agent to “teach” are not compared with
conditions where the method is provided to students without the agent.
Design problems. Very few agent studies control for the type of hypothesized
learning and/or motivational support the agent is providing in a balanced, alternative
condition where the same type of learning support is provided by a lower technology,
non-agent condition. If the agent is providing a specific type of instructional support,
study designs should include a “low technology” alternative method of providing the
same type of support to a comparison or control group. Any pedagogical support
provided by an agent can also be provided in a ‘lean’ format. Dehn & van Mulken (2000)
explain that without this type of design control, “...differences between the two
conditions cannot be attributed exclusively to [the agent]” (p. 18). An adequate test
requires that the non-agent or control condition provide all of the learning and
motivational support available from the agent condition, otherwise a comparison will be
potentially confounded by the uncontrolled effects of the instructional methods the agent
provides and the agent itself.
Confusion about the source of measured benefits. For example, Atkinson (2002)
compared a “voice plus agent” group with “voice only” and “text only” groups
(Experiment 2). In the voice plus agent group, participants listened to the agent’s verbal
explanations and saw the agent highlighting relevant information on the screen
simultaneously by using pointing gestures. Alternatively, participants in the voice only
and text only conditions only received explanations delivered either in voice or text,
respectively. In other words, participants in the voice only and text only groups did not
have the benefit of a visual, highlighting indicator for important information, which
might have forced the participants to use their scarce cognitive resources to connect
verbal explanation with related visual information on the screen. Therefore, although the
voice plus agent group outperformed the other two groups in far-transfer performance, it
is problematic to attribute the obtained learning benefit exclusively to the presence of the
agent. The critical learning support provided by the agent--directing learner’s attention to
the key information in the screen display, was not available to the two comparison
groups. A leaner version of the agent’s pointing gesture would be to simply use an
animated arrow and/or to underline the same information selected by the agent in the
comparison conditions. Other studies which also failed to control the types of
Five Principles 13

instructional and motivation supports provided in agent and alternative conditions include
Moundridou & Virvou (2002) and Ryokai, et al. (2002).
Adequately designed studies provide consistent results. André and colleagues
(1999) conducted a well- controlled study that avoided this design pitfall. To find
empirical support for the affective and cognitive benefits of their “PPP Persona” agent,
they exposed participants to two different memory tasks--a technical description (the
operation of pulley systems) and an informational presentation that included the names,
pictures, and office locations of fictitious employees. Both experiment and control
versions provided the same treatments except that the control groups did not have the
PPP Persona agent. The control group heard a voice conveying the same explanations as
the agent provided to the experimental group. The agent’s pointing gesture was replaced
with an arrow that pointed to important information in the control condition. Following
the presentations, participants’ affective reaction to the agent and control condition was
measured through a questionnaire whereas the cognitive impact was measured by
comprehension and recall questions. The results showed significant differences only in
the affective measures. Participants interacting with the PPP Persona agent for the
technical description found the presentation less difficult and more entertaining. The
positive effects, however, disappeared for the informational presentation about the
fictitious employees. Participants reported that the PPP Persona agent was less
appropriate for employee information and less helpful as an attention direction aid. No
significant achievement differences were found between the experimental and control
groups for either the technical description or information presentation tasks on
comprehension or recall measures. Thus, in this well-designed study, the agent did not
provide learning or motivational benefits that translated to greater learning. Yet, because
of the adequate design, there is the serendipitous finding that learners may believe that
agents are more appropriate and likeable in some learning tasks but not in others.
Craig, et al. (2002) also employed an adequate design where participants learned
the process by which lightning occurs presented through an agent and through alternative
multimedia (i.e., picture, narration, or animation). An animated agent that pointed to
important instructional elements on a computer screen was contrasted with a sudden
onset highlighting (i.e., color singleton or electronic flashing) and animation of the same
information (without the agent) for comparison groups. The narrative information was
synchronized simultaneously with the agent’s pointing gestures, separated and provided
prior to the agent’s pointing, or in a third condition, with a sudden onset of highlighting
and animation of relevant parts of an instructional picture. Craig, et al.’s results indicated
that the agent made no difference in learners’ performance both in cognitive load
assessment and performance tests (i.e., retention, matching, and transfer). Rather, they
reported a significant benefit from both a sudden onset and animation of parts of the
pictures for focusing learners’ attention. This may be an example of an effect that van
Merrienboer (1997) calls “just in time” learning support.
Conclusion – animated agents do not increase learning. These results provide
evidence that in multimedia studies of agents, measured differences in student learning
may not be due to the agent by itself or any increased motivation or attention caused by
the agent, but rather due to the pedagogical method provided by the agent. Thus we
should ask a question: Is the animated pedagogical agent the only way to deliver these
types of instructional methods in a multimedia learning environment? If alternative ways
Five Principles 14

can deliver the same instruction with the same learning and motivation, but with less cost,
shouldn’t we choose the least expensive option?
Erickson (1997) argued that the adaptive functionality of an instructional system
is often enough for learners to perform a task and achieve the same outcome without the
guidance of an agent. He further suggested that when including an agent, instructional
designers should think about what benefits and costs the agent would bring, and far more
research should be conducted on how people experience agents. Furthermore, Nass and
Steuer (1993) found that simply using a human voice without the image of an agent was
sufficient to induce learners to use social rules when interacting with a computer. Moreno
and colleagues (2001) also noted that learners may form a social relationship with a
computer itself without the help of an agent and thus, the image of an agent might not be
necessary to invoke a social agency metaphor in a computer-based learning environment.
Baylor (2002), Craig, et al. (2004) and Mayer, et al. (2003) found no effect of agent
image on learning outcomes. This research is also reviewed in Chapter 13 of this volume
by Mayer.
Principle #5: Multimedia Instruction Provides Learner Control and Discovery Pedagogy
to Enhance Learning
There is a persistent belief among some segments of the education and training
communities that the most effective learning experiences are those in which learners
navigate unstructured multimedia learning environments or solve novel problems
presented without instructional supports (Land & Hannafin, 1996). However, this
assumption about pure discovery learning has been tested repeatedly over forty years of
research and found to lack empirical validation when its efficacy, efficiency, and impact
on successful transfer of skills have been compared to well-structured, guided instruction
(Mayer, 2004). Several factors have been found to play key roles in enactive learning
environments that have significant impacts on student success, specifically cognitive
load, instructional supports, and prior knowledge.
Cognitive load theory. Developed by John Sweller and his colleagues, cognitive
load theory reliably predicts instructional learning outcomes by analyzing the
pedagogical materials and features of the learning environment to determine the amounts
of relevant and irrelevant load placed on working memory (Sweller, 1988, 1989, 1999;
Sweller, van Merriënboer, & Paas, 1998; Chapter 2 of this volume by Sweller). Because
working memory capacity is limited, unnecessary features function as artificial
constraints on the amount of mental resources that can be directed toward the necessary
semantic elements for new knowledge to be successfully acquired. As novice learners
develop skills and organizational schemas within the domain of instruction, the
information occupies significantly less “space,” which allows for the processing of more
advanced (i.e. higher load) elements and complex problem solving. Because adaptive
organizational schemas are difficult to acquire, resources that could otherwise be
dedicated to conceptual understanding must be dedicated to imposing meaningful
structure on the material presented if external supports and carefully controlled
presentation of material is not utilized. If these supports are not used, fewer cognitive
resources are available to be focused on the mastery of conceptual content (Chandler &
Sweller, 1991; Sweller, Chandler, Tierney, & Cooper, 1990).
Instructional support. As learners gain mastery of basic knowledge and
organizational structures, their need for external supports to optimize their learning
Five Principles 15

efforts decreases. Because the schemas organize the information presented effectively, it
becomes redundant for those frameworks to be provided externally within the learning
environment. Thus, providing more structure than is appropriate to the level of the
learner can impose extraneous cognitive load and redirect working memory resources
away from the target material. Known as the expertise reversal effect, it has been
demonstrated that optimal instruction utilizes instructional supports that fade in
proportion to the learner’s level of expertise for a particular skill or concept (Kalyuga,
Ayres, Chandler, & Sweller, 2003).
These findings present a complex picture for the appropriate use of discovery
learning environments. Because by definition, pure discovery learning does not use
instructional supports, it imposes large amounts of extraneous cognitive load on novice
and intermediate learners, thereby increasing the amount of time and mental effort
expended on learning while decreasing post-learning performance relative to more
structured approaches (Tuovinen & Sweller, 1999). However, learners with high levels
of expertise in the material presented have been found to perform better after learning in
unstructured environments that do not impose unnecessary scaffolding. As such, pure
discovery learning is maximally beneficial only to those learners who require additional
training least.
Types of support. The specific nature of the instructional supports that have been
used to guide discovery learning processes also plays a major role in the efficacy of the
instruction (see also Chapter 14 by de Jong). De Jong and van Joolingen (1998) reviewed
a variety of tools that were used in computer-based discovery learning environments and
concluded that providing enhanced task structure for learners consistently improved
learner outcomes. For example, a number of studies found that up to four times as many
students were able to grasp concepts central to a simulation-based discovery program
when instructions specifying how to proceed in solving the problem were provided when
compared with subjects who experienced the pure discovery mode (e.g. Gruber, Graf,
Mandl, Renkl, & Stark, 1995; Linn & Songer, 1991). Even when overall results did not
indicate a significant difference in subsequent student performance between guided and
pure discovery learning environments, deeper analyses indicated that students with lower
levels of ability in the target domain who received guidance did attain posttest scores
significantly higher than their unguided counterparts, providing a replication of the
expertise reversal effect discussed above (Veenman & Elshout, 1995). Further, several
studies demonstrated high correlations between intelligence and success in discovery
learning environments across a number of domains, suggesting that such forms of
instruction are less able to generate strong results for all learners (Veenman, 1993).
Indeed, Funke (1991) reported that correlations between intelligence and achievement
linked to discovery learning simulations increase as the level of guidance offered by the
environment falls. Similarly, Shute and Glaser (1990) found that embedding guidance
tools into the learning environment resulted in only a very low correlation between
achievement and intelligence.
Thrashing. One of the problems that learners frequently encounter in pure
discovery learning environments is that of “thrashing” (Lewis, Bishay, & McArthur,
1993) or “floundering” (Goodyear, Njoo, Hijne, & van Berkum, 1991), in which learners
lack an effective and/or systematic approach to interacting with the learning environment
and consequently are unable to draw valid or helpful inferences from simulation
Five Principles 16

outcomes or events. In Lewis, et al.’s study, students were directed to engage in

discovery learning tasks using a geometry software tool to identify formulas describing
mathematical relationships between geometric figures. When they reached an impasse
and no scaffolding or assistance was available, students generally persisted in their
attempts to use a strategy that had been previously effective. After multiple attempts at
using the maladaptive strategy, students were then observed to progressively attempt less
and less appropriate solution strategies until they eventually quit the program or selected
a new, unrelated goal to pursue. Such thrashing in pursuit of a solution did not yield a
successful solution for any of the study participants but occupied as much as 25% of their
total instructional time. Similarly, Goodyear, et al.’s study found that when students
engaged in this kind of behavior, they were unable to identify the causal relationships that
existed between their actions and resulting events within the learning environment. The
lack of a systematic approach prevented them from adequately tracking their own actions,
and learners were thus unable to extract functional principles from the interactions.
Even when goals and processes are relatively clear, discovery learning
environments can produce impediments to learning through uncontrolled sequencing of
material. Kester, Kirschner, van Merrienboer, & Baumer (2001) found in an exploratory
study that the timeliness of information presentation predicts performance in learning
tasks designed to facilitate complex skill acquisition. Their just-in-time instructional
model holds that abstract “supportive” information (e.g. mental model explanation) must
be presented prior to learner attempts to solve authentic complex tasks, whereas
prerequisite information (e.g. facts relevant to a specific problem scenario) should be
presented to the learner during the execution of the tasks. Further, in a recent study,
Clarke, Ayres, and Sweller (in press) found that when students were given a learning task
to master mathematics concepts through the manipulation of a spreadsheet, those students
who were not provided with specific instructions for using the spreadsheet program prior
to attempting to learn the mathematics material performed at a much lower level than
those who had acquired spreadsheet knowledge prior to attempting the learning task.
Similarly, many of the studies reviewed by de Jong and van Joolingen (1998) that
required students to discover scientific principles within simulated environments found
that only those students who acquired strong scientific inquiry skills prior to attempting
identification of the science concepts were able to achieve at high levels. Although many
such environments provided related procedural support if requested by the learner during
the discovery task, meta-analyses of students’ self-assessment with regard to their
learning needs have found consistent evidence that students—especially novices—do not
accurately determine which pedagogical formats and tools will be most beneficial for
them (Clark, 1982, 1989).
General Conclusion
Multimedia instruction offers extraordinary benefits to education including a wide
range of instructional options and, with adequate instructional design, considerable
reductions in the time required to learn, the time required of expert teachers, and when
large numbers of students are involved, the cost of learning (Clark, 2001). Like all new
and exciting educational innovations it also suffers from mistaken beliefs about its
potential and achievements. This chapter reviewed five commonly held beliefs about
multimedia that have not been supported by research. For example, multimedia does not
increase student learning beyond any other media including live teachers.
Five Principles 17

It also appears that studies examining the motivational benefits of multimedia

instruction provide good news and bad news. While multimedia may be a more attractive
option for instructions by students than older media, the bad news is that their interest
most often seems to lead them to reduce their effort to learn. Meta-analytic evidence
from many studies suggests that as student interest in multimedia courses increases,
learning tends to decrease because students may feel that learning in these courses
requires less work.
If multimedia does not produce more learning than other options, and if
motivation to choose multimedia courses produces an ironic reduction in course
achievement, the solution seems to require increasing the focus on pedagogical support in
multimedia courses. A pedagogical approach that seems very common in multimedia
courses is an attempt to tailor instructional sequences for learners with different learning
styles. The flexibility of multimedia permits the tailoring of instruction to a variety of
learning styles by providing different versions of the same lesson to accommodate
different styles. However, attempts to validate this assumption over the past thirty years
have generally failed. New efforts, such as those initiated by Mayer and Massa (2003),
may meet with success in the future. However given the data currently available, it
appears that the two most promising individual differences that can be used to shape
adaptive instructional programs are the prior knowledge and learning goal orientation of
students, and nothing inherent to these factors seems to require use of multimedia for
tailored accommodation.
Attempts to insert socially engaging learning support into multimedia courses
with animated pedagogical agents also seem not to increase learning and sometimes
appear to diminish instructional effectiveness, because agents often produce cognitive
overload for students. Evidence from well-designed studies suggests that agents may be
expensive and unnecessary, because appropriately designed narration and instructional
methods embedded into instruction can achieve similar learning outcomes at less cost.
Finally, multimedia advocates have often embraced constructivist-based
discovery and problem-based learning pedagogy. The flexibility of multimedia
technology permits the design of courses where students can control not only the
(beneficial) pacing of instruction, but also students’ navigation between and within
lessons. The latter type of control combined with unguided or minimally guided
instruction seems most often to harm learning for students with less prior knowledge of
course subject matter. In another ironic twist, strong instructional guidance and
scaffolding seems to interfere with the learning of more advanced students. Thus,
tailoring instruction to student prior knowledge does seem to be beneficial, but it does not
require most of the features of multimedia instruction.
The main concern addressed in this chapter is the need to check research evidence
for the presumed benefits of all instructional media and related pedagogies. Research
sometimes provides counterintuitive evidence and so prevents us from unintentionally
causing damage or investing scarce resources in instruction that does not support
learning. It can also point in directions that can lead to dramatically increases in
achievement such as Corbett’s (2001) two sigma gain in learning accompanied by a 40
percent reduction in learning time.
Five Principles 18

References
Abrahamson, C. E. (1998). Issues in interactive communication in distance education.
College Student Journal, 32(1), 33-42.
Ackerman, P. L. (1987). Individual differences in skill learning: An integration of
psychometric and information processing perspectives. Psychological Bulletin,
102, 3–27.
Ackerman, P. L. (1988). Determinants of individual differences during skill acquisition:
Cognitive abilities and information processing. Journal of Experimental
Psychology: General, 117, 288–318.
Ackerman, P. L. (1990). A correlational analysis of skill specificity: Learning, abilities,
and individual differences. Journal of Experimental Psychology: Learning,
Memory, and Cognition, 16, 883–901.
Ackerman, P. L. (1992). Predicting individual differences in complex skill acquisition:
Dynamics of ability determinants. Journal of Applied Psychology, 77, 598–614.
American Society for Training and Development (2001). E-Learning: If you build it, will
they come?. Alexandria VA: ASTD.
Anderson, J.R., & Gluck, K. (2001). What role do cognitive architectures play in
intelligent tutoring systems. In D. Klahr & S. Carver (Eds.) Cognition and
instruction: 25 years of progress (pp. 227-262). Mahwah, NJ: Lawrence
Erlbaum.
André, E., Rist, T., & Müller, J. (1999). Employing AI methods to control the behavior of
animated interface agents. Applied Artificial Intelligence, 13, 415-448.
Atkinson, R. K. (2002). Optimizing learning from examples using animated pedagogical
agents. Journal of Educational Psychology, 94(2), 416-427.
Bandura, A. (1997). Self-Efficacy: The exercise of control. New York: Freeman.
Barron, K. E., & Harackiewicz, J. M. (2001). Achievement goals and optimal
motivation: Testing multiple goal models. Journal of Personality & Social
Psychology, 80(5), 706-722.
Baylor, A. L. (2002). Expanding preservice teachers’ metacognitive awareness of
instructional planning through pedagogical agents. Educational Technology
Research & Development, 50(2), 5-22.
Bernard, R., Abrami, P., Lou, Y., Borokhovski, E., Wade, A., Wozney, L., Wallet, P.
A., Fiset, M., and Binru Huang. (in press). How does distance education
compare to classroom instruction? A meta-analysis of the empirical literature.
Review of Educational Research.
Bloom, B.(1984) The 2-sigma problem: The search for methods of group instruction as
effective as one-to-one tutoring, Educational Researcher, 13, 4-16.
Bosseler, A., & Massaro, D. (2003). Development and evaluation of a computer-
animated tutor for vocabulary and language learning in children with autism.
Journal of Autism and Developmental Disorders, 33(6), 653-672.
Cassidy, S. (2004). Learning styles: An overview of theories, models, and measures.
Educational Psychology, 24(4), 419-444.
Cattell, R.B. (1987). Intelligence: Its structure, growth and action. Amsterdam: North
Holland.
Five Principles 19

Ceci, S. J., & Liker, J. K. (1986). A day at the races: A study of IQ, expertise, and
cognitive complexity. Journal of Experimental Psychology, 115, 255-266.
Chandler, P. & Sweller, J. (1991). Cognitive load theory and the format of instruction.
Cognition and Instruction, 8, 293-332.
Choi, S., & Clark, R. E. (2004). Five suggestions for the design of experiments on the
effects of animated pedagogical agents. Symposium paper presented at the
Annual Meeting of the American Educational Research Association in San Diego,
California.
Clark, R. E. (1982) Antagonism between achievement and enjoyment in ATI studies.
Educational Psychologist, 17(2). 92-101.
Clark, R. E. (1989), When teaching kills learning: Research on mathemathantics. In H.N.
Mandl, N. Bennett, E. de Corte and H.F. Freidrich (Eds.). Learning and
Instruction. European Research in an International Context. Volume II. London:
Pergamon Press Ltd.
Clark, R. E. (2000). Evaluating distance education: Strategies and cautions. The
Quarterly Journal of Distance Education, 1(1), 5-18.
Clark, R. E. (Ed.). (2001). Learning from Media: Arguments, Analysis and Evidence.
Greenwich, CT: Information Age Publishers.
Clark, R.E., & Salomon, G. (1986). Media in teaching. In M. Wittrock (Ed.), Handbook
of Research on Teaching (3rd ed.). New York: Macmillan.
Clarke, T., Ayres, P., & Sweller, J. (in press). The impact of sequencing and prior
knowledge on learning mathematics through spreadsheet applications.
Educational Technology Research and Development.
Corbett, A. (2001). Cognitive computer tutors: Solving the two-sigma problem. In M.
Bauer, P. Gmytrasiewicz & J. Vassileva (Eds.), User Modeling 2001:
Proceedings of the 8th International Conference, UM2001 (pp. 137-146). New
York: Springer.
Craig, S., Driscoll, D. M., & Gholson, B. (2004). Constructing knowledge from dialog in
an intelligent tutoring system: Interactive learning, vicarious learning, and
pedagogical agents. Journal of Educational Multimedia and Hypermedia, 13(12),
163-183.
Craig, S. D., Gholson, B., & Driscoll, D. M. (2002). Animated pedagogical agents in
multimedia educational environments: Effects of agent properties, picture features
and redundancy. Journal of Educational Psychology, 94(2), 428-434.
Cronbach, L., & Snow, R. E. (1977). Aptitudes and instructional methods. New York:
Irvington.
de Jong, T., & van Joolingen, W.R. (1998). Scientific discovery learning with computer
simulations of conceptual domains. Review of Educational Research, 68(2), 179-
201.
Dehn, D. M. & van Mulken, S. (2000). The impact of animated interface agents: a review
of empirical research. International Journal of Human-Computer Studies, 52, 1-
22.
Doll, J., & Mayr, U. (1987). Intelligenz und schachleistung—eine untersuchung an
schachexperten. [Intelligence and achievement in chess—a study of chess
masters.] Psychologische Beiträge, 29, 270-289.
Duff, A., & Duffy, T. (2002). Psychometric properties of Honey and Mumford’s
Five Principles 20

Learning Styles Questionnaire. Learning and Individual Differences, 33, 147-

163.
Eccles, J. S., & Wigfield, A. (2000). Schooling's influences on motivation and
achievement. In S. Danziger and J. Waldfogel (Eds.), Securing the Future:
Investing in Children from Birth to College (pp. 153-181). New York: Russell
Sage Foundation.
Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual
Review of Psychology, 53, 109-132.
Ericsson, K. A., & Lehmann, A. C. (1996). Expert and exceptional performance:
Maximal adaptation to task constraints. Annual Review of Psychology, 47, 273-
305.
Erickson, T. (1997). Designing agents as if people mattered. In J. M. Bradshaw (Ed.),
Software Agents (pp. 79-96). Menlo Park, CA: MIT Press.
Frankola, K. (2001). Why online learners drop out. Workforce, 80(10), 52-60.
Fredericksen, E., Pickett, A., Shea, P., Pelz, W., & Swan, K. (2000). Student satisfaction
and perceived learning with on-line courses: Principles and examples from the
SUNY learning network. Retrieved June 30, 2002, from [Link]
alnweb/journal/Vol4_issue2/le/Fredericksen/[Link]
Funke, J. (1991). Solving complex problems: exploration and control of complex
systems. In R.J. Sternberg & P.A. Frensch (Eds.), Complex problem solving:
Principles and mechanisms (pp. 185-223). Hillsdale, NJ: Erlbaum.
Gimino, A. (2000). Factors that influence students’ investment of mental effort in
academic tasks: A validation and exploratory study. Unpublished doctoral
dissertation, University of Southern California, Los Angeles, CA.
Goodyear, P., Njoo, M., Hijne, H., & van Berkum, J.J.A. (1991). Learning processes,
learner attributes and simulations. Education & Computing, 6, 263-304.
Gruber, H., Graf, M., Mandl, H., Renkl, & Stark, R. (1995). Fostering applicable
knowledge by multiple perspectives and guided problem solving. Proceedings
from the Annual Conference of the European Association for Research on
Learning and Instruction, Nijmegen, The Netherlands.
Henson, R. K., & Hwang, D. (2002). Variability and prediction of measurement error in
Kolb’s Learning Style Inventory scores: A reliability generalization study.
Educational and Psychological Measurement, 62(4), 712-727.
Hulin, C. L., Henry, R. A., & Noon, S. L. (1990). Adding a dimension: Time as a
factor in the generalizability of predictive relationships. Psychological Bulletin,
107, 328-340.
Kalyuga, S., Ayres, P., Chandler, P. & Sweller, J. (2003) Expertise reversal effect.
Educational Psychologist, 38, 23-31.
Kalyuga, S., Chandler, P., Tuovinen, J., & Sweller, J. (2001). When problem solving is
superior to studying worked examples. Journal of Educational Psychology, 93,
579-588.
Kalyuga, S., & Sweller, J. (in press-a). Measuring knowledge to optimize cognitive load
factors during instruction. Journal of Educational Psychology.
Kalyuga, S., & Sweller, J. (in press-b). Rapid dynamic assessment of expertise to
improve the efficiency of adaptive E-learning. Educational Technology: Research
& Development.
Five Principles 21

Kavale, K. A., & Forness, S. R. (1987). Substance over style: Assessing the efficacy of
modality testing and teaching. Exceptional Children, 54(3), 228-239.
Kennedy, C. (2000). Quick online survey summary. Retrieved June 30, 2002, from
[Link]
Kester, L., Kirschner, P., van Merrienboer, J., & Baumer, A. (2001). Just-in-time
information presentation and the acquisition of complex cognitive skills.
Computers in Human Behavior, 17, 373-391.
Kozma, R. B. (1994). Will media influence learning? Reframing the debate. Educational
Technology Research and Development, 42(2), 7-19.
Kozma, R. (2000). Reflections on the state of educational technology. Educational
Technology, Research and Development, 48, 5-15.
Land, S., & Hannafin, M.J. (1996). A conceptual framework for the development of
theories in action with open learning environments. Educational Technology
Research and Development, 44(3), 37–53.
Levin, H. M, Glass, G., & Meister, G. R. (1987). Cost-effectiveness of computer assisted
instruction. Evaluation Review, 11(1), 50-72.
Levin, H. M., & McEwan, P. J. (2001). Cost-effectiveness analysis: Methods and
applications (2nd ed.). Thousand Oaks, CA: Sage.
Lewis, M., Bishay, M., & McArthur, D. (1993). The macrostructure and microstructure
of inquiry activities: Evidence from students using a microworld for mathematical
discovery. Proceedings from the World Conference on Artificial Intelligence and
Education, Edinburgh.
Li, L., O’Neil, H. F., & Feldon, D. F. (in press). The effects of effort and worry on
distance learning with text and video. The American Journal of Distance
Education.
Linn, M. C., & Songer, N. B. (1991). Teaching thermodynamics to middle school
students: What are appropriate cognitive demands? Journal of Research in
Science Teaching, 28, 885-918.
Loo, R. (1997). Evaluating change and stability in learning styles: A methodological
concern. Educational Psychology, 17, 95-100.
Masunaga, H., & Horn, J. (2001). Expertise and age-related changes in components
of intelligence. Psychology and Aging, 16(2), 293-311.
Mayer, R. E. (2004). Should there be a three-strikes rule against pure discovery
learning? American Psychologist, 59(1), 14-19.
Mayer, R. (2001). Multimedia learning. Cambridge, England: Cambridge University
Press.
Mayer, R. E., Dow, G. T., & Mayer, S. (2002). Multimedia learning in an interactive self-
explaining environment: What works in the design of agent-based microworlds?
Journal of Educational Psychology, 95(4), 806-813
Mayer, R. E., & Massa, L. J. (2003). Three facets of visual and verbal learners:
Cognitive ability, cognitive style, and learning preference. Journal of
Educational Psychology, 95(4), 833-846.
Mayer, R. E., Sobko, K., & Mautone, P. D. (2003). Social cues in multimedia learning:
Role of speaker's voice. Journal of Educational Psychology, 94, 419-425.
Mielke, K. W. (1968). Questioning the questions of ETV research. Educational
Broadcasting, 2, 6-15.
Five Principles 22

Mitrovic, A. & Suraweera, P. (2000). Evaluating an Animated Pedagogical Agent.

Lecture notes in computer science, No. 1839, 73-82.
Moreno, R., & Mayer, R. (2000). A learner centered approach to multi media
explanations: Deriving instructional design principles from cognitive theory.
Retrieved from [Link]
Moreno, R., Mayer, R. E., Spires, H. A., & Lester, J. C. (2001). The case for social
agency in computer-based teaching: Do students learn more deeply when they
interact with animated pedagogical agents? Cognition and Instruction, 19(2), 177-
213.
Morrison, G. R. (2001). The media effects question: “Unresolvable” or asking the right
question. In R. E. Clark (Ed.). Learning from Media: Arguments, Analysis and
Evidence (ch. 9). Greenwich, CT: Information Age Publishers
Moundridou, M., & Virvou, M. (2002). Evaluating the persona effect of an interface
agent in a tutoring system. Journal of Computer Assisted Learning, 18(3), 253-
261
Mousavi, S. Y., Low, R., & Sweller, J. (1995). Reducing cognitive load by mixing
auditory and visual presentation modes. Journal of Educational Psychology, 87,
319-334.
Nass, C., & Steuer, J. (1993). Anthropomorphism, agency, and ethopoeia: Computers as
social actors. Human Communication Research, 19(4), 504-527.
Office of Technology Assessment. (1988, September). Power on: New tools for teaching
and learning. Retrieved June 30, 2002, from [Link]
disk2/1988/8831_n.html
Perkins, D. N., & Grotzer, T. A. (1997). Teaching intelligence. American Psychologist,
52(10), 1125-1133.
Pintrich, P. R. and Schunk, D. H. (2002). Motivation in Education: Theory, Research and
Applications (2nd Ed.). Englewood Cliffs, NJ: Prentice Hall.
Richardson, J. T. (2000). Researching students’ learning: Approaches to studying in
campus-based and distance learning. Buckingham, UK: Society for Research
into Higher Education and Open University Press.
Richardson, J. A., & Turner, T. E. (2000). Field dependence revisited I: Intelligence.
Educational Psychology, 20(3), 255-270.
Riding, R. J., & Cheema, I. (1991). Cognitive styles: An overview and integration.
Educational Psychology, 11, 193-215.
Ryokai, R., Vaucelle, C., & Cassell, J. (2003). Virtual peers as partners in storytelling
and literacy learning. Journal of computer assisted learning, 19, 195-208.
Salomon, G. (1984). Television is “easy” and print is “tough”: The differential
investment of mental effort in learning as a function of perceptions and
attributions. Journal of Educational Psychology, 76(4), 647-658.
Sampson, D., Karagiannidis, C., & Kinshuk. (2002). Personalised learning: Educational,
technological and standardisation perspective. Interactive Educational
Multimedia, 4, 24-39.
Schramm, W. (1977). Big media, little media. Beverly Hills, CA: Sage.
Shute, V.J., & Glaser, R. (1990). A large-scale evaluation of an intelligent discovery
world: Smithtown. Interactive Learning Environments, 1, 51-77.
Stahl, S.A. (1999). Different strokes for different folks? A critique of learning styles.
Five Principles 23

American Educator, 23(3), 27-31.

Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive
Science, 12, 257-285.
Sweller, J. (1989). Cognitive technology: Some procedures for facilitating learning and
problem solving in mathematics and science. Journal of Cognitive Psychology,
81(4), 457-466.
Sweller, J. (1999). Instruction design in technical areas. Camberwell, Australia: ACER.
Sweller, J., & Chandler, P. (1994). Why some material is difficult to learn. Cognition and
Instruction, 12, 185-233.
Sweller, J., Chandler, P., Tierney, P., & Cooper, M. (1990). Cognitive load as a factor in
the structuring of technical material. Journal of Experimental Psychology:
General, 119(2), 176-192.
Sweller, J., van Merriënboer, J. G., & Paas, F. G. (1998). Cognitive architecture and
instructional design. Educational Psychology Review, 10, 251-296.
Tuovinen, J. E., & Sweller, J. (1999). A comparison of cognitive load associated with
discovery learning and worked examples. Journal of Educational Psychology,
91(2), 334-341.
van Merriënboer, J. J. G. (1997). Training complex cognitive skills: A four-component
instructional design model for technical training. Englewood Cliffs, NJ:
Educational Technology Publications.
van Merriënboer, J. G., Kirschner, P. A., & Kester, L. (2003). Taking the load off a
learner's mind: Instructional design for complex learning. Educational
Psychologist, 38(1), 5-13.
Veenman, M.V.J. (1993). Intellectual ability and metacognitive skill: Determinants of
discovery learning in computerized learning environments. Unpublished
dissertation. Amsterdam: University of Amsterdam.
Veenman, M. V. J., & Elshout, J. J. (1995). Differential effects of instructional support
on learning in simulation environments. Instructional Science, 22, 363-383.