Developmental Research

The purpose of developmental research is to assess changes over an extended period of time. For
example, developmental research would be an ideal choice to assess the differences in academic and
social development in low-income versus high-income neighborhoods. It is most common when
working with children as subjects for obvious reasons and can be undertaken using several methods:
longitudinal, cross sectional, and cross sequential.

Longitudinal Studies.

Longitudinal studies assess changes over an extended period of time by looking at the same groups of
subjects for months or even years. Looking at academic and social development, we may choose a small
sample from each of the low- and high-income areas and assess them on various measures every six
months for a period of ten years. The results of longitudinal studies can provide valuable qualitative and
quantitative data regarding the differences in development between various groups.

The major concern with longitudinal research, aside from the obvious lack of control, randomization,
and standardization, is the length of time it takes to complete the study. Imagine starting a project that
must be constantly maintained for a period of ten or more years. The subject mortality rate due to
illness, relocation, and other factors alone could result in major concerns, not to mention the amount of
energy and time that must be devoted to the research.

Cross Sectional Studies.

One way to reduce the amount of time and the mortality rate in a developmental study is to assess
different ages at the same time rather than using the same groups over an extended period. A cross
sectional study might look at the same theory regarding academic and social development but assess a
small group of three year olds, six year olds, nine year olds and 12 year olds at the same time.

The assumption is that the differences between the age ranges represent natural development and that
of a longitudinal study had been used, similar results would be found. The obvious benefit is in the
length of time it takes to complete the study, but the assumptions that the six year old group will
achieve the same academic and social development as the nine year old group can be invalid.
Cross Sequential Studies.

Cross sequential studies combine both longitudinal and cross sectional methods in an attempt to both
shorten the length of the research and minimize developmental assumptions. For this method, groups
of different age children (three, six, and nine for example) may be studied for a period of three years to
both assess developmental changes and assure that the typical three year old is similar to the typical 6
year old after three years of development.
DEVELOPMENTAL RESEARCH: STUDIES OF

INSTRUCTIONAL DESIGN AND DEVELOPMENT

Rita C. Richey

Wayne State University

James D. Klein

Arizona State University

Wayne A. Nelson

Southern Illinois University at Edwardsville

41.1 INTRODUCTION

The field of instructional technology has traditionally involved

a unique blend of theory and practice. This blend is most ob-

vious in developmental research, those studies that involve the

production of knowledge with the ultimate aim of improving

the processes of instructional design, development, and evalu-

ation. Such research is based on either situation-specific prob-

lem solving or generalized inquiry procedures. Developmental

research, as opposed to simple instructional development, has

been defined as “the systematic study of designing, developing

and evaluating instructional programs, processes and products

that must meet the criteria of internal consistency and effec-

tiveness” (Seels & Richey, 1994, p. 127). In its simplest form,

developmental research can be either

rthe study of the process and impact of specific instructional

design and development efforts; or

ra situation in which someone is performing instructional de-

sign, development, or evaluation activities and studying the

process at the same time; or

rthe study of the instructional design, development, and evalu-

ation process as a whole or of particular process components.

In each case the distinction is made between performing a pro-

cess and studying that process. Reports of developmental re-

search may take the form of a case study with retrospective

analysis, an evaluation report, or even a typical experimental

research report.

The purposes of this chapter are to

rexplore the nature and background of developmental re-

search;

rdescribe the major types of developmental research by exam-

ining a range of representative projects;

ranalyze the methodological approaches used in the various

types of developmental research;

rdescribe the issues, findings, and trends in recent develop-

mental research; and

rdiscuss the future of this type of research in our field.

41.2 THE NATURE OF DEVELOPMENTAL

RESEARCH

Today, even amid the calls for increased use of alternative re-

search methodologies, the notion of developmental research is

1099

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

1100 •RICHEY, KLEIN, NELSON

often unclear, not only to the broader community of educational

researchers, but to many instructional technology researchers

as well. An understanding of this topic is rooted in the nature

of development and research in general, as well as a more spe-

cific understanding of the purpose, focus, and techniques of

developmental research itself.

41.2.1 The Character of Development

Development, in its most generic sense, implies gradual growth,

evolution, and change. This concept has been applied to diverse

areas of study and practice. For example, organization develop-

ment is a strategy for changing “the beliefs, attitudes, values,

and structure of organizations so that they can better adapt to

new...challenges”(Bennis, 1969, p. 2). Educators are familiar

with the notion of professional or staff development. Lieberman

and Miller (1992) define this as “the knowledge, skills, abilities,

and the necessary conditions for teacher learning on the job”

(p. 1045). This same concept is often applied to other profes-

sional areas. In the corporate arena, the term “executive de-

velopment”also refers to learning processes, and in this setting

learning, as a developmental activity, often integrates both class-

room instruction and work experience (Smith, 1993). The most

common use of the term “development,”however, is in relation

to human growth and the field of developmental psychology.

The term developmental research is most often confused with

research in this field that concentrates on particular age groups,

such as in the areas of adolescent development or life-span de-

velopment.

In the field of instructional technology, development has a

particular, somewhat unique, connotation. The most current

definition views development as “the process of translating the

design specifications into physical form”(Seels & Richey, 1994,

p. 35). In other words, it refers to the process of producing

instructional materials. Development is viewed as one of the five

major domains of theory and practice in the field.1Even though

this varies from many other uses of the term development, it is

consistent with the fundamental attribute of being a process of

growth, and in our field development is a very creative process.

Historically development has been an ambiguous term to

many instructional technologists and has generated consider-

able discussion regarding its proper interpretation. This de-

bate has focused typically on the distinctions between instruc-

tional design and instructional development. Heinich, Molenda,

Russell, and Smaldino (2002) define instructional development

as “the process of analyzing needs, determining what content

must be mastered, establishing educational goals, designing ma-

terials to reach the objectives, and trying out and revising the
program in terms of learner achievement”(p. 445). They have

been consistent in this orientation since the early editions of

their influential book. Yet to many, this is a definition of the

instructional systems design (ISD) process.

The confusion has been further exacerbated. In 1977, Briggs

defined instructional design as “the entire process of analysis of

1In addition to development, the domains also include design, management, utilization, and evaluation.

learning needs and goals and the development of a delivery sys-

tem to meet the needs; includes development of instructional

materials and activities; and tryout and revision of all instruction

and learner assessment activities”(p. xx). In this interpretation

design is the more generic term, encompassing both planning

and production. The 1994 definition of the field attempts to clar-

ify these issues by viewing design as the planning phase in which

specifications are constructed, and development as the produc-

tion phase in which the design specifications are actualized

(Seels & Richey, 1994). This is not a new distinction (Cronbach

& Suppes, 1969; National Center for Educational Research and

Development, 1970), even though the past use of the term in-

structional developer (see Baker, 1973) typically referred to a

person who was doing what today we would call both design

and development. All would agree, however, that design and de-

velopment are related processes, and Connop-Scollard (1991)

has graphically demonstrated these relationships in a complex

chart which identified hundreds of interrelated concepts.

However, the word development has a broader definition

when it is used within the research context than it has when

used within the context of creating instructional products. The

focus is no longer only on production, or even on both planning

and production. It also includes comprehensive evaluation. As

such, developmental research may well address not only for-

mative, but also summative and confirmative evaluation. It may

address not only needs assessment, but also broad issues of front-

end analysis, such as contextual analysis issues as conceived by

Tessmer and Richey (1997). When evaluation is approached in

a comprehensive manner, the scope of the research effort is

often correspondingly expanded to encompass product utiliza-

tion and management, as well as product creation. Table 41.1

displays the scope of development as discussed in this chapter.

The next step beyond “Utilization & Maintenance”in the

Table 41.1 schemata would be “Impact,”the follow-up analysis

of the effects of an instructional product or program on the

organization or the learner. This type of research typically falls

within the scope of traditional evaluation research.

41.2.2 The Character of Research

Although research methodologies vary, there are key attributes

that transcend the various research orientations and goals. An

understanding of these characteristics can shed light on the

process of developmental research.

[Link] The Dimensions of Research. Research is typi-

cally viewed as a type of systematic investigation, and in edu-

cation it typically is an empirical process that employs the sys-

tematic method (Crowl, 1996). One result of such efforts is the

creation of knowledge. Even though research is typically rooted

in societal problems, all knowledge produced by research is not

necessarily in a form conducive to quick resolution of society’s

problems. Some knowledge (which is usually generated by basic

research) must be specifically transformed to enable its appli-

cation to a given problem. Other knowledge (which is usually

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

41. DEVELOPMENTAL RESEARCH •1101

TABLE 41.1. The Scope of Development in a Research Context

Utilization &

Design Development Maintenance

Analysis and Planning for Production & Formative Usage, Management,

Development, Evaluation, Evaluation Summative &

Utilization, & Maintenance Confirmative Evaluation

generated by applied research) lends itself to the immediate

solution of practical problems. Developmental research clearly

falls in the latter category. In this respect, it is similar to other

methodologies such as action research.

Although the objective of research is to produce knowledge,

these products may take a variety of forms. Diesing (1991) has

noted that

. . .the philosophers suggest that social science produces at least three

kinds of knowledge: (1) systems of laws which describe interconnected

regularities in society; (2) descriptions, from the inside, of a way of life,

community, person, belief system, or scientific community’s beliefs;

(3) structural models, mathematical or verbal, of dynamic processes

exemplified in particular cases. (p. 325)

Research orientations tend to conform to the pursuit of a

particular type of knowledge. Experimental research tends to

contribute to the construction of a system of laws. Character-

istic of this method is a series of routine checks to ensure self-

correction throughout the research, and in the logical positivist

tradition, such checks are considered to be rooted in objectiv-

ity (Kerlinger, 1964). Qualitative research primarily contributes

to the development of “mirrors for man”so that we can see

ourselves better (Kluckhohn, as noted in Diesing, 1991). In the

various forms of qualitative research, context and contextual in-

fluences become an integral part of the investigation (Driscoll,

1991; Mishler, 1979).

Diesing’s third type of knowledge is process knowledge pre-

sented in model form. This is usually of great interest to instruc-

tional designers and developers, given our history of working

with many kinds of process models, such as the graphic models

of systematic design procedures and the models of media se-

lection. When inquiry procedures result in this type of knowl-

edge, these endeavors can legitimately be placed in the research

realm.

Another traditional characterization of research is as a facilita-

tor of understanding and prediction. In this regard “understand-

ing results from a knowledge of the process or dynamics of a

theory. Prediction results from investigation of the outcomes of

a theory”(Schwen, 1977, p. 8). These goals can be achieved

by either (a) providing a logical explanation of reality, (b) an-

ticipating the values of one variable based on those of other

variables, or (c) determining the states of a model (Dubin, as

cited by Schwen, 1977). While these ends, especially the first

two, can be achieved through traditional research methodolo-

gies, the third is uniquely matched to the goals of developmental

research. This was emphasized by Schwen (1977):

Inquiry in educational technology may be associated with the planning,

implementing, and/or evaluation of the management-of-learning pro-

cess, where that process employs systematic technological analysis and

synthesis . . . current definitions of technological process may be found

in development models . . . (having) the common attributes of 1) dis-

ciplined analysis of problem, context, constraints, learners, and task,

and 2) disciplined synthesis involving the design of replicable forms of

instruction and formative and summative evaluation. (p. 9)

[Link] The Relationships Between Research and De-

velopment. The traditional view of research is as the discovery

of new knowledge and that of development is as the translation

of that knowledge into a useful form (Pelz, 1967). This concep-

tual framework not only has been commonly subscribed to, but

was subsequently extended into the research, development, and

diffusion model (Brickell, Clark & Guba, as cited in Havelock,

1971). Early research methods texts addressed development as

the “research and development”process (Borg & Gall, 1971).

The processes were separate, though related, dependent, and

sequential. In some situations this orientation still prevails. This

view emphasizes development’s function of linking practice to

research and theory and recognizes the likelihood that instruc-

tional development can highlight researchable problems and,

thus, serve as a vehicle for stimulating new research (Baker,

1973).

Stowe (1973) has shown the parallels between scientificre-

search and the general methodology of instructional systems

design (ISD). He notes that both

rare objective, empirical problem-solving approaches;

remploy procedural models “couched in the language of math-

ematics”(p. 167);

rhave a predictive power dependent on the degree to which

they represent the most critical aspects of reality; and

rcan generate new problems and hypotheses.

Nonetheless, Stowe rejected the proposition that systematic

design procedures can be viewed as research to a great extent

because of the inability of ISD to discover generalizable prin-

ciples and its intent to produce context-specific solutions. In

addition, Stowe cited the distinctions between ISD’s orienta-

tion toward explanations of “how”as opposed to research’s

orientation toward explanations of “why.”

In the contemporary orientation toward research, Stowe’s

arguments can be interpreted as an overly rigid expression of

positivist philosophy. The contextual richness of the typical de-

sign and development task increases the likelihood that research

on such topics be especially ripe for a qualitative orientation.

The ability to provide process explanations exemplifies a type of

knowledge production using the Diesing framework, and an av-

enue to understanding and prediction using Schwen’s paradigm.

Stowe’s arguments, drafted 30 years ago, could lead to diamet-

rically opposite conclusions today, providing one accepts the

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

1102 •RICHEY, KLEIN, NELSON

premise that research can have a broader function than the cre-

ation of generalizable statements of law. We are taking the posi-

tion that research can also result in context-specific knowledge

and can serve a problem-solving function. This is true of devel-

opmental research, as it has commonly thought to be true of

evaluation research.

While instructional development typically builds on previ-

ous research, developmental research attempts to produce the

models and principles that guide the design, development, and

evaluation processes. As such, doing development and studying

development are two different enterprises.

41.2.3 The Background of Developmental Research

The field of instructional technology as it exists today emerged

primarily from a convergence of the fields of audiovisual edu-

cation and instructional psychology. In audiovisual education

the emphasis was on the role of media as an enhancement of

the teaching/learning process and an aid in the communication

process, and there was much interest in materials production.

On the other hand, in instructional psychology the nature of

the learner and the learning process took precedence over the

nature of the delivery methodology, and there was much in-

terest in instructional design. Complementing the instructional

psychology roots was the application of systems theory to in-

struction that resulted in the instructional systems design move-

ment (Seels & Richey, 1994). This conceptual and professional

merger came to fruition in the 1960s and 1970s. During this

period instructional design and development came to assume

the role of the “linking science”that John Dewey had called for

at the turn of the century (Reigeluth, 1983).

Not surprisingly, it was during this same period that the

term developmental research emerged. This new orientation

was exemplified by the shift in topics between the first and

the second Handbook of Research on Teaching (Gage, 1963;

Travers, 1973). In the 1963 handbook, media was addressed as

an area of research with a major emphasis on media compari-

son research, and all research methodologies considered were

quantitative. In the 1973 handbook, media continued to be in-

cluded as a research area, but the research methodologies were

varied, including Eva Baker’s chapter on “The Technology of In-

structional Development.”2This chapter describes in detail the

process of systematic product design, development, and eval-

uation. Of significance is the fact that the entire methodology

section was titled “Methods and Techniques of Research and

Development”.

This was a period in which federal support of educational

research mushroomed. Regional research and development lab-

oratories were established and the ERIC system was devised for

dissemination. Clifford (1973) estimated that appropriations for

educational “research and development for 1966 through 1968

alone equaled three-fourths of all funds ever made available”

(p. 1). Research-based product and program development had

2A history of instructional development is given by Baker (1973), who primarily summarizes the work in
research-based product development from

the turn of the century to 1970. Baker, however, does not address developmental research as it is
presented in this chapter.

become firmly established as part of the scientific movement in

education. At this time, Wittrock (1967) hailed the use of em-

pirical measurement and experimentation to explain product

effectiveness. Such activities “could change the development of

products into research with empirical results and theory gener-

alizable to new problems”(p. 148).

Hilgard (1964) characterized research as a continuum from

basic research on topics not directly relevantto lear ning through

the advocacy and adoption stages of technological develop-

ment. Saettler (1990) maintained that the last three of Hil-

gard’s research categories were directly within the domain of

instructional technology. These included laboratory, classroom,

and special teacher research; tryout in “normal”classrooms;

and advocacy and adoption. Note that these are portrayed as

types of research, rather than applications of research, and they

are all encompassed within the framework of developmental

research.

Although instructional technology is not the only field con-

cerned with learning in applied settings, few would dispute

the critical role played by these three types of research in our

field. Moreover, our uniqueness among educational fields is not

only our concern with technology, but rather our emphasis on

the design, development, and use of processes and resources for

learning (Seels & Richey, 1994). Given this definition of the field,

developmental research is critically important to the evolution

of our theory base.

41.2.4 The Character of Developmental Research

The distinctions between “doing”and “studying”design and

development provide further clarification of developmental re-

search activities. These distinctions can be described in terms

of examining the focus, techniques, and tools of developmental

research.

[Link] The Focus of Developmental Research. The gen-

eral purposes of research have been described as knowledge

production, understanding, and prediction. Within this frame-

work, developmental researchhas par ticular emphases that vary

in terms of the extent to which the conclusions are generalizable

or contextually specific. Table 41.2 portrays the relationships

between the two general types of developmental research.

The most straightforward developmental research projects

fall into the first category in Table 41.2. This category typically

involves situations in which the product development process

used in a particular situation is described and analyzed and

the final product is evaluated, such as McKenney’s (2002)

documentation of the use of CASCADE-SEA, a computer-based

support tool for curriculum development. Driscoll (1991) has

used the term systems-based evaluation to describe a similar

research paradigm. van den Akker (1999), on the other hand,

prefers to label Type I research formative research. He further

defines this type of research as “activities performed during

the entire development process of a specific intervention,

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

41. DEVELOPMENTAL RESEARCH •1103

TABLE 41.2. A Summary of the Two Types

of Developmental Research

Type 1 Type 2

Emphasis Study of specific product or

program design,

development, &/or

evaluation projects

Study of design,

development, or

evaluation processes,

tools, or models

Product Lessons learned from

developing specific

products and analyzing

the conditions that

facilitate their use

New design, development,

and evaluation

procedures &/or models,

and conditions that

facilitate their use

Context-specific Generalized

Conclusions ⇒⇒⇒ ⇒Conclusions

from exploratory studies through (formative and summative)

evaluation studies”(p. 6).

Some Type 1 developmental studies reflect traditional eval-

uation orientations in which the development process is not

addressed, and only the product or program evaluation is de-

scribed. An example of this type of study is O’Quin, Kinsey, and

Beery’s (1987) report on the evaluation of a microcomputer

training workshop for college personnel. Regardless of the na-

ture of the Type 1 study, the results are typically context and

product specific, even though the implications for similar situ-

ations may be discussed.

The second type of developmental study is oriented to-

ward a general analysis of design, development, or evaluation

processes, addressed either as a whole or in terms of a par-

ticular component. They are similar to those studies Driscoll

(1991) calls “model development and technique development

research.”van den Akker (1999) calls them “reconstructive stud-

ies.”His term emphasizes the common (but not exclusive) situ-

ation in which this research takes place after the actual design

and development process is completed.

Tracey’s (2002) study is an example of Type 2 developmental

research that has a global design orientation. She constructed

and validated an instructional systems design model that incor-

porated Gardner’s notion of multiple intelligences. Taylor and

Ellis’(1991) study had a similar orientation, although their orien-

tation was far different. They evaluated the use of instructional

systems design in the Navy. Other studies in this category fo-

cus on only one phase of the design/development/evaluation

process, such as Jonassen’s (1988) case study of using needs

assessment data in the development of a university program.

Type 2 research may draw its population either from one target

project such as King and Dille’s (1993) study of the application

of quality concepts in the systematic design of instruction at

the Motorola Training and Education Center or from a variety of

design and development environments. Examples of the latter

approach include Riplinger’s (1987) survey of current task analy-

sis procedures and Hallamon’s similar study conducted in 2001.

Typically, conclusions from Type 2 developmental research are

generalized, even though there are instances of context-specific

conclusions in the literature.

[Link] Nondevelopmental Research in the Field. A crit-

ical aspect of any concept definition is the identification of

nonexamples as well as examples. This is especially important

with respect to developmental research, as it often seems to

overlap with other key methodologies used in the field. Even

so, developmental research does not encompass studies such

as the following:

rinstructional psychology studies,

rmedia or delivery system comparison or impact studies,

rmessage design and communication studies,

rpolicy analysis or formation studies, and

rresearch on the profession.

While results from research in these areas impact the de-

velopment process, the study of variables embedded in such

topics does not constitute developmental research. For exam-

ple, design and development is dependent on what we know

about the learning process. We have learned from the research

literature that transfer of training is impacted by motivation,

organizational climate, and previous educational experiences.

Therefore, one may expand a front-end analysis to address such

issues, or even construct design models that reflect this infor-

mation, but the foundational research would not be considered

developmental. If the new models were tested, or programs de-

signed using such models were evaluated, this research would

qualify as developmental.

A fundamental distinction should be made between reports

that analyze actual development projects and descriptions of

recommended design and development procedural models. Al-

though these models may represent a synthesis of the research,

they do not constitute research in themselves. A good example

of the latter situation is Park and Hannafin’s (1993) guidelines

for designing interactive multimedia. These guidelines are gen-

eralized principles that speak to the development process, and

they are based on a large body of research. Nonetheless, the

identification and explanation of the guidelines is not in itself

an example of developmental research. The instructional tech-

nology literature includes many examples of such work. They of-

ten provide the stimulus for a line of new research, even though

these articles themselves are not considered to be research re-

ports themselves. There are many examples today of such work,

including explorations of topics such as cognitive task analysis

(Ryder & Redding, 1993) and the nature of design and designer

decision making (Rowland, 1993).

[Link] The Techniques and Tools of Developmental

Research. Developmental researchers employ a variety of re-

search methodologies, applying any tool that meets their re-

quirements. Summative evaluation studies often employ classi-

cal experimental designs. Needs assessments may incorporate

qualitative approaches. Process studies may adopt descriptive

survey methods. Even historical research methods may be used

in developmental projects.

Traditional research tools and traditional design tools facil-

itate the developmental endeavor. Expertise is often required

in statistical analysis, measurement theory, and methods of

establishing internal and external validity. Likewise, the devel-

opmental researcher (even those studying previously designed

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

1104 •RICHEY, KLEIN, NELSON

instruction) requires a command of design techniques and

theory. Additional design proficiency is frequently required

when using electronic design systems and aids, conducting

environmental analyses, and defining ways to decrease design

cycle time.

A developmental research project may include several dis-

tinct stages, each of which involves reporting and analyzing a

data set. Merely conducting a comprehensive design and devel-

opment project does not constitute conducting a developmen-

tal research project even using its most narrow Type 1 defini-

tion. One must also include the analysis and reporting stage to

warrant being classified as developmental research.

Developmental research projects may include a number of

component parts. Substudies may be conducted to analyze and

define the instructional problem, to specify the content, or to

determine instrument reliability and validity. Substudies may be

conducted to provide a formative evaluation, a summative eval-

uation, or a follow-up of postinstruction performance. Conse-

quently, reports of developmental research are frequently quite

long, often prohibiting publication of the full study.

Reports of developmental projects can often be found in

rprofessional journals,

rdoctoral dissertations,

rEducational Resource Information Center (ERIC) collections

of unpublished project reports, and

rconference proceedings.

The nature of these reports varies depending on the dis-

semination vehicle. Sometimes, full developmental projects are

split into more easily publishable units (or even summarized) to

facilitate publication in the traditional research journals. Devel-

opmental research reports are also published in practitioner-

oriented journals and magazines, and the methodology and

theoretical base of the studies are omitted to conform to the

traditions of those periodicals. The next section further defines

the nature of developmental research by summarizing studies

that are representative of the wide range of research in this

category.

41.3 A REVIEW OF REPRESENTATIVE

DEVELOPMENTAL RESEARCH

We have identified representative developmental research stud-

ies in the literature so that we might

rfurther describe the character of this type of research,

ridentify the range of settings and foci of developmental re-

search,

rsummarize developmental tools and techniques commonly

used,

ridentify the range of research methodologies used, and

rdescribe the nature of the conclusions in such research.

The literature is described in terms of the two types of devel-

opmental research. This review covers research from the past

20 years, with a concentration on the most recent work.

41.3.1 Type 1 Developmental Research

Type 1 research is the most context-specific inquiry. These stud-

ies are essentially all forms of case studies and emanate from a

wide range of educational needs. Table 41.3 presents an analysis

of 56 studies representative of this category of research. These

studies are described in terms of their focus, their methodology,

and the nature of their conclusions. Focus is examined in terms

of the

rtype of program or product developed;

rparticular design, development, or evaluation process empha-

sized in the study;

rparticular tools and techniques emphasized; and

rorganizational context for which the product is intended.

The most common characteristics among the studies are

found in relation to their process foci, the research methodolo-

gies employed, and the nature of their conclusions. The product

and technique focus and the user context seem to reflect indi-

vidual researcher interests or participant availability more than

they reflect the inherent nature of this type of research.

[Link] Process Foci of Type 1 Developmental Re-

search. Type 1 research studies originate with the design and

development of an instructional product or program. This is the

crux of Type 1 research. Frequently, the entire design, develop-

ment, and evaluation process is documented. Consistent with

predominant practice in the field, the procedures employed

usually follow the tenets of ISD, encompassing front-end analy-

sis through formative or summative evaluation. One-third of the

studies cited in Table 41.3 describe in detail the entire design,

development, and evaluation process as it occurred in a partic-

ular environment. (See Table 41.3 for studies classified in terms

of an “A”process focus.)

Two studies provide an example of this kind of research.

Petry and Edwards’s (1984) description of the systematic de-

sign, development, and evaluation of a university applied pho-

netics course is a classic Type 1 study with a course focus. They

describe the application of a particular ISD model as well as

the use of elaboration theory in content sequencing. The study

also addresses the production of course materials, as well as the

results of an evaluation of student performance and attitudes in

the revised course. Hirumi, Savenye, and Allen’s (1994) report

of the analysis, design, development, implementation, and eval-

uation of an interactive videodisc museum exhibit is an example

of a Type 1 study with a program focus. This study provides evi-

dence that an ISD model can be adapted to informal educational

settings.
Studies that did not document the entire design, develop-

ment, and evaluation process tended to emphasize a particular

phase of ISD such as needs assessment or formative evaluation.

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

41. DEVELOPMENTAL RESEARCH •1105

For example, Link and Cherow-O’Leary (1990) document the

needs assessment procedures followed by the Children’s Tele-

vision Workshop (CTW) to determine the needs and interests

of elementary school teachers. They also describe formative

evaluation techniques for testing the effectiveness of children’s

and parent’s magazines. Klein et al. (2000) describe a needs

assessment conducted to determine the optimal instructional

content and delivery method for an introductory course in ed-

ucational technology. The results of the needs assessment were

used to revise an existing course. Fischer, Savenye, and Sullivan

(2002) report on the formative evaluation of a computer-based

training course for an on-line financial and purchasing system.

The purpose of the study was to evaluate the effectiveness of

the training and to identify appropriate revisions to incorporate

into the program.

Type 1 developmental research studies often concentrate on

the production aspect of the ISD approach. (See Table 41.3 for

studies classified in terms of an “E”process focus.) Often these

studies concerned the development of technology-based in-

struction, such as Bowers and Tsai’s (1990), Crane and Mylonas’

(1988), and Harris and Cady’s (1988) research on the use of hy-

pertext as a vehicle for creating computer-based instructional

materials. The reports describe authoring procedures so specif-

ically that one could replicate the innovative development pro-

cesses. These same tactics were employed with respect to de-

veloping instructional television by Albero-Andes (1983), and

they were used in interactive videodisc projects by Alessi (1988)

and C. M. Russell (1990). Similar studies can be found in the

research of our field as each new technological advancement

emerges.3

Type 1 developmental studies demonstrate the range of de-

sign and development procedures currently available to practi-

tioners. In addition, these studies commonly encompassed an

evaluation of the products and programs that were created, in-

cluding an examination of the changes in learners who had in-

teracted with the newly developed products.

[Link] Research Methodologies Employed in Type 1

Developmental Research. A basic premise of this chapter is

that the design–development–evaluation process itself can be

viewed as a form of inquiry. This is accomplished in a devel-

opmental research project by embedding traditional research

methodologies into the development projects. The type of re-

search method used in a particular project tends to vary with the

type of developmental research, and Type 1 studies frequently

employ case study techniques. (See Table 41.3 for studies clas-

sified in terms of an A methodology use.)

The manner in which case study techniques are used varies

widely in developmental research. Most commonly, the case

study is seen as a way in which one can explore or describe com-

plex situations, which consist of a myriad of critical contextual

3Although this is not the emphasis of this chapter, one could construct a type of history of the field of
instructional technology by tracing the
Type 1 developmental research. The audiovisual emphasis could be documented through a series of
studies such as Greenhill’s (1955) description

of producing low-cost motion pictures with sound for instructional purposes. Procedural changes could
be traced through studies such as those

by Rouch (1969) and Gordon (1969), who describe early applications of the systems approach to
designing education and training programs.

Programmatic trends could be seen in studies of the design and development of competency-based
teacher education programs, such as Cook and

Richey (1975). Today the development of new technologies is documented in a variety of studies, many
of which are discussed here.

variables as well as process complexities. For example, Dick

(1991) described a corporate design project in Singapore aimed

at training instructional designers. The report of his Type 1 study

focuses upon the needs assessment and actual design phases

rather than the entire design–development–evaluation process.

In another Type 1 study, Carr-Chellman, Cuyar, and Breman

(1998) document how user design was implemented to cre-

ate an information technology system in the context of home

health care. This case study explored the complexities of sys-

temic change as the organization used an educational systems

design process. Both of these projects were approached from a

research point of view in a qualitative manner.

Less often do studies heed the admonition of Yin (1992),

who indicates that case studies are more appropriate when one

is attempting to establish causal relationships rather than sim-

ply provide detailed descriptions. This approach requires a more

quantitative orientation to the development case study as one

seeks to explain (as well as describe) the nature of the devel-

opment process. More often quantitative aspects of a Type 1

developmental research project concern efforts to determine

the effectiveness or impact of the resulting instruction. In these

cases the studies have experimental designs, with or without

the case study component. (See Table 41.3 for studies classified

in terms of an E methodology use.) An example of this method-

ological approach is the study by Plummer, Gillis, Legree, and

Sanders (1992), which used two research methodologies—case

study and experimental—in conjunction with the design and

development task. This project involved the development of a

job aid used by the military when operating a complicated piece

of communications equipment. The experimental phase of the

evaluation of the job aid was a study to evaluate the effectiveness

of the job aid. Three instructional situations were compared—

using the job aid alone, using it in combination with a demon-

stration, and using the technical manual in combination with a

demonstration. Consequently, not only was impact information

secured, but also information relating to the superior conditions

for using the newly developed product was obtained.

Evaluation methods are frequently used in Type 1 devel-

opmental research studies (see Table 41.3 for studies classi-

fied as a “D”methodology use). Evaluation instruments such

as learner surveys, achievement tests, and performance mea-

sures are often used to collect data. For example, Martin and

Bramble (1996) conducted an in-depth evaluation of five video

teletraining courses for military personnel by collecting sur-

vey data from students and instructors who participated in

the courses. Achievement and proficiency tests were also used.

Quinn (1994) evaluated a curriculum project in which graduate

students served as instructional developers in a corporate envi-

ronment. He used learner, instructor, and client evaluations to

determine the impact of the project. Sullivan, Ice, and Nieder-

meyer (2000) field-tested a K–12 energy education curriculum

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

TABLE 41.3. An Analysis of Representative Type 1 Developmental Research Literature

Product or Tools and Techniques Research

Reference Program Focus Process Focus Use Context Emphasized Method(s) Used Nature of Conclusions

Albero-Andes (1983) C, F, I A, E, F ABAB

Alessi (1988) J A, H B H, K A C

Aukerman (1987) J G, J D H D, G, I A

Bednar (1988) BBBA,BAB

Blackstone (1990) H E, F, I, J F A, E, D, J A

Borras (1993) B, J, L D, F B K E B

Bowers & Tsai (1990) J E, G G K A, E D

Buch (1989) C A, J C B, F A, D, E B

Cantor (1988) D, E, F, G E E K A, D B

Capell & Dannenberg

(1993)

J C,D A,B F,G,L A,D B

Carr-Chellman, Cuyar, &

Breman (1998)

J D,F,G D L A B

Chou & Sun (1996) J A B,H L D A

Corry, Frick, & Hansen

(1997)

L B,F B B,L A A

Coscarelli & White (1982) A A, G, H B E A, D B

Coyle (1986) B, J G, J B A, G B

Coyle (1992) B E, G B D A
Crane & Mylonas (1988) J E B K A A

Cregger Metzler (1992) A A, G, H B G A, D, J B

Dabbagh et al. (2000) A A B A B

Dick (1991) B B, D C, H B, F, A A

Fischer, Savenye, &

Sullivan (2002)

J F GJ DB

Fox & Klein (2002) ABBBJ B

Gay & Mazur (1993) J E, H G K A, F C

Gettman, McNelly, &

Muraida (1999)

MDE DA

Gustafson & Reeves (1990) K A G B, C, D, E, H D C

Harris and Cady (1988) J E A K A A

Herrington & Oliver (2000) J, L D, E, G B K I B

Hirumi, Savenye, & Allen

(1994)

BAF AB

Jonassen (1988) B B, D B B J A

Jones (1986) C J B A, C, E, J A

Kanyarusoke (1985) A, F A, B, C, J B A, B, J A

Kearsley (1985) J G I K D B

Klein et al. (2000) A B, C B B J A

Lewis, Stern, & Linn (1993) H, J E A A, B A

Li & Merrill (1991) J C, D I A, D C

1106

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

Link & Cherow-O’Leary

(1990)

D B,F A,F,G B,G,J B

Martin & Bramble (1996) AAEJ DB

McKenney (2001) J, M A A, H B, C, L G, I, J B

Medsker (1992) A, B A, H, J B, C, G A A, B D

Mendes & Mendes (1996) J D, E, F A, H D A

Mielke (1990) I A A, F A A

Munro & Towne (1992) J D, E A, B, E, G K A, D B

Nieveen & van den Akker

(1999)

J, M F I L A, G, J C

Noel (1991) B A, B A, H C A C

O’Quin, Kinsey, & Beery

(1987)

C, J I, J B, G D, E A

Petry & Edwards (1984) AABGAA

Pirolli (1991) J C, D I F, G, K A, D B

Pizzuto (1983) C, H A, C, E, F, I, J C A, D, G, J A

Plummer, Gillis, Legree, &

Sanders (1992)

G E,I C A,E,J A

Quinn (1994) A A B, C, G J D D

Ross (1998) A G B, C I A

Russell (1990) J A, E, F, G, J B D, E D, E B

Russell (1988) J C, D I F, G, K A, D B

Sullivan (1984) B D, F, G A A C

Sullivan, Ice, &

Niedermeyer (2000)

B D,F,G A J D D
Watson & Belland (1985) E

A. Full course

B. Full program

C. Workshop

D. Gen. print mat.

E. Instruct. module

F. Study guide

G. Job aid

H. Games/simulation

I. Instruct. television

J. Computer-based

K. Any project

L. Multimedia/Web-

based

M. Performance

support system

A. Gen. des/devel./

evaluation proc.

B. Needs assessment

C. Content selection

D. Design/development

E. Production

F. Form. evaluation/

usability

G. Implementation/

utilization/delivery

H. Management

I. Summative

evaluation
J. Learner outcomes

K. No dev. involved

A. K–12 schools

B. Postsecondary

C. Business and

industry

D. Health care

E. Military and govt.

F. Continuing and

community educ.

G. Employee

trng.—other

H. International

I. Context-free

A. Problem analysis

B. Needs assessment

C. Environm’tal anal.

D. Job analysis

E. Task analysis

F. Learning hierarchy

G. Sequencing

H. Cost analysis

I. Dissemination

J. Learner verification

K. Specific technique

L. Prototype/usability

testing

A. Case study

B. Descriptive
C. Ethnography

D. Evaluation

E. Experimental

F. Historical

G. Observational

H. Philosophical

I. Qualitative

J. Survey

K. Other

A. Context/product

specific

B. Context/product

specific with some

generalization

C. Generalizations

with some

context/product

specific

D. Generalizations

1107

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

1108 •RICHEY, KLEIN, NELSON

by implementing student and teacher attitude surveys and stu-

dent achievement tests. In most cases, Type 1 developmental

studies using evaluation research methods employ several tech-

niques for collecting data.

[Link] The Nature of Conclusions from Type 1 Devel-

opmental Research. Type 1 studies are characterized by their

reliance on contextually specific projects and contextually spe-

cific conclusions that emerge from such research. This is consis-

tent with the more qualitative orientation of the research, with

the applied nature of the research, and with the field’s history

of using practitioner experience as a source of knowledge.

Over three-quarters of the Type 1 representative develop-

mental studies identified in Table 41.3 have conclusions that

are either exclusively directed toward the target product and

situation in which the project occurred or have predominantly

context-specific conclusions, even though they did generalize to

some extent to other situations. (See Table 41.3 for studies clas-

sified in terms of A and B types of conclusions.) These context-

specific conclusions address issues such as the following.

rSuggested improvements in the product or program (Albero-

Andres, 1983; Coyle, 1986; Corry, Frick, & Hansen, 1997;

Crane & Mylonas, 1988; Cregger & Metzler, 1992; Fischer,

Savenye, & Sullivan, 2002; Kanyarusoke, 1985; Klein et al.,

2000; Link & Cherow-O’Leary, 1990; Martin & Bramble, 1996;

McKenney, 2002; Mendes & Mendes, 1996; Munro & Towne,

1992; C. M. Russell, 1990)

rThe conditions that promote successful use of the product

or program (Borras, 1993; Dabbagh et al., 2000; Hirumi et

al., 1994; McKenney, 2002; Munro & Towne, 1992; Pizzuto,

1983; Quinn, 1994; Ross, 1998)

rThe impact of the particular product or program (Auker-

man, 1987; Blackstone, 1990; Cantor, 1988; Fischer et al.,

2002; Jones, 1986; Kearsley, 1985; Martin & Bramble, 1996;

Petry & Edwards, 1984; Pirolli, 1991; Plummer et al., 1992;

D. M. Russell, 1988)
rThe conditions that are conducive to efficient design, de-

velopment, and/or evaluation of the instructional product

or program (Albero-Andres, 1983; Bednar, 1988; Blackstone,

1990; Buch, 1989; Capell & Dannenberg, 1993; Chou & Sun,

1996; Coyle, 1992; Dick, 1991; Harris & Cady, 1988; Herring-

ton & Oliver, 2000; Jonassen, 1988; Jones, 1986; Kearsley,

1985; Link & Cherow-O’Leary, 1990; Martin & Bramble, 1996;

McKenney, 2002; Mielke, 1990; Munro & Towne, 1992; Petry

& Edwards, 1984; Pirolli, 1991; Quinn, 1994; C. M. Russell,

1990; D. M. Russell, 1988; Sullivan et al., 2000; Watson &

Bellend, 1985)

Even though these conclusions are directed toward a partic-

ular product or program, it is clear that the foundational design

and development procedures are as important as the product.

However, it is possible for these conclusions, even though they

are context specific, to provide direction to others who are con-

fronting similar design and development projects.

Some Type 1 studies did present conclusions that were di-

rected toward applications of a more general nature. It should

be noted that these studies (see C and D classifications, Table

41.3) were published in journals with a national audience. This

mayhaveinfluenced the manner in which the report was writ-

ten, or it may have been recommended by a reviewer or even

a condition of publication. In any case, many journal articles

tend to be less focused on context-specific conclusions than

dissertation research reports. The conclusions of these stud-

ies, especially those with a less formal quantitative design or

with a more descriptive qualitative design, tend to be presented

as “lessons learned”rather than as hypothesized relationships

or predictions. These generalized lessons are often supported

by and discussed in the context of current related literature as

well as the design, development, and evaluation project that

prompted them. The content of the conclusions in the studies

identified in Table41.3 with more generalized conclusions relate

to the conditions that are conducive to efficient design, develop-

ment, and/or evaluation of instructional products or programs

(Alessi, 1988; Bowers & Tsai, 1990; Gay & Mazur, 1993; Medsker,

1992; Noel, 1991; Quinn, 1994; Sullivan, 1984; Sullivan et al.,

2000).

[Link] Typical Type 1 Developmental Studies. This sec-

tion has described the nature of Type 1 developmental re-

search in terms of their focus, specifically examining the

design, development, and evaluation processes addressed, the

research methodologies employed, and the nature of their con-

clusions. While recent studies have been used to exemplify the

range of Type 1 research, thus far particular studies have not

been described in detail. Here, we summarize the research by

McKenney (2002) and Sullivan et al. (2000) as studies that are

representative of the group. These studies reflect the two key

Type 1 research formats; McKenney’s work was a doctoral dis-

sertation and Sullivan and his co-researchers’was a long-term

ID project published in a research journal.

The McKenney (2002) study examined the development of

a computer program to support curriculum materials devel-

opment in the context of secondary science and mathematics

education in southern Africa. As with any traditional research

project, the study was guided by predetermined questions and

was embedded into a conceptual framework supported by cur-

rent literature on curriculum development, teacher professional

development, and computer-based performance support. The

project was a careful and extensive documentation of the phases

of ISD—needs and context analysis, design, development, and

formative evaluation of several prototypes, and summative eval-

uation. Each phase included different expert and novice par-

ticipant groups; a total of 510 participants from 15 countries

was involved. Like many developmental research projects, the

researcher used several different types of instruments for data

collection including interviews, questionnaires, observations,

logbooks, and document analysis. However, this study included

more data collection than most; a total of 108 instruments was

employed. Also typical of many Type 1 studies, the conclusions

were focused on context-specific variables. McKenney (2002)

discusses the validity, practicality, and potential impact of the

computer-based performance support program. In addition, she

discusses the design principles followed and provides sugges-

tions for conducting developmental research.

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

41. DEVELOPMENTAL RESEARCH •1109

McKenney’s work (2002) is a detailed report of an extensive

study that includes both context-specific and some generalized

conclusions, although the clear focus of the study is on the de-

sign, development, and evaluation of a computer program to

support curriculum development in Africa. As it is a study com-

pleted as part of the requirements of a doctoral degree, length is

not an issue and the format was undoubtedly dictated by the de-

gree requirements. On the other hand, the Sullivan et al. (2000)

study was reported in a research journal. Consequently, it takes

on a different form, even though it too is representative of Type

1 developmental research.

Sullivan et al. (2000) describe the development and im-

plementation of a comprehensive K–12 energy education

curriculum. When the article was published, the program had

been implemented over a 20-year period and used by more than

12 million students in the United States. The report describes

the components of the program itself including instructional ob-

jectives and test items covering a variety of learning outcomes

such as facts, concepts, problem solving, and behavior change.A

description of the instructional materials is also provided. Field

test data such as posttest scores and student and teacher atti-

tudes are reported and the implementation phase of the project

is discussed in detail. Based on two decades of experience with

the program, the authors provide 10 guidelines for long-term in-

structional development projects. The conclusions and recom-

mendations are written in a more general tone; they are lessons

learned from a specific practical situation.

The McKenney (2002) dissertation and Sullivan et al. (2000)

article exemplify Type 1 developmental research. They are stud-

ies that

rdescribe and document a particular design, development, and

evaluation project;

remploy standard design and development procedures;

rutilize a range of research methodologies and data collection

instruments;

rdraw conclusions from their research which are context spe-

cific; and
rtend to serve as dissemination vehicles for exemplary design,

development, and/or evaluation strategies.

41.3.2 Type 2 Developmental Research

Type 2 developmental research is the most generalized of the

various orientations and typically addresses the design, develop-

ment, and evaluation processes themselves rather than a demon-

stration of such processes. The ultimate objective of this re-

search is the production of knowledge, often in the form of

a new (or an enhanced) design or development model. This

research tends to emphasize

rthe use of a particular design technique or process, such as

formative evaluation; or

rthe use of a comprehensive design, development, or evalua-

tion model; or

ra general examination of design and development as it is com-

monly practiced in the workplace.

Fifty-eight examples of Type 2 research have been analyzed

using the same framework previously employed in the analysis

of Type 1, and the results are presented in Table 41.4.

[Link] Process Foci of Type 2 Developmental Re-

search. We have previously distinguished between doing de-

velopment and studying development. In Type 1 research there

is a pattern of combining the doing and the studying in the pro-

cess of discovering superior procedures. However, Type 2 re-

search often does not begin with the actual development of an

instructional product or program. These studies are more likely

to focus on the more generic use of development processes,

offering implications for any design or development project.

Type 2 studies are more likely to concentrate upon a par-

ticular ISD process, rather than the more comprehensive view

of development. (See those studies in the Table 41.4 Process

Focus column that are labeled B through J as opposed to those

labeled A.) For example, Abdelraheem (1990) and Twitchell,

Holton, and Trott (2000) both studied evaluation processes, one

with respect to instructional module design and the other in ref-

erence to the practices of technical trainers in the United States.

But while Abdelraheem was involved in a specific design and

development project, Twitchell et al. were not. They studied

the extent to which technical trainers employ accepted eval-

uation methods and techniques. The key difference between

Type 1 and Type 2 studies that focus on a particular aspect of

the total process is that goals of Type 2 studies tend to be more

generalized, striving to enhance the ultimate models employed

in these procedures. Type 1 research, on the other hand, is more

confined to the analysis of a given project.

Type 2 developmental research projects span the entire

range of design and development process components from

needs assessment and performance analysis (Cowell, 2000;

Kunneman & Sleezer, 2000; Tessmer, McCann, & Ludvigsen,

1999) to evaluation (Le Maistre, 1998; Phillips, 2000; Twitchell

et al., 2000). In addition, there are studies that address design

processes in a more generic fashion. Sample studies of this na-

ture include those by Adamski (1998), Nelson (1990), Rowland

(1992), Rowland and Adams (1999), Tracey (2002), Visscher-

Voerman (1999), Klimczak and Wedman (1997), and Wedman

and Tessmer (1993). This research is part of the growing body of

literature contributing to an understanding of the nature of de-

signer decision making. It provides another topical orientation

in addition to those studies that examine the various compo-

nents of traditional instructional systems design models.

Although the majority of the studies cited in Table 41.4 do

not tackle the entire design and development process in a com-

prehensive fashion, some do. For example, Spector, Muraida,

and Marlino (1992) proposed an enhanced ISD model for use

in courseware authoring, which is grounded in cognitive the-

ory. This use of this model was then described and evaluated

in a military training environment. This is a good example of a

Type 2 study. Likewise, Richey (1992) proposed a modified ISD

procedural model that was based on an empirical examination

of those factors that influence employee training effectiveness.

This is another example of a comprehensive Type 2 study.

Recently, a few Type 2 studies have examined how ISD mod-

els can be applied in settings outside of traditional education

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

TABLE 41.4. An Analysis of Representative Type 2 Developmental Research Literature

Product or Tools and Techniques Research

Reference Program Focus Process Focus Use Context Emphasized Method(s) Used Nature of Conclusions

Abdelraheem (1990) E F, J A, H J, K E C

Adamski (1998) G A C A, J, K E, I, K, L C

Beauchamp (1991) K AABJ D

Burkholder (1981–82) E, A F B K E D

Capell (1990) J A B B, I D

Carliner (1998) D F I C

Cowell (2000) K BCBI D

Cummings (1985) K B C J D

Driscoll & Tessmer (1985) E, K D, J A G, J E D

English & Reigeluth (1996) D D B G D, I D

Ford & Wood (1992) K D I E, F, G D C

Forsyth (1997) B A F I, D, K C

Fuchs & Fuchs (1986) A, K F A J K D

Gauger (1987) B I B, D D C

Goel & Pirolli (1988) K A, K C G, I D

Hallamon (2001) K C I E J D

Harrison et al. (1991) B A, G, H B I, J C

Higgins & Reiser (1985) K D C E D

Jones & Richey (2000) EACA,JQD

Julian (2001) K A I I C

Keller (1987) A, K D A, I J E, J, K C

Kerr (1983) A, E A A, B J C

Kim (1984) K A H C

King & Dille (1993) K A C H A B

Klimczak & Wedman

(1997)

KAG I,JC

Kress (1990) A A C E, J D

Kunneman & Sleezer

(2000)

KKC AB

Le Maistre (1998) K F I I B

Le Maistre & Weston

(1996)

DF C E B

Luiz (1983) K A I H D

McAleese (1988) K D I E, F, G D C

Means, Jonassen, & Dwyer

(1997)

D, E J B E B

Nadolski, Kirschner, van

Merrienboer, &

Hummel (2001)

J D B E, G A, D, I C

Nelson (1990) K A, K G A, D, E G, I D

Nicolson (1988) K D, E B E, F, G D C

Phillips (2000) K I C K I D

Piper (1991) K H C J A D

Pirolli (1991) K D, E I E, F, G D D

Plass & Salisbury (2002) L A C, E L D C

1110

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

Richey (1992) C, K A, J C B, C I, J D

Riplinger (1987) K B, C J E B, J D

Rojas (1988) C I C D D

Rowland (1992) K A, K B, C, E G, I D

Rowland & Adams (1999) K A B, C J D

Roytek (1999) K D C L A, I, K B

Shambaugh & Magliaro

(2001)

AABJ AC

Shellnut (1999) K A A, B, C, D, E K J D

Spector, Muraida, &

Marlino (1992)

JA,EEKGB

Taylor & Ellis (1991) A A E D B

Tessmer, McCann, &

Ludvigsen (1999)

BBC,DBAD

Tracey (2002) K A I E, I, K D

Twitchell, Holton, & Trott

(2000)

KF,IC,G J D

Visscher-Voerman, (1999) K A I A, I D

Wagner (1984) A, K I B J, K E D

Wedman & Tessmer (1993) K A, K C J D

Weston, McAlpine, &

Bordonaro (1995)

KFI BD

Wreathall & Connelly

(1992)

K I,J E J,K A,J C

Zemke (1985) K A, K C J B

A. Full course

B. Full program

C. Workshop

D. Gen. print mat.

E. Instruct. module

F. Study guide

G. Job aid

H. Games/simulation

I. Instruct. television

J. Computer-based

K. Any project

L. Multimedia/Web-
based

M. Performance

support system

A. Gen. des/devel./

evaluation proc.

B. Needs assessment

C. Content selection

D. Design/development

E. Production

F. Form. evaluation/

usability

G. Implementation/

utilization/delivery

H. Management

I. Summative

evaluation

J. Learner outcomes

K. No dev. involved

A. K–12 schools

B. Postsecondary

C. Business and

industry

D. Health care

E. Military and govt.

F. Continuing and

community educ.

G. Employee

trng.—other

H. International
I. Context-free

A. Problem analysis

B. Needs assessment

C. Environmental anal.

D. Job analysis

E. Task analysis

F. Learning hierarchy

G. Sequencing

H. Cost analysis

I. Dissemination

J. Learner verification

K. Specific tech.

L. Prototype/usability

testing

A. Case study

B. Descriptive

C. Ethnography

D. Evaluation

E. Experimental

F. Historical

G. Observational

H. Philosophical

I. Qualitative

J. Survey

K. Other

A. Context/product

specific

B. Context/product

specific with some

generalization

C. Generalizations

with some

context/product

specific

D. Generalizations

1111

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

1112 •RICHEY, KLEIN, NELSON

and training. For example, Plass and Salisbury (2002) described

and evaluated a design model for a Web-based knowledge man-

agement system. Carliner (1998) conducted a naturalistic study

of design practices in a museum setting and proposed an en-

hanced model of instructional design for informal learning in

museums. Both of these studies included empirical data to sup-

port the use of the ISD approach to nontraditional products and

settings.

[Link] Research Methodologies Employed in Type 2

Developmental Research. While the case study served as the

dominant methodological orientation of Type 1 developmental

research, it is far less prominent in Type 2 studies. This is not

surprising given the fact that Type 2 research typically does not

involve a specific design and development project. There is a

much greater diversity of research methods employed in the

representative Type 2 studies identified in Table 41.4. Not only

are experimental and quasi-experimental studies common, but

this table even shows a study using philosophical techniques, as

well as a variety of qualitative methodologies. (See Table 41.4’s

Research Methods Used column.)

Even though case studies are not the overwhelming norm

in Type 2 research, they are nonetheless found. (See Table 41.4

for studies classified in terms of an A-type research method-

ology.) As with other developmental studies, case study tech-

niques are sometimes employed in Type 2 research, which

includes a description of the actual design and development

processes followed in the creation of a particular product or in

the demonstration of a particular process. Wreathall and Con-

nelly (1992) document the development of a method of using

performance indicators as a technique for determining training

effectiveness. In true Type 1 fashion they describe the technique

and the manner in which they applied it in a given context. This

facet utilizes case study methods. However, they extend their

study to verify the relevance of their technique through the use

of a structured survey interview of other professionals in the

field. (The latter phase of the research warrants its classification

as a Type 2 study.) The Wreathall and Connelly study is typi-

cal of the Type 2 research, which uses case study methods in

that the methods are used in combination with other research

techniques.

Experimental and quasi-experimental techniques can serve

a variety of functions in Type 2 developmental research. (See

Table 41.4 for studies classified in terms of an E-type research

methodology.) They are often used as a way of verifying the

procedure in terms of learner outcomes. Means, Jonassen, and

Dwyer (1997) examined the impact of embedding relevance

strategies into instruction following the ARCS model by using ex-

perimental design to determine their influence on achievement

and motivation. Keller (1987) used quasi-experimental methods

to test the impact of the ARCS model as it was applied to the de-

velopment of teacher in-service training workshops. Le Maistre

and Weston (1996) used a counterbalanced design to exam-

ine the revision practices of instructional designers when they

are provided with various types of data sources. In these studies,

the experimentation is critical to the verification and evaluation

of a particular design and development technique. This is not un-

like the use of experimental and quasi-experimental techniques

in other types of research.

Surveys are frequently used in Type 2 developmental studies.

In Type 1 studies surveys were typically another means of de-

termining product impact or effectiveness with learners and/or

instructors in a given education or training situation. This tech-

nique is also used in Type 2 research as in the studies by Keller

(1987) and Richey (1992). However, in Type 2 studies the survey

is frequently used as a means of gathering data from designers

in a variety of settings. Beauchamp (1991), Hallamon (2002),

Klimczak and Wedman (1997), Rowland and Adams (1999),

Twitchell et al. (2000), and Wedman and Tessmer (1993) all con-

ducted surveys across a range of typical design environments—

Beauchamp to validate the use a wide range of affective variables

in the instructional design process, Hallamon to investigate the

factors that influence the use of task analysis, Klimczak and Wed-

man to examine success factors in instructional design projects,

Rowland and Adams to explore designers’perspectives toward

systems thinking, Twitchell et al. to determine evaluation prac-

tices in technical training settings, and Wedman and Tessmer to

analyze the nature of designer decisions.

Qualitative research methods are often employed in Type 2

developmental research studies. (See Table 41.4 for studies clas-

sified in terms of an I-type research methodology.) These studies

frequently employ a structured interview to gather data from

participants. For example, Nadolski, Kirschner, van Merrien-

boer, and Hummel (2001) used structured interviews to deter-

mine how different types of practitioners approached cognitive

task analysis. Jones and Richey (2000) interviewed instructional

designers and customers to investigate the use of rapid prototyp-

ing. Interviews are sometimes used in combination with other

data collection methods. Visscher-Voerman (1999) conducted

interviews of professional designers and analyzed project docu-

ments to determine the design strategies used in various training

and education contexts. Richey (1992) conducted structured

personal interviews with trainees to verify and/or expand the

more quantitative data collected in her study. Similarly, Row-

land (1992) used a systematic posttask interview in addition to

other data collection techniques designed to document one’s

decision-making processes.

In addition to interviews, other qualitative methods are used

in Type 2 studies. English and Reigeluth (1996) used qualitative

data analysis techniques to examine the use of elaboration the-

ory for sequencing instruction. Harrison et al. (1991) used qual-

itative data in the development of a distance education assess-

ment instrument. Nelson’s (1990) Le Maistre’s (1998), Nelson’s

(1990), and Rowland’s (1992) use of “think-aloud protocols”re-

flects not only a cognitive orientation, but also a qualitative one.

[Link] The Nature of Conclusions from Type 2 Devel-

opmental Research. Type 2 studies are unique among de-

velopmental research in that they are ultimately directed to-

ward general principles, which are applicable in a wide range

of design and development projects. Those studies described in

Table 41.4 are typical. Most of these studies present generalized

conclusions (see the studies in Table 41.4 that are classified in

terms of C and D types of conclusions), and the majority of these

studies have all generalized conclusions.

The nature of the conclusions in Type 2 research also varies

somewhat from that in Type 1. The most noticeable difference is

that the Type 2 conclusions pertain to a technique or model, as

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

41. DEVELOPMENTAL RESEARCH •1113

opposed to a product or program. The issues that areaddressed

in these conclusions can be summarized in the following

manner.

rEvidence of the validity and/or effectiveness of a particu-

lar technique or model (Adamski, 1998; Beauchamp, 1991;

Burkholder, 1981–1982; Driscoll & Tessmer, 1985; English &

Reigeluth, 1996; Forsyth, 1998; Fuchs & Fuchs, 1986; Gauger,

1987; Harrison et al., 1991; Keller, 1987; Kim, 1984; Kress,

1990; Kunneman & Sleezer; 2000; Means et al., 1997; Nadol-

ski et al., 2001; Plass & Salisbury, 2002: Richey, 1992; Sham-

baugh & Magliaro, 2001; Shellnut, 1999; Tessmer et al., 1999;

Twitchell et al., 2000; Tracey, 2002; Wagner, 1984; Weston,

McAlpine, & Bordonaro, 1995; Wreathall & Connelly, 1992)

rConditions and procedures that facilitate the successful

use of a particular technique or model (Abdelraheem,

1990; Adamski, 1998; Beauchamp, 1991; Burkholder, 1981–

1982; Cowall, 2001; Cummings, 1985; English & Reigeluth,

1996; Ford & Wood, 1992; Forsyth, 1998; Fuchs & Fuchs,

1986; Gauger, 1987; Goel & Pirolli, 1988; Hallamon, 2002;

Jones & Richey, 2000; Keller, 1987; Kim, 1984; King &

Dille, 1993; Kress, 1990; Le Maistre & Weston, 1996; Nelson,

1990; Nicholson, 1988; Phillips, 2000; Piper, 1991; Rowland,

1992; Rowland & Adams, 1999; Roytek, 2000; Shambaugh &

Magliaro, 2001; Shellnut, 1999; Spector et al., 1992; Tracey,

2002; Twitchell et al., 2000; Visscher-Voerman, 1999)

rExplanations of the successes or failures encountered in us-

ing a particular technique or model (Cowall, 2001; Higgins

& Reiser, 1985; Jones & Richey, 2000; Kerr, 1983; King & Dille,

1993; Klimczak & Wedman, 1997; Kress, 1990; Le Maistre &

Weston, 1996; McAleese, 1988; Means et al., 1997; Nicolson,

1988; Pirolli, 1991; Roytek, 2000; Taylor & Ellis, 1991; Tracey,

2002; Wedman & Tessmer, 1993; Zemke, 1985)

rA synthesis of events and/or opinions related to the use

of a particular technique or model (Carliner, 1998; Cowall,

2001; Jones & Richey, 2000; Julian, 2001; Le Maistre; 1998;

Luiz, 1983; Nadolski et al., 2001; Phillips, 2000; Pirolli, 1991;

Riplinger, 1987; Rojas, 1988; Roytek, 2000; Visscher-Voerman,

1999; Weston et al., 1995)

rA new or enhanced design, development, and/or evaluation

model (Adamski, 1998; Beauchamp, 1991; Carliner, 1998;

Forsyth, 1998; Jones & Richey, 2000; King & Dille, 1993;

McAleese, 1988; Nadolski et al., 2001; Plass & Salisbury, 2002:

Richey, 1992; Spector et al., 1992; Tessmer et al., 1999; Tracey,

2002; Wedman & Tessmer, 1993)

It is not uncommon for a given Type 2 study to generate more

than one type of conclusion. This was less likely to be the case

with respect to Type 1 developmental research.

[Link] Typical Type 2 Developmental Studies. We now

illustrate two typical Type 2 research studies in more detail.

Tessmer et al. (1999)4described a theoretical and procedural

model for conducting a type of needs assessment, called Needs

4The article by Tessmer et al. won the 1999 ETR&D Award for Outstanding Research on Instructional
Development. The award is given by the

development section of Educational Technology Research and Development (ETR&D) for the best paper
describing research findings that can be

used to improve the process of instructional design, development, and evaluation.

reassessment. According to the authors, “Needs reassessment is

a hybrid process, one that has the purposes of needs assessment

and the timing of summative evaluation”(p. 86). The model is

used to reassess existing training programs to determine if train-

ing excesses and deficiencies exist. Like many other authors in

the field who propose a new ISD model, Tessmer et al. (1999)

use an acronym for their model—CODE—which stands for the

four foci of the needs reassessment process (criticality, oppor-

tunity, difficulty, and emphasis). However, unlike many recent

ISD models, the validity of the CODE model was empirically

tested. Tessmer et al. (1999) report on two studies—a needs re-

assessment of corporate training program and a medical training

program reassessment. Both employed the case study method

to examine the application of the CODE model. A variety of

data sources was used including surveys, interviews, and extant

data analysis. The authors provide generalized conclusions but

wisely caution that their model requires further validation in

other instructional settings.

Another example of Type 2 developmental research is a study

by Jones and Richey (2000). These authors conducted an in-

depth examination of the use of rapid prototyping methods in

two instructional design projects in natural work settings. The

study used qualitative methods; data were collected from struc-

tured personal interviews, activity logs, and a review of extant

data. Participants included two experienced instructional de-

signers who employed rapid prototyping to the design and de-

velopment of two projects—a 1-day instructor led course with

a matching on-line tutorial for the automotive industry and a

1-day instructor led training program in the health-care industry.

In addition, one client was interviewed to obtain perceptions

of product quality and usability, cycle time reduction, and cus-

tomer satisfaction. After discussing how rapid prototyping was

successfully used in the context of the study, Jones and Richey

(2000) provide generalized conclusions and suggest a revised

model of ISD that includes rapid prototyping.

Each of these studies represents Type 2 developmental re-

search. They are typical of Type 2 studies because of the follow-

ing characteristics:

ran orientation toward studying the design and development

process, rather than demonstrating particular strategies;

ra tendency toward the use of multiple sources of data;

ra tendency to develop generalized principles and conclusions

with respect to the design and development process, or a part

of the process; and

ran effort to identify, describe, explain, or validate those con-

ditions that facilitate successful design and development.

41.4 THE METHODOLOGY OF

DEVELOPMENTAL RESEARCH

The aim of this section is to provide some methodological direc-

tion to those entertaining a developmental research project. In

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

1114 •RICHEY, KLEIN, NELSON

essence, it is a discussion of establishing the credibility of a given

developmental study by assuring authenticity and methodolog-

ical rigor. Because many topics can be addressed in a number of

ways, this section may also help one recognize the potential of

a given problem to be addressed as a developmental topic. This

section describes developmental methodologies in terms of the

traditional stages of planning and reporting research projects:

rproblem definition,

rliterature reviews, and

rresearch procedures.

41.4.1 Defining the Research Problem

The perceived authenticity of a particular developmental re-

search project often depends on the problem selected. Is the

problem one that is common to many designers and develop-

ers? Is it one that is currently critical to the profession? Does

the problem reflect realistic constraints and conditions typically

faced by designers? Does the problem pertain to cutting-edge

technologies and processes? Answers to these questions predict

not only interest in the project, but also whether the research is

viewed as relevant. “Explorations of research relevance are typ-

ically examinations of shared perceptions, the extent to which

researchers’notions of relevance are congruent with the per-

ceptions and needs of practitioners”(Richey, 1998, p. 8). This is

particularly true with developmental research, where the object

of such research is clearly not simply knowledge, but knowledge

that practitioners consider usable.

[Link] Focusing the Problem. Given a relevant topic, the

research project must first be given a “developmental twist.”

This begins in the problem definition stage. It is done by focus-

ing the research problem on a particular aspect of the design,

development, or evaluation process, as opposed to focusing on a

particular variable that impacts learning or, perhaps, the impact

of a type of media (to name two alternative approaches). Type 1

developmental studies focus upon a given instructional product,

program, process, or tool. They reflect an interest in identify-

ing either general development principles or situation-specific

recommendations. These studies may ultimately validate a par-

ticular design or development technique or tool. Type 2 stud-

ies, on the other hand, focus on a given design, development

or evaluation model or process. They may involve constructing

and validating unique design models and processes, as well as

identifying those conditions that facilitate their successful use.

The problem definition stage must also establish the research

parameters. At the minimum, this involves determining whether

the research will be conducted as the design and development

is occurring or whether retrospective data will be collected on

apreviously developed program or set of materials. Then the

scope of the study must be established. How much of the design

and development process will be addressed? Will the research

address

rall parts of the design of the instruction?

rthe development (or part of the development) of the instruc-

tion?

rthe evaluation of the instruction? If so, will formative, and

summative, and confirmative evaluation be addressed?

rthe revision and retesting of the instruction?

Developmental studies often are structured in phases. For

example, comprehensive Type 1 studies may have an analysis

phase, design phase, a development phase, and a try-out and

evaluation phase. Another organization of a Type 1 study would

include phases directed toward first analysis, then prototype

development and testing, and, finally, prototype revision and

retesting. McKinney (2001) is an example of this type of study.

Type 2 studies may have a model construction phase, a model

implementation phase, and a model validation phase. Forsyth

(1998), Adamski (1998), and Tracey (2002) followed this pat-

tern. In these studies the model construction phase was fur-

ther divided to include comprehensive literature reviews, model

construction, and model revision phases.

[Link] Framing the Problem. Seels (1994) describes typ-

ical processes one uses to explain the goals of a developmen-

tal research project. For example, research questions, rather

than hypotheses, commonly serve as the organizing framework

for developmental studies. This tactic is appropriate if there

is not a firm base in the literature that one can use as a basis

for formulating a hypothesis. This is often the case with such

research, especially if the problem focuses on emerging tech-

nologies. However, research questions are also more appropri-

ate for qualitative research, a common developmental method-

ology.

[Link] Identifying Limitations. Because developmental

research is often context specific, one must be particularly con-

cerned with the limitations or unique conditions that may be

operating in a particular study. Such limitations will effect the

extent to which one may generalize the conclusions of the study.

The results may be applicable only in the situation studied or to

others with similar characteristics, rather than being generaliz-

able to a wider range of instructional environments.

41.4.2 Review of Related Literature

Typically, literature reviews concentrate on the specific vari-

ables being studied, usually the independent and dependent

variables. This orientation may not prove useful in many de-

velopmental studies. The goal of the literature review, how-

ever, remains the same as with other types of research

projects. It is to establish the conceptual foundations of the

study.

The literature review in Type 1 developmental studies, for

example, may address topics, such as

rprocedural models that might be appropriate for the task at

hand;

rcharacteristics of similar effective instructional products, pro-

grams, or delivery systems;

rfactors that have impacted the use of the target development

processes in other situations; and

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

41. DEVELOPMENTAL RESEARCH •1115

rfactors impacting the implementation and management of the

target instructional product, program, or delivery system in

other situations.
Literature reviews in Type 2 studies may address topics such as

ra description of models (either formally published or cur-

rently in use) similar to the one being studied, including their

strengths and weaknesses;

rresearch on the targeted process (for example, task analysis

or evaluation of organizational impact); and

rresearch on factors impacting the use of a given model or

process (for example, factors that facilitate the use of rapid

prototyping models).

Both types of studies often address the methodology of devel-

opmental research itself in the literature review.

In developmental studies directed toward innovative instruc-

tional environments or innovative design and development pro-

cesses, it would not be unusual to find little research in the

literature that is directly relevant. In such cases the researcher

must still identify literature that is relevant to the foundational

theory of the project, even though the link may be indirect.

For example, there is literature on factors that affect the use

of computers and other media in instruction, but there may be

little on factors related to the specific use of virtual reality as

a delivery system. In developmental research the conceptual

framework for the study may be found in literature from actual

practice environments (such as an evaluation report) as well

as from traditional research literature directed toward theory

construction.

41.4.3 Research Procedures

Often developmental research occurs in natural work environ-

ments. This tends to enhance the credibility of the research,

as well as create methodological dilemmas for the researcher.

Nonetheless, whether the research is conducted during the

design and development process or retrospectively, the best

research pertains to actual projects, rather than simulated or

idealized projects. Perhaps it is the “real-life”aspect of develop-

mental research that results in studies that frequently take even

more time to complete than other types of research. There are

often more changes in one’s research plans and procedures as

a result of unanticipated events than is typical in other types of

research. Consequently, detailed research procedures and time

lines are most important.

[Link] Participants. There are often multiple types of par-

ticipants in a given developmental research project, and if the

study is conducted in phases, the participants may vary among

phases. The nature of the participating populations tends to

vary with the type of developmental research being conducted.

Typical populations include

rdesigners, developers, and evaluators;

rclients;

rinstructors and/or program facilitators;

TABLE 41.5. Common Participants in Developmental

Research Studies

Developmental

Research Function/Phase Type of Participant

Type 1 Product design &

development

Designers, Developers,

Clients

Type 1 Product

evaluation
Evaluators, Clients,

Learners, Instructors,

Organizations

Type 1 Validation of tool

or technique

Designers, Developers,

Evaluators, Users

Type 2 Model

development

Designers, Developers,

Evaluators, Researchers,

Theorists

Type 2 Model use Designers, Developers,

Evaluators, Clients

Type 2 Model validation Designers, Developers,

Evaluators, Clients,

Learners, Instructors,

Organizations

rorganizations;

rdesign and development researchers and theorists; and

rlearners and other types of users.

Table 41.5 shows the array of persons that most commonly par-

ticipate in these projects and the various phases of the project

in which they tend to contribute data.

For example, participants in the Tracey (2002) study in-

cluded researchers and theorists in model development, design-

ers in model use, and learners and instructors in model valida-

tion. This was a comprehensive Type 2 study. The participants

in Jones and Richey’s (2000) Type 2 study were designers, de-

velopers, and clients. This study primarily addressed model use.

McKenney’s research (2002) was a Type 1 study. Here, in the

phase addressing the construction and prototyping of an elec-

tronic design tool, the participants were developers, users, eval-

uators, and a variety of ID experts. Then, in the phase evaluating

and validating the tool, participants were users (preservice and

in-service teachers and curriculum developers, i.e., designers)

and experts.

[Link] Research Design. It is not uncommon for a de-

velopmental research project to also utilize multiple research

methodologies and designs, with different designs again being

used for different phases of the project. Table 41.6 presents a

summary of those research methods that are most frequently

used in the various types and phases of developmental re-

search. This table reflects those studies presented in Tables 41.3

and 41.4.

Table 41.6 highlights some typical methodology patterns

used in developmental studies. In Type 1 studies critical de-

sign and development processes are often explicated using case

study methods. Interviews, observations, and document analy-

sis are techniques used to gather the case study data and to doc-

ument the processes used and the conditions under which they

are employed as well. Pizzuto’s (1983) development of a simu-

lation game is a classic example of this type of methodology.

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

1116 •RICHEY, KLEIN, NELSON

TABLE 41.6. Common Research Methods Employed in Developmental Research Studies

Developmental Research Methodologies

Research Type Function/Phase Employed

Type 1 Product design & development Case study, In-depth interview, Field

observation, Document analysis

Type 1 Product evaluation Evaluation, Case study, Survey,

In-depth interview, Document

analysis

Type 1 Validation of tool or technique Evaluation, Experimental, Expert

review, In-depth interview, Survey

Type 2 Model development Literature review, Case study, Survey,

Delphi, Think-aloud protocols

Type 2 Model use Survey, In-depth interview, Case study,

Field observation, Document

analysis

Type 2 Model validation Experimental, In-depth interview,

Expert review, Replication

Evaluation research techniques are often employed in Type

1 studies to determine the effectiveness of the resulting product

or the particular techniques used during the design and devel-

opment project. As with all evaluation research, a variety of data

collection techniques is possible. Sullivan and co-Researchers’

(2000) examination of the development and use of a K-12 en-

ergy education program employed a formal evaluation design,

in addition to a case study. Smith (1993) used in-depth inter-

views and document analysis techniques in her evaluation of an

executive development program. Sometimes, a full experiment

is constructed to test the product or technique.

In Type 2 research models of the full design and development

process, or of a particular part of the process, are constructed

in a variety of ways, including the following.

rBy conducting surveys of designers and developers with re-

gard to projects in which they have been involved, such as

Shellnut’s (1999) study of motivation design or Phillip’s (2000)

study of organizational impact evaluation techniques

rBy synthesizing models from the literature, such as Adamski’s

(1998) reviews of instructional technology, human factors,

and aviation literature in his efforts to devise an initial model

for designing job performance aids for use in high-risk settings

rBy arriving at a consensus of opinion of respects experts in

the field using Delphi techniques, such as Tracey’s (2002)

methods of finalizing her multiple intelligence design model

rBy conducting experiments to validate particular design and

development models, such as Tracey’s (2002) and Adamski’s

(1998) projects

Developmental researchers are commonly confronted by

methodological dilemmas. One is the need to account for con-

textual variables, especially in those studies taking place in a

natural work setting. For example, to what extent do the client’s

design experience and sophistication, or designer expertise, or

time pressures impact the success of a particular project? Be-

cause it is typically not possible to control such factors in the

research design, the researcher is then obligated to describe

(and measure, if possible) these variables carefully in an effort

to account for their impact.

Another common situation that is potentially problematic is

when the researcher is also a participant in the study, such as

when the researcher is also the designer or developer. Although

this situation is not preferable, it is not unusual. Care must be

taken to ensure objectivity through consistent, systematic data

collection techniques and the collection of corroborating data, if

possible. Often structured logs and diaries completed by several

project participants according to a regularly established sched-

ule create a structure that facilitates the generation of reliable

and comparable data.

Another frequent problem is maintaining the integrity of re-

call data. Many studies rely on self-reports of past projects. Oth-

ers use structured interviews of participants. Using previously

prepared documents or data from others involved in the same

project facilitates a triangulation process to validate the data

collected.

[Link] Collecting, Analyzing, and Reporting Data.

Data collection in a developmental study takes a variety of forms

depending on the focus of the research. The validity of the con-

clusions is often dependent on the richness of the data set as

well as the quality of the research design. Typical types of data

collected in developmental research relate to

rdocumentation of the design, development, and evaluation

tasks, including profiling the design and development context

and collecting data such as work time and expenses, problems

encountered and decisions made, adjustments made in the

original plans, designer reactions and attitudes, or records of

concurrent work patterns;

rdocumentation of the conditions under which the devel-

opment and implementation took place, including factors

such as equipment and resources available, participant ex-

pertise and background, or time and client constraints;

and

ridentification of the results of predesign needs assess-

ments, formative, summative, and confirmative evaluations,

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

41. DEVELOPMENTAL RESEARCH •1117

including documentation of the target populations and the

implementation context and measures of learning, transfer,

and the impact of the intervention on the organization.

As with all research projects, participants must be fully in-

formed of the nature of the research, not be coerced to be in-

volved, and be assured that the data will be both anonymous

and confidential. Not only must the participants be informed

and give their consent, but often written consent from their

organizations is required as well. Data may be sensitive and pro-

prietary in nature, and researchers must be attuned to these

issues and their ramifications.

Data analysis and synthesis in a developmental study are not

unlike those in other research projects. There are likely to be de-

scriptive data presentations and qualitative data analyses using

data from documentation, interviews, and observations. Tradi-

tional quantitative data analyses techniques are used as well.

The best techniques for reporting developmental data, how-

ever, have not been firmly established. There can be a massive

amount of data, especially with Type 1 studies. Journals do not

provide for a detailed documentation of such data. Often, the

raw data sets are too massive even to include in a dissertation

appendix.

In response to this dilemma, some researchers are using Web

sites as data repositories. For example, full results of a needs

assessment may be included in a Web site, or complete copies

of electronic tools can be provided, or full transcripts of designer

interviews can be presented. Assuming that these Web sites are

stable and accurate, this solution allows for full disclosure of

data that should prove valuable to practitioners and researchers

alike. With full data sets, practitioners should be to apply general

lessons to their own work environments. With full data sets,

researchers should have opportunities for secondary analysis of

data (an option seldom available to researchers in this field),

as well as opportunities to replicate fully the research in other

settings.

41.5 RECENT INNOVATIVE DEVELOPMENTAL

RESEARCH

Recently, innovative lines of developmental research have ex-

tended the boundaries of the more traditional orientation to de-

velopmental research previously discussed in this chapter. This

work concerns mainly the development of instruction using

newer models and procedures of instructional design that re-

flect the influence of cognitive science, especially with respect

to higher-level cognitive skills (van Merrienboer, Jelsma, & Paas,

1992). Moreover, constructivist influences are evident in the em-

phasis on the role of context in design (Richey & Tessmer, 1995),

in examination of the social and collaborative nature of learning

(Duffy & Jonassen, 1992), and in the development of new ap-

proaches to instruction such as anchored instruction (Bransford,

Sherwood, Hasselbring, Kinzer,& Williams, 1990), or case-based

instruction (Schank, Berman, & Macpherson, 1999). The more

recent research to be considered here addresses areas such as

designer decision making, knowledge acquisition tools, and the

use of automated development tools.

By its very nature, the research on designer decision making

is Type 2 research aimed at understanding and improving the

design process. The other research tends to focus on improving

the design process through the development of new tools and

techniques. Sometimes this is Type 1 research involving a partic-

ular product or context, but more frequently it is Type 2 research

oriented toward the development of generic tools. The follow-

ing section summarizes important findings and issues emerging

from these research activities.

41.5.1 Trends in Research on Design and Designer

Decision Making

Design has been described as decision making, simulation, a

creative activity, a scientific process or “a very complicated act

of faith”(Freeman, 1983, p. 3). Research on how people design,

regardless of what is being designed, has shown that designers

select certain elements from a large number of possibilities and

combine these elements to develop a functional and aesthet-

ically pleasing solution to a problem (Zaff, 1987). Although a

definitive, unified description of how people design is yet to be

fully developed (Shedroff, 1994), most researchers agree that

design combines both rational and intuitive thought processes

that are derived from the knowledge and creativity of the de-

signer (Nadin & Novak, 1987).

To understand the design process, it is necessary to distin-

guish design activities from other forms of thinking. The theo-

retical basis for most studies of design comes from the literature

on human problem solving, where Simon (1981) suggests an

all-encompassing view of design that incorporates nearly any

kind of planning activity. In fact, he considers design as a funda-

mental characteristic of human thought: Everyone designs who

devises courses of action aimed at changing existing situations

into preferred ones. Simon’s conception of design as problem

solving leads to the characterization of design as a process of

optimization among various solution alternatives.

An alternative theoretical orientation views design as an ex-

periential, constructive process where an individual designer

shapes the problem and solution through cycles of situated

action and reflection (Suchman, 1987). In this sense, design

problems are constructed by the designer through a process of

“dialogue”with the situation in which the designer engages in

metaphorical processes that relate the current design state to

the repertoire of objects/solutions known by the designer. De-

sign typically flows through four major stages: naming (where

designers identify the main issues in the problem), framing (es-

tablishing the parameters of the problem), moving (taking an

experimental design action), and reflecting (evaluating and crit-

icizing the move and the frame). Schon (1983, 1985, 1987) has

noted that designers reflect on moves in three ways: by judging

the desirability and consequences of the move, by examining

the implications of a move in terms of conformity or violation of

earlier moves, and by understanding new problems or potentials

the move has created. In part, this involves “seeing”the current

situation in a new way (Rowland, 1993). As a designer moves

through the design process, the situation “talks back”to the

designer and causes a reframing of the problem. This reframing

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

1118 •RICHEY, KLEIN, NELSON

is often accomplished by relating the current situation to pre-

vious experiences. Obstacles or difficulties in the design situa-

tion provide opportunities for new insights into the problem.

Because of its cyclical nature, design thinking naturally benefits

from reflection in action, and designers often maintain sketch-

books and diaries to support reflection (Cheng, 2000; Webster,

2001). These and other aspects of reflection assume that a de-

signer possesses a willingness to be thoughtful and reflective,

is able to understand the context in which assumptions and ac-

tions are formed, and is willing to explore alternatives and be

exposed to interpretive considerations through dialogue with

others (Moallem, 1998). In some sense, a designer is engaged

in learning as well as design, because the designer’s personal

knowledge structures are altered by the information present in

the design environment (McAleese, 1988).

From a social view, design is a collaborative activity where

conversation, argumentation, and persuasion are used to

achieve consensus about perspectives and actions that might

be taken to solve the design problem (Bucciarelli, 2001; Lave &

Wenger, 1991; Stumpf & McDonnell, 1999). The design pro-

cess, therefore, includes both shared and distributed cognition

(Lanzara, 1983; Roth, 2001), with the design team developing

a shared understanding of the problem through conversations

and representations (Hedberg & Sims, 2001; Rowland, 1996),

and a solution to the problem through individual and collabora-

tive design efforts (Hutchins, 1991; Walz, Elam, & Curtis, 1993).

Often, collaborative design processes generate questions and

requests for information and opinions among group members.

Conflicting viewpoints are debated and differences of opinion

are negotiated. Mutually agreeable solutions result from rethink-

ing, restructuring, and synthesizing alternate points of view. In

this way, dialogue transforms individual thinking, creating col-

lective thought and socially constructed knowledge within the

team (Sherry & Myers, 1998).

[Link] The Instructional Design Task Environment.

What makes design a special form of problem solving, in part,

is the nature of design problems. Reitman (1964) has identified

a category of problems that he calls “ill defined,”where starting

states, goal states, and allowable transformations of the problem

are not specified. Simon (1973) proposes a similar classification

but notes that problems can fall along a continuum between

well defined and ill defined. The point at which a problem be-

comes ill defined is largely a function of the problem solver, in

that the goals, attitudes, and knowledge of the problem solver

determine the degree to which a problem may be ill defined.

Goel and Pirolli (1988) take issue with Simon’s broad character-

ization of design. They identify several features that distinguish

design from other forms of problem solving, including the ini-

tial “fuzziness”of the problem statement, limited or delayed

feedback from the world during problem-solving activities, an

artifact that must function independently from the designer,

and “no right or wrong answers, only better and worse ones”

(Goel & Pirolli, 1988, p. 7). Chandrasekaran (1987) concurs, not-

ing that at one extreme are those rare design problems that re-

quire innovative behavior where neither the knowledge sources

nor the problem-solving strategies are known in advance. Such

activity might be more properly termed creating or inventing,

and results in a completely new product. Other design problems

are closer to routine but may require some innovation because

of the introduction of new requirements for a product that has

already been designed.

Certainly, how a designer functions in the design environ-

ment is related to what is being designed (the design task envi-

ronment). Initially, design goals are often poorly specified and

can involve the performance goals for the object or system,

constraints on the development process (such as cost), or con-

straints on the design process (such as time required for com-

pletion). Part of the designer’s task is to formulate more specific

goals based on the constraints of the problem. The designer then

identifies criteria for selecting and eliminating various elements

and, finally, makes decisions based on these criteria (Kerr, 1983).

Pirolli and Greeno (1988) noted that the instructional design

task environment consists of “the alternatives that a problem

solver has available and the various states that can be produced

during problem solving by the decisions that the problem solver

makes in choosing among alternatives”(p. 182). They suggest

that instructional design has three levels of generality: global,

intermediate, and local. At each level, designers are concerned

with three types of issues: goals and constraints, technological

resources, and theoretical resources. Global-level design deci-

sions are concerned mainly with the content and goals of in-

struction. At the intermediate level, lessons and activities are

identified and sequenced. At the local level, instructional design-

ers make decisions about how information is to be presented

to learners and how specific learning tasks are to be organized.

This description of the instructional design task environment is

similar in many respects to the “variables”of instruction identi-

fied by Reigeluth (1983a). Hwoever, Pirolli and Greeno (1988)

also noted an interesting gap in instructional design research:

nearly all of the descriptive and prescriptive theories of instruc-

tional design focus on instructional products, while there is little

research dedicated to describing the instructional design pro-

cess. A few researchers have subsequently addressed issues sur-

rounding the instructional design task environment (Dijkstra,

2001; Murphy, 1992), attempting to draw parallels to other de-

sign domains. Rathbun (1999) has provided a comprehensive

activity-oriented analysis of the work of instructional design that

confirms many of the theoretical predictions of design studies in

other areas. Other researchers have established that the way the

process is portrayed visually impacts a designer’s impression of

the process (Branch, 1997; Branch & Bloom, 1995; Rezabek &

Cochenour, 1996). New approaches to instructional design have

emerged as designers explore procedural techniques from other

fields, such as rapid prototyping (Rathbun, 1997) and situated

instructional design (Rowland, 1993; Wilson, 1995). Research

has also focused on collaborative aspects of the instructional

design and development process, to understand and improve

design team interactions (Hedberg & Sims, 2001), content pro-

duction procedures (Keppel, 2001), and project management

strategies (McDaniel & Liu, 1996; Phillips, 2001)

[Link] Design Thinking and Instructional Design.

Studies of the cognitive processes of designers in domains

other than instructional design indicate that the design pro-

cess is iterative and cyclical, with two distinct categories of

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

41. DEVELOPMENTAL RESEARCH •1119

designer behavior: problem structuring and problem solving

(Akin, Chen, Dave, & Pithavadian, 1986). Problem structuring

transforms the information obtained through functional analysis

into scenarios that partition the design space into a hierarchical

organization of design units along with the parameters of the

units and relationships between units. The design units are then

arranged in various ways until a solution is found that meets the

requirements and constraints established earlier. Furthermore,

the structure of the problem can affect the designer’s perfor-

mance. When the information of a design problem is provided in

a more hierarchical structure, solutions tend to be faster, more

clustered, stable, and more successful in satisfying design re-

quirements (Carroll, Thomas, Miller, & Friedman, 1980). While

problem structuring establishes the problem space, problem

solving completes the design task by producing solutions that

satisfy the requirements and constraints for each design unit.

Solution proceeds under a control structure that is established

through problem structuring and consists of successive gener-

ate/test actions that progress toward the final solution. Prob-

lem solving in design also contains a feedback component that

communicates results of the solution to higher levels, where

restructuring of the problem space can be undertaken if the

partial solution demands such an action (Akin et al., 1986).

In their studies of the design process, Goel and Pirolli (1988)

noted a number of additional characteristics of design problem

solving. Observations of three experienced designers (an archi-

tect, a mechanical engineer, and an instructional designer) using

think-aloud protocols revealed that during the design task these

designers engaged in extensive problem structuring (decom-

posing the problem into modules) and performance modeling

of the artifact at various stages of its design. Each designer also

employed evaluation functions and “stopping rules”that con-

trolled the decomposition of the problem. Designers tended to

evaluate their decisions at several levels continuously through

the process, employing cognitive strategies to decompose prob-

lems into “leaky modules”(Goel & Pirolli, 1988, p. 20). Although

extensive restructuring of the design problem into several mod-

ules was observed, the designers did not encapsulate each mod-

ule, but rather they monitored the design process to assure that

decisions made in one module did not adversely affect other

modules. The designers handled “leaks”between modules by

“plugging”the leaks, that is, ignoring the effects of low-level

decisions in one module after making high-level assumptions

about the other modules.

Several other studies have provided valuable insights into the

design process. Kerr (1983) studied the thought processes of

26 novice instructional designers using interviews and planning

documents produced by the designers during a graduate course

in instructional design. He found that the processes employed

by the designers were not systematic, that their solutions were

generated based largely on their personal experiences in vari-

ous instructional settings, and that problem constraints greatly

influenced their solutions. Using a think-aloud task and proto-

col analysis, Nelson (1990) studied the initial phase of prob-

lem structuring in four experienced instructional designers. Al-

though the experimental task only approximated a “first pass”

at the problem, the designers tended to focus on information

related to available resources, the learners, and the skills to

be trained. Specific information regarding content and learn-

ing tasks was not examined in detail. It was also apparent that

specific pieces of information constrained the possible solutions

that the designers considered. In other words, a “stopping rule”

was evoked that led the designers to a particular solution with-

out considering additional alternatives (Goel & Pirolli, 1988).

A more comprehensive study of instructional designers was re-

ported by Rowland (1992), where four expert and four novice

designers were given a task to design instruction for an industrial

setting involving training employees to operate two hypothet-

ical machines. Verbal reports of the designers’thoughts while

completing the task were coded and analyzed, and the results

of this study suggest that the design process alternates between

two phases that Rowland terms problem understanding and

solution generation (p. 71). Experts tended to take much more

time in the problem understanding phase, constructing a rich

representation of the problem that was guided by a template, or

mental model, of the process (Rowland, 1992). Novices did not

develop a rich representation of the problem, relying on the

materials given rather than making inferences about the con-

straints of the problem or the structure of the content. They

also quickly began to generate possible solutions after briefly

examining the problem materials and frequently returned to

the problem materials as the process unfolded. Consequently,

their problem representation grew as they progressed through

the solution generation phase.

Observations of designer behavior using a case study

methodology (Spector et al., 1992) have revealed that many vari-

ables affect the ability of designers to author computer-based

instruction effectively, especially prior experience (both as a

designer and with computer-based authoring systems). In an-

other context, a naturalistic study of instructional design for

informal learning settings noted the effects of constraints on

the design process (Carliner, 1998). Other studies of instruc-

tional design practice have employed self-report methods using

surveys to elicit designers’opinions about how they actually

practice instructional design. Zemke’s (1985) survey indicated

that instructional designers in business and industry are selec-

tive in terms of the design activities in which they engage, often

ignoring needs assessment, task analysis, and follow-up evalu-

ations. Similar results were found by Mann (1996). Wedman

and Tessmer (1993) extended this work by examining the fac-

tors that might contribute to the selective use of instructional

design activities. Surveys of more than 70 instructional design

professionals indicated that activities such as writing learning

objectives, developing test items, and selecting media and in-

structional strategies are nearly always completed. On the other

hand, these designers reported that needs assessment, task anal-

ysis, assessing entry-level skills, and pilot testing instruction are

performed selectively, if ever. Reasons for not performing some

activities included lack of time and that the decisions had already

been made by others. Interestingly, lack of money or expertise

in performing the activity was rarely cited as a reason the activ-

ities were not performed.

[Link] The Role of Knowledge in the Design Process.

The success of the designer’s problem-solving processes is

directly related to the designer’s experience and knowledge

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

1120 •RICHEY, KLEIN, NELSON

in the design task environment. Researchers have speculated

about general cognitive structures for design that contain both

the elements of a designed product and the processes necessary

for generating the design (Akin, 1979; Jeffries, Turner, Polson, &

Atwood, 1981). Goel and Pirolli (1988) also discuss the role

of knowledge in the design process, noting that the personal

knowledge of expert designers is organized in rich and intri-

cate memory structures that contain both general knowledge

(extracted from the totality of an individual’s life experiences)

and domain-specific knowledge derived from their professional

training. This knowledge is used in many cases as a template for

understanding the characteristics of the design task, as well as

generating solutions.

So it seems that a well-organized knowledge base for instruc-

tional design is crucial to the process. The knowledge available

to instructional designers, either individual knowledge stored in

memory or other forms of knowledge embodied in the models

and tools used for design, will influence the kinds of instruction

they create (Nelson & Orey, 1991). An instructional designer’s

knowledge base should include not only conceptual and proce-

dural structures for controlling the design process, but also cases

or scenarios of exemplary instructional products and solutions

that can be recalled and applied to particular situations (Nelson,

Magliaro, & Sherman, 1988; Rowland, 1993). It is also necessary

for an instructional designer to be familiar with concepts and

procedures derived from general systems theory, psychological

theory, instructional theory, and message design (Richey, 1993).

In fact, it has been argued that successful implementation of in-

structional design procedures ultimately depends on whether

designers have “adequate understanding and training in higher-

order problem-solving principles and skills such that the neces-

sary expertise can be applied in the process”(McCombs, 1986,

p. 78). More recent studies suggest that complex case studies

grounded in real-world, ill-defined problems are effective in de-

veloping the kind of knowledge and expertise necessary to be

an effective instructional designer.

[Link] Overview of Designer Decision-Making Studies

as Developmental Research. Designer decision-making re-

search is typically Type 2 research and has the ultimate goal

of understanding the design process and, at times, produc-

ing design models that more closely match actual design ac-

tivity. The populations of the studies are naturally designers—

not learners—and frequently designers are classified as either

novice or expert. The effort to identify the impact of various

design environments is a common secondary objective. Con-

sequently, the project itself is often a second unit of analysis

and data are collected on the nature of the content, work re-

sources, and constraints. Methodologically the studies tend to

be more qualitative in nature, although survey methods are not

uncommon.

41.5.2 Trends in Research on Automated

Instructional Design and Development

The systematic instructional design and development proce-

dures common to our field have been developed as a means to

organize and control what is essentially a very complicated en-

gineering process. Even with systematic methods, however, the

instructional design process can become very time-consuming

and costly (O’Neil, Faust, & O’Neil, 1979). As computer technol-

ogy has proliferated in society, and as more training has become

computer based, it is not surprising that efforts in the instruc-

tional design field have concentrated on the development of

computer-based tools intended to streamline the design and de-

velopment of instruction, for both novice designers and expert

designers.

[Link] Tools for Content Acquisition. One of the major

tasks to be completed by the instructional designer is to identify

and structure the content of instruction. Both instructional de-

signers and instructional developers consult with subject matter

experts to determine the content that must be acquired by the

learners, and the forms in which this content might best be

communicated to the learners. In this respect, instructional de-

sign activities are similar to the knowledge engineering process

that is utilized in the development of expert systems and intel-

ligent tutoring systems (McGraw, 1989; Nelson, 1989; Rushby,

1986). Knowledge engineers typically work with subject matter

experts to identify and organize the knowledge that is necessary

for solving problems in a domain. Although the ultimate goals

of content acquisition for instructional design and knowledge

engineering differ, the processes and tools used by instructional

designers and knowledge engineers are complementary. In fact,

many of the techniques and tools used in knowledge engineer-

ing are becoming common in instructional design.

Until recently, the procedures utilized for content acquisi-

tion in instructional design and development relied largely on

task analysis (e.g., Merrill, 1973) and information processing

analysis (Gagn´e, Briggs, & Wager, 1988). A large number of task

analysis procedures have been employed for instructional de-

sign and development, as summarized by Jonassen, Hannum,

and Tessmer (1989). In general, these analysis techniques are

used by designers to describe the nature of the learning tasks

and to identify and sequence the content of the instruction.

Information processing analysis further delineates the decision-

making processes that may underlie a task, allowing the de-

signer to identify any concepts or rules that may be necessary

to complete the task. The process of content acquisition gen-

erally follows several stages, including (a) a descriptive phase,

where basic domain concepts and relations are identified; (b) a

structural phase, where the information is organized into inte-

grated structures; (c) a procedural phase, where the reasoning

and problem-solving strategies are identified; and (d) a refine-

ment phase, where the information structures procedures, and

strategies are tested with a wider range of problems and mod-

ified as needed (Ford & Wood, 1992; McGraw, 1989). Using

structured interviews and a variety of documentation activi-

ties, concepts and relations can be identified during the de-

scriptive phase of content acquisition. McGraw (1989) recom-

mends the use of concept dictionaries or cognitive maps to

represent the major domain concepts graphically. Many of the

automated tools for instructional design discussed later feature

some kind of graphical representation tool to help define do-

main concepts and relations (e.g., McAleese, 1988; Nicolson,

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

41. DEVELOPMENTAL RESEARCH •1121

1988; Pirolli, 1991). During the structural phase, the concepts

and relations identified earlier are organized into larger units.

This phase has been described as “a precondition to effective in-

structional design”(Jones, Li, & Merrill, 1990, p. 7) and is critical

to the specification of content. Interviewing is the most widely

employed technique, however, at this stage it is important to

begin prompting the expert to clarify distinctions between con-

cepts in order to structure the knowledge (Ford & Wood, 1992).

Besides interviews, various statistical procedures such as mul-

tidimensional scaling (e.g., Cooke & McDonald, 1986), path

analysis (Schvaneveldt, 1990), ordered trees (Naveh-Benjamin,

McKeachie, Lin, & Tucker, 1986), and repertory grid techniques

(Boose, 1986; Kelly, 1955) can be used to help structure the

knowledge.

Content does not consist solely of concepts and relations. Ex-

perts also possess highly proceduralized knowledge in the form

of rules and heuristics that help them solve problems. A variety

of methods can be used to identify and describe these proce-

dures. Task analysis methods yield a description of the tasks

that constitute performance, along with the skills necessary to

perform each task (Jonassen et al., 1989). Additional methods

that provide useful descriptions of domain expertise include

time-line analysis for sequencing tasks, information processing

analysis for describing decision making, and process tracing to

determine the thinking strategies used by the expert during

task performance (McGraw, 1989). Protocol analysis (Ericsson

& Simon, 1984) is a particularly effective process tracing tech-

nique, although it is time-consuming because of the extensive

data analysis that is required. Observations, constrained tasks

(Hoffman, 1987), and simulations can also be useful at this stage

of knowledge acquisition, especially when two or more experts

collaborate to solve or “debug”a problem (Tenney & Kurland,

1988). It is also advisable to perform similar analyses of novices

in order to identify specific difficulties that learners are likely to

encounter (Means & Gott, 1988; Orey & Nelson, 1993).

Advances in database technologies and Web-based delivery

systems have created new opportunities for research in content

acquisition, management, and delivery. The focus in recent re-

search has been on the feasibility of creating and maintaining

reusable, scalable, and distributed content. Much research has

been devoted to the definition and organization of “learning

objects”or “knowledge objects”(Wiley, 2000; Zielinski, 2000).

Systems to store and deliver content modules that are “tagged”

with descriptors related to various learning or instructional char-

acteristics of the content modules, as well as the characteristics

of the content itself, are being developed and tested. The mod-

ules can be retrieved and dynamically assembled by the system

based on learner actions or requests.

Standards for structuring and tagging content objects have

been developed and evaluated (Cuddy, 2000). Several models

for automating the delivery process have been proposed and

tested, including Merrill’s (1999) instructional transaction the-

ory, interactive remote instruction (Maly, Overstreet, Gonzalez,

Denbar, Cutaran, & Karunaratne, 1998), and learning engines

(Fritz & Ip, 1998). Other research has focused on assembling

content and demonstrating the effectiveness of learning object

technologies (Anderson & Merrill, 2000; Merrill, 2001). The im-

pact of these approaches is evident in the rapid adoption of open

architectures for learning objects (Open Courseware, 2002) and

the proliferation of shared content databases (Labeuf & Spalter,

2001).

[Link] Knowledge-Based Design Tools. Knowledge-

based design systems are becoming common in many design

professions as researchers strive to acquire and represent in

computer systems the kinds of knowledge and reasoning nec-

essary to interpret design problems, control design actions, and

produce design specifications (Coyne, Rosenman, Radford, Bal-

achandran, & Gero, 1990). Some computer-based design envi-

ronments have been developed as extensions of the tools used

by designers, such as the computer-aided design (CAD) systems

used in professions where sketches or blueprints are used as a

part of the design specifications (architecture, electronics, au-

tomobiles, etc.). Other systems have been developed around

“libraries”or “templates”of primitive design elements or as cri-

tiquing environments that monitor the designer’s activities and

suggest alternatives when an action that may be inappropriate

is taken (Fischer, 1991). These types of knowledge-based design

systems require an extensive database of rules that functions in

the background as the designer works with the system.

Although a wide variety of approaches to knowledge-based

instructional design has been attempted, all are characterized by

the need to formalize the knowledge and decision-making pro-

cesses necessary for effective instructional design. These efforts

began with attempts to provide “job aids”for novice instruc-

tional designers, especially those in military settings (Schulz &

Wagner, 1981). Early efforts using computers to automate vari-

ous instructional development tasks focused on the production

of print-based materials for programmed instruction (Braby &

Kincaid, 1981; Brecke & Blaiwes, 1981), adaptive testing sys-

tems (Weiss, 1979), and, more recently, paper-based training

materials (Cantor, 1988). A more ambitious project was under-

taken in conjunction with the PLATO system. Beginning with

libraries of code to produce various types of test items, Schulz

(1979) completed extensive research to produce and test a num-

ber of on-line authoring aids designed to be integrated with the

military’s IPISD model. Various activities specified by the model

were identified and the associated tasks analyzed. Flowcharts

of the design activities were then converted to interactive in-

structional segments that could be accessed by users as on-line

aids during authoring activities. Findings indicated that the aids

were accepted by the authors and that development time was

significantly decreased.

[Link] Design Productivity Tools. Numerous tools based

on expert system technology have been developed to aid in-

structional designers in making decisions about various aspects

of instructional design. These tools function as intelligent “job

aids”where the designer enters information about the present

situation in response to system queries, and the system then

provides a solution based on domain-specific rules and reason-

ing strategies. Such tools do not provide a complete working

environment for the design task but, rather, can serve as an

expert consultant when the designer is not sure how to pro-

ceed in a particular situation (Jonassen & Wilson, 1990). A num-

ber of expert system tools for instructional design have been

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

1122 •RICHEY, KLEIN, NELSON

developed by Kearsley (1985) to provide guidance for classi-

fication of learning objectives, needs assessment, cost benefit

analysis, and decisions about the appropriateness of comput-

ers for delivery of instruction. Other expert system tools have

been developed to aid in media selection (Gayeski, 1987; Har-

mon & King, 1985) and job/task analysis (Hermanns, 1990),

while researchers in artificial intelligence are developing tools

to analyze a curriculum for an intelligent tutoring system based

on prerequisites and lesson objectives (Capell & Dannenberg,

1993).

Expert system technology has also been employed in the

development of integrated instructional design systems that

provide guidance and support for the complete design pro-

cess. Merrill (1987) was an early advocate for the development

of authoring systems for computer-based instruction that pro-

vide guidance for the user throughout the design process. His

work has focused on a comprehensive instructional design en-

vironment that queries the user about the nature of the prob-

lem, guides in the specification and structuring of content,

and recommends strategies and instructional transactions (Li

& Merrill, 1991; Merrill & Li, 1989; Merrill & Thompson, 1999).

Nicolson (1988) has described a similar system named SCALD

(Scriptal CAL Designer) that was developed to produce auto-

matically computer code for computer-assisted instruction sys-

tems. Using a script representation, SCALD users enter descrip-

tions of the instructional components of the system by filling

out forms to specify the script for each component. The sys-
tem then creates an appropriately organized and sequenced

“shell”from the scripts, and content (specific questions, infor-

mation, graphics, etc.) can then be added to the shell by the

developer.

Alternative approaches to rule-based expert system tech-

nologies for knowledge-based instructional design have been

examined by other researchers. Some of these systems serve as

working environments for the instructional designer but with-

out the system-controlled advisory sessions common to expert

system approaches. Instead, these systems provide structured

environments and tools to facilitate the instructional design

process, sometimes in very specific domains such as the tools

described by Munro and Towne (1992) for the design and devel-

opment of computer-based simulations and sometimes compre-

hensive systems to support all aspects of the instructional design

process, including tutorials and support for novice designers

(Gayeski, 1987; Seyfer & Russell, 1986; Spector & Song, 1995;

Whitehead & Spector, 1994). IDioM (Gustafson & Reeves, 1990)

is an instructional design environment used by Apple Computer

employees to aid in their training design activities. Consisting of

several modules related to analysis, design, development, evalu-

ation, implementation, and management activities, IDioM helps

to impose structure on the design process while maintaining

flexibility for the designer to engage in various design and devel-

opment activities. The Instructional Design Environment (IDE),

developed at Xerox, is a similar system that allows designers to

enter and manipulate the information necessary for analysis and

specification activities (Pirolli, 1991). The environment is com-

pletely open, allowing the designer to represent the problem

in a way that is comfortable and appropriate for the situation.

Tools are provided to assist the designer in activities related to

task and content analysis, sequencing, delivery, and evaluation.

Another set of tools, the IDE-Interpreter, has been developed

to generate automatically instructional plans based on the spec-

ifications previously stored in the IDE knowledge base by the

designer (D. M. Russell, 1988). Research in this area has ex-

panded in recent years to include new commercial products

such as Designer’s Edge (Chapman, 1995) and Design Station

2000 (Gayeski, 1995), as well as tools for Web-based learning

environments (Wild, 2000)

[Link] The Practicality of Automated Instructional De-

sign. The question that remains unanswered with respect to

research in the area of knowledge-based instructional design

systems is whether such systems will be used by practicing in-

structional designers. Gayeski (1988, 1990) has pointed out sev-

eral problems with automating instructional design processes,

especially the difficulty of representing instructional design ex-

pertise as decision algorithms that can be executed by a com-

puter. She also speculates that using an automated instructional

design system may stifle creativity and that systems may need

to be customized and tailored to the design procedures and

practices of a particular organization. On the other hand, there

has been growth in the development of these tools (Kasowitz,

1998) and increased interest in design and utilization issues

(Tennyson & Baron, 1995) The open-ended workbench ap-

proach to knowledge-based instructional design can pose prob-

lems for some users, even though it may be a more appropriate

architecture given the nature of the instructional design task

(Duchastel, 1990). Because of the high degree of user control,

an instructional design workbench can be useful for experi-

enced instructional designers, helping them to streamline their

work and providing assistance in documenting and managing

the design process. But a system such as Designer’s Edge is not

designed to provide guidance to novice instructional design-

ers and, therefore, may not be useful in many situations where

trained instructional designers are in short supply.

The most appropriate architecture for knowledge-based in-

structional design systems may be a hybrid system that incor-

porates open-ended tools for experts along with an advisory

system and solution library for novices. Fischer (1991) has de-

veloped such a system for architectural design that includes

several modules: a construction kit, a simulation generator, a hy-

pertext database of design principles, and a design catalog. As

the designer works, an on-line design critic monitors activities,

interrupting the user when a design principle has been violated.

At such times, the design critic module presents an argument

for modification of the design, supporting the argument with a

design principle retrieved from a hypertext database along with

examples of correct designs in the design catalog. The user can

then browse both modules to obtain more information about

the principle that was violated and other related principles and

examples. The user can also access the simulation generator to

test the current design using “what if”scenarios that simulate

usage in various situations.

Even if we never use a knowledge-based instructional de-

sign system on a daily basis, research in this area has not been

futile. In part, the interest in developing automated instruc-

tional design systems has precipitated research into the nature

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

41. DEVELOPMENTAL RESEARCH •1123

of the instructional design process (Pirolli, personal commu-

nication). Work in this area may also help to identify those

instructional design tasks that can be successfully automated,

leaving the tasks that require creativity and “fuzzy”reasoning

to human designers. And finally, attempts to acquire and rep-

resent instructional design expertise will result in a better un-

derstanding of the nature of that expertise, suggesting alter-

native approaches to teaching instructional design (Rowland,

1992).

[Link] Overview of Studies of Automated Instructional

Design and Development. These studies are Type 2 research

directed toward the production and testing of tools that would

change design procedures, although some studies focus on only

one phase of the design process—predesign content identifica-

tion and analysis. The research on content is directed primarily

toward producing new content analysis tools and procedures

and then determining the conditions under which they can be

best used. There are often multiple units of analysis in these stud-

ies, including designers, subject-matter experts, and the design

tasks themselves. Much of this research is based on “artificial”

design tasks, or projects that have been devised solely for the

purposes of the research. Again, it is common for researchers

to use qualitative techniques in these studies.

Research on automated design tools and systems is also pri-

marily Type 2 research, however, Type 1 studies that would de-

scribe and analyze design projects using these new automated

tools and evaluate the impact on learners of the materials pro-

duced using these tools are not precluded. The focus of analysis

in the Type 2 studies of automated design tools tends to be the

tools themselves, and in some instances the designers who use

such tools are also studied. The studies are typically descriptive

and observational in nature and typically seek to identify those

conditions that facilitate successful use.

41.6 CONCLUSIONS

Developmental research methodologies facilitate the study of

new models, tools, and procedures so that we can reliably an-

ticipate their effectiveness and efficiency and, at the same, time

address the pressing problems of this field. Such research can

identify context-specificfindings and determine their relevance

for other teaching and learning environments. It can also identify

new principles of design, development, and evaluation. Devel-

opmental research techniques not only expand the empirical

methodologies of the field, but also expand the substance of

instructional technology research. As such, it can be an impor-

tant vehicle in our field’s efforts to enhance the learning and

performance of individuals and organizations alike.

The history of developmental research parallels the history

and growth of instructional technology. Findings from past re-

search provide a record of the technological development of the

field, as well as a record of the development of the instructional

design movement. The impetus for developmental research lies

in the concerns of the field at any point in time. For example,

the field is currently preoccupied with the idiosyncrasies of

e-learning, while it continues to grapple with ways of reducing

design and development cycle time while maintaining quality

standards. It is concerned with the implications of globaliza-

tion and diversity for instructional design. Issues such as these

should give rise to new research, and developmental projects

are well suited to address many of these problems.

References

Abdelraheem, A. Y. (1990). Formative evaluation in instructional de-

velopment: A multiple comparison study (Doctoral dissertation,

Indiana University, 1989). Dissertation Abstracts International—

A, 50(10), 3381.

Adamski, A. J. (1998). The development of a systems design model for

job performance aids: A qualitative developmental study (Doctoral

dissertation, Wayne State University, 1998). Dissertation Abstracts

International—A, 59(03), 789.

Akin, O. (1979). Exploration of the design process. Design Methods and

Theories, 13(3/4), 115–119.

Akin, O., Chen, C. C., Dave, B., & Pithavadian, S. (1986). A schematic

representation of the designer’s logic. Proceedings of the Joint In-

ternational Conference on CAD and Robotics in Architecture and

Construction (pp. 31–40). London: Kogan-Page.

Albero-Andres, M. (1983). The use of the Agency for Instructional Tele-

vision instructional development model in the design, production,

and evaluation of the series of Give and Take (Doctoral dissertation,

Indiana University, 1982). Dissertation Abstracts International—A,

43(11), 3489.

Alessi, S. M. (1988). Learning interactive videodisc development: A case

study. Journal of Instructional Development, 11(2), 2–7.

Anderson, J. R., Boyle, C. F., Corbett, A. T., & Lewis, M. W. (1990).

Cognitive modeling and intelligent tutoring. Artificial Intelligence,

42(1), 7–49.

Anderson, T. A., & Merrill, M. D. (2000). A design for standards-based

knowledge components. Journal of Computing in Higher Educa-

tion, 11(2), 3–29.

Andrews, D. H., & Goodson, L. A. (1980). A comparative analysis of mod-

els of instructional design. Journal of Instructional Development,

3(4), 2–16.

Aukerman, M. E. (1987). Effectiveness of an interactive video ap-

proach for CPR recertification of registered nurses (Doctoral dis-

sertation, University of Pittsburgh, 1986). Dissertation Abstracts

International—A, 47(06), 1979.

Baghdadi, A. A. (1981). A comparison between two formative evaluation

methods (Doctoral dissertation, Indiana University, 1980). Disserta-

tion Abstracts International, 41(08), 3387.

Baker, E. L. (1973). The technology of instructional development. In

R. M. W. Travers (Ed.), Second handbook of research on teaching

(pp. 245–285). Chicago: Rand McNally.

Beauchamp, M. (1991). The validation of an integrative model of stu-

dent affect variables and instructional systems design (Doctoral

dissertation, Wayne State University, 1990). Dissertation Abstracts

International—A, 51(6), 1885.

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

1124 •RICHEY, KLEIN, NELSON

Bednar, A. K. (1988). Needs assessment as a change strategy: A case

study. Performance Improvement Quarterly, 1(2), 31–39.

Bennis, W. G. (1969). Organizational development: Its nature, origins,

and prospects. Reading, MA: Addison–Wesley.

Blackstone, B. B. (1990). The development and evaluation of a simu-

lation game about being a believer in the Soviet Union (Doctoral

dissertation, University of Pittsburgh, 1989). Dissertation Abstracts

International—A, 50(7), 2024.

Boose, J. H. (1986). Expertise transfer for expert system design. New

York: Elsevier.

Borg, W. R., & Gall, M. D. (1971). Educational research: An introduc-

tion (2nd ed.). New York: David McKay.

Borras, I. (1993). Developing and assessing practicing spoken French:

A multimedia program for improving speaking skills. Educational

Technology Research and Development, 41(4), 91–103.

Bowers, D., & Tsai, C. (1990). Hypercard in educational research: An

introduction and case study. Educational Technology, 30(2), 19–24.

Braby, R., & Kincaid, J. P. (1981). Computer aided authoring and editing.

Journal of Educational Technology Systems, 10(2), 109–124.

Branch, R. M. (1997, October). Perceptions of instructional design pro-

cess models. Paper presented at the annual meeting of the Interna-

tional Visual Literacy Association, Cheyenne, WY.

Branch, R. M., & Bloom, J. R. (1995, October). The role of graphic

elements in the accurate portrayal of instructional design. Paper

presented at the annual meeting of the International Visual Literacy

Association, Tempe, AZ.

Bransford, J. D., Sherwood, R. D., Hasselbring, T. S., Kinzer, C. K., &

Williams, S. M. (1990). Anchored instruction: Why we need it and

how technology can help. In D. Nix & R. Spiro (Eds.), Cognition,

education, and multimedia (pp. 115–142). Hillsdale, NJ: Lawrence

Erlbaum.

Brecke, F., & Blaiwes, A. (1981). CASDAT: An innovative approach to

more efficient ISD. Journal of Educational Technology Systems,

10(3), 271–283.
Briggs, L. J. (Ed.). (1977). Instructional design: Principles and applica-

tions. Englewood Cliffs, NJ: Educational Technology.

Bucciarelli, L. (2001). Design knowing and learning: A socially mediated

activity. In C. M. Eastman, W. M. McCracken, & W. C. Newstetter

(Eds.), Design knowing and learning: Cognition in design educa-

tion (pp. 297–314). Amsterdam, NY: Elsevier Science.

Buch, E. E. (1989). A systematically developed training program

for microcomputer users in an industrial setting (Doctoral dis-

sertation, University of Pittsburgh, 1988). Dissertation Abstracts

International—A, 49(4), 750.

Burkholder, B. L. (1981–1982). The effectiveness of using the instruc-

tional strategy diagnostic profile to prescribe improvements in self-

instructional materials. Journal of Instructional Development, 5(2),

2–9.

Cantor, J. A. (1988). An automated curriculum development process for

Navy Technical Training. Journal of Instructional Development,

11(4), 3–12.

Capell, P. (1990). A content analysis approach used in the study of

the characteristics of instructional design in three intelligent tu-

toring systems: The LISP tutor, Bridge, and Piano Tutor (Doctoral

dissertation, University of Pittsburgh, 1989). Dissertation Abstracts

International—A, 50(7), 2024.

Capell, P., & Dannenberg, R. B. (1993). Instructional design and intelli-

gent tutoring: Theory and the precision of design. Journal of Artifi-

cial Intelligence in Education, 4(1), 95–121.

Carl, D. (1978). A front-end analysis of instructional development in

instructional television (Doctoral dissertation, Indiana University,

1977). Dissertation Abstracts International—A, 38(9), 5198.

Carliner, S. (1998). How designers make decisions: A descriptive model

of instructional design for informal learning in museums. Perfor-

mance Improvement Quarterly, 11(2), 72–92.

Carr-Chellman, A., Cuyar, C., & Breman,J. (1998). User-design: A case ap-

plication in health care training. Educational Technology Research

and Development, 46(4), 97–114.

Carroll, J. M., Thomas, J. C., Miller, L. A., & Friedman, H. P. (1980).

Aspects of solution structure in design problem solving. American

Journal of Psychology, 95(2), 269–284.

Chandrasekaran, B. (1987). Design as knowledge-based problem solving

activity. In B. Zaff (Ed.), Proceedings of a Symposium on the Cog-

nitive Condition of Design (pp. 117–147). Columbus: Ohio State

University.

Chapman, B. L. (1995). Accelerating the design process: A tool for in-

structional designers. Journal of Interactive Instructional Develop-

ment, 8(2), 8–15.

Cheng, N. (2000, September). Web-based teamwork in design

[Link] presented at SiGraDi 2000: 4th Iberoamerican

Congress of Digital Graphics, Rio de Janiero.

Chou, C., & Sun C. (1996). Constructing a cooperative distance learning

system: The CORAL experience. Educational Technology Research

and Development, 44(4), 71–84.

Clifford, G. J. (1973). A history of the impact of research on teaching. In

R. M. W. Travers (Ed.), Second handbook of research on teaching

(pp. 1–46). Chicago: Rand McNally.

Connop-Scollard, C. (1991). The ID/D chart: A representation of instruc-

tional design and development. Educational Technology, 31(12),

47–50.

Cook, F. S., & Richey, R. C. (1975). A competency-based program for

preparing vocational teachers. Journal for Industrial Teacher Edu-

cation, 12(4), 29–38.

Cooke, N. M., & McDonald, J. E. (1986). A formal methodology for

acquiring and representing expert knowledge. Proceedings of the

IEEE, 74(1), 1422–1430.

Corry, M. D., Frick, T. W., & Hansen, L. (1997). User-centered design and

usability testing of a web site: An illustrative case study. Educational

Technology Research and Development, 45(4), 65–76.

Coscarelli, W. C., & White, G. P. (1982). Applying the ID process to the

guided design teaching strategy: A case study. Journal of Instruc-

tional Development, 5(4), 2–6.

Cowell, D. M. (2001). Needs assessment activities and techniques

of instructional designers: A qualitative study (Doctoral disser-

tation, Wayne State University, 2000). Dissertation Abstracts

International—A, 61(10), 3873.

Coyle, K. (1986). The development and evaluation of an experimen-

tal computer simulation for animal science students (Doctoral

dissertation, Iowa State University, 1985). Dissertation Abstracts

International—A, 46(12), 3581.

Coyle, L. (1992). Distance education project: Extending extension pro-

gramming via telecommunications technology. Educational Tech-

nology, 32(8), 57–58.

Coyne, R. D., Rosenman, M. A., Radford, A. D., Balachandran, M., &

Gero, J. S. (1990). Knowledge-based design systems. Reading, MA:

Addison–Wesley.

Crane, G., & Mylonas, E. (1988). The Perseus project: An interactive

curriculum on classical Greek civilization. Educational Technology,

28(11), 25–32.

Cregger, R., & Metzler, M. (1992). PSI for a college physical educa-

tion basic instructional program. Educational Technology, 32(8),

51–56.

Cronbach, L. J., & Suppes, P. (Eds.). (1969). Research for tomorrow’s

schools—Disciplined inquiry for education. Report of the Commit-

tee on Educational Research of the National Academy of Education.

London: Macmillan, Callien Macmillan.

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

41. DEVELOPMENTAL RESEARCH •1125

Crowl, T. K. (1996). Fundamentals of educational research (2nd ed.).

Madison, WI: Brown & Benchmark.

Cuddy, C. (2000). Metadata evaluation: The road toward meeting our

objectives. Proceedings of the ASIS Annual Meeting, 37, 67.

Cummings, O. W. (1985). Comparison of three algorithms for analyzing

questionnaire-type needs assessment data to establish need priori-

ties. Journal of Instructional Development, 8(2), 11–16.

Dabbagh, N. H., Jonassen, D. H., Yueh, H., & Samouilova, M. (2000).

Assessing a problem-based learning approach to an introductory in-

structional design course: A case study. Performance Improvement

Quarterly, 13(2), 61–83.

Dick, W. (1991). The Singapore project: A case study in instructional

design. Performance Improvement Quarterly, 4(1), 14–22.

Diesing, P. (1991). How does social science work? Reflections on prac-

tice. Pittsburgh, PA: University of Pittsburgh Press.

Dijkstra, S. (2001). The design space for solving instructional design

problems. Instructional Science, 29(4–5), 275–290.

Driscoll, M. P. (1991). Paradigms for research in instructional systems. In

G. Anglin (Ed.), Instructional technology: Past, present, and future

(pp. 310–317). Englewood, CO: Libraries Unlimited.

Driscoll, M. P., & Tessmer, M. (1985). The rational set generator: A

method for creating concept examples for teaching and testing. Ed-

ucational Technology, 25(2), 29–32.

Duchastel, P. C. (1990). Cognitive design for instructional design. In-

structional Science, 19, 437–444.

Duffy, T. M., & Jonassen, D. H. (1992). Constructivism and the technol-

ogy of instruction. Mahwah, NJ: Lawrence Erlbaum Associates.

English, R. E., & Reigeluth, C. M. (1996). Formative research on sequenc-

ing instruction with the elaboration theory. Educational Technology

Research and Development, 44(1), 23–42.

Ericsson, K. A., & Simon, H. A. (1984). Protocol analysis. Cambridge,

MA: MIT Press.

Fischer, G. (1991). Supporting learning on demand with design envi-

ronments. In L. Birnbaum (Ed.), The International Conference on

the Learning Sciences: Proceedings of the 1991 Conference. Char-

lottesville, VA: Association for the Advancement of Computing in

Education.

Fischer, K. M., Savenye, W. C., & Sullivan, H. J. (2002). Formative eval-

uation of computer-based training for a university financial system.

Performance Improvement Quarterly, 15(1), 11–24.

Ford, J. M., & Wood, L. E. (1992). Structuring and documenting interac-

tions with subject-matter experts. Performance Improvement Quar-

terly, 5(1), 2–24.

Forsyth, J. E. (1998). The construction and validation of a model for the

design of community-based train-the-trainer instruction (Doctoral

dissertation, Wayne State University, 1997). Dissertation Abstracts

International—A, 58(11), 4242.

Fox, E. J., & Klein, J. D. (2002). What should instructional designers

know about human performance technology? Paper presented at

the annual meeting of the Association for Educational Communica-

tion and Technology, Dallas, TX.

Freeman, P. (1983). Fundamentals of design. In P. Freeman (Ed.), Soft-

ware design tutorial (pp. 2–22). New York: IEEE Computer Society

Press.

Fritz, P., & Ip, A. (1998, June). Learning engines—A functional ob-

ject model for developing learning resources for the WWW. Paper

presented at the World Conference on Educational Multimedia and

Hypermedia, Freiburg, Germany.

Fuchs, L. S., & Fuchs, D. (1986). Effects of systematic formative

evaluation: A meta-analysis. Exceptional Children, 53(3), 199–

208.

Gage, N. L. (Ed.). (1963). Handbook of research on teaching. Chicago:

Rand McNally.

Gagn´e, R. M. (1985). The conditions of learning and theory of instruc-

tion (4th ed.). New York: Holt, Rinehart and Winston.

Gagn´e, R. M., Briggs, L. J., & Wager, W. W. (1988). Principles of instruc-

tional design (3rd ed.). New York: Holt, Rinehart and Winston.

Gauger, E. P. (1987). The design, development, and field test of a proto-

type program evaluation approach for medical residency rotations

(Doctoral dissertation, Michigan State University, 1985). Disserta-

tion Abstracts International—A, 47(1), 69.

Gay, G., & Mazur, J. (1993). The utility of computer tracking tools for

user-centered design. Educational Technology, 33(4), 45–59.

Gayeski, D. M. (1987). Interactive toolkit. Ithaca, NY: OmniCom Asso-

ciates.

Gayeski, D. M. (1988). Can (and should) instructional design be auto-

mated? Performance and Instruction, 27(10), 1–5.

Gayeski, D. M. (1990). Are you ready for automated design? Training

and Development Journal, 44(1), 61–62.

Gayeski, D. M. (1995). Design Station 2000: Imagining future realities in

learning systems design. Educational Technology, 35(3), 43–47.

Gettman, D., McNelly, T., & Muraida, D. (1999). The guided approach

to instructional design advising (GAIDA): A case-based approach

to developing instructional design expertise. In J. van den Akker,

R. M. Branch, K. Gustafson, N. Nieveen, & T. Plomp (Eds.), Design

approaches and tools in education and training (pp. 175–181).

Dordrecht: Kluwer.

Goel, V., & Pirolli, P. (1988). Motivating the notion of generic design

within information processing theory: The design problem space

(Report No. DPS-1). Washington, DC: Office of Naval Research.

(ERIC Document Reproduction Service No. ED 315 041)

Gordon, J. J. (1969). A system model for civil defense training and edu-

cation. Educational Technology, 9(6), 39–45.

Greenhill, L. P. (1955). A study of the feasibility of local production of

minimum cost sound motion pictures (Pennsylvania State Univer-

sity Instructional Film Research Program). Port Washington, NY:U.S.

Naval Training Device Center, Office of Naval Research, Technical

Report No. SDC 269-7-48.

Gustafson, K. L., & Reeves, T. C. (1990). IDioM: A platform for a course

development expert system. Educational Technology, 30(3), 19–25.

Gustafson, K. L., & Reeves, T. C. (1991). Introduction. In L. J. Briggs,

K. L. Gustafson, & M. H. Tillman (Eds.), Instructional design: Prin-

ciples and applications (2nd ed., pp. 3–16). Englewood Cliffs, NJ:

Educational Technology.

Hallamon, T. C. (2002). A study of factors affecting the use of task anal-

ysis in the design of instruction (Doctoral dissertation, Wayne State

University, 2001). Dissertation Abstracts International—A, 62(12),

4131.
Harmon, P., & King, D. (1985). Expert systems: Artificial intelligence in

business. New York: Wiley.

Harris, M., & Cady,M. (1988). The dynamic process of creating hypertext

literature. Educational Technology, 28(11), 33–40.

Harrison, P. J., Seeman, B. J., Behm, R., Saba, F., Molise, G., & Williams, M.

D. (1991). Development of a distance education assessment instru-

ment. Educational Technology Research and Development, 39(4),

65–77.

Havelock, R. G. (1971). The utilisation of educational research and devel-

opment. British Journal of Educational Technology, 2(2), 84–98.

Hedberg, J., & Sims, R. (2001). Speculations on design team interactions.

Journal of Interactive Learning Research, 12(2–3), 193–208.

Heinich, R., Molenda, M., Russell, J. D., & Smaldino, S. E. (2002). In-

structional media and the new technologies of instruction (7th

ed.). Englewood Cliff, NJ: Prentice Hall.

Hermanns, J. (1990). Computer-aided instructional systems develop-

ment. Educational Technology, 30(3), 42–45.

Herrington, J., & Oliver, R. (2000). An instructional design framework

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

1126 •RICHEY, KLEIN, NELSON

for authentic learning environments. Educational Technology Re-

search and Development, 48(3), 23–48.

Higgins, N., & Reiser, R. (1985). Selecting media for instruction: An

exploratory study. Journal of Instructional Development, 8(2),

6–10.

Hilgard, E. R. (Ed.). (1964). Theories of learning and instruction. 63rd

National Society for the Study of Education Yearbook. Chicago: Uni-

versity of Chicago Press.

Hirumi, A., Savenye, W., & Allen, B. (1994). Designing interactive

videodisc-based museum exhibits: A case study. Educational Tech-

nology Research and Development, 42(1), 47–55.

Hoffman, R. R. (1987). The problem of extracting the knowledge of

experts from the perspective of experimental psychology. AI Mag-

azine, 8(2), 53–67.

Hutchins, E. (1991). The technology of team navigation. In L. B. Resnick,

J. M. Levine, & S. D. Teasley (Eds.), Perspectives on socially shared

cognition (pp. 283–307).Washington, DC: American Psychological

Association.

Jeffries, R., Turner, A. A., Polson, P. G., & Atwood, M. E. (1981). The

processes involved in designing software. In J. R. Anderson (Ed.),

Cognitive skills and their acquisition (pp. 255–283). Hillsdale, NJ:

Lawrence Erlbaum.

Jonassen, D. H. (1988). Using needs assessment data to design a grad-

uate instructional development program. Journal of Instructional

Development, 11(2), 14–23.

Jonassen, D. H., Grabinger, R. S., & Harris, N. D. C. (1991). Analyzing

and selecting instructional strategies and tactics. Performance Im-

provement Quarterly, 4(2), 77–97.

Jonassen, D. H., Hannum, W. H., & Tessmer, M. (1989). Handbook of

task analysis procedures. New York: Praeger.

Jonassen, D. H., & Wilson, B.G. (1990). Analyzing automated instruc-

tional design systems: Metaphors from related design professions.

In M. R. Simonson (Ed.). Proceedings of Selected Research Paper

Presentations at the 1990 Annual Convention of the Association

of Educational Communications and Technology (pp. 309–328).

Washington, DC: Association of Educational Communications and

Technology. (ERIC Document Reproduction Service No. ED 323

912)

Jones, M. K., Li, Z., & Merrill, M. D. (1990). Domain knowledge repre-

sentation for instructional analysis. Educational Technology, 30(10),

7–32.

Jones, R. H. (1986). The development and analysis of an interactive

video program for paralegal students in a university setting (Doctoral

dissertation, University of Pittsburgh, 1985). Dissertation Abstracts

International—A, 46(6), 1604.

Jones, T., & Richey, R. C. (2000). Rapid prototyping methodology in

action: A developmental study. Educational Technology Research &

Development, 48(2), 63–80.

Julian, M. F. (2001). Learning in action: The professional preparation

of instructional designers (Doctoral dissertation, University of Vir-

ginia). Dissertation Abstracts International—A, 62(01), 136.

Kanyarusoke, C. M. S. (1985). The development and evaluation of an

external studies form of the introductory course in educational com-

munications and technology (Doctoral dissertation, University of

Pittsburgh, 1983). Dissertation Abstracts International—A, 45(2),

387.

Kasowitz, A. (1998). Tools for Automating Instructional Design. ERIC

Digest Report No. EDO-IR-98-01.

Kearsley, G. (1985). An expert system program for making decisions

about CBT. Performance and Instruction, 24(5), 15–17.

Kearsley, G. (1986). Automated instructional development using per-

sonal computers: Research issues. Journal of Instructional Devel-

opment, 9(1), 9–15.

Kelly, G. A. (1955). The psychology of personal constructs. New York:

W. W. Norton.

Keppel, M. (2001). Optimizing instructional designer-subject matter ex-

pert communication in the design and development of multimedia

projects. Journal of Interactive Learning Research, 12(2/3), 209–

227.

Kerlinger, F. N. (1964). Foundations of behavioral research: Educa-

tional and psychological inquiry. New York: Holt, Rinehart and

Winston.

Kerr, S. T. (1983). Inside the black box: Making design decisions

for instruction. British Journal of Educational Technology, 14(1),

45–58.

Kim, S. O. (1984). Design of an instructional systems development pro-

cess for Korean education in harmony with Korean culture (Doc-

toral dissertation, Ohio State University, 1983). Dissertation Ab-

stract International—A, 44(4), 974.

King, D., & Dille, A. (1993). An early endeavor to apply quality con-

cepts to the systematic design of instruction: Successes and lessons

learned. Performance Improvement Quarterly, 6(3), 48–63.

Klein, J. D., Brinkerhoff, J., Koroghlanian, C., Brewer, S., Ku, H., &

MacPherson-Coy, A. (2000). The foundations of educational tech-

nology: A needs assessment. Tech Trends, 44(6), 32–36.

Klimczak, A. K., & Wedman,J. F. (1997). Instructional design project suc-

cess factors: An empirical basis. Educational Technology Research

& Development, 45(2), 75–83.

Kress, C. (1990). The effects of instructional systems design techniques

on varying levels of adult achievement in technical training (Doctoral

dissertation, Wayne State University, 1989). Dissertation Abstracts

International—A, 50(11), 3447.

Kunneman, D. E., & Sleezer, C. M. (2000). Using performance analysis

for training in an organization implementing ISO-9000 manufactur-

ing practices: A case study. Performance Improvement Quarterly,

13(4), 47–66.

Labeuf, J. R., & Spalter, A. M. (2001, June). A component repository

for learning objects: A progress report. Paper presented at the Joint

Conference on Digital Libraries, Roanoke, VA.

Lanzara, G. F. (1983). The design process: Frames, metaphors and games.

In U. Briefs, C. Ciborra, & L. Schneider (Eds.), Systems design for,

with and by the users. Amsterdam: North-Holland.

Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral

[Link], MA: Cambridge University Press.

Le Maistre, C. (1998). What is an expert instructional designer? Evidence

of expert instructional designer? Evidence of expert performance

during formative evaluation. Educational Technology Research &

Development, 46(3), 21–36.

Le Maistre, K., & Weston, C. (1996). The priorities established among

data sources when instructional designers revise written materi-

als. Educational Technology Research and Development, 44(1),

61–70.

Lewis, E. L., Stern, J. L., & Linn, M. C. (1993). The effect of computer

simulations on introductory thermodynamics understanding. Edu-

cational Technology, 33(1), 45–58.

Li, Z., & Merrill, M. D. (1991). ID Expert 2.0: Design theory and pro-

cess. Educational Technology Research and Development, 39(2),

53–69.

Lieberman, A., & Miller, L. (1992). Professionaldevelopment of teachers.

In M.C. Alkin (Ed.), Encyclopedia of educational research (6th ed.,

pp. 1045–1051). New York: Macmillan.

Link, N., & Cherow-O’Leary, R. (1990). Research and development of

print materials at the Children’s Television Workshop. Educational

Technology Research & Development, 38(4), 34–44.

Luiz, T. (1983). A comparative study of humanism and pragmatism

as they relate to decision making in instructional development

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

41. DEVELOPMENTAL RESEARCH •1127

processes (Doctoral dissertation, Michigan State University, 1982).

Dissertation Abstracts International—A, 43(12), 3839.

Maly, K. Overstreet, C. M., Gonzalez, A., Denbar, M. L., Cutaran, R., &

Karunaratne, N. (1998, June). Automated content synthesis for in-

teractive remote instruction. Paper presented at the World Con-

ference on Educational Multimedia and Hypermedia, Freiburg,

Germany.

Mann, E. (1996). A case study of instructional design practices: Impli-

cations for instructional designers. In M. R. Simonson, M. Hays, &

S. Hall (Eds.), Proceedings of selected research papers presented

at the 1996 Annual Convention of the Association of Educational

Communications and Technology (pp. 455–463). Washington, DC:

Association of Educational Communications and Technology.

Martin, B., & Bramble, W. (1996). Designing effective video teletraining

instruction: The Florida teletraining project. Educational Technol-

ogy Research and Development, 44(1), 85–99.

McAleese, R. (1988). Design and authoring: A model of cognitive pro-

cesses. In H. Mathias, N. Rushby, & R. Budgett (Eds.), Designing new

systems for learning (pp. 118–126). New York: Nichols.

McCombs, B. L. (1986). The instructional systems development model:

A review of those factors critical to its implementation. Educational

Technology and Communications Journal, 34(2), 67–81.

McDaniel, K., & Liu, M. (1996). A study of project management tech-

niques for developing interactive multimedia programs: A practi-

tioner’s perspective. Journal of Research on Computing in Educa-

tion, 29(1), 29–48.

McGraw, K. L. (1989). Knowledge acquisition for intelligent instruc-

tional systems. Journal of Artificial Intelligence in Education, 1(1),

11–26.

McKenney, S. (2002). Computer-based support for science education

materials development in Africa: Exploring potentials (Doctoral dis-

sertation, Universiteit Twente, The Netherlands, 2001). Dissertation

Abstracts International—C, 63(03), 355.

Means, B., & Gott, S. (1988). Cognitive task analysis as a basis for tutor

development: Articulating abstract knowledge representations. In

J. Psotka, L. D. Massey, & S. A. Mutter (Eds.), Intelligent tutoring

systems: Lessons learned (pp. 35–57). Hillsdale, NJ: Erlbaum.

Means, T. B., Jonassen, D. H., & Dwyer, F. M. (1997). Enhancing rele-

vance: Embedded ARCS strategies vs. purpose. Educational Tech-

nology Research & Development, 45(1), 5–17.

Medsker, K. L. (1992). NETwork for excellent teaching: A case study in

university instructional development. Performance Improvement

Quarterly, 5(1), 35–48.

Mendes, A. J., & Mendes, T. (1996). AIDA: An integrated authoring en-

vironment for educational software. Educational Technology Re-

search and Development, 44(4), 57–70.

Merrill, M. D. (1973). Cognitive and instructional analysis for cognitive

transfer tasks. Audio Visual Communications Review, 21(1), 109–

125.

Merrill, M. D. (1987). Prescriptions for an authoring system. Journal of

Computer-Based Instruction, 14(1), 1–10.

Merrill, M. D. (1999). Instructional transaction theory: Instructional de-

sign based on knowledge objects. In C. M. Reigeluth (Ed.), Instruc-

tional design theories and models (Vol. 2). Mahwah, NJ: Lawrence

Erlbaum Associates.

Merrill, M. D. (2001). A knowledge object and mental model approach

to a physics lesson. Educational Technology, 41(1), 36–47.

Merrill, M. D., & Li, Z. (1989). An instructional design expert system.

Journal of Computer-Based Instruction, 16(3), 95–101.

Merrill, M. D., Li, Z., & Jones, M. K. (1990a). Limitations of first genera-

tion instructional design. Educational Technology, 30(1), 7–11.

Merrill, M. D., Li, Z., & Jones, M. K. (1990b). Second generation instruc-

tional design. Educational Technology, 30(2), 7–14.

Merrill, M. D., Li, Z., & Jones, M. K. (1991). Instructional transaction

theory: An introduction. Educational Technology, 31(6), 7–12.

Merrill, M. D., & Thompson, B. M. (1999). The IDXelerator: Learning-

centered instructional design. In J. Vd Akker, R. M. Branch, K.

Gustafson, N. Nieveen, & T. Plomp (Eds.), Design approaches and

tools in education and training (pp. 265–277). Dordrecht: Kluwer

Academic.

Mielke, K. W. (1990). Researchand development at the Children’s Televi-

sion Workshop. Educational Technology Research & Development,

38(4), 7–16.

Milazzo, C. (1981). Formative evaluation process in the development

of a simulation game for the socialization of senior nursing students

(Doctoral dissertation, Columbia University Teachers College). Dis-

sertation Abstracts International—A, 42(5), 1935.

Mishler, E. G. (1979). Meaning in context: Is there any other kind? Har-

vard Educational Review, 49(1), 1–19.

Moallem, M. (1998). Reflection as a means of developing expertise in

problem solving, decision making, and complex thinking of design-

ers. In N. J. Maushak, C. Schlosser, T. N. Lloyd, & M. Simonson (Eds.),

Proceedings of Selected Research and Development Presentations

at the National Convention of the Association for Educational

Communications and Technology (pp. 281–289). Washington, DC:

Association for Educational Communications and Technology.

Munro, A., & Towne, D. M. (1992). Productivity tools for simulation-

centered training development. Educational Technology Research

and Development, 40(4), 65–80.

Murphy, D. (1992). Is instructional design truly a design activity? Edu-

cational and Training Technology International, 29(4), 279–282.

Nadin, M., & Novak, M. (1987). MIND: A design machine. In P. J.

W. ten Hagen & T. Tomiyama (Eds.), Intelligent CAD Systems I

(pp. 146–171). New York: Springer-Verlag.

Nadolski, R., Kirschner, P., van Merrienboer, J., & Hummel, H. (2001).

A model for optimizing step size of learning tasks in competency-

based multimedia practicals. Educational Technology Research &

Development, 49(3), 87–103.

National Center for Educational Research and Development. (1970).

Educational research and development in the United States.

Washington, DC: United States Government Printing Office.

Naveh-Benjamin, J., McKeachie, W. J., Lin, Y., & Tucker, D. G. (1986).

Inferring students’cognitive structures and their development using

the “ordered tree technique.”Journal of Educational Psychology,

78, 130–140.

Nelson, W. A. (1989). Artificial intelligence knowledge acquisition tech-

niques for instructional development. Educational Technology Re-

search and Development, 37(3), 81–94.

Nelson, W. A. (1990). Selection and utilization of problem informa-

tion by instructional designers (Doctoral dissertation, Virginia Poly-

technic Institute and State University, 1988). Dissertation Abstract

International—A, 50(4), 866.

Nelson, W. A., Magliaro, S. G., & Sherman, T. M. (1988). The intellectual

content of instructional design. Journal of Instructional Develop-

ment, 11(1), 29–35.

Nelson, W. A., & Orey, M. A. (1991). Reconceptualizing the instruc-

tional design process: Lessons learned from cognitive science. Pa-

per presented at the annual meeting of the American Educational

Research Association, Chicago, IL. (ERIC Document Reproduction

Service No. ED 334 268)

Nicolson, R. (1988). SCALD—Towards an intelligent authoring system.

In J. Self (Ed.), Artificial intelligence and human learning (pp.

236–254). London: Chapman and Hall.

Nieveen, N., & van den Akker, J. (1999). Exploring the potential of a

computer tool for instructional developers. Educational Technol-

ogy Research & Development, 47(3), 77–98.

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

1128 •RICHEY, KLEIN, NELSON

Noel, K. (1991). Instructional systems development in a large-scale edu-

cation project. Educational Technology Research & Development,

39(4), 91–108.

O’Neil, H. L., Faust, G. W., & O’Neil, A. F. (1979). An author training

course. In H. L. O’Neil (Ed.), Procedures for instructional systems

development (pp. 1–38). New York: Academic Press.

O’Quin, K., Kinsey, T. G., & Beery, D. (1987). Effectiveness of a

microcomputer-training workshop for college professionals. Com-

puters in Human Behavior, 3(2), 85–94.

Open Courseware. (2002). Open Courseware: Simple idea, profound

implications. Syllabus, 15(6), 12–14.

Orey, M. A., & Nelson, W. A. (1993). Development principles for in-

telligent tutoring systems: Integrating cognitive theory into the de-

velopment of computer-based instruction. Educational Technology

Research and Development, 41(1), 59–72.

Parer, M. (1978). A case study in the use of an instructional development

model to produce and evaluate an instructional television program

(Doctoral dissertation, Indiana University, 1977). Dissertation Ab-

stracts International—A, 38(6), 3229.

Park, I., & Hannafin, M. J. (1993). Empirically-based guidelines for the

design of interactive multimedia. Educational Technology Research

and Development, 41(3), 63–85.

Pelz, D. C. (1967, July). Creative tensions in the research and develop-

ment climate. Science, 160–165.

Petry, B. A., and Edwards, M. L. (1984). Systematic development of an

applied phonetics course. Journal of Instructional Development,

7(4), 6–11.

Phillips, J. H. (2000). Evaluating training programs for organizational

impact (Doctoral dissertation, Wayne State University, 2000). Dis-

sertation Abstracts International—A61(03), 840.

Phillips, R. (2001). A case study of the development and project man-

agement of a web/CD hybrid application. Journal of Interactive

Learning Research, 12(2/3), 229–247.

Piper, A. J. (1991). An analysis and comparison of selected project

management techniques and their implications for the instructional

development process (Doctoral dissertation, Michigan State Uni-

versity, 1990). Dissertation Abstracts International—A, 51(12),

4010.

Pirolli, P. (1991). Computer-aided instructional design systems. In

H. Burns, J. W. Parlett, & C. L. Redfield (Eds.), Intelligent tutoring sys-

tems: Evolutions in design (pp.105–125). Hillsdale, NJ: Lawrence

Erlbaum Associates.

Pirolli, P. L., & Greeno, J. G. (1988). The problem space of instruc-

tional design. In J. Psotka, L. D. Massey, & S. A. Mutter (Eds.), Intel-

ligent tutoring systems: Lessons learned (pp. 181–201). Hillsdale,

NJ: Lawrence Erlbaum.

Pizzuto, A. E. (1983). The development and evaluation of a simulation

game demonstrating diffusion communications in a corporate or-

ganization (Doctoral dissertation, University of Pittsburgh, 1982).

Dissertation Abstracts International—A, 43(4), 1273.

Plass, J. L., & Salisbury, M. W. (2002). A living-systems design model for

web-based knowledge management systems. Educational Technol-

ogy Research & Development, 50(1), 35–57.

Plummer, K. H., Gillis, P. D., Legree, P. J., & Sanders, M. G. (1992). The de-

velopment and evaluation of a job aid to support mobile subscriber

radio-telephone terminal (MSRT). Performance Improvement Quar-

terly, 5(1), 90–105.

Quinn, J. (1994). Connecting education and practice in an instructional

design graduate program. Educational Technology Research and

Development, 42(3), 71–82.

Rathbun, G. A. (1999). Portraying the work of instructional design:

An activity-oriented analysis. In K. E. Sparks & M. R. Simonson

(Eds.), Proceedings of Selected Research and Development Papers

Presented at the 1999 Annual Convention of the Association of

Educational Communications and Technology (pp. 391–402).

Washington, DC: Association of Educational Communications and

Technology.

Rathbun, G., Saito, R., & Goodrum, D. A. (1997). Reconceiving ISD:

Three perspectives on rapid prototyping as a paradigm shift. In

O. Abel, N. J. Maushak, K. E. Wright, & M. R. Simonson (Eds.),

Proceedings of Selected Research and Development Presentations

at the 1997 Annual Convention of the Association of Educational

Communications and Technology (pp. 291–296). Washington, DC:

Association of Educational Communications and Technology.

Reigeluth, C. M. (1983). Instructional design: What is it and why is it? In

C. M. Reigeluth (Ed.), Instructional design theories and models: An

overview of their current status (pp. 3–36). Hillsdale, NJ: Lawrence

Erlbaum Associates, Publishers.

Reigeluth, C. M., & Curtis, R. V. (1987). Learning situations and instruc-

tional models. In R. M. Gagn´e (Ed.), Instructional technology: Foun-

dations (pp. 175–206). Hillsdale, NJ: Lawrence Erlbaum.

Reitman, W. (1964). Cognition and thought. New York: John Wiley.

Rezabek, L. L., & Cochenour, J. J. (1996). Perceptions of the ID pro-

cess: The influence of visual display. In M. R. Simonson, M. Hays &

S. Hall (Eds.), Proceedings of Selected Research and Development

Presentations at the 1996 Annual Convention of the Association

of Educational Communications and Technology (pp. 582–593).

Washington, DC: Association of Educational Communications and

Technology.

Richey, R. C. (1992). Designing instruction for the adult learner:

Systemic training theory and practice. London/Bristol, PA: Kogan

Page/Taylor and Francis.

Richey, R. C. (1993). The knowledge base of instructional design. Pa-

per presented at the 1993 Annual Meeting of the Association for

Educational Communications and Technology in New Orleans, LA.

Richey, R. C. (1998). The pursuit of useable knowledge in instructional

technology. Educational Technology Research and Development,

46(4), 7–22.
Richey, R. C., & Tessmer, M. (1995). Enhancing instructional systems de-

sign through contextual analysis. In B. Seels (Ed.), Instructional de-

sign fundamentals: A reconsideration (pp. 189–199). Englewood

Cliffs, NJ: Educational Technology.

Riplinger, J. A. (1987). A survey of task analysis activities used by instruc-

tional designers (Doctoral dissertation, University of Iowa, 1985).

Dissertation Abstracts International—A, 47(3), 778.

Rojas, A. M. (1988). Evaluation of sales training impact: A case study us-

ing the organizational elements model. Performance Improvement

Quarterly, 1(2), 71–84.

Ross, K. R. (1998). Blending authentic work projects and instructional

assignments: An adaptation process. Educational Technology Re-

search and Development, 46(3), 67–79.

Roth, W. M. (2001). Modeling design as a situated and distributed pro-

cess. Learning and Instruction, 11(3), 211–239.

Rouch, M. A. (1969). A system model for continuing education for the

ministry. Educational Technology, 9(6), 32–38.

Rowland, G. (1992). What do instructional designers actually do? An

initial investigation of expert practice. Performance Improvement

Quarterly, 5(2), 65–86.

Rowland, G. (1993). Designing and instructional design. Educational

Technology Research and Development, 41(1), 79–91.

Rowland, G. (1996). “Lighting the fire”of design conversations. Educa-

tional Technology, 36(1), 42–45.

Rowland, G., & Adams, A. (1999). Systems thinking in instructional

design. In J. van den Akker, R. M. Branch, K. Gustafson, N. Nieveen,

& T. Plomp (Eds.), Design approaches and tools in education and

training (pp. 29–44). Dordrecht: Kluwer.

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

41. DEVELOPMENTAL RESEARCH •1129

Rowland, G., Parra, M. L., & Basnet, K. (1994). Educating instructional

designers: Different methods for different outcomes. Educational

Technology, 34(6), 5–11.

Roytek, M. A. (2000). Contextual factors affecting the use of rapid proto-

typing within the design and development of instruction (Doctoral

dissertation, Wayne State University, 1999). Dissertation Abstracts

International—A, 61(01), 76.

Rushby, N. (1986). A knowledge-engineering approach to instructional

design. Computer Journal, 29(5), 385–389.

Russell, C. M. (1990). The development and evaluation of an interactive

videodisc system to train radiation therapy technology students on

the use of the linear accelerator (Doctoral dissertation, University of

Pittsburgh, 1988). Dissertation Abstracts International—B, 50(3),

919.

Russell, D. M. (1988). IDE: The interpreter. In J. Psotka, L. D. Massey, &

S. A. Mutter (Eds.), Intelligent tutoring systems: Lessons learned

(pp. 323–349). Hillsdale, NJ: Lawrence Erlbaum.

Saettler, P. (1990). The evolution of American educational technology.

Englewood, CO: Libraries Unlimited.

Schank, R. C., Berman, T. R., & Macpherson, K. A. (1999). Learning by

doing. In C. M. Reigeluth (Ed.), Instructional design theories and

models:Vol. II. Mahwah, NJ: Lawrence Erlbaum Associates.

Schon, D. A. (1983). The reflective practitioner: How professionals

think and act. New York: Basic Books.

Schon, D. (1985). The design studio: An exploration of its traditions

and potentials. Portland, OR: RIBA Publications for RIBA Building

Industry Trust.
Schon, D. A. (1987). Educating the reflective practitioner: Toward

a new design for teaching and learning in the professions. San

Francisco: Jossey–Bass.

Schulz, R. E. (1979). Computer aids for developing tests and instruction.

In H. L. O’Neil (Ed.) Procedures for instructional systems develop-

ment (pp. 39–66). New York: Academic Press.

Schulz, R. E., & Wagner, H. (1981). Development of job aids for instruc-

tional systems development (ARI TechnicalRepor t No. ARI-TR-527).

Alexandria, VA: U.S. Army Research Institute. (ERIC Document Re-

production Service No. 205 202)

Schvaneveldt, R. W. (1990). Pathfinder associative networks: Studies

in knowledge organization. Norwood, NJ: Ablex.

Schwen, T. M. (1977). Professional scholarship in educational technol-

ogy: Criteria for judging inquiry. Audio Visual Communication Re-

view, 25(1), 5–24.

Seels, B. (1994). An advisor’s view: Lessons learned from developmen-

tal research dissertations. Paper presented at the 1994 Annual Meet-

ing of the Association for Educational Communications and Technol-

ogy, Nashville, TN.

Seels, B. B., & Richey, R. C. (1994). Instructional technology: The def-

inition and domains of the field. Washington, DC: Association for

Educational Communications and Technology.

Seyfer, C., & Russell, J. D. (1986). Success story: Computer managed

instruction development. Performance and Instruction, 25(9),

5–8.

Shambaugh, N., & Magliaro, S. (2001). A reflexive model for teaching

instructional design. Educational Technology Research & Develop-

ment, 49(2), 69–92.

Shedroff, N. (1994). Information interaction design: A unified field

theory of design [On-line]. Available at: [Link]

thoughts/unified/.

Shellnut, B. J. (1999). The influence of designer and contextual vari-

ables on the incorporation of motivational components to instruc-

tional design and the perceived success of a project (Doctoral

dissertation, Wayne State University, 1999). Dissertation Abstracts

International—A, 60(06), 1993.

Sherry, L., & Myers, K. M. (1998). The dynamics of collaborative design.

IEEE Transactions on Professional Communication, 41(2), 123–

139.

Simon, H. A. (1973). The structure of ill-structured problems. Artificial

Intelligence, 4, 181–201.

Simon, H. A. (1981). The sciences of the artificial (2nd ed.). Cambridge,

MA: MIT Press.

Smith, M. (1993). Evaluation of executive development: A case study.

Performance Improvement Quarterly, 6(1), 26–42.

Spector, J. M., Muraida, D. J., & Marlino, M. R. (1992). Cognitively based

models of courseware development. Educational Technology Re-

search and Development, 40(2), 45–54.

Spector, J. M., & Song, D. (1995). Automated instructional design advis-

ing. In R. D. Tennyson & A. E. Baron (Eds.), Automating instruc-

tional design: Computer-based development and delivery tools.

New York: Springer-Verlag.

Stowe, R. A. (1973). Research and the systems approach as methodolo-

gies for education. Audio-Visual Communications Review, 21(2),

165–175.

Stumpf, S. C., & McDonnell, J. T. (1999). Relating argument to design

problem framing. Proceedings of the 4th Design Thinking Research

Symposium, Cambridge, MA.

Suchman, L. (1987). Plans and situated actions: The problem of hu-

man/machine communication. New York: Cambridge University

Press.

Sullivan, H. J. (1984). Instructional development through a national

industry-education partnership. Journal of Instructional Develop-

ment, 7(4), 17–22.

Sullivan, H., Ice, K., & Niedermeyer, F. (2000). Long-term instructional

development: A 20-year ID and implementation project. Educa-

tional Technology Research and Development, 48(4), 87–99.

Taylor, B., & Ellis, J. (1991). An evaluation of instructional systems de-

velopment in the Navy. Educational Technology Research and De-

velopment, 39(1), 93–103.

Taylor,R. L. M. (1987). An analysis of development and design models for

microcomputer-based instruction (Doctoral dissertation, Syracuse

University, 1986). Dissertation Abstracts International—A, 47(8),

3011.

Tenney, Y. J., & Kurland, L. J. (1988). The development of troubleshoot-

ing expertise in radar mechanics. In J. Psotka, L. D. Massey, &

S. A. Mutter (Eds.), Intelligent tutoring systems: Lessons learned

(pp. 59–83). Hillsdale, NJ: Erlbaum.

Tennyson, R. D., & Baron, A. E. (1995). Automating instructional de-

sign: Computer-based development and delivery [Link]:

Springer-Verlag.

Tessmer, M., McCann, D., & Ludvigsen, M. (1999). Reassessing training

programs: A model for identifying training excesses and deficien-

cies. Educational Technology Research and Development, 47(2),

86–99.

Tessmer, M., & Richey, R. C. (1997) The role of context in learning

and instructional design. Educational Technology Research and

Development, 45(2), 85–115.

Tracey, M. W. (2002). The construction and validation of an instruc-

tional design model for incorporating multiple intelligences (Doc-

toral dissertation, Wayne State University, 2001). Dissertation Ab-

stracts International—A, 62(12), 4135.

Travers, R. M. W. (Ed.). (1973). Second handbook of research on teach-

ing. Chicago: Rand McNally.

Twitchell, S., Holton, E. F., & Trott, J. W. (2000). Technical training eval-

uation practices in the United States. Performance Improvement

Quarterly, 13(3), 84–110.

van den Akker, J. (1999). Principles and methods of development re-

search. In J. van den Akker, R. M. Branch, K. Gustafson, N. Nieveen,

P1: MRM/FYX P2: MRM/UKS QC: MRM/UKS T1: MRM

PB378-41 [Link] September 8, 2003 15:16 Char Count= 0

1130 •RICHEY, KLEIN, NELSON

& T. Plomp (Eds.), Design approaches and tools in education and

training (pp. 1–14). Dordrecht: Kluwer Academic.

van Merrienboer, J. J. G. V., Jelsma, O., & Paas, R. G. W. C. (1992).

Training for reflective expertise: A four-component instructional de-

sign model for complex cognitive skills. Educational Technology

Research and Development, 40(2), 23–43.

Visscher-Voerman, J. I. A. (1999). Design approaches in training and

education: A reconstructive study. Doctoral dissertation, Univer-

siteit Twente, The Netherlands.

Wagner, N. (1984). Instructional product evaluation using the staged

innovation design. Journal of Instructional Development, 7(2),

24–27.

Walz, D., Elam, J., & Curtis, B. (1993). Inside a design team: Knowledge

acquisition, sharing and integration. Communications of the ACM,

36(10), 63–77.

Watson, J. E., & Belland, J. C. (1985). Use of learner data in selecting

instructional content for continuing education. Journal of Instruc-

tional Development, 8(4), 29–33.

Webster, H. (2001, September). The design diary: Promoting reflective

practice in the design studio. Paper presented at the Architectural

Education Exchange, Cardiff, UK.

Wedman,J., & Tessmer, M. (1993). Instructional designers’decisions and

priorities: A survey of design practice. Performance Improvement

Quarterly, 6(2), 43–57.

Weiss, D. J. (1979). Computerized adaptive achievement testing. In

H. L. O’Neil (Ed.), Procedures for instructional systems develop-

ment (pp. 129–164). New York: Academic Press.

Wenger, E. (1987). Artificial intelligence and tutoring systems. Los

Altos, CA: Morgan Kaufmann.

Weston, C., McAlpine, L., & Bordonaro, T. (1995). A model for under-

standing formative evaluation in instructional design. Educational

Technology Research and Development, 43(3), 29–48.

Whitehead, L. K., & Spector, M. J. (1994, February). A guided ap-

proach to instructional design advising. Paper presented at the

International Conference of the Association for the Development of

Computer-Based Instructional Systems, Nashville, TN.

Wild, M. (2000). Designing and evaluating an educational performance

support system. British Journal of Educational Technology 31(1),

5–20.

Wiley, D. A. (Ed.). (2000). Instructional use of learning objects. Indi-

anapolis, IN: Association for Educational Communications and Tech-

nology.

Wilson, B. G. (1995, April). Situated instructional design. Paper pre-

sented at the annual meeting of the American Educational Research

Association, San Francisco, CA.

Wittrock, M. C. (1967). Product-oriented research. Educational Forum,

31(1), 145–150.

Wreathall, J., & Connelly, E. (1992). Using performance indicators to

evaluate training effectiveness: Lessons learned. Performance Im-

provement Quarterly, 5(3), 35–43.

Yin, R. K. (1992). Case study design. In M. C. Alkin (Ed.), Encyclopedia

of educational research (pp. 134–137). New York: Macmillan.

Zaff, B. S. (Ed.). (1987). The cognitive condition of design. Proceedings

of a Symposium on the Cognitive Condition of Design. Columbus:

Ohio State University.

Zemke, R. (1985, October). The systems approach: A nice theory but

....Training, 103–108.

Zielinski, D. (2000). Objects of desire. Training, 37(9), 126–134.

Citations (25)

References (282)

 ... Dalam kajian pembangunan modul, kajian analisis keperluan adalah sebahagian daripada
proses kajian reka bentuk dan pembangunan modul. Menurut Richey dan Klein (2007), kekuatan
kajian reka bentuk dan pembangunan modul ialah kajian ini digunakan untuk menyelesaikan
masalah dalam konteks yang spesifik. Fasa analisis ialah fasa permulaan kajian iaitu maklumat
dalam konteks dan persekitaran yang hendak dikaji dikumpulkan (Saedah Siraj, Norlidah Alias,
Dorothy DeWitt, & Zaharah Hussin, 2013). ...

... Dalam kajian ini, analisis keperluan dilakukan setelah masalah dikenal pasti dan maklumat tentang
perincian dan spesifikasi modul diperoleh daripada guru. Menurut Richey dan Klein (2007) analisis
keperluan adalah untuk menentukan spesifikasi produk. Jadi, kajian analisis keperluan adalah penting
dalam menentukan produk bagaimanakah yang sesuai dihasilkan untuk guru supaya permasalahan sedia
ada dapat diselesaikan demi pengajaran yang lebih berkesan. ...

... Fasa analisis memberikan maklumat yang penting untuk digunakan dalam fasa reka bentuk dan
pembangunan modul seterusnya ( Gagne et al., 2005). Maklumat hasil analisis keperluan digunakan
untuk menentukan kandungan bahan instruksi secara optimal dan kaedah penyampaian
pengajaran (Richey & Klein, 2007). ...
Analisis keperluan terhadap pengguna sasaran modul pendekatan berasaskan bermain bagi
pengajaran dan pembelajaran kemahiran bahasa kanak-kanak prasekolah

Article

Full-text available

o Jan 2015

Aliza Ali

View

Show abstract

 ... By name Developmental research is still unclear and is confused with research conducted for
the study of human growth and development. Richey et al. [6] in their bid to clear up the
confusion, states the simplest form of developmental research as used in instructional design as
either ...

... • The study of instructional design, development and evaluation process as a whole or particular
process components [6]. ...

E-LEARNING GRAPHICAL USER INTERFACE DEVELOPMENT USING THE ADDIE INSTRUCTION DESIGN
MODEL AND DEVELOPMENTAL RESEARCH: THE NEED TO ESTABLISH VALIDITY AND RELIABILITY

Article

Full-text available

o Mar 2019

Yaw Ofosu-Asare

o
Harry Barton Essel

Frank Mensah Bonsu

View

Show abstract

 ... Validasi model pembelajaran dengan metode FGD dilakukan dengan tujuan untuk
memperoleh masukan dan perbaikan terhadap model pembelajaran yang telah dikembangkan.
Metode validasi ini sesuai dengan yang dinyatakan Richey (2005) (Borich, 1994). ...

mahrsoen78

Article

Full-text available

o Aug 2018

Muh. Makhrus

View

 ... Akker (1999) mengemukakan bahwa perkembangan bidang media dan teknologi pendidikan
begitu cepat menempatkan penelitian pengembangan sangat menonjol. Misalnya, Flagg (1990)
menggarisbawahi peran evaluasi formatif untuk (media) peningkatan program dan
penelitian; Richey & Nelson (1996) telah menggambarkan hal yang sama dalam analisis paling
komprehensif dari developmental research dalam sub-domain media dan teknologi pendidikan
(Gambar 2) dan menyatakan sebagai berikut: its ultimate aim: "improving the processes of
instructional design, development, and evaluation based on either situation- specific problem
solving or generalized inquiry procedures". Jadi tujuan penelitian pengembangan dalam sub-
domain media dan teknologi ini adalah untuk menigkatkan proses desain instruksional,
pengembangan, dan evaluasi, baik yang didasarkan pada situasi pemecahan masalah spesifik
maupun pada inquiry procedures yang digeneralisasikan. ...

PHILOSOPHICAL PERSPECTIVES THAT DESCRIBE HOW THE WORK IN CHEMISTRY EDUCATION

Conference Paper

Full-text available
 o Jun 2018

Albinus Silalahi

View

Show abstract

 ... Akker (1999) mengemukakan bahwa perkembangan bidang media dan teknologi pendidikan
begitu cepat menempatkan penelitian pengembangan sangat menonjol. Misalnya, Flagg (1990)
menggarisbawahi peran evaluasi formatif untuk (media) peningkatan program dan
penelitian; Richey & Nelson (1996) telah menggambarkan hal yang sama dalam analisis paling
komprehensif dari developmental research dalam sub-domain media dan teknologi pendidikan
(Gambar 2) dan menyatakan sebagai berikut: its ultimate aim: "improving the processes of
instructional design, development, and evaluation based on either situation- specific problem
solving or generalized inquiry procedures". Jadi tujuan penelitian pengembangan dalam sub-
domain media dan teknologi ini adalah untuk menigkatkan proses desain instruksional,
pengembangan, dan evaluasi, baik yang didasarkan pada situasi pemecahan masalah spesifik
maupun pada inquiry procedures yang digeneralisasikan. ...

DEVELOPMENT RESEARCH & RESEARCH AND DEVELOPMENT

Presentation

Full-text available

o Jun 2018

Albinus Silalahi

View

Show abstract

 English Teaching Efficacy Enhancement Module (MPEBI) For Early Childhood Education
Teachers: Measuring Its Validity Through Experts Viewpoints

Article

Full-text available

o Jan 2019
 o

Nur Nazuha

View

Show abstract

 References

Chapter

o Oct 2018

Marc Trestini

View

 Design and Development of Constructivist Multimedia Learning Environment Enhancing Skills

in Computer Programming

Conference Paper

o Jul 2017

Charuni Samat

Sumalee Chaijaroen

Issara Kanjug

o
Pornsawan Vongtathum

View

 The many facets of design and research in instructional design

Chapter

o Jan 2014

o W.A. Nelson

View

 Design and development research

Chapter

o Jan 2014

o R.C. Richey

o J.D. Klein

View

Show abstract

 Show more

Join ResearchGate to find the people and research you need to help your work.

 15+ million members

 118+ million publications

 700k+ research projects

Join for free

Recommended publications

Discover more publications, questions and projects in Problem Solving

Article

Full-text available

Developmental research: Studies of instructional design and development


Rita C. Richey


James D. Klein

View full-text

Article

Full-text available

Developmental research methods: Creating knowledge from instructional design and development
practic...

March 2005 · Journal of Computing in Higher Education


Rita C. Richey


James D. Klein

THE PURPOSE OF THIS ARTICLE is to provide direction to those entertaining a developmental research
project. There are two categories of developmental research, both of which are examined here. The two
types vary in terms of the extent to which the conclusions resulting from the research are generalizable
or contextually specific. This article describes developmental research in terms of the ... [Show full
abstract]

View full-text

Chapter

Full-text available

Developmental research

January 1995


Rita C. Richey


Wayne A. Nelson
The field of instructional technology has traditionally involved a unique blend of theory and practice.
This blend is most obvious in developmental research, which involves the production of knowledge with
the ultimate aim of improving the processes of instructional design, development, and evaluation. It is
based on either situation-specific problem solving or generalized inquiry procedures. ... [Show full
abstract]

View full-text

Article

Research on instructional development

September 1997 · Educational Technology Research and Development


Rita C. Richey

The instructional technology field is periodically faced with discussion of the appropriate direction of its
research and more specifically of the relationship between research and practice. This paper extends the
discussion by examining the nature and scope of developmental research. Two categories of
developmental research are explored. They vary in terms of the extent to which the conclusions ...
[Show full abstract]

Read more

Discover more
abject

of the most contemptible kind

aberration

a state or condition markedly different from the norm

abjure

formally reject or disavow a formerly held belief

abnegation

the denial and rejection of a doctrine or belief

abrogate

revoke formally

abscond

run away, often taking something or somebody along

abstruse
difficult to penetrate

accede

yield to another's wish or opinion

accost

approach and speak to someone aggressively or insistently

accretion

an increase by natural growth or addition

acumen

shrewdness shown by keen insight

adamant

impervious to pleas, persuasion, requests, or reason

admonish

scold or reprimand; take to task

adumbrate

describe roughly or give the main points or summary of

adverse

in an opposing direction

advocate

a person who pleads for a person, cause, or idea

affluent

having an abundant supply of money or possessions of value

aggrandize

embellish; increase the scope, power, or importance of

alacrity

liveliness and eagerness

alias

a name that has been assumed temporarily

ambivalent

uncertain or unable to decide about what course to follow

amenable

disposed or willing to comply

amorphous

having no definite form or distinct shape

anachronistic

chronologically misplaced

anathema

a formal ecclesiastical curse accompanied by excommunication

annex

attach to

antediluvian

of or relating to the period before the biblical flood

antiseptic

thoroughly clean and free of disease-causing organisms

apathetic

showing little or no emotion or animation

antithesis

exact opposite

apocryphal

being of questionable authenticity

approbation

official acceptance or agreement

arbitrary

based on or subject to individual discretion or preference

arboreal

of or relating to or formed by trees

arcane

requiring secret or mysterious knowledge

archetypal
of an original type after which other things are patterned

arrogate

seize and take control without authority

ascetic

someone who practices self denial as a spiritual discipline

aspersion

a disparaging remark

assiduous

marked by care and persistent effort

atrophy

a decrease in size of an organ caused by disease or disuse

bane

something causing misery or death

bashful

self-consciously timid

beguile

influence by slyness

bereft

sorrowful through loss or deprivation

blandishment

flattery intended to persuade

bilk

cheat somebody out of what is due, especially money

bombastic

ostentatiously lofty in style

cajole

influence or urge by gentle urging, caressing, or flattering

callous

emotionally hardened
calumny

a false accusation of an offense

camaraderie

the quality of affording easy familiarity and sociability

candor

the quality of being honest and straightforward

capitulate

surrender under agreed conditions

carouse

engage in boisterous, drunken merrymaking

carp

any of various freshwater fish of the family Cyprinidae

caucus

meet to select a candidate or promote a policy

cavort

play boisterously

circumlocution

an indirect way of expressing something

circumscribe

draw a geometric figure around another figure

circumvent

surround so as to force to give up

clamor

utter or proclaim insistently and noisily

cleave

separate or cut with a tool, such as a sharp instrument

cobbler

a person who makes or repairs shoes

cogent
powerfully persuasive

cognizant

having or showing knowledge or understanding or realization

commensurate

corresponding in size or degree or extent

complement

something added to embellish or make perfect

compunction

a feeling of deep regret, usually for some misdeed

concomitant

following or accompanying as a consequence

conduit

a passage through which water or electric wires can pass

conflagration

a very intense and uncontrolled fire

congruity

the quality of agreeing; being suitable and appropriate

connive

form intrigues (for) in an underhand manner

consign

give over to another for care or safekeeping

constituent

one of the individual parts making up a composite entity

construe

make sense of; assign a meaning to

contusion

an injury in which the skin is not broken

contrite

feeling or expressing pain or sorrow for sins or offenses

contentious

showing an inclination to disagree

contravene

go against, as of rules and laws

convivial

occupied with or fond of the pleasures of good company

corpulence

the property of excessive fatness

covet

wish, long, or crave for

cupidity

extreme greed for material wealth

dearth

an insufficient quantity or number

debacle

a sudden and complete disaster

debauch

a wild gathering involving excessive drinking

debunk

expose while ridiculing

defunct

no longer in force or use; inactive

demagogue

a leader who seeks support by appealing to popular passions

denigrate

attack the good name and reputation of someone

derivative

a compound obtained from another compound

despot
a cruel and oppressive dictator

diaphanous

so thin as to transmit light

didactic

instructive, especially excessively

dirge

a song or hymn of mourning as a memorial to a dead person

disaffected

discontented as toward authority

discomfit

cause to lose one's composure

disparate

fundamentally different or distinct in quality or kind

dispel

to cause to separate and go in different directions

disrepute

the state of being held in low esteem

divisive

causing or characterized by disagreement or disunity

dogmatic

pertaining to a code of beliefs accepted as authoritative

dour

showing a brooding ill humor

duplicity

the act of deceiving or acting in bad faith

duress

compulsory force or threat

eclectic

selecting what seems best of various styles or ideas

edict

a formal or authoritative proclamation

ebullient

joyously unrestrained

egregious

conspicuously and outrageously bad or reprehensible

elegy

a mournful poem; a lament for the dead

elicit

call forth, as an emotion, feeling, or response

embezzlement

the fraudulent appropriation of funds or property

emend

make corrections to

emollient

a substance with a soothing effect when applied to the skin

empirical

derived from experiment and observation rather than theory

emulate

strive to equal or match, especially by imitating

enervate

weaken mentally or morally

enfranchise

grant freedom to, as from slavery or servitude

engender

call forth

ephemeral

anything short-lived, as an insect that lives only for a day

epistolary
written in the form of letters or correspondence

equanimity

steadiness of mind under stress

equivocal

open to two or more interpretations

espouse

choose and follow a theory, idea, policy, etc.

evanescent

tending to vanish like vapor

evince

give expression to

exacerbate

make worse

exhort

spur on or encourage especially by cheers and shouts

execrable

unequivocally detestable

exigent

demanding immediate attention

expedient

appropriate to a purpose

expiate

make amends for

expunge

remove by erasing or crossing out or as if by drawing a line

extraneous

not belonging to that in which it is contained

extol

praise, glorify, or honor

extant

still in existence; not extinct or destroyed or lost

expurgate

edit by omitting or modifying parts considered indelicate

fallacious

containing or based on incorrect reasoning

fatuous

devoid of intelligence

fetter

a shackle for the ankles or feet

flagrant

conspicuously and outrageously bad or reprehensible

foil

hinder or prevent, as an effort, plan, or desire

forbearance

good-natured tolerance of delay or incompetence

fortuitous

lucky; occurring by happy chance

fractious

easily irritated or annoyed

garrulous

full of trivial conversation

gourmand

a person who is devoted to eating and drinking to excess

grandiloquent

lofty in style

gratuitous

unnecessary and unwarranted

hapless
unfortunate and deserving pity

hegemony

the dominance or leadership of one social group over others

heterogenous

consisting of elements that are not of the same kind

iconoclast

someone who attacks cherished ideas or institutions

idiosyncratic

peculiar to the individual

impecunious

not having enough money to pay for necessities

impetuous

characterized by undue haste and lack of thought

impinge

infringe upon

impute

attribute or credit to

inane

devoid of intelligence

inchoate

only partly in existence; imperfectly formed

incontrovertible

impossible to deny or disprove

incumbent

necessary as a duty or responsibility; morally binding

inexorable

not to be placated or appeased or moved by entreaty

inimical

not friendly
injunction

a judicial remedy to prohibit a party from doing something

inoculate

inject or treat with the germ of a disease to render immune

insidious

working or spreading in a hidden and usually injurious way

instigate

provoke or stir up

insurgent

in opposition to a civil authority or government

interlocutor

a person who takes part in a conversation

intimation

a slight suggestion or vague understanding

inure

cause to accept or become hardened to

invective

abusive language used to express blame or censure

intransigent

impervious to pleas, persuasion, requests, or reason

inveterate

habitual

irreverence

a mental attitude showing lack of due respect

knell

the sound of a bell rung slowly to announce a death

laconic

brief and to the point

largesse
liberality in bestowing gifts

legerdemain

an illusory feat

libertarian

an advocate of freedom of thought and speech

licentious

lacking moral discipline

linchpin

a central cohesive source of support and stability

litigant

a party to a lawsuit

maelstrom

a powerful circular current of water

maudlin

effusively or insincerely emotional

maverick

someone who exhibits independence in thought and action

mawkish

effusively or insincerely emotional

maxim

a saying that is widely accepted on its own merits

mendacious

given to lying

modicum

a small or moderate or token amount

morass

a soft wet area of low-lying land that sinks underfoot

mores

the conventions embodying the fundamental values of a group

munificent

very generous

multifarious

having many aspects

nadir

the lowest point of anything

negligent

characterized by undue lack of attention or concern

neophyte

any new participant in some activity

noisome

offensively malodorous

noxious

injurious to physical or mental health

obdurate

stubbornly persistent in wrongdoing

obfuscate

make obscure or unclear

obstreperous

noisily and stubbornly defiant

officious

intrusive in a meddling or offensive manner

onerous

burdensome or difficult to endure

ostensible

appearing as such but not necessarily so

ostracism

the act of excluding someone from society by general consent

palliate
lessen or to try to lessen the seriousness or extent of

panacea

hypothetical remedy for all ills or diseases

paradigm

a standard or typical example

pariah

a person who is rejected from society or home

partisan

a fervent and even militant proponent of something

paucity

an insufficient quantity or number

pejorative

expressing disapproval

pellucid

transparently clear; easily understandable

penchant

a strong liking

penurious

excessively unwilling to spend

pert

characterized by a lightly saucy or impudent quality

pernicious

exceedingly harmful

pertinacious

stubbornly unyielding

phlegmatic

showing little emotion

philanthropic

of or relating to charitable giving

pithy

concise and full of meaning

platitude

a trite or obvious remark

plaudit

enthusiastic approval

plenitude

a full supply

plethora

extreme excess

portent

a sign of something about to happen

potentate

a ruler who is unconstrained by law

preclude

make impossible, especially beforehand

predilection

a predisposition in favor of something

preponderance

exceeding in heaviness; having greater weight

presage

a foreboding about what is about to happen

probity

complete and confirmed integrity

proclivity

a natural inclination

profligate

unrestrained by convention or morality

promulgate
state or announce

proscribe

command against

protean

taking on different forms

prurient

characterized by lust

puerile

displaying or suggesting a lack of maturity

pugnacious

ready and able to resort to force or violence

pulchritude

physical beauty, especially of a woman

punctilious

marked by precise accordance with details

quaint

attractively old-fashioned

quixotic

not sensible about practical matters

quandary

state of uncertainty in a choice between unfavorable options

recalcitrant

stubbornly resistant to authority or control

redoubtable

inspiring fear

relegate

assign to a lower position

remiss

failing in what duty requires

reprieve

postpone the punishment of a convicted criminal

reprobate

a person without moral scruples

rescind

cancel officially

requisition

an authoritative demand

rife

excessively abundant

sanctimonious

excessively or hypocritically pious

sanguine

confidently optimistic and cheerful

scurrilous

expressing offensive reproach

semaphore

an apparatus for visual signaling

serendipity

good luck in making unexpected and fortunate discoveries

sobriety

the state of being unaffected or not intoxicated by alcohol

solicitous

full of anxiety and concern

solipsism

the philosophical theory that the self is all that exists

spurious

plausible but false

staid
characterized by dignity and propriety

stolid

having or revealing little emotion or sensibility

subjugate

make subservient; force to submit or subdue

surfeit

indulge (one's appetite) to satiety

surreptitious

marked by quiet and caution and secrecy

swarthy

naturally having skin of a dark color

tangential

of superficial relevance if any

tome

a large and scholarly book

toady

a person who tries to please someone to gain an advantage

torpid

in a condition of biological rest or suspended animation

travesty

a composition that imitates or misrepresents a style

trenchant

having keenness and forcefulness and penetration in thought

trite

repeated too often; overfamiliar through overuse

truculent

defiantly aggressive

turpitude

a corrupt or depraved or degenerate act or practice

ubiquitous

being present everywhere at once

umbrage

a feeling of anger caused by being offended

upbraid

express criticism towards

utilitarian

having a useful function

veracity

unwillingness to tell lies

vestige

an indication that something has been present

vicissitude

a variation in circumstances or fortune

vilify

spread negative information about

virtuoso

someone who is dazzlingly skilled in any field

vitriolic

harsh, bitter, or malicious in tone

vituperate

spread negative information about

vociferous

conspicuously and offensively loud

wanton

a lewd or lascivious person

winsome

charming in a childlike or naive way

yoke
join with stable gear, as two draft animals

zephyr

a slight wind

wily

marked by skill in deception

tirade

a speech of violent denunciation

Chat Conversation End

Type a message...

Detour : a round about route, deviation

Detract (v) : divert, reduce

Detrain (v) : discharge from train

Detriment : damage, injury

Deuce : plying cards of dice with two spots

Devastate (v) : lay waste, ravage, plunder

Develop (v) : to grow larger, mature, unfold

Deviate (v) : wander from the right course, turn away, take a round about route

Device : an emblem, a scheme, a piece of machinery

Devil (v) : an evil spirit, temper of mankind, Satan

Devious (adj.) : indirect, round about, cooked

Devise (v) : plan, invent, contrive, bequeath

Devoid (adj.) : free from, not having empty, not possessing

Devolve (v) : to dedicate completely, to set aside for

Devote (v) : dedicate, set apart

Devoted (v) : dedicate, loving, zealous, loyal

Devotee : an earnest follower

Sponsored Content
Apply for Canadian Visa. Start Your New Life in Canada.
Apply for Canadian Visa. Start Your New Life in Canada.
[Link]
15 insanely cool Gadgets that are selling like crazy in 2019!
15 insanely cool Gadgets that are selling like crazy in 2019!
[Link]
New WiFi Booster Stops Expensive Internet in Philippines
New WiFi Booster Stops Expensive Internet in Philippines
Next Tech
The Top 10 Reasons Why People Prefer DuckDuckGo Over Google
The Top 10 Reasons Why People Prefer DuckDuckGo Over Google
DuckDuckGo Blog
Say Goodbye to Botox. Grandma's ₱1,755 Trick Erases Wrinkles
Say Goodbye to Botox. Grandma's ₱1,755 Trick Erases Wrinkles
Health News Online
25 Insanely Cool Gadgets That Are Going to Sell out This Year
25 Insanely Cool Gadgets That Are Going to Sell out This Year
Next Tech
Recommended by
Devotion : great love, prayer, worship, piety, earnest
Devotional (adj.) : devotion

Devour (v) : eat greedily, consume to absorb wholly, destroy

Devout (adj.) : holy, given to religious duties, solemn

Dew : tiny drops of moisture, formed from water vapour in the air

Dewlap : fold of loose skin hanging under the throat of cattle, dogs, etc.,

Dexterity : cleverness, ability, expertness

Dexterous (adj.) : adroit, clever, skilful

Dhal : a kind of pulse

Diabetes : a kind of disease

Diabolical (adj) : devilish, impious

Diadem : royal crown, a head band

Diagnose (v) : find out from the symptoms

Diagnosis : ascertaining disease by symptoms

Diagonal (adj.) : joining opposite points

Diagram : a drawing to illustrate something

Dial : a place to show the numbers of the clock or watch

Dialect : language of a community or district

Dialogue : conversation or talk between two or more people

Diameter : line through the center, breath, thickness

Diamond : a bright precious tone, a shape like

Diaper : baby’s napkin

Diarrhoea : looseness of the bowels

Chat Conversation End

Type a message...

Detour : a round about route, deviation

Detract (v) : divert, reduce

Detrain (v) : discharge from train

Detriment : damage, injury

Deuce : plying cards of dice with two spots

Devastate (v) : lay waste, ravage, plunder

Develop (v) : to grow larger, mature, unfold

Deviate (v) : wander from the right course, turn away, take a round about route

Device : an emblem, a scheme, a piece of machinery

Devil (v) : an evil spirit, temper of mankind, Satan

Devious (adj.) : indirect, round about, cooked

Devise (v) : plan, invent, contrive, bequeath

Devoid (adj.) : free from, not having empty, not possessing

Devolve (v) : to dedicate completely, to set aside for

Devote (v) : dedicate, set apart

Devoted (v) : dedicate, loving, zealous, loyal

Devotee : an earnest follower

Sponsored Content
Apply for Canadian Visa. Start Your New Life in Canada.
Apply for Canadian Visa. Start Your New Life in Canada.
[Link]
15 insanely cool Gadgets that are selling like crazy in 2019!
15 insanely cool Gadgets that are selling like crazy in 2019!
[Link]
New WiFi Booster Stops Expensive Internet in Philippines
New WiFi Booster Stops Expensive Internet in Philippines
Next Tech
The Top 10 Reasons Why People Prefer DuckDuckGo Over Google
The Top 10 Reasons Why People Prefer DuckDuckGo Over Google
DuckDuckGo Blog
Say Goodbye to Botox. Grandma's ₱1,755 Trick Erases Wrinkles
Say Goodbye to Botox. Grandma's ₱1,755 Trick Erases Wrinkles
Health News Online
25 Insanely Cool Gadgets That Are Going to Sell out This Year
25 Insanely Cool Gadgets That Are Going to Sell out This Year
Next Tech
Recommended by
Devotion : great love, prayer, worship, piety, earnest

Devotional (adj.) : devotion

Devour (v) : eat greedily, consume to absorb wholly, destroy

Devout (adj.) : holy, given to religious duties, solemn

Dew : tiny drops of moisture, formed from water vapour in the air

Dewlap : fold of loose skin hanging under the throat of cattle, dogs, etc.,

Dexterity : cleverness, ability, expertness

Dexterous (adj.) : adroit, clever, skilful

Dhal : a kind of pulse

Diabetes : a kind of disease

Diabolical (adj) : devilish, impious

Diadem : royal crown, a head band

Diagnose (v) : find out from the symptoms

Diagnosis : ascertaining disease by symptoms

Diagonal (adj.) : joining opposite points

Diagram : a drawing to illustrate something

 Dial : a place to show the numbers of the clock or watch

Dialect : language of a community or district

Dialogue : conversation or talk between two or more people

Diameter : line through the center, breath, thickness

Diamond : a bright precious tone, a shape like

Diaper : baby’s napkin

Diarrhoea : looseness of the bowels

Chat Conversation End
Type a message...


 ourse Programs: University of Michigan; Harvard University; Boston University; Syracuse University; New York University;
University of Massachusetts (Amherst); University of Pittsburgh; Indiana University; University of California at Berkeley; Curry
School, University of Virginia; SUNY Albany; ERIC: Education Resources Information Center; University of Toronto (Canada), and
University of Alberta (Canada).
 Adult Education
 Andragogy
 Lifelong Learning
 Adult Education Programs
 English as a Second Language for Adults
 Industrial & Work-Based Education
 Adult Learning Theory
 Citizenship classes
 Career Transition Education Resources
 GED Programs
 Adult Literacy Programs


 Alternative Education
 Magnet Schools
 Tribal Education
 Alternative College Prep Programs
 Private Schools
 Military Education
 Agricultural Education
 Naturalist Teaching & Learning
 Religious Education in the U.S.
 Vocational Education
 Environmental Education
 Pilot Schools
 Waldorf Schools
 American Heritage Education
 Home Schooling
 Overview of Alternative Education
 Charter Schools
 Outdoor Adventure Education


 Curriculum Organization
 Elective Courses
 Field Trips
 Integrated Curriculum
 Same-Sex Classrooms
 Understanding by Design
 Academic Learning Time
 Parent-Teacher Conferences
 Student Internships
 KIPP Program, The
 School Orientation & Open House
 Anti-Bias Curriculum
 Block Scheduling
 Home Visits
 Open Classrooms
 Project-Based Learning
 Pedagogical Content Knowledge
 Ability Grouping
 Looping
 Core Curriculum, The
 Continuous Progress Approach
 Information Technology Literacy (ITL)
 Health & Nutrition Studies
 Student Exchange Programs
 School Enrichment Activities
 Advanced Placement Programs
 State Curriculum Guidelines
 Self-Contained Classrooms
 Nongraded Instruction


 Early Childhood Education

 Reggio Emilia Approach, The
 Emergent Literacy
 Universal Pre-Kindergarten
 Child Development Programs
 Early Intervention
 Day Care Programs
 Caring for Infants & Young Children
 The Montessori Method
 Childhood Development
 The Head Start Program
 Developmental Risks
 Language Development
 Transition Planning
 Early Childhood/Preschool Education


 Education & the Law

 Academic Freedom
 Title IX
 School Prayer
 No Child Left Behind Act of 2001
 Teacher Misconduct
 Random Drug Testing in School
 Faculty Rights
 Inclusive Education Laws
 Freedom of Religion & Public Education
 Freedom of Expression & Public Education
 School Desegregation
 Fear of Litigation
 Student Rights in the Public Schools
 Evolution vs. Creation
 Sexual Misconduct in the Schools
 Out of School Faculty Behavior
 Parental Rights & Public Education
 Search & Seizure in the Public Schools
 Dispensing Birth Control in Public Schools
 Equal Educational Opportunity
 School Discipline
 Undocumented Immigrant Students


 Educational Psychology
 Intelligence Scales
 Token Economy
 Mathematics Anxiety
 Personality Theories
 Emerging Adulthood
 Moral Development
 Brain-Based Learning
 Socio-emotional Development
 Humanism
 Conation
 Abstract Thinking Skills
 Cognitive Neuroscience
 Psycholinguistics
 Cognitive Development
 Self-Determination Theory
 Transactional Models
 Adolescent Development
 Social Cognition
 Critical Thinking: An Academic Perspective
 Heuristics
 Learning Process, The
 Developmental Psychology
 Motivation


 Educational Theory
 Psychomotor Domain, The
 Cognitive Theories
 Progressive Education
 Attribution Theory
 Theory of Mind
 Sociology of Education
 Behaviorism
 Multiple Intelligences
 Experiential Learning
 Constructivism
 Educational Anthropology
 Affective Domain, The
 Social Justice
 Postmodernism & Education
 Learning Styles
 Problem-based learning
 Taxonomy of Educational Objectives-Cognitive Domain
 Ausubel's Assimilation Theory
 Social Learning Theory
 Structural Learning Theory
 Sternberg's Triarchic Theory
 Programmed Instruction
 Gagne's Conditions of Learning
 Goal Theory
 Cognitive Dissonance Theory
 Script Theory
 Seymour Papert & Constructionism
 Epistemology
 Herbart & Apperception
 Keller's ARCS Model
 Social Change Education
 Instructional Design
 Essentialism & Perennialism
 Transformative Learning
 Social Development Model, The
 Educology


 English as a Second Language

 Structured English Immersion in ESL Instruction
 Audiolingual Method, The
 Error Analysis
 Four Language Skills, The
 Corpus Linguistics
 Bilingual Education
 Communicative Competence
 World Englishes
 Sociolinguistics
 Teaching Heritage Languages
 Second Language Acquisition
 Speech Act
 Interlanguage Pragmatics
 Lexical Approach, The
 Writing Assessment for Second Language Learners
 Teaching English as a Second Language
 Suggestopedia


 Extended Learning
 Affective Variable, The
 Educational Television in the Classroom
 Extracurricular Activities & Programs
 Out of School Time Programs
 Community-Youth Programs
 Full-Day Kindergarten
 Supplemental Education Services
 Extended School Year


 Guidance & Counseling

 College Placement Counseling
 Substance Abuse Counseling
 Academic Interventions
 Counseling Methods
 Counseling Students Who Are Gay, Lesbian, Bisexual, Transgender or Questioning
 Course Scheduling
 School Guidance Centers
 Testing & Evaluation
 Multicultural Counseling Programs
 Peer Counseling Programs
 Vocational Counseling
 School Psychologist, The


 Higher Education
 Admissions policies
 Community College Education
 College & University Student Recruitment
 Virtual Universities
 Rhodes Scholarship, The
 Non-Traditional Minority Students
 Nontraditional Older Students
 Accelerated Degree Programs
 Grants & Private funding
 Institutional Effectiveness in Higher Education
 Access to Higher Education
 E-learning Programs in Higher Education
 Affirmative Action in Higher Education
 Generation Y: Educational Considerations
 Tribal Colleges
 Publish or Perish Paradox, The
 African American Colleges & Universities
 Land-Grant Universities
 Financial Aspects/Budget aspects


 History of Education
 Modern European Influences on American Education
 Chautauqua Movement, The
 G.I. Bill & American Education, The
 Minorities & Education in America
 Women in Education
 History of Teacher Education
 Education of Women in the U.S.
 History of Public Education in the U.S.
 Social History of American Education
 Educational Reform Movements
 Early Roots of Modern American Education, The


 International Perspectives
 Inclusive Education in Developing Countries
 National Education Plans
 Charities & Global Education
 Esperanto
 Study Abroad Programs
 Comparative Education
 International Scholarship Programs
 Globalization & Education
 International Education Indicators
 Teacher Shortages
 UNESCO/Education for All
 Training for International Teachers
 Skills Development Programs
 Gender Disparities in World Education
 Poverty Reduction Strategies
 Teaching in Developing Countries


 Multicultural & Diversity Education

 Non-Formal Education
 Multicultural Education: An Overview
 Intercultural Programs
 Issues of Class in U.S. Education
 Cultural Variable, The
 Culturally Responsive Teaching
 Faculty / Staff Diversity
 Critical Pedagogy
 Achievement Gap, The
 Multicultural Curricula
 Ebonics: African American Vernacular English
 Multicultural Resource Centers
 Cultural Transmission
 Monoculturalism
 Cosmopolitanism
 Cultural Pluralism


 Physical Education
 Hazing in School Sports
 Principles of Conditioning
 Character & Moral Development in Sport & Physical E
 Sport Psychology
 Sportsmanship
 Exercise Science
 Gender in School Athletics
 Interscholastic Coaching
 Health Education
 Assessment in Physical Education
 Motivation & Achievement
 Kinesiology
 Body Image Curriculum
 Competitive Stress
 Performance Enhancing Drugs
 Adaptive Physical Education
 Intramural Sports
 Physical Therapy
 First Aid-CPR Training in Schools
 Athletics & Academics
 Athletic Training
 School Playground Design


 Politics, Government & Education

 English-only Movement
 Teacher Tenure
 De Facto Segregation
 Teacher's Unions -- Past & Present
 Tuition-Free College
 Spellings Report, The
 Meritocracy
 Education Interest Groups
 Cultural Capital & U.S. Education
 Education in National Politics
 Minority Teacher Shortages
 Overcrowding in Schools
 Identity Politics in Education
 U.S. Educational Indicators
 Collective Bargaining & Teachers' Unions
 Busing
 Public Education Finance
 School Choice
 Education & the Economy
 School Vouchers
 Equalization Aid


 Public School Education

 Teaching Health & Sex Education
 Teaching Foreign Languages
 Strategies in Teaching Math
 Tutoring
 High School Education
 Teaching Humanities
 Elementary Education
 Urban Education
 Teaching Social Studies
 Teaching Study Skills
 Teaching Science in U.S. Public Schools
 Public School Education: Middle Grades
 Teaching History
 School Size
 Teaching the Gifted Student
 School Librarianship
 School Nursing
 Rural Education
 Teaching Creative Writing
 Teaching English & Literature
 Teaching Industrial Arts-Technology Education
 Teaching Reading & Writing
 Teaching Life Skills
 Library & Resource Instruction
 Teaching Physical Ed.


 Research in Education
 Experimental (Trial) Research
 Survey Research in Education
 Curriculum Theory
 Action Research
 Empirical Research
 Educational Research Overview
 Statistical Analysis
 Ethnography
 Qualitative & Quantitative Research
 Field Research


 School Administration & Policy

 Professional & Staff Development Opportunities
 School Leadership
 Marketing of Education, The
 School Accreditation
 School Culture & School Reform Efforts
 Parent Involvement in the Schools
 Performance-Based Salaries
 Public School Accountability
 Student Governments
 Recruiting, Hiring & Retaining Quality Teachers
 Education Leadership Policy
 Public Relations in Schools
 School Based Management
 "Green" Schools
 Decision-Making Structures
 Centralized Administration
 School Superintendency
 External Partnering with Public Schools
 Teacher Supervising
 Teacher Burnout
 Staff Performance Evaluation
 School Dropout Issues
 School Boards & Committees
 Instructional Leadership
 Becoming a School Administrator
 Classroom Autonomy
 "Sick" School Buildings


 School Safety
 Zero Tolerance Policies
 Conflict Mediation
 Drug & Alcohol Prevention Programs
 Juvenile Delinquency/Truancy
 Laboratory Safety in the Schools
 School Health Services
 Hazing
 Violence Prevention
 School Security
 Weapons in the Schools
 Bullying
 Crisis Management
 Dress Codes & Uniforms in Public Schools
 Internet Safety
 Social Concerns Among Students


 Service Learning
 Student Leadership
 Community Service Education
 Service Learning for Disabled Students
 Civics Education in the Schools
 Community-School Partnerships
 Parent Volunteers in Schools
 Volunteer Teaching Opportunities
 Values, Character & Moral Education
 Evaluation of Service Learning Programs


 Special Education
 Overidentification
 Service Coordination
 Multidisciplinary Evaluation
 Inclusive Education
 Communication-Based Intervention
 Graphic Organizers
 Instructional Design for SPED
 Homebound Education
 Response to Intervention (RTI)
 Teaching Dyslexic Students
 Sheltered Workshops
 Deficit Model, The
 Remedial Education
 Individualized Education Plan
 Residential Care Education
 Title 1
 Learners who are Exceptional
 Functional Behavioral Assessment
 Teaching Students with Autism
 Mainstreaming in the Public Schools
 Applied Behavioral Analysis
 Behavior Intervention -- Seclusion & Restraint
 Assessment for Exceptional Learners
 Resource Room in Special Education, The
 Family Involvement in Special Education
 Normalization
 Teaching Students with ADHD
 Low-incidence & High-incidence Disabilities
 Universal Design for Learning
 Teaching Students with Physical Disabilities
 Education for the Deaf
 Adaptive Skills & Behavior
 Education for Learners with Visual Impairment
 Movement Education for the Disabled


 Teacher Education
 Teacher Competency Evaluation
 Pre-practicum & Practicum
 Master Teachers
 Continuing Education
 Teacher Certification & Licensure
 Highly Qualified Teachers
 Government Assistance for Teacher Education
 Teacher Mentoring
 Capstone Courses & Experiences
 Teachers' Computer Literacy
 In-Service Education Programs
 Microteaching Labs
 Employment Opportunities in Teaching
 Teacher Peer Assessment
 Student Evaluation of Teachers


 Teaching Methods
 Reader-response methods
 Classroom Environment
 Phonics
 Peer Interaction
 Student Mentoring
 Concept Mapping
 Addressing Plagiarism
 Lesson Planning
 Schema Theory
 Classroom Management
 Reflective Teaching
 Differentiated Instruction
 Multiage Classrooms
 Team Teaching
 Self-Directed Learning
 Cooperative learning
 Socratic Method, The
 Whole Language
 Contract Learning
 Modeling
 Process Writing
 Direct Instruction
 Metacognition


 Technology in Education
 Artificial Intelligence in Schools
 Technology Access Gap
 Integrated Learning Systems
 Computer Assisted Instruction
 Web-based Instruction
 STEM fields & STEM Education
 Educational Software
 Virtual Learning in the Public Schools
 Hypermedia in Education
 Intelligent Tutoring Systems
 Computers in the Classroom
 Anchored Instruction
 Free & Open Source Software in Education
 Assistive Technology in Education


 Testing & Evaluation

 Commercial Test Preparation Industry, The
 Psychometrics
 Aptitude Testing
 Minimum Competencies
 Test Validity
 Test Reliability
 Classroom Evaluation
 Grades & Report Cards
 Computer-Assisted Testing
 Student Peer Evaluation
 Formative & Summative Assessment
 Rubrics
 Teacher Competency Requirements
 Achievement Tests
 Norm-referenced testing
 Authentic Assessment
 Intelligence Testing
 Psychological Testing
 State and National Examinations
 College Entrance Exams
 Classical Test Theory
 Diagnostic Teaching & Testing
 Alternative Assessment
 Grade Point Averages
 Item Response Theory
 Test Readiness
 Performance-Based Assessment
 High School Exit Exams
 High-Stakes Testing
 Portfolios
 Progress Monitoring
 Evaluation of Student Writing
 Test Bias
 Race, Gender & Testing
 Criterion-Referenced Testing
 Homework