On-Line Market Research

Author(s): Thomas W. Miller and Peter R. Dickson

Source: International Journal of Electronic Commerce, Vol. 5, No. 3, Marketing in the E-
Channel (Spring, 2001), pp. 139-167
Published by: Taylor & Francis, Ltd.
Stable URL: [Link]
Accessed: 02-05-2020 06:25 UTC

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact support@[Link].

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
[Link]

Taylor & Francis, Ltd. is collaborating with JSTOR to digitize, preserve and extend access to
International Journal of Electronic Commerce

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
On-line Market Research
Thomas W. Miller and Peter R. Dickson

ABSTRACT: This article focuses upon innovation and issues in on-line market research.
On-line research, on-line consumer behavior, and e-commerce are new areas of aca
demic study in marketing. Most of the early work in these areas was done by practitio
ners, as illustrated by research reports and case studies presented at professional
conferences. The present article reviews technologies and methods of on-line research
and points to various methodological and ethical issues. On-line research is evaluated
from two perspectives: orthodox thinking about the validity of research, and out-of-the
box thinking about how on-line research can increase the impact of market research and
develop the core competitive capabilities of firms. Further academic research and learn
ing from the data of on-line research and electronic commerce are encouraged.

KEY WORDS AND PHRASES: Consumer behavior, consumer research, interactive re

search, Internet research, new product development, on-line focus groups, on-line shop
ping, Web surveys.

On-line research is research conducted over the Internet, including electronic

mail surveys, Web-browser-based surveys and concept tests, on-line inter
views and focus groups. The thesis of this paper is that on-line research is
more than just a new research modality. It represents a cultural and techno
logical change in the way market research is done. What researchers do and
how they think about research are changing because of what they can do on
line. Recalling the words of McLuhan:

In a culture like ours, long accustomed to splitting and dividing all

things as a means of control, it is sometimes a bit of a shock to be
reminded that, in operational and practical fact, the medium is the
message. This is merely to say that the personal and social conse
quences of any medium?that is, of any extension of ourselves?re
sult from the new scale that is introduced into our affairs by each
extension of ourselves, or by any new technology. [64, p. 7]

Traditional market research is time- and paper-intensive, involving sepa

rate phases of survey design, data collection, coding, and analysis. The time
to complete a study?a cross-sectional survey of 1,000 consumers, for ex
ample?is measured in weeks or months. Traditional research methods are
being replaced by on-line methods in many organizations. Compared with
traditional market research methods, on-line research provides a faster and
less expensive method of collecting information. On-line research is of great
benefit to firms desiring faster product development and faster implementa
tion of marketing programs. The time to complete an on-line consumer study
is often measured in hours or days.
In one sense, the new Internet medium poses a historic test for the market
research profession. Market research had not yet been invented in the era
when the telephone was introduced, so the discipline has never before had to
adjust to a new 'Voice of the market" medium. The content and evolution of

International Journal of Electronic Commerce / Spring 2001, Vol. 5, No. 3, pp. 139-167.
Copyright ? 2001 M.E. Sharpe, Inc. All rights reserved.
1086-4415/2001 $9.50 + 0.00.

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
140 THOMAS W. MILLER AND PETER R. DICKSON

the discourse about on-line market research will tell a lot about the profes
sion. Much of the discourse to date has emphasized market research ortho
doxy, with its concerns about sampling representativeness and its focus upon
traditional market research applications.
A popular creativity game or problem presents a solitary player with a set
of dots in a box and asks the player to find some small number of lines that
will connect the dots. Solving the problem requires creativity, drawing lines
outside the box. There is a need for creative, out-of-the-box thinking ("out-of
boxy" instead of orthodoxy) about the effects of on-line media. How will on
line processes and methods change the way decisions are made in companies
and redefine the role and impact of market research?
What cannot be contested is that the new medium is having a seismic im
pact on the market research industry. Changes in the way research is done are
occurring so fast that few in the academic world have been able to keep up
with and study them. This paper presents some observations about current
methods of on-line research and suggestions for improving them in the fu
ture. The understanding of the state of on-line research reflected here is in
formed by the authors' readings in the academic research literature on
consumer behavior, market research, and the information sciences, as well as
by their experience with information systems. Given the currency of on-line
research as a topic and the lack of an academic research literature about on
line research, this review draws upon documentation gathered at recent pro
fessional conferences, such as the EXPLOR Forum.1

Technology and Methods of On-line Research

Enamored of technology, many researchers sometimes ignore the consider
able work to be done in order to begin the process of on-line research. Con
ducting on-line research requires appropriate hardware, software, and
systems expertise. It requires an integrated market research system and a
passing familiarity with new terms and acronyms that describe technology
and systems.
An architecture for on-line research should facilitate transparent end-to
end communication between remote respondents and the researcher. Based
upon internationally accepted networking protocols (TCP/IP and higher-level
services, including telnet for terminal communications and FTP for file trans
fers), the Internet serves as a medium of communication across a diverse ar
ray of computing systems. A common gateway interface (CGI) allows Web
servers to communicate with Web browsers. CGI application programs pro
vide the link between data entered at the respondent's workstation and data
stored in the researcher's database.
An architecture for on-line research also includes utilities for the develop
ment and deployment of Web-browser-based surveys. Software can be devel
oped using a combination of code generators, general-purpose and scripting
languages, and database and Web-development tools. Object-oriented pro
gramming in Java, the preferred general-purpose language for on-line appli
cations, is aided by the existence of extensive class libraries, allowing faster

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 141

development of customized research applications. Market researchers can

begin to learn about the technology of on-line research by studying reviews of
information systems technology for electronic commerce [1, 82].

Electronic Mail Surveys and Distributed Applications

An electronic mail (e-mail) survey involves sending survey questions as part
of the text of an e-mail message or as an attachment to an e-mail message.
Early studies comparing e-mail with traditional methods showed that the re
sponse rates and data quality of e-mail surveys and traditional mail surveys
were similar [66, 77, 92], but there is great variability in e-mail survey effec
tiveness across different types of samples [85]. The cycle time, or length, of a
typical e-mail survey project is from 1 to 10 days, considerably less than for a
traditional mail or telephone survey [12].
Experience shows the importance of obtaining permission before sending
an e-mail survey to a possible respondent. Permission may be obtained dur
ing a customer or panelist registration process (associated with a promotion
or some other Internet activity) or by using traditional methods, such as mail
or telephone. The Council of American Survey Research Organizations
(CASRO) recommends that e-mail surveys include a subject line showing the
topic of the survey. The initial text of the survey should indicate how the
respondent's e-mail address was obtained, who is conducting the survey, how
the research data will be used, and how long the survey will take. As with
telephone interviews, 15 to 20 minutes is thought to be the maximum survey
time, unless appropriate incentives are offered to respondents [12].
E-mail surveys have fallen from favor as a method of data collection over
the Internet. The problem is that long surveys and surveys with skip patterns
are difficult to administer using standard e-mail technology. As Schaefer and
Dillman maintain [85], e-mail is an interim surveying technology, useful to
researchers before the development of Web-browser-based surveys. Simple 5
to 10-question surveys, such as an annual survey of 100 academic depart
ments about the supply and demand for academic marketing positions, can
easily be undertaken by e-mail [28]. Survey prototypes and question formats
can be tested via e-mail before being implemented on a Web browser. E-mail
is also useful in recruiting for on-line Web-browser-based surveys.
Sophisticated stand-alone applications can be sent as e-mail attachments,
providing an on-line analogue to the disk-by-mail survey popularized by
Sawtooth Software for use in conjoint studies [98]. The respondents store the
e-mail attachments as files on their workstation (e.g., on removable disk me
dia), execute the survey applications, and then send their survey response
files back to the researcher as e-mail attachments. Completing all the steps
successfully requires motivated and computer-literate respondents, as well
as an appropriate systems environment on the respondents' workstations.
Adequate respondent incentives and training are required, as when concept
testing new drug protocols and medical appliances with doctors and medical
specialists.
Advances in computer graphics and multimedia technologies make it pos
sible to render realistic images of products (virtual products) and to play video

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
142 THOMAS W. MILLER AND PETER R. DICKSON

clips on respondent workstations. Respondents read or listen to product de

scriptions and view advertisements for products. In some systems, respon
dents have the ability to manipulate or interact with virtual products and to
ask questions of developers and researchers during the tests. Hypertext pre
sentation of product images and information has distinct advantages. Respon
dents might click on parts of an object to see enlarged images or explanations
of product features. Distributed survey applications and multimedia-capable
respondent workstations provide a rich environment for consumer research,
including conjoint studies, concept testing, virtual shopping experiments, and
advertising-copy testing for video and Web pages.

Web-Browser-Based Surveys and Concept Tests

The technological precursors of Web-browser-based surveys include various
computer-assisted technologies. Useful reviews of these methods and the as
sociated research literature may be found in Saris [831 and Couper et al. [221.
In particular, computer-assisted telephone interviewing (CATI), a popular tech
nology, dates back to early work in the 1970s, as documented by Fink [37] and
Palit and Sharp [78]. CATI offers automatic coding of responses and consis
tency checks, leading to higher data quality. Automatic branching, a feature
of most computer-assisted systems, simplifies the development and adminis
tration of tailored surveys. Web-browser-based surveys build upon the capa
bilities of computer-assisted systems, offering distributed, automated research
over the Internet. Many application service providers (ASPs) and software
firms offer Web-browser-based survey development tools and services.
Web-browser-based surveys are by far the most popular method of on-line
research, and most large market research firms have the capability to under
take them. At the most basic level, Web browsers can locate files on a
respondent's workstation or on the World Wide Web using a uniform resource
locator (URL), and can convert hypertext mark-up language (rTTML) files into
images on the respondent's monitor. Current versions of popular Web brows
ers, including Netscape and Microsoft Internet Explorer, can execute Java
based distributed applications. Standard text file formats and Java applications
are transportable, which means that they may be executed across a diverse
array of computing systems.
Java-based applications distributed over the World Wide Web are more
reliable than e-mail attachments. The respondent uses a Java-enabled Web
browser to automatically download and execute the Java application or applet.
An applet is a Java program that is embedded into HTML pages and is usu
ally restricted in its ability to read from or write to fixed disk media on the
respondent workstation. Complex surveys and multimedia presentations may
be distributed in this way.
Concept tests using Web-browser technology hold considerable promise.
As described by Holt, "A concept is a simple written expression of a product's
positioning, benefit, and reason for being, or unique selling proposition.. . .
New product concept tests determine the appeal of a potential new product.
Benefit positioning concept tests identify the one product benefit that should
be emphasized in product advertising" ([50], pp. 505, 508). Traditional con

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 143

cept tests, often conducted with fewer than 200 respondents thought to repre
sent the target population for a product, involve mail surveys or, more often,
personal interviews and focus groups. In-person demonstrations of product
prototypes are commonly used. On-line concept tests distributed over the Web
could satisfy the goals of most concept tests at considerably less expense.
Dahan and Srinivasan executed complex choice-based conjoint studies us
ing a Web-browser-based distributed application [24]. Respondents were pre
sented with one of four types of product-concept renderings of bicycle pumps:
physical pumps (the baseline condition), verbal descriptions, static render
ings, or Web-based animations. Respondents in each condition chose among
nine product concepts. They were given information about five features of a
bicycle pump: (1) how long it would take to inflate a tire, (2) whether it could
fit into a small space, (3) whether it was suitable only for adults or for children
and adults, (4) how durable it was, and (5) price. Dahan and Srinivasan found
that Web-browser-based virtual prototypes, both static and animated, pro
vided market-share estimates similar to those obtained with physical proto
types tested in person [24].
Many studies have demonstrated the utility of Web-browser-based surveys
and distributed applications. Eichman reviewed surveys for tracking IBM's
e-business branding success [29]. Lilie described the advantages of self-ad
ministered Web-browser-based surveys to track the effectiveness of PC Com
puting print ads in driving reader inquiries to the Web [59]. Blackwood
described the extensive surveying of America Online (AOL) customer-sup
port satisfaction [4]. Every 24 hours a sample of 50,000 AOL subscribers re
ceive e-mail invitations to participate in a short survey. Using data from the
approximately 2 million responses over a period of 18 months, AOL was able
to provide detailed monthly evaluations of customer-support performance
by type of service.
Zimring demonstrated the viability of Web-browser-based and distributed
applications for concept testing with children between the ages of 6 and 12
[100]. Working with two research providers, the Larson Group and Education
Market Resources (EMR), researchers at the Quaker Oats Company adminis
tered 15-minute on-line interviews. Each child viewed and ranked five Cap'n
Crunch cereal concepts. What was unique about this study was that it was
undertaken at schools, with parental permission and teachers and aids avail
able to assist the children in the concept testing task. Substantial time and cost
savings were realized over traditional methods of concept testing.
Miller of Cooper Research, Inc., used Web-based concept tests of medical
instruments with nurses and physicians [69]. Two-dimensional and three-di
mensional engineering designs and animations on the Web were compared
with concept-board drawings in person (the traditional method). The cost of a
Web-based concept test involving 200 medical professionals was $48,500, com
pared with $59,500 using traditional methods. Miller noted that costs with
actual medical-product prototypes could be two to three times higher. The
time to complete the Web-based test was 38 days, whereas the traditional
method, executed in a central location and three test markets, took 47 days.
Goglia recently conducted a study at American Power Conversion (APC)
exploring consumer preferences for various features of desktop power-pro

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
244 THOMAS W. MILLER AND PETER R. DICKSON

tection devices, such as surge protectors and battery back-up systems (UPS
devices), for personal computers [41]. The goal of the research was to study
product feature and pricing preferences among aware and unaware consum
ers, and to determine why some people were unaware of APC products and
how awareness could be increased through advertising and promotion. Tar
get respondents were consumers with home computers and Internet access.
Recruiting from the general Internet population, using America Online ban
ner ads, allowed APC to reduce recruitment time and costs. Choice tasks and
data were created and analyzed using software and systems from Sawtooth
Software. Due to the speed at which a large number of respondents could be
recruited over the Internet and the relatively low cost of doing so, models
were estimated with large sample sizes, enabling meaningful comparisons
across consumer subgroups.
In recent years Intel has taken its Intel Inside? program on-line, encourag
ing original equipment manufacturers (OEMs) to place the Intel Inside logo
on Web pages for personal computer products. Marketing managers at Intel
wondered whether they could improve upon this on-line marketing program,
making better use of OEM Web pages by linking information about proces
sors to logos or by adding text next to the Intel Inside logo. In one study, the
processor-selection and system-configuration pages of DelTs on-line store were
modified to permit the administration of experimental conditions designed to
encourage "sell up," the selling of higher-speed Intel processors [65]. Under
heavy sell-up conditions, special Web pages with processor information were
automatically displayed. Under light sell-up conditions, shoppers had to click
on an icon in order to receive special pages of information about processors.
The results showed a 20 percent increase in sales under both heavy and light
sell-up conditions when compared with a base-level (no treatment) condition.
Another study assessed participant beliefs about personal computers that had
been described on OEM Web pages [48]. The stimulus materials included OEM
Web pages with standard Intel Inside logos and text-enhanced logos. The re
sults of the on-line experiment, however, showed no improvement in partici
pant learning from Web pages with text-enhanced logos.

Qualitative On-line Research

Qualitative on-line research includes on-line focus groups, in-depth interviews,
bulletin boards, and list-serve discussions. Both focus groups and in-depth
interviews can be implemented using chat room software or custom software
designed for multiuser real-time interaction. Windowed environments per
mit viewing by selected groups of observers. Popular commercial systems,
such as those offered by Greenfield Online and Itracks, provide participant,
moderator, and client views of a focus group. Clients see what participants
see, as well as their interactions with the moderator. The moderator has par
ticipant and client visibility. In some systems the moderator also has the abil
ity to have private conversations with individual participants. Heckman and
Walkowski review general issues regarding on-line qualitative research [47,96].
Saris reported multitrait-multimethod studies indicating that on-line inter
views yield higher-quality data than telephone or in-person interviews [84].

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 145

There is evidence that on-line interviews may be less susceptible to response

bias (experimenter and social-desirability effects). Kalfs observed respondents
reporting higher daily television watching in on-line surveys than in telephone
surveys [54]. Bon and van Doom found that drug-use reporting was higher in
an on-line setting than in face-to-face interviews [5]. Tourangeau and Smith
noted that self-administered surveys yield higher levels of reporting of sensi
tive behaviors than interviewer-administered surveys. They argued that it is
self-administration, rather than computerization, that explains the differences
[91]. On-line focus groups have also been successful in working with children
8 to 12 years of age [9].
A recent study by the A.C. Nielsen Center for Marketing Research at the
University of Wisconsin-Madison compared on-line, telephone, and face-to
face focus-group modalities [31]. Subjects were junior and senior undergradu
ate students at the University of Wisconsin-Madison. The moderators utilized
versions of a standardized discussion guide differing only in the ways in which
they encouraged interaction among focus group participants. The experiment,
using Greenfield Online focus-group facilities, examined recruitment success
and completion times across modalities, moderators, and majors, as well as
social interaction and reactions to a variety of question types (including pro
jective and sensitive questions). Response measures included counts of the
number of individuals volunteering to participate, and the number of volun
teers actually attending and completing focus-group discussions, as well as
measures derived from textual analysis of focus-group transcripts. The re
sults showed important differences across modalities. Recruitment required
less effort, and participation rates were higher for on-line and face-to-face
groups than for telephone groups. On-line focus groups took longer to com
plete than face-to-face groups, and face-to-face groups took longer than tele
phone groups. When asked about their willingness to participate in future
focus groups, participants indicated a preference for face-to-face and on-line
groups over telephone groups. With each of the three modalities there was
wide variability in the degree of participant interaction. No significant differ
ences were observed across modalities in participant responses to sensitive
questions.
Feld and Lindorf describe one firm's use of live interviewers, called "e
interviewers/' to administer on-line surveys in real-time [36]. The premise
underlying the e-interviewer concept is that the interaction and oversight of a
live interviewer improves sample-frame coverage and data quality by observ
ing and probing for respondent input and retaining respondents through sur
vey completion. To provide evidence of the effects of using e-interviewers to
execute surveys, Feld and Lindorf executed a baseline study as a computer
assisted telephone interviewing (CATI) survey, a self-administered on-line
survey, and an e-interviewer-assisted survey [36]. The three versions included
list rotations, error traps, forced responses, and complicated skip patterns.
They also prevented respondents from reading or answering questions out of
order. The results were consistent with previous findings comparing self-ad
ministered versus interviewer-administered surveys. Although CATI surveys
had a higher cooperation and lower refusal rate than on-line surveys, e-inter
viewer-assisted surveys had better rates than self-administered on-line sur

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
246 THOMAS W. MILLER AND PETER R. DICKSON

veys. Furthermore, an analysis of open-ended responses across the three ver

sions revealed that open-ended responses from on-line surveys yielded more
information than those collected from CATI surveys, and that e-interviewer
assisted responses to open-ended questions yielded more information than
self-administered responses for on-line surveys.
Walkowski has reviewed the methodological options for qualitative on
line research, including chat or focus groups, Web-based bulletin boards, and
e-mail-based list-serve discussion groups [96]. Chat or focus groups offer real
time interaction among participants and are the most popular option for on
line qualitative research. Bulletin boards and list-serve discussions, however,
offer the option of well-crafted questions and well-thought-out answers. Re
spondents are free to participate at times convenient to them. Walkowski pro
vided the following summary advice on the use of qualitative on-line research
[96]: Use it when the target population is comfortable with the Internet, when
there are few concerns about the "leaking" of competitive information, and
when one is willing to forgo seeing and hearing respondents. Qualitative
on-line research, like on-line survey research, is especially useful when the
target population is small, very specialized in its skills, and difficult to find
and recruit, and when the issue studied relates to high-tech products and
services.

Future Developments
Much of what has been seen to date involves the conversion of traditional
research methods to the on-line medium. True innovation in on-line research
may come not from the implementation of on-line surveys and on-line focus
groups, but from new, experimental uses of the medium. How can the Internet
and the power of distributed computing be used to go beyond traditional
methods for learning about consumers?
What more could a researcher want? With today's technology it is possible
to present static or moving-picture advertisements, virtual product displays
that one can "walk around" with a cursor, joystick, or mouse. Digital audio
and video-teleconferencing capabilities can be added. Responses can be pro
vided with a keyboard, joysticks, mice, touch-pads, touch-screens, or digital
tablets. Or one can rely on automated response monitoring that records much
of what people do on-line, including mouse clicks and response choices. While
end-to-end response times are subject to occasional measurement errors asso
ciated with uncontrolled network delays, reliable response time measurements
can be obtained using distributed applications running on respondent work
stations. Add additional devices and software to the mix, and one is able to
record the facial expressions, eye movements, and voices of consumers re
sponding to multimedia stimuli.
What might we expect in the future? Bandwidth will increase with the imple
mentation of fiber optics and wireless channels [55]. Hardware costs will de
cline, leading to extensive, if not ubiquitous, access to on-line distribution
channels in developed countries. Software will be easier to develop with the
availability of sophisticated class libraries for object-oriented programming
in Java. Intelligent consumer devices will be installed in most households.

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 147

Opportunities for meaningful consumer research will increase with techno

logical development in these areas.
Experimental research methods are no longer the restricted domain of the
psychology laboratory. A researcher with an Internet connection can work
with subjects around the globe. Smith and Leigh provide a useful review of
the issues in experimental research on the Internet [88]. Virtual shopping ex
periments could be conducted over the Web. Virtual shopping experiments
involve multimedia presentations of product images in an interactive envi
ronment [10]. Urban, Weinberg, and Hauser [93] and Urban et al. [94] de
scribe an "information acceleration" technology with integrated virtual concept
testing and virtual shopping. With increased bandwidth and computer speeds,
one may expect distributed applications of this type in the future. Increased
bandwidth will also expand possibilities for on-line qualitative research, per
mitting the use of visual as well as textual interaction among focus-group
participants.
Glimp and Tartar described partially automated data reporting imple
mented for Microsoft's Web site, [Link], a site which, at the time of
their study, included 1.4 million Web pages and averaged more than 5 million
unique visitors per day [40]. Microsoft implemented an Online Data Report
ing Tool (ODRT) to provide direct, timely data analysis and management re
porting. The ODRT gave Microsoft employees the ability to view on-line survey
results minutes after a survey launch. The tool processed, cleaned, updated,
and analyzed data immediately after each respondent's submission. Using a
satisfaction-measurement system developed in conjunction with NFO Inter
active, Microsoft reported satisfaction scores for various areas of its Web site.
Areas with low satisfaction scores were marked for improvement. The avail
ability of real-time reporting of customer-satisfaction data has led to manage
ment action, yielding greater satisfaction with Microsoft's Web site over time.
Technologies associated with artificial intelligent agents should be of spe
cial interest to consumer and market researchers. Just as search engines can
look for text strings on the World Wide Web, artificial shopping agents can
look for products, reducing consumer search costs [15]. Artificial broker agents
can do the bidding for consumers at electronic auctions, reducing negotiation
costs [76]. Consumer research can be facilitated by having artificial agents
collect information from respondents about the history of their Internet search
behavior and purchases. The marriage of agent and data-mining technologies
could provide a system for automated or partially automated data analysis.
Intelligent agents were the subject of a special issue of the International Journal
of Electronic Commerce [58]. Many aspects of on-line research will, no doubt, be
touched by agent technologies, whose evolution as an extraordinarily low
cost, extraordinarily broad and fast shopping/search capability is still in a
very early stage.
Developments in on-line research and e-commerce present a challenge for
researchers whose last major adoption to new technology occurred with the
use of computers to undertake data analysis some 30 years ago. It is all the
more challenging because the technology is changing so quickly. Web pages
and on-line stores are modified long before click-streams, log files, and trans
action data are fully analyzed and interpreted. Automated or partially auto

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
148 THOMAS W. MILLER AND PETER R. DICKSON

mated data analysis, possibly implemented with the aid of artificial agents,
may be an answer (perhaps the only answer) to keep in touch with consumer
responses to the new technologies, newly designed Web pages, and new shop
ping and buying processes.

Issues in On-line Research

On-line research presents many technical challenges. Heavy traffic on the Web
affects response times. Lacking sufficient bandwidth, graphical images can be
slow in downloading. There are difficulties associated with the distribution
and execution of software within an uncontrolled systems environment. Not
enough is known about respondent hardware and software configurations to
assume that Web-based surveys and applications are being displayed as one
might like them to be displayed. Web-browser-based surveys often look dif
ferent on different browsers, on screens of different sizes, and at different reso
lutions. Browser software versions differ in their ability to execute Java
applications. Large Web surveys and distributed applications run the risk of
overloading or even crashing respondent workstations. Accordingly, on-line
research suffers from a lack of standardization, raising fundamental issues
regarding the reliability and validity of on-line measures.
On-line research also suffers from research problems associated with self
administered surveys. When running a study over the Internet, one has little
control of respondent viewing behavior. Respondents can resize survey and
application windows, run concurrent applications, shift attention to alterna
tive applications, walk away from their workstations, or ask someone else to
respond to the survey. The fact that these problems also occur with self-ad
ministered mail surveys is little consolation to researchers trying to justify on
line methods. To add to these on-line woes, evidence from Georgia Tech's
Graphic, Visualization & Usability (GVU) Center indicates that more than 40
percent of experienced and frequent Internet users provide false information
on on-line surveys at least 25 percent of the time [43].
Acknowledging the technical difficulty of administering on-line product
concept tests, Miller posed some additional questions for on-line researchers
[69]: Are on-line respondents representative of the target population? How
do we know who the respondents are (respondent identity)? How do we
maintain security for information about new-product concepts when distrib
uting detailed illustrations and documentation over the Web? A recent survey
of marketing research professionals identified sampling issues as a critical
area of weakness for on-line surveys [71]. The issues considered here relate to
on-line survey design (reliability and validity), sampling, recruitment, panel
development and maintenance, and ethical problems.

Reliability and Validity of On-line Measures

The reliability of a scale or measure is the degree to which it yields consistent
measurements. One way to assess reliability is to use test-retest designs: mea
suring respondent attributes twice using the same scale. The correlation be

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 149

tween test and retest scores provides an estimate of reliability. Another way to
assess reliability is to administer parallel forms of a scale. The correlation be
tween scores on the parallel forms provides an estimate of reliability. Reliabil
ity can also be estimated from a single administration of a multi-item scale:
Do respondents with the same scale scores have similar patterns of item re
sponses? If they do, then the scale is reliable or internally consistent.
Measurement is the assignment of numbers to attributes according to rules.
How do we know that we are measuring what we say we are measuring? This
is the fundamental question of validity. To demonstrate content validity, one
must show that the item content of a scale or measure relates to the attribute
being measured. To demonstrate predictive validity, one must show that the
measure of one attribute, an explanatory variable, can be used to predict the
measure of another attribute, a response variable. The meaning of a scale or
measure is defined in terms of its relationship to other scales or measures.
This is the concept of construct validity or measurement validity. The word
"validity" is also used in reference to the internal validity of study designs
and the external validity or generalizability of research findings.
Campbell and Fiske showed how multitrait-multimethod studies could be
used to examine the reliability and validity of measures [11]. Churchill showed
how the concepts of reliability and validity developed by social science re
searchers could be applied in the context of marketing research [16]. These
measurement concepts were more fully explained by Nunnally and Bernstein
[75] and Churchill [17]. Clearly, there is a need for basic research concerning
the reliability and validity of on-line measures. This is an area where aca
demic researchers could make a valuable contribution.
Pending the availability of systematic studies to serve as a guide in the
design of on-line measures, researchers rely upon their best judgment and the
advice of experts in the area of survey design, both traditional and Web-based
[7, 19, 28, 30, 62, 68]. Practitioner studies reported to date have shown the
comparability of on-line and traditional measures at an aggregate level.
Wilke reported on ACNielsen BASES' parallel-testing program, which in
volved 50 parallel tests in 1998, and another 25 in 1999 and 2000 [97]. The
participating research clients included 16 major consumer package goods
manufacturers. In addition to being faster and less costly, on-line concept and
product tests were methodologically sound. High correlations were observed
between mall and on-line product tests. Test-retest reliability estimates for
aggregate data (estimates of purchase intent for groups of consumers) were
high for on-line tests. More extensive open-ended comments were obtained
from on-line panelists than from mall-intercept panelists. Parallel testing
showed that panels of on-line respondents were more willing than mall-inter
cept respondents to express negative feelings about new products. Wilke noted
that valid forecasts can be obtained using traditional or on-line methods as
long as researchers take care to obtain demographically representative
panels and calibrate forecasts against appropriate norms [97]. After its
extensive parallel testing with traditional mall-intercept and on-line inter
viewing methods over the Internet, ACNielsen BASES moved approximately
50 percent of its new product research on-line.
Sch?fer and Wydra reported on a series of 60 parallel tests performed in

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
150 THOMAS W. MILLER AND PETER R. DICKSON

food, health and beauty aid, and household-product categories [86]. The tests
involved on-line and mail surveys asking questions relating to consumers'
likelihood of product purchase. The aggregate data (the proportions of con
sumers indicating purchase intent and ratings of liking, value, and unique
ness) showed high correlations (0.88 to 0.97) between on-line and mail
modalities. But, noting that the immediate responders were those with more
time on their hands (e.g., retired people, and people without children in the
home), Sch?fer and Wydra recommended that on-line surveys remain in the
field for at least four days [86].
When should on-line surveys be conducted? What times of day are most
appropriate? How long should surveys be conducted? Are single-day sur
veys appropriate, or should all surveys be run from four to seven days in
order to cover both the work week and the weekend? How large a sample is
appropriate? How large a panel? These are important questions. But more
important, particularly to orthodox market research practice, are questions
relating to whether on-line samples are representative.

Sampling and Panel Recruitment

"The more things change, the more they stay the same." So it is with sam
pling. Certain sampling problems, discussed in the best marketing research
texts [17, 63], have haunted social science researchers since the 1920s, when
statistical sampling theory and practice were developed.
On-line surveys present special problems. It is difficult to specify a sam
pling frame when there is no centralized listing of Internet users, and when
users are both mobile and global. Nor is there an on-line analogue of tradi
tional random digit dialing. Many on-line samples and panels are subject to
criticism as biased or non-representative. A common practice of on-line mar
keters is to invite Web-site visitors to evaluate Web sites, on-line stores, or
customer service. Researchers hope, and sometimes trust, that survey respon
dents are representative of the population of site visitors or on-line shoppers.
Perhaps they are, perhaps not.
Self-selected panels are also susceptible to sample bias. If self-selection were
merely a matter of income and education, then attempts to adjust for bias
through preliminary quotas for demographic segments or through post-hoc
weighting of segments might prove successful. But it is more difficult, per
haps impossible, to adjust for sample bias when self-selection is a matter of
attitude and experience. Research by Forrester Research and the NPD Group
suggests that Internet adoption and shopping behaviors are affected by con
sumer "technographics," their attitudes toward technology itself [72]. And
response rates, low with self-administered mail surveys, are often lower with
on-line surveys [28].
For some purposes, self-selected panels may be appropriate. Technology
firms and direct marketers selling exclusively on-line, for example, may have
good reasons to work with self-selected panels consisting of current custom
ers and on-line prospects (defined as visitors to their Web sites). Shapiro and
Varian predict that more and more businesses will be using customer-loyalty
programs in the future [87]. Customer databases built from loyalty-program

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 151

registration forms and business transactions have become a rich source of

survey respondents.
Are Internet users a representative sample of the adult population? Not
yet, but the growth in Internet access and usage suggests that in the near fu
ture adult Internet users will be demographically similar to the total adult
population of the United States. A study completed in the fall of 1999 showed
that 49 percent of American adults had access to the Internet either at home or
at work, and that 39 percent were active users of the Internet [73]. Another
study, from July 1999, showed that 33 percent of American adults were active
users [23]. While a few years ago users were more likely to be men than women,
recent usage is almost equally split between men and women [23, 73]. Data
for occupations, household income levels, and U.S. Census geographical re
gions show similar trends: As it grows in size, the population of Internet users
is looking more like the general population of the United States [73].
Correcting for non-representative panels, as much art as science, has been
utilized at firms involved with political polling as well as consumer and mar
ket research. Such corrections are most useful when working with panels of
hundreds of thousands or millions of potential respondents, such as those
created by firms like America Online's Digital Marketing Services [46] and
Harris Interactive [3]. Terhanian suggested case weighting by propensity scores
and using composite measures derived from traditional and on-line survey
methods [90]. Cross-design synthesis, a form of meta-analysis with propen
sity scores, could also be used as a vehicle for better survey design [6]. Still,
opinions differ about the utility of case weighting to adjust for non-represen
tative panels. Sch?fer and Wydra recommended against the weighting of on
line samples [86]. Miller provided a useful summary of the case weighting
issue: "Weighting is one method purported to make on-line 'look7 like the
general population. Even after balancing demographics, many 'intrinsic' dif
ferences remain. Do these remaining differences impact survey results? It prob
ably depends upon the type of study and category under examination. Many
believe the remaining differences impact technology related categories more
than [other] categories" [68, p. 113].
Knowledge Networks (formerly InterSurvey) has taken a different approach
to sampling and recruitment for on-line studies. As reviewed by Rivers and
Fallat [81], Knowledge Networks uses random-digit dialing of all households,
offering contacted households the use of a free Internet appliance, Web TV,
for browsing and survey taking. The demographics for the Knowledge Net
works panel bear a close resemblance to the U.S. population. With free access
to the Web TV appliance and other incentives, response rates in excess of 80
percent have been observed.
How much of the representativeness problem is contrived, over-blown by
orthodox methodologists who are applying a gold standard to Internet sam
pling that they do not apply to telephone or mail survey sampling? When do
firms need a representative sample of the entire adult population of the United
States? The concept of market segmentation is supposedly ingrained in the
thinking of marketers and market researchers. But apparently not when it
comes to evaluating the sampling representativeness of Internet research. A
firm is always targeting its interest, services, products, distribution, commu

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
152 THOMAS W. MILLER AND PETER R. DICKSON

nications, and price promotions to a subset of the entire population. Just as

political pollsters target adults most likely to vote, market researchers target
consumers most likely to buy.
The question of sample representativeness is idiosyncratic to the target
population and its comparative accessibility via the Internet versus mall in
tercept, telephone, and mail. The question of representativeness reduces to
this: Is the sample or panel representative of the segment? Inference to the
general population may be neither necessary nor desirable. Large on-line re
search-provider firms often have specialized or target panels for each major
product category being studied [3,461. Well-recruited panels, Web communi
ties, and Web-site visitors to on-line stores provide meaningful survey groups
for many studies [56]. Furthermore, there is growing evidence that traditional
survey methods, the baselines in comparative studies, are themselves subject
to sampling issues. Bickart and Schmittlein estimate that 20 to 23 percent of
American adults account for all survey responses, and that only 4 to 5 percent
of American adults account for more than half of all survey responses [2].
In many research contexts, on-line surveys may be more efficient and more
effective than traditional telephone and mail surveys. Hard-to-reach respon
dents are often more accessible using the Internet. This is true, for example,
for busy executives or professionals, for people in low-incidence segments of
the general population, and for people who have rare medical conditions or
are unwilling to talk about their conditions or behaviors in face-to-face or
telephone interviews. Geographically remote respondents may be contacted
more easily through the Internet, making on-line methods the preferred method
of global research.

Ethical Issues in On-line Research

Whatever the research modality, market researchers have an obligation to con
duct research in a responsible manner, recruiting with respondent opt-in and
opt-out, protecting respondent confidentiality, being respectful of people's time,
and protecting individual privacy. Privacy is a public concern worldwide.
The European Union has adopted strict privacy standards in its Directive on
the Protection of Individuals with Regard to the Processing of Personal Data and on
the Free Movement of Such Data [33], which has the force of law in member
countries. And, given the number of recommendations that have been adopted
by organizations associated with the European Union [21,33], additional leg
islation should be expected in coming years. Human-subjects research poli
cies and protocols in the United States will, no doubt, require clarification and
revision in order to address the world of on-line research.
Gate observed that 1,000 of the 7,945 bills introduced in the 104th U.S. Con
gress related to privacy issues [13]. In 1998 Congress passed the Children's
Online Privacy Protection Act (COPPA), which went into effect in April 2000.
Many people have concerns about privacy and the rights of on-line research
participants. Several popular books have documented the technological threats
to privacy [38, 60, 89].
Consumer and market researchers argue that products and services be
come successful by responding to the needs of the consumer. Hand in hand

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 153

with privacy issues go better understanding of individual customer needs

and tailored customer services and products. But consumers do not always
buy into the "marketing concept." People are fickle and wary of technology.
They like it when firms use Internet information to save them money or pro
vide special new products and services (such as Travelocity's prompting of
price promotions on frequently flown routes) but consider other information
uses an invasion of privacy. Consumers who voluntarily provide information
in exchange for free e-mail access or pay-for-performance also have concerns
about privacy [14].
Cookies are another concern of Internet users. Just as the storybook charac
ters Hansel and Gretel used cookie crumbs to mark their trail through the
woods, digital cookies provide markings on computers. Cookies are data stored
on the respondent's workstation that may be used to identify the respondent
and to keep information about his or her past acts. Although the technically
informed can easily erase their cookie directories, many respondents do not
even know of their existence. Some people regard cookies as an unnecessary
and overly intrusive technology. But Web-based survey researchers see cook
ies as a way to protect respondents from receiving multiple invitations to par
ticipate in surveys. Moreover, if a survey application is interrupted, information
about the application in a cookie file would allow the respondent to continue
with the survey at the exact place where the process broke off. Cookie files can
also identify respondent preferences for a certain type of survey or for the
look and feel of an application.
To accomplish many of the same things that cookies accomplish, some re
searchers turn to data-augmented URL strings. As respondents move from
one survey Web page to the next, clicking on hyperlinks, they send URLs to
the researcher's Web server. The URLs contain coded information relating not
only to the location of the next Web page, but also to the location of the
respondent's data. The researcher can maintain a unique directory or file of
data for each respondent, including passwords, personal information, and
survey status information. The advantage of using data-augmented URLs in
stead of cookies is that the data are maintained on the researcher's server, not
on the respondent's workstation.
A click-stream is a record of mouse clicks. A click-stream shows what a
respondent does when viewing a Web site. It is the on-line analogue of behav
ioral observation. Analysis of click-streams is in its infancy, but is viewed as
having special relevance to advertising research and Web-site design. Some
people view click-stream research as overly intrusive and a possible invasion
of privacy.
The principle of informed consent should be the researcher's guide in mak
ing decisions about using cookies, data-augmented URL strings, or click
streams. Is the respondent fully informed of the consequences of survey taking?
If a respondent agrees to take a survey, does this give the researcher permis
sion to use cookies, data-augmented URLs, or click-stream recording? It is
the researcher's duty to see that the respondent has given fully informed
consent.
MacElroy discussed some questionable techniques used by researchers to
recruit respondents to on-line panels [61]. One such technique is the surrepti

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
254 THOMAS W. MILLER AND PETER R. DICKSON

tious use of respondent e-mail addresses without notifying the individuals

that they are being "recruited" for research purposes. Artificial agents
(spambots, spiders, and sniffers) are often used for such collections. Mislead
ing or off-topic newsgroup postings are sometimes used to recruit people for
panels. Purchased e-mail lists often include people who have not opted in to
be contacted for surveys. The use of such lists for recruitment constitutes
"spamming."
Many observers view spamming, the sending of unsolicited e-mail, as in
appropriate. A California law that took effect in January 1999 makes it illegal
to send unsolicited e-mail [45], Other states may follow in California's foot
steps. There is by no means a unanimous opinion on the question of spamming.
Consider Hall's challenge to others to "spam me": "It is fashionable these
days to decry spammers in the strongest of terms. After all, they sully the
Internet with their tawdry come-ons. They clutter our mail queues and waste
our time. They clog corporate networks. But why all the fuss? Doesn't anyone
but me have a delete key?" [44, p. 281.
There is evidence of emerging self-regulation among industry representa
tives. Organizations issue privacy statements. On-line analogues of the Better
Business Bureau, such as the Information Technology Industry Council (ITIC)
and TRUSTe, issue privacy seals of approval [89]. Professional organizations
are addressing ethical issues and best practices in on-line research. Established
organizations include the American Association for Public Opinion Research
(AAPOR), the American Marketing Association (AMA), the Advertising Re
search Foundation (ARF), the Council of American Survey Research Organi
zations (CASRO), and the European Society for Opinion and Marketing
Research (ESOMAR). And new organizations like the Association of Internet
Researchers (AIR) and the Interactive Marketing Research Organization
(IMRO), have been formed, in part, to address special concerns with regard to
on-line research methods.
Based in the United States, CASRO is active in educating marketing re
searchers about survey methods and appropriate standards of practice for
on-line research [12]. CASRO's European counterpart, ESOMAR, has issued
guidelines for the conduct of Internet research. Included in ESOMAR's guide
lines are the following recommendations: cooperation must be voluntary, the
researcher's identity must be disclosed, respondents' rights to anonymity must
be safeguarded, respondent data must be protected and given secure trans
mission, the public must not be misled about the reliability and validity of
Internet research findings, the permission of a responsible adult must be ob
tained before interviewing minors, and unsolicited e-mail must be kept to a
minimum [32].
Finally, it is worth noting that the ethics of Internet research must be placed
in the larger context of ethical priorities for the market research profession.
The industry is confronted with a far more important and long-standing ethi
cal problem precipitated by the practice of salespeople posing as market re
searchers undertaking telephone surveys of customers. Not only is it fraudulent
to sell under the guise of research, but the collective result of this practice has
been to severely hurt the credibility and integrity of market research, result
ing in plummeting telephone-survey response rates.

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 155

From Orthodoxy to "Out-of-boxy"

The intensive discussion of the validity, integrity, and ethics of on-line market
research raises the question of whether market research orthodoxy is over
whelming other, more creative ways of evaluating on-line market research.
Even with its phenomenal growth, are market researchers and the profession
too risk-averse in evaluating on-line market research? Is market research, as a
discipline, playing the role it should in helping businesses understand how to
use the Internet to do market research? Or is the discipline too immersed in
orthodoxy? Thinking beyond the industry's conventional concerns about
market research processes, some constructive speculation about how on-line
market research can affect marketing decision-making and resource deploy
ment is in order.
Supply-chain logistics has seen a great deal of innovation in the last 20
years, and market researchers could learn a lot from these developments. High
speed information about the order-delivery cycle has greatly reduced supply
chain system uncertainty, eliminated expensive safety stocks, and identified
places, nodes, and activities in the supply chain that were notoriously ineffi
cient. The direct implication of the use of electronic data interchange and sup
ply-chain information sharing is that a great deal of information is generated
about the what, when, and where of consumer demand. Information is gener
ated about which distributors and distribution systems are hot, and what and
who are not. Yet there is a noticeable absence of discussion in the marketing
research literature as to how some of the vital information in supply chains
can be used by market researchers and combined with other market research
information to inform and direct management decision-making. As a result,
sales forecasting is becoming the responsibility of logistics rather than market
research.
Market researchers need to learn about on-line research and how they can
use the Internet to capture vital market information, at the same time revolu
tionizing many business-to-business and business-to-consumer processes.
Since market research is responsible for researching all aspects of the market
(not just consumer attitudes and behavior), it must strive to incorporate on
line supply-chain information in its analyses, including analyses of trading
partner profitability and competitiveness.
The less direct but perhaps more important message from the logistics revo
lution is how information can transform business processes. Can lessons from
logistics be applied to other managerial processes, such as product develop
ment, advertising campaigns, and marketing program improvisation? Can
high-speed, quality information improve managerial decision-making, reduce
uncertainty, and speed up the delivery of marketing programs?
What is the common problem with product and service innovation? Firms
are not in touch with their markets?they don't hear the voice of the cus
tomer. Personal and organizational politics get in the way, slowing decision
making and implementation [20]. How can creative, out-of-the-box thinking
about on-line market research, or what is here called "out-of-boxy," be used to
address the problem of product and service innovation?
Suppose firms were to create on-line panels of salespeople, distributors,

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
256 THOMAS W. MILLER AND PETER R. DICKSON

and customers from various target segments. Suppose these panels were con
sulted on a regular basis throughout the product-development process, giv
ing ample time for field testing of the product. Using willing and properly
motivated panelists, product development could proceed much faster with
on-line processes than with traditional processes. A prototype design and field
testing loop might become a process lasting two or three days rather than two
or three weeks [26, p. 398]. For example, Procter & Gamble has been able to
reformulate products and concept-test them with a panel of customers using
a form of conjoint analysis. The product-development process has been tak
ing 48 hours with on-line methods. This approach to field testing of concepts
should be extended to salespeople and distributors where appropriate.
The sort of market research described here is very different from traditional
customized market research with its focus upon special one-time projects. Not
uncommonly, traditional customized research takes weeks to launch in the
field, is weighed down with everybody's pet issue or question, and results in
lengthy research reports that are ignored by management. The reports pro
duced by traditional customized research often bury critical answers in nice
to-know report chaff. Information often arrives too late to influence decisions,
because organizational beliefs, attitudes, and political positions have become
set in the interim. The monolithic research study should be replaced by sev
eral, perhaps many, small studies. On-line research is well suited to the devel
opment of iterative studies. Organizational beliefs and attitudes may be
updated on a regular basis. Iterative on-line research would provide a smooth
learning path, with information organized into small chunks or steps that are
within the capacity of management teams to understand.
Some researchers may question the generalizabililty of iterative on-line stud
ies, especially those conducted over a day or two with small samples.
Generalizability may be a problem when only 55 percent of respondents say
yes to a product concept. But when 80 percent or more say yes, longer or
larger samples would seem unnecessary. The objective of new-product re
search should be to lead us to a winning idea. Rather than invest in larger
samples to obtain statistical significance for a 55 percent idea, management
should invest in new studies to find a better 80 percent idea, a product con
cept with a larger, very positive consumer response. The market research in
dustry has much to learn from the consulting industry about convenience
samples and effect sizes. It should note, too, that replication and meta-analy
sis across similar studies can address research reliability and validity issues.
Perhaps the greatest contribution of continuous-development test looping
(comparable to continuous replenishment in supply chains) is how it can
change other organizational processes. Iterative on-line research offers many
organizational benefits, such as the following:

Greater direct involvement of research and development designers

and engineers whose ideas and questions can be tested overnight.
Direct involvement encourages continuous thinking about and
reaction to the voice of the customer, responding to innovative
development ideas as well as "why don't you?" suggestions from
the field. Many R&D engineers work on several projects at once. In

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 157

the traditional process, with market research results coming after

several weeks in the field, it takes an engineer at least a day to get
up to speed on a project. There is inevitable forgetting about issues
and problems. The same concern applies to groups, because several
more weeks may be needed just to get everyone working together
again in the same room. With iterative on-line research, there will
be less competition for management attention from other projects,
as well as more creative thinking and cumulative learning. Fast,
iterative market research exploits the capabilities of company
research and development by keeping people on track and
up-to-speed.

Reduced level of political processes and activities. Anyone who has

worked in the real world knows about the ubiquitous influence of
personal and functional politics in business decision-making and
implementation. This is not written about in the academic literature
because its surreptitious and devious nature make it almost impos
sible to study. But it is there and is the frequent enemy of product
development successes. Traditional market research can be the
friend of organizational politics. When senior managerial oppo
nents of projects insist on further market research, they buy time to
find other reasons for canceling or slowing the momentum of
projects. But when research results flow quickly and continuously,
market research is less likely to be used as a delaying tactic.

Faster project progress builds team enthusiasm, increasing cross

learning and cooperation among team members. Faster field testing
and market research force other processes to speed up. There is
faster prototyping, costing, and manufacturing design. Speeding up
these processes keeps participants more focused, creative, and
capable, and brings out the best in concurrent engineering pro
cesses. There is more interaction, intensity, commitment, and
"buzz." E-mail between team members becomes filled with ques
tions, suggestions, compromises, and decisions about projects.
Iterative on-line research has the potential to change the entire
product-development process, resulting in higher quality and
higher productivity.

Market research organizations need to attend to their internal customers

just as much as to the details of research design. They need to think about the
impact of research just as much as the quality of research. For years research
ers have talked about the need to provide timely market research. Now on
line technologies offer huge improvements in research timeliness and potential
impact.
Not yet fully explored are the possibilities for developing organizational
consensus on-line. The principle of triangulation in qualitative research sug
gests that multiple sources of information should be found to support research
conclusions. Consider the possibilities for triangulation using a Delphi-like

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
158 THOMAS W. MILLER AND PETER R. DICKSON

sharing of beliefs from parallel and sequential surveying of salespeople and

distributors.2 In high-technology markets where early, fast insights are gener
ated by sales and marketing interacting with engineering and design, on-line
communication promotes creativity. The speed of on-line research can create
first-mover advantages, providing substantial economic returns and eco
nomic rents.
Turning to the issue of validity, there are admittedly some problems associ
ated with on-line research. Because on-line samples can be less representa
tive, questionnaires are self-administered or contain fewer items, and survey
processes are less standardized, on-line surveys may be less valid than tradi
tional surveys. But the validity of on-line surveys improves when repeated
measures are used to track customer preferences over time or when the focus
is upon target segments. Iterative on-line surveys may be less susceptible to
one-time framing biases, and in addition offer the advantages of iterative model
fitting and testing. On-line surveys may be highly segment-focused (facili
tated by frequent communication with on-line panels).
What affects the utility of market research? Three factors can be identified:
(1) the importance of the project, measured in terms of maximum net-present
value contribution to profits, (2) the probability of impact, and (3) validity, an
indication of generalizability (measurement validity, internal and external
validity). To provide a rough formula, think of the utility of a market research
project as the product of these three factors:

Research Utility = (Maximum Value) x (Probability of Impact) x (Validity)

Consider a situation in which management is evaluating a new-product

development project or a decision to choose or switch distribution alliances in
a market. Suppose the project has the potential of making a $10 million net
present-value contribution to profits if the impact on the decision-making is
at the maximum of 1.0 and if the validity of the research is at the maximum of
1.0. The probability of traditional market research having an effect on deci
sion-making might be 0.5 with a validity of 0.8. Investing in iterative on-line
research, because of the advantages described above (involvement of engi
neers, timely information, less forgetting, faster learning, less politics, more
coordination, the speeding up of all processes that force further efficiencies),
might increase the probability of impact to 0.8. Assuming a drop in validity
from 0.8 to 0.7 associated with on-line research, the value of the approach is
$5.6 million ($10 million x 0.8 x 0.7), compared with $4 million for the tradi
tional approach ($10 million x 0.5 x 0.8). Lower validity can be tolerated (if
indeed on-line research has lower validity) when the probability of impact is
high.

Conclusion

Just as many businesses are being transformed by the e-commerce or e-busi

ness revolution, as noted by Evans and Wurster [351, market research is being
transformed by on-line research. Research clients and providers alike have

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 159

observed dramatic time and cost savings associated with moving from tradi
tional modes of research to on-line research. Automation of information gath
ering and research functions fosters change in the information supply chain
[70]. Many predict that custom research provider firms must adopt on-line
methods in order to survive [42, 51].
Peterson and Hopkins noted that in 1998 General Mills conducted less than
1 percent of its consumer research on-line, and in 1999 about 20 percent [80].
But in 2000 General Mills planned to conduct 60 percent of its consumer re
search on-line. Client firms can acquire software tools to become research pro
viders themselves. On-line research was a top news item in the Minneapolis
Star Tribune:

Taking its first major step onto the information highway, General Mills
announced Wednesday that it has bought a stake in a small, privately
held California software company as part of a venture to use the
Internet for market research. Apart from developing a joint venture,
to be called InsightTools, Golden Valley-based General Mills and
Sausalito-based MarketTools, Inc. will sell software and technical sup
port to other companies eager to do customer research via the Internet.
[74, p. D3]

On-line research is a fast-moving area. Yesterday's orthodox questions were:

Should we do on-line research? Is it valid? How does it compare with tradi
tional methods of research? Today's out-of-box questions are: Who should we
partner with to change our market research processes? Which new technol
ogy should we try next? How can we use our new market research processes
to drive change and improvements in our core capabilities of research and
development and implementation? The field has moved from questions of
legitimacy to questions of core competitive capabilities.
Comparative studies are valuable, because it is important to learn how
measures differ across modalities. Multitrait-multimethod studies ? la
Campbell and Fiske [11] help to demonstrate both the reliability and the va
lidity of on-line measures. But market researchers need to do more than estab
lish the reliability and validity of on-line methods. It is necessary to innovate,
to explore the potential of new media and methods.
It is time to move beyond conventional modes of consumer and market
research. Time to use computer power and networks to change research and
management processes, to promote creativity and managerial innovation.
Market researchers can take a lesson from logistics and supply-chain man
agement. High-speed information exchanges, both within firms and between
firms, improve processes and lower costs across the supply chain. On-line
research, automated data analysis, and the speed at which research results
can be communicated over intranets and the Internet offer similarly benefi
cial results. Instead of slowing down the process of innovation in on-line con
sumer and market research by asking questions about its reliability and validity,
perhaps the time has come for speeding it up.
Much has been written about Web-page and Web-site design from the points
of view of artists and engineers. Much has been written about human factors
and usability testing. What is needed now is consumer and market informa

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
160 THOMAS W. MILLER AND PETER R. DICKSON

tion about Web pages and sites. How does the design of Web pages and sites
affect consumer attitudes and behavior? How does design affect product per
ceptions? Does design affect sales response? There are many opportunities for
meaningful Web-page and Web-site research. Mena has reviewed software
tools for the analysis of click-streams and Web-site logs [67]. Young has illus
trated the uses of Web measurements derived from a variety of methods, ex
tending from focus groups to eye-tracking devices [99].
If it is true, as some argue, that the future of shopping is on-line [39], then
academic researchers need to address basic questions about on-line shopping
behavior. How can firms and investors predict the long-term impact of Internet
searching and shopping when such behaviors are still in their early stages of
evolution? While most business analysts predict substantial growth in e-com
merce activities, it is difficult to forecast exact rates of growth or the effects of
such growth upon business operations and financial performance. One ap
proach to improving upon the bases of most current forecasts, including the
exuberant forecasts of some business and stock market analysts, is to examine
consumer on-line behavior. How do Internet searchers and shoppers spend
their time on-line and off-line? What is the relationship between time spent
and dollars spent? How does consumer long-term experience with comput
ers and the Internet affect searching and shopping behavior?
More studies of on-line searching and shopping behavior are needed, as
well as studies of consumer attitudes about on-line searching and shopping
[52]. Firms, especially firms engaged in on-line research and e-commerce, are
collecting more data than ever before. Researchers also need to explore new
methods and models for learning from the data of on-line research and e
commerce. On-line researchers, working in cooperation with modelers, statis
ticians, database managers, and computer and information scientists, can learn
to employ appropriate and efficient methods for analyzing data. Working to
gether, practitioners and academics can answer fundamental questions about
the design and implementation of on-line research.
When on-line, connected to a network, whether using a Web browser, elec
tronic mail, or a Java-based application, we are participants in an open, po
tentially interactive, communication medium. To the extent that it captures
people's attention, this medium is more efficient than mass media broadcast,
telephone, or mail. On-line communication is fast and has low variable cost.
Messages sent are easily modified, offering the benefits of adaptive or tai
lored messaging. Messages received come from address-identifiable sources.
Data are easily collected, stored, and analyzed.
As McLuhan first observed, "the medium is the message" [64]. Following
his lead, what one sees with an on-line medium is the integration of research
with product testing, advertising, promotion, product display, pricing (includ
ing dynamic pricing in electronic auctions), sales order taking, processing,
and fulfillment. In the future, further integration of market research with
marketing and sales activities can be expected, because an on-line medium
allows it. Much of market research is moving on-line. Progress in on-line re
search is just a matter of bandwidth and technical expertise and creativity.
Bandwidth, with wire-bound and wireless connections, is growing fast. One
hopes that the expertise and creativity will follow.

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 161

NOTES

1. The EXPLOR Awards Program at the A.C. Nielsen Center for Marketing
Research recognizes "exemplary performance and leadership in on-line research."
The annual EXPLOR Forums, jointly presented by the A.C. Nielsen Center for
Marketing Research and the American Marketing Association, focus upon the best
practices in on-line research, superlative research applications, and case studies.
2. A discussion of integrating competitive intelligence with consumer research
is beyond the scope of this paper, but surely an on-line orientation to market
research, including on-line access to inexpensive competitive intelligence services
and feedback from distributors and salespeople, will naturally facilitate such
integration. This offers further opportunities for triangulation.

REFERENCES

1. Amor, D. The E-business (R)evolution: Living and Working in an Intercon

nected World. Upper Saddle River, NJ: Prentice Hall, 2000.
2. Bickart, B., and Schmittlein, D. The distribution of survey contact and
participation in the United States: Constructing a survey-based estimate.
Journal of Marketing Research, 26 (May 1999), 286-294.
3. Black, G. Harris Interactive: A new era in marketing and advertising
research, and polling. Paper presented at the 1999 EXPLOR Forum, Madi
son, WI, October 20,1999.
4. Blackwood, M. Taking action on satisfaction measurement. Paper
presented at the 1999 EXPLOR Forum, Madison, WI, October 21,1999.
5. Bon, E., and van Doom, L. Some results and experiences of a new
consumer panel technique: Tele-interview. In ESOMAR Seminar on Improv
ing the Use of Consumer Panels for Marketing Decisions. Amsterdam:
ESOMAR, 1988, pp. 85-101.
6. Boruch, R. F., and Terhanian, G. Cross-design synthesis. In A.J. Reynolds
and H.J. Walberg (eds.), Advances in Educational Productivity, vol. 7. Green
wich, CT: JAI Press, 1998, pp. 59-85.
7. Bradford, D. Conducting on-line focus groups. Tutorial presented at the
2000 EXPLOR Forum, Chicago, November 16, 2000.
8. Bradshaw, J.M. (ed.). Software Agents. Menlo Park, CA: AAAI Press/MIT
Press, 1997.
9. Bulkeley, W.M. Nickelodeon sets up on-line focus group. Wall Street
Journal, March 29,1993, B4.
10. Burke, R.R. Virtual shopping. ORMS Today, 22, 4 (1995), 28-34.
11. Campbell, D.T., and Fiske, D.W. Convergent and discriminant validation
by the multitrait-multimethod matrix. Psychological Bulletin, 56, 2 (1959), 81
105.
12. CASRO. Frequently Asked Questions About Conducting On-line Research:
New Methodologies for Traditional Techniques. Port Jefferson, NY: Council of
American Survey Research Organizations, 2000. Retrieved from
[Link]/faq/[Link], January 30, 2000.
13. Cate, F.H. Privacy in the Information Age. Washington, DC: Brookings
Institution, 1997.
14. Chang, A-M.; Kannan, P.K.; and Whinston, A.B. The economics of

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
 162 THOMAS W. MILLER AND PETER R. DICKSON

freebies in exchange for consumer information on the Internet. International

Journal of Electronic Commerce, 4,1 (fall 1999), 85-102.
15. Chun, I.; Lee, J.; and Lee, E. I-SEE: An intelligent search agent for
electronic commerce. International Journal of Electronic Commerce, 4, 2 (winter
1999-2000), 83-98.
16. Churchill, G.A., Jr. A paradigm for developing better measures of
marketing constructs. Journal of Marketing Research, 16 (February 1979),
64-73.
17. Churchill, G.A., Jr. Marketing Research: Methodological Foundations, 7th ed.
Fort Worth, TX: Dryden, 1999.
18. Clayton, R.L., and Werking, G.S. Business surveys of the future: The
World Wide Web as a data collection methodology. In M.R Couper, R.R
Baker, J. Bethlehem, C.Z.F. Clark, J. Martin, W.L. Nichols II, and J.M.
O'Reilly (eds.), Computer-Assisted Survey Information Collection. New York:
Wiley, 1998, pp. 543-562.
19. Cook, S. Online marketing research. Tutorial presented at the IQPC
conference on Online Market Research & Web-Based Surveys, San Francisco
(January 18, 2000).
20. Cooper, R.G. From experience: The invisible success factors in product
innovation. Journal of Product Innovation Management, 16 (1999), 115-133.
21. Council of Europe. Guidelines for the Protection of Individuals with
Regard to the Collection and Processing of Personal Data on Information
Highways. (Recommendation No. R (99) 5 of the Committee of Ministers to
Member States for the Protection of Privacy on the Internet), (February 23,
1999). Retrieved from [Link]/cm/ta/rec/1999/[Link], January 30,
2000.
22. Couper, M.P; Baker, R.P.; Bethlehem, J.; Clark, C.Z.F; Martin, J.; Nichols
II, W.L.; and O'Reilly, J.M. (eds.). Computer-Assisted Survey Information
Collection. New York: Wiley, 1998.
23. Cyber Dialogue. American Internet User Survey. New York: Cyber Dialo
gue, July 1999. Retrieved from [Link]/free_data/
[Link] (January 30, 2000).
24. Dahan, E., and Srinivasan, V. The predictive power of Internet-based
product concept testing using visual depiction and animation. Report no.
1502. Palo Alto, CA: Stanford Graduate School of Business, 1998.
25. D'Aveni, R.A. Hyper-Competition: Managing the Dynamics of Strategic
Maneuvering. New York: Free Press, 1994.
26. Dickson, PR. Marketing Management, 2d ed. Fort Worth, TX: Dryden,
1997.
27. Dickson, PR. 2000 Labor Market Survey for Academic Marketing
Positions. A.C. Nielsen Center for Marketing Research, College of Business,
University of Wisconsin-Madison, 2000.
28. Dillman, D.A. Mail and Internet Surveys: The Tailored Design Method. New
York: Wiley, 2000.
29. Eichman, C. The complexities of website research: IBM's online learn
ings. Paper presented at the 1999 EXPLOR Forum, Madison, WI, October
21,1999.
30. Eigenbauer, D.F. Building online surveys that produce meaningful

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 163

research. Tutorial presented at the IQPC conference on On-line Market

Research & Web-Based Surveys, San Francisco, January 18, 2000.
31. Esipova, N.I.; Miller, T.W.; Zarnecki, M.D.; Elzaurdia, J.; and Ponnaiya,
S. Exploring the possibilities of online focus groups. Paper presented at the
American Association for Public Opinion Research Conference, Portland,
OR, May 20, 2000.
32. ESOMAR. Conducting Marketing and Opinion Research Using the Internet.
Amsterdam: European Society of Opinion and Marketing Research, 2000.
Retrieved from [Link]/guidelines/internet_guidelines.htm
(February 1, 2000).
33. European Communities. Directive on the Protection of Individuals with
Regard to the Processing of Personal Data and on the Free Movement of
Such Data (October 24,1995). Retrieved from [Link]/eur-lex/en/lif/
dat/1995/en_395L0046.html (January 30, 2000).
34. European Communities. Recommendation on 1/99 on Invisible and
Automatic Processing of Personal Data on the Internet Performed by
Software and Hardware. (The Working Party on the Protection of Individu
als with Regard to the Processing of Personal Data) (February 23,1999).
Retrieved from [Link]/comm/dgl5/en/media/dataprot/wpdocs/
[Link] (January 30, 2000).
35. Evans, P., and Wurster, T.S. Blown to Bits: How the New Economics of
Information Transforms Strategy. Boston: Harvard Business School Press, 2000.
36. Feld, K., and Lindorf, R. Using E-interviewers in Conjunction with Online
Surveys at humanvoice . Paper presented at the 2000 EXPLOR Forum,
Chicago, November 17,2000.
37. Fink, J.C. CATI's first decade: The Chilton experience. Sociological
Methods and Research, 12 (1983), 153-168.
38. Garfinkel, S. Database Nation. Sebastopol, CA: O'Reilly, 2000.
39. Gerbert, P.; Schneider, D.; and Birch, A. The Age ofE-tail: Conquering the
New World of Electronic Shopping. Milford, CT: Capstone, 1999.
40. Glimp, D., and Tartar, J. Microsoft's online data reporting tool. Paper
presented at the 2000 EXPLOR Forum, Chicago, November 17,2000.
41. Goglia, C. Online studies of product awareness and product choice: The
American power conversion experience. Paper presented at the 2000
EXPLOR Forum, Chicago, November 16, 2000.
42. Gonier, D.E. The research emperor gets new clothes. CASRO Journal
1999, 109-114.
43. GVU's Tenth WWW User Survey, 1998. Retrieved from
[Link]/user_surveys/survey-1998-10 (October 1, 2000).
44. Hall, M. Spam me. Computerworld (August 14,2000), 28.
45. Hanson, W. Principles of Internet Marketing. Cincinnati: South-Western
College Publishing, 2000.
46. Harmon, G., and Miller, C. Online research: State of the industry. Paper
presented at the 1999 EXPLOR Forum, Madison, WI, October 20,1999.
47. Heckman, J. Turning the focus online. Marketing News (February 28,
2000), 15.
48. Helmrich, L. Assessment of Intel Inside? logos and text-enhanced logos.
Paper presented at the 2000 EXPLOR Forum, Chicago, November 17, 2000.

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
 264 THOMAS W. MILLER AND PETER R. DICKSON

49. Hinkkanen, A.; Kalakota, R.; Saengcharoenrat, R; Stallaert,}.; and

Whinston, A.B. Distributed decision support systems for real-time supply
chain management using agent technologies. In R. Kalakota and A.B.
Whinston (eds.), Readings in Electronic Commerce. Reading, MA: Addison
Wesley, 1997, pp. 275-291.
50. Holt, J.J. Advertising Research: Theory and Practice. Upper Saddle River,
NJ: Prentice Hall, 1997.
51. James, D. The future of online research. Marketing News (January 3,
2000), 1,11.
52. Jarvenpaa, S.L., and Todd, P.A. Consumer reactions to electronic shop
ping on the World Wide Web. International Journal of Electronic Commerce, 1,2
(winter 1996-1997), 59-88.
53. Jennings, N.R., and Wooldridge, M.J. (eds.), Agent Technology: Founda
tions, Applications, and Markets. New York: Springer, 1998.
54. Kalfs, N. Hour by Hour: Effects of the Data Collection Mode in Time Use
Research. Amsterdam: NIMMO, 1993.
55. Kannan, P.K. Wireless technology and marketing research. Paper pre
sented at the American Marketing Association Summer Marketing Educa
tors Conference, Chicago, August 6, 2000.
56. Kincaid, L. Incubation of the online sampling frame. CASRO Journal
1999, 119-121.
57. Knoblock, C.A., and Ambite, J.L. Agents for gathering information. In
J.M. Bradshaw (ed.), So?ware Agents. Menlo Park, CA: AAAI Press/MIT
Press, 1997, pp. 347-373.
58. Liang, T-P. Introduction to the special issue: Intelligent agents for
electronic commerce. International Journal of Electronic Commerce, 4,3 (spring
2000), 3-4.
59. Lilie, J. Reader response for the new millennium. Paper presented at the
1999 EXPLOR Forum, Madison, WI, October 21,1999.
60. Lyon, D. The Electronic Eye: The Rise of Surveillance Society. Minneapolis:
University of Minnesota Press, 1994.
61. MacElroy, B. Recruitment and sampling on the Internet. Paper presented
at the IIR conference on "Mastering Web-Based Surveys & Online Research
Techniques," San Diego, June 13, 2000.
62. MacElroy, B.; Miller, C; and Stark, M. Best practices in online research.
Tutorial presented at the 2000 EXPLOR Forum, Chicago, November 16,2000.
63. Malhotra, N.K. Marketing Research: An Applied Orientation, 2d ed. Upper
Saddle River, NJ: Prentice Hall, 1996.
64. McLuhan, M. Understanding Media: The Extensions of Man. New York:
McGraw-Hill, 1965.
65. Mechling, J. Taking Intel Inside? online: Extending an existing brand
into a new channel. Paper presented at the 1999 EXPLOR Forum, Madison,
WI, October 20,1999.
66. Mehta, R., and Sivadas, E. Comparing response rates and response
content in mail versus electronic mail surveys. Journal of the Marketing
Research Society, 37, 4 (October 1995), 429-435.
67. Mena, J. Data Mining Your Website. Woburn, MA: Butterworth
Heinemann, 1999.

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 165

68. Miller, J. Conducting Internet research. Tutorial presented at the Ameri

can Marketing Association Applied Research Methods All-Tutorial Event,
San Antonio, TX, September 25, 2000.
69. Miller, R. Interactive data electronically acquired. Paper presented at the
1999 EXPLOR Forum, Madison, WI, October 21,1999.
70. Miller, T.W. Marketing research and the information industry. CASRO
Journal 2000, 21-26.
71. Miller, T.W., and Gupta, A. Studies of Information, Research, and Consulting
Activities (S1RCS): Fall 2000 Survey of Organizations, A.C. Nielsen Center for
Marketing Research, University of Wisconsin-Madison, Madison, WI,
January 2001.
72. Modahl, M. Now or Never: How Companies Must Change Today to Win the
Battle for Internet Consumers. New York: HarperCollins, 2000.
73. MRI CyberStats. Internet Access and Usage, Percent of Adults 18+. New
York: Mediamark Research, fall 1999. Retrieved from
[Link]/MRI/docs/cs_a.htm, January 30, 2000.
74. Myers, M. General Mills joins Internet venture that will conduct market
research. Minneapolis Star Tribune, June 22, 2000, D23.
75. Nunnally, J.C, and Bernstein, I.H. Psychometric Theory, 3d ed. New York:
McGraw-Hill, 1994.
76. Oliver, J.R. Artificial agents learn policies for multi-issue negotiation.
International Journal of Electronic Commerce, 1,4 (summer 1997), 49-88.
77. Oppermann, M. E-mail surveys: Potentials and pitfalls. Marketing
Research, 7, 3 (summer 1995), 29-33.
78. Palit, C, and Sharp, H. Microprocessor-assisted telephone interviewing.
Sociological Methods and Research, 12 (1983), 169-189.
79. Peppers, D., and Rogers, M. The One to One Future: Building Relationships
One Customer at a Time. New York: Doubleday, 1993.
80. Peterson, D., and Hopkins, N. Going Online with consumer research:
The General Mills experience. Paper presented at the 2000 EXPLOR Forum,
Chicago, November 17, 2000.
81. Rivers, D., and Fallat, D. The Web survey problem. Paper presented at
the IQPC conference on "Online Market Research & Web-Based Surveys,"
San Francisco, January 20, 2000.
82. Rosenfeld, L., and Morville, P. Information Architecture for the World Wide
Web. Sebastopol, CA: O'Reilly, 1998.
83. Saris, W.E. Computer-Assisted Interviewing. Newbury Park, CA: Sage,
1991.
84. Saris, W.E. Ten years of interviewing without interviewers: The
telepanel. In M.P. Couper et al. (eds.), Computer-Assisted Survey Information
Collection. New York: Wiley, 1998, pp. 409-429.
85. Schaefer, D.R., and Dillman, D.A. Development of a standard e-mail
methodology: Results of an experiment. Public Opinion Quarterly, 62, 3 (fall
1998), 378-397.
86. Schafer, H., and Wydra, D. The "E-volution" of parallel testing: Internet
vs. mail & mall. Paper presented at the IIR conference on "Mastering Web
Based Surveys & Online Research," San Diego, June 12, 2000.
87. Shapiro, C, and Varian, H.R. Information Rules: A Strategic Guide to the

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
 266 THOMAS W. MILLER AND PETER R. DICKSON

Network Economy. Boston: Harvard Business School Press, 1999.

88. Smith, M.A., and Leigh, B. Virtual subjects: Using the Internet as an
alternative source of subjects and research environment. Behavior Research
Methods, Instruments & Computers, 29, 4 (1997), 496-505.
89. Stefik, M. The Internet Edge: Social, Legal, and Technological Challenges for a
Networked World. Cambridge, MA: MIT Press, 1999.
90. Terhanian, G. Harris Interactive. Paper presented at the IQPC conference
on "Online Market Research & Web-Based Surveys," San Francisco, January
20, 2000.
91. Tourangeau, R., and Smith, T.W. Collecting sensitive information with
different modes of data collection. In M.P. Couper et al. (eds.), Computer
Assisted Survey Information Collection. New York: Wiley, 1998, pp. 431-453.
92. Tse, A.C.B.; Tse, K.C.; Yin, C.H.; Ting, C.B.; Yi, K.W.; Yee, K.P.; and Hong,
W.C. Comparing two methods of sending out questionnaires: E-mail versus
mail. Journal of the Marketing Research Society, 3,4 (October 1995), 441-446.
93. Urban, G.; Weinberg, B.D.; and Hauser, J.R. Premarket forecasting of
really new products. Journal of Marketing, 60 (January 1996), 47-60.
94. Urban, G.; Hauser, J.; Quails, W.; Weinberg, B.; Bohlmann, J.; and Chicos,
R. Information acceleration: Validation and lessons from the field. Journal of
Marketing Research, 34 (February 1997), 143-153.
95. Vanthuyne, J., and Bates, D. Online research: An Itracks survey of
market research and management consulting firms. Paper presented at the
2000 EXPLOR Forum, Chicago, November 17,2000.
96. Walkowski, J. Internet-based qualitative research: A hands-on overview.
Tutorial presented at the IIR conference on "Mastering Web-Based Surveys
& On-line Research Techniques," San Diego, June 14, 2000.
97. Wilke, J. A Landmark study of the differences between mall-intercept
and Internet interviewing from bases. Paper presented at the Eleventh
Annual Advanced Research Techniques Forum, Monterey, CA, June 5, 2000.
98. Witt, K., and Bernstein, S. Best Practices in Disk-by-Mail Surveys. Sequim,
WA: Sawtooth Software, 1996. Retrieved from [Link]/
[Link], February 1, 2000.
99. Young, S. Using research to improve Web marketing. Tutorial presented
at the IIR conference on "Mastering Web-Based Surveys & Online Research
Techniques," San Diego, June 14, 2000.
100. Zimring, J. Not your traditional online survey: Overcoming obstacles of
kids research online. Paper presented at the 1999 EXPLOR Forum, Madison,
WI, October 21,1999.
THOMAS W. MILLER (twmiller@[Link]) is director of the A.C. Nielsen
Center for Marketing Research at the University of Wisconsin-Madison School of Busi
ness. He manages the center's research program and EXPLOR, an annual awards
program that recognizes "exemplary performance and leadership in online research."
He teaches graduate-level marketing research for the Marketing Department and serves
as academic program adviser for graduate students associated with the Nielsen Cen
ter. For ten years he worked in the computer industry, where he held technical, sales,
and marketing positions. Miller's research work has been interdisciplinary, involving
marketing, measurement, statistics, and computer science. He has designed and con
ducted numerous empirical and simulation studies comparing traditional and mod
em methods for classification and regression. He has published papers about statistical

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]
 INTERNATIONAL JOURNAL OF ELECTRONIC COMMERCE 167

model selection and has served as an invited speaker, trainer, and consultant on
modern, data-adaptive modeling methods. His active research includes explora
tions of online research methods and studies of consumer choice and use of technol
ogy products.

PETER R. DICKSON (dicksonp@[Link]) is the Knight-Ridder Eminent Scholar in

Global Marketing at Florida International University (FIU). He was previously the
Arthur C. Nielsen Jr., Chair of Marketing Research at the University of Wisconsin
Madison (1995-2000), the Crane Professor of Strategic Marketing and an adjunct Pro
fessor of Industrial Design at Ohio State University (1981-1994). He has written some
70 articles on buyer and seller behavior, particularly on market segmentation, pricing
and price search, decision-making biases, and competitive rationality. He is setting up
a process-thinking research laboratory at FIU's Knight-Ridder Center for Manage
ment Excellence. His current major research streams are process thinking, global com
petitive dynamics, and marketing planning practices. His 1992 paper on competitive
rationality won the Journal of Marketing Maynard Award and he has won other awards
for his research, teaching, and service. His book Marketing Management as been used
at more than a hundred business schools. He is also a major contributor to an intro
ductory marketing text. Professor Dickson has been a vice-president of marketing in a
successful start-up company, is president of another start-up, National Family Ar
chives, and is a partner in a software development firm, Stratmesh Software. He is
currently the chair of the American Marketing Association Strategy special-interest
group and as a senior fellow is helping found the Guanghua Strategic Marketing
Research Center at Beijing University.

This content downloaded from [Link] on Sat, 02 May 2020 06:25:44 UTC
All use subject to [Link]