From the Ph.D.
thesis of Pedro Quelhas Brito
[Link] Sample definition
Kish (1995) observed that a sample design should balance four broad criteria: (1)
goal orientation, (2) measurability, (3) practicality, and (4) economy. In addition to these
criteria, and in spite of the fact that this survey is not a panel, there are some specific
aspects borrowed from the longitudinal studies that should be taken into account. All
together the aspects influencing the definition of the sample design are:
- Large amount of data to be obtained;
- Wide variety of question formats applied;
- Use of multimethod approach – questions asked verbally and self-completed;
- Task complexity – the administration procedure involves not only the collection
of data but also the mode and the time of response has to be recorded;
- Need to keep and reassure continuity in the respondent willingness to participate
in at least three interactions over a span of three months;
- Market research companies’ advice and the researcher’s experience in managing
consumer panels.
Thornton et al. (1982) advocate that long-term and successful cooperation depends
on the quality of the interaction between the interviewer and the respondent. Thus, both
individuals involved in the data collection process have an equal part in that success.
Those authors, based on their experience in maintaining the respondent’s cooperation,
document some techniques to explain their high success rate for completing interviews
over a period of eighteen years. The respondents grant access to their time, space and
psychological availability for the researcher to explore their inner beliefs and actions.
The interviewers offer their expertise, persistence and tolerance for contact to set up
appointments, to listen, to probe, to wait, to learn and to report. They have to ensure
trust and understanding. The output of this interaction has to be satisfying and socially
valuable. Motivation is the key factor for both elements. Additionally, the interviewers’
morale, motivation and skill accounts for some of their expected psychological profile.
According to Thornton et al. (1982) the attractiveness of the study content, flexibility in
some aspects of procedures implementation, and some form of financial reward or gift,
help to raise the respondents’ willingness to cooperate. Similarly, on the interviewer’s
side the following aspects play a relevant role in enhancing their achievement: technical
training; communication of the importance of the research and the rationale behind the
1
From the Ph.D. thesis of Pedro Quelhas Brito
questions; researcher availability to incorporate procedural advice which they suggested;
and a facilitating questionnaire design.
The two-stage data collection technique consists of recruiting existing customers and
prospective respondents by asking them to participate in a telephone interview. The
second stage is where those consumers who accepted to participate were contacted. This
approach generated higher cooperation rates, data accuracy and sufficient quantity (Pol,
1992; Pol and Pak, 1994; Pak and Pol, 1995).
The goal and scope of this thesis research covers an extensive dimension of peoples’
lives in their role as consumers. The family decision-making process is dynamic because
it evolves according to the life-cycle and also as a function of the situation itself (Spiro,
1983). Accepting the premise that the couple and/or their children, rather than the
individual, is the decision-making unit, the relative influence of the members varies with
the nature of the purchase and the potential user or users of that purchase (Spiro, 1983;
Kim and Lee, 1989). Despite the culturally framed egalitarian movement, mainly
stimulated by families having double incomes which results from female workforce
emancipation, many husbands and wives still lead perform separate roles (Ford et al.,
1995). For example, grocery shopping is still quite limited among husbands (Dholakia et
al., 1995). Domestic chores are extensively taken care of by wives, whereas financial
decisions are dominated by husbands (Kim and Lee, 1989). Though the cultural
environment moderates any determinism, it is reasonable to assume that the wives and
mothers have an accurate and prevalent knowledge of the situation, preferences and
conditions under which the family and each family member behaves. Her intuition,
emotional traits, sensibility and specific interests makes her particularly well-positioned
to embark upon and to be more involved in certain shopping rituals, such as gift-giving
(Fischer and Arnold, 1990). Hence, the sample units are households but the sampling
element unit is the wife, or head of the household in the case of a single-parent home.
Drawing together the above arguments regarding: (1) the attention to be drawn
both to the respondent and the interviewer, (2) the benefit of previously defining a list of
respondents in order to assure their cooperation before starting the survey, (3) the focus
on the best informed family member, it is possible to explain the rationale and to
describe the procedure used to set up the sampling process (see Table 4.4).
Initially, approximately one thousand families volunteered to participate. They
were student’s relatives, friends and a network of respondents belonging to a field work
agency. Subsequently, based on their demographic profile record including indicators
2
From the Ph.D. thesis of Pedro Quelhas Brito
such as age, family size, income and mainly residential area, they were distributed in
five groups. Within each group, approximately, half were randomly selected and
recruited. The first step of this procedure obtained a sample frame. The main criterion
specified in the instructions used to allocate the prospective respondents into specific
groups was the residential area dispersion. An equal geographical distribution of
respondents is considered important; due to the relevance of the convenience attribute
among the definition of the preferences of consumers in their choice of the retail setting.
In practical terms, this concern was translated into the preparation of a list of households
located in different streets, neighbourhoods and geographic locations of the Porto
metropolitan area by each interviewer. Regarding the other demographic variables
mentioned above, excepting for the gender where the preference fell to females as was
justified above, this research sought diversity and a high level of variety among the
possible values taken by each variable. The interviewers themselves, as a condition of
recruitment, had to fulfil a set of requirements which were subsequently evaluated:
motivation, willingness and availability to carry on the interviewing task until its end.
The major benefit of these procedures is seen in the sample control. It refers to how
efficiently and effectively the specified survey elements are reached and motivated (Yu
and Cooper, 1983; Childers and Skinner, 1985).
Table 4.4 Sampling’s procedures and activities
1º Twenty of twenty-six prospective interviewers that showed interest in this project
were initially selected;
2º After an explanation and description of the study, written and verbal instructions of
the procedures to develop the interview process were given;
3º The training was a simulation of the interview first between the researcher and the
interviewer and then the interviewers between themselves. The main concern was the
measure of response time. As a chronometer could not be used during the interviews,
each interviewer had to develop the ability to internally measure 4 seconds and 10
seconds. The chronometer pace and interviewer counting cadence had to match each
other in order to manage the speed of the questioning sequence and to respect the
automatic cognitive processing approach;
4º Once interviewers had acquired enough confidence and knowledge about the
questionnaire and the administration procedures they started the pilot test stage. That
3
From the Ph.D. thesis of Pedro Quelhas Brito
was to test themselves. The results were discussed and some changes were incorporated
in to the instructions;
5º There were two supervisors – one from research and one responsible for the field
work agency – whose tasks were to monitor their performance, to help and to solve any
incurring problem;
6º A cover letter from the University of Porto was sent to all volunteer respondents,
pointing out the purpose, importance, and duration of the research. Appreciation of their
commitment for more than six months involvement in this study was reinforced by a
gift– a book about the Euro and a euro-escudo converter calculator. Finally, the
researcher gave his University address, phone number, electronic mail and mobile phone
number and expressed his availability for any further inquiry (see Appendices 4.1 and
4.2).
Recruiting a large number of interviewers contributes to minimising the impact
of the interviewer variability. Kish (1962) suggests a maximum workload of about 30.
The precautions in the grounding of interviewers’ performance and fieldwork planning
benefit from the advice and guidelines of Collins (1980). The actions undertaken for
reducing interviewer variability and fieldwork operations were inspired in Kinner and
Taylor’s recommendations (1991).
The sample definition so far described above represents a trade-off between the
need to foster the respondents’ cooperation during the time span and the need to assure
the representativeness of the sample. This latter goal denotes the aims of adequately
representing the population in the sample, in order to draw conclusions susceptible to
expressing the nature of that population. The constraints mentioned above hinder the
application of a probability sampling based on mechanical population randomization,
where each element has the same probability of being selected. Instead, the applied
method was a combination of a convenience sample and then a statistically-random
generated sample. Thus, this latter option allows the use of objective statistical inference
(Kish, 1995).
The sample size can be determined by taking qualitative and quantitative factors
into consideration. If a probability sample method was used it would be possible to
apply the formula:
2
2 ⎛ Z ⎞
n =σ ⎜ ⎟
⎝ SE ⎠
4
From the Ph.D. thesis of Pedro Quelhas Brito
where the Z designates the confidence level generally ranging from 90% to 95%; the
allowed maximum sampling error (SE) usually adopted is 0.5; finally the (σ2) stands for
the population variation of the relevant variable under scrutiny (e.g., Aaker and Day,
1990; Kish, 1995; Malhotra, 1996; Aaker, Kumar and Day, 1995).
However, the latter parameter (σ2) is generally unknown, giving rise to the
problem of obtaining an estimation of its value. Assuming that there are not comparable
studies where the research can extrapolate a proxy, one option to solve this problem
consists of running a pilot sample aimed at estimating the variance (Shiffler and Adams,
1987; Gillet, 1989). An additional practical problem occurs when a decision has to be
made concerning the choice of the relevant variable. In this thesis research there are
more than one thousand variables, among which the research does not take the risk of
attributing one variable as being more relevant than any other. A short but somewhat
arbitrary list of relevant variables could be taken, and used to estimate the variance and
to achieve a sample size. Then the highest value could be taken as the reference. The
main drawback would be the three stages of interviews necessary to obtain that list!
Alternatively, variance as function of a proportional value of the width of the confidence
interval could be taken into consideration and then, by using the Seelbinder’ table
(1953), get the sample size directly.
Another clue to setting up the sample size stems from the theoretical
requirements associated with to the different multivariate data analysis techniques. In
general, researchers use other academics’ recommendations based on some conventional
value, presumed to be appropriate in the field study, also called rules-of-thumb (Green,
1991). Therefore, depending upon the multivariate technique, the minimum number of
subjects varies.
- Regression analysis – One possible rule to estimate the minimum sample size is
expressed through “N≥ 50+8m”, “m” being the number of predictors (Green,
1991);
- Factor analysis – The recommended number of subjects-to-number of variables
ratio is ten-to-one (e.g., Malhotra, 1996; Hair et al., 1998);
- Structural equation modelling – Regardless of the majority of the goodness-of-fit
indexes being incrementally affected by the increase of sample size, in general,
the accuracy of causal models improves as the number of subjects expands
(Bearden et al.1982; Marsh et al., 1988). The size of sample must take into
5
From the Ph.D. thesis of Pedro Quelhas Brito
account the model size, the normality assumption and the estimation procedure.
Considering all this, conditions with a ratio of 15 respondents for each parameter
is acceptable. Conversely, a large sample size (exceeding 400 to 500), brings
about a “too sensitive” and unstable causal model estimation, leading
occasionally to a poor fit (Tanaka, 1987; Marsh et al., 1988; Ding et al., 1995;
Wang et al., 1996). Consequently, although there is no pre-determined sample
size, a sample size ranging between 100 and 200 could be good starting point.
[Link] Sources of errors and accuracy
The literature divides the total error into two categories. The variance between
the population mean and the observed mean value results from the effect of random
sampling error and nonsampling error (Malhotra, 1996; Aaker, Kumar and Day, 1995).
The former error, addressed in the earlier section, is associated with the sampling
method and its contribution to the total error is minimal (Assael and Keon, 1982). The
latter is composed of response and nonresponse error. Ultimately, the researcher is
responsible for all measurement errors. Even the nonresponse errors, due to refusal to
cooperate, is in some degree a function of the characteristics of the data collection
procedure, such as the length of the interview. That error becomes eminent when the
pattern of responses between the two groups of respondents varies significantly (Aaker,
Kumar and Day, 1995).
Among the response errors, there are three possible categories of errors: the
interviewer errors, respondent errors and, researcher errors (Malhotra, 1996). The
remedies to minimise interviewer error were addressed in the previous section. Figure
4.2 gives a checklist of respondents’ sources of error and questionnaire design axioms,
which will be studied in detail in the next section.
Measurement is a central issue in science and therefore in marketing.
Measurement “consists of rules for assigning symbols to objects so as to represent
quantities of attributes numerically” (Nunnally and Bernstein, 1994, p.3).
The standardization implied by rules yields unambiguity, clarity and practicality
in the universal application. Scaling, or the representation of attributes by numbers
expressing quantities, is not enough to reassure meaningfulness. One aspect of
meaningfulness stems from the ability to observe an attribute, also called measurement,
6
From the Ph.D. thesis of Pedro Quelhas Brito
to portray some inaccuracies inherent to that measurement process. Measures of a
construct are obscured by systematic and random errors.
Figure 4.2 Some sources of error in questionnaire design and from respondents
Respondent inaccuracy some solutions
- Telescoping - recency and importance of the event
Inability to respond - Averaging - avoid retrospective questions
- Omission
- No conscious awareness - measure response time lag
of motives
- Invasion of privacy - grouping the questions and take
- Time pressure and fatigue coffee breaks
Unwillingness to - Social desirability response bias - splitting the questionnaire into
respond accurately - Courtesy bias three phases
- Uninformed response error - probing and silence
- Response style - face to face personal interview
Questionnaire design some solutions
- Order effects
-Ambiguity
- Sensitivity
Error in questions - Unbalanced response alternatives see section 4.3.2
- Lack of relevance
- Lack of clarity-vocabulary
- Wording bias
Sources: Weiers (1988), Tull and Hawkins (1990), Reynolds et al. (1993), Malhotra (1996),
Dutka and Frankel (1997), Aaker, Kumar and Day (1995).
7
From the Ph.D. thesis of Pedro Quelhas Brito
The importance of the source of variance which is not due to attribute or construct
characteristic can be considerable (Cote and Buckley, 1987). Thus, its size or relative
importance has to be assessed. The method used to evaluate the extent of the random
error is the reliability test. Reliability captures the degree to which a scale consistently
brings in similar results if a repeated measurement is done (Peter, 1979). There are
various approaches for testing the psychometric stability of data and constructs,
including test-retest, alternative forms and internal consistency (e.g., Burns and
Harrison, 1979; Parameswaran et al, 1979; Peter, 1979). This thesis research applies
many of them. Test-retest and alternative forms are not used in an absolutely
conventional way. The spreading of similar questions across two phases of the
questionnaire administration closely follows the first approach. However, the use of two
equivalent content questions, which do not express identical format, partially departs in
the current practice from the instruction underlying those methods. The same content
question, regarding the naming of a respondent’s meaningful store and nature of
attributes associated to that specific store, was assessed in the first phase through the
automatic cognitive processing approach assumption. In the third phase, framed by the
same shopping scenario, respondents were asked to name their preferred retailer and rate
previously assigned attributes. Then the incoming data of both phases are correlated.
Cronbach’s alpha is the most used internal consistency reliability method. Despite the
coefficient being influenced by response options, such as the number of scales, and
some respondent characteristics, like age (Alwin and Krosnick, 1991), in general its
magnitude is not related to any specific research design (Peterson, 1994).
Reliability assessment is just one side of the scientific verification. The other
side is validity. This refers to the accuracy with which a scale actually measures what it
purports to measure (e.g., Peter, 1981; Malhotra, 1996). Content validity is the most
immediate way of subjectively assessing the suitability of a measurement scale,
especially during the pre-testing phase (Calder et al., 1982). Construct validity is a more
sophisticated form of validity, concerned with the theoretical meaning of the association
of a construct at a conceptual level, and it has the purported measure translated at the
operational level (Peter, 1981). Construct validity comprises three types of
complementary procedures: convergent validity, discriminant validity and nomological
validity. In the former technique two sets of scales belonging to the same construct
8
From the Ph.D. thesis of Pedro Quelhas Brito
should correlate positively. If there is a lack of correlation among constructs which are
supposed to differ, then it means the discriminant validity will be confirmed. The
nomological validity works whenever different sets of scales are correlated as
theoretically predicted (e.g., Peter, 1981; Malhotra, 1996).
4.3.2 Derivation of the questionnaire
The questionnaire is the basic quantitative research tool used in the data
collection process. Thus, its scientific quality and performance determines, among other
factors, the scientific investigation and outcome. Its function is measurement. This
section describes the conditions underlying the questionnaire development, including the
contribution of the pre-test stage. The response scale formats, sequencing and nature of
questions used are justified under the objective of minimising error, stated in the
previous section, and under the thesis research objectives. Finally, a brief representation
of constructs is offered and data quality assessment results are presented.
[Link] Questionnaire development
Four assumptions underlie the questionnaire conception and determine the
interview process:
1) Questionnaire capacity of organising and capturing all information
needed to fulfil the theoretical goals of this research;
2) Respondents hold the required information;
3) Respondents can access that information;
4) Respondents are able to verbalise their knowledge accurately.
Interviewers’ selection and training, as it is described in the previous sections,
reflect the interviewers’ responsibility for the quality of the data collection. The
questionnaire design makes sure that it will be comprehensive and inclusive regarding
all needed information, easing the respondents’ decoding-encoding task. Overall, Figure
4.3 portrays the flow of activities associated in the development of the questionnaire
(Foddy, 1999), (see Appendices 4.3 and 4.4).
9
From the Ph.D. thesis of Pedro Quelhas Brito
Figure 4.3 Questionnaire development flow-chart
Define Topics
(research goals)
Define the nature and the quality of
information required about the topic
- Context
- Wording – unintended
meaning, similar words.
Formulation of the - Grammatical complexities
information requested: - Negative/positive sentences
nature and format - Questions introduction
- Non-response options
- Multi-attribute rating
- Rating scales format
- Sequence of items/questions
Questionnaire
pre-testing
Source: Adapted from Foddy (1999)
The first draft was edited after defining the content and the structure of the
questionnaire. This research instrument had two problems during the design process: (1)
sometimes new scales had to be introduced taking into consideration the specific
theoretical orientations and the information gathered during the qualitative stage, (2)
other authors’ scales had to be translated and adapted to the Portuguese language, which
required an additional effort to make sure that the fundamental meaning and
understanding was kept intact. In this research process, the pre-test stage looked to
address the following issues:
- Meaning, probing, familiar terms and understanding response;
- Clarity, ambiguity and relevance;
- Response time;
- Order sequence.
10