CHAPTER 7

Survey Sampling

7.1 Introduction
Resting on the probabilistic foundations of the preceding chapters, this chapter marks
the beginning of our study of statistics by introducing the subject of survey sampling.
As well as being of considerable intrinsic interest and practical utility, the development
of the elementary theory of survey sampling serves to introduce several concepts and
techniques that will recur and be amplified in later chapters.
Sample surveys are used to obtain information about a large population by exam-
ining only a small fraction of that population. Sampling techniques have been used
in many fields, such as the following:
• Governments survey human populations; for example, the U.S. government con-
ducts health surveys and census surveys.
• Sampling techniques have been extensively employed in agriculture to estimate
such quantities as the total acreage of wheat in a state by surveying a sample of
farms.
• The Interstate Commerce Commission has carried out sampling studies of rail and
highway traffic. In one such study, records of shipments of household goods by
motor carriers were sampled to evaluate the accuracy of preshipment estimates of
charges, claims for damages, and other variables.
• In the practice of quality control, the output of a manufacturing process may be
sampled in order to examine the items for defects.
• During audits of the financial records of large companies, sampling techniques may
be used when examination of the entire set of records is impractical.

The sampling techniques discussed here are probabilistic in nature—each mem-

ber of the population has a specified probability of being included in the sample, and
the actual composition of the sample is random. Such techniques differ markedly from
199
200 Chapter 7 Survey Sampling

the type of sampling scheme in which particular population members are included
in the sample because the investigator thinks they are typical in some way. Such a
scheme may be effective in some situations, but there is no way mathematically to
guarantee its unbiasedness (a term that will be precisely defined later) or to estimate
the magnitude of any error committed, such as that arising from estimating the popu-
lation mean by the sample mean. We will see that using a random sampling technique
has a consequence that estimates can be guaranteed to be unbiased and probabilistic
bounds on errors can be calculated. Among the advantages of using random sampling
are the following:
• The selection of sample units at random is a guard against investigator biases, even
unconscious ones.
• A small sample costs far less and is much faster to survey than a complete enumer-
ation.
• The results from a small sample may actually be more accurate than those from a
complete enumeration. The quality of the data in a small sample can be more easily
monitored and controlled, and a complete enumeration may require a much larger,
and therefore perhaps more poorly trained, staff.
• Random sampling techniques make possible the calculation of an estimate of the
error due to sampling.
• In designing a sample, it is frequently possible to determine the sample size neces-
sary to obtain a prescribed error level.

Peck et al. (2005) contains several interesting papers about applications of

sampling.

7.2 Population Parameters

This section defines those numerical characteristics, or parameters, of the population
that we will estimate from a sample. We will assume that the population is of size
N and that associated with each member of the population is a numerical value of
interest. These numerical values will be denoted by x1 , x2 , · · ·, x N . The variable xi
may be a numerical variable such as age or weight, or it may take on the value 1 or
0 to denote the presence or absence of some characteristic such as gender. We will
refer to the latter situation as the dichotomous case.

EXAMPLE A This is the first of many examples in this chapter in which we will illustrate ideas
by using a study by Herkson (1976). The population consists of N = 393 short-
stay hospitals. We will let xi denote the number of patients discharged from the ith
hospital during January 1968. A histogram of the population values is shown in Fig-
ure 7.1. The histogram was constructed in the following way: The number of hospitals
that discharged 0–200, 201– 400, . . . , 2801–3000 patients were graphed as horizon-
tal lines above the respective intervals. For example, the figure indicates that about
 7.2 Population Parameters 201

80

60

Count
40

20

0
0 500 1000 1500 2000 2500 3000 3500
Number of discharges

F I G U R E 7.1 Histogram of the numbers of patients discharged during January 1968

from 393 short-stay hospitals.

40 hospitals discharged from 601 to 800 patients. The histogram is a convenient

graphical representation of the distribution of the values in the population, being
more quickly assimilated than would a list of 393 values. ■

We will be particularly interested in the population mean, or average,

N
1 ⎪
µ= xi
N i=1

For the population of 393 hospitals, the mean number of discharges is 814.6. Note
the location of this value in Figure 7.1. In the dichotomous case, where the presence
or absence of a characteristic is to be determined, µ equals the proportion, p, of
individuals in the population having the particular characteristic, because in the sum
above, each xi is either 0 or 1. The sum thus reduces to the number of 1s and when
divided by N , gives the proportion, p.
The population total is
N
⎪
τ= xi = N µ
i=1

The total number of people discharged from the population of hospitals is τ =

320,138. In the dichotomous case, the population total is the total number of members
of the population possessing the characteristic of interest.
We will also need to consider the population variance,
N
1 ⎪
σ2 = (xi − µ)2
N i=1
202 Chapter 7 Survey Sampling

A useful identity can be obtained by expanding the square in this equation:

⎪ N N
⎩
1 ⎨ ⎨
2
σ = xi2 − 2µ xi + N µ 2
N i=1 i=1
⎪ N ⎩
1 ⎨
= xi2 2
− 2N µ + N µ 2
N i=1
N
1 ⎨ 2
= x − µ2
N i=1 i

In the dichotomous case, the population variance reduces to p(1 − p):

N
1 ⎨ 2
σ2 = x − µ2
N i=1 i
= p − p2
= p(1 − p)

Here we used the fact that because each xi is 0 or 1, each xi2 is also 0 or 1.
The population standard deviation is the square root of the population variance
and is used as a measure of how spread out, dispersed, or scattered the individual values
are. The standard deviation is given in the same units (for example, inches) as are the
population values, whereas the variance is given in those units squared. The variance
of the discharges is 347,766, and the standard deviation is 589.7; examination of
the histogram in Figure 7.1 makes it clear that the latter number is the more reasonable
description of the spread of the population values.

7.3 Simple Random Sampling

The most elementary form of sampling is simple random sampling (s.r.s.): Each
particular
⎡N ⎢ sample of size n has the same probability of occurrence; that is, each of the
n
possible samples of size n taken without replacement has the same probability.
We assume that sampling is done without replacement so that each member of the
population will appear in the sample at most once. The actual composition of the
sample is usually determined by using a table of random numbers or a random number
generator on a computer. Conceptually, we can regard the population members as
balls in an urn, a specified number of which are selected for inclusion in the sample
at random and without replacement.
Because the composition of the sample is random, the sample mean is random.
An analysis of the accuracy with which the sample mean approximates the population
mean must therefore be probabilistic in nature. In this section, we will derive some
statistical properties of the sample mean.
 7.3 Simple Random Sampling 203

7.3.1 The Expectation and Variance of the Sample Mean

We will denote the sample size by n (n is less than N ) and the values of the sample
members by X 1 , X 2 , . . . , X n . It is important to realize that each X i is a random vari-
able. In particular, X i is not the same as xi : X i is the value of the ith member of the sam-
ple, which is random and xi is that of the ith member of the population, which is fixed.
We will consider the sample mean,
n
1⎨
X= Xi
n i=1
as an estimate of the population mean. As an estimate of the population total, we will
consider
T = NX
Properties of T will follow readily from those of X . Since each X i is a random
variable, so is the sample mean; its probability distribution is called its sampling
distribution. In general, any numerical value, or statistic, computed from a random
sample is a random variable and has an associated sampling distribution. The sampling
distribution of X determines how accurately X estimates µ; roughly speaking, the
more tightly the sampling distribution is centered on µ, the better the estimate.

EXAMPLE A To illustrate the concept of a sampling distribution, let us look again at the population
of 393 hospitals. In practice, of course, the population would not be known, and only
one sample would be drawn. For pedagogical purposes here, we can consider the
sampling distribution of the sample mean from this known population. Suppose, for
example, that we want to find the sampling
⎡ ⎢ distribution of the mean of a sample of size
16. In principle, we could form all 393 16
samples and compute the mean of each one—
this would give the sampling distribution. But because the number of such samples is
of the order 1033 , this is clearly not practical. We will thus employ a technique known
as simulation. We can estimate the sampling distribution of the mean of a sample of
size n by drawing many samples of size n, computing the mean of each sample, and
then forming a histogram of the collection of sample means. Figure 7.2 shows the
results of such a simulation for sample sizes of 8, 16, 32, and 64 with 500 replications
for each sample size. Three features of Figure 7.2 are noteworthy:
1. All the histograms are centered about the population mean, 814.6.
2. As the sample size increases, the histograms become less spread out.
3. Although the shape of the histogram of population values (Figure 7.1) is not
symmetric about the mean, the histograms in Figure 7.2 are more nearly so.
These features will be explained quantitatively. ■

As we have said, X is a random variable whose distribution is determined by

that of the X i . We thus examine the distribution of a single sample element, X i . It
should be noted that the following lemma holds whether sampling is with or without
replacement.
204 Chapter 7 Survey Sampling

120
100
80

Count
60
40
20
0
200 400 600 800 1000 1200 1400 1600 1800
(a)

140

100
Count

60

20
0
200 400 600 800 1000 1200 1400 1600 1800
(b)

120
100
80
Count

60
40
20
0
200 400 600 800 1000 1200 1400 1600 1800
(c)

140

100
Count

60

20
0
200 400 600 800 1000 1200 1400 1600 1800
(d)

F I G U R E 7.2 Histograms of the values of the mean number of discharges in 500

simple random samples from the population of 393 hospitals. Sample sizes: (a) n = 8,
(b) n = 16, (c) n = 32, (d) n = 64.

We need to be careful about the values that the random variable X i can assume.
The i th sample member is equally likely to be any of the N population members. If
all the population values were distinct, we would then have P(X 1 = x j ) = 1/N .
But the population values may not be distinct (for example, in the dichotomous case
 7.3 Simple Random Sampling 205

there are only two values, 0 and 1). If k members of the population have the same
value ζ , then P(X i = ζ ) = k/N . We use this construction in proving the following
lemma.

LEMMA A
Denote the distinct values assumed by the population members by ζ1 , ζ2 , . . . , ζm ,
and denote the number of population members that have the value ζ j by n j , j =
1, 2, . . . , m. Then X i is a discrete random variable with probability mass
function
nj
P(X i = ζ j ) =
N
Also,
E(X i ) = µ
Var(X i ) = σ 2

Proof
The only possible values that X i can assume are ζ1 , ζ2 , . . . , ζm . Since each mem-
ber of the population is equally likely to be the ith member of the sample, the
probability that X i assumes the value ζ j is thus n j /N . The expected value of the
random variable X i is then
m
⎨ m
1 ⎨
E(X i ) = ζ j P(X i = ζ j ) = n jζj = µ
j=1
N j=1

The last equation follows because n j population members have the value ζ j
and the sum is thus equal to the sum of the values of all the population members.
Finally,
⎡ ⎢
Var(X i ) = E X i2 − [E(X i )]2
m
1 ⎨
= n j ζ j2 − µ2
N j=1

= σ2
⎣N 2
⎣ m 2
Here we have used the fact that i=1 x i = j=1 n j ζ j and the identity for the
population variance derived in Section 7.2. ■

As a measure of the center of the sampling distribution, we will use E(X ). As a

measure of the dispersion of the sampling distribution about this center, we will use
the standard deviation of X . The key results that will be obtained shortly are that the
sampling distribution is centered at µ and that its spread is inversely proportional to
the square root of the sample size, n. We first show that the sampling distribution is
centered at µ.
206 Chapter 7 Survey Sampling

THEOREM A
With simple random sampling, E(X ) = µ.

Proof
Since, from Lemma A, E(X i ) = µ, it follows from Theorem A in Section 4.1.2
that
n
1⎨
E(X ) = E(X i ) = µ ■
n i=1

From Theorem A, we have the following corollary.

COROLLARY A
With simple random sampling, E(T ) = τ.

Proof

E(T ) = E(N X )
= N E(X )
= Nµ
=τ ■

In the dichotomous case, µ = p, and X is the proportion of the sample that

possesses the characteristic of interest. In this case, X will be denoted by p̂. We have
shown that E( p̂) = p.
It is important to keep in mind that X is random. The result E(X ) = µ can be
interpreted to mean that “on the average” X = µ. In general, if we wish to estimate
a population parameter, θ say, by a function θ̂ of the sample, X 1 , X 2 , . . . , X n , and
E(θ̂) = θ, whatever the value of θ may be, we say that θ̂ is unbiased. Thus, X
and T are unbiased estimates of µ and τ . On average they are correct. We next
investigate how variable they are, by deriving their variances and standard deviations.
Section 4.2.1 introduced the concepts of bias and variance in the context of a model
of measurement error, and these concepts are also relevant in this new context. In
Chapter 4, it was shown that
Mean squared error = variance + bias2
Since X and T are unbiased, their mean squared
⎣n errors are equal to their variances.
We next find Var(X ). Since X = n −1 i=1 X i , it follows from Corollary A of
Section 4.3 that
n n
1 ⎨⎨
Var(X ) = 2 Cov(X i , X j )
n i=1 j=1
 7.3 Simple Random Sampling 207

Suppose that sampling were done with replacement. Then the X i would be inde-
pendent, and for i = " j we would have Cov(X i , X j ) = 0, whereas Cov(X i , X i ) =
Var(X i ) = σ 2 . It would then follow that
n
1 ⎨
Var X = Var(X i )
n 2 i=1
σ2
=
n
and that the standard deviation of X , also called its standard error, would be
σ
σX = √
n

Sampling without replacement induces dependence among the X i , which com-

plicates this simple result. However, we will see that if the sample size n is small
relative to the population size N , the dependence is weak and this simple result holds
to a good approximation.
To find the variance of the sample mean in sampling without replacement we
need to find Cov(X i , X j ) for i =
" j.

LEMMA B
For simple random sampling without replacement,
Cov(X i , X j ) = −σ 2 /(N − 1) if i =
" j
Using the identity for covariance established at the beginning of Section 4.3,
Cov(X i , X j ) = E(X i X j ) − E(X i )E(X j )
and
m ⎨
⎨ m
E(X i X j ) = ζk ζl P(X i = ζk and X j = ζl )
k=1 l=1
m
⎨ m
⎨
= ζk P(X i = ζk ) ζl P(X j = ζl |X i = ζk )
k=1 l=1

from the multiplication law for conditional probability. Now,

⎤
n l /(N − 1), if k =
" l
P(X j = ζl |X i = ζk ) =
(n l − 1)/(N − 1), if k = l
Now if we express
m
⎨ ⎨ nl nk − 1
ζl P(X j = ζl |X i = ζk ) = ζl + ζk
l=1 l=
" k
N −1 N −1
m
⎨ nl 1
= ζl − ζk
l=1
N −1 N −1
208 Chapter 7 Survey Sampling

the expression for E(X i X j ) becomes

m
⎪ m ⎩ ⎪ m
⎩
⎨ nk ⎨ nl ζk 1 ⎨
2 2
ζk ζl − = τ − ζk n k
k=1
N l=1 N − 1 N −1 N (N − 1) k=1
⎨m
τ2 1
= − ζ 2nk
N (N − 1) N (N − 1) k=1 k
N µ2 1
= − (µ2 + σ 2 )
N −1 N −1
σ2
= µ2 −
N −1
2
Finally, subtracting E(X i )E(X j ) = µ from the last equation, we have
σ2
Cov(X i , X j ) = −
N −1
for i =
" j. ■

(Alternative proofs of Lemma B are outlined in Problems 25 and 26 at the end of

this chapter.) This lemma shows that X i and X j are not independent of each other for
i=" j, but that the covariance is very small for large values of N . We are now able to
derive the following theorem.

THEOREM B
With simple random sampling,
⎥ ⎦
σ2 N −n
Var(X ) =
n N −1
⎥ ⎦
σ2 n−1
= 1−
n N −1

Proof
From Corollary A of Section 4.3,
n n
1 ⎨⎨
Var(X ) = Cov(X i , X j )
n 2 i=1 j=1
n n
1 ⎨ 1 ⎨⎨
= 2 Var(X i ) + 2 Cov(X i , X j )
n i=1 n i=1 j =" i

σ2 1 σ2
= − 2 n(n − 1)
n n N −1
After some algebra, this gives the desired result. ■
 7.3 Simple Random Sampling 209

Notice that the variance of the sample mean in sampling without replacement
differs from that in sampling with replacement by the factor
⎥ ⎦
n−1
1−
N −1
which is called the finite population correction. The ratio n/N is called the sampling
fraction. Frequently, the sampling fraction is very small, in which case the standard
error (standard deviation) of X is
σ
σX ≈ √
n
We see that, apart from the usually small finite population correction, the spread of the
sampling distribution and therefore the precision of X are determined by the sample
size (n) and not by the population size (N ). As will be made more explicit later,
the appropriate measure of the precision of the sample mean is its standard error,
which is inversely proportional to the square root of the sample size. Thus, in order
to double the accuracy, the sample size must be quadrupled. (You might examine
Figure 7.2 with this in mind.) The other factor that determines the accuracy of the
sample mean is the population standard deviation, σ . If σ is small, the population
values are not very dispersed and a small sample will be fairly accurate. But if the
values are widely dispersed, a much larger sample will be required in order to attain
the same accuracy.

EXAMPLE B If the population of hospitals is sampled without replacement and the sample size is
n = 32,
*
σ n−1
σX = √ 1−
n N −1
*
589.7 31
= √ 1−
32 392
= 104.2 × .96
= 100.0

Notice that because the sampling fraction is small, the finite population correction
makes little difference. To see that σ X = 100.0 is a reasonable measure of accuracy,
examine part (b) of Figure 7.2 and observe that the vast majority of sample means
differed from the population mean (814) by less than two standard errors; i.e., the
vast majority of sample means were in the interval (614, 1014). ■

EXAMPLE C Let us apply this result to the problem of estimating a proportion. In the population of
hospitals, a proportion p = .654 had fewer than 1000 discharges. If this proportion
were estimated from a sample as the sample proportion p̂, the standard error of p̂
210 Chapter 7 Survey Sampling

could be found by applying Theorem B to this dichotomous case:

* *
p(1 − p) n−1
σ p̂ = 1−
n N −1

For example, for n = 32, the standard error of p̂ is

* *
.654 × .346 31
σ p̂ = 1−
32 392
= .08 ■

The precision of the estimate of the population total does depend on the population
size, N .

COROLLARY B
With simple random sampling,
⎥ ⎦
σ2 N −n
Var(T ) = N 2
n N −1

Proof
Since T = N X ,
Var(T ) = N 2 Var(X ) ■

7.3.2 Estimation of the Population Variance

A sample survey is used to estimate population parameters, and it is desirable also
to assess and quantify the variability of the estimates. In the previous section, we
saw how the standard error of an estimate may be determined from the sample size
and the population variance. In practice, however, the population variance will not
be known, but as we will show in this section, it can be estimated from the sample.
Since the population variance is the average squared deviation from the population
mean, estimating it by the average squared deviation from the sample mean seems
natural:

n
1⎨
σ̂ 2 = (X i − X )2
n i=1
 7.3 Simple Random Sampling 211

The following theorem shows that this estimate is biased.

THEOREM A
With simple random sampling,
⎥ ⎦
n−1 N
E(σ̂ 2 ) = σ 2
n N −1

Proof
Expanding the square and proceeding as in the identity for the population variance
in Section 7.2, we find
n
1⎨ 2
σ̂ 2 = X − X2
n i=1 i
Thus,
n
1 ⎨ ⎡ 2⎢
E(σ̂ 2 ) = E X i − E(X 2 )
n i=1
Now, we know that
⎡ ⎢
E X i2 = Var(X i ) + [E(X i )]2
= σ 2 + µ2
Similarly, from Theorems A and B of Section 7.3.1,
E(X 2 ) = Var(X ) + [E(X )]2
⎥ ⎦
σ2 n−1
= 1− + µ2
n N −1
Substituting these expressions for E(X i2 ) and E(X 2 ) in the preceding equation
for E(σ̂ 2 ) gives the desired result. ■

Because N > n, it follows with a little algebra that

n−1 N
<1
n N −1

so that E(σ̂ 2 ) < σ 2 ; σ̂ 2 thus tends to underestimate σ 2 . From Theorem A, we see

that an unbiased estimate of σ 2 may be obtained by multiplying σ̂⎣ 2
by the factor
2 1 1 n
n(N −1)/[(n −1)N ]. Thus, an unbiased estimate of σ is n−1 (1− N ) i=1 (X i − X )2 .
We also have the following corollary.
212 Chapter 7 Survey Sampling

COROLLARY A
An unbiased estimate of Var(X ) is
⎥ ⎦⎥ ⎦⎥ ⎦
σ̂ 2 n N −1 N −n
s X2 =
n n−1 N N −1
s2 + n,
= 1−
n N
where
n
2 1 ⎨
s = (X i − X )2
n − 1 i=1

Proof
Since
⎥ ⎦
σ2 N −n
Var(X ) =
n N −1
an unbiased estimate of Var(X ) may be obtained by substituting in an unbiased
estimate of σ 2 . Algebra then yields the desired result. ■

Similarly, an unbiased estimate of the variance of T , the estimator of the popu-

lation total, is
sT2 = N 2 s X2
For the dichotomous case, in which each X i is 0 or 1, note that
n n
1⎨ 1⎨ 2 2
(X i − X )2 = Xi − X
n i=1 n i=1
= p̂(1 − p̂)
Therefore,
n
s2 = p̂(1 − p̂)
n−1
Thus, as a special case of Corollary A, we have the following corollary.

COROLLARY B
An unbiased estimate of Var( p̂) is
p̂(1 − p̂) + n,
s 2p̂ = 1− ■
n−1 N

In many cases, the sampling fraction, n/N , is small and may be neglected. Fur-
thermore, it often makes little difference whether n − 1 or n is used as the divisor.
 7.3 Simple Random Sampling 213

The quantities s X , sT , and s p̂ are called estimated standard errors. If we knew

them, the actual standard errors, σ X , σT and σ p̂ , would be used to gauge the accuracy
of the estimates X , T and p̂. If they are not known, which is the typical case, the
estimated standard errors are used in their place.

EXAMPLE A A simple random sample of 50 of the 393 hospitals was taken. From this sample,
X = 938.5 (recall that, in fact, µ = 814.6) and s = 614.53 (σ = 590). An estimate
of the variance of X is
s2 + n,
s X2 = 1− = 6592
n N
The estimated standard error of X is
s X = 81.19
-
49
(Note that the true value is σ X = √σ50 1 − 392 = 78.) This estimated standard error
gives a rough idea of how accurate the value of X is; in this case, we see that the
magnitude of the error is of the order 80, as opposed to 8 or 800, say. In fact, the error
was 123.9, or about 1.5 s X . ■

EXAMPLE B From the same sample, the estimate of the total number of discharges in the population
of hospitals is
T = N X = 368,831
Recall that the true value of the population total is 320,139. The estimated standard
error of T is
sT = N s X = 31,908
Again, this estimated standard error can be used as a rough gauge of the estimation
error. ■

EXAMPLE C Let p be the proportion of hospitals that had fewer than 1000 discharges—that is,
p = .654. In the sample of Example A, 26 of 50 hospitals had fewer than 1000
discharges, so
26
p̂ = = .52
50
The variance of p̂ is estimated by
p̂(1 − p̂) + n,
s 2p̂ = 1− = .0045
n−1 N
Thus, the estimated standard error of p̂ is
s p̂ = .067
214 Chapter 7 Survey Sampling

Crudely, this tells us that the error of p̂ is in the second or first decimal place—that
we are probably not so fortunate as to have an error only in the third decimal place.
In fact, the error was .134 or about 2 × s p̂ . ■

These examples show how, in simple random sampling, we can not only form
estimates of unknown population parameters, but can also gauge the likely size of the
errors of the estimates, by estimating their standard errors from the data in the sample.
We have covered a lot of ground, and the presence of the finite population cor-
rection complicates the expressions we have derived. It is thus useful to summarize
our results in the following table:

Population
Parameter Estimate Variance of Estimate Estimated Variance
1
⎣n σ2
⎡ N −n ⎢ s2
⎡ n
⎢
µ X= n i=1
Xi σ X2 = n N −1
s X2 = n
1− N
p(1− p)
⎡ N −n ⎢ p̂(1− p̂)
⎡ n
⎢
p p̂ = sample proportion σ p̂2 = n N −1
s 2p̂ = n−1
1− N

τ T = NX σT2 = N 2 σ X2 sT2 = N 2 s X2
⎡ 1
⎢
σ2 1− N
s2

1
⎣n
where s 2 = n−1 2
i=1 (X i − X ) .
The square roots of the entries in the third column are called standard errors,
and the square roots of the entries in the fourth column are called estimated standard
errors. The former depend on unknown population parameters, so the latter are used
to gauge the accuracy of the parameter estimates. When the population is large relative
to the sample size, the finite population correction can be ignored, simplifying the
preceding expressions.

7.3.3 The Normal Approximation to the Sampling

Distribution of X
We have found the mean and the standard deviation of the sampling distribution of X .
Ideally, we would like to know the sampling distribution, since it would tell us every-
thing we could hope to know about the accuracy of the estimate. Without knowledge
of the population itself, however, we cannot determine the sampling distribution. In
this section, we will use the central limit theorem to deduce an approximation to
the sampling distribution—the normal, or Gaussian, distribution. This approximation
will be used to find probabilistic bounds for the estimation error.
In Section 5.3, we considered a sequence of independent and identically dis-
tributed (i.i.d.) random variables, X 1 , X 2 , . . . having the common mean and variance
µ and σ 2 . The sample mean of X 1 , X 2 , . . . , X n is
n
1⎨
Xn = Xi
n i=1
 7.3 Simple Random Sampling 215

This sample mean has the properties

E(X n ) = µ
and
σ2
Var(X n ) =
n
The central limit theorem says that, for a fixed number z,
⎥ ⎦
Xn − µ
P √ ≤ z → %(z) as n → ∞
σ/ n
where % is the cumulative distribution function of the standard normal distribution.
Using a more compact and suggestive notation, we have
⎪ ⎩
Xn − µ
P ≤ z → %(z)
σX n
The context of survey sampling is not exactly like that of the central limit theorem
as stated above—as we have seen, in sampling without replacement, the X i are not
independent of each other, and it makes no sense to have n tend to infinity while N
remains fixed. But other central limit theorems have been proved that are appropriate
to the sampling context. These show that if n is large, but still small relative to N ,
then X n , the mean of a simple random sample, is approximately normally distributed.
To demonstrate the use of the central limit theorem, we will apply it to approx-
imate P(|X − µ| ≤ δ), the probability that the error made in estimating µ by X is
less than some constant δ
P(|X − µ| ≤ δ) = P(−δ ≤ X − µ ≤ δ)
⎥ ⎦
δ X −µ δ
=P − ≤ ≤
σX σX σX
⎥ ⎦ ⎥ ⎦
δ δ
≈% −% −
σX σX
⎥ ⎦
δ
= 2% −1
σX
since %(−z) = 1 − %(z), from the symmetry of the standard normal distribution
about zero.

EXAMPLE A Let us again consider the population of 393 hospitals. The standard deviation of the
mean of a sample of size n = 64 is, using the finite population correction,
*
σ n−1
σX = √ 1−
n N −1
*
589.7 63
= 1− = 67.5
8 392
We can use the central limit theorem to approximate the probability that the
sample mean differs from the population mean by more than 100 in absolute value; i.e.,
216 Chapter 7 Survey Sampling

P(|X − µ| > 100). First, from the symmetry of the normal distribution,
P(|X − µ| > 100) ≈ 2P(X − µ > 100)
and
P(X − µ > 100) = 1 − P(X − µ < 100)
⎥ ⎦
X −µ 100
= 1− P <
σ σX
⎥ X⎦
100
≈ 1−%
67.5
= .069
Thus the probability that the sample mean differs from the population mean by more
than 100 is approximately .14. In fact, among the 500 samples of size 64 in Example
A in Section 7.3.1, 82, or 16.4%, differed by more than 100 from the population mean.
Similarly, the central limit theorem approximation gives .026 as the probability of
deviations of more than 150 from the population mean. In the simulation in Example
A in Section 7.3.1, 11 of 500, or 2.2%, differed by more than 150. If we are not too
finicky, the central limit theorem gives us reasonable and useful approximations. ■

EXAMPLE B For a sample of size 50, the standard error of the sample mean number of discharges
is
σ X = 78
For the particular sample of size 50 discussed in Example A in Section 7.3.2, we
found X = 938.35, so X − µ = 123.9. We now calculate an approximation of the
probability of an error this large or larger:
P(|X − µ| ≥ 123.9) = 1 − P(|X − µ| < 123.9)
. ⎥ ⎦ /
123.9
≈ 1 − 2% −1
78
= 2 − 2%(1.59)
= .11
Thus, we can expect an error this large or larger to occur about 11% of the time. ■

EXAMPLE C In Example C in Section 7.3.2, we found from the sample of size 50 an estimate
p̂ = .52 of the proportion of hospitals that discharged fewer than 1000 patients; in
fact, the actual proportion in the population is .65. Thus, | p̂ − p | = .13. What is the
probability that an estimate will be off by an amount this large or larger?
We have
* *
p(1 − p) n−1
σ p̂ = 1−
n N −1
= .068 × .94 = .064
 7.3 Simple Random Sampling 217

We can therefore calculate

P(| p − p̂| > .13) = 1 − P(| p − p̂| ≤ .13)
⎥ ⎦
| p − p̂| .13
= 1− P ≤
σ p̂ σ p̂
≈ 2[1 − %(2.03)] = .04
We see that the sample was rather “unlucky”—an error this large or larger would
occur only about 4% of the time. ■

We can now derive a confidence interval for the population mean, µ. A confi-
dence interval for a population parameter, θ, is a random interval, calculated from the
sample, that contains θ with some specified probability. For example, a 95% confi-
dence interval for µ is a random interval that contains µ with probability .95; if we
were to take many random samples and form a confidence interval from each one,
about 95% of these intervals would contain µ. If the coverage probability is 1 − α,
the interval is called a 100(1 − α)% confidence interval. Confidence intervals are
frequently used in conjunction with point estimates to convey information about the
uncertainty of the estimates.
For 0 ≤ α ≤ 1, let z(α) be that number such that the area under the standard
normal density function to the right of z(α) is α (Figure 7.3). Note that the symmetry
of the standard normal density function about zero implies that z(1 − α) = −z(α).
If Z follows a standard normal distribution, then, by definition of z(α),
P(−z(α/2) ≤ Z ≤ z(α/2)) = 1 − α
From the central limit theorem, (X − µ)/σ X has approximately a standard normal
distribution, so
⎥ ⎦
X −µ
P −z(α/2) ≤ ≤ z(α/2) ≈ 1 − α
σX

.4

.3
f (z)

.2

.1

!
0
!3 !2 !1 0 1 2 3
z z (!)

F I G U R E 7.3 A standard normal density showing α and z(α).

218 Chapter 7 Survey Sampling

Elementary manipulation of the inequalities gives

P(X − z(α/2)σ X ≤ µ ≤ X + z(α/2)σ X ) ≈ 1 − α
That is, the probability that µ lies in the interval X ± z(α/2)σ X is approximately
1 − α. The interval is thus called a 100(1 − α)% confidence interval. It is important
to understand that this interval is random and that the preceding equation states that
the probability that this random interval covers µ is 1 − α. In practice, α is assigned a
small value, such as .1, .05, or .01, so that the probability that the interval covers µ will
be large. Also, since the population variance is typically not known, s X is substituted
for σ X . For large samples, it can be shown that the effect of this substitution is
practically negligible. It is impossible to give a precise answer to the question “How
large is large?” As a rule of thumb, a value of n greater than 25 or 30 is usually
adequate.
To illustrate the concept of a confidence interval, 20 samples each of size n = 25
were drawn from the population of hospital discharges. From each of these 20 samples,
an approximate 95% confidence interval for µ, the mean number of discharges, was
computed. These 20 confidence intervals are displayed as vertical lines in Figure 7.4;
the dashed line in the figure is drawn at the true value, µ = 814.6. Notice that it so
1200
1000
Number of discharges
800
600
400

F I G U R E 7.4 Vertical lines are 20 approximate 95% confidence intervals for µ. The
horizontal line is the true value of µ.
 7.3 Simple Random Sampling 219

happened that all the confidence intervals included µ; since these are 95% intervals,
on the average 5%, or 1 out of 20, would not include µ.
The following example illustrates the procedure for calculating confidence
intervals.

EXAMPLE D A particular area contains 8000 condominium units. In a survey of the occupants, a
simple random sample of size 100 yields the information that the average number of
motor vehicles per unit is 1.6, with a sample standard deviation of .8. The estimated
standard error of X is thus
*
s n
sX = √ 1−
n N
*
.8 100
= 1−
10 8000
= .08
Note that the finite population correction makes almost no difference. Since z(.025) =
1.96, a 95% confidence interval for the population average is X ± 1.96s X , or (1.44,
1.76).
An estimate of the total number of motor vehicles is T = 8000 × 1.6 = 12,800.
The estimated standard error of T is

sT = N s X = 640

A 95% confidence interval for the total number of motor vehicles is T ± 1.96sT , or
(11,546, 14,054).
In the same survey, 12% of the respondents said they planned to sell their condos
within the next year; p̂ = .12 is an estimate of the population proportion p. The
estimated standard error is
* *
p̂(1 − p̂) 100
s p̂ = 1− = .03
n−1 8000
A 95% confidence interval for p is p̂ ± 1.96s p̂ , or (.06, .18).
The total number of owners planning to sell is estimated as T = N p̂ = 960. The
estimated standard error of T is sT = N s p̂ = 240. A 95% confidence interval for the
number in the population planning to sell is T ± 1.96sT , or (490, 1430). The proper
interpretation of this interval, (490, 1430), is a little subtle. We cannot state that the
probability is 0.95 and that the number of owners planning to sell is between 490 and
1430, because that number is either in this interval or not. What is true is that 95% of
intervals formed in this way will contain the true number in the long run. This interval
is like one of those shown in Figure 7.4; in the long run, 95% of those intervals will
contain the true number of discharges, but in the figure any particular interval either
does or doesn’t contain the true number. ■

The width of a confidence interval is determined by the sample size n and the
population standard deviation σ . If σ is known approximately, perhaps from earlier
220 Chapter 7 Survey Sampling

samples of the population, n can be chosen so as to obtain a confidence interval close

to some desired length. Such analysis is usually an important aspect of planning the
design of a sample survey.

EXAMPLE E The interval for the total number of owners planning to sell in Example D might be
considered too wide for practical purposes; reducing its width would require a larger
sample size. Suppose that an interval with a half-width of 200 is desired. Neglecting
the finite population correction, the half-width is
*
p̂(1 − p̂) 5095
1.96sT = 1.96N =√
n−1 n−1
Setting the last expression equal to 200 and solving for n yields n = 650 as the
necessary sample size. ■

Let us summarize: The fundamental result of this section is that the sampling
distribution of the sample mean is approximately Gaussian. This approximation can be
used to quantify the error committed in estimating the population mean by the sample
mean, thus giving us a good understanding of the accuracy of estimates produced
by a simple random sample. We next introduced the idea of a confidence interval,
a random interval that contains a population parameter with a specified probability
and thus provides an assessment of the accuracy of the corresponding estimate of that
parameter. We have seen in our examples that the width of the confidence interval is a
multiple of the estimated standard deviation of the estimate; for example, a confidence
interval for µ is X ± ks X , where the constant k depends on the coverage probability
of the interval.

7.4 Estimation of a Ratio

The foundations of the theory of survey sampling have been laid in the preceding sec-
tions on simple random sampling. This and the next section build on that foundation,
developing some advanced topics in survey sampling.
In this section, we consider the estimation of a ratio. Suppose that for each member
of a population, two values, x and y, may be measured. The ratio of interest is
N
⎣
yi
i=1 µy
r= N
=
⎣ µx
xi
i=1

Ratios arise frequently in sample surveys; for example, if households are sampled,
the following ratios might be calculated:
• If y is the number of unemployed males aged 20–30 in a household and x is the
number of males aged 20–30 in a household, then r is the proportion of unemployed
males aged 20–30.
 7.4 Estimation of a Ratio 221

• If y is weekly food expenditure and x is number of inhabitants, then r is weekly

food cost per inhabitant.
• If y is the number of motor vehicles and x is the number of inhabitants of driving
age, then r is the number of motor vehicles per inhabitant of driving age.
In a survey of farms, y might be the acres of wheat planted and x the total acreage.
In an inventory audit, y might be the audited value of an item and x the book value.
In this section, we first consider directly the problem of estimating a ratio. Later,
we will use the estimation of a ratio as a technique for estimating µ y . We will produce
a new estimate, the ratio estimate, which we will compare to the ordinary estimate, Y .
Before continuing, we note the elementary but sometimes overlooked fact that
N
1 ⎨ yi
r =
"
N i=1 xi
Suppose that a sample is drawn consisting of the pairs (X i , Yi ); the natural
estimate of r is R = Y /X . We wish to derive expressions for E(R) and Var(R), but
since R is a nonlinear function of the random variables X and Y , we cannot do this
in closed form. We will therefore employ the approximate methods of Section 4.6.
In order to calculate the approximate variance of R, we need to know Var(X ),
Var(Y ), and Cov(X , Y ). The first two quantities we know from Theorem B of Section
7.3.1. For the last quantity, we define the population covariance of x and y to be
N
1 ⎨
σx y = (xi − µx )( yi − µ y )
N i=1
It can then be shown, in a manner entirely analogous to the proof of Theorem B in
Section 7.3.1, that
⎥ ⎦
σx y n−1
Cov(X , Y ) = 1−
n N −1
From Example C in Section 4.6, we have the following theorem.

THEOREM A
With simple random sampling, the approximate variance of R = Y /X is
1 ⎡ ⎢
Var(R) ≈ 2 r 2 σ X2 + σY2 − 2r σ X Y
µx
⎥ ⎦
1 n−1 1 ⎡ 2 2 ⎢
= 1− r σx + σ y2 − 2r σx y ■
n N − 1 µx 2

The population correlation coefficient is defined as

σx y
(=
σx σ y
and is used as a measure of the strength of the linear relationship between the x and
y values in the population. It can be shown that −1 ≤ ( ≤ 1; large values of (
222 Chapter 7 Survey Sampling

indicate a strong positive relationship between x and y, and small values indicate a
strong negative relationship. (See Figure 4.7 for some illustrations of correlation.)
The equation in Theorem A can be expressed in terms of the population correlation
coefficient as follows:
⎥ ⎦
1 n−1 1 ⎡ 2 2 ⎢
Var(R) ≈ 1− r σx + σ y2 − 2r(σx σ y
n N − 1 µx 2

From this expression, we see that strong correlation of the same sign as r decreases the
variance. We also note that the variance is affected by the size of µx —if µx is small,
the variance is large, essentially because small values of X in the ratio R = Y /X
cause R to fluctuate wildly.
We now consider the approximate expectation of R. From Example C in Section
4.6 and the preceding calculations, we have the following theorem.

THEOREM B
With simple random sampling, the expectation of R is given approximately by
⎥ ⎦
1 n−1 1 ⎡ 2 ⎢
E(R) ≈ r + 1− r σ x − (σ x σ y ■
n N − 1 µ2x

From the equation in Theorem B, we see that strong correlation of the same
sign as r decreases the bias and that the bias is large if µx is small. Furthermore,
note that the bias is of the order 1/n, so its contribution to the mean squared error is
of the order 1/n 2 . In comparison, the contribution of the variance is of the order 1/n.
Therefore, for large samples, the bias is negligible compared to the standard error of
the estimate.
For large samples, truncating the Taylor series after the linear term provides a
good approximation, since the deviations X − µ X and Y − µY are likely to be small.
To this order of approximation, R is expressed as a linear combination of X and Y ,
and an argument based on the central limit theorem can be used to show that R is
approximately normally distributed. Approximate confidence intervals can thus be
formed for r by using the normal distribution.
In order to estimate the standard error of R, we substitute R for r in the formula
of Theorem A. The x and y population variances are estimated by sx2 and s y2 . The
population covariance is estimated by
n
1 ⎨
sx y = (X i − X )(Yi − Y )
n − 1 i=1
⎪ n ⎩
1 ⎨
= X i Yi − n X Y
n − 1 i=1
(as can be seen by expanding the product), and the population correlation is estimated
by
sx y
(ˆ =
sx s y
 7.4 Estimation of a Ratio 223

The estimated variance of R is thus

⎥ ⎦
1 n−1 1
s R2 = 1− (R 2 sx2 + s y2 − 2Rsx y )
n N − 1 X2
An approximate 100(1 − α)% confidence interval for r is R ± z(α/2)s R .

EXAMPLE A Suppose that 100 people who recently bought houses are surveyed, and the monthly
mortgage payment and gross income of each buyer are determined. Let y denote the
mortgage payment and x the gross income. Suppose that
X = $3100 Y = $868
s y = $250 sx = $1200
(ˆ = .85 R = .28
Neglecting the finite population correction, the estimated standard error of R is
⎥ ⎦0
1 1
sR = .282 × 12002 + 2502 − 2 × .28 × .85 × 250 × 1200
10 3100
= .006
An approximate 95% confidence interval for r is .28 ±(1.96) × (.006), or .28 ± .012.
Note that the high correlation between x and y causes the standard error of R to be
small. We can use the observed values for the variances, covariances, and means to
gauge the order of magnitude of the bias by substituting them in place of the population
parameters in the formula of Theorem B. Doing so, and again neglecting the finite
population correction, gives the value .00015 for the bias, which is negligible relative
to s R . Note that the large value of X and the large positive correlation coefficient
cause the bias to be small. ■

Ratios may also be used as tools for estimating population means and totals.
To illustrate the concept, we return to the example of hospital discharges. For this
population, the number of beds in each hospital is also known; let us denote the number
of beds in the ith hospital by xi and the number of discharges by yi . Suppose that
all the xi are known, perhaps from an earlier enumeration, before a sample has been
taken to estimate the number of discharges, and that we would like to take advantage
of this information. One way to do this is to form a ratio estimate of µ y :
µx
YR = Y = µx R
X
where X is the average number of beds and Y is the average number of discharges in
the sample. The idea is fairly simple: We expect xi and yi to be closely related in the
population, since a hospital with a large number of beds should tend to have a large
number of discharges. This is borne out by Figure 7.5, a scatterplot of the number
of discharges versus the number of beds. If X < µx , the sample underestimates the
number of beds and probably the number of discharges as well; multiplying Y by
µx /X increases Y to Y R .
224 Chapter 7 Survey Sampling

3000

2500

Discharges 2000

1500

1000

500

0
0 200 400 600 800 1000
Beds

F I G U R E 7.5 Scatterplot of the number of discharges versus the number of beds for
the 393 hospitals.

120
Count

80
40
0
500 600 700 800 900 1000 1100
Mean of simple random sample
(a)

120
Count

80
40
0
500 600 700 800 900 1000 1100
Ratio estimate
(b)

F I G U R E 7.6 (a) A histogram of the means of 500 simple random samples of size 64
from the population of discharges; (b) a histogram of the values of 500 ratio estimates
of the mean number of discharges from samples of size 64.

To see how this ratio estimate works in practice, it was simulated from 500 sam-
ples of size 64 from the population of hospitals. The histogram of the results is shown
in Figure 7.6 along with the histogram of the means of 500 simple random samples
of size 64. The comparison shows dramatically how effective the ratio estimate is at
reducing variability.
 7.4 Estimation of a Ratio 225

Two more examples will illustrate the scope of the ratio estimation method.

EXAMPLE B Suppose that we want to estimate the total number of unemployed males aged 20–30
from a sample of households and that we know τx , the total number of males aged
20–30, from census data. The ratio estimate is
Y
TR = τx
X
where Y is the average number of unemployed males aged 20–30 per household in
the sample, and X is the sample average number of males aged 20–30 per house-
hold. ■

EXAMPLE C A sample of items in an inventory is taken to estimate the total value of the inventory.
Let Yi be the audited value of the ith sample item, and let X i be its book value. We
assume that τx , the total book value of the inventory, is known, and we estimate the
total audited value by
Y
TR = τx ■
X

We will now analyze the observed success of the ratio estimate. Since Y R = µ X R,
Var(Y R ) = µ2X Var(R). From Theorem A, we thus have the following.

COROLLARY A
The approximate variance of the ratio estimate of µ y is
⎥ ⎦
1 n−1 ⎡ 2 2 ⎢
Var(Y R ) ≈ 1− r σx + σ y2 − 2r(σx σ y ■
n N −1

Similarly, from Theorem B, we have another corollary.

COROLLARY B
The approximate bias of the ratio estimate of µ y is
⎥ ⎦
1 n−1 1 ⎡ 2 ⎢
E(Y R ) − µY ≈ 1− r σx − (σx σ y ■
n N − 1 µx

When will the ratio estimate Y R be better than the ordinary estimate Y ? In the fol-
lowing, the finite population correction is neglected for simplicity. Since the variance
of the ordinary estimate Y is
σ y2
Var(Y ) =
n
226 Chapter 7 Survey Sampling

the ratio estimate has a smaller variance if

r 2 σx2 − 2r(σx σ y < 0
or (provided r > 0, for example)
2(σ y > r σx
Letting C x = σx /µx and C y = σ y /µ y , this last inequality is equivalent to
⎥ ⎦
1 Cx
(>
2 Cy
C x and C y are called coefficients of variation and give the standard deviation as a
proportion of the mean. (Coefficients of variation are often more meaningful than
standard deviations. For example, a standard deviation of 10 means one thing if the
true value of the quantity being measured is 100 and something entirely different if
the true value is 10,000.)
In order to assess the accuracy of Y R , Var(Y R ) can be estimated from the sample.

COROLLARY C
The variance of Y R can be estimated by
⎥ ⎦
1 n−1 ⎡ 2 2 ⎢
sY2 R = 1− R sx + s y2 − 2Rsx y
n N −1
and an approximate 100(1 − α)% confidence interval for µ y is (Y R ±
z( α2 )sY R ). ■

EXAMPLE D For the population of 393 hospitals, we have

µx = 274.8 σx = 213.2
µ y = 814.6 σ y = 589.7
r = 2.96 ( = .91
Thus,
1
Var(Y R ) ≈ (2.962 × 213.22 + 589.72 − 2 × 2.96 × .91 × 213.2 × 589.7)
n
68,697.4
=
n
and
262.1
σY R ≈ √
n
Including the finite population correction, the linearized approximation predicts that,
with n = 64,
*
1 63
σY R = (262.1) 1 − = 30.0
8 392
 7.5 Stratified Random Sampling 227

The actual standard deviation of the 500 sample values displayed in Figure 7.6 is
29.9, which is remarkably close. The mean of the 500 values is 816.2, compared to
the population mean of 814.6; the slight apparent bias is consistent with Corollary B.
In contrast, the standard deviation of Y from a simple random sample of size
n = 64 is
*
σ n−1
σY = √ 1−
n N −1
*
589.7 63
= 1−
8 329
= 66.3
The comparison of σY to σY R is consistent with the substantial reduction in variability
accomplished by using a ratio estimate of µ y shown in Figure 7.6.
The following is another way of interpreting this comparison. If a simple random
sample of size n 1 is taken, the variance of the estimate is Var(Y ) = 589.72 /n 1 . A
ratio estimate from a sample of size n 2 will have the same variance if
262.12 589.72
=
n2 n1
or
⎥ ⎦2
262.1
n2 = n1 = .1975n 1
589.7
Thus, in this case, we can obtain the same precision from a ratio estimate using a
sample about 80% smaller than the simple random sample. Note that this comparison
neglects the bias of the ratio estimate, which is justifiable in this case because the bias
is quite small. Here is a case in which a biased estimate performs substantially better
than an unbiased estimate, the bias being quite small and the reduction in variance
being quite large. ■

7.5 Stratified Random Sampling

7.5.1 Introduction and Notation
In stratified random sampling, the population is partitioned into subpopulations, or
strata, which are then independently sampled. The results from the strata are then
combined to estimate population parameters, such as the mean.
Following are some examples that suggest the range of situations in which strat-
ification is natural:
• In auditing financial transactions, the transactions may be grouped into strata on
the basis of their nominal values. For example, high-value, medium-value, and
low-value strata might be formed.
• In samples of human populations, geographical areas often form natural strata.
• In a study of records of shipments of household goods by motor carriers, the carriers
were grouped into three strata: large carriers, medium carriers, and small carriers.
228 Chapter 7 Survey Sampling

Stratified samples are used for a variety of reasons. We are often interested in
obtaining information about each of a number of natural subpopulations in addition
to information about the population as a whole. The subpopulations might be defined
by geographical areas or age groups. In an industrial application in which the popula-
tion consists of items produced by a manufacturing process, relevant subpopulations
might consist of items produced during different shifts or from different lots of raw
material. The use of a stratified random sample guarantees a prescribed number of
observations from each subpopulation, whereas the use of a simple random sample
can result in underrepresentation of some subpopulations. A second reason for using
stratification is that, as will be shown below, the stratified sample mean can be con-
siderably more precise than the mean of a simple random sample, especially if the
population members within each stratum are relatively homogeneous and if there is
considerable variation between strata.
In the next section, properties of the stratified sample mean are derived. Since
a simple random sample is taken within each stratum, the results will follow easily
from the derivations of earlier sections. The section after that takes up the problem
of how to allocate the total number of observations, n, among the various strata.
Comparisons will be made of the efficiencies of different allocation schemes and
also of the precisions of these allocation schemes relative to that of a simple random
sample of the same total size.

7.5.2 Properties of Stratified Estimates

Suppose there are L strata in all. Let the number of population elements in stratum
1 be denoted by N1 , the number in stratum 2 be N2 , etc. The total population size
is N = N1 + N2 + . . . + N L . The population mean and variance of the lth stratum
are denoted by µl and σl2 . The overall population mean can be expressed in terms of
the µl as follows. Let xil denote the ith population value in the lth stratum and let
Wl = Nl /N denote the fraction of the population in the lth stratum. Then
L Nl
1 ⎨⎨
µ= xil
N l=1 i=1
L
1 ⎨
= Nl µl
N l=1
L
⎨
= Wl µl
l=1

Within each stratum, a simple random sample of size n l is taken. The sample
mean in stratum l is denoted by
nl
1 ⎨
Xl = X il
n l i=1

Here X il denotes the ith sample value in the lth stratum. Note that X l is the mean of
a simple random sample from the population consisting of the lth stratum, so from
Theorem A of Section 7.3.1, E(X l ) = µl . By analogy with the preceding relationship
 7.5 Stratified Random Sampling 229

between the overall population mean and the population means of the various strata,
the obvious estimate of µ is
⎨L
Nl X l
Xs =
l=1
N
L
⎨
= Wl X l
l=1

THEOREM A
The stratified estimate, X s , of the population mean is unbiased.

Proof
L
⎨
E(X s ) = Wl E(X l )
l=1
L
1 ⎨
= Nl µl
N l=1
=µ ■

Since we assume that the samples from different strata are independent of one
another and that within each stratum a simple random sample is taken, the variance
of X s can be easily calculated.

THEOREM B
The variance of the stratified sample mean is given by
⎨ L ⎥ ⎦⎥ ⎦
1 nl − 1
Var(X s ) = Wl2 1− σl2
l=1
n l N l − 1

Proof
Since the X l are independent,
L
⎨
Var(X s ) = Wl2 Var(X l )
l=1

From Theorem B of Section 7.3.1, we have

⎥ ⎦
1 nl − 1
Var(X l ) = 1− σl2
nl Nl − 1
Therefore, the desired result follows. ■
230 Chapter 7 Survey Sampling

If the sampling fractions within all strata are small,

⎨L
Wl2 σl2
Var(X s ) ≈
l=1
nl

EXAMPLE A We again consider the population of hospitals. As we did in the discussion of ratio
estimates, we assume that the number of beds in each hospital is known but that the
number of discharges is not. We will try to make use of this knowledge by stratifying
the hospitals according to the number of beds. Let stratum A consist of the 98 smallest
hospitals, stratum B of the 98 next larger, stratum C of the 98 next larger, and stratum
D of the 99 largest. The following table shows the results of this stratification of
hospitals by size:

Stratum Nl Wl µl σl

A 98 .249 182.9 103.4

B 98 .249 526.5 204.8
C 98 .249 956.3 243.5
D 99 .251 1591.2 419.2

Suppose that we use a sample of total size n and let

n
n1 = n2 = n3 = n4 =
4
so that we have equal sample sizes in each stratum. Then, from Theorem B, neglecting
the finite population corrections and using the numerical values in the preceding table,
we have
⎨ 4
Wl2 σl2
Var(X s ) =
l=1
n1
4
4⎨ 2 2
= W σ
n l=1 l l
72, 042.6
=
n
and
268.4
σX s = √
n
The standard deviation of the mean of a simple random sample is
587.7
σX = √
n
Comparing the two standard deviations, we see that a tremendous gain in precision
has resulted from the stratification. The ratio of the variances is .20; thus a stratified
estimate based on a total sample size of n/5 is as precise as a simple random sample
of size n. The reduction in variance due to stratification is comparable to that achieved
 7.5 Stratified Random Sampling 231

by using a ratio estimate (Example D in Section 7.4). In later parts of this section, we
will look more analytically at why the stratification done here produced such dramatic
improvement. ■

Let us next consider the stratified estimate of the population total, Ts = N X s .

From Theorem B, we have the following corollary.

COROLLARY A
The expectation and variance of the stratified estimate of the population total are
E(Ts ) = τ
and
Var(Ts ) = N 2 Var(X s )
⎨L ⎥ ⎦⎥ ⎦
1 nl − 1
= Nl2 1− σl2 ■
l=1
n l N l − 1

In order to estimate the standard errors of X s and Ts , the variances of the individual
strata must be separately estimated and substituted into the preceding formulae. The
estimate of σl2 is given by
n
1 ⎨ l

sl2 = (X il − X l )2
n l − 1 i=1

Var(X s ) is estimated by
L
⎨ ⎥ ⎦⎥ ⎦
1 nl
s X2 s = Wl2 1− sl2
l=1
nl Nl

The next example illustrates how this variance estimate can be used to find
approximate confidence intervals for µ based on X s .

EXAMPLE B A sample of size 10 was drawn from each of the four strata of hospitals described in
Example A, yielding the following:

X 1 = 240.6 s12 = 6827.6

X 2 = 507.4 s22 = 23,790.7
X 3 = 865.1 s32 = 42,573.0
X 4 = 1716.5 s42 = 152,099.6
232 Chapter 7 Survey Sampling

Therefore, X s = 832.5. The variance of the stratified sample mean is estimated by

4 ⎥ ⎦
1 ⎨ 2 nl − 1
s X2 s = Wl 1 − s2
10 l=1 Nl − 1 l
= 1282.0

Thus,

s X s = 35.8

An approximate 95% confidence interval for the population mean number of dis-
charges is X s ± 1.96sx̄s , or (762.4, 902.7).
The total number of discharges is estimated by Ts = 393X s = 327,172. The
standard error of Ts is estimated by sTs = 393s X s = 14,069. An approximate
95% confidence interval for the population total is Ts ± 1.96sTs , or (299,596, 354,
748). ■

7.5.3 Methods of Allocation

In Section 7.5.2, it was shown that, neglecting the finite population correction,

⎨L
Wl2 σl2
Var(X s ) =
l=1
nl

If the resources of a survey allow only a total of n units to be sampled, the question
arises of how to choose n 1 , . . . , n L to minimize Var(X s ) subject to the constraint
n 1 + · · · + n L = n.
For the sake of simplicity, the calculations in this section ignore the finite popu-
lation correction within each stratum. The analysis may be extended to include these
corrections, but at the cost of some additional algebra. More complete results are
contained in Cochran (1977).

THEOREM A
The sample sizes n 1 , . . . , n L that minimize Var(X s ) subject to the constraint
n 1 + · · · + n L = n are given by
Wl σl
nl = n L
⎣
Wk σk
k=1

where l = 1, . . . , L .
 7.5 Stratified Random Sampling 233

Proof
We introduce a Lagrange multiplier, and we must then minimize
L
⎪ L ⎩
⎨ Wl2 σl2 ⎨
L(n 1 , . . . , n L , )) = +) nl − n
l=1
nl l=1

For l = 1, . . . , L , we have
∂L W 2σ 2
=− l2l +)
∂n l nl
Setting these partial derivatives equal to zero, we have the system of equations
Wl σl
nl = √
)
for l = 1, . . . , L. To determine ), we first sum these equations over l:
L
1 ⎨
n= √ Wl σl
) l=1
Thus,
1 n
√ = L
) ⎣
Wl σl
l=1
and
Wl σl
nl = n L
⎣
Wl σl
l=1

which proves the theorem. ■

This theorem shows that those strata for which Wl σl is large should be sampled
heavily. This makes sense intuitively. If Wl is large, the stratum contains a large
fraction of the population; if σl is large, the population values in the stratum are
quite variable, and in order to obtain a good determination of the stratum’s mean, a
relatively large sample size must be used. This optimal allocation scheme is called
Neyman allocation.
Substituting the optimal values of n l as given in Theorem A into the equation for
Var(X s ) given in Theorem B in Section 7.5.2 gives us the following corollary.

COROLLARY A
Denoting by X so , the stratified estimate using the optimal allocations as given in
Theorem A and neglecting the finite population correction,
⎥L ⎦2
⎣
Wl σl
Var(X so ) = l=1 ■
n
234 Chapter 7 Survey Sampling

⎣
EXAMPLE A For the population of hospitals, the weights for optimal allocation, Wl σl / Wl σl ,
are, from the table of Example A of Section 7.5.2,

Stratum
A B C D
Weight .106 .210 .250 .434

Note that, because of its larger standard deviation, stratum D is sampled more than
four times as heavily as stratum A. ■

The optimal allocations depend on the individual variances of the strata, which
generally will not be known. Furthermore, if a survey measures several attributes
for each population member, it is usually impossible to find an allocation that is
simultaneously optimal for all of those variables. A simple and popular alternative
method of allocation is to use the same sampling fraction in each stratum,
n1 n2 nL
= =···=
N1 N2 NL
which holds if
Nl
nl = n = nWl
N
for l = 1, . . . , L. This method is called proportional allocation. The estimate of the
population mean based on proportional allocation is
L
⎨
X sp = Wl X l
l=1
L
⎨ nl
1 ⎨
= Wl X il
l=1
n l i=1
L nl
1 ⎨⎨
= X il
n l=1 i=1

since Wl /n l = 1/n. This estimate is simply the unweighted mean of the sample
values.

THEOREM B
With stratified sampling based on proportional allocation, ignoring the finite
population correction,
L
1⎨
Var(X sp ) = Wl σl2
n l=1
 7.5 Stratified Random Sampling 235

Proof
From Theorem B of Section 7.5.2, we have
L
⎨
Var(X sp ) = Wl2 Var(X l )
l=1
L
⎨ σl2
= Wl2
l=1
nl
Using n l = nWl , the result follows. ■

We now compare Var(X sp ) and Var(X so ) in order to discover the circumstances

under which optimal allocation is substantially better than proportional allocation.

THEOREM C
With stratified random sampling, the difference between the variance of the
estimate of the population mean based on proportional allocation and the variance
of that estimate based on optimal allocation is, ignoring the finite population
correction,
L
1⎨
Var(X sp ) − Var(X so ) = Wl (σl − σ̄ )2
n l=1
where
L
⎨
σ̄ = Wl σl
l=1

Proof
 ⎪ L ⎩2 
⎨L ⎨
1
Var(X sp ) − Var(X so ) =  Wl σl2 − Wl σl 
n l=1 l=1
⎣L
The term within the large brackets equals l=1 Wl (σl − σ̄ )2 , which may be
verified by expanding the square and collecting terms. ■

According to Theorem C, if the variances of the strata are all the same, propor-
tional allocation yields the same results as optimal allocation. The more variable these
variances are, the better it is to use optimal allocation.
236 Chapter 7 Survey Sampling

EXAMPLE B Let us calculate how much better optimal allocation is than proportional allocation
for the population of hospitals. From Theorem C and Corollary A, we have
1⎨
Var(X sp ) = Var(X so ) + Wl (σl − σ̄ )2
n
Therefore,
1⎣
Var(X sp ) Wl (σl − σ̄ )2
= 1+ n
Var(X so ) Var(X so )
⎣
Wl (σl − σ̄ )2
= 1+ ⎣
( Wl σl )2
= 1 + .218
Thus, under proportional allocation, the variance of the mean is about 20% larger
than it is under optimal allocation. ■

We can also compare the variance under simple random sampling with the vari-
ance under proportional allocation. The variance under simple random sampling is,
neglecting the finite population correction,
σ2
Var(X ) =
n
In order to compare this equation with that for the variance under proportional allo-
cation, we need a relationship between the overall population variance, σ 2 , and the
strata variances, σl2 . The overall population variance may be expressed as
L Nl
1 ⎨⎨
σ2 = (xil − µ)2
N l=1 i=1
Also,
(xil − µ)2 = [(xil − µl ) + (µl − µ)]2
= (xil − µl )2 + 2(xil − µl )(µl − µ) + (µl − µ)2
When both sides of this last equation are summed
⎣ Nl over l, the middle term on the
right-hand side becomes zero since Nl µl = l=1 xil , so we have
Nl
⎨ nl
⎨
2
(xil − µ) = (xil − µl )2 + Nl (µl − µ)2
i=1 i=1
= Nl σl2 + Nl (µl − µ)2
Dividing both sides by N and summing over l, we have
L
⎨ L
⎨
σ2 = Wl σl2 + Wl (µl − µ)2
l=1 l=1
 7.5 Stratified Random Sampling 237

Substituting this expression for σ 2 into Var(X ) = σ 2 /n and using the formula for
Var(X sp ) given in Theorem B completes a proof of the following theorem.

THEOREM D
The difference between the variance of the mean of a simple random sample and
the variance of the mean of a stratified random sample based on proportional
allocation is, neglecting the finite population correction,
L
1⎨
Var(X ) − Var(X sp ) = Wl (µl − µ)2 ■
n l=1

Thus, stratified random sampling with proportional allocation always gives a

smaller variance than does simple random sampling, providing that the finite popu-
lation correction is ignored. Comparing the equations for the variances under simple
random sampling, proportional allocation, and optimal allocation, we see that strat-
ification with proportional allocation is better than simple random sampling if the
strata means are quite variable and that stratification with optimal allocation is even
better than stratification with proportional allocation if the strata standard deviations
are variable.

EXAMPLE C We calculate the improvement that would result from using stratification with propor-
tional allocation rather than simple random sampling for the population of hospitals.
From Theorems B and D, we have
⎣
Var(X sr s ) Wl (µl − µ̄)2
= 1+ ⎣
Var(X sp ) Wl σl2
= 1 + 3.83

As is frequently the case, the gain from using stratification with proportional allocation
rather than simple random sampling is much greater than the gain from using optimal
allocation rather than proportional allocation. Furthermore, proportional allocation
requires knowledge only of the sizes of the strata, whereas optimal allocation requires
knowledge of the standard deviations of the strata, and such knowledge is usually
unavailable. ■

Typically, stratified random sampling can result in substantial increases in preci-

sion for populations containing values that vary greatly in size. For example, a pop-
ulation of transactions, a sample of which is to be audited for errors, might contain
transactions in the hundreds of thousands of dollars and transactions in the hundreds
of dollars. If such a population were divided into several strata according to the dollar
amounts of the transactions, there might well be considerable variation in the mean
transaction errors between the strata, since there may be rather large errors on large
238 Chapter 7 Survey Sampling

transactions and small errors on small transactions. The variability of the errors might
also be larger in the former strata as well.
We have not addressed the question of how many strata to form and how to
define the strata. In order to construct the optimal number of strata, the population
values themselves, which are of course unknown, would have to be used. Stratification
must therefore be done on the basis of some related variable that is known (such as
transaction amount in the preceding paragraph) or on the results of earlier samples.
In practice, it usually turns out that such relationships are not strong enough to make
it worthwhile constructing more than a few strata.

7.6 Concluding Remarks

This chapter introduced survey sampling. It first covered the most elementary method
of probability sampling—simple random sampling. The theory of this method under-
lies the theory of more complex sampling techniques. Stratified sampling was also in-
troduced and shown to increase the precision of estimates substantially in many cases.
Several concepts and techniques introduced here recur throughout statistics: the
concept of a random estimate of a population parameter, such as the population mean;
bias; the standard error of an estimate; confidence intervals based on the central limit
theorem; and linearization, or propagation of error.
The theory and technique of survey sampling go far beyond the material in
this introduction. One method that deserves mention because of its widespread use
is systematic sampling. The population members are given in a list. If, say, a 10%
sample is desired, every tenth member of the list is sampled starting from some random
point among the first ten. If the list is in totally random order, this method is similar
to simple random sampling. If, however, there is some correlation or relationship
between successive members, the method is more similar to stratified sampling. The
clear danger of this method is that there may be some periodic structure in the list, in
which case bias can ensue.
Another commonly used method is cluster sampling. In sampling residential
households, a survey might choose blocks randomly and then either sample every
dwelling on each chosen block or further subsample the dwellings. Because one
would expect dwellings within a single block to be relatively homogeneous, this
method is typically less precise than a simple random sample of the same size.
We have developed a mathematical model for survey sampling and have deduced
consequences of that model, including probabilistic error bounds for the estimates.
As is always the case, reality never quite matches the mathematical model. The
basic assumptions of the model are (1) that every population member appears in
the sample with a specified probability and (2) that an exact measurement or response
is obtained from every sample member. In practice, neither assumption will hold pre-
cisely. Converse and Traugott (1986) provide an interesting discussion of the practical
difficulties of polls and surveys and consequences for the variability of the estimates.
The first assumption may fail because of the difficulty of obtaining an ex-
act enumeration of the population or because of imprecision in its definition. For
example, political surveys can be putatively based on all adults, all registered voters,
or all “likely” voters. However, the most serious problem with respect to the first
 7.7 Problems 239

assumption is that of nonresponse. Response levels of only 60% to 70% are common
in surveys of human populations. The possibility of substantial bias clearly arises if
there is a relationship of potential answers to survey questions to the propensity to
respond to those questions. For example, adults living in families are easier to contact
by a telephone survey than those living alone, and the opinions of these two groups
may well differ on certain issues. It is important to realize that the standard errors
of estimates that we have developed earlier in this chapter account only for random
variability in sample composition, not for systematic biases.
The Literary Digest poll of 1936, which predicted a 57% to 43% victory for
Republican Alfred Landon over incumbent president Franklin Roosevelt, is one of
the most famous of flawed surveys. Questionnaires were mailed to about 10 million
voters, who were selected from lists such as telephone books and club memberships,
and approximately 2.4 million of the questionnaires were returned. There were two
intrinsic problems: (1) nonresponse—those who did not respond may have voted dif-
ferently from those who did—and (2) selection bias—even if all 10 million voters
had responded, they would not have constituted a random sample; those in lower
socioeconomic classes (who were more likely to vote for Roosevelt) were less likely
to have telephone service or belong to clubs and thus less likely to be included in
the sample than were wealthier voters. The assumption that an exact measurement is
obtained from every member of the sample may also be in error. In surveys conducted
by interviewers, the interviewer’s approach and personality may affect the response.
In surveys that use questionnaires, the wording of the questions and the context within
which they are lodged can have an effect. An interesting example is a poll conducted
by Stanley Presser, (New Yorker, Oct 18, 2004). Half of the sample was asked, “Do
you think the United States should allow public speeches against democracy?” The
other half was asked, “Do you think the United States should forbid public speeches
against democracy?” 56% said no to the first question, and 39% said yes to the second.
The interesting paper by Hansen in Tanur et al. (1972) reports on efforts of the U.S.
Bureau of the Census to investigate these sorts of problems.

7.7 Problems
1. Consider a population consisting of five values—1, 2, 2, 4, and 8. Find the
population mean and variance. Calculate the sampling distribution of the mean
of a sample of size 2 by generating all possible such samples. From them, find
the mean and variance of the sampling distribution, and compare the results to
Theorems A and B in Section 7.3.1.
2. Suppose that a sample of size n = 2 is drawn from the population of the preceding
problem and that the proportion of the sample values that are greater than 3 is
recorded. Find the sampling distribution of this statistic by listing all possible
such samples. Find the mean and variance of the sampling distribution.
3. Which of the following is a random variable?
a. The population mean
b. The population size, N
240 Chapter 7 Survey Sampling

c. The sample size, n

d. The sample mean
e. The variance of the sample mean
f. The largest value in the sample
g. The population variance
h. The estimated variance of the sample mean
4. Two populations are surveyed with simple random samples. A sample of size n 1
is used for population I, which has a population standard deviation σ1 ; a sample of
size n 2 = 2n 1 is used for population II, which has a population standard deviation
σ2 = 2σ1 . Ignoring finite population corrections, in which of the two samples
would you expect the estimate of the population mean to be more accurate?
5. How would you respond to a friend who asks you, “How can we say that the
sample mean is a random variable when it is just a number, like the population
mean? For example, in Example A of Section 7.3.2, a simple random sam-
ple of size 50 produced x̄ = 938.5; how can the number 938.5 be a random
variable?”
6. Suppose that two populations have equal population variances but are of different
sizes: N1 = 100,000 and N2 = 10,000,000. Compare the variances of the sample
means for a sample of size n = 25. Is it substantially easier to estimate the mean
of the smaller population?
7. Suppose that a simple random sample is used to estimate the proportion of families
in a certain area that are living below the poverty level. If this proportion is roughly
.15, what sample size is necessary so that the standard error of the estimate is .02?
8. A sample of size n = 100 is taken from a population that has a proportion
p = 1/5.
a. Find δ such that P(| p̂ − p| ≥ δ) = 0.025.
b. If, in the sample, p̂ = 0.25, will the 95% confidence interval for p contain
the true value of p?
9. In a simple random sample of 1,500 voters, 55% said they planned to vote for a
particular proposition, and 45% said they planned to vote against it. The estimated
margin of victory for the proposition is thus 10%. What is the standard error of
this estimated margin? What is an approximate 95% confidence interval for the
margin?
10. True or false (and state why):
If a sample from a population is large, a histogram of the values in the sample
will be approximately normal, even if the population is not normal.
11. Consider a population of size four, the members of which have values x1 , x2 , x3 , x4 .
a. If simple random sampling were used, how many samples of size two are
there?
b. Suppose that rather than simple random sampling, the following sampling
scheme is used. The possible samples of size two are
{x1 , x2 }, {x2 , x3 }, {x3 , x4 }, {x1 , x4 }
 7.7 Problems 241

and the sampling is done in such a way that each of these four possible samples
is equally likely. Is the sample mean unbiased?
12. Consider simple random sampling with replacement.
a. Show that
n
1 ⎨
s2 = (X i − X )2
n − 1 i=1

is an unbiased estimate of σ 2 .
b. Is s an unbiased estimate of σ ?
c. Show that n −1 s 2 is an unbiased estimate of σ X2 .
d. Show that n −1 N 2 s 2 is an unbiased estimate of σT2 .
e. Show that p̂(1 − p̂)/(n − 1) is an unbiased estimate of σ p̂2 .
13. Suppose that the total number of discharges, τ , in Example A of Section 7.2 is
estimated from a simple random sample of size 50. Denoting the estimate by T ,
use the central limit theorem to sketch the approximate probability density of the
error T − τ .
14. The proportion of hospitals in Example A of Section 7.2 that had fewer than 1000
discharges is p = .654. Suppose that the total number of hospitals having fewer
than 1000 discharges is estimated from a simple random sample of size 25. Use
the central limit theorem to sketch the approximate sampling distribution of the
estimate.
15. Consider estimating the mean of the population of hospital discharges (Exam-
ple A of Section 7.2) from a simple random sample of size n. Use the normal
approximation to the distribution of X in answering the following:
a. Sketch P(|X − µ| > 200) as a function of n for 20 ≤ n ≤ 100.
b. For n = 20, 40, and 80, find + such that P(|X − µ| > +) ≈ .10. Similarly,
find + such that P(|X − µ| > +) ≈ .50.
16. True or false?
a. The center of a 95% confidence interval for the population mean is a random
variable.
b. A 95% confidence interval for µ contains the sample mean with probability
.95.
c. A 95% confidence interval contains 95% of the population.
d. Out of one hundred 95% confidence intervals for µ, 95 will contain µ.
17. A 90% confidence interval for the average number of children per household
based on a simple random sample is found to be (.7, 2.1). Can we conclude that
90% of households have between .7 and 2.1 children?
18. From independent surveys of two populations, 90% confidence intervals for the
population means are constructed. What is the probability that neither interval
contains the respective population mean? That both do?
19. This problem introduces the concept of a one-sided confidence interval. Using
the central limit theorem, how should the constant k be chosen so that the interval
242 Chapter 7 Survey Sampling

(−∞, X + ks X ) is a 90% confidence interval for µ—i.e., so that P(µ ≤ X +

ks X ) = .9? This is called a one-sided confidence interval. How should k be
chosen so that (X − ks X , ∞) is 95% one-sided confidence interval?
20. In Example D of Section 7.3.3, a 95% confidence interval for µ was found to be
(1.44, 1.76). Because µ is some fixed number, it either lies in this interval or it
doesn’t, so it doesn’t make any sense to claim that P(1.44 ≤ µ ≤ 1.76) = .95.
What do we mean, then, by saying this is a “95% confidence interval?”
21. In order to halve the width of a 95% confidence interval for a mean, by what factor
should the sample size be increased? Ignore the finite population correction.
22. An investigator quantifies her uncertainty about the estimate of a population mean
by reporting X ± s X . What size confidence interval is this?
23. a. Show that the standard error of an estimated proportion is largest when p =
1/2.
b. Use this result and Corollary B of Section 7.3.2 to conclude that the
quantity
5
1 N −n
2 N (n − 1)
is a conservative estimate of the standard error of p̂ no matter what the value
of p may be.
c. Use the central limit theorem to conclude that the interval
5
N −n
p̂ ±
N (n − 1)
contains p with probability at least .95.
24. For a random sample of size n from a population of size N , consider the following
as an estimate of µ:
n
⎨
Xc = ci X i
i=1

where the ci are fixed numbers and X 1 , . . . , X n is the sample.

a. Find a condition on the ci such that the estimate is unbiased.
b. Show that the choice of ci that minimizes the variances of the estimate subject
to this condition is ci = 1/n, where i = 1, . . . , n.
25. Here is an alternative proof of Lemma B in Section 7.3.1. Consider a random
permutation Y1 , Y2 , . . . , Y N of x1 , x2 , . . . , x N . Argue that the joint distribution of
any subcollection, Yi1 , . . . , Yin , of the Yi is the same as that of a simple random
sample, X 1 , . . . , X n . In particular,
Var(Yi ) = Var(X k ) = σ 2
and
Cov(Yi , Y j ) = Cov(X k , X l ) = ,
 7.7 Problems 243

if i = " l. Since Y1 + Y2 + · · · + Y N = τ ,
" j and k =
⎪ N ⎩
⎨
Var Yi = 0
i=1
⎣N
(Why?) Express Var( i=1 Yi ) in terms of σ 2 and the unknown covariance, , .
Solve for , , and conclude that
σ2
, =−
N −1
for i =
" j.
26. This is another proof of Lemma B in Section 7.3.1. Let Ui be a random vari-
able with Ui = 1 if the ith population member is in the sample and equal to 0
otherwise.
⎣N
a. Show that the sample mean X = n −1 i=1 Ui xi .
b. Show that P(Ui = 1) = n/N . Find E(Ui ), using the fact that Ui is a Bernoulli
random variable.
c. What is the variance of the Bernoulli random variable Ui ?
d. Noting that Ui U j is a Bernoulli random variable, find E(Ui U j ), i =
" j. (Be
careful to take into account that the sample is drawn without replacement.)
e. Find Cov(Ui , U j ), i =
" j.
f. Using the representation of X above, find Var(X ).
27. Suppose that the population size N is not known, but it is known that n ≤ N .
Show that the following procedure will generate a simple random sample of
size n. Imagine that the population is arranged in a long list that you can read
sequentially.
a. Let the sample initially consist of the the first n elements in the list.
b. For k = 1, 2, . . . , as long as the end of the list has not been encountered:
i. Read the (n + k)-th element in the list.
ii. Place it in the sample with probability n/(n + k) and, if it is placed in the
sample, randomly drop one of the exisiting sample members.
28. In surveys, it is difficult to obtain accurate answers to sensitive questions such as
“Have you ever used heroin?” or “Have you ever cheated on an exam?” Warner
(1965) introduced the method of randomized response to deal with such sit-
uations. A respondent spins an arrow on a wheel or draws a ball from an urn
containing balls of two colors to determine which of two statements to respond
to: (1) “I have characteristic A,” or (2) “I do not have characteristic A.” The inter-
viewer does not know which statement is being responded to but merely records
a yes or a no. The hope is that an interviewee is more likely to answer truthfully
if he or she realizes that the interviewer does not know which statement is being
responded to. Let R be the proportion of a sample answering Yes. Let p be the
probability that statement 1 is responded to ( p is known from the structure of
the randomizing device), and let q be the proportion of the population that has
characteristic A. Let r be the probability that a respondent answers Yes.
a. Show that r = (2 p−1)q +(1− p). [Hint: P(yes) = P(yes given question 1) ×
P(question 1) + P(yes given question 2) × P(question 2).]
244 Chapter 7 Survey Sampling

b. If r were known, how could q be determined?

c. Show that E(R) = r , and propose an estimate, Q, for q. Show that the estimate
is unbiased.
d. Ignoring the finite population correction, show that

r (1 − r )
Var(R) =
n
where n is the sample size.
e. Find an expression for Var(Q).
29. A variation of the method described in Problem 28 has been proposed. Instead
of responding to statement 2, the respondent answers an unrelated question for
which the probability of a “yes” response is known, for example, “Were you born
in June?”
a. Propose an estimate of q for this method.
b. Show that the estimate is unbiased.
c. Obtain an expression for the variance of the estimate.
30. Compare the accuracies of the methods of Problems 28 and 29 by comparing their
standard deviations. You may do this by substituting some plausible numerical
values for p and q.
31. Referring to Example D in Section 7.3.3, how large should the sample be in order
that the 95% confidence interval for the total number of owners planning to sell
will have a width of 500?
32. Referring again to Example D in Section 7.3.3, suppose that a survey is done of
another condominium project of 12,000 units. The sample size is 200, and the
proportion planning to sell in this sample is .18.
a. What is the standard error of this estimate? Give a 90% confidence interval.
b. Suppose we use the notation p̂1 = .12 and p̂2 = .18 to refer to the proportions
in the two samples. Let dˆ = p̂1 − p̂2 be an estimate of the difference, d, of
the two population proportions p1 and p2 . Using the fact that p̂1 and p̂2 are
independent random variables, find expressions for the variance and standard
ˆ
error of d.
c. Because p̂1 and p̂2 are approximately normally distributed, so is d. ˆ Use this
fact to construct 99%, 95%, and 90% confidence intervals for d. Is there clear
evidence that p1 is really different from p2 ?
33. Two populations are independently surveyed using simple random samples of
size n, and two proportions, p1 and p2 , are estimated. It is expected that both
population proportions are close to .5. What should the sample size be so that the
standard error of the difference, p̂1 − p̂2 , will be less than .02?
34. In a survey of a very large population, the incidences of two health problems are
to be estimated from the same sample. It is expected that the first problem will
affect about 3% of the population and the second about 40%. Ignore the finite
population correction in answering the following questions.
 7.7 Problems 245

a. How large should the sample be in order for the standard errors of both esti-
mates to be less than .01? What are the actual standard errors for this sample
size?
b. Suppose that instead of imposing the same limit on both standard errors, the
investigator wants the standard error to be less than 10% of the true value in
each case. What should the sample size be?
35. A simple random sample of a population of size 2000 yields the following
25 values:
104 109 111 109 87
86 80 119 88 122
91 103 99 108 96
104 98 98 83 107
79 87 94 92 97

a. Calculate an unbiased estimate of the population mean.

b. Calculate unbiased estimates of the population variance and Var(X ).
c. Give approximate 95% confidence intervals for the population mean and total.
2
36. With simple random sampling, is X an unbiased estimate of µ2 ? If not, what is
the bias?
37. Two surveys were independently conducted to estimate a population mean, µ.
Denote the estimates and their standard errors by X 1 and X 2 and σ X 1 and σ X 2 .
Assume that X 1 and X 2 are unbiased. For some α and β, the two estimates can
be combined to give a better estimator:

X = αX1 + β X2

a. Find the conditions on α and β that make the combined estimate unbiased.
b. What choice of α and β minimizes the variances, subject to the condition of
unbiasedness?
n
1⎨ 3
38. Let X 1 , . . . , X n be a simple random sample. Show that X is an unbiased
N n i=1 i
1 ⎨
estimate of x 3.
N i=1 i
39. Suppose that of a population of N items, k are defective in some way. For exam-
ple, the items might be documents, a small proportion of which are fraudulent.
How large should a sample be so that with a specified probability it will contain
at least one of the defective items? For example, if N = 10,000, k = 50, and
p = .95, what should the sample size be? Such calculations are useful in planning
sample sizes for acceptance sampling.
40. This problem presents an algorithm for drawing a simple random sample from a
population in a sequential manner. The members of the population are considered
for inclusion in the sample one at a time in some prespecified order (for example,
the order in which they are listed). The ith member of the population is included
246 Chapter 7 Survey Sampling

in the sample with probability

n − ni
N −i +1

where n i is the number of population members already in the sample before the
ith member is examined. Show that the sample selected in this way is in fact
a simple random sample; that is, show that every possible sample occurs with
probability
1
⎥ ⎦
N
n
41. In accounting and auditing, the following sampling method is sometimes used to
estimate a population total. In estimating the value of an inventory, suppose that
a book value exists for each item and is readily accessible. For each item in the
sample, the difference D, audited value minus book value, is determined. The
inventory value is estimated by the sum of the book values of the population and
N D, where N is the population size.
a. Show that the estimate is unbiased.
b. Find an expression for the variance of the estimate.
c. Compare the expression obtained in part (b) to the variance of the usual es-
timate, which is the product of N and the average audited value. Under what
circumstances would the proposed method be more accurate?
d. How could a ratio estimate be employed in this situation? Would there be any
advantage or disadvantage to using a ratio estimate rather than the proposed
method?
42. Show that the population correlation coefficient is less than or equal to 1 in
absolute value.
43. Suppose that for Example D in Section 7.3.3, the average number of occupants
per condominium unit in the sample is 2.2 with a sample standard deviation of
.7 and the sample correlation coefficient between the number of occupants and
the number of motor vehicles is .85. Estimate the population ratio of the number
of motor vehicles per occupant and its standard error. Find an approximate 95%
confidence interval for the estimate.

44. Show that

⎥ ⎦
Var(Y R ) Cx Cx
≈1+ − 2(
Var(Y ) Cy Cy

Sketch the graph of this ratio as a function of C x /C y .

45. In the population of hospitals, the correlation of the number of beds and the num-
ber of discharges is ( = .91 (Example D of Section 7.4). To see how Var(Y R )
would be different if the correlation were different, plot Var(Y R ) for n = 64 as
a function of ( for −1 < ( < 1.
 7.7 Problems 247

46. Use the central limit theorem to sketch the approximate sampling distribution
of Y R for n = 64 for the population of hospitals. Compare to the approximate
sampling distribution of Y .
47. For the population of hospitals and a sample size of n = 64, find the approxi-
mate bias of Y R by applying Corollary B of Section 7.4 and compare it to the
approximate standard deviation of the estimate. Repeat for n = 128.
48. A simple random sample of 100 households located in a city recorded the number
of people living in the household, X , and the weekly expenditure for food, Y . It
is known that there are 100,000 households in the city. In the sample
⎨
X i = 320
⎨
Yi = 10,000
⎨
X i2 = 1250
⎨
Yi2 = 1,100,000
⎨
X i Yi = 36,000
Neglect the finite population correction in answering the following.
a. Estimate the ratio r = µ y /µx .
b. Form an approximate 95% confidence interval for µ y /µx .
c. Using only the data on Y estimate the total weekly food expenditure, τ , for
households in the city and form a 90% confidence interval.
49. In a wildlife survey, an area of desert land was divided into 1000 squares, or
“quadrats,” a simple random sample of 50 of which were surveyed. In each sur-
veyed quadrat, the number of birds, Y , and the area covered by vegetation, X ,
were determined. It was found that
⎨
X i = 3000
⎨
Yi = 150
⎨
X i2 = 225,000
⎨
Yi2 = 650
⎨
X i Yi = 11,000
a. Estimate the ratio of the average number of birds per quadrat to the average
vegetation cover per quadrat.
b. Estimate the standard error of your estimate and find an approximate 90%
confidence interval for the population average.
c. Estimate the total number of birds and find an approximate 95% confidence
interval for the population total.
d. Suppose that from an aerial survey, the total area covered by vegetation could
easily be determined. How could this information be used to provide another
248 Chapter 7 Survey Sampling

estimate of the number of birds? Would you expect this estimate to be better
than or worse than that found in part (c)?
50. Hartley and Ross (1954) derived the following exact bound on the relative size
of the bias and standard error of a ratio estimate:
5 ⎥ ⎦
|E(R) − r | σX σx 1 n−1
≤ = 1−
σR µx µx n N −1

a. Derive this bound from the relation

⎥ ⎦ ⎥ ⎦
Y Y
Cov(R, X ) = E X −E E(X )
X X

b. Apply the bound to Problem 43 using sample estimates in place of the given
population parameters.
51. This problem introduces a technique called the “jackknife,” originally proposed
by Quenouille (1956) for reducing bias. Many nonlinear estimates, including the
ratio estimator, have the property that

b1 b2
E(θ̂) = θ + + 2 +···
n n

where θ̂ is an estimate of θ . The jackknife forms an estimate θ̂ J , which has a

leading bias term of the order n −2 rather than n −1 . Thus, for sufficiently large
n, the bias of θ̂ J is substantially smaller than that of θ̂. The technique involves
splitting the sample into several subsamples, computing the estimate for each
subsample, and then combining the several estimates. The sample is split into p
groups of size m, where n = mp. For j = 1, . . . , p, the estimate θ̂ j is calculated
from the m( p − 1) observations left after the jth group has been deleted. From
the preceding expression,

b1 b2
E(θ̂ j ) = θ + + +···
m( p − 1) [m( p − 1)]2

Now, p “pseudovalues” are defined:

V j = p θ̂ − ( p − 1)θ̂ j

The jackknife estimate, θ̂ J , is defined as the average of the pseudovalues:

p
1⎨
θ̂ J = Vj
p j=1

Show that the bias of θ̂ J is of the order n −2 .

52. A population consists of three strata with N1 = N2 = 1000 and N3 = 500.
A stratified random sample with 10 observations in each stratum yields the
 7.7 Problems 249

following data:

Stratum 1 94 99 106 106 101 102 122 104 97 97

Stratum 2 183 183 179 211 178 179 192 192 201 177
Stratum 3 343 302 286 317 289 284 357 288 314 276

Estimate the population mean and total and give a 90% confidence interval.
53. The following table (Cochran 1977) shows the stratification of all farms in a
county by farm size and the mean and standard deviation of the number of acres
of corn in each stratum.

Farm Size Nl µl σl

0–40 394 5.4 8.3

41–80 461 16.3 13.3
81–120 391 24.3 15.1
121–160 334 34.5 19.8
161–200 169 42.1 24.5
201–240 113 50.1 26.0
241 + 148 63.8 35.2

a. For a sample size of 100 farms, compute the sample sizes from each stratum
for proportional and optimal allocation, and compare them.
b. Calculate the variances of the sample mean for each allocation and compare
them to each other and to the variance of an estimate formed from simple
random sampling.
c. What are the population mean and variance?
d. Suppose that ten farms are sampled per stratum. What is Var(X s )? How large
a simple random sample would have to be taken to attain the same variance?
Ignore the finite population correction.
e. Repeat part (d) using proportional allocation of the 70 samples.
54. a. Suppose that the cost of a survey is C = C0 + C1 n, where C0 is a startup
cost and C1 is the cost per observation. For a given cost C, find the al-
location n 1 , . . . , n L to L strata that is optimal in the sense that it mini-
mizes the variance of the estimate of the population mean subject to the cost
constraint.
b. Suppose that the cost of an observation varies from stratum to stratum—in
some strata the observations might be relatively cheap and in others relatively
expensive. The cost of a survey with an allocation n 1 , . . . , n L is
L
⎨
C = C0 + Cl n l
l=1

For a fixed total cost C, what choice of n 1 , · · ·, n L minimizes the variance?

c. Assuming that the cost function is as given in part (b), for a fixed variance,
find n l to minimize cost.
250 Chapter 7 Survey Sampling

55. The designer of a sample survey stratifies a population into two strata, H and L.
H contains 100,000 people, and L contains 500,000. He decides to allocate 100
samples to stratum H and 200 to stratum L, taking a simple random sample in
each stratum.
a. How should the designer estimate the population mean?
b. Suppose that the population standard deviation in stratum H is 20 and the
standard deviation in stratum L is 10. What will be the standard error of his
estimate?
c. Would it be better to allocate 200 samples to stratum H and 100 to stratum L?
d. Would it be better to use proportional allocation?
56. How might stratification be used in each of the following sampling problems?
a. A survey of household expenditures in a city.
b. A survey to examine the lead concentration in the soil in a large plot of land.
c. A survey to estimate the number of people who use elevators in a large building
with a single bank of elevators.
d. A survey of programs on a television station, taken to estimate the proportion
of time taken up by advertising on Monday through Friday from 6 P.M. until
10 P.M. Assume that 52 weeks of recorded broadcasts are available for analysis.
57. Consider stratifying the population of Problem 1 into two strata: (1, 2, 2) and (4,
8). Assuming that one observation is taken from each stratum, find the sampling
distribution of the estimate of the population mean and the mean and standard
deviation of the sampling distribution. Compare to Theorems A and B in Section
7.5.2 and the results of Problem 1.
58. (Computer Exercise) Construct a population consisting of the integers from 1 to
100. Simulate the sampling distribution of the sample mean of a sample of size
12 by drawing 100 samples of size 12 and making a histogram of the results.
59. (Computer Exercise) Continuing with Problem 58, divide the population into
two strata of equal size, allocate six observations per stratum, and simulate
the distribution of the stratified estimate of the population mean. Do the same
thing with four strata. Compare the results to each other and to the results of
Problem 58.
60. A population consists of two strata, H and L, of sizes 100,000 and 500,000 and
standard deviations 20 and 12, respectively. A stratified sample of size 100 is to
be taken.
a. Find the optimal allocation for estimating the population mean.
b. Find the optimal allocation for estimating the difference of the means of the
strata, µ H − µ L .
61. The value of a population mean increases linearly through time: µ(t) = α + βt
while the variance remains constant. Independent simple random samples of size
n are taken at times t = 1, 2, and 3.
a. Find conditions on w1 , w2 , and w3 such that

β̂ = w1 X 1 + w2 X 2 + w3 X 3
 7.7 Problems 251

is an unbiased estimate of the rate of change, β. Here X i denotes the sample

mean at time ti .
b. What values of the wi minimize the variance subject to the constraint that the
estimate is unbiased?
62. In Example B of Section 7.5.2, the standard error of X s was estimated to be
s X s = 35.8. How good is this estimate—what is the actual standard error of X s ?
63. (Open-ended) Monte Carlo evaluation of an integral was introduced in Example
A of Section 5.2. Refer to that example for the following notation. Try to interpret
that method from the point of view of survey sampling by considering an “infinite
population” of numbers in the interval [0, 1], each population member x having
a value f (x). Interpret Î ( f ) as the mean of a simple random sample. What is
the standard error of Î ( f )? How could it be estimated? How could a confidence
interval for I ( f ) be formed? Do you think that anything could be gained by
stratifying the “population?” For example, the strata could be the intervals [0, .5)
and [.5, 1]. You might find it helpful to consider some examples.
64. The value of an inventory is to be estimated by sampling. The items are stratified
by book value in the following way:
Stratum Nl µl σl

$1000 + 70 3000 1250

$200–1000 500 500 100
$1–200 10,000 90 30

a. What should the relative sampling fraction in each stratum be for proportional
and for optimal allocation? Ignore the finite population correction.
b. How do the variances under each type of allocation compare to each other and
to the variance under simple random sampling?
65. The disk file cancer contains values for breast cancer mortality from 1950 to
1960 (y) and the adult white female population in 1960 (x) for 301 counties in
North Carolina, South Carolina, and Georgia.
a. Make a histogram of the population values for cancer mortality.
b. What are the population mean and total cancer mortality? What are the pop-
ulation variance and standard deviation?
c. Simulate the sampling distribution of the mean of a sample of 25 observations
of cancer mortality.
d. Draw a simple random sample of size 25 and use it to estimate the mean and
total cancer mortality.
e. Estimate the population variance and standard deviation from the sample of
part (d).
f. Form 95% confidence intervals for the population mean and total from the
sample of part (d). Do the intervals cover the population values?
g. Repeat parts (d) through (f) for a sample of size 100.
h. Suppose that the size of the total population of each county is known and that
this information is used to improve the cancer mortality estimates by forming
a ratio estimator. Do you think this will be effective? Why or why not?
252 Chapter 7 Survey Sampling

i. Simulate the sampling distribution of ratio estimators of mean cancer mortal-

ity based on a simple random sample of size 25. Compare this result to that
of part (c).
j. Draw a simple random sample of size 25 and estimate the population mean and
total cancer mortality by calculating ratio estimates. How do these estimates
compare to those formed in the usual way in part (d) from the same data?
k. Form confidence intervals about the estimates obtained in part ( j).
l. Stratify the counties into four strata by population size. Randomly sample six
observations from each stratum and form estimates of the population mean
and total mortality.
m. Stratify the counties into four strata by population size. What are the sam-
pling fractions for proportional allocation and optimal allocation? Compare
the variances of the estimates of the population mean obtained using simple
random sampling, proportional allocation, and optimal allocation.
n. How much better than those in part (m) will the estimates of the population
mean be if 8, 16, 32, or 64 strata are used instead?
66. A photograph of a large crowd on a beach is taken from a helicopter. The photo
is of such high resolution that when sections are magnified, individual people
can be identified, but to count the entire crowd in this way would be very time-
consuming. Devise a plan to estimate the number of people on the beach by using
a sampling procedure.
67. The data set families contains information about 43,886 families living in
the city of Cyberville. The city has four regions: the Northern region has 10,149
families, the Eastern region has 10,390 families, the Southern region has 13,457
families, and the Western region has 9,890. For each family, the following infor-
mation is recorded:
1. Family type
1: Husband-wife family
2: Male-head family
3: Female-head family
2. Number of persons in family
3. Number of children in family
4. Family income
5. Region
1: North
2: East
3: South
4: West
6. Education level of head of household
31: Less than 1st grade
32: 1st, 2nd, 3rd, or 4th grade
33: 5th or 6th grade
34: 7th or 8th grade
35: 9th grade
36: 10th grade
37: 11th grade
 7.7 Problems 253

38: 12th grade, no diploma

39: High school graduate, high school diploma, or equivalent
40: Some college but no degree
41: Associate degree in college (occupation/vocation program)
42: Associate degree in college (academic program)
43: Bachelor’s degree (e.g., B.S., B.A., A.B.)
44: Master’s degree (e.g., M.S., M.A., M.B.A.)
45: Professional school degree (e.g., M.D., D.D.S., D.V.M., LL.B., J.D.)
46: Doctoral degree (e.g., Ph.D., Ed.D.)
In these exercises, you will try to learn about the families of Cyberville by using
sampling.
a. Take a simple random sample of 500 families. Estimate the following popula-
tion parameters, calculate the estimated standard errors of these estimates, and
form 95% confidence intervals:
i. The proportion of female-headed families
ii. The average number of children per family
iii. The proportion of heads of households who did not receive a high school
diploma
iv. The average family income
Repeat the preceding parameters for five different simple random samples of
size 500 and compare the results.
b. Take 100 samples of size 400.
i. For each sample, find the average family income.
ii. Find the average and standard deviation of these 100 estimates and make
a histogram of the estimates.
iii. Superimpose a plot of a normal density with that mean and standard devi-
ation of the histogram and comment on how well it appears to fit.
iv. Plot the empirical cumulative distribution function (see Section 10.2). On
this plot, superimpose the normal cumulative distribution function with
mean and standard deviation as earlier. Comment on the fit.
v. Another method for examining a normal approximation is via a normal
probability plot (Section 9.9). Make such a plot and comment on what it
shows about the approximation.
vi. For each of the 100 samples, find a 95% confidence interval for the pop-
ulation average income. How many of those intervals actually contain the
population target?
vii. Take 100 samples of size 100. Compare the averages, standard deviations,
and histograms to those obtained for a sample of size 400 and explain how
the theory of simple random sampling relates to the comparisons.
c. For a simple random sample of 500, compare the incomes of the three family
types by comparing histograms and boxplots (see Chapter 10.6).
d. Take simple random samples of size 400 from each of the four regions.
i. Compare the incomes by region by making parallel boxplots.
ii. Does it appear that some regions have larger families than others?
iii. Are there differences in education level among the four regions?
254 Chapter 7 Survey Sampling

e. Formulate a question of your choice and attempt to answer it with a simple

random sample of size 400.
f. Does stratification help in estimating the average family income? From a simple
random sample of size 400, estimate the average income and also the standard
error of your estimate. Form a 95% confidence interval. Next, allocate the 400
observations proportionally to the four regions and estimate the average income
from the stratified sample. Estimate the standard error and form a 95% confi-
dence interval. Compare your results to the results of the simple random sample.
Chapter 2

Thinking Like a Researcher

Conducting good research requires first retraining your brain to think like a researcher.
This requires visualizing the abstract from actual observations, mentally “connecting the dots”
to identify hidden concepts and patterns, and synthesizing those patterns into generalizable
laws and theories that apply to other contexts beyond the domain of the initial observations.
Research involves constantly moving back and forth from an empirical plane where
observations are conducted to a theoretical plane where these observations are abstracted into
generalizable laws and theories. This is a skill that takes many years to develop, is not
something that is taught in graduate or doctoral programs or acquired in industry training, and
is by far the biggest deficit amongst Ph.D. students. Some of the mental abstractions needed to
think like a researcher include unit of analysis, constructs, hypotheses, operationalization,
theories, models, induction, deduction, and so forth, which we will examine in this chapter.

Unit of Analysis
One of the first decisions in any social science research is the unit of analysis of a
scientific study. The unit of analysis refers to the person, collective, or object that is the target
of the investigation. Typical unit of analysis include individuals, groups, organizations,
countries, technologies, objects, and such. For instance, if we are interested in studying people’s
shopping behavior, their learning outcomes, or their attitudes to new technologies, then the
unit of analysis is the individual. If we want to study characteristics of street gangs or teamwork
in organizations, then the unit of analysis is the group. If the goal of research is to understand
how firms can improve profitability or make good executive decisions, then the unit of analysis
is the firm. In this case, even though decisions are made by individuals in these firms, these
individuals are presumed to represent their firm’s decision rather than their personal decisions.
If research is directed at understanding differences in national cultures, then the unit of analysis
becomes a country. Even inanimate objects can serve as units of analysis. For instance, if a
researcher is interested in understanding how to make web pages more attractive to its users,
then the unit of analysis is a web page (and not users). If we wish to study how knowledge
transfer occurs between two firms, then our unit of analysis becomes the dyad (the combination
of firms that is sending and receiving knowledge).

Understanding the units of analysis can sometimes be fairly complex. For instance, if we
wish to study why certain neighborhoods have high crime rates, then our unit of analysis
becomes the neighborhood, and not crimes or criminals committing such crimes. This is
because the object of our inquiry is the neighborhood and not criminals. However, if we wish to
compare different types of crimes in different neighborhoods, such as homicide, robbery,

9
10 | S o c i a l S c i e n c e R e s e a r c h

assault, and so forth, our unit of analysis becomes the crime. If we wish to study why criminals
engage in illegal activities, then the unit of analysis becomes the individual (i.e., the criminal).
Like, if we want to study why some innovations are more successful than others, then our unit
of analysis is an innovation. However, if we wish to study how some organizations innovate
more consistently than others, then the unit of analysis is the organization. Hence, two related
research questions within the same research study may have two entirely different units of
analysis.

Understanding the unit of analysis is important because it shapes what type of data you
should collect for your study and who you collect it from. If your unit of analysis is a web page,
you should be collecting data about web pages from actual web pages, and not surveying people
about how they use web pages. If your unit of analysis is the organization, then you should be
measuring organizational-level variables such as organizational size, revenues, hierarchy, or
absorptive capacity. This data may come from a variety of sources such as financial records or
surveys of Chief Executive Officers (CEO), who are presumed to be representing their
organization (rather than themselves). Some variables such as CEO pay may seem like
individual level variables, but in fact, it can also be an organizational level variable because each
organization has only one CEO pay at any time. Sometimes, it is possible to collect data from a
lower level of analysis and aggregate that data to a higher level of analysis. For instance, in
order to study teamwork in organizations, you can survey individual team members in different
organizational teams, and average their individual scores to create a composite team-level
score for team-level variables like cohesion and conflict. We will examine the notion of
“variables” in greater depth in the next section.

Concepts, Constructs, and Variables

We discussed in Chapter 1 that although research can be exploratory, descriptive, or
explanatory, most scientific research tend to be of the explanatory type in that they search for
potential explanations of observed natural or social phenomena. Explanations require
development of concepts or generalizable properties or characteristics associated with objects,
events, or people. While objects such as a person, a firm, or a car are not concepts, their specific
characteristics or behavior such as a person’s attitude toward immigrants, a firm’s capacity for
innovation, and a car’s weight can be viewed as concepts.

Knowingly or unknowingly, we use different kinds of concepts in our everyday

conversations. Some of these concepts have been developed over time through our shared
language. Sometimes, we borrow concepts from other disciplines or languages to explain a
phenomenon of interest. For instance, the idea of gravitation borrowed from physics can be
used in business to describe why people tend to “gravitate” to their preferred shopping
destinations. Likewise, the concept of distance can be used to explain the degree of social
separation between two otherwise collocated individuals. Sometimes, we create our own
concepts to describe a unique characteristic not described in prior research. For instance,
technostress is a new concept referring to the mental stress one may face when asked to learn a
new technology.

Concepts may also have progressive levels of abstraction. Some concepts such as a
person’s weight are precise and objective, while other concepts such as a person’s personality
may be more abstract and difficult to visualize. A construct is an abstract concept that is
specifically chosen (or “created”) to explain a given phenomenon. A construct may be a simple
concept, such as a person’s weight, or a combination of a set of related concepts such as a
 T h i n k i n g L i k e a R e s e a r c h e r | 11

person’s communication skill, which may consist of several underlying concepts such as the
person’s vocabulary, syntax, and spelling. The former instance (weight) is a unidimensional
construct, while the latter (communication skill) is a multi-dimensional construct (i.e., it
consists of multiple underlying concepts). The distinction between constructs and concepts are
clearer in multi-dimensional constructs, where the higher order abstraction is called a construct
and the lower order abstractions are called concepts. However, this distinction tends to blur in
the case of unidimensional constructs.

Constructs used for scientific research must have precise and clear definitions that
others can use to understand exactly what it means and what it does not mean. For instance, a
seemingly simple construct such as income may refer to monthly or annual income, before-tax
or after-tax income, and personal or family income, and is therefore neither precise nor clear.
There are two types of definitions: dictionary definitions and operational definitions. In the
more familiar dictionary definition, a construct is often defined in terms of a synonym. For
instance, attitude may be defined as a disposition, a feeling, or an affect, and affect in turn is
defined as an attitude. Such definitions of a circular nature are not particularly useful in
scientific research for elaborating the meaning and content of that construct. Scientific research
requires operational definitions that define constructs in terms of how they will be
empirically measured. For instance, the operational definition of a construct such as
temperature must specify whether we plan to measure temperature in Celsius, Fahrenheit, or
Kelvin scale. A construct such as income should be defined in terms of whether we are
interested in monthly or annual income, before-tax or after-tax income, and personal or family
income. One can imagine that constructs such as learning, personality, and intelligence can be
quite hard to define operationally.

Figure 2.1. The theoretical and empirical planes of research

A term frequently associated with, and sometimes used interchangeably with, a

construct is a variable. Etymologically speaking, a variable is a quantity that can vary (e.g., from
low to high, negative to positive, etc.), in contrast to constants that do not vary (i.e., remain
constant). However, in scientific research, a variable is a measurable representation of an
abstract construct. As abstract entities, constructs are not directly measurable, and hence, we
look for proxy measures called variables. For instance, a person’s intelligence is often measured
as his or her IQ (intelligence quotient) score, which is an index generated from an analytical and
pattern-matching test administered to people. In this case, intelligence is a construct, and IQ
score is a variable that measures the intelligence construct. Whether IQ scores truly measures
one’s intelligence is anyone’s guess (though many believe that they do), and depending on
12 | S o c i a l S c i e n c e R e s e a r c h

whether how well it measures intelligence, the IQ score may be a good or a poor measure of the
intelligence construct. As shown in Figure 2.1, scientific research proceeds along two planes: a
theoretical plane and an empirical plane. Constructs are conceptualized at the theoretical
(abstract) plane, while variables are operationalized and measured at the empirical
(observational) plane. Thinking like a researcher implies the ability to move back and forth
between these two planes.

Depending on their intended use, variables may be classified as independent,

dependent, moderating, mediating, or control variables. Variables that explain other variables
are called independent variables, those that are explained by other variables are dependent
variables, those that are explained by independent variables while also explaining dependent
variables are mediating variables (or intermediate variables), and those that influence the
relationship between independent and dependent variables are called moderating variables.
As an example, if we state that higher intelligence causes improved learning among students,
then intelligence is an independent variable and learning is a dependent variable. There may be
other extraneous variables that are not pertinent to explaining a given dependent variable, but
may have some impact on the dependent variable. These variables must be controlled for in a
scientific study, and are therefore called control variables.

Figure 2.2. A nomological network of constructs

To understand the differences between these different variable types, consider the
example shown in Figure 2.2. If we believe that intelligence influences (or explains) students’
academic achievement, then a measure of intelligence such as an IQ score is an independent
variable, while a measure of academic success such as grade point average is a dependent
variable. If we believe that the effect of intelligence on academic achievement also depends on
the effort invested by the student in the learning process (i.e., between two equally intelligent
students, the student who puts is more effort achieves higher academic achievement than one
who puts in less effort), then effort becomes a moderating variable. Incidentally, one may also
view effort as an independent variable and intelligence as a moderating variable. If academic
achievement is viewed as an intermediate step to higher earning potential, then earning
potential becomes the dependent variable for the independent variable academic achievement,
and academic achievement becomes the mediating variable in the relationship between
intelligence and earning potential. Hence, variable are defined as an independent, dependent,
moderating, or mediating variable based on their nature of association with each other. The
overall network of relationships between a set of related constructs is called a nomological
network (see Figure 2.2). Thinking like a researcher requires not only being able to abstract
constructs from observations, but also being able to mentally visualize a nomological network
linking these abstract constructs.
 T h i n k i n g L i k e a R e s e a r c h e r | 13

Propositions and Hypotheses

Figure 2.2 shows how theoretical constructs such as intelligence, effort, academic
achievement, and earning potential are related to each other in a nomological network. Each of
these relationships is called a proposition. In seeking explanations to a given phenomenon or
behavior, it is not adequate just to identify key concepts and constructs underlying the target
phenomenon or behavior. We must also identify and state patterns of relationships between
these constructs. Such patterns of relationships are called propositions. A proposition is a
tentative and conjectural relationship between constructs that is stated in a declarative form.
An example of a proposition is: “An increase in student intelligence causes an increase in their
academic achievement.” This declarative statement does not have to be true, but must be
empirically testable using data, so that we can judge whether it is true or false. Propositions are
generally derived based on logic (deduction) or empirical observations (induction).

Because propositions are associations between abstract constructs, they cannot be

tested directly. Instead, they are tested indirectly by examining the relationship between
corresponding measures (variables) of those constructs. The empirical formulation of
propositions, stated as relationships between variables, is called hypotheses (see Figure 2.1).
Since IQ scores and grade point average are operational measures of intelligence and academic
achievement respectively, the above proposition can be specified in form of the hypothesis: “An
increase in students’ IQ score causes an increase in their grade point average.” Propositions are
specified in the theoretical plane, while hypotheses are specified in the empirical plane. Hence,
hypotheses are empirically testable using observed data, and may be rejected if not supported
by empirical observations. Of course, the goal of hypothesis testing is to infer whether the
corresponding proposition is valid.

Hypotheses can be strong or weak. “Students’ IQ scores are related to their academic
achievement” is an example of a weak hypothesis, since it indicates neither the directionality of
the hypothesis (i.e., whether the relationship is positive or negative), nor its causality (i.e.,
whether intelligence causes academic achievement or academic achievement causes
intelligence). A stronger hypothesis is “students’ IQ scores are positively related to their
academic achievement”, which indicates the directionality but not the causality. A still better
hypothesis is “students’ IQ scores have positive effects on their academic achievement”, which
specifies both the directionality and the causality (i.e., intelligence causes academic
achievement, and not the reverse). The signs in Figure 2.2 indicate the directionality of the
respective hypotheses.

Also note that scientific hypotheses should clearly specify independent and dependent
variables. In the hypothesis, “students’ IQ scores have positive effects on their academic
achievement,” it is clear that intelligence is the independent variable (the “cause”) and academic
achievement is the dependent variable (the “effect”). Further, it is also clear that this
hypothesis can be evaluated as either true (if higher intelligence leads to higher academic
achievement) or false (if higher intelligence has no effect on or leads to lower academic
achievement). Later on in this book, we will examine how to empirically test such cause-effect
relationships. Statements such as “students are generally intelligent” or “all students can
achieve academic success” are not scientific hypotheses because they do not specify
independent and dependent variables, nor do they specify a directional relationship that can be
evaluated as true or false.
14 | S o c i a l S c i e n c e R e s e a r c h

Theories and Models

A theory is a set of systematically interrelated constructs and propositions intended to
explain and predict a phenomenon or behavior of interest, within certain boundary conditions
and assumptions. Essentially, a theory is a systemic collection of related theoretical
propositions. While propositions generally connect two or three constructs, theories represent
a system of multiple constructs and propositions. Hence, theories can be substantially more
complex and abstract and of a larger scope than propositions or hypotheses.

I must note here that people not familiar with scientific research often view a theory as
a speculation or the opposite of fact. For instance, people often say that teachers need to be less
theoretical and more practical or factual in their classroom teaching. However, practice or fact
are not opposites of theory, but in a scientific sense, are essential components needed to test
the validity of a theory. A good scientific theory should be well supported using observed facts
and should also have practical value, while a poorly defined theory tends to be lacking in these
dimensions. Famous organizational research Kurt Lewin once said, “Theory without practice is
sterile; practice without theory is blind.” Hence, both theory and facts (or practice) are
essential for scientific research.

Theories provide explanations of social or natural phenomenon. As emphasized in

Chapter 1, these explanations may be good or poor. Hence, there may be good or poor theories.
Chapter 3 describes some criteria that can be used to evaluate how good a theory really is.
Nevertheless, it is important for researchers to understand that theory is not “truth,” there is
nothing sacrosanct about any theory, and theories should not be accepted just because they
were proposed by someone. In the course of scientific progress, poorer theories are eventually
replaced by better theories with higher explanatory power. The essential challenge for
researchers is to build better and more comprehensive theories that can explain a target
phenomenon better than prior theories.

A term often used in conjunction with theory is a model. A model is a representation of

all or part of a system that is constructed to study that system (e.g., how the system works or
what triggers the system). While a theory tries to explain a phenomenon, a model tries to
represent a phenomenon. Models are often used by decision makers to make important
decisions based on a given set of inputs. For instance, marketing managers may use models to
decide how much money to spend on advertising for different product lines based on
parameters such as prior year’s advertising expenses, sales, market growth, and competing
products. Likewise, weather forecasters can use models to predict future weather patterns
based on parameters such as wind speeds, wind direction, temperature, and humidity. While
these models are useful, they may not necessarily explain advertising expenditure or weather
forecasts. Models may be of different kinds, such as mathematical models, network models, and
path models. Models can also be descriptive, predictive, or normative. Descriptive models are
frequently used for representing complex systems, for visualizing variables and relationships in
such systems. An advertising expenditure model may be a descriptive model. Predictive
models (e.g., a regression model) allow forecast of future events. Weather forecasting models
are predictive models. Normative models are used to guide our activities along commonly
accepted norms or practices. Models may also be static if it represents the state of a system at
one point in time, or dynamic, if it represents a system’s evolution over time.

The process of theory or model development may involve inductive and deductive
reasoning. Recall from Chapter 1 that deduction is the process of drawing conclusions about a
 T h i n k i n g L i k e a R e s e a r c h e r | 15

phenomenon or behavior based on theoretical or logical reasons and an initial set of premises.
As an example, if a certain bank enforces a strict code of ethics for its employees (Premise 1)
and Jamie is an employee at that bank (Premise 2), then Jamie can be trusted to follow ethical
practices (Conclusion). In deduction, the conclusions must be true if the initial premises and
reasons are correct.

In contrast, induction is the process of drawing conclusions based on facts or observed

evidence. For instance, if a firm spent a lot of money on a promotional campaign (Observation
1), but the sales did not increase (Observation 2), then possibly the promotion campaign was
poorly executed (Conclusion). However, there may be rival explanations for poor sales, such as
economic recession or the emergence of a competing product or brand or perhaps a supply
chain problem. Inductive conclusions are therefore only a hypothesis, and may be disproven.
Deductive conclusions generally tend to be stronger than inductive conclusions, but a deductive
conclusion based on an incorrect premise is also incorrect.

As shown in Figure 2.3, inductive and deductive reasoning go hand in hand in theory
and model building. Induction occurs when we observe a fact and ask, “Why is this happening?”
In answering this question, we advance one or more tentative explanations (hypotheses). We
then use deduction to narrow down the tentative explanations to the most plausible
explanation based on logic and reasonable premises (based on our understanding of the
phenomenon under study). Researchers must be able to move back and forth between
inductive and deductive reasoning if they are to post extensions or modifications to a given
model or theory, or built better ones, which are the essence of scientific research.

Figure 2.3. The model-building process

Chapter 3

The Research Process

In Chapter 1, we saw that scientific research is the process of acquiring scientific

knowledge using the scientific method. But how is such research conducted? This chapter
delves into the process of scientific research, and the assumptions and outcomes of the research
process.

Paradigms of Social Research

Our design and conduct of research is shaped by our mental models or frames of
references that we use to organize our reasoning and observations. These mental models or
frames (belief systems) are called paradigms. The word “paradigm” was popularized by
Thomas Kuhn (1962) in his book The Structure of Scientific Revolutions, where he examined the
history of the natural sciences to identify patterns of activities that shape the progress of
science. Similar ideas are applicable to social sciences as well, where a social reality can be
viewed by different people in different ways, which may constrain their thinking and reasoning
about the observed phenomenon. For instance, conservatives and liberals tend to have very
different perceptions of the role of government in people’s lives, and hence, have different
opinions on how to solve social problems. Conservatives may believe that lowering taxes is the
best way to stimulate a stagnant economy because it increases people’s disposable income and
spending, which in turn expands business output and employment. In contrast, liberals may
believe that governments should invest more directly in job creation programs such as public
works and infrastructure projects, which will increase employment and people’s ability to
consume and drive the economy. Likewise, Western societies place greater emphasis on
individual rights, such as one’s right to privacy, right of free speech, and right to bear arms. In
contrast, Asian societies tend to balance the rights of individuals against the rights of families,
organizations, and the government, and therefore tend to be more communal and less
individualistic in their policies. Such differences in perspective often lead Westerners to
criticize Asian governments for being autocratic, while Asians criticize Western societies for
being greedy, having high crime rates, and creating a “cult of the individual.” Our personal
paradigms are like “colored glasses” that govern how we view the world and how we structure
our thoughts about what we see in the world.

Paradigms are often hard to recognize, because they are implicit, assumed, and taken
for granted. However, recognizing these paradigms is key to making sense of and reconciling
differences in people’ perceptions of the same social phenomenon. For instance, why do
liberals believe that the best way to improve secondary education is to hire more teachers, but
conservatives believe that privatizing education (using such means as school vouchers) are

17
18 | S o c i a l S c i e n c e R e s e a r c h

more effective in achieving the same goal? Because conservatives place more faith in
competitive markets (i.e., in free competition between schools competing for education dollars),
while liberals believe more in labor (i.e., in having more teachers and schools). Likewise, in
social science research, if one were to understand why a certain technology was successfully
implemented in one organization but failed miserably in another, a researcher looking at the
world through a “rational lens” will look for rational explanations of the problem such as
inadequate technology or poor fit between technology and the task context where it is being
utilized, while another research looking at the same problem through a “social lens” may seek
out social deficiencies such as inadequate user training or lack of management support, while
those seeing it through a “political lens” will look for instances of organizational politics that
may subvert the technology implementation process. Hence, subconscious paradigms often
constrain the concepts that researchers attempt to measure, their observations, and their
subsequent interpretations of a phenomenon. However, given the complex nature of social
phenomenon, it is possible that all of the above paradigms are partially correct, and that a fuller
understanding of the problem may require an understanding and application of multiple
paradigms.

Two popular paradigms today among social science researchers are positivism and
post-positivism. Positivism, based on the works of French philosopher Auguste Comte (1798-
1857), was the dominant scientific paradigm until the mid-20th century. It holds that science or
knowledge creation should be restricted to what can be observed and measured. Positivism
tends to rely exclusively on theories that can be directly tested. Though positivism was
originally an attempt to separate scientific inquiry from religion (where the precepts could not
be objectively observed), positivism led to empiricism or a blind faith in observed data and a
rejection of any attempt to extend or reason beyond observable facts. Since human thoughts
and emotions could not be directly measured, there were not considered to be legitimate topics
for scientific research. Frustrations with the strictly empirical nature of positivist philosophy
led to the development of post-positivism (or postmodernism) during the mid-late 20th
century. Post-positivism argues that one can make reasonable inferences about a phenomenon
by combining empirical observations with logical reasoning. Post-positivists view science as
not certain but probabilistic (i.e., based on many contingencies), and often seek to explore these
contingencies to understand social reality better. The post-positivist camp has further
fragmented into subjectivists, who view the world as a subjective construction of our subjective
minds rather than as an objective reality, and critical realists, who believe that there is an
external reality that is independent of a person’s thinking but we can never know such reality
with any degree of certainty.

Burrell and Morgan (1979), in their seminal book Sociological Paradigms and
Organizational Analysis, suggested that the way social science researchers view and study social
phenomena is shaped by two fundamental sets of philosophical assumptions: ontology and
epistemology. Ontology refers to our assumptions about how we see the world, e.g., does the
world consist mostly of social order or constant change. Epistemology refers to our
assumptions about the best way to study the world, e.g., should we use an objective or
subjective approach to study social reality. Using these two sets of assumptions, we can
categorize social science research as belonging to one of four categories (see Figure 3.1).

If researchers view the world as consisting mostly of social order (ontology) and hence
seek to study patterns of ordered events or behaviors, and believe that the best way to study
such a world is using objective approach (epistemology) that is independent of the person
conducting the observation or interpretation, such as by using standardized data collection
 T h e R e s e a r c h P r o c e s s | 19

tools like surveys, then they are adopting a paradigm of functionalism. However, if they
believe that the best way to study social order is though the subjective interpretation of
participants involved, such as by interviewing different participants and reconciling differences
among their responses using their own subjective perspectives, then they are employing an
interpretivism paradigm. If researchers believe that the world consists of radical change and
seek to understand or enact change using an objectivist approach, then they are employing a
radical structuralism paradigm. If they wish to understand social change using the subjective
perspectives of the participants involved, then they are following a radical humanism
paradigm.

Figure 3.1. Four paradigms of social science research

(Source: Burrell and Morgan, 1979)

To date, the majority of social science research has emulated the natural sciences, and
followed the functionalist paradigm. Functionalists believe that social order or patterns can be
understood in terms of their functional components, and therefore attempt to break down a
problem into small components and studying one or more components in detail using
objectivist techniques such as surveys and experimental research. However, with the
emergence of post-positivist thinking, a small but growing number of social science researchers
are attempting to understand social order using subjectivist techniques such as interviews and
ethnographic studies. Radical humanism and radical structuralism continues to represent a
negligible proportion of social science research, because scientists are primarily concerned with
understanding generalizable patterns of behavior, events, or phenomena, rather than
idiosyncratic or changing events. Nevertheless, if you wish to study social change, such as why
democratic movements are increasingly emerging in Middle Eastern countries, or why this
movement was successful in Tunisia, took a longer path to success in Libya, and is still not
successful in Syria, then perhaps radical humanism is the right approach for such a study.
Social and organizational phenomena generally consists elements of both order and change.
For instance, organizational success depends on formalized business processes, work
procedures, and job responsibilities, while being simultaneously constrained by a constantly
changing mix of competitors, competing products, suppliers, and customer base in the business
environment. Hence, a holistic and more complete understanding of social phenomena such as
why are some organizations more successful than others, require an appreciation and
application of a multi-paradigmatic approach to research.
20 | S o c i a l S c i e n c e R e s e a r c h

Overview of the Research Process

So how do our mental paradigms shape social science research? At its core, all scientific
research is an iterative process of observation, rationalization, and validation. In the
observation phase, we observe a natural or social phenomenon, event, or behavior that
interests us. In the rationalization phase, we try to make sense of or the observed
phenomenon, event, or behavior by logically connecting the different pieces of the puzzle that
we observe, which in some cases, may lead to the construction of a theory. Finally, in the
validation phase, we test our theories using a scientific method through a process of data
collection and analysis, and in doing so, possibly modify or extend our initial theory. However,
research designs vary based on whether the researcher starts at observation and attempts to
rationalize the observations (inductive research), or whether the researcher starts at an ex ante
rationalization or a theory and attempts to validate the theory (deductive research). Hence, the
observation-rationalization-validation cycle is very similar to the induction-deduction cycle of
research discussed in Chapter 1.

Most traditional research tends to be deductive and functionalistic in nature. Figure 3.2
provides a schematic view of such a research project. This figure depicts a series of activities to
be performed in functionalist research, categorized into three phases: exploration, research
design, and research execution. Note that this generalized design is not a roadmap or flowchart
for all research. It applies only to functionalistic research, and it can and should be modified to
fit the needs of a specific project.

Figure 3.2. Functionalistic research process

The first phase of research is exploration. This phase includes exploring and selecting
research questions for further investigation, examining the published literature in the area of
inquiry to understand the current state of knowledge in that area, and identifying theories that
may help answer the research questions of interest.
 T h e R e s e a r c h P r o c e s s | 21

The first step in the exploration phase is identifying one or more research questions
dealing with a specific behavior, event, or phenomena of interest. Research questions are
specific questions about a behavior, event, or phenomena of interest that you wish to seek
answers for in your research. Examples include what factors motivate consumers to purchase
goods and services online without knowing the vendors of these goods or services, how can we
make high school students more creative, and why do some people commit terrorist acts.
Research questions can delve into issues of what, why, how, when, and so forth. More
interesting research questions are those that appeal to a broader population (e.g., “how can
firms innovate” is a more interesting research question than “how can Chinese firms innovate in
the service-sector”), address real and complex problems (in contrast to hypothetical or “toy”
problems), and where the answers are not obvious. Narrowly focused research questions
(often with a binary yes/no answer) tend to be less useful and less interesting and less suited to
capturing the subtle nuances of social phenomena. Uninteresting research questions generally
lead to uninteresting and unpublishable research findings.

The next step is to conduct a literature review of the domain of interest. The purpose
of a literature review is three-fold: (1) to survey the current state of knowledge in the area of
inquiry, (2) to identify key authors, articles, theories, and findings in that area, and (3) to
identify gaps in knowledge in that research area. Literature review is commonly done today
using computerized keyword searches in online databases. Keywords can be combined using
“and” and “or” operations to narrow down or expand the search results. Once a shortlist of
relevant articles is generated from the keyword search, the researcher must then manually
browse through each article, or at least its abstract section, to determine the suitability of that
article for a detailed review. Literature reviews should be reasonably complete, and not
restricted to a few journals, a few years, or a specific methodology. Reviewed articles may be
summarized in the form of tables, and can be further structured using organizing frameworks
such as a concept matrix. A well-conducted literature review should indicate whether the initial
research questions have already been addressed in the literature (which would obviate the
need to study them again), whether there are newer or more interesting research questions
available, and whether the original research questions should be modified or changed in light of
findings of the literature review. The review can also provide some intuitions or potential
answers to the questions of interest and/or help identify theories that have previously been
used to address similar questions.

Since functionalist (deductive) research involves theory-testing, the third step is to

identify one or more theories can help address the desired research questions. While the
literature review may uncover a wide range of concepts or constructs potentially related to the
phenomenon of interest, a theory will help identify which of these constructs is logically
relevant to the target phenomenon and how. Forgoing theories may result in measuring a wide
range of less relevant, marginally relevant, or irrelevant constructs, while also minimizing the
chances of obtaining results that are meaningful and not by pure chance. In functionalist
research, theories can be used as the logical basis for postulating hypotheses for empirical
testing. Obviously, not all theories are well-suited for studying all social phenomena. Theories
must be carefully selected based on their fit with the target problem and the extent to which
their assumptions are consistent with that of the target problem. We will examine theories and
the process of theorizing in detail in the next chapter.

The next phase in the research process is research design. This process is concerned
with creating a blueprint of the activities to take in order to satisfactorily answer the research
22 | S o c i a l S c i e n c e R e s e a r c h

questions identified in the exploration phase. This includes selecting a research method,
operationalizing constructs of interest, and devising an appropriate sampling strategy.

Operationalization is the process of designing precise measures for abstract

theoretical constructs. This is a major problem in social science research, given that many of
the constructs, such as prejudice, alienation, and liberalism are hard to define, let alone
measure accurately. Operationalization starts with specifying an “operational definition” (or
“conceptualization”) of the constructs of interest. Next, the researcher can search the literature
to see if there are existing prevalidated measures matching their operational definition that can
be used directly or modified to measure their constructs of interest. If such measures are not
available or if existing measures are poor or reflect a different conceptualization than that
intended by the researcher, new instruments may have to be designed for measuring those
constructs. This means specifying exactly how exactly the desired construct will be measured
(e.g., how many items, what items, and so forth). This can easily be a long and laborious
process, with multiple rounds of pretests and modifications before the newly designed
instrument can be accepted as “scientifically valid.” We will discuss operationalization of
constructs in a future chapter on measurement.

Simultaneously with operationalization, the researcher must also decide what research
method they wish to employ for collecting data to address their research questions of interest.
Such methods may include quantitative methods such as experiments or survey research or
qualitative methods such as case research or action research, or possibly a combination of both.
If an experiment is desired, then what is the experimental design? If survey, do you plan a mail
survey, telephone survey, web survey, or a combination? For complex, uncertain, and multi-
faceted social phenomena, multi-method approaches may be more suitable, which may help
leverage the unique strengths of each research method and generate insights that may not be
obtained using a single method.

Researchers must also carefully choose the target population from which they wish to
collect data, and a sampling strategy to select a sample from that population. For instance,
should they survey individuals or firms or workgroups within firms? What types of individuals
or firms they wish to target? Sampling strategy is closely related to the unit of analysis in a
research problem. While selecting a sample, reasonable care should be taken to avoid a biased
sample (e.g., sample based on convenience) that may generate biased observations. Sampling is
covered in depth in a later chapter.

At this stage, it is often a good idea to write a research proposal detailing all of the
decisions made in the preceding stages of the research process and the rationale behind each
decision. This multi-part proposal should address what research questions you wish to study
and why, the prior state of knowledge in this area, theories you wish to employ along with
hypotheses to be tested, how to measure constructs, what research method to be employed and
why, and desired sampling strategy. Funding agencies typically require such a proposal in
order to select the best proposals for funding. Even if funding is not sought for a research
project, a proposal may serve as a useful vehicle for seeking feedback from other researchers
and identifying potential problems with the research project (e.g., whether some important
constructs were missing from the study) before starting data collection. This initial feedback is
invaluable because it is often too late to correct critical problems after data is collected in a
research study.
 T h e R e s e a r c h P r o c e s s | 23

Having decided who to study (subjects), what to measure (concepts), and how to collect
data (research method), the researcher is now ready to proceed to the research execution
phase. This includes pilot testing the measurement instruments, data collection, and data
analysis.

Pilot testing is an often overlooked but extremely important part of the research
process. It helps detect potential problems in your research design and/or instrumentation
(e.g., whether the questions asked is intelligible to the targeted sample), and to ensure that the
measurement instruments used in the study are reliable and valid measures of the constructs of
interest. The pilot sample is usually a small subset of the target population. After a successful
pilot testing, the researcher may then proceed with data collection using the sampled
population. The data collected may be quantitative or qualitative, depending on the research
method employed.

Following data collection, the data is analyzed and interpreted for the purpose of
drawing conclusions regarding the research questions of interest. Depending on the type of
data collected (quantitative or qualitative), data analysis may be quantitative (e.g., employ
statistical techniques such as regression or structural equation modeling) or qualitative (e.g.,
coding or content analysis).

The final phase of research involves preparing the final research report documenting
the entire research process and its findings in the form of a research paper, dissertation, or
monograph. This report should outline in detail all the choices made during the research
process (e.g., theory used, constructs selected, measures used, research methods, sampling, etc.)
and why, as well as the outcomes of each phase of the research process. The research process
must be described in sufficient detail so as to allow other researchers to replicate your study,
test the findings, or assess whether the inferences derived are scientifically acceptable. Of
course, having a ready research proposal will greatly simplify and quicken the process of
writing the finished report. Note that research is of no value unless the research process and
outcomes are documented for future generations; such documentation is essential for the
incremental progress of science.

Common Mistakes in Research

The research process is fraught with problems and pitfalls, and novice researchers often
find, after investing substantial amounts of time and effort into a research project, that their
research questions were not sufficiently answered, or that the findings were not interesting
enough, or that the research was not of “acceptable” scientific quality. Such problems typically
result in research papers being rejected by journals. Some of the more frequent mistakes are
described below.

Insufficiently motivated research questions. Often times, we choose our “pet”

problems that are interesting to us but not to the scientific community at large, i.e., it does not
generate new knowledge or insight about the phenomenon being investigated. Because the
research process involves a significant investment of time and effort on the researcher’s part,
the researcher must be certain (and be able to convince others) that the research questions
they seek to answer in fact deal with real problems (and not hypothetical problems) that affect
a substantial portion of a population and has not been adequately addressed in prior research.
24 | S o c i a l S c i e n c e R e s e a r c h

Pursuing research fads. Another common mistake is pursuing “popular” topics with
limited shelf life. A typical example is studying technologies or practices that are popular today.
Because research takes several years to complete and publish, it is possible that popular
interest in these fads may die down by the time the research is completed and submitted for
publication. A better strategy may be to study “timeless” topics that have always persisted
through the years.

Unresearchable problems. Some research problems may not be answered adequately

based on observed evidence alone, or using currently accepted methods and procedures. Such
problems are best avoided. However, some unresearchable, ambiguously defined problems
may be modified or fine tuned into well-defined and useful researchable problems.

Favored research methods. Many researchers have a tendency to recast a research

problem so that it is amenable to their favorite research method (e.g., survey research). This is
an unfortunate trend. Research methods should be chosen to best fit a research problem, and
not the other way around.

Blind data mining. Some researchers have the tendency to collect data first (using
instruments that are already available), and then figure out what to do with it. Note that data
collection is only one step in a long and elaborate process of planning, designing, and executing
research. In fact, a series of other activities are needed in a research process prior to data
collection. If researchers jump into data collection without such elaborate planning, the data
collected will likely be irrelevant, imperfect, or useless, and their data collection efforts may be
entirely wasted. An abundance of data cannot make up for deficits in research planning and
design, and particularly, for the lack of interesting research questions.
Chapter 4

Theories in Scientific Research

As we know from previous chapters, science is knowledge represented as a collection of

“theories” derived using the scientific method. In this chapter, we will examine what is a
theory, why do we need theories in research, what are the building blocks of a theory, how to
evaluate theories, how can we apply theories in research, and also presents illustrative
examples of five theories frequently used in social science research.

Theories
Theories are explanations of a natural or social behavior, event, or phenomenon. More
formally, a scientific theory is a system of constructs (concepts) and propositions (relationships
between those constructs) that collectively presents a logical, systematic, and coherent
explanation of a phenomenon of interest within some assumptions and boundary conditions
(Bacharach 1989).1

Theories should explain why things happen, rather than just describe or predict. Note
that it is possible to predict events or behaviors using a set of predictors, without necessarily
explaining why such events are taking place. For instance, market analysts predict fluctuations
in the stock market based on market announcements, earnings reports of major companies, and
new data from the Federal Reserve and other agencies, based on previously observed
correlations. Prediction requires only correlations. In contrast, explanations require causations,
or understanding of cause-effect relationships. Establishing causation requires three
conditions: (1) correlations between two constructs, (2) temporal precedence (the cause must
precede the effect in time), and (3) rejection of alternative hypotheses (through testing).
Scientific theories are different from theological, philosophical, or other explanations in that
scientific theories can be empirically tested using scientific methods.

Explanations can be idiographic or nomothetic. Idiographic explanations are those

that explain a single situation or event in idiosyncratic detail. For example, you did poorly on an
exam because: (1) you forgot that you had an exam on that day, (2) you arrived late to the exam
due to a traffic jam, (3) you panicked midway through the exam, (4) you had to work late the
previous evening and could not study for the exam, or even (5) your dog ate your text book.
The explanations may be detailed, accurate, and valid, but they may not apply to other similar
situations, even involving the same person, and are hence not generalizable. In contrast,

1Bacharach, S. B. (1989). “Organizational Theories: Some Criteria for Evaluation,” Academy of

Management Review (14:4), 496-515.

25
26 | S o c i a l S c i e n c e R e s e a r c h

nomothetic explanations seek to explain a class of situations or events rather than a specific
situation or event. For example, students who do poorly in exams do so because they did not
spend adequate time preparing for exams or that they suffer from nervousness, attention-
deficit, or some other medical disorder. Because nomothetic explanations are designed to be
generalizable across situations, events, or people, they tend to be less precise, less complete,
and less detailed. However, they explain economically, using only a few explanatory variables.
Because theories are also intended to serve as generalized explanations for patterns of events,
behaviors, or phenomena, theoretical explanations are generally nomothetic in nature.

While understanding theories, it is also important to understand what theory is not.

Theory is not data, facts, typologies, taxonomies, or empirical findings. A collection of facts is
not a theory, just as a pile of stones is not a house. Likewise, a collection of constructs (e.g., a
typology of constructs) is not a theory, because theories must go well beyond constructs to
include propositions, explanations, and boundary conditions. Data, facts, and findings operate
at the empirical or observational level, while theories operate at a conceptual level and are
based on logic rather than observations.

There are many benefits to using theories in research. First, theories provide the
underlying logic of the occurrence of natural or social phenomenon by explaining what are the
key drivers and key outcomes of the target phenomenon and why, and what underlying
processes are responsible driving that phenomenon. Second, they aid in sense-making by
helping us synthesize prior empirical findings within a theoretical framework and reconcile
contradictory findings by discovering contingent factors influencing the relationship between
two constructs in different studies. Third, theories provide guidance for future research by
helping identify constructs and relationships that are worthy of further research. Fourth,
theories can contribute to cumulative knowledge building by bridging gaps between other
theories and by causing existing theories to be reevaluated in a new light.

However, theories can also have their own share of limitations. As simplified
explanations of reality, theories may not always provide adequate explanations of the
phenomenon of interest based on a limited set of constructs and relationships. Theories are
designed to be simple and parsimonious explanations, while reality may be significantly more
complex. Furthermore, theories may impose blinders or limit researchers’ “range of vision,”
causing them to miss out on important concepts that are not defined by the theory.

Building Blocks of a Theory

David Whetten (1989) suggests that there are four building blocks of a theory:
constructs, propositions, logic, and boundary conditions/assumptions. Constructs capture the
“what” of theories (i.e., what concepts are important for explaining a phenomenon),
propositions capture the “how” (i.e., how are these concepts related to each other), logic
represents the “why” (i.e., why are these concepts related), and boundary
conditions/assumptions examines the “who, when, and where” (i.e., under what circumstances
will these concepts and relationships work). Though constructs and propositions were
previously discussed in Chapter 2, we describe them again here for the sake of completeness.

Constructs are abstract concepts specified at a high level of abstraction that are chosen
specifically to explain the phenomenon of interest. Recall from Chapter 2 that constructs may
be unidimensional (i.e., embody a single concept), such as weight or age, or multi-dimensional
(i.e., embody multiple underlying concepts), such as personality or culture. While some
 T h e o r i e s i n S c i e n t i f i c R e s e a r c h | 27

constructs, such as age, education, and firm size, are easy to understand, others, such as
creativity, prejudice, and organizational agility, may be more complex and abstruse, and still
others such as trust, attitude, and learning, may represent temporal tendencies rather than
steady states. Nevertheless, all constructs must have clear and unambiguous operational
definition that should specify exactly how the construct will be measured and at what level of
analysis (individual, group, organizational, etc.). Measurable representations of abstract
constructs are called variables. For instance, intelligence quotient (IQ score) is a variable that
is purported to measure an abstract construct called intelligence. As noted earlier, scientific
research proceeds along two planes: a theoretical plane and an empirical plane. Constructs are
conceptualized at the theoretical plane, while variables are operationalized and measured at
the empirical (observational) plane. Furthermore, variables may be independent, dependent,
mediating, or moderating, as discussed in Chapter 2. The distinction between constructs
(conceptualized at the theoretical level) and variables (measured at the empirical level) is
shown in Figure 4.1.

Figure 4.1. Distinction between theoretical and empirical concepts

Propositions are associations postulated between constructs based on deductive logic.

Propositions are stated in declarative form and should ideally indicate a cause-effect
relationship (e.g., if X occurs, then Y will follow). Note that propositions may be conjectural but
MUST be testable, and should be rejected if they are not supported by empirical observations.
However, like constructs, propositions are stated at the theoretical level, and they can only be
tested by examining the corresponding relationship between measurable variables of those
constructs. The empirical formulation of propositions, stated as relationships between
variables, is called hypotheses. The distinction between propositions (formulated at the
theoretical level) and hypotheses (tested at the empirical level) is depicted in Figure 4.1.

The third building block of a theory is the logic that provides the basis for justifying the
propositions as postulated. Logic acts like a “glue” that connects the theoretical constructs and
provides meaning and relevance to the relationships between these constructs. Logic also
represents the “explanation” that lies at the core of a theory. Without logic, propositions will be
ad hoc, arbitrary, and meaningless, and cannot be tied into a cohesive “system of propositions”
that is the heart of any theory.

Finally, all theories are constrained by assumptions about values, time, and space, and
boundary conditions that govern where the theory can be applied and where it cannot be
applied. For example, many economic theories assume that human beings are rational (or
28 | S o c i a l S c i e n c e R e s e a r c h

boundedly rational) and employ utility maximization based on cost and benefit expectations as
a way of understand human behavior. In contrast, political science theories assume that people
are more political than rational, and try to position themselves in their professional or personal
environment in a way that maximizes their power and control over others. Given the nature of
their underlying assumptions, economic and political theories are not directly comparable, and
researchers should not use economic theories if their objective is to understand the power
structure or its evolution in a organization. Likewise, theories may have implicit cultural
assumptions (e.g., whether they apply to individualistic or collective cultures), temporal
assumptions (e.g., whether they apply to early stages or later stages of human behavior), and
spatial assumptions (e.g., whether they apply to certain localities but not to others). If a theory
is to be properly used or tested, all of its implicit assumptions that form the boundaries of that
theory must be properly understood. Unfortunately, theorists rarely state their implicit
assumptions clearly, which leads to frequent misapplications of theories to problem situations
in research.

Attributes of a Good Theory

Theories are simplified and often partial explanations of complex social reality. As such,
there can be good explanations or poor explanations, and consequently, there can be good
theories or poor theories. How can we evaluate the “goodness” of a given theory? Different
criteria have been proposed by different researchers, the more important of which are listed
below:

 Logical consistency: Are the theoretical constructs, propositions, boundary conditions,

and assumptions logically consistent with each other? If some of these “building blocks”
of a theory are inconsistent with each other (e.g., a theory assumes rationality, but some
constructs represent non-rational concepts), then the theory is a poor theory.

 Explanatory power: How much does a given theory explain (or predict) reality? Good
theories obviously explain the target phenomenon better than rival theories, as often
measured by variance explained (R-square) value in regression equations.

 Falsifiability: British philosopher Karl Popper stated in the 1940’s that for theories to
be valid, they must be falsifiable. Falsifiability ensures that the theory is potentially
disprovable, if empirical data does not match with theoretical propositions, which
allows for their empirical testing by researchers. In other words, theories cannot be
theories unless they can be empirically testable. Tautological statements, such as “a day
with high temperatures is a hot day” are not empirically testable because a hot day is
defined (and measured) as a day with high temperatures, and hence, such statements
cannot be viewed as a theoretical proposition. Falsifiability requires presence of rival
explanations it ensures that the constructs are adequately measurable, and so forth.
However, note that saying that a theory is falsifiable is not the same as saying that a
theory should be falsified. If a theory is indeed falsified based on empirical evidence,
then it was probably a poor theory to begin with!

 Parsimony: Parsimony examines how much of a phenomenon is explained with how

few variables. The concept is attributed to 14th century English logician Father William
of Ockham (and hence called “Ockham’s razor” or “Occam’s razor), which states that
among competing explanations that sufficiently explain the observed evidence, the
simplest theory (i.e., one that uses the smallest number of variables or makes the fewest
 T h e o r i e s i n S c i e n t i f i c R e s e a r c h | 29

assumptions) is the best. Explanation of a complex social phenomenon can always be

increased by adding more and more constructs. However, such approach defeats the
purpose of having a theory, which are intended to be “simplified” and generalizable
explanations of reality. Parsimony relates to the degrees of freedom in a given theory.
Parsimonious theories have higher degrees of freedom, which allow them to be more
easily generalized to other contexts, settings, and populations.

Approaches to Theorizing
How do researchers build theories? Steinfeld and Fulk (1990)2 recommend four such
approaches. The first approach is to build theories inductively based on observed patterns of
events or behaviors. Such approach is often called “grounded theory building”, because the
theory is grounded in empirical observations. This technique is heavily dependent on the
observational and interpretive abilities of the researcher, and the resulting theory may be
subjective and non-confirmable. Furthermore, observing certain patterns of events will not
necessarily make a theory, unless the researcher is able to provide consistent explanations for
the observed patterns. We will discuss the grounded theory approach in a later chapter on
qualitative research.

The second approach to theory building is to conduct a bottom-up conceptual analysis

to identify different sets of predictors relevant to the phenomenon of interest using a
predefined framework. One such framework may be a simple input-process-output framework,
where the researcher may look for different categories of inputs, such as individual,
organizational, and/or technological factors potentially related to the phenomenon of interest
(the output), and describe the underlying processes that link these factors to the target
phenomenon. This is also an inductive approach that relies heavily on the inductive abilities of
the researcher, and interpretation may be biased by researcher’s prior knowledge of the
phenomenon being studied.

The third approach to theorizing is to extend or modify existing theories to explain a

new context, such as by extending theories of individual learning to explain organizational
learning. While making such an extension, certain concepts, propositions, and/or boundary
conditions of the old theory may be retained and others modified to fit the new context. This
deductive approach leverages the rich inventory of social science theories developed by prior
theoreticians, and is an efficient way of building new theories by building on existing ones.

The fourth approach is to apply existing theories in entirely new contexts by drawing
upon the structural similarities between the two contexts. This approach relies on reasoning by
analogy, and is probably the most creative way of theorizing using a deductive approach. For
instance, Markus (1987)3 used analogic similarities between a nuclear explosion and
uncontrolled growth of networks or network-based businesses to propose a critical mass
theory of network growth. Just as a nuclear explosion requires a critical mass of radioactive
material to sustain a nuclear explosion, Markus suggested that a network requires a critical
mass of users to sustain its growth, and without such critical mass, users may leave the
network, causing an eventual demise of the network.

2 Steinfield, C.W. and Fulk, J. (1990). “The Theory Imperative," in Organizations and Communications
Technology, J. Fulk and C. W. Steinfield (eds.), Newbury Park, CA: Sage Publications.
3 Markus, M. L. (1987). “Toward a ‘Critical Mass’ Theory of Interactive Media: Universal Access,

Interdependence, and Diffusion,” Communication Research (14:5), 491-511.

30 | S o c i a l S c i e n c e R e s e a r c h

Examples of Social Science Theories

In this section, we present brief overviews of a few illustrative theories from different
social science disciplines. These theories explain different types of social behaviors, using a set
of constructs, propositions, boundary conditions, assumptions, and underlying logic. Note that
the following represents just a simplistic introduction to these theories; readers are advised to
consult the original sources of these theories for more details and insights on each theory.

Agency Theory. Agency theory (also called principal-agent theory), a classic theory in
the organizational economics literature, was originally proposed by Ross (1973)4 to explain
two-party relationships (such as those between an employer and its employees, between
organizational executives and shareholders, and between buyers and sellers) whose goals are
not congruent with each other. The goal of agency theory is to specify optimal contracts and the
conditions under which such contracts may help minimize the effect of goal incongruence. The
core assumptions of this theory are that human beings are self-interested individuals,
boundedly rational, and risk-averse, and the theory can be applied at the individual or
organizational level.

The two parties in this theory are the principal and the agent; the principal employs the
agent to perform certain tasks on its behalf. While the principal’s goal is quick and effective
completion of the assigned task, the agent’s goal may be working at its own pace, avoiding risks,
and seeking self-interest (such as personal pay) over corporate interests. Hence, the goal
incongruence. Compounding the nature of the problem may be information asymmetry
problems caused by the principal’s inability to adequately observe the agent’s behavior or
accurately evaluate the agent’s skill sets. Such asymmetry may lead to agency problems where
the agent may not put forth the effort needed to get the task done (the moral hazard problem)
or may misrepresent its expertise or skills to get the job but not perform as expected (the
adverse selection problem). Typical contracts that are behavior-based, such as a monthly salary,
cannot overcome these problems. Hence, agency theory recommends using outcome-based
contracts, such as a commissions or a fee payable upon task completion, or mixed contracts that
combine behavior-based and outcome-based incentives. An employee stock option plans are is
an example of an outcome-based contract while employee pay is a behavior-based contract.
Agency theory also recommends tools that principals may employ to improve the efficacy of
behavior-based contracts, such as investing in monitoring mechanisms (such as hiring
supervisors) to counter the information asymmetry caused by moral hazard, designing
renewable contracts contingent on agent’s performance (performance assessment makes the
contract partially outcome-based), or by improving the structure of the assigned task to make it
more programmable and therefore more observable.

Theory of Planned Behavior. Postulated by Azjen (1991)5, the theory of planned

behavior (TPB) is a generalized theory of human behavior in the social psychology literature
that can be used to study a wide range of individual behaviors. It presumes that individual
behavior represents conscious reasoned choice, and is shaped by cognitive thinking and social
pressures. The theory postulates that behaviors are based on one’s intention regarding that
behavior, which in turn is a function of the person’s attitude toward the behavior, subjective

4 Ross, S. A. (1973). “The Economic Theory of Agency: The Principal’s Problem,” American Economic
Review (63:2), 134-139.
5 Ajzen, I. (1991). “The Theory of Planned Behavior,” Organizational Behavior and Human Decision

Processes (50), 179-211.

 T h e o r i e s i n S c i e n t i f i c R e s e a r c h | 31

norm regarding that behavior, and perception of control over that behavior (see Figure 4.2).
Attitude is defined as the individual's overall positive or negative feelings about performing the
behavior in question, which may be assessed as a summation of one's beliefs regarding the
different consequences of that behavior, weighted by the desirability of those consequences.
Subjective norm refers to one’s perception of whether people important to that person expect
the person to perform the intended behavior, and represented as a weighted combination of the
expected norms of different referent groups such as friends, colleagues, or supervisors at work.
Behavioral control is one's perception of internal or external controls constraining the behavior
in question. Internal controls may include the person’s ability to perform the intended behavior
(self-efficacy), while external control refers to the availability of external resources needed to
perform that behavior (facilitating conditions). TPB also suggests that sometimes people may
intend to perform a given behavior but lack the resources needed to do so, and therefore
suggests that posits that behavioral control can have a direct effect on behavior, in addition to
the indirect effect mediated by intention.

TPB is an extension of an earlier theory called the theory of reasoned action, which
included attitude and subjective norm as key drivers of intention, but not behavioral control.
The latter construct was added by Ajzen in TPB to account for circumstances when people may
have incomplete control over their own behaviors (such as not having high-speed Internet
access for web surfing).

Figure 4.2. Theory of planned behavior

Innovation diffusion theory. Innovation diffusion theory (IDT) is a seminal theory in

the communications literature that explains how innovations are adopted within a population
of potential adopters. The concept was first studied by French sociologist Gabriel Tarde, but the
theory was developed by Everett Rogers in 1962 based on observations of 508 diffusion
studies. The four key elements in this theory are: innovation, communication channels, time,
and social system. Innovations may include new technologies, new practices, or new ideas, and
adopters may be individuals or organizations. At the macro (population) level, IDT views
innovation diffusion as a process of communication where people in a social system learn about
a new innovation and its potential benefits through communication channels (such as mass
media or prior adopters) and are persuaded to adopt it. Diffusion is a temporal process; the
diffusion process starts off slow among a few early adopters, then picks up speed as the
innovation is adopted by the mainstream population, and finally slows down as the adopter
population reaches saturation. The cumulative adoption pattern therefore an S-shaped curve,
as shown in Figure 4.3, and the adopter distribution represents a normal distribution. All
adopters are not identical, and adopters can be classified into innovators, early adopters, early
majority, late majority, and laggards based on their time of their adoption. The rate of diffusion
32 | S o c i a l S c i e n c e R e s e a r c h

also depends on characteristics of the social system such as the presence of opinion leaders
(experts whose opinions are valued by others) and change agents (people who influence others’
behaviors).

At the micro (adopter) level, Rogers (1995)6 suggests that innovation adoption is a
process consisting of five stages: (1) knowledge: when adopters first learn about an innovation
from mass-media or interpersonal channels, (2) persuasion: when they are persuaded by prior
adopters to try the innovation, (3) decision: their decision to accept or reject the innovation, (4)
implementation: their initial utilization of the innovation, and (5) confirmation: their decision
to continue using it to its fullest potential (see Figure 4.4). Five innovation characteristics are
presumed to shape adopters’ innovation adoption decisions: (1) relative advantage: the
expected benefits of an innovation relative to prior innovations, (2) compatibility: the extent to
which the innovation fits with the adopter’s work habits, beliefs, and values, (3) complexity: the
extent to which the innovation is difficult to learn and use, (4) trialability: the extent to which
the innovation can be tested on a trial basis, and (5) observability: the extent to which the
results of using the innovation can be clearly observed. The last two characteristics have since
been dropped from many innovation studies. Complexity is negatively correlated to innovation
adoption, while the other four factors are positively correlated. Innovation adoption also
depends on personal factors such as the adopter’s risk-taking propensity, education level,
cosmopolitanism, and communication influence. Early adopters are venturesome, well
educated, and rely more on mass media for information about the innovation, while later
adopters rely more on interpersonal sources (such as friends and family) as their primary
source of information. IDT has been criticized for having a “pro-innovation bias,” that is for
presuming that all innovations are beneficial and will be eventually diffused across the entire
population, and because it does not allow for inefficient innovations such as fads or fashions to
die off quickly without being adopted by the entire population or being replaced by better
innovations.

Figure 4.3. S-shaped diffusion curve

6 Rogers, E. (1962). Diffusion of Innovations. New York: The Free Press. Other editions 1983, 1996, 2005.
 T h e o r i e s i n S c i e n t i f i c R e s e a r c h | 33

Imple-
Knowledge Persuasion Decision Confirmation
mentation

Figure 4.4. Innovation adoption process

Elaboration Likelihood Model. Developed by Petty and Cacioppo (1986)7, the

elaboration likelihood model (ELM) is a dual-process theory of attitude formation or change in
the psychology literature. It explains how individuals can be influenced to change their attitude
toward a certain object, events, or behavior and the relative efficacy of such change strategies.
The ELM posits that one’s attitude may be shaped by two “routes” of influence, the central route
and the peripheral route, which differ in the amount of thoughtful information processing or
“elaboration” required of people (see Figure 4.5). The central route requires a person to think
about issue-related arguments in an informational message and carefully scrutinize the merits
and relevance of those arguments, before forming an informed judgment about the target
object. In the peripheral route, subjects rely on external “cues” such as number of prior users,
endorsements from experts, or likeability of the endorser, rather than on the quality of
arguments, in framing their attitude towards the target object. The latter route is less
cognitively demanding, and the routes of attitude change are typically operationalized in the
ELM using the argument quality and peripheral cues constructs respectively.

Figure 4.5. Elaboration likelihood model

Whether people will be influenced by the central or peripheral routes depends upon
their ability and motivation to elaborate the central merits of an argument. This ability and
motivation to elaborate is called elaboration likelihood. People in a state of high elaboration
likelihood (high ability and high motivation) are more likely to thoughtfully process the
information presented and are therefore more influenced by argument quality, while those in
the low elaboration likelihood state are more motivated by peripheral cues. Elaboration
likelihood is a situational characteristic and not a personal trait. For instance, a doctor may
employ the central route for diagnosing and treating a medical ailment (by virtue of his or her
expertise of the subject), but may rely on peripheral cues from auto mechanics to understand

7Petty, R. E., and Cacioppo, J. T. (1986). Communication and Persuasion: Central and Peripheral Routes to
Attitude Change. New York: Springer-Verlag.
34 | S o c i a l S c i e n c e R e s e a r c h

the problems with his car. As such, the theory has widespread implications about how to enact
attitude change toward new products or ideas and even social change.

General Deterrence Theory. Two utilitarian philosophers of the eighteenth century, Cesare
Beccaria and Jeremy Bentham, formulated General Deterrence Theory (GDT) as both an explanation
of crime and a method for reducing it. GDT examines why certain individuals engage in deviant,
anti-social, or criminal behaviors. This theory holds that people are fundamentally rational (for
both conforming and deviant behaviors), and that they freely choose deviant behaviors based
on a rational cost-benefit calculation. Because people naturally choose utility-maximizing
behaviors, deviant choices that engender personal gain or pleasure can be controlled by
increasing the costs of such behaviors in the form of punishments (countermeasures) as well as
increasing the probability of apprehension. Swiftness, severity, and certainty of punishments
are the key constructs in GDT.

While classical positivist research in criminology seeks generalized causes of criminal

behaviors, such as poverty, lack of education, psychological conditions, and recommends
strategies to rehabilitate criminals, such as by providing them job training and medical
treatment, GDT focuses on the criminal decision making process and situational factors that
influence that process. Hence, a criminal’s personal situation (such as his personal values, his
affluence, and his need for money) and the environmental context (such as how protected is the
target, how efficient is the local police, how likely are criminals to be apprehended) play key
roles in this decision making process. The focus of GDT is not how to rehabilitate criminals and
avert future criminal behaviors, but how to make criminal activities less attractive and
therefore prevent crimes. To that end, “target hardening” such as installing deadbolts and
building self-defense skills, legal deterrents such as eliminating parole for certain crimes, “three
strikes law” (mandatory incarceration for three offenses, even if the offenses are minor and not
worth imprisonment), and the death penalty, increasing the chances of apprehension using
means such as neighborhood watch programs, special task forces on drugs or gang-related
crimes, and increased police patrols, and educational programs such as highly visible notices
such as “Trespassers will be prosecuted” are effective in preventing crimes. This theory has
interesting implications not only for traditional crimes, but also for contemporary white-collar
crimes such as insider trading, software piracy, and illegal sharing of music.
Chapter 5

Research Design

Research design is a comprehensive plan for data collection in an empirical research

project. It is a “blueprint” for empirical research aimed at answering specific research
questions or testing specific hypotheses, and must specify at least three processes: (1) the data
collection process, (2) the instrument development process, and (3) the sampling process. The
instrument development and sampling processes are described in next two chapters, and the
data collection process (which is often loosely called “research design”) is introduced in this
chapter and is described in further detail in Chapters 9-12.

Broadly speaking, data collection methods can be broadly grouped into two categories:
positivist and interpretive. Positivist methods, such as laboratory experiments and survey
research, are aimed at theory (or hypotheses) testing, while interpretive methods, such as
action research and ethnography, are aimed at theory building. Positivist methods employ a
deductive approach to research, starting with a theory and testing theoretical postulates using
empirical data. In contrast, interpretive methods employ an inductive approach that starts
with data and tries to derive a theory about the phenomenon of interest from the observed
data. Often times, these methods are incorrectly equated with quantitative and qualitative
research. Quantitative and qualitative methods refers to the type of data being collected
(quantitative data involve numeric scores, metrics, and so on, while qualitative data includes
interviews, observations, and so forth) and analyzed (i.e., using quantitative techniques such as
regression or qualitative techniques such as coding). Positivist research uses predominantly
quantitative data, but can also use qualitative data. Interpretive research relies heavily on
qualitative data, but can sometimes benefit from including quantitative data as well.
Sometimes, joint use of qualitative and quantitative data may help generate unique insight into
a complex social phenomenon that are not available from either types of data alone, and hence,
mixed-mode designs that combine qualitative and quantitative data are often highly desirable.

Key Attributes of a Research Design

The quality of research designs can be defined in terms of four key design attributes:
internal validity, external validity, construct validity, and statistical conclusion validity.

Internal validity, also called causality, examines whether the observed change in a
dependent variable is indeed caused by a corresponding change in hypothesized independent
variable, and not by variables extraneous to the research context. Causality requires three
conditions: (1) covariation of cause and effect (i.e., if cause happens, then effect also happens;
and if cause does not happen, effect does not happen), (2) temporal precedence: cause must

35
36 | S o c i a l S c i e n c e R e s e a r c h

precede effect in time, (3) no plausible alternative explanation (or spurious correlation).
Certain research designs, such as laboratory experiments, are strong in internal validity by
virtue of their ability to manipulate the independent variable (cause) via a treatment and
observe the effect (dependent variable) of that treatment after a certain point in time, while
controlling for the effects of extraneous variables. Other designs, such as field surveys, are poor
in internal validity because of their inability to manipulate the independent variable (cause),
and because cause and effect are measured at the same point in time which defeats temporal
precedence making it equally likely that the expected effect might have influenced the expected
cause rather than the reverse. Although higher in internal validity compared to other methods,
laboratory experiments are, by no means, immune to threats of internal validity, and are
susceptible to history, testing, instrumentation, regression, and other threats that are discussed
later in the chapter on experimental designs. Nonetheless, different research designs vary
considerably in their respective level of internal validity.

External validity or generalizability refers to whether the observed associations can be

generalized from the sample to the population (population validity), or to other people,
organizations, contexts, or time (ecological validity). For instance, can results drawn from a
sample of financial firms in the United States be generalized to the population of financial firms
(population validity) or to other firms within the United States (ecological validity)? Survey
research, where data is sourced from a wide variety of individuals, firms, or other units of
analysis, tends to have broader generalizability than laboratory experiments where artificially
contrived treatments and strong control over extraneous variables render the findings less
generalizable to real-life settings where treatments and extraneous variables cannot be
controlled. The variation in internal and external validity for a wide range of research designs
are shown in Figure 5.1.

Cone of Validity
Single Multiple
case study case study Field
experiment
Ethnography
Longitudinal
External
Cross-sectional field survey
validity
field survey
Simulation Multiple lab
experiment
Validity
frontier
Math Single lab
proofs experiment
Internal validity

Figure 5.1. Internal and external validity

Some researchers claim that there is a tradeoff between internal and external validity:
higher external validity can come only at the cost of internal validity and vice-versa. But this is
not always the case. Research designs such as field experiments, longitudinal field surveys, and
multiple case studies have higher degrees of both internal and external validities. Personally, I
prefer research designs that have reasonable degrees of both internal and external validities,
i.e., those that fall within the cone of validity shown in Figure 5.1. But this should not suggest
that designs outside this cone are any less useful or valuable. Researchers’ choice of designs is
 R e s e a r c h D e s i g n | 37

ultimately a matter of their personal preference and competence, and the level of internal and
external validity they desire.

Construct validity examines how well a given measurement scale is measuring the
theoretical construct that it is expected to measure. Many constructs used in social science
research such as empathy, resistance to change, and organizational learning are difficult to
define, much less measure. For instance, construct validity must assure that a measure of
empathy is indeed measuring empathy and not compassion, which may be difficult since these
constructs are somewhat similar in meaning. Construct validity is assessed in positivist
research based on correlational or factor analysis of pilot test data, as described in the next
chapter.

Statistical conclusion validity examines the extent to which conclusions derived using
a statistical procedure is valid. For example, it examines whether the right statistical method
was used for hypotheses testing, whether the variables used meet the assumptions of that
statistical test (such as sample size or distributional requirements), and so forth. Because
interpretive research designs do not employ statistical test, statistical conclusion validity is not
applicable for such analysis. The different kinds of validity and where they exist at the
theoretical/empirical levels are illustrated in Figure 5.2.

Figure 5.2. Different Types of Validity in Scientific Research

Improving Internal and External Validity

The best research designs are those that can assure high levels of internal and external
validity. Such designs would guard against spurious correlations, inspire greater faith in the
hypotheses testing, and ensure that the results drawn from a small sample are generalizable to
the population at large. Controls are required to assure internal validity (causality) of research
designs, and can be accomplished in four ways: (1) manipulation, (2) elimination, (3) inclusion,
and (4) statistical control, and (5) randomization.

In manipulation, the researcher manipulates the independent variables in one or more

levels (called “treatments”), and compares the effects of the treatments against a control group
where subjects do not receive the treatment. Treatments may include a new drug or different
38 | S o c i a l S c i e n c e R e s e a r c h

dosage of drug (for treating a medical condition), a, a teaching style (for students), and so forth.
This type of control is achieved in experimental or quasi-experimental designs but not in non-
experimental designs such as surveys. Note that if subjects cannot distinguish adequately
between different levels of treatment manipulations, their responses across treatments may not
be different, and manipulation would fail.

The elimination technique relies on eliminating extraneous variables by holding them

constant across treatments, such as by restricting the study to a single gender or a single socio-
economic status. In the inclusion technique, the role of extraneous variables is considered by
including them in the research design and separately estimating their effects on the dependent
variable, such as via factorial designs where one factor is gender (male versus female). Such
technique allows for greater generalizability but also requires substantially larger samples. In
statistical control, extraneous variables are measured and used as covariates during the
statistical testing process.

Finally, the randomization technique is aimed at canceling out the effects of extraneous
variables through a process of random sampling, if it can be assured that these effects are of a
random (non-systematic) nature. Two types of randomization are: (1) random selection,
where a sample is selected randomly from a population, and (2) random assignment, where
subjects selected in a non-random manner are randomly assigned to treatment groups.

Randomization also assures external validity, allowing inferences drawn from the
sample to be generalized to the population from which the sample is drawn. Note that random
assignment is mandatory when random selection is not possible because of resource or access
constraints. However, generalizability across populations is harder to ascertain since
populations may differ on multiple dimensions and you can only control for few of those
dimensions.

Popular Research Designs

As noted earlier, research designs can be classified into two categories – positivist and
interpretive – depending how their goal in scientific research. Positivist designs are meant for
theory testing, while interpretive designs are meant for theory building. Positivist designs seek
generalized patterns based on an objective view of reality, while interpretive designs seek
subjective interpretations of social phenomena from the perspectives of the subjects involved.
Some popular examples of positivist designs include laboratory experiments, field experiments,
field surveys, secondary data analysis, and case research while examples of interpretive designs
include case research, phenomenology, and ethnography. Note that case research can be used
for theory building or theory testing, though not at the same time. Not all techniques are suited
for all kinds of scientific research. Some techniques such as focus groups are best suited for
exploratory research, others such as ethnography are best for descriptive research, and still
others such as laboratory experiments are ideal for explanatory research. Following are brief
descriptions of some of these designs. Additional details are provided in Chapters 9-12.

Experimental studies are those that are intended to test cause-effect relationships
(hypotheses) in a tightly controlled setting by separating the cause from the effect in time,
administering the cause to one group of subjects (the “treatment group”) but not to another
group (“control group”), and observing how the mean effects vary between subjects in these
two groups. For instance, if we design a laboratory experiment to test the efficacy of a new drug
in treating a certain ailment, we can get a random sample of people afflicted with that ailment,
 R e s e a r c h D e s i g n | 39

randomly assign them to one of two groups (treatment and control groups), administer the
drug to subjects in the treatment group, but only give a placebo (e.g., a sugar pill with no
medicinal value). More complex designs may include multiple treatment groups, such as low
versus high dosage of the drug, multiple treatments, such as combining drug administration
with dietary interventions. In a true experimental design, subjects must be randomly
assigned between each group. If random assignment is not followed, then the design becomes
quasi-experimental. Experiments can be conducted in an artificial or laboratory setting such
as at a university (laboratory experiments) or in field settings such as in an organization where
the phenomenon of interest is actually occurring (field experiments). Laboratory experiments
allow the researcher to isolate the variables of interest and control for extraneous variables,
which may not be possible in field experiments. Hence, inferences drawn from laboratory
experiments tend to be stronger in internal validity, but those from field experiments tend to be
stronger in external validity. Experimental data is analyzed using quantitative statistical
techniques. The primary strength of the experimental design is its strong internal validity due
to its ability to isolate, control, and intensively examine a small number of variables, while its
primary weakness is limited external generalizability since real life is often more complex (i.e.,
involve more extraneous variables) than contrived lab settings. Furthermore, if the research
does not identify ex ante relevant extraneous variables and control for such variables, such lack
of controls may hurt internal validity and may lead to spurious correlations.

Field surveys are non-experimental designs that do not control for or manipulate
independent variables or treatments, but measure these variables and test their effects using
statistical methods. Field surveys capture snapshots of practices, beliefs, or situations from a
random sample of subjects in field settings through a survey questionnaire or less frequently,
through a structured interview. In cross-sectional field surveys, independent and dependent
variables are measured at the same point in time (e.g., using a single questionnaire), while in
longitudinal field surveys, dependent variables are measured at a later point in time than the
independent variables. The strengths of field surveys are their external validity (since data is
collected in field settings), their ability to capture and control for a large number of variables,
and their ability to study a problem from multiple perspectives or using multiple theories.
However, because of their non-temporal nature, internal validity (cause-effect relationships)
are difficult to infer, and surveys may be subject to respondent biases (e.g., subjects may
provide a “socially desirable” response rather than their true response) which further hurts
internal validity.

Secondary data analysis is an analysis of data that has previously been collected and
tabulated by other sources. Such data may include data from government agencies such as
employment statistics from the U.S. Bureau of Labor Services or development statistics by
country from the United Nations Development Program, data collected by other researchers
(often used in meta-analytic studies), or publicly available third-party data, such as financial
data from stock markets or real-time auction data from eBay. This is in contrast to most other
research designs where collecting primary data for research is part of the researcher’s job.
Secondary data analysis may be an effective means of research where primary data collection is
too costly or infeasible, and secondary data is available at a level of analysis suitable for
answering the researcher’s questions. The limitations of this design are that the data might not
have been collected in a systematic or scientific manner and hence unsuitable for scientific
research, since the data was collected for a presumably different purpose, they may not
adequately address the research questions of interest to the researcher, and interval validity is
problematic if the temporal precedence between cause and effect is unclear.
40 | S o c i a l S c i e n c e R e s e a r c h

Case research is an in-depth investigation of a problem in one or more real-life settings

(case sites) over an extended period of time. Data may be collected using a combination of
interviews, personal observations, and internal or external documents. Case studies can be
positivist in nature (for hypotheses testing) or interpretive (for theory building). The strength
of this research method is its ability to discover a wide variety of social, cultural, and political
factors potentially related to the phenomenon of interest that may not be known in advance.
Analysis tends to be qualitative in nature, but heavily contextualized and nuanced. However,
interpretation of findings may depend on the observational and integrative ability of the
researcher, lack of control may make it difficult to establish causality, and findings from a single
case site may not be readily generalized to other case sites. Generalizability can be improved by
replicating and comparing the analysis in other case sites in a multiple case design.

Focus group research is a type of research that involves bringing in a small group of
subjects (typically 6 to 10 people) at one location, and having them discuss a phenomenon of
interest for a period of 1.5 to 2 hours. The discussion is moderated and led by a trained
facilitator, who sets the agenda and poses an initial set of questions for participants, makes sure
that ideas and experiences of all participants are represented, and attempts to build a holistic
understanding of the problem situation based on participants’ comments and experiences.
Internal validity cannot be established due to lack of controls and the findings may not be
generalized to other settings because of small sample size. Hence, focus groups are not
generally used for explanatory or descriptive research, but are more suited for exploratory
research.

Action research assumes that complex social phenomena are best understood by
introducing interventions or “actions” into those phenomena and observing the effects of those
actions. In this method, the researcher is usually a consultant or an organizational member
embedded within a social context such as an organization, who initiates an action such as new
organizational procedures or new technologies, in response to a real problem such as declining
profitability or operational bottlenecks. The researcher’s choice of actions must be based on
theory, which should explain why and how such actions may cause the desired change. The
researcher then observes the results of that action, modifying it as necessary, while
simultaneously learning from the action and generating theoretical insights about the target
problem and interventions. The initial theory is validated by the extent to which the chosen
action successfully solves the target problem. Simultaneous problem solving and insight
generation is the central feature that distinguishes action research from all other research
methods, and hence, action research is an excellent method for bridging research and practice.
This method is also suited for studying unique social problems that cannot be replicated outside
that context, but it is also subject to researcher bias and subjectivity, and the generalizability of
findings is often restricted to the context where the study was conducted.

Ethnography is an interpretive research design inspired by anthropology that

emphasizes that research phenomenon must be studied within the context of its culture. The
researcher is deeply immersed in a certain culture over an extended period of time (8 months
to 2 years), and during that period, engages, observes, and records the daily life of the studied
culture, and theorizes about the evolution and behaviors in that culture. Data is collected
primarily via observational techniques, formal and informal interaction with participants in
that culture, and personal field notes, while data analysis involves “sense-making”. The
researcher must narrate her experience in great detail so that readers may experience that
same culture without necessarily being there. The advantages of this approach are its
sensitiveness to the context, the rich and nuanced understanding it generates, and minimal
 R e s e a r c h D e s i g n | 41

respondent bias. However, this is also an extremely time and resource-intensive approach, and
findings are specific to a given culture and less generalizable to other cultures.

Selecting Research Designs

Given the above multitude of research designs, which design should researchers choose
for their research? Generally speaking, researchers tend to select those research designs that
they are most comfortable with and feel most competent to handle, but ideally, the choice
should depend on the nature of the research phenomenon being studied. In the preliminary
phases of research, when the research problem is unclear and the researcher wants to scope
out the nature and extent of a certain research problem, a focus group (for individual unit of
analysis) or a case study (for organizational unit of analysis) is an ideal strategy for exploratory
research. As one delves further into the research domain, but finds that there are no good
theories to explain the phenomenon of interest and wants to build a theory to fill in the unmet
gap in that area, interpretive designs such as case research or ethnography may be useful
designs. If competing theories exist and the researcher wishes to test these different theories or
integrate them into a larger theory, positivist designs such as experimental design, survey
research, or secondary data analysis are more appropriate.

Regardless of the specific research design chosen, the researcher should strive to collect
quantitative and qualitative data using a combination of techniques such as questionnaires,
interviews, observations, documents, or secondary data. For instance, even in a highly
structured survey questionnaire, intended to collect quantitative data, the researcher may leave
some room for a few open-ended questions to collect qualitative data that may generate
unexpected insights not otherwise available from structured quantitative data alone. Likewise,
while case research employ mostly face-to-face interviews to collect most qualitative data, the
potential and value of collecting quantitative data should not be ignored. As an example, in a
study of organizational decision making processes, the case interviewer can record numeric
quantities such as how many months it took to make certain organizational decisions, how
many people were involved in that decision process, and how many decision alternatives were
considered, which can provide valuable insights not otherwise available from interviewees’
narrative responses. Irrespective of the specific research design employed, the goal of the
researcher should be to collect as much and as diverse data as possible that can help generate
the best possible insights about the phenomenon of interest.
 Technological Forecasting & Social Change 130 (2018) 99–113

Contents lists available at ScienceDirect

Technological Forecasting & Social Change

journal homepage: [Link]/locate/techfore

Big Data sources and methods for social and economic analyses T
*
Desamparados Blazquez, Josep Domenech
Department of Economics and Social Sciences, Universitat Politècnica de València, Camí de Vera s/n., Valencia 46022, Spain

A R T I C L E I N F O A B S T R A C T

Keywords: The Data Big Bang that the development of the ICTs has raised is providing us with a stream of fresh and
Big Data architecture digitized data related to how people, companies and other organizations interact. To turn these data into
Forecasting knowledge about the underlying behavior of the social and economic agents, organizations and researchers must
Nowcasting deal with such amount of unstructured and heterogeneous data. Succeeding in this task requires to carefully plan
Data lifecycle
and organize the whole process of data analysis taking into account the particularities of the social and economic
Socio-economic data
Non-traditional data sources
analyses, which include the wide variety of heterogeneous sources of information and a strict governance policy.
Non-traditional analysis methods Grounded on the data lifecycle approach, this paper develops a Big Data architecture that properly integrates
most of the non-traditional information sources and data analysis methods in order to provide a specifically
designed system for forecasting social and economic behaviors, trends and changes.

1. Introduction extended to 4Vs, adding the “Value” dimension (process to extract

valuable information from data, known as Big Data Analytics). Cur-
What comes to your mind when talking about “The Digital Era”? For rently, the “Big Data” concept is starting to be defined in terms of the
sure, concepts as the “Internet”, “Smartphones” or “Smart sensors” 5Vs model (Bello-Orgaz et al., 2016), which added the “Veracity” di-
arise. These technologies are progressively being used in most of the mension (related to proper data governance and privacy concerns).
everyday activities of companies and individuals. For instance, many This new data paradigm is called to transform the landscape for
companies conduct marketing campaigns through social networks, sell socio-economic policy and research (Einav and Levin, 2014; Varian,
their products online, monitor the routes followed by sales re- 2014) as well as for business management and decision-making. Thus,
presentatives with smartphones or register the performance of ma- identifying which data sources are available, what type of data they
chinery with specific sensors. At the other side, individuals make use of provide, and how to treat these data is basic to generate as much value
computers, smartphones and tablets in order to buy products online, as possible for the company or organization. In this context, a Big Data
share their opinions, chat with friends or check the way to some place. architecture adapted to the specific domain and purpose of the orga-
Moreover, citizens' movements and activities are daily registered by nization contributes to systematize the process of generating value. This
sensors placed in any part of cities or roads and in public places such as architecture should be capable of managing the complete data lifecycle
supermarkets. in the organization, including data ingestion, analysis and storage,
Therefore, all of these technologies are generating tons of digitized among others.
and fresh data about people and firms' activities that properly analyzed, Furthermore, the design of a Big Data architecture should consider
could help reveal trends and monitor economic, industrial and social the numerous challenges that this paradigm implies. These include:
behaviors or magnitudes. These data are not only updated, but also scalability, data availability, data integrity, data transformation, data
massive, given that daily data generation has been recently estimated in quality, data provenance (related to generation of right metadata that
2.5 Exabytes (IBM, 2016). For this reason, they are commonly referred identify the origin of data as well as the processes applied to them
to as “Big Data”, concept which first appeared in the late 90s (Cox and during the data lifecycle, to assure traceability), management of huge
Ellsworth, 1997) and was defined in the early 2000s in terms of the 3Vs volumes of information, data heterogeneity (structured and un-
model (Laney, 2001), which refers to: Volume (size of data), Velocity structured, with different time frequencies), integration of data from
(speed of data transfers), and Variety (different types of data, ranging different sources, data matching, bias, availability of tools for properly
from video to data logs for instance, and with different structures). This analyzing such kind of data, processing complexity, privacy and legal
model evolved to adapt to the changing digital reality, so that it was issues, and data governance (Fan et al., 2014; Jagadish et al., 2014;

*
Corresponding author.
E-mail addresses: mdeblzso@[Link] (D. Blazquez), jdomenech@[Link] (J. Domenech).

[Link]
Received 1 March 2017; Received in revised form 7 July 2017; Accepted 25 July 2017
Available online 08 September 2017
0040-1625/ © 2017 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license ([Link]
D. Blazquez, J. Domenech Technological Forecasting & Social Change 130 (2018) 99–113

Hashem et al., 2015). cases. This work describes a number of functionalities expected to be
The Big Data paradigm also offers many advantages and benefits for considered when designing a Big Data architecture for a specific
the companies, governments, and the society. Jin et al. (2015) highlight knowledge field, business or industrial process. These include: Data
its potential contribution to national and industrial development, as it sources, data extraction, data loading and preprocessing, data proces-
enforces to change and upgrade research methods, promotes and makes sing, data analysis, data transformation, interfacing and visualization,
it easy to conduct interdisciplinary research, helps to nowcast the data storage and model specification. Besides that, Assunção et al.
present and to forecast the future more precisely. In this vein, first Big (2015) reflected on some components that should be present in any Big
Data architectures designed for specific fields are being proposed in Data architecture by depicting the four most common phases within a
order to surpass the previously mentioned challenges and make the Big Data analytics workflow: Data sources, data management (including
most of the data available with the aim of nowcasting and forecasting tasks such as preprocessing and filtering), modelling, and result analysis
variables of interest. and visualization. This scheme was put in relation to cloud computing,
However, no specific architecture for social and economic fore- whose potential and benefits for storing huge amounts of data and
casting has been proposed yet. This emerges as a necessity, in the one performing powerful calculus are positioning it as a desirable tech-
hand, because of the particular nature of socio-economic data, which nology to be included in the design of a Big Data architecture. Con-
have important components of uncertainty and human behavior that cretely, the role of cloud computing as part of a Big Data system has
are particularly complex to model; and, in the other hand, because of been explored by Hashem et al. (2015).
the great benefits that can be derived from the use of Big Data to About architectures for specific domains, Zhang et al. (2017) pro-
forecast economic and social changes. For instance, Big Data ap- posed a Big Data analytics architecture with the aim of exploiting in-
proaches have been proved to improve predictions of economic in- dustrial data to achieve cleaner production processes and optimize the
dicators such as the unemployment level (Vicente et al., 2015), help product lifecycle management. This architecture works in four main
managers detect market trends so that they can anticipate opportu- stages: in stage 1, services of product lifecycle management, such as
nities, and also help policy-makers monitor faster and more precisely design improvement, are applied; in stage 2, the architecture acquires
the effects of a wide range of policies and public grants (Blazquez and and integrates Big Data from different industrial sources, such as sen-
Domenech, 2017). sors; in stage 3, Big Data is processed and stored depending on their
In this context, this paper aims to i) establish a framework about the structure; finally, in stage 4, Big Data mining and knowledge discovery
new and potentially useful available sources of socio-economic data and is conducted by means of four layers: the data layer (mixing data), the
new methods devoted to deal with these data, ii) propose a new data method layer (data extraction), the result layer (data mining) and the
lifecycle model that encompasses all the processes related to working application layer (meeting the demands of the enterprise). Results from
with Big Data, and iii) propose an architecture for a Big Data system last stage fill the ERP systems and are used along with decision support
able to integrate, process and analyze data from different sources with systems to improve product-related services and give feedback in all
the objective to forecast economic and social changes. product lifecycle stages.
The remainder of the paper is organized as follows: Section 2 re- In the domain of healthcare, a complete and specific Big Data
views the Big Data architectures proposed in the literature; Section 3 analytics architecture was developed by Wang et al. (2016a). This ar-
compiles the new socio-economic data sources emerged in the Digital chitecture was based on the experiences about best practices in im-
Era and proposes a classification of them; Section 4 reviews the new plementing Big Data systems in the industry, and was composed of five
methods and analytics designed to deal with Big Data and establishes a major layers: first, the data layer, which includes the data sources to be
taxonomy of these methods; Section 5 depicts the data lifecycle on used for supporting operations and problem solving; second, the data
which the proposed Big Data architecture is based; Section 6 proposes a aggregation layer, which is in charge of acquiring, transforming and
Big Data architecture for nowcasting social and economic variables, storing data; third, the analytics layer, which is in charge of processing
explaining its different modules; finally, Section 7 draws some con- and analyzing data; fourth, the information exploration layer, which
cluding remarks. works by generating outputs for clinical decision support, such as real-
time monitoring of potential medical risks; last, the data governance
2. Related work layer, which is in charge of managing business data throughout its
entire lifecycle by applying the proper standards and policies of security
Since the advent of the concept of “Big Data” two decades ago, some and privacy. This layer is particularly necessary in this case given the
architectures to manage and analyze such data in different fields have sensibility of clinical data.
been proposed, having their technical roots in distributed computing The review of these architectures evidenced some common modules
paradigms such as grid computing (Berman et al., 2003). However, the or functionalities. After homogenizing the different names for modules
current data explosion also referred to as “Data Big Bang” (Pesenson very similar responsibilities, and considering their sequence in the
et al., 2010) in which there is a daily generation of vast quantities of process, they can be summarized as follows: first, a data module, which
data from a variety of formats and sources, is revealing the fullest includes different sources of data with different formats; second, a data
meaning of “Big Data”. preprocessing module, which includes data extraction, integration and
The particular properties and challenges that the current Big Data transformation; third, a data analytics module, which includes model-
context opens require specific architectures for information systems ling and analysis techniques for knowledge discovery; and fourth, a
particularly designed to retrieve, process, analyze and store such vo- results and visualization module, which includes tools for representing
lume and variety of data. Therefore, we are living the constant births of the results in a way useful for the firm or organization.
new technologies conceived to be useful in this context such as, to However, there are other functionalities whose location within the
mention some, cloud and exascale computing (Bahrami and Singhal, Big Data architecture is not homogeneous across the different propo-
2014; Reed and Dongarra, 2015). Given this recent technological and sals. For instance, the data storage responsibilities, which are basic for
data revolution, research in this topic is in its early stage (Chen et al., enabling data reuse and bringing access to previous results, have been
2014). In this section, we review the novel and incipient research works included in a variety of places, ranging from being included in the data
that develop general frameworks and specific architectures for adopting module (Assunção et al., 2015) or the preprocessing module (Wang
the Big Data approach in different fields from the point of view of data et al., 2016a; Zhang et al., 2017), to being a macro-functionality present
analytics applications. in each module of the architecture (Pääkkönen and Pakkala, 2015). The
Pääkkönen and Pakkala (2015) proposed a reference architecture last approach is better reflecting the nature and complexity of Big Data
for Big Data systems based on the analysis of some implementation analysis, given that not only the original data requires storage, but also

100
D. Blazquez, J. Domenech Technological Forecasting & Social Change 130 (2018) 99–113

the integrated data, processed data, and the results derived from data behaviors (Askitas and Zimmermann, 2015).
analytics.
Other functionalities whose consideration in the literature has been 3.1.1. Google Trends: the power of search engines
divergent are those related to data governance, which is concerned to Google Trends (GT) is an Internet-based facility, released on May
preserve privacy, security and assure the accomplishment of data-re- 2006, which provides up-to-date reports on the volume of search
lated regulations. Despite its importance, data governance was only queries on a specific keyword or text, with historic searches available
considered by Wang et al. (2016a). As long as the 5Vs model expands, since January 2004. It captures how the demand of information under
data governance is expected to gain relevance and become a require- certain topics varies over time, providing useful data to detect emerging
ment in the design of any Big Data architecture. trends and underlying interests and concerns of society. The use of GT
For the case of Big Data for social or economic domains, no specific data to nowcast social and economic (particularly macroeconomic)
architecture has been proposed yet in the literature. Given their parti- variables was introduced by Choi and Varian (2009a,b), who showed
cular characteristics and increasing potential for detecting and mon- that some search categories in the Google search engine helped to
itoring behaviors and trends, which is basic to anticipate events, design predict car and home sales, incoming tourists or unemployment claims.
better action plans and make more informed decisions, an architecture Afterwards, various studies in different countries focused on improving
specifically devoted to treat these data emerges as necessary. Thus, this unemployment-related variables' forecasts by using GT data, obtaining
work proposes a Big Data architecture designed for nowcasting and successful results (Askitas and Zimmermann, 2009; McLaren and
forecasting social and economic changes. This proposal aims to help Shanbhogue, 2011; Fondeur and Karamé, 2013; Vicente et al., 2015).
business implement the most appropriate architecture for their decision The aggregate consumer behavior in different sectors has also been
making needs, make the most of the data available and assure that it is successfully predicted with GT data. For instance, using GT data as
treated according to the ethic and legal standards. predictors has been proved to improve forecasts of tourist inflows
(Artola et al., 2015; Bangwayo-Skeete and Skeete, 2015), of trading
3. Non-traditional sources of social and economic data decisions and transaction volumes on the stock market (Preis et al.,
2013; Moat et al., 2014), of private purchases of different goods and
The digital footprint left by individuals has caused an exponential services (Vosen and Schmidt, 2011) or of cinema admissions (Hand and
growth of the data sources available for social and economic analyses, Judge, 2012). Recently, GT data have proven to be useful for fore-
which broadens the possibilities for conducting socio-economic studies casting political inquiries' results (Mavragani and Tsagarakis, 2016).
beyond traditional data sources, namely surveys and official records. However, elections results and topics with such components of opinion
Although the reasons why these new data are generated are numerous, and ideology have been particularly studied through data from sites
the way they are generated has important ethical and legal implica- focused on social interaction, as are Social Networking Sites (SNS) such
tions. For instance, personal data in a purchase order cannot be used for as Facebook and Twitter and opinion platforms such as Ciao.
the same purposes as the data from a public profile in Twitter. To some
extent, the usage of the data is limited by how they are generated. This
3.1.2. Social Networking Sites and blogs
fact motivated us to review and classify the newborn non-traditional
SNS are online places specifically addressed to encourage users
sources of social and economic data according to the purpose of the user
express their feelings and opinions about any kind of topic. Therefore,
generating the data, as Fig. 1 shows.
the information they contain is to some extent a reflection of what
The first level in the taxonomy includes five categories: i) purpose of
happens in society. Indeed, the term “Social Big Data” is becoming
searching for information; ii) purpose of conducting a transaction,
popular to refer to data generated by SNS and blogs (Bello-Orgaz et al.,
which could be of a financial or non-financial nature; iii) purpose of
2016). For that reason, more attention is being paid to SNS as sources of
disseminating information; iv) purpose of doing a social interaction;
data potentially useful in forecasting social variables.
and v) not a deliberate purpose. The first four categories correspond to
Among SNS, the microblogging service Twitter is one of the most
an active generation of data, while the last correspond to an inactive
popular, with 332 million users who are active monthly and send on
generation: that is, data is not intentionally generated as a result of a
average more than 500 million tweets per day. This huge amount of
particular purpose, but just derived from the own use of any device (PC,
“user-generated” information, though implies some issues, weaknesses
smartphone, tablet…) with any of the purposes explained above. Data
and challenges that require further research (Gayo-Avello, 2013;
that fall in this category have been divided in three types: usage data,
Schoen et al., 2013), could help to predict both present and future so-
location data and personal data. A brief description of each purpose
cial and economic events, as verified in different works. For instance,
from which data is generated and examples of sources involved in each
tweets' contents have helped to describe political preferences and
data generation process is shown in Table 1.
forecast elections results (Tumasjan et al., 2011; Kim and Park, 2012;
The majority of non-traditional sources of social and economic data
Ceron et al., 2014), to predict stock market movements (Bollen et al.,
mentioned above needs the Internet for working. Indeed, the increasing
2011), to forecast box office in the motion pictures industry (Kim et al.,
penetration and importance of the Internet in almost every social and
2015; Gaikar et al., 2015) or to monitor the public opinion on new
economic activity has positioned it as a basic means for the generation
policies (Ceron and Negri, 2016).
of such kind of data.
Facebook, which is the third most visited site worldwide1 with
1,650 million active users, doubtlessly also represents a source of
3.1. The Internet as basic means for generating socio-economic data
powerful data for analyzing social and economic behaviors. However,
given that its contents are more heterogeneous and user-adjustable,
The “Data Big Bang” originated in the Internet, which unstoppable
they are also more difficult to retrieve and analyze. Notwithstanding
expands, is transforming the way of interacting in the economic and
this, incipient studies have shown the ability of Facebook data to de-
social framework. Myriad individuals, companies and public organisms
termine consuming profiles, which are useful for marketing purposes
search, post and generate tons of information daily through the
(Arrigo et al., 2016), and to predict election results and the population's
Internet. These online activities leave behind a digital footprint that can
political orientation (Cameron et al., 2016; David et al., 2016).
be tracked and, if treated with the proper Big Data architecture, could
Other principal SNS are LinkedIn, Youtube, Instagram, Google+,
help to describe their behavior, decisions and intentions, and thus, to
Tumblr and Flickr (Bello-Orgaz et al., 2016). They are also rich sources
monitor key economic and social changes and trends. Indeed, recent
research highlighted the increasing role of the Internet as a provider of
data for explaining, modelling, nowcasting and forecasting social 1
[Link].

101
D. Blazquez, J. Domenech Technological Forecasting & Social Change 130 (2018) 99–113

Fig. 1. Taxonomy of non-traditional sources of social and economic data.

Table 1
Classification of sources of socio-economic Big Data.

User's purpose Description Examples of sources

Information search The user aims to find information about a topic of his interest. Data is actively generated Search engines, Google Trends
Transactions The user interacts with an individual and/or machine to achieve an agreement in which
the user demands and obtains a product or service in exchange for a financial or non-
financial compensation. Data is actively generated.
• Financial transactions Event in which the user makes a payment to obtain a product or service E-banking, e-commerce, urban sensors (tolls, credit card
readers, retail scanners, public transport card readers)
• Non-financial Event in which the user provides the counterpart with required information to obtain a E-government, e-recruiting
transactions product or service
Information diffusion The user aims to spread information or knowledge. This includes marketing purposes, in Corporate websites, apps, Wiki pages
order to establish a public image of the user or the agent he represents. Data is actively
generated.
Social interaction The user wants to share information, opinions and ideas with other users. Data is Social Networking Sites, opinion platforms, blogs
actively generated.
Non-deliberate The user does not pursue to generate data with his/her action, but data are generated by
the use of some means. Data is passively generated as a result of any other user action.
• Usage The simple fact of using any device generates data related to how, when and where an Web cookies, Internet Protocol, Sensors for self-tracking
action has been done.
• Location The use of mobile phones generates data particularly related to the position of the user. GPS, GSM, Call Detail Records, Bluetooth, WiFi Points
• Personal Personal data (age, sex, etc.) is generated consciously (e.g. filling a form to complete a Forms, profiles, type of searches or purchases
purchase) or unconsciously (e.g. data about the type of information we look for is used
to infer our incomes) as a consequence of using any device or tool to achieve a purpose.

of social and economic data, which could eventually be used to find opinion in real-time. Nevertheless, these sources are not without lim-
changes in the unemployment patterns or detect what entertainment itations. It is common that they are biased towards one segment of the
activities people prefer, among other topics (Russell, 2013). However, population, e.g., young people, and English language, e.g., blogs in non-
the diverse and complex formats of the information provided, along English language link more frequently English content than the other
with the newness in some of these SNS, makes them remained almost way round (Thelwall, 2007). Thus, some correcting measures should be
unexplored. It should be noted that blogs are also important generators considered before generalization (Gayo-Avello, 2012).
of “Social Big Data”, though research in relating blogs' data to fore-
casting is also in its early stage. The pioneer work of Liu et al. (2007)
3.1.3. Websites and apps: transactional, opinion platforms and information
examined the usefulness of opinions and sentiments extracted from
diffusion
blogs to forecast sales performance, while more recently Saleiro et al.
In the Digital Era, firms generally establish their official public
(2015) combined data from news, blogs and SNS to track political
image on the Internet by implementing corporate websites. Through

102
D. Blazquez, J. Domenech Technological Forecasting & Social Change 130 (2018) 99–113

these sites, companies inform about their products, services, organiza- given that their comments may influence other consumers' purchase
tional structure and intentions, such as exporting and opening a branch behavior and, thus, detecting and monitoring them is essential. For
office abroad. Corporate websites encompass all kind of websites im- instance, Arenas-Márquez et al. (2014) successfully identified influen-
plemented by firms in relation to their economic activity, ranging from cers in [Link] by retrieving and analyzing characteristics of the
websites used only to give information about the firm, to transactional product reviews such as the rating received by other users.
websites devoted not only to provide information but also to offer on- Other sites that provide potentially useful data for detecting social
line services (e-commerce, e-banking…), about which users are some- and economic trends are, for instance, [Link], whose information
times allowed to give their opinion in the website itself. That is, cor- has been helpful to explain price differences among remanufactured,
porate websites may present three different functionalities: spreading used and new items (Frota Neto et al., 2016), [Link], which
information about firms (related to establishing a public image), con- has been successfully used to detect tourist preferences thus helping
ducting transactions (e-business processes) and facilitating opinion hotel managers to adapt their offers (Li et al., 2015), and [Link],
sharing (electronic word-of-mouth (eWOM) booster). which organizes the available job offers and helps to track changes in
It is remarkable that websites have a complex structure which differ job search (Edelman, 2012).
from one case to another, so that standardizing the retrieval and ana- When using opinion platforms as sources for social and economic
lysis of their information requires from a specific Big Data architecture. analyses, limitations related to the veracity of the contents must be
That difficulty has contributed to corporate websites being an almost considered. Sellers and marketers may have the temptation to generate
unexplored source of data. However, their public, updated and “busi- fake consumer reviews to influence in the consumer decision (Malbon,
ness generated” nature makes them potential sources of economic data. 2013). In this context, some techniques for detecting such manipula-
Moreover, business characteristics could emerge on the web and be tions could be applied to alleviate this limitation (Hu et al., 2012).
monitored by massively analyzing corporate websites, as recent re- Apps provide access to information and services that may or may
search shows. not be offered by other means, such as websites. Since the use of apps is
Applying Big Data approaches (particularly web data mining and becoming widespread in the daily activities of individuals and organi-
machine learning) to the “spreading information” functionality of cor- zations, they have become a source of data with great potential for
porate websites, firms' sales growth and business activities such as forecasting social and economic topics. Although accessing data gen-
strategies of technology adoption, innovation and R & D have been erated by them is currently a difficult task, some incipient research
successfully detected (Arora et al., 2016, 2013; Gök et al., 2015; Li works are appearing. To date, apps data logs have been proved to be
et al., 2016). In addition, by using a specifically designed web data successful for forecasting users' intentions to use a specific app, auto-
mining system for analyzing corporate websites (Domenech et al., matically forecasting depression (Wang et al., 2016b; Suhara et al.,
2012) the export orientation of firms has also been successfully de- 2017) or helping to detect mobility patterns as reviewed by Pan and
tected (Blazquez and Domenech, 2017). In addition, there exist other Yang (2016).
type of websites created with the specific aim of spreading information,
such as are Wiki pages, from which Wikipedia is the most important 3.2. Urban and mobile sensors
representative nowadays with more than 730 million unique visitors
monthly (Wikimedia Foundation, 2017). Its penetration in the society Ubiquitous computing is one of the technological areas that has
along with its collaborative nature have positioned it as a potential experimented the greatest development in the context of the Digital
source of social and behavioral data. Concretely, Wikipedia page views, Era. Its advances have resulted in the generation of wireless, incon-
edits and contents have already proven to be useful for socio-economic spicuous and inexpensive sensors to gather information on our ev-
forecasting. Incipient research works have successfully used Wikipedia eryday life activities (Krishnan and Cook, 2014). Specifically, urban
data to better forecast stock market movements (Moat et al., 2014) and sensors and mobile embedded sensors are potential generators of social
tourism demand (Alis et al., 2015; Khadivi and Ramakrishnan, 2016). and economic data.
This kind of studies aim to create new indicators in advance or to Among urban sensors, one of the most widespread and used
complement those used in current official statistics. worldwide is the credit card reader. Credit card transactions are re-
The prominent role of the Internet in today's economy and society corded and provide data potentially useful for firms to detect and
has promoted the emergence of e-business services, which firms can use predict, for instance, personal bankruptcy (Xiong et al., 2013), frau-
to sell their products and do transactions in an online base with cus- dulent purchases in online stores (Van Vlasselaer et al., 2015) and de-
tomers (E-commerce), recruit candidates (E-recruiting) or offer their fault and repayment, which in the context of credit card companies is
services online (e.g. E-banking). E-business may even go a step further useful for defining marketing strategies (Einav and Levin, 2014).
and represent not only a complementary tool for firms (e.g., selling Retail scanners are also very extended, and their function is to re-
locally and online), but a new type of business model characterized by cord the characteristics of customers' everyday purchases. These data
operating just online. Many of these sites offer users the chance to post has proven to be useful for forecasting consumer behaviors, sales and
their opinions and do reviews on the product or service acquired, which prices, as recent research shows. For instance, Dey et al. (2014) suc-
may range from any manufacture to a hotel stay or an experience in a cessfully used retail level scanner data to model market trends, prices
restaurant. This web feature is generally known as opinion platform and sales in the industry of catfish products, suggesting a particular
(even a website can be designed just to act as opinion platform), which competition strategy based on the results obtained. Another study, fo-
is used to bring together online communities of users, whose opinions cused on explaining human behavior, employed weekly scanner data to
are basic information for social science research. detect consumer boycotts in response to an international conflict
One of the most important e-commerce and opinion platform (Pandya and Venkatesan, 2016).
worldwide is Amazon. It is one of the biggest online retailers, with more A pioneer study by Askitas and Zimmermann (2013) successfully
than 300 million active customers' accounts. This website provides used data from tolls to nowcast business cycles, creating a Toll Index
customers' reviews and opinions on millions of products and services, that represents a technological, innovation-driven economic telemetry.
being therefore a source of data potentially useful to detect consumer Other sensor networks that provide useful data for forecasting a
preferences or predict sales. For instance, the forecast of consumer manifold of socio-economic variables are smart grid, WiFi access points
product demands in [Link] has been significantly improved by and public transport card readers, among others (Kitchin, 2014; Chou
using the textual contents of consumer reviews (Chong et al., 2015; and Ngo, 2016).
Schneider and Gupta, 2016). Another noteworthy topic for managers is Some sensors embedded in mobile phones are also potential sources
to detect the so-called “influencers” in consumer-opinion platforms, of social data: GSM, GPS, Bluetooth, accelerometer or sensors for

103
D. Blazquez, J. Domenech Technological Forecasting & Social Change 130 (2018) 99–113

connecting to the telephonic network through Base Transceiver Stations information can be more easily extracted. Some interesting NLP tech-
(which produce the so-called “Call Detail Records”, with information niques for social analysis are Sentiment Analysis (also referred to as
regarding all call-related activities, such as sending SMS and phoning, Opinion Mining), Latent Semantic Analysis (LSA), Latent Dirichlet Al-
conducted by mobile phone users in the network). These sensors gen- location (LDA), TF-IDF (Term Frequency - Inverse Document Fre-
erate data related to the user location that have been successfully used quency) and Word Embeddings. Liu (2012), Evangelopoulos et al.
to study social behaviors, preferences and mobility patterns. Properly (2012), Blei et al. (2003), Moro et al. (2015), Armentano et al. (2014)
treated, these data can contribute to better understand in which way and Rudolph et al. (2016), among others, provide some reference for
human mobility affects well-being and human behaviors at the micro these methods.
level, and social organization and change at the macro level (Williams Linking records from the same user (or entity) across different data
et al., 2015). sources is also an important challenge for analyzing social and eco-
Concretely, data from such sensors have been useful for detecting nomic information. Data Matching (which is also commonly known as
places of interest, that is, places where people go and stay for a while Record Linkage or Entity Resolution) is a computational process used to
(Montoliu et al., 2013), and for detecting personality traits, which identify, match and merge records from several databases that corre-
companies may use to personalize their services (Chittaranjan et al., spond to the same entities. A special case of data matching is dedu-
2013). Moreover, Laurila et al. (2013) summarized different human plication, which consists in the identification and matching of records
behaviors analyzed to date with such mobile embedded sensors data, about the same entities within just one database (this step is crucial in
including: mobility patterns and their relation with the weather, the data cleaning). Matched data are becoming more important because
perceived level of safeness and intimacy of a given location, the relation they may contain information impossible to obtain by means of other
among moves from individuals and from their friends and acquain- sources or processes. This technique is a complex process encompassing
tances, and the transition between spatial habitats. Other recent ap- five steps, from data cleaning and standardization to data quality and
plications of mobile phones' data in relation to mobility are recreating completeness measuring. For a detailed description, see the work by
and drawing maps of population distribution (Deville et al., 2014; Vatsalan et al. (2013).
Graells-Garrido et al., 2016) and detecting anomalous behavioral pat-
terns associated to emergency (e.g. earthquakes) and non-emergency 4.2. Methods for modelling data
(e.g. holidays) events (Dobra et al., 2015).
Modelling data (and their relationships) is the main process in a Big
4. Non-traditional methods for processing social and economic Data analysis. This includes reducing the dimensionality of data sets,
data applying modelling techniques to data and obtaining outcomes.
Depending on the type of data available and the objective of the ana-
Data obtained from non-traditional socio-economic sources are lysis, two different paradigms for modelling data may be applied:
generally large, heterogeneous and unstructured or semi-structured. Supervised Learning and Unsupervised Learning (Hastie et al., 2013).
These characteristics imply a number of challenges when it comes to On the one hand, Supervised Learning refers to problems in which
retrieving, processing, analyzing and storing data. Accordingly, each observation in a data set has inputs (also referred to as in-
methods and techniques related to machine learning and Big Data are dependent variables, features or predictors) and outputs (also referred
being developed. Many of such methods have been widely applied in to as targets, responses or dependent variables), and the main goal is to
other knowledge fields such as engineering, medicine and biostatistics. use inputs in order to infer the values of outputs. These problems can be
Despite their potential for treating socio-economic data, their applica- further categorized as classification problems, in which outputs are
tion in this field is still at an early stage (Varian, 2014). expressed as categories, or as regression problems, in which outputs are
This section enumerates and describes the most relevant methods expressed in a continuous space. On the other hand, Unsupervised
for treating socio-economic data from a Big Data approach, with the Learning refers to problems in which each observation has some inputs
objective of providing a framework. The reviewed techniques are but no outputs, and the main goal is to find the relationships or struc-
summarized and classified in a taxonomy illustrated in Fig. 2. ture among inputs. These problems can be further categorized into
clustering problems, in which the goal is to discover groupings in the
4.1. Methods for structuring data data, and association problems, in which the objective is to find rules
that describe the behavior of part of the data.
Big Data sources can be classified as structured (tabular data), semi- Depending on the learning paradigm, different machine learning
structured (data with machine-readable tags that do not follow a strict techniques can be applied. For nowcasting and forecasting applications,
standard) or unstructured (data that lacks from any scheme allowing supervised methods are generally employed. The most common su-
machines to understand them, e.g. a video). Since analysis algorithms pervised machine learning techniques successfully applied in other
require some structure to interpret the data and given that about 95% of disciplines, such as medicine and engineering, and that are potentially
Big Data is unstructured (Gandomi and Haider, 2015), the process of useful for the social sciences, are enumerated below.
structuring the information is basic. This includes transforming the data Linear and logistic regressions are two useful machine learning
into an organized set, with clearly defined variables and the relations techniques widely applied by economists and social scientists.
among them identified. Below, some of the most common methods for However, alternative methods to regressions have been developed and
structuring data with applications to social and economic analyses are demonstrated to perform as well as or better when using big data sets
surveyed. (Varian, 2014). For instance, Decision Trees, which are a type of pre-
Almost any source of data, and particularly the Internet, is plenty of dictive models that can be used to represent both classifiers and re-
human generated text that requires proper retrieval and processing. To gression models; Support Vector Network (Cortes and Vapnik, 1995),
exploit the full potential of text in databases, specific techniques for more commonly known as Support Vector Machine (SVM), which is a
processing natural language are required. Natural Language Processing learning machine for two-group classification; Artificial Neural Net-
(NLP) is a research area focused on exploring how computers can be works (ANN), which are two-stage regression or classification models
used to understand and shape natural language text so that it can be able to identify non-linear relations among a set of input variables, and
useful for different applications (Chowdhury, 2005). NLP is in itself a generate forecasts about the variable under study by modelling and
computational method that comprehends a series of techniques that weighting those relations (Hastie et al., 2013); and Deep Learning
provide an easy interface for information retrieval systems and, at the methods, which develop a layered and hierarchical architecture where
same time, to structure texts in different ways so that the underlying higher-level (more abstract) features are obtained by transforming

104
D. Blazquez, J. Domenech Technological Forecasting & Social Change 130 (2018) 99–113

Fig. 2. Taxonomy of non-traditional methods to treat social and economic data.

lower-level (less abstract) features. For classification, higher-level fea- learning techniques, from regressions to support vector machines, and
tures highlight aspects of the raw input that are relevant for dis- include, to mention some examples: the Least Absolute Shrinkage and
crimination. Deep Learning methods can deal with huge quantities of Selection Operator (LASSO), which was one of the first regularization
unstructured data, reason why they are positioning as a promising tool methods (Tibshirani, 1996); the regularization for support vector ma-
in Big Data analysis (LeCun et al., 2015; Najafabadi et al., 2015). ANN chines (Hastie et al., 2004); the Elastic Net, which is a mixture of the
and Deep Learning are special cases, given that their learning algo- LASSO and Ridge Regression (Zou and Hastie, 2005); and a regular-
rithms can be either supervised or unsupervised. ization scheme for neural networks, aimed at improving the classifi-
In addition, there exist a group of techniques which are focused on cation margin (Ludwig et al., 2014).
improving the performance of the previously described ones, and that Finally, Bayesian Statistics constitute an alternative approach to
are starting to be known as “Ensemble algorithms”. Some of these al- frequentist statistics (as are the methods describe above) in both terms
gorithms work by adding randomness to data, which is a useful pro- of decision theory and inference. Though their potential in the social
cedure to deal with overfitting. These techniques include the Bootstrap, sciences and economy was pointed out almost 40 years ago (Harsanyi,
Bagging, Boosting and Random Forests (Varian, 2014). 1978), the complex numerical integrations needed made them remain
Regularization methods are another group of supervised learning unused. However, the recent advances in computation methods have
techniques, whose objective is to obtain sparse solutions and that, due made it possible to easily apply Bayesian methods (Congdon, 2007).
to the increased amount of information available, have been increas- To mention some, Bayesian Model Averaging (BMA) is a multi-
ingly studied in recent years by the scientific community (Friedman modelling method that is starting to be applied to linear regression and
et al., 2010). These methods can be applied to a number of supervised generalized linear models (Ley and Steel, 2012). Naive Bayes, which is

105
D. Blazquez, J. Domenech Technological Forecasting & Social Change 130 (2018) 99–113

a machine learning tool for classification whose popularity is starting to oversampling, undersampling, synthetic sampling and kernel methods
increase due to its simplicity for being implemented, being fast and can be applied (He and Garcia, 2009). Unbalanced data sets are
computationally efficient, and obtaining high classification accuracy, common in social sciences, so it is expected that the use of these pro-
especially for Big Data (Wu et al., 2015). Also, the Spike-and-Slab Re- cedures in socio-economic research will expand in the near future.
gression, which is a variable selection method for linear regression Moreover, think if what we are trying to predict is if someone is
models (Varian, 2014). Besides this, the Bayesian Structural Time Series infected with a disease; then, the best situation would be to obtain a
(BSTS) technique is devoted to treating panel or longitudinal data, true negative (individual not infected). That is, not only false positives,
which are very common in the social sciences. This is a method for but also false negatives, imply costs. It is important to assign a mone-
variable selection and time series forecasting and nowcasting, used as tary value to these costs in order to influence the decision making of a
an alternative to traditional time series analysis methods such as Au- model. This process is known as “Cost-sensitive analysis” (Sun et al.,
toregressive (AR) and Moving Average (MA) models. 2007). It makes use of the Cost Matrix, which reflects the costs and
benefits associated to each of the four possible outcomes of a classifier.
4.3. Methods for assessing models' performance and robustness Providing this information to a classifier, it can be influenced to mini-
mize the most costly errors or to maximize beneficial classifications, so
A basic objective in any data analysis focused on forecasting is to that we obtain a “weighted accuracy”. Similarly, by using Loss Func-
obtain a robust model with the best out-of-sample predictive precision tions, classifiers are forced to give preference to predictors that help to
possible. In this subsection, a brief review on techniques for improving predict true probabilities accurately (Witten et al., 2016).
the performance of forecasting and nowcasting models is provided. To check the predictive accuracy of classifiers, methods such as the
Assessing the performance and robustness of predictive models is Lift analysis, Precision-Recall Curves, ROC Curves and the Confusion
essential to determine their validity and applicability, and the quality of Matrix are pertinent, whose fundamentals and applications regarding
the predictions. In this case, performance refers to how well a model fits the social sciences can be looked at (Fawcett, 2006; Witten et al., 2016).
the data and how accurate it is, while robustness refers to how well a When the output variable is not categorical, but numerical, other
model works on alternate data, that is, on data which is different from measures are available, such as the Root Mean Squared Error (RMSE),
that used to build the model. If a model has a good performance, then it the Percentage Error (PE), the Fractional Bias and the Index of Agree-
is capable of detecting the characteristics in a data set and providing ment (IA), whose popularity is starting to increase.
highly accurate predictions. Moreover, if it is robust, then the predic-
tions obtained could generalize and so the model is valid and useful 5. The data lifecycle
with new data. The goal in any Big Data analysis is to build models that
simultaneously are robust and provide accurate outputs: this is the only Digital data have many advantages, such as being easy to share,
path to use them as reliable tools for forecasting and nowcasting whose replicate and recombine, which make them potentially reusable.
results can be used for decision-making. Business and researchers can take advantage of this to boost research in
To compare and select different kind of models depending on how progress and leverage past investments, for instance. However, to ex-
well they fit to data and how complex they are, there exist uncountable ploit all the benefits of digital data, they must be properly collected,
classically applied tests such as Nagelkerke's R2, Hosmer-Lemeshow, processed and preserved. Data loss or damage may imply economic
Mallows' Cp, Akaike Information Criterion (AIC), Bayesian Information costs as well as lost chances, reason why funder agents (public or pri-
Criterion (BIC), Deviance and Log-Likelihood, among others. Although vate) are increasingly demanding institutions to document and run
these tests and indices provide useful information about model per- data-management plans taking into account the whole lifecycle of data
formance, they were not conceived for treating the huge amount of (Lynch, 2008). For this reason, it is basic to define what phases and
complex data with which we work nowadays. The particular char- processes form this lifecycle in order to implement robust and flexible
acteristics and issues of Big Data (size, bias, imbalanced sets, complex architectures to manage data in the context of the Digital Era.
relations…) make necessary to complement classical tests with more The data lifecycle is the sequence of stages that data follow from the
recently developed techniques that are capable to better deal with these moment they enter a system to the moment they are erased from the
issues (Varian, 2014). system or stored (Simonet et al., 2015). Between the data entrance and
First of all, to ensure that the predictions obtained are robust it is exit or storage, data go through different stages, which may differ de-
recommended to build the models by conducting a holdout process in pending on the type of data and purpose to achieve as documented in
which the initial sample is split into two subsets: the training set and the the compilation of classic data lifecycles (Committee on Earth
test set. The former is used to train the model, and generally includes Observation Satellites — Working Group on Information Systems and
about 75% to 90% of the initial observations, while the latter is used to Services, 2012). The Knowledge Discovery in Databases (KDD) process
evaluate its predictive performance and includes the remaining per- was the first proposal of a model to manage digital data (Fayyad et al.,
centage of observations. Even if data is large enough, it may be divided 1996). It refers to the complete (non-cyclical) process of extracting
in three sets: a train set (the largest), a validation set and a test set. This knowledge from data, and includes five main stages: data selection,
method ensures that the predictions obtained are robust, so that they data preprocessing, data transformation, data mining and data inter-
can be generalized to an independent data set. Another approach with pretation. As databases started to exponentially grow in size and
the same objective is K-Fold Cross-Validation. In this method, data are complexity, the necessity of a wider scheme to appropriately manage
split into K parts of equal size and the model is fitted K times, where K-1 these data was highlighted, especially by the industry. This derived into
parts are used to train the model and the remaining is used to test its the development of the Cross-Industry Standard Process for Data
predictive performance. Finally, the K estimates of the prediction error Mining (CRISP-DM process), which is an expanded implementation of
are combined. In case each part includes just one observation, then the the KDD process that introduced the management of digital data as a
process is called Leave-one-out Cross Validation (Hastie et al., 2013). cycle (Chapman et al., 2000). It comprises six stages: business under-
For big data sets, the first method is recommended. standing, data understanding, data preparation, modelling, evaluation
In addition, for properly training classifiers, at least the train set and deployment. If both are compared, the first and last stages of
should be balanced, because this way the model is built to successfully CRISP-DM process are new with respect to the KDD process, while the
detect each of the categories equally. Otherwise the learning process “data understanding” stage of CRISP-DM is similar to the “data pre-
could be endangered (Menardi and Torelli, 2014). A sample is balanced processing ” and “data transformation” stages of KDD.
when each of the categories of the response variable is present in the The next approach to data management within a digital environ-
same proportion. To balance an unbalanced data set, solutions such as ment that the scientific and industrial community focused on, and to

106
D. Blazquez, J. Domenech Technological Forecasting & Social Change 130 (2018) 99–113

which most research efforts have been paid since these days, was called to metadata. The metadata include information related to the source
itself the “data lifecycle”. The Data Documentation Initiative Alliance of origin, data format, technical details on the retrieval process and
(DDI Alliance) was one of the first voices to focus their efforts on this accessing dates, among others, thus enabling their reuse and correct
idea (DDI Alliance, 2008). It proposed a data lifecycle including the referencing. Second, data quality and validity should be assured. It
following five stages: first, discovery and planning; second, initial data is required to verify the trustworthiness of the data sources as well
collection; third, final data preparation and analysis; fourth, publication as of the own data, to control for any data inconsistencies, such as
and sharing; and last, long-term management. This departing point unexpected values and typing errors, and to clean and anonymize
considers from planning the project (what is being studied, what data data if necessary.
are needed and how they are going to be treated, etc.) to determining 4. Data integration: This stage consists in fusing data obtained from
how to store and preserve data in the long-term. With respect to KDD different data sources with a coherent and homogeneous structure,
and CRISP-DM processes, this is a more extensive approach that in- which helps to make data traceable and easier to access and ma-
cludes important concepts within digital data such as sharing and long- nipulate in successive projects. This includes activities such as es-
term management. tablishing relations among variables of different data sources,
Afterwards, Corti et al. (2014) described the phases and activities adapting units, translating, and creating a single database with all
typically undertaken within the research data lifecycle. These phases, the acquired data. Data integration should also incorporate privacy
each of which included a number of specific activities, are the fol- constraints to avoid disclosing some private information in the in-
lowing: discovery and planning, data collection, data processing and tegrated data. This is a major concern because rich integrated data
analysis, publishing and sharing, long-term management and reusing may facilitate discovering some personal details otherwise anon-
data. This proposal extends the initial one by DDI Alliance to include an ymous.
additional stage at the end of the cycle devoted to data reuse. A more 5. Data preparation: This stage consists in transforming data so that
exhaustive data lifecycle to date was proposed by Rüegg et al. (2014), they meet the format requirements of the analysis tools and tech-
who included up to eight stages: the first four stages (planning, data niques that are going to be applied. This includes activities such as
collection, data quality control, and analysis) correspond to managing transcribing, digitizing, interpolating, establishing a tabular format
data in a traditional project which is new (no previously results or data in the data set and deriving new data by operating with the existing
exist). If the project relies on existing data (referred to as “data reuse”), data.
then it follows the third first stages and continues with additional data 6. Data analysis: This stage consists in analyzing data, obtaining and
discovery, data integration, and finally, the analysis. interpreting results, and achieving conclusions. A huge range of
While Corti et al. (2014) consider data reuse as a step itself, Rüegg statistical techniques and computational tools are called to be used
et al. (2014) reference data reuse as a type of project in which existing in this stage. The final selection of the most appropriate techniques
data are used, including some steps within this lifecycle. The context of will depend on the type of data analyzed and research objectives.
economic and social analyses makes it more appropriate to consider The interpretation of the results and conclusions achieved, as well as
data reuse as a step itself, given that as a project that started from the results themselves, are basic inputs for the next stage.
scratch develops and data are obtained and exploited, these data may 7. Publishing and sharing: This stage consists in publishing results and
be reused many times in the same project with different purposes. That conclusions derived from data analysis, or the generated data sets
is, the view that a project is new or departs from data seems excessively themselves. The outputs of this stage aim to facilitate the decision-
static for economic and social nowcasting purposes. Additionally, to making process of managers or policy-makers (when data is pre-
complete each of the data lifecycles, this work includes two more steps: sented in reports, for instance), to spread knowledge (if a research
data documentation and data archiving in a public repository, which article is published, for instance) and to feed automatic systems of
we consider basic for preserving and publishing data. companies with information of relevance to help the staff make
The review of these works of reference allowed us to integrate and decisions such as ordering supplies, among many others. Other re-
fully describe the different stages of a full data lifecycle in the context of lated activities in this stage are establishing copyright of data and
economic and social analyses. Its aim is to standardize the concept of results, authoring publications, citing data sources, distributing data
data lifecycle and serve as framework when it comes to designing a and controlling data access.
proper data management architecture in this context. Our proposal for a 8. Data storage and maintenance: This stage consists in archiving and
data lifecycle includes nine stages, as reflected in Fig. 3. These stages registering all the data gathered, processed and analyzed, for al-
are described as follows: lowing long-term data preservation, curation and reuse. Actions to
be done may include storing data in specific repositories or com-
1. Study and planning: This first stage consists in designing the re- putational systems, migrating them to other platforms or mediums,
search or business project to achieve the desired goals of funders or regularly backing up the data, producing associated metadata, pre-
managers. Once each phase of the study is defined, it is necessary to serving the documentation generated during the whole process,
plan what procedures to treat data (collected or generated controlling data security and privacy and erasing data if required by
throughout the research) will be applied. For instance, this includes legal regulations, for instance.
planning what type of data are going to be collected, how and from 9. Data reuse: This stage consists in reusing data that have been pre-
which sources, which methods will be used for their processing and viously gathered, processed, analyzed and stored. This action can be
analysis, where will they be stored, and to find out what legal reg- originated in a variety of different purposes such as testing new
ulations and privacy issues affect the type of data that is going to be hypotheses related to the same project for which data were col-
analyzed, in order to adapt the operating procedures. lected, sharing or selling data to companies, conducting new pro-
2. Data collection: This stage consists in accessing the sources, which jects for which existing data can be useful and using data with in-
can be internal or external, and collecting initial or raw data. structive purposes.
Depending on the field of knowledge and the data required for de-
veloping the project, activities such as phenomena observation, 6. A Big Data architecture for nowcasting and forecasting social
experimentation, recording, simulating, scraping and negotiating and economic changes
with third-party data providers will be part of this stage.
3. Data documentation and quality assurance: This stage consists in Our proposal of a Big Data architecture for nowcasting social and
documenting the acquired data and checking their quality. First, the economic changes is presented in Fig. 4. Departing from the approach
data acquisition process should be documented by associating data of the data lifecycle in the organization, it includes layers and modules

107
D. Blazquez, J. Domenech Technological Forecasting & Social Change 130 (2018) 99–113

Fig. 3. The data lifecycle within a Big Data

paradigm.

to manage the processing and integration of social and economic data, frequencies. This includes surveys, market research data, and non-
including the storage, processing policies and publication of results. periodic report data.
This architecture is organized in three layers. The data analysis The second group of data sources are those external, that is, those
layer contains the main processes of generating knowledge from the sources which are not controlled by the organization, though they may
input data: from the ingestion of data from multiple sources to the contain information relevant to its operation. A wide variety of sources
publication of reports. Together with this layer, there are two other may be considered as relevant for the organization purposes. For in-
layers that work as support to the data analysis: The governance layer is stance, some open data offered by public institutions might provide
in charge of applying policies and regulations to the whole data life- some information on the context of the company customers. Similarly,
cycle, as well as managing the licenses related to the data sets. The social and economic data published by the official statistics institutions
persistence layer deals with the storage and management of data to have also potential for explaining the context in which individuals
make them available to the different modules in the data analysis layer. make decisions. Google Trends and social media platforms, such as
Twitter and Facebook, are useful sources for detecting trends and re-
6.1. Data analysis layer lations between relevant topics. Furthermore, many other websites or
RSS providing product opinions, political comments, product releases,
The data analysis layer is the part of the architecture that imple- etc. might be explored to find some other contextual variables that
ments the main processes required to generate knowledge, in form of could complement own data sources.
reports or predictions, from the different data sources to which the Since the access to these sources is widely heterogeneous, the ele-
organization has access. It is composed of six modules that work se- ments of this data receiving module must hide the complexity for ac-
quentially, from the data reception to the publishing of results. cessing the sources. The access to these sources by the different ele-
ments in the module may be done by means of an Application
Programming Interface (API) when available from the data provider, or
6.1.1. Data receiving module
by means of specific software developed for this purpose, e.g., web
This module constitutes the data ingestion point in the architecture,
scraper.
so that data external to the system are made accessible to the other
All elements in this module will receive the data with the format
modules of the architecture. This way, external data are connected to
and structure provided by the origin, which could be incompatible
the processing stream of the nowcasting system. It is composed of dif-
among them. According to their structure, data can be classified as
ferent elements, as many as different data sources are used as input.
structured, semi-structured or unstructured. Structured data includes
Connected data sources can be classified into two main groups:
information organized at high level, such as in relational databases,
First, those sources owned by the organization implementing the ar-
which apart from data, contains a schema with restrictions and rela-
chitecture. These sources may include relational databases managing
tions. Semi-structured data also have some organization of the in-
the daily operation of the company, that is, business-specific informa-
formation, although the schema is embedded in the data. That is, its
tion, such as sales, customers, purchases, website analytics, and so on.
structure is self-describing, as in XML documents. Unstructured data
Own sources also involve data not directly generated by the business
provide no structure at all, and can be considered as a collection of
operation, but collected or requested by the organization at variable

108
D. Blazquez, J. Domenech Technological Forecasting & Social Change 130 (2018) 99–113

Fig. 4. Big Data architecture for nowcasting and forecasting social and economic changes.

elements. This does not mean that each element does not have a with the data structure (if any) and other technical details such as the
structure, but that the schema has not been described, so additional codification, format and so on.
techniques to infer some structure should be applied. Text documents The second step is to check the internal validity of each source.
are typical unstructured sources. Data structure is a key factor to suc- Although structured sources usually keep all observations in the right
ceed integrating data from different sources, as it is the first step to format, other sources may be internally inconsistent. Thus, this step
establish the links between them. Structured data are usually related to involves checking observations for anomalous values (e.g., text when a
SQL databases, while NoSQL are more suitable for storing unstructured number is expected) and dealing with them, for instance, by marking
and semi-structured data. them as missing or wrong. This may result in a number of useless ob-
The elements in this module could access origins in batch or in servations, that is, those with an excessive number of missing features,
stream. Stream processing is possible when the source allows access which may be cleaned up to avoid including noise in the data analysis
with high bandwidth and low latency conditions, e.g., when accessing process.
an own relational database. However, when access conditions are not so The third step is related to the extraction of features and the gen-
favourable, the elements should work in batch, thus requiring persistent eration of new data derived from the original source. At this step, only
storage for the accessed data. In such event, the type of storage (SQL or derived data at entity level should be created. That is, if the origin
NoSQL) must be consistent with the data source type. In any case, the provides rows, only data from each row may be used to generate new
data access that this module provides to the remaining modules of the features. If the origin provides documents, only document contents may
architecture should be given as in stream processing. be used to generate variables describing the document, for instance, by
applying natural language processing techniques. Examples of entity-
6.1.2. Data preprocessing module level derived data may include counting the number of words of a
This module departs from the data connections prepared in the data comment, detecting the language and computing the term frequency.
receiving module. It aims to validate and preprocess data to leave them Derived data whose computation requires analyzing several entities
ready for integrating different sources. This preprocessing is divided in (e.g., computing an average) should be generated in the data analytics
three steps. module. When the computational effort to generate new data is high,
The first step is to record and document the procedure of data ac- the resulting features should be persisted to allow for reusing them in
quisition by attaching metadata to the original source. These metadata subsequent data integrations. This involves using a database consistent
should include information about the data source, the version of the with the structure type of the origin.
collector (in the receiving module) used for the retrieval, the schema

109
D. Blazquez, J. Domenech Technological Forecasting & Social Change 130 (2018) 99–113

6.1.3. Data integration module which could be treated with traditional statistical techniques. The sto-
The objective of this module is to merge the different data sources rage in this module must be analysis-driven, unlike the previous mod-
and provide homogeneous access to all data available to the organiza- ules, whose storage is source-driven.
tion. To do so, data integration must deal with five challenges: i) het-
erogeneous sources, whose access was homogenized by the data re- 6.1.5. Data analytics module
ceiving module; ii) related data structures whose relation has not been This module applies statistical and machine learning methods to
explicitly established by the sources; iii) a variety of data sizes and extract knowledge and make predictions from the data prepared by the
probably inconsistent formats; iv) heterogeneous time frequencies, previous module. To do so, descriptive and predictive techniques are
ranging from milliseconds to years; and v) heterogeneous geographic applied. The descriptive analysis could provide some insights on the
groupings, ranging from detailed GPS coordinates to state or country characteristics and evolution of the socio-economic variables under
level. study. Its results will be used in the results publishing module to create
To relate data from different sources, it is required to define tables and graphics representing the relationship among variables.
schemes that establish the relation among them. For instance, estab- Predictive techniques are based on models that help explain, clas-
lishing the relation of a commercial establishment to a region, it will be sify, forecast or nowcast the socio-economic variables under study. To
possible to relate its sales to the average income of the area in which it do so, the models are estimated or trained by using learning methods
is located. These data usually come from different sources: sales are and relying on any of the methods described in Section 4 for selecting
accessed through internal sources, while the average income could be the most meaningful variables and improving predictions. The com-
provided by an official statistics institute. puting-intensive nature of these techniques makes it more challenging
To establish such relations, some linkage techniques and hier- to deal with large data sets, since they may not properly scale when
archical groupings might be applied. Geographic hierarchies are useful data size grows.
to link records to the most appropriate geographic level, which is not Before using the models, they must be validated with a different set
necessarily the lowest one. For instance, street-level economic situation of data than that used for estimation or training. The validation pro-
may be useful for analyzing housing prices, but it is too specific for a vides an estimation on the robustness of the models and the quality of
business whose influence area is wider, e.g., an airline office. Linkage the predictions, so that the risk related to an inaccurate prediction can
techniques are required when the same entity does not receive the same be taken into account.
identifier across the different sources. This could happen simply be- The methods used in this module may be applied in stream (i.e., the
cause it is written in a different language (e.g., the country name), si- models are continuously being trained with new data), scheduled (i.e.,
tuation which can be solved with a simple translation; but also because the models are trained periodically), or on demand (i.e, the user
of lacking of a standardized or public id. In such cases, some analysis to manually requests to train again the models). Choosing one or other
match the record could help find relations and provide new insights on approach depends mainly on the computational resources available for
the data. this module.
Adapting time frequencies is also included in this category. It is The main output of the predictive techniques are the trained
required to adopt some criteria to generate high frequency data from models, whose application can guide the operative and the strategy of
lower frequencies and vice versa. Reducing time frequencies may in- the organization. They are made available to the rest of the organiza-
volve computing some summarizing statistics (e.g., average and max- tion by means of the results publishing module.
imum), while increasing time frequencies may involve interpolating
data or selecting the closest value in time. 6.1.6. Results publishing module
Once several data sources are integrated and their relations are The purpose of this module is to provide the organization with a
established, they could be stored in the persistence layer and feed the decision-making tool. To do so, it makes the results of the analysis
data analysis layer again as a new element in the data receiving module. conducted in the data analytics module available to the organization,
This way, it is possible to use these sources as a single one when in- which includes the people that make decisions, but also other in-
tegrating with additional sources. formation systems that could benefit from the data analysis. For this
reason, this module should offer the results in different formats,
6.1.4. Data preparation module adapted to the different consumers of information in the organization.
The organization in which data are stored after the integration may The publication of results for decision-makers should be done in the
not be suitable to perform the analysis. This module takes the data as form of reports, including tables, graphics and other visual elements
prepared by the data integration module and transforms them to match that help understand the social and economic behavior behind the data.
the format expected by the data analytics module. Since each element The main objective of these reports is to support decisions at strategic
in the analytics module may expect data in a different format, data or tactical levels.
preparation is also specific to each analytics element. Making the analysis results available to other information systems
These transformations may involve grouping some elements or in the organization contributes to support the decision-making at op-
joining data from different tables to enrich the information about each erational level. There is a wide variety of options to do so. For instance,
entity or individual. This is also the most suitable module to alleviate a trained model can be stored in a database or in any other storage for
the missing data, which may be estimated or interpolated to avoid being applied by different business units. The model could also be of-
losing cases. fered as a service (under the SaaS paradigm) so that when a new event
A common operation in this module is the pivot transformation. occurs, the service offers the prediction as a result of applying the
Storing information as key-value pairs, in which entity features are model. This way, the trained models can be successfully applied to
spread among many rows, is quite convenient in Big Data environ- some operational actions such as purchases and financial resources
ments. However, this may not be the table format expected by the management.
analysis software. By applying the pivot transformation, all features
regarding the same entity are arranged in the same row, which is the 6.2. Governance layer
data organization commonly required to feed the analysis.
The resulting data after the preparation process should be stored to This layer is horizontal to the rest of the system and applies the
provide consistent input to the analysis. After conducting data in- organization policies and regulations to the whole data lifecycle: from
tegration and depending on the purpose of each particular study, it is the data ingestion to the disposal. It is composed of five modules, four
possible to obtain small data sets derived from the initial big data set, of them related to the data lifecycle, plus one for auditing purposes.

110
D. Blazquez, J. Domenech Technological Forecasting & Social Change 130 (2018) 99–113

• Ingestion module: It deals with the management of the sources, in- them according to the phase of the data analysis they operate.
cluding the licenses and allowed uses, credentials for accessing In order to frame the data analysis in an organizational perspective
them, internal user permissions, completeness of metadata, and so and allow its management in a robust and flexible architecture, the data
on. lifecycle approach has been taken. Different perspectives on this ap-
• Processing module: It manages the privacy and anonymization po- proach have been reviewed and synthesized to establish and define all
licies, controls processing for ethical principles, keeps track of the the involved phases and processes.
transformations, as well as of the permissions for accessing the data Finally, the main contribution of the paper is the proposal of a Big
and computing resources. Data architecture adapted to the particularities of the economic and
• Results module: It is concerned with the traceability of the results social analyses, and grounded on the data lifecycle approach for the
(from the sources to the final report), the permissions for accessing management of data in the organization. At the same time, the proposal
the reports and results, along with the privacy aspects that may aimed to be general enough to be implemented with different tech-
affect the reports. nologies, computing paradigms and analytical software depending on
• Archival and disposal module: It implements the policy for archiving the requirements and purposes of each particular case. By im-
and disposing the information related to data sources, processing plementing this architecture, an organization will be able to make the
procedures and generated reports. most of all social and economic sources of information to which it has
• Auditing module: It inspects that the implementation of the archi- access. Not only the organization of sources is advantageous, but also
tecture is consistent with the current regulations, as well as with the their integration and connection to Big Data analytics tools able to run
security and privacy policies. It may also include checking the the models for nowcasting and forecasting socio-economic variables.
overall performance of the architecture in order to ensure that the The wide variety of data sources and techniques considered in the ar-
system has an acceptable response time. chitecture results in potentially more accurate and granular predictions.
Governments and official statistics institutions may also benefit
6.3. Persistence layer from the implementation of an information system with the proposed
architecture. Integrating the multiple sources to which they have access
The persistence layer supports the other layers by managing all is- may result in improved predictions about key economic indicators and
sues related to the storage needs. Its main function is associated with planning economic policies accordingly.
the storage of the data used as input in the data analysis layer, including Although the proposed architecture is general enough to be im-
the schema for describing the relations among sources and other me- plemented with any technology, its adoption is not without obstacles.
tadata. Not only the data itself is covered, but also the storage of the To mention some of them, the integration of the architecture in the
procedures followed in the different modules to access and transform existing organizational information systems is a critical process to en-
the data. sure the smooth generation of forecasts and nowcasts. The im-
Furthermore, this layer serves the data analytics and results pub- plementation of the modules in a proper cloud computing environment
lishing modules by providing storage for the results. This includes so that the system can scale easily is also crucial. As future work, we
storing the models and providing them with the inputs required for plan to implement the proposed Big Data architecture, in order to
computing new predictions or estimations as part of the publication of generate and publish real-time nowcasts and forecasts of some socio-
the results. economic variables using Internet data.
It is in the persistence layer where the storage infrastructure is
controlled and managed. This layer will typically use distributed sto- Acknowledgments
rage systems, combining local storage with cloud solutions that allow
elastic storage and large volume data. The decision on whether to use This work has been partially supported by the Spanish Ministry of
local or cloud storage mainly depends on where (on- or off-premises) Economy and Competitiveness under Grant TIN2013-43913-R; and by
the modules intensive in computing power (e.g., data analytics) are the Spanish Ministry of Education under Grant FPU14/02386.
implemented.
References
7. Conclusions
Alis, C.M., Letchford, A., Moat, H.S., Preis, T., 2015. Estimating tourism statistics with
In the Digital Era, most economic and social behaviors leaves be- Wikipedia page views. In: WebSci-15 Proceedings of the ACM Web Science
Conference, Oxford, United Kingdom, [Link]
hind a huge digital footprint, which is incipiently being used with Arenas-Márquez, F., Martínez-Torres, M., Toral, S., 2014. Electronic word-of-mouth
nowcasting and forecasting purposes. Despite the enormous potential of communities from the perspective of social network analysis. Tech. Anal. Strat.
these data, integrating and analyzing the wide variety of heterogeneous Manag. 26 (8), 927–942. [Link]
Armentano, M.G., Godoy, D., Campo, M., Amandi, A., 2014. NLP-based faceted search:
sources cannot be tackled with the traditional methods used in eco- experience in the development of a science and technology search engine. Expert
nomics and social sciences. To succeed in this purpose, it is mandatory Syst. Appl. 41 (6), 2886–2896. [Link]
to carefully plan and implement the whole process of data extraction, Arora, S.K., Li, Y., Youtie, J., Shapira, P., 2016. Using the Wayback Machine to mine
websites in the social sciences: a methodological resource. J. Assoc. Inf. Sci. Technol.
transformation and analysis. This is the point in which the Big Data and
67 (8), 1904–1915. [Link]
data lifecycle paradigms arise as helpful perspectives on how to deal Arora, S.K., Youtie, J., Shapira, P., Gao, L., Ma, T., 2013. Entry strategies in an emerging
with this process. technology: a pilot web-based study of graphene firms. Scientometrics 95 (3),
1189–1207. [Link]
This paper has proposed a novel Big Data architecture that accounts
Arrigo, E., Liberati, C., Mariani, P., 2016. A multivariate approach to Facebook data for
for the particularities of the economic and social behavior analyses in marketing communication. In: Proceedings of the 1st International Conference on
the Digital Era. The first particularity is related to the variety of sources Advanced Research Methods and Analytics (CARMA 2016). UPV Press, Universitat
that could provide information about economic and social topics. Our Politécnica de Valéncia, Valencia, Spain. [Link]
2016.2974.
first contribution addresses this issue by reviewing the multiple data Artola, C., Pinto, F., de Pedraza García, P., 2015. Can internet searches forecast tourism
sources and proposing a taxonomy to classify them according to the inflows? Int. J. Manpow. 36 (1), 103–116. [Link]
purpose of the agent generating the data. 0259.
Askitas, N., Zimmermann, K.F., 2009. Google econometrics and unemployment fore-
Following the Big Data paradigm, this wide variety of hetero- casting. Appl. Econ. Q. 55 (2), 107–120. [Link]
geneous sources requires specific methods for processing them. The Askitas, N., Zimmermann, K.F., 2013. Nowcasting business cycles using toll data. J.
second contribution of the paper addresses this issue by reviewing those Forecast. 32 (4), 299–306. [Link]
Askitas, N., Zimmermann, K.F., 2015. The internet as a data source for advancement in
methods not so commonly used in the social sciences, and classifying

111
D. Blazquez, J. Domenech Technological Forecasting & Social Change 130 (2018) 99–113

social sciences. Int. J. Manpow. 36 (1), 2–12. [Link] Intelligent Agent Technologies (IAT), Macau, China, pp. 573–578. [Link]
2015-0029. 10.1109/WI-IAT.2012.92.
Assunção, M.D., Calheiros, R.N., Bianchi, S., Netto, M.A., Buyya, R., 2015. Big Data Edelman, B., 2012. Using Internet data for economic research. J. Econ. Perspect. 26 (2),
computing and clouds: trends and future directions. J. Parallel Distrib. Comput. 79- 189–206. [Link]
80, 3–15. [Link] Einav, L., Levin, J., 2014. The data revolution and economic analysis. Innov. Policy Econ.
Bahrami, M., Singhal, M., 2014. The Role of Cloud Computing Architecture in Big Data. 14 (1), 1–24. [Link]
vol. 8. Springer International Publishing, Cham, pp. 275–295. [Link] Evangelopoulos, N., Zhang, X., Prybutok, V.R., 2012. Latent semantic analysis: five
1007/978-3-319-08254-7_13. methodological recommendations. Eur. J. Inf. Syst. 21 (1), 70–86. [Link]
Bangwayo-Skeete, P.F., Skeete, R.W., 2015. Can Google data improve the forecasting 10.1057/ejis.2010.61.
performance of tourist arrivals? Mixed-data sampling approach. Tour. Manag. 46, Fan, J., Han, F., Liu, H., 2014. Challenges of big data analysis. Nat. Sci. Rev. 1 (2),
454–464. [Link] 293–314. [Link]
Bello-Orgaz, G., Jung, J.J., Camacho, D., 2016. Social big data: recent achievements and Fawcett, T., 2006. An introduction to ROC analysis. Pattern Recogn. Lett. 27 (8),
new challenges. Inf. Fusion 28, 45–59. [Link] 861–874. [Link]
005. Fayyad, U., Piatetsky-Shapiro, G., Smyth, P., 1996. The KDD process for extracting useful
Berman, F., Fox, G., Hey, A.J., 2003. Grid Computing: Making the Global Infrastructure a knowledge from volumes of data. Commun. ACM 39 (11), 27–34.
Reality. Communications Networking & Distributed Systems John Wiley and Sons. Fondeur, Y., Karamé, F., 2013. Can Google data help predict French youth unemploy-
Blazquez, D., Domenech, J., 2017. Web data mining for monitoring business export or- ment? Econ. Model. 30, 117–125. [Link]
ientation. Technol. Econ. Dev. Econ. Online, 1–23. [Link] 017.
20294913.2016.1213193. Friedman, J., Hastie, T., Tibshirani, R., 2010. Regularization paths for Generalized Linear
Blei, D.M., Ng, A.Y., Jordan, M.I., 2003. Latent Dirichlet allocation. J. Mach. Learn. Res. Models via coordinate descent. J. Stat. Softw. 33 (1), 1–22.
3, 993–1022. Frota Neto, J.Q., Bloemhof, J., Corbett, C., 2016. Market prices of remanufactured, used
Bollen, J., Mao, H., Zeng, X., 2011. Twitter mood predicts the stock market. J. Comput. and new items: evidence from eBay. Int. J. Prod. Econ. 171 (3), 371–380. [Link]
Sci. 2 (1), 1–8. [Link] [Link]/10.1016/[Link].2015.02.006.
Cameron, M.P., Barrett, P., Stewardson, B., 2016. Can social media predict election re- Gaikar, D.D., Marakarkandy, B., Dasgupta, C., 2015. Using Twitter data to predict the
sults? Evidence from New Zealand. J. Polit. Mark. 15 (4), 416–432. [Link] performance of Bollywood movies. Ind. Manag. Data Syst. 115 (9), 1604–1621.
org/10.1080/15377857.2014.959690. [Link]
Ceron, A., Curini, L., Iacus, S.M., Porro, G., 2014. Every tweet counts? How sentiment Gandomi, A., Haider, M., 2015. Beyond the hype: big data concepts, methods, and ana-
analysis of social media can improve our knowledge of citizens' political preferences lytics. Int. J. Inf. Manag. 35 (2), 137–144. [Link]
with an application to Italy and France. New Media Soc. 16 (2), 340–358. [Link] 2014.10.007.
[Link]/10.1177/1461444813480466. Gayo-Avello, D., 2012, Nov, Nov. No, you cannot predict elections with twitter. IEEE
Ceron, A., Negri, F., 2016. The “social side” of public policy: monitoring online public Internet Comput. 16 (6), 91–94. [Link]
opinion and its mobilization during the policy cycle. Policy Internet 8 (2), 131–147. Gayo-Avello, D., 2013. A meta-analysis of state-of-the-art electoral prediction from
[Link] Twitter data. Soc. Sci. Comput. Rev. 31 (6), 649–679. [Link]
Chapman, P., Clinton, J., Kerber, R., Khabaza, T., Reinartz, T., Shearer, C., Wirth, R., 0894439313493979.
2000. CRISP-DM 1.0 - Step-by-Step Data Mining Guide. [Link] Gök, A., Waterworth, A., Shapira, P., 2015. Use of web mining in studying innovation.
[Link]/[Link] (accessed 1st June, 2017). Scientometrics 102 (1), 653–671. [Link]
Chen, M., Mao, S., Liu, Y., 2014. Big Data: a survey. Mob. Netw. Appl. 19 (2), 171–209. Graells-Garrido, E., Peredo, O., García, J., 2016. Sensing urban patterns with antenna
[Link] mappings: the case of Santiago, Chile. Sensors 16 (7), 1098–1123. [Link]
Chittaranjan, G., Blom, J., Gatica-Perez, D., 2013. Mining large-scale smartphone data for 10.3390/s16071098.
personality studies. Pers. Ubiquit. Comput. 17 (3), 433–450. [Link] Hand, C., Judge, G., 2012. Searching for the picture: forecasting UK cinema admissions
1007/s00779-011-0490-1. using Google Trends data. Appl. Econ. Lett. 19 (11), 1051–1055. [Link]
Choi, H., Varian, H., 2009a. Predicting Initial Claims for Unemployment Benefits. http:// 10.1080/13504851.2011.613744.
[Link]/archive/papers/[Link] (accessed 10th October, Harsanyi, J.C., 1978. Bayesian decision theory and utilitarian ethics. Am. Econ. Rev. 68
2016). (2), 223–228.
Choi, H., Varian, H., 2009b. Predicting the Present with Google Trends. [Link] Hashem, I.A.T., Yaqoob, I., Anuar, N.B., Mokhtar, S., Gani, A., Ullah Khan, S., 2015. The
[Link]/external_content/untrusted_dlcp/[Link]/en// rise of “big data” on cloud computing: review and open research issues. Inf. Syst. 47,
googleblogs/pdfs/google_predicting_the_present.pdf (accessed 10th October, 2016). 98–115. [Link]
Chong, A.Y.L., Ch’ng, E., Liu, M.J., Li, B., 2015. Predicting consumer product demands via Hastie, T., Rosset, S., Tibshirani, R., Zhu, J., 2004. The entire regularization path for the
Big Data: the roles of online promotional marketing and online reviews. Int. J. Prod. support vector machine. J. Mach. Learn. Res. 5, 1391–1415.
Res. Online, 1–15. [Link] Hastie, T., Tibshirani, R., Friedman, J., 2013. The Elements of Statistical Learning: Data
Chou, J.-S., Ngo, N.-T., 2016. Smart grid data analytics framework for increasing energy Mining, Inference and Prediction. Springer Series in Statistics, 3rd. Springer.
savings in residential buildings. Autom. Constr. 72 (3), 247–257. [Link] He, H., Garcia, E., 2009. Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21
10.1016/[Link].2016.01.002. (9), 1263–1284. [Link]
Chowdhury, G.G., 2005. Natural language processing. Annu. Rev. Inf. Sci. Technol. 37 Hu, N., Bose, I., Koh, N.S., Liu, L., 2012. Manipulation of online reviews: an analysis of
(1), 51–89. [Link] ratings, readability, and sentiments. Decis. Support. Syst. 52 (3), 674–684. [Link]
Committee on Earth Observation Satellites — Working Group on Information Systems and [Link]/10.1016/[Link].2011.11.002.
Services, 2012. CEOS Data Life Cycle Models and Concepts. [Link] IBM, 2016. Big Data and Analytics. [Link]
confluence/download/attachments/82935852/Data%20Lifecycle%20Models [Link] (accessed 21st December, 2016).
%20and%20Concepts% (accessed 27th September, 2016). Jagadish, H., Gehrke, J., Labrinidis, A., Papakonstantinou, Y., Patel, J.M., Ramakrishnan,
Congdon, P., 2007. Bayesian Statistical Modelling. Wiley Series in Probability and R., Shahabi, C., 2014. Big data and its technical challenges. Commun. ACM 57 (7),
Statistics, 2nd. John Wiley & Sons. 86–94. [Link]
Cortes, C., Vapnik, V., 1995. Support-vector networks. Mach. Learn. 20 (3), 273–297. Jin, X., Wah, B.W., Cheng, X., Wang, Y., 2015. Significance and challenges of big data
[Link] research. Big Data Res. 2 (2), 59–64. [Link]
Corti, L., Van den Eynden, V., Bishop, L., Woollard, M., 2014. Managing and sharing Khadivi, P., Ramakrishnan, N., 2016. Wikipedia in the tourism industry: forecasting de-
research data: a guide to good practice, 1st. Sage Publications. mand and modeling usage behavior. In: Thirtieth AAAI Conference on Artificial
Cox, M., Ellsworth, D., 1997. Managing Big Data for scientific visualization. ACM Intelligence. February 12–17, 2016, Phoenix, Arizona, pp. 4016–4021.
Siggraph, MRJ/NASA Ames Res. Cent. 5, 1–17. Kim, M., Park, H.W., 2012. Measuring Twitter-based political participation and delib-
David, E., Zhitomirsky-Geffet, M., Koppel, M., Uzan, H., 2016. Utilizing Facebook pages eration in the South Korean context by using social network and Triple Helix in-
of the political parties to automatically predict the political orientation of Facebook dicators. Scientometrics 90 (1), 121–140. [Link]
users. Online Inf. Rev. 40 (5), 610–623. [Link] 0508-5.
0308. Kim, T., Hong, J., Kang, P., 2015. Box office forecasting using machine learning algo-
Alliance, D.D.I., 2008. DDI Lifecycle 3.0. [Link] (accessed 29th rithms based on SNS data. Int. J. Forecast. 31 (2), 364–390. [Link]
September, 2016). 1016/[Link].2014.05.006.
Deville, P., Linard, C., Martin, S., Gilbert, M., Stevens, F.R., Gaughan, A.E., Blondel, V.D., Kitchin, R., 2014. The real-time city? Big data and smart urbanism. GeoJournal 79 (1),
Tatem, A.J., 2014. Dynamic population mapping using mobile phone data. Proc. Natl. 1–14. [Link]
Acad. Sci. 111 (45), 15888–15893. [Link] Krishnan, N.C., Cook, D.J., 2014. Activity recognition on streaming sensor data. Pervasive
Dey, M.M., Rabbani, A.G., Singh, K., Engle, C.R., 2014. Determinants of retail price and Mob. Comput. 10, 138–154. [Link]
sales volume of catfish products in the United States: an application of retail scanner Laney, D., 2001. 3D Data Management: Controlling Data Volume, Velocity, and Variety.
data. Aquac. Econ. Manag. 18 (2), 120–148. [Link] Application Delivery Strategies. pp. 949. [Link]
2014.903312. 2012/01/ad949-3D-Data-Management-Controlling-Data-Volume-Velocity-and-
Dobra, A., Williams, N.E., Eagle, N., 2015. Spatiotemporal detection of unusual human [Link] (accessed 21st December, 2016).
population behavior using mobile phone data. PLOS ONE 10 (3), 1–20. [Link] Laurila, J.K., Gatica-Perez, D., Aad, I., Blom, J., Bornet, O., Do, T.M.T., Dousse, O., Eberle,
org/10.1371/[Link].0120449.s001. J., Miettinen, M., 2013. From big smartphone data to worldwide research: the mobile
Domenech, J., de la Ossa, B., Pont, A., Gil, J.A., Martinez, M., Rubio, A., 2012. An in- data challenge. Pervasive Mob. Comput. 9, 752–771. [Link]
telligent system for retrieving economic information from corporate websites. In: pmcj.2013.07.014.
IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and LeCun, Y., Bengio, Y., Hinton, G., 2015. Deep learning. Nature 521 (7553), 436–444.

112
D. Blazquez, J. Domenech Technological Forecasting & Social Change 130 (2018) 99–113

[Link] 528–543. [Link]

Ley, E., Steel, M.F., 2012. Mixtures of g-priors for Bayesian model averaging with eco- Simonet, A., Fedak, G., Ripeanu, M., 2015. Active data: a programming model to manage
nomic applications. J. Econ. 171 (2), 251–266. [Link] data life cycle across heterogeneous systems and infrastructures. Futur. Gener.
2012.06.009. Comput. Syst. 53, 25–42. [Link]
Li, G., Law, R., Vu, H.Q., Rong, J., Zhao, X.R., 2015. Identifying emerging hotel pre- Suhara, Y., Xu, Y., Pentland, A., 2017. Deepmood: forecasting depressed mood based on
ferences using Emerging Pattern Mining technique. Tour. Manag. 46, 311–321. self-reported histories via recurrent neural networks. In: Proceedings of the 26th
[Link] International Conference on World Wide Web - WWW ’17. ACM Press, pp. 715–724.
Li, Y., Arora, S., Youtie, J., Shapira, P., 2016. Using web mining to explore Triple Helix Sun, Y., Kamel, M.S., Wong, A.K., Wang, Y., 2007. Cost-sensitive boosting for classifica-
influences on growth in small and mid-size firms. Technovation Online, 1–12. http:// tion of imbalanced data. Pattern Recogn. 40 (12), 3358–3378. [Link]
[Link]/10.1016/[Link].2016.01.002. 1016/[Link].2007.04.009.
Liu, B., 2012. Sentiment analysis and opinion mining. Synth. Lect. Hum. Lang. Technol. 5 Thelwall, M., 2007. Blog searching: the first general-purpose source of retrospective
(1), 1–167. [Link] public opinion in the social sciences? Online Inf. Rev. 31 (3), 277–289. [Link]
Liu, Y., Huang, X., An, A., Yu, X., 2007. ARSA: sentiment-aware model for predicting sales org/10.1108/14684520710764069.
performance using blogs. In: Proceedings of the 30th Annual International ACM Tibshirani, R., 1996. Regression shrinkage and selection via the Lasso. J. R. Stat. Soc. Ser.
SIGIR Conference on Research and Development in Information Retrieval. ACM, pp. B Methodol. 58, 267–288.
607–614. Tumasjan, A., Sprenger, T.O., Sandner, P.G., Welpe, I.M., 2011. Election forecasts with
Ludwig, O., Nunes, U., Araujo, R., 2014. Eigenvalue decay: a new method for neural Twitter: how 140 characters reflect the political landscape. Soc. Sci. Comput. Rev. 29
network regularization. Neurocomputing 124, 33–42. [Link] (4), 402–418. [Link]
neucom.2013.08.005. Van Vlasselaer, V., Bravo, C., Caelen, O., Eliassi-Rad, T., Akoglu, L., Snoeck, M., Baesens,
Lynch, C., 2008. Big data: how do your data grow? Nature 455, 28–29. [Link] B., 2015. APATE: a novel approach for automated credit card transaction fraud de-
10.1038/455028a. tection using network-based extensions. Decis. Support. Syst. 75, 38–48. [Link]
Malbon, J., 2013. Taking fake online consumer reviews seriously. J. Consum. Policy 36 [Link]/10.1016/[Link].2015.04.013.
(2), 139–157. [Link] Varian, H.R., 2014. Big Data: new tricks for econometrics. J. Econ. Perspect. 28, 3–28.
Mavragani, A., Tsagarakis, K.P., 2016. YES or NO: predicting the 2015 GReferendum [Link]
results using Google Trends. Technol. Forecast. Soc. Chang. 109, 1–5. [Link] Vatsalan, D., Christen, P., Verykios, V.S., 2013. A taxonomy of privacy-preserving record
org/10.1016/[Link].2016.04.028. linkage techniques. Inf. Syst. 38, 946–969. [Link]
McLaren, N., Shanbhogue, R., 2011. Using internet search data as economic indicators. 005.
Bank Engl. Q. Bull. 2011 Q2, 134–140. Vicente, M.R., López-Menéndez, A.J., Pérez, R., 2015. Forecasting unemployment with
Menardi, G., Torelli, N., 2014. Training and assessing classification rules with imbalanced internet search data: does it help to improve predictions when job destruction is
data. Data Min. Knowl. Disc. 28, 92–122. [Link] skyrocketing? Technol. Forecast. Soc. Chang. 92, 132–139. [Link]
0295-5. 1016/[Link].2014.12.005.
Moat, H.S., Curme, C., Stanley, H.E., Preis, T., 2014. Anticipating Stock Market Vosen, S., Schmidt, T., 2011. Forecasting private consumption: survey-based indicators
Movements with Google and Wikipedia. NATO Science for Peace and Security Series vs. Google Trends. J. Forecast. 30 (6), 565–578. [Link]
C: Environmental Security Springer Science, pp. 47–59. [Link] Wang, Y., Kung, L., Byrd, T.A., 2016a. Big data analytics: understanding its capabilities
978-94-017-8704-8_4. and potential benefits for healthcare organizations. Technol. Forecast. Soc. Chang.
Montoliu, R., Blom, J., Gatica-Perez, D., 2013. Discovering places of interest in everyday Online, 1–11. [Link]
life from smartphone data. Multimedia Tools Appl. 62, 179–207. [Link] Wang, Y., Yuan, N.J., Sun, Y., Zhang, F., Xie, X., Li, Q., Chen, E., 2016b. A contextual
10.1007/s11042-011-0982-z. collaborative approach for app usage forecasting. In: Proceedings of the 2016 ACM
Moro, S., Cortez, P., Rita, P., 2015. Business intelligence in banking: a literature analysis International Joint Conference on Pervasive and Ubiquitous Computing — UbiComp
from 2002 to 2013 using text mining and latent Dirichlet allocation. Expert Syst. ’16. ACM Press, pp. 1247–1258.
Appl. 42, 1314–1324. [Link] Wikimedia Foundation, 2017. Dashboards and Data Downloads for Wikimedia Projects.
Najafabadi, M.M., Villanustre, F., Khoshgoftaar, T.M., Seliya, N., Wald, R., Muharemagic, [Link] (accessed 7th July, 2017).
E., 2015. Deep learning applications and challenges in big data analytics. J. Big Data Williams, N.E., Thomas, T.A., Dunbar, M., Eagle, N., Dobra, A., 2015. Measures of human
2 (1), 1–21. [Link] mobility using mobile phone records enhanced with GIS data. PLOS ONE 10, 1–16.
Pääkkönen, P., Pakkala, D., 2015. Reference architecture and classification of technolo- [Link]
gies, products and services for big data systems. Big Data Res. 2 (4), 166–186. http:// Witten, I.H., Frank, E., Hall, M.A., Pal, C.J., 2016. Data Mining: Practical Machine
[Link]/10.1016/[Link].2015.01.001. Learning Tools and Techniques, 4th. Morgan Kaufmann - Elsevier.
Pan, B., Yang, Y., 2016. Monitoring and Forecasting Tourist Activities with Big Data. Wu, J., Pan, S., Zhu, X., Cai, Z., Zhang, P., Zhang, C., 2015. Self-adaptive attribute
Apple Academic Press, pp. 43–62 chap. 3. weighting for Naive Bayes classification. Expert Syst. Appl. 42 (3), 1487–1502.
Pandya, S.S., Venkatesan, R., 2016. French roast: consumer response to international [Link]
conflict — evidence from supermarket scanner data. Rev. Econ. Stat. 98 (1), 42–56. Xiong, T., Wang, S., Mayers, A., Monga, E., 2013. Personal bankruptcy prediction by
[Link] mining credit card data. Expert Syst. Appl. 40, 665–676. [Link]
Pesenson, M.Z., Pesenson, I.Z., McCollum, B., 2010. The data big bang and the expanding [Link].2012.07.072.
digital universe: high-dimensional, complex and massive data sets in an inflationary Zhang, Y., Ren, S., Liu, Y., Si, S., 2017. A big data analytics architecture for cleaner
epoch. Adv. Astron. 2010, 1–16. [Link] manufacturing and maintenance processes of complex products. J. Clean. Prod. 142
Preis, T., Moat, H.S., Stanley, H.E., 2013. Quantifying trading behavior in financial (2), 626–641. [Link]
markets using Google Trends. Sci Rep 3, 1–6. [Link] Zou, H., Hastie, T., 2005. Regularization and variable selection via the elastic net. J. R.
Reed, D.A., Dongarra, J., 2015. Exascale computing and big data. Commun. ACM 58, Stat. Soc. Ser. B 67, 301–320. [Link]
56–68. [Link]
Rudolph, M., Ruiz, F., Mandt, S., Blei, D., 2016. Exponential family embeddings. In: Desamparados Blazquez received a B.S. in Business Administration and Management
Advances in Neural Information Processing Systems, pp. 478–486. and a M.S. in Data Analytics Engineering from the Universitat Politècnica de València
Rüegg, J., Gries, C., Bond-Lamberty, B., Bowen, G.J., Felzer, B.S., McIntyre, N.E., (Spain). She is currently a Ph.D. student and a predoctoral research fellow under the
Soranno, P.A., Vanderbilt, K.L., Weathers, K.C., 2014. Completing the data life cycle: Programme for the Training of University Lecturers (FPU) from the Spanish Ministry of
using information management in macrosystems ecology research. Front. Ecol. Education. She develops her work at the Department of Economics and Social Sciences of
Environ. 12, 24–30. [Link] the Universitat Politècnica de València. Her research interests include web economic
Russell, M.A., 2013. Mining the Social Web: Data Mining Facebook, Twitter, LinkedIn, indicators and internet economics.
Google+, GitHub and More, 2nd. O’Reilly Media, pp. 448.
Saleiro, P., Amir, S., Silva, M., Soares, C., 2015. Popmine: tracking political opinion on the
web. In: 2015 IEEE International Conference on Computer and Information Josep Domenech received a B.S., M.S. and Ph.D. in Computer Science from the
Technology; Ubiquitous Computing and Communications; Dependable, Autonomic Universitat Politècnica de València, and a B.S. and M.S. in Business Administration and
Economics from the Universitat de València. Since 2009, he is an associate professor at
and Secure Computing; Pervasive Intelligence and Computing (CIT/IUCC/DASC/
PICOM). IEEE, Liverpool, United Kingdom, pp. 1521–1526. the Department of Economics and Social Sciences of the Universitat Politècnica de
València. He is currently leading a research project on planning and using cloud com-
Schneider, M.J., Gupta, S., 2016. Forecasting sales of new and existing products using
consumer reviews: a random projections approach. Int. J. Forecast. 32, 243–256. puting platforms efficiently, funded by the Ministry of Economy and Competitiveness of
[Link] Spain. His research interests are focused on multidisciplinary approaches to internet
Schoen, H., Gayo-Avello, Panagiotis Takis Metax, D., Takis Metaxas, P., Mustafaraj, E., systems and digital economics, including web economic indicators, internet economics
Strohmaier, M., 2013. The power of prediction with social media. Internet Res. 23, and web performance characterization.

113