Sampling Theory

Sisay Awoke (MSc)

sisaybest2000@[Link] /[Link]@[Link]
Statistics Department, Addis Ababa University

2024
Course Contents
1. Introduction to Sample Survey Methods
• Purposes and uses of sample surveys
• Basic concepts of sampling
• Types of inquiry
• Sample surveys and census
• Types of sampling techniques
2. Simple Random Sampling
• Basic concepts
• Selection procedures
• Review of sampling distribution
• Properties of estimates
• Variances and standard errors of the estimates
• Estimation of standard error from a sample
• Confidence limits
• Estimation of sub-populations
• Sample size for estimating population mean and total
• Relative error
3. Sampling Proportions
• Basic concepts
• Variances and standard errors of the estimates
• Estimation of the standard error from a sample
• Confidence limits
• Sample size estimation
• Relative error
• Proportions and totals over sub-populations
4. Stratified Random Sampling
• Basic concepts of stratification
• Major advantages of stratification
• Estimation of population mean and total
• Allocating samples to strata
• Relative precision of stratified random sampling and SRS
• Relative gain in precision
• Stratified sampling for proportion
• Estimation of sample size
• Design effect
5. Ratio Estimation
• Basic concepts of ratio estimation
• Estimation of a ratio under SRS
• The ratio estimator and its variance
• Bias of the ratio estimates
• Comparison of ratio estimators with SRS
• Ratio estimates in stratified sampling
6. Regression Estimation
• Basic concepts of regression estimation
• The linear regression estimate
• Large sample comparison of regression with SRS and ratio estimators
• Regression estimates in stratified sampling
7. Cluster Sampling
• Basic concepts of cluster sampling
• Cluster of equal size
• Estimation of cluster of equal size
• Comparison of cluster of equal size with SRS
• Multi-stage cluster sampling
8. Systematic Sampling
• Basic concepts of systematic sampling
• Systematic sample selection procedure under different cases
• Comparison with SRS
• Estimation of the variance from a single sample
• Estimation of mean/total from repeated systematic sampling (sub-samples)
• Population with linear trend and with periodicity
• Advantages and disadvantages of systematic sampling
9. Unequal Probability sampling: Sampling with Probability
Proportional to Size (PPS)
• Basic concepts of varying probability sampling
• Procedure for the selection of sample units with PPS
• Estimating procedures
 CHAPTER ONE

Introduction to Sample Survey

Methods
1.1 Purposes and Uses of Sample Surveys
• Why sample surveys are conducted as a method of data collection?
What is a sample survey? Answering these two important
questions will give us a general view of the purpose of sample
survey.
• One must get statistical data by directly investigating the
characteristics of elements of the population.
• In most of the cases this direct investigation of the whole elements
of the population is not either possible or feasible.
• This could be due to resource constraints (funds and time) or for
some other reasons such as nature of the population in which
investigation of all population elements is impossible.
• Sampling is done because you usually cannot gather data from
the entire population
• The population size may be very large: A typical example is that
housing and population census of Ethiopia is conducted every 10
years
• The population may be homogeneous
• Even in relatively small populations, the data may be needed
urgently, and including everyone in the population in your data
collection may take too long.
• To handle such problems, sample survey is an alternative and
feasible method of data gathering.
• Thus, sample survey is a method of collecting statistical data from
sample elements of the population to provide information or
statistical data in the form of population estimates, which help to
make inferences about the entire population.

• These population estimates are relevant to researchers, planners,

and policy makers at all levels of economic and social sectors.
• Information or statistical data needed for both research and
administrative purposes are used mainly by
• government and non-government institutions,
• individuals or groups,
• and wider community in making concrete decision, planning,
formulating economic and social policies, and promoting research work
and studies.
• For example, responsible offices and institutions of a central
government and other development agencies conduct sample
surveys to inform the public about their respective activities and
performances, in particular about populations in terms of
distribution, employment status, income and expenditure, housing
conditions, education, health, nutrition, agriculture, transport
services, trade, and many other economic and social phenomena.
• They also conduct sample surveys of organizations or
institutions such as schools, hospitals, manufacturers, retail
outlets, farms, hotels, markets, etc. to meet the ever-
increasing demand of statistics in these areas.
• In general, as the demand of statistics rises and expands its
scope conducting sample surveys frequently has become a
feasible fast data generation mechanism.
• The main objective of sample survey is to make an inference
about the population parameters based on the estimators
computed from the sample.
• Uses of Sample Survey
• Sample survey can be used to update the values of census results.
In Ethiopia, population and housing census is conducted every 10
years by the FDRE constitution article 103/5. The characteristics of
the population such as population size, sex composition between
two consecutive censuses can be updated by sample survey
• It uses to estimate the population characteristics at a point in time
• Demographic Surveys
• These surveys are conducted to collect the demographic data, e.g., household
surveys, family size, number of males in families, etc. Such surveys are useful in the
policy formulation for any city, state or country for the welfare of the people
• Educational Surveys
• These surveys are conducted to collect the educational data, e.g., how many children
go to school, how many persons are graduate, etc. Such surveys are conducted to
examine the educational programs in schools and colleges. Generally, schools are
selected first and then the students from each school constitute the sample
• Economic Surveys
• These surveys are conducted to collect the economic data, e.g., data related to export
and import of goods, industrial production, consumer expenditure etc. Such data is
helpful in constructing the indices indicating the growth in a particular sector of
economy or even the overall economic growth of the country
• Employment Surveys
• These surveys are conducted to collect the employment related data, e.g.,
employment rate, labour conditions, wages, etc. in a city, state or country. Such data
helps in constructing various indices to know the employment conditions among the
people
• Health and Nutrition Surveys
• These surveys are conducted to collect the data related to health and nutrition issues,
e.g., number of visits to doctors, food given to children, nutritional value etc. Such
surveys are conducted in cities, states as well as countries by the national and
international organizations like UNICEF, WHO etc.
• Agricultural Surveys
• These surveys are conducted to collect the agriculture related data to estimate, e.g.,
the acreage and production of crops, livestock numbers, use of fertilizers, use of
pesticides and other related topics. The government bases its planning related to the
food issues for the people based on such surveys
• Marketing Surveys
• These surveys are conducted to collect the data related to marketing. They are
conducted by major companies, manufacturers or those who provide services to
consumer etc. Such data is used for knowing the satisfaction and opinion of
consumers as well as in developing the sales, purchase and promotional activities etc.
• Election Surveys
• These surveys are conducted to study the outcome of an election or a poll. For
example, such polls are conducted in democratic countries to have the opinions of
people about any candidate who is contesting the election.
• Campus Surveys
• These surveys are conducted on the students of any educational institution to study
about the educational programs, living facilities, dining facilities, sports activities,
etc.
 1.2 Basic Concepts of Sampling and Definition
of Some Statistical Terms
• Population: Population is defined statistically as the totality of all subjects
having certain common characteristics that are being studied in specific area
and time. Expressing this in a different way, it is the collection of all the
units or elements under investigation, which consists of a specified type of
persons or objects over a given space and time.
• During actual survey undertakings, a researcher can limit definition of the
population in scope on the basis of the objective of the study. For example,
it could mean:
• the total human population of a county or a village, or a state in 1994;
• it could mean the total number of plants in a field in 2003;
• the total number of farmers receiving fertilizer on credit in a given area in 2006;
• all herds of cattle in a specific region in 2000;
• total number of school age population in a specified rural part of the country in 2007;
• total number of women in the reproductive age in Addis Ababa City in 2005;
• total ledgers of textile industries to be audited in 2007, etc.
• Moreover, dealing with population requires that one should note the
nature of the population, that is, the population can be finite, in which
one can enumerate all persons or objects, or infinite in which
enumeration of all elements is not possible.
• A sample consists of elements selected from a population with
statistical methods for the purpose of investigation and with the aim of
estimating the characteristics of the population.
• Sampling: Sampling is a statistical process in which one can select and
examine sample units and provide statistical information by involving a
variety of techniques instead of studying the whole population units.
• In other words, it is a process that allows the investigator to obtain
accurate information from a sample and relate that information to the
population characteristic without examining every unit of that
population.
• Each observation, or item, taken from the population
contains a certain amount of information about the
population parameter or parameters of interest
• Because information costs money, the
experimenter/data collector must determine how
much information he/she should buy
• The quantity of information obtained in the sample
depends on the number of items sampled and on the
amount of variation in the data Too little information prevents the experimenter
• The variation in the data can be controlled somewhat from making good estimate, whereas too much
by the method of selecting the sample, called the of it results in a waste of money
design of the sample survey
• The design of survey and the sample size determine the quantity of information in the
sample pertinent to a population parameter, provided that accuracy of measurements are
obtained on each sampled element
• An element is an object on which a measurement is taken
• An elementary unit (unit of analysis) is an element or group of
elements on which information is required. Thus, persons, vehicles,
households, farms, animals, steel cable, etc., are examples of elementary
unit. In other words, it is the object that we observe or measure.
• Sampling unit: For the purpose of sample selection, the population is
divided into a finite number of distinct, non-overlapping and identifiable
units called sampling units, so that each member of the population belongs
to only one sampling unit.
• Naturally, the type of sampling unit depends on the nature of the study.
Hence, the sampling unit may or may not correspond to the units of analysis
or elementary unit.
• For example, the sampling units may be households, whereas the unit of
analysis would be persons.
• Commonly the sampling unit could be the farm, the farmers, a household or
a person, an event such as a market transaction, establishments, etc.
• Therefore, one can select individual persons, households, villages, urban
kebele (woreda), or any other unit. The use of different sampling units has
different effects on the case of conducting a survey and on the precision of
the findings.
• A characteristics of the units which may take any of the values of
a specified set with a specific relative frequency for different unit
is called random variable
• If such a characteristics is to be studies on the basis of information
obtained through an inquiry, the variable is termed study variable
• An ancillary variable is a variable which, although not a subject
of study of a particular inquiry, is one on which information can
be collected that may help in providing, for a study variable,
estimators with better approximation to the unknown true value
• Sampling Frame is a list of elements or sampling units. This frame may
consist either of a list of the units or of a map areas, in case a sample of areas
is to be selected
• The sampling frame should be
• Accurate,
• Free from omissions and duplication,
• Adequate and up-to-date, and
• The units should be identified without ambiguity
• Some sampling schemes may involve multiple frames. For example, in
sampling voters, we could start by sampling city blocks, then sampling
housing units. One frame is a list of city blocks and the second sampling
frame is a list of housing units within those blocks
• A Sample is a collection of elementary units drawn from a single sampling
frame or from multiple sampling frames
• Example: A researcher is Addis Ababa University has discovered
that students are complaining about the quality of cafeteria service.
She has decided to prepare a solution about this problem. To do
this, she wants to conduct sample survey. As a result, she planned
to consider students of CNCS registered for 2021/22 academic
year. To ease selection of students, she grouped the students by
their department and manage to select 20 students from the two
departments that are randomly selected. Identify target population,
elements, sampling units and sampling frame
• Population Values: Survey research is aimed at estimating specified
population values. These population values are called parameters. A
Population parameter is a numerical expression that summarizes the
values of some characteristics for all units of an entire population.
• Experience has shown that mostly four values of the population are
of interest in surveys. These are population total, population mean,
population proportion, and population ratio. The following are
examples of the population values.
a) Population Total:
• Example: Total number of acres of cropland harvested; total production of
crops by type; total number of persons economically active; total number of
farming households; the total amount of fertilizer used.
b) Population Mean:
• Example: The average size of agricultural holdings in acres; the average
size of fertilizer consumed per household; the average income per
household; the average number of patients treated per hospital; yearly or
monthly average production; the average age of school age population.
c) Population proportion:
• Example: The proportion of cultivated area under oil seeds (cotton,
pulses, etc.); The proportion of illiterate head of agricultural household;
Proportion of farmers who own tractors; The proportion of coffee export;
The proportion of health facility users.
d) Population Ratio:
• Example: The ratio of male population to female population; the total
number of loans for fertilizer to total loans in a bank; the ratio of
expenditure on food to that on rent.
Steps in Sample Survey
1. Objective of the survey
– Identifying, understanding and formulating the problem is a blueprint
– Understanding the problem will help us to formulate the objective
– Clarifying the objective will guide us to ask right questions
2. Identifying Target Population
– Based on the objective of the survey, decide the population from which the
information can be obtained
– Identifying the target population will help us to locate some resources
3. Data to be collected
– Decide what type of information is required to answer the objective
– List out all variables that are either directly or indirectly related with the objective of
the survey
4. Decide Degree of Precision Required
– The results of any sample survey are always subjected to some uncertainty. Such
uncertainty can be reduced by taking larger samples or using superior instruments.
This involves more cost and more time
– Decide the sample size
5. Method of Measurement
– The choice of measuring instrument and the method to measure the data from the
population needs to be specified clearly
– For example, the data has to be collected through interview, questionnaire, personal
visit, combination of any of these approaches, etc.
6. Prepare Sampling Frame
– The sampling frame has to be clearly specified
– The population is divided into sampling units such that the units cover the whole
population and every sampling unit is tagged with identification
7. Selection of Sample
– Use random numbers or lottery method to select sample elements randomly from target
population
8. Organization of the Field Work
– How to conduct the survey, how to handle business administrative issues, providing
proper training to surveyors, procedures, plans for handling the non-response and
missing observations etc. are some of the issues which need to be addressed for
organizing the survey work in the fields.
– The procedure for early checking of the quality of return should be prescribed
9. Pre-test
– Before actual data collection is undergone, testing survey instrument must be conducted
– The pre-test (piloting) must be conducted on a small scale and the feedback obtained
from the pretest will help us to improve the instrument
10. Summary and Analysis of Data
– It is to be noted that based on the objectives of the data, the suitable statistical tool is
decided which can answer the relevant questions
1.3 Types of Inquiry
• Inquiry can be described simply as the process of finding answers to
questions.
• Several types of inquiry for the gathering of statistical data on various topics
can be distinguished. We consider here only censuses and sample surveys,
which are the two main ways of collecting information for statistical
purposes.
• In sample survey, we have plenty of methods to gather answers to our
questions
‒ Direct observation
‒ Face-to-face interview
‒ Phone interview
‒ Self administered questionnaire
a) Census: This is an investigation that covers every individual unit in the
population being studied. In other words, census is characterized by four
essential features:
• Individual enumeration of all units, which means that the census counts each
and every unit in the designated territory separately and the characteristics of
each unit are recorded separately.
• Universality within a defined territory, which implies that all units are required
to be included in the census
• Simultaneity to express population with reference to point of time,
• Defined periodicity to assess changes of population, that is, censuses should be
taken at regular intervals so that comparable information is made available in a
fixed sequence.
• The best known examples are the national censuses of population and
housing, agriculture and industrial, which are conducted by many countries
on regular basis.
• Since these censuses aim at exhaustive coverage of all the units of the
population of interest, they are usually massive operations and are therefore
conducted at regular intervals once in five or ten years.
• Although it aims at enumerating all units in the country, this goal is seldom
fully achieved in practice. Units may be accidentally or erroneously omitted
from a census for various reasons, and certain well-defined categories of units
may be deliberately excluded from the scope of the census.
• The existence of either omissions or exclusions does not mean a census is a
survey.
• A census can also relate to a much smaller and more specific population. For
example:
‒ a population census of an individual village;
‒ an investigation covering all extension workers in a district;
‒ a study of all farmers receiving credit from an agricultural bank,
‒ a total patients treated in a hospital,
‒ total number of female students in AAU.
• Census is likely to be appropriate for these examples because of two general
circumstances.
• One is where basic information is needed for general planning purposes, so
that it is necessary to gather data on a range of topics, from all sectors of the
population with a good level of accuracy.
• The other is where the population under investigation is small and readily
identifiable, but may be fairly heterogeneous, so that it is both feasible and
advisable to include the whole of the population in the study.
• b) Sample Survey is an investigation of a certain population to provide
quantitative and qualitative characteristics of the population. It deals with the
collection of information about the characteristics of socioeconomic interest
from some or all units of a population using well-defined concepts, methods
and procedures, and the compilation of such information into a useful
summary form, which could be used for analysis.
• This type of study is appropriate mainly when resources are not sufficient to
cover the whole population or when it is not feasible to consider the whole
population in terms of time and level of treatment. The information gathered
through sample survey must be generalized to the whole population in order
to make overall conclusion.
• A sample survey can be of any size. It can be of large-scale operation such as
a national demographic or agricultural survey for general planning purposes
or it may be a very small-scale investigation, such as a sample survey of
farmers in a project area. Usually the selection is made at random, so that the
sample is known to be representative of the whole population.
• In sample surveys it is useful to distinguish between the target population, for
which the results are required, and the survey population, which is the
population actually covered.
• Target population is the population, defined at the planning stage, for which
the results are expected.
• Survey population is the population actually covered during the
implementation.
• The difference could exist due to the exclusion of some units from a survey
because of non-coverage and non-response.
1.4 Sample Surveys and Census
1.4.1 Advantages of Sample Survey
• The advantages of sample survey against complete coverage (census) have become
obvious in recent years and need be stated briefly.
• Sample survey saves money. It is possible to collect information from sample
households and obtain estimates that reasonably approximate the actual
characteristics of a large population. It is obviously cheaper to gather information
from 500 households than from 10,000 households. Therefore, the cost of obtaining
information through a sample would be a lot less than obtaining it through a census.
It is important to note that the cost per unit is higher in sample than in census.
• Sample survey saves labor. A smaller staff is required both for fieldwork and for
data processing. The number of enumerators, supervisors, data editors, and data
entry clerks required are very much less in sample survey than in census.
• Sample survey saves time. Sampling requires a smaller scale of operations and it
reduces data collection and processing time.
• Sample survey provides higher level of accuracy. This accuracy can be
achieved through more selective recruiting of interviewers and supervisors,
more extensive training programs, a closer supervision of the personnel
involved and a more efficient monitoring of the fieldwork.
• Sample survey could be the only option for the study in some specialized
areas. There are some cases where information of technical nature requires
highly trained personnel and specialized equipment like in medical areas.
Observation or experimentation could be destructive in nature like testing
industrial products, such as testing the average duration of burning of bulbs,
and testing the quality of wine, beer, etc. In this case sampling is the only
feasible means of study.
• Other practical advantages of sample survey may include assessment of
census (tests of census procedures, evaluation of census and for updating
census results), quality control at field level and at data processing level,
advance tabulation.
1.4.2 Advantages of Census
• Census can provide information for lowest administrative units.
• It is more efficient in providing information on rare events, such as the
total areas under some rare crops or the number of persons with specified
disability.
• In census better responses can be obtained and these results serve as a
basis for various surveys. In addition, there is no sampling error, and it is
easier to reduce coverage error.
1.5 Types of sampling techniques
• Sample surveys can be categorized into two very broad classes on the basis of how the
sample is selected, namely probability sampling and non-probability sampling.
• Probability sampling: It involves the selection of units from a population, based on the
principle of randomization. It has the characteristic that every element in the population
has a known, nonzero probability of being included in the sample.
• The selection process is random and the laws of probability can be applied. In probability
sampling the reliability of the resulting population estimates can be evaluated statistically.
• This gives the users of the survey estimates insight into how much value can be placed on
the estimates. There is also a theoretical basis for the process of inferring the sample
results back to the population.
• There are various types of probability sampling techniques in which a sample design uses
a combination of one or more of these techniques. These include simple random sampling,
stratified sampling, systematic sampling, cluster sampling and multi-stage sampling. They
will be discussed further in subsequent chapters of this course.
• Whatever probability sampling method is used, the results of a sample survey are
affected by sampling errors and non-sampling errors.
• Sampling errors are the resulting discrepancies between the sample estimates and
the population values. In other words, the estimates obtained from sample survey
differ from the values that would have been obtained from complete census,
conducted under the same general conditions. Such errors are measured by the
standard error.
• The magnitude of sampling errors will naturally depend on the population under
study, the size of the sample, the manner in which the sample is drawn and the
method of estimation.
• Non-sampling errors comprise all other things that are not the result of sampling
and contribute to survey error. It would arise even if the whole population is studied.
• Non-sampling errors are often thought of as being due entirely to mistakes and
deficiencies during the development and execution of survey procedures.
• Non-sampling errors are said to arise at every stage of survey, for example from
wrongly conceived definitions, imperfections in the tabulation plans, data processing,
during data collection such as failure to obtain responses from all sample members
and so on.
Non-probability sampling
• It is a method in which a sample is selected on a basis other than probability considerations.
They are generally less statistically accurate than probability sampling methods, but they
are cheaper and more convenient.
• Some survey situations call for non-probability samples, samples in which the investigator
does not know the probability of each proportion element being included in the sample.
• Although, non-probability sampling can be very useful, they do have some important
limitations.
• First, no real claim of representatives can be made without the use of probability in the
selection of elements for the sample. This question of representation greatly limits the ability
to generalize findings beyond the level of the sample cases.
• Second, the degree of sampling error remains unknown and unknowable. The lack of
probability in the selection of units means that the techniques employed for estimating
sampling error with probability samples are not appropriate. This also means that the
techniques for estimating sample size are also not applicable to non- probability sampling.
In this case it only suggested that sufficient samples be selected to allow the types of data
analysis that are planned.
• Finally, non-probability sampling cannot use statistical tests of significance.
The commonly used statistical tests of significance, which are based on
various laws of probability, indicate to the investigator (researcher) whether
relationships found in sample data are sufficiently strong to be generalized to
the whole population.
• Because non-probability sampling violates the basic assumption of these tests
they should not be used on data derived from non-probability sampling.
• However, despite these drawbacks, non-probability sampling is used quite
frequently in experimental research, market research and public opinion
surveys.
• They are used because probability sampling is often a time-consuming and
expensive procedure and in fact, may not feasible in many situations.
Source of Errors in Survey
• Generally, the data observed in a sample don’t precisely mirror the data in the
population from which that sample was selected, even if the sampling and
measuring are done with extreme care and accuracy
• Survey errors can be divided into two major groups
– Errors of non-observation where the sampled elements make up only part of the
target
– Errors of observation where recorded data deviates from the truth
• Errors of non-observation can be attributed to
– Sampling,
– Coverage, or
– Nonresponse
• Errors of observation can be attributed to the
– Interviewer (data collector),
– Respondent,
– Instrument, or method of data collection
• The deviation between an estimate from an ideal sample and true population
value is called sampling error. Suppose θ is the parameter of interest (true
population value) and 𝜃 is an estimator of θ, then
Error of estimation = |θ - 𝜃|
• Sampling error can be measured theoretically and estimated from the sample
data for probability sampling techniques
• It is important to note that sampling error can be reduced by good survey
design and appropriate choice of sample size
• The nonresponse error arises in one of three ways
– The inability to contact the sampled element,
– The inability of the person responding to come up with the answer to the question of
interest
– Refusal to answer
Reducing Errors in Surveys
• Both errors of non-observation and errors of observation can seriously affect the
accuracy of a survey. Errors cannot be eliminated from a survey, but their effects
can be reduced by careful adherence to a good sampling plan
• Callback:
− Nonresponse can be reduced by having a carefully prepared plan for callbacks on
sampled elements
• Rewards and Incentives:
− Sometimes, an appropriate tactic for encouraging respondents is to offer a reward for
responding
• Trained interviews:
− The skill of the interviewer is directly related to the quantity and quality of data
resulting from a survey, whether the interview is in person or over the telephone
− Good interviewers can ask questions in such a way as to encourage honest responses
and can tell the difference between those who really don’t know the answer and those
who are simply reluctant to answer
• Data Check: Completed questionnaires should be scrutinized carefully by
someone other than the interviewer to see that the form has been filled out
correctly
− Back Checks and
− High Frequency Check
• Back checks
− It’s an important tool to detect fraud, for instance, enumerators filling out
questionnaires themselves
− It can be conducted by in-person visits, or through phone calls. A complementary
approach to in-person back checks is conducting random audio audits
− It allows researcher to resolve issues in data collection by improving enumerator
training, or replacing low-performing or problematic enumerators
• The following points are important when selecting the sample for the back check
survey
− Sample size: Aim to back check 10 - 20% of the total observations. Administer 20% of
back checks within the first two weeks of fieldwork. This helps the research team to
identify quickly whether the questionnaire is effective, whether enumerators are doing
their jobs well, and what changes to make to ensure high quality data collection.
− Stratified sampling: The back check sample should be stratified, that is it must cover
all survey teams and enumerators. Back check every team, and every enumerator
regularly, and as frequently as possible
− Independent team: The back check team should be independent from the rest of the
survey staff. Train them separately, and ensure that there is very little or no contact
between the back check team and the survey team
− Include missing respondents: This is to verify that there is no bias in the sample just
because enumerators did not track hard-to-find respondents.
− Include other flagged observations: Include observations that were flagged in other
quality tests like high frequency checks. Also include respondents who were
interviewed by enumerators suspected of cheating.
• Questions for the back check survey (or simply back check) are drawn from
the actual questionnaire which is used for data collection. There are five types
of questions that should be included in a back check to get a clear idea of the
data quality, as well as the enumerator’s skills
− Questions to verify respondent and interview information: Verify the identity of the
respondent and check if, when, and where the original survey took place. Useful for
verifying reported completion rates
− Questions to detect fraud: Questions that ask for straightforward information which
has no expected variation or room for error. They do not require particularly skilled
enumerators, and do not vary over time—especially the time period between the actual
interview and the back check. For example, questions about type of dwelling, education
level, marital status, occupation etc.
• Questions to detect errors in survey execution: Questions that have complex loops or
skip patterns, or check for consistency of recorded answers. For example, if household size
is recorded as 4, then the number of repeat groups for household members should not be
more than 4. Capable enumerators should get the true answer for these questions. If values
for these questions differ between the questionnaire and the back check survey, then the
enumerator may need more training
• Questions to detect problems with the questionnaire or key outcomes: Provide
additional checks for accuracy, and flag difficulties and/or inconsistencies in the
interpretation of the questions by enumerators. If these values differ between the actual
survey and the back check, then the enumerator may need more training
• Questions that repeat multiple times: Check whether enumerators are falsifying data to
reduce the length of interviews. For example, if there is a long series of questions about
each household member, verify that the number of times these questions repeat is equal to
the number of household members
• High Frequency Check: This check should be run daily, to provide real-time
information to the field team and research team for all surveys
− Completeness: Compare collected responses with the sample to make sure all data
points actually appear in the data set. Verify that there are no duplicate observations
− Consistency: Check if data shared by any particular enumerator is significantly
different from the data shared by other enumerators. Some parameters to check
enumerator performance include percentage of “Don’t know" responses, or average
interview duration
− Anomalous data points: One of the most common ways to deal with anomalous data
points (like outliers) is through carefully programmed questionnaires that set the
accepted range and/or type of values for each question