MODULE 3 SAMPLING AND

QUESTIONNAIRE DESIGN
AND CONSTRUCTION DR. HARDEE UDAYAN
SAMPLING
The process of obtaining information from a subset (sample) of a larger group
(population)
The results for the sample are then used to make estimates of the larger group
Faster and cheaper than asking the entire population
Two keys
1. Selecting the right people
Have to be selected scientifically so that they are representative of the
population
2. Selecting the right number of the right people
To minimize sampling errors I.e. choosing the wrong people by chance
Sample -- contacting a portion of the population
(e.g., 10% or 25%)
Best with a very large population (n)
Easiest with a homogeneous population

Census -- the entire population – most useful is the

population ("n") is small
or the cost of making an error is high
 POPULATION of
Interest

SAMPLE

We measure the sample

using statistics in order to
Population Sample(10) draw inferences about the
population and its
parameters.
Parameter Statistic
CHARACTERISTIC OF A GOOD
SAMPLE
Representative
Accessible
Low cost
 DEFINING THE TARGET
POPULATION
It is critical to the success of the research project to clearly
define the target population.

Rely on logic and judgment.

The population should be defined in connection with the

objectives of the study.
 TECHNICAL TERMINOLOGY
Population :The entire group of people of interest from whom the
researcher needs to obtain information.
Element (sampling unit) :One unit from a population
Sampling :The selection of a subset of the population
Sampling Frame : Listing of population from which a sample is
chosen
Census : A polling of the entire population Survey
A polling of the sample
 TECHNICAL TERMS
Parameter: The variable of interest, numerical
characteristic of a population
Statistic: The information obtained from the sample
about the parameter
Goal: To be able to make inferences about the population
parameter from knowledge of the relevant statistic - to
draw general conclusions about the entire body of units
Critical Assumption: The sample chosen is representative
of the population
STEPS IN SAMPLING PROCESS
1. Define the population
2. Identify the sampling frame
3. Select a sampling design or procedure
4. Determine the sample size
5. Draw the sample
 1. DEFINE THE TARGET POPULATION

It addresses the question “Ideally, who do you want to survey?”

i.e. those who have the information sought What are their characteristics.
Who should be excluded?
 age, gender, product use, those in industry – Geographic area
The target population for a toy store can be defined as all households with children
The Element ...... Individuals, families
Sampling Unit…. individuals over 20 families, with 2 kids seminar groups at ”new”
university
Extent ............ individuals who have bought “one” families who eat fast food
Timing .......... bought over the last seven days
 2. DETERMINE THE SAMPLING
FRAME
Obtaining a “list” of population (how will you reach sample)
Students who eat at McDonalds?
young people at random in the street?
phone book
Procedures; E.g. individuals who have spent two or more
hours on the internet in the last week
Select “sample units” like Individuals, Household, Streets,
Telephone numbers, Companies
 3. SELECTING A SAMPLING PROCEDURE
A) Probability sampling - equal chance of being included in the sample (random)
– simple random sampling
– systematic sampling
– stratified sampling
– cluster sampling
B) Non-probability sampling - - unequal chance of being included in the sample (non-
random)
– convenience sampling
– judgement sampling
– snowball sampling
– quota sampling
 PROBABILITY SAMPLING
means that every member of the population has a chance of being
selected.
It is mainly used in quantitative research.
If you want to produce results that are representative of the whole
population, probability sampling techniques are the most valid choice.
An objective procedure in which the probability of selection is
nonzero and is known in advance for each population unit.
Sampling error can be computed
Survey results can be projected to the population
1. SIMPLE RANDOM
SAMPLING
In a simple random sample, every member of the population has an
equal chance of being selected.
Your sampling frame should include the whole population.
Population members are selected directly from the sampling frame
(sample size/population size)
To conduct this type of sampling, you can use tools like random
number generators or other techniques that are based entirely on
chance
 SYSTEMATIC SAMPLING
Systematic sampling is often used instead of random sampling. It is also called an Nth
name selection technique.

After the required sample size has been calculated, every Nth record is selected from a list
of population members.

As long as the list does not contain any hidden order, this sampling method is as good as the
random sampling method.

Its only advantage over the random sampling technique is simplicity (and possibly cost
effectiveness).
 STRATIFIED SAMPLING

Stratified sampling is commonly used probability method that is superior to random

sampling because it reduces sampling error.

A stratum is a subset of the population that share at least one common characteristic; such as
males and females.

Identify relevant stratums and their actual representation in the population.

Random sampling is then used to select a sufficient number of subjects from each stratum.

Stratified sampling is often used when one or more of the stratums in the population have a low
incidence relative to the other stratums.
 CLUSTER SAMPLING
Cluster Sample: a probability sample in which each sampling unit is a
collection of elements.
Effective under the following conditions:
 A good sampling frame is not available or costly, while a frame listing clusters is
easily obtained

 The cost of obtaining observations increases as the distance separating the elements
increases
Examples of clusters:
 City blocks – political or geographical
 Housing units – college students
 Hospitals – illnesses
 Automobile – set of four tires
 NON-PROBABILITY SAMPLING
In a non-probability sample, individuals are selected based on non-random
criteria, and not every individual has a chance of being included.
This type of sample is easier and cheaper to access, but it has a higher risk of 
sampling bias.
That means the inferences you can make about the population are weaker than
with probability samples, and your conclusions may be more limited.
If you use a non-probability sample, you should still aim to make it as
representative of the population as possible.
Mostly in exploratory or qualitative research
 CONVENIENCE SAMPLING
Convenience sampling is used in exploratory research where the researcher is interested in
getting an inexpensive approximation.
A convenience sample simply includes the individuals who happen to be most accessible
to the researcher.
This is an easy and inexpensive way to gather initial data, but there is no way to tell if the
sample is representative of the population, so it can’t produce generalizable results.

The sample is selected because they are convenient.

It is a nonprobability method.

 Often used during preliminary research efforts to get an estimate without incurring the cost or time required to
select a random sample
 JUDGMENT SAMPLING
Also known as purposive sampling
involves the researcher using their expertise to select a sample that is most useful to the
purposes of the research.
The sample is selected based upon judgment.

an extension of convenience sampling

It is often used in qualitative research, where the researcher wants to gain detailed knowledge about a
specific phenomenon rather than make statistical inferences, or where the population is very small and
specific. An effective purposive sample must have clear criteria and rationale for inclusion.

When using this method, the researcher must be confident that the chosen sample is truly
representative of the entire population.
 QUOTA SAMPLING
Quota sampling is the nonprobability equivalent of stratified sampling.

 First identify the stratums and their proportions as they are represented in the
population

 Then convenience or judgment sampling is used to select the required number of

subjects from each stratum.
 SNOWBALL SAMPLING
Snowball sampling is a special nonprobability method used when the desired sample
characteristic is rare.

It may be extremely difficult or cost prohibitive to locate respondents in these situations.

This technique relies on referrals from initial subjects to generate additional subjects.

It lowers search costs; however, it introduces bias because the technique itself reduces the
likelihood that the sample will represent a good cross section from the population.
 SAMPLE SIZE?

The more heterogeneous a population is, the larger the sample needs to be.

Depends on topic – frequently it occurs?

For probability sampling, the larger the sample size, the better.

With nonprobability samples, not generalizable regardless – still consider

stability of results
 RESPONSE RATES
About 20 – 30% usually return a questionnaire

Follow up techniques could bring it up to about 50%

Still, response rates under 60 – 70% challenge the

integrity of the random sample

How the survey is distributed can affect the quality of

sampling
ERRORS IN SURVEY
Random Sampling Error
– random error
- the sample selected is not representative of the population due to chance
– the level of it is controlled by sample size
– a larger sample size leads to a smaller sampling error.
Population mean (μ)
gross income = $42,300 Sample 1 (400/250,000) mean (Χ) = $41,100 Sample 1
(400/250,000) mean (Χ) = $43,400
NON-SAMPLING ERRORS
The basic types of non-sampling error
Non-response error: A non-response error occurs when units
selected as part of the sampling procedure do not respond in whole
or in part
If non-respondents are not different from those that did respond,
there is no non-response error
Response or data error: A response or data error is any systematic
bias that occurs during data collection, analysis or interpretation
Respondent error (e.g., lying, forgetting, etc.) – Interviewer bias –
Recording errors – Poorly designed questionnaires
 PRIMARY DATA

Observation studies:
A scientific inquiry conducted to answer a research question.
Versatility of observation makes it indispensable
Besides observation; it involves listening, reading, smelling, touching,
OBSERVATION AND RESEARCH PROCESS
SELECTING DATA COLLECTION METHOD
SELECTING A OBSERVATION APPROACH
QUESTIONNAIRE DESIGNING
 WHAT IS A
QUESTIONNAIRE?
A tool for collecting information
to describe, compare, or explain
an event or situation, as well as,
knowledge, attitudes, behaviors,
and/or sociodemographic
characteristics on a particular
target group.
 QUESTIONNAIRE GENERAL
FORMAT

Self-administered (by mail or

personal contact)
In person (face-to-face)
Telephone interviews
 THE QUESTIONS OR ITEMS
Are the focus on any survey or
questionnaire
It is crucial to know how to ask
the questions in written and
spoken form
The way you ask the questions
determines the answers
 QUESTIONS CONTEXT

Identify the questionnaire’s specific

purpose(s)
Clarify the terms used to state the
questionnaire’s purpose(s)
Be sure to have the specific
objectives of the questionnaire
Know the respondents
 QUESTIONS
CONTEXT
Standardize the interviewer
Standardize the response format
Ask questions in a social, cultural, and
economic context
Keep confidentiality and voluntary
participation
Include a letter of introduction or
presentation
 QUESTIONS
FORMAT
Open questions - more
information but difficult to
codify, enter, and analyze
Closed questions - less
information but easy to codify,
enter, and analyze
 CHOICES AND
MEASUREMENT OF
RESPONSES
Nominal or categorical choices
Ordinal
Numerical
 HOW TO OBTAIN
VALID INFORMATION

Ask purposeful questions

Ask concrete questions
Use time periods based on
importance of the questions
Use conventional language
 HOW TO OBTAIN
VALID
INFORMATION
Use complete sentences
Avoid abbreviations
Review questions with experts
and potential respondents
Use shorter questions
 HOW TO OBTAIN
VALID
INFORMATION

Avoid two-edged questions

Avoid negative questions
Adopt/adapt questions used
successfully in other
questionnaires
BEFORE THE DESİGN OF
QUETİONNAİRE
Your approach should take consideration of:
Your topic
Who you want survey
The type of information you want
Your budget
How quickly you need the information
QUESTİONNAİRE DESİGN
Questionnaire design is a systematic process in which the researcher contemplates
various question formats, considers a number of factors characterizing the survey at
hand, ultimately words the various questions very carefully, and organizes the
questionnaire’s layout.
TYPES OF QUESTIONS IN A QUESTIONNAIRE

Open questions
 more information but difficult to codify, enter, and analyze

Closed questions
 less information but easy to codify, enter, and analyze
OPEN-ENDED QUESTIONS

Helps collect qualitative data in a questionnaire where

the respondent can answer in a free form with little to
no restrictions.

 What are your expectations/ requirements for this product

(Information gathering)
 What do you see as the next action steps?(Qualifying)
 What’s the most important priority to you with this? Why?
(Establishing rapport)
CLOSE-ENDED QUESTIONS
These are multiple choice type in which a respondent has to select one (single
select multiple choice question) or many (Multi-select multiple choice
question) responses from a given list of options. 

 Have you been stressed lately?

 Yes
 No
TYPES OF MULTİPLE CHOİCE
QUESTİONS
Likert Scale
TYPES OF MULTİPLE CHOİCE
QUESTİONS
Checklist Type:
TYPES OF MULTİPLE CHOİCE
QUESTİONS
Rank Order
TYPES OF MULTİPLE CHOİCE
QUESTİONS
• Rating Scale
COMMON MISTAKES
What might be What was the first drink you had
wrong with this
question? today?
 Tea
 Coffee
 Water
Need to cover  Juice
all options  Haven’t had a drink today
 Other (please state)
COMMON MISTAKES
What might be
wrong with this What is your age group?
question?
0-18 18-30 30-45 45+ 

0-18 19-30 31-45 46+

Options 
should be
mutually
exclusive.
COMMON MISTAKES
What might be
wrong with this What did you think about the
question?
waiting time?
Excellent Very Good Averag Poo
good e r

Options
should be ‘Good’ responses ‘Bad’
response
balanced
COMMON MISTAKES
What might be
wrong with this How would you rate the
question?
appointment booking service
using the:
Good Average Poor N/A
Website
Use opt-out Phone
Reception
responses
appropriately
DEMOGRAPHICS
Need to know how opinions and behaviors vary across different categories of people

Such things as
 Gender, age,
 Income, education levels
 Political beliefs

Put at end of instrument

 Easy to answer
TYPES OF QUESTİONNAİRE
SELF-COMPLETED
Respondents complete on their own

Best designed for

 Measuring variables with numerous values or response categories
 Investigating attitudes and opinions not usually observable
 Describing characteristics of a large population
 Studying ‘private’ or ‘difficult’ behaviors
SELF-COMPLETED
Closed-ended items limit the researcher
 Adjust for differences in respondents
 Clarify misunderstood items
 Explain ambiguity

Not suitable for all audiences

 Young children
 Visually impaired
 Learning/reading disabilities
HOW TO OBTAIN VALID INFORMATION

Ask purposeful questions

Ask concrete questions
Use time periods based on importance of the questions
Use conventional language
Use complete sentences
Use shorter questions
HOW TO OBTAIN VALID
INFORMATION
Avoid abbreviations
Avoid two-edged questions
Avoid negative questions
Adopt/adapt questions used successfully in other questionnaires
 WORDS TO AVOID IN QUESTIONNAIRE
DEVELOPMENT

These words: all, any, anybody, best, ever, every, never, etc. are all
EXTREME ABSOLUTES…
They place respondents in a situation where they must either fully agree or
they must completely disagree with the extreme position in the question.
 Do you always observe traffic signs?
 Would you say all cats have four legs?
COMPUTER ASSISTED AND
WEB-BASED
Way to create and administer self-administered questionnaires

Marketing researchers find response rates increase

 This is not across the board

Questionnaires (especially short ones) can be sent via email

 Or provide internet link to site which hosts survey

Maybe less time and costs

Access is a huge issue

INTERVIEWS
Reading questionnaire items in a face-to-face or telephone situation

Unstructured or in-depth
 Suited for exploratory research
 Either with one person or in focus groups
 Open-ended items
 Cannot standardize
 Good for complex situations
INTERVIEWS
Response rates tend to be highest with face-to-face interviews

Issues
 More time
 Smaller samples
 Higher cost
TELEPHONE
Most popular
 Less costly
 Less time
 Less subjective to interviewer
 As compared to face-to-face

Often conducted with computers

Can probe for information/clarification

Threshold about 20 minutes

CONSTRUCTIONS
Regardless of survey type, construct in a way

 Allows for candid answers

 Accurately

 Consistently

 Addresses goals, hypotheses, research questions

INSTRUCTIONS
Beginning of each section, should include clear instructions

Disclose needed information for respondents to decide whether

to complete the survey

Tell whether
 Anonymous: no names or IDs
 Confidential: names or IDs, not attached responses
 INSTRUCTIONS
Give an information about questionnaire time

Be consistent with instructions and format

 A little variation might break up ‘routine’

Avoid ambiguity, confusion, and vagueness.

 Make sure it is absolutely clear what you are asking and how you want it answered.
 Avoid indefinite words or response categories.
INSTRUCTIONS
Avoid asking questions beyond a respondent's capabilities

 People have cognitive limitations, especially when it comes to memory of past events.

 It is pointless to ask people about things that are not natural ways for them to think.
ONLİNE SURVEYİNG
PLATFORMS
[Link]/forms

[Link]

[Link]
 Önizleme

Soru tipi Soru ekle

seçenekleri
Başlık ve
açıklama ekle

Fotoğraf
ve video
ekle

Bölüm
ekle
 Ön izleme

Soru tipi
seçenekleri
Soru tipi
seçenekleri
CONSIDERATIONS WHEN
WRITING QUESTIONS
Type of question

Response formats

Question wording
TYPES OF QUESTIONS
Open-ended questions

Close-ended questions
 Ordered response categories
 Unordered response categories

Partially close-ended questions

OPEN-ENDED QUESTIONS
Responses are not provided to the respondent
Advantages:
Researcher does not need to know universe of possible answers
Respondent not influenced by specific alternatives suggested
Respondent can reveal what is most salient
Useful in exploratory work
Can be used to build rapport in interview
OPEN-ENDED QUESTIONS
Disadvantages:
Effort required of respondent
Respondents may vary in ability/willingness to articulate
Respondents may be reluctant to reveal detailed information or socially unacceptable
opinions or behaviors
Large amount of information may be revealed, information may be vague or
irrelevant
Difficulties in recording and in reducing and coding material
CLOSE-ENDED QUESTIONS
A list of acceptable responses is provided to the respondent
Advantages:
Easier for respondent
Communicates same frame of reference to respondents
Standardization
Less variability in interviewer performance
Less time to administer and record response
CLOSE-ENDED QUESTIONS
Disadvantages:
Need to know appropriate response categories in advance
Lack of spontaneity permitted respondent
Respondent may be forced into an unnatural frame of reference
May suggest response categories respondent has not thought of
Respondent may not feel as involved or motivated by questionnaire
RESPONSE FORMATS
Multiple categories that exhaust all meaningful answers and are mutually exclusive

During a typical work week (40 hours), how many hours do you spend on health
promotion in the community?:
_____less than 10 hours
_____10 to 19 hours
_____20 to 29 hours
_____30 to 39 hours
_____40 hours or more
ORDERED RESPONSE
FORMATS
Response categories are ordered along a gradient. Examples:
Strongly agree to strongly disagree (3 to 7 point scale; include or not include a neutral response
category)
Excellent, Good, Fair, Poor
Numerical rating scales
No
Complete Confidence
Confidence at All
______________________________
1 2 3 4 5 6 7 8 9 10
UNORDERED RESPONSE
FORMAT
No single dimension underlies response categories. Respondent must evaluate each.
Example:
Which one of the following do you think is most responsible for the long waiting
period in the clinic? (Choose only ONE answer.)
1. Low staff morale
2. Poor staff training.
3. Many patients.
4. No other healthcare options near by.
PARTIALLY CLOSE-ENDED
QUESTIONS
Answer choices are provided and respondents have the opportunity to create their
own responses. Example:
What is your position at this school?
Classroom teacher
Principal
Guidance counselor
Nurse
Other position (Please specify:____________)
VISUAL ANALOG SCALE
Subjective format for collecting data
Instead of defining all the categories, you define only the extremes
Leads to a more personal (but variable) perspective of response
Often used to assess pain levels
Sometimes uses symbols that are recognizable, especially for children or illiterate

 . . .  . . . 
FORMATTING
Create a form to obtain consent
 Explain reasons for study
 Explain how to respond to questions

Put questions in blocks that are related

Begin with emotionally neutral questions – demographics
Remember questionnaire fatigue and don’t put anything important at
the end
 CONSIDER SKIP
PATTERNS
 These are questions that are not appropriate for everyone
 Use of these may cause confusion
 Use may depend on whether the questionnaire is self-administered or administered by a computer or
interviewer
WORDING
Use language that is simple, free of ambiguity and encourages accurate and honest
answers
Avoid embarrassing or offending respondent
Make sure that there is clarity in how to express questions (if interviewer led)
Use vocabulary appropriate for your audience
Write questions like people talk not like people write
Translate and back-translate
QUESTION WORDING - DO
Use techniques for enhancing recall
 Shorten the reference period
 Use landmarks to aid dating
 Provide a helpful context
 Provide cues to stimulate recall
 Ask about typical behavior
QUESTION WORDING – DON’T
Don’t use double-barreled questions
Where do you go to get information about agricultural technologies and obtain seeds?

When I get ill, I know it is because I have not been eating right or washing my hands.
QUESTION WORDING – DON’T
Do not use leading questions
Do you agree that all children should be vaccinated?
With economic conditions the way they are these days, is it fair to have more
than one or two children?
Can you tell me when you last visited the clinic?
QUESTION WORDING – DON’T
Provide incomplete or overlapping response categories
Where have you received health care in the past 12 months?
____Health clinic
____Hospitals
____Private clinic

*** Use check-all-that-apply format

QUESTION WORDING – DON’T
Do not ask respondents to make unnecessary calculations.
Out of 100 women your age, how many to you think take pills?

Do not use loaded questions or loaded words

Have you ever stolen anything?
PRACTICAL STANDARDS
FOR EVALUATING QUESTIONS
Is this a question that will mean the same thing to everyone?
Is this a question that people can answer?
Is this a question that people will be willing to answer, given the
data collection procedures (i.e., sexual health or income
questions)?
CONSTRUCTING THE QUESTIONNAIRE:
PUTTING THE QUESTIONS IN ORDER
Beginning – inviting, interesting, non-threatening questions
 Demographic information

Middle – most important, put difficult and sensitive toward end

 Sexual health
 income

Closing – easy questions again, often routine, background

 Participation in community activities
QUESTION DESIGN
Include instructions, as needed, with questions - not at the beginning of
the questionnaire
Clearly differentiate response categories from questions
Be consistent in placement of answer boxes
Ask one question at a time: don’t stack side-by-side
Number questions consecutively and simply from beginning to end
OTHER ELEMENTS
Consent forms
Title/cover page
General instructions
Identifiers (e.g., respondent ID)
Transitions
CREATING NEW
QUESTIONNAIRES
Generate potential items for the instrument
 Use qualitative data collection to inform – focus group or in-depth interviews

Test it
Correct it
Pilot it again
Train interviewers and data entry people well
STEPS IN ASSEMBLING
INSTRUMENTS
List of variables potentially useful (conceptual framework)
Collect existing measures (justification)
Draft survey (long version to be revised and shortened later)
Pre-test
Validate –– are items measuring what you think they are…
ADMINISTERING
INSTRUMENT
Questionnaires vs. Interviews
 Questionnaires –
 Self-administered (may cause bias in responses)
 Less expensive
 Interviews –
 Administered verbally (advantage when person is illiterate)
 Helps for complicated surveys
 More costly and time-consuming

Choice depends on costs and complexity of study

INTERVIEWING
Standardize approach
 Train, train, train,
 Document, document, document
 Standardize wording, stick to it
 Avoid interviewer bias
 Neutral probing
DATA CODING, ENTRY, AND
ANALYSIS
Beyond the scope of this workshop
Many available resources
Some are in your CD in your packet, including the UNICEF, ORC Macro, CARE,
and UNAID survey guides
SUMMARY
Decide what information (variables) is needed
Draft or obtain questions to elicit that information
Put questions in meaningful order
Add other elements of questionnaire
Pretest questionnaire
Repeat
Remember: you can use these guidelines for pre- and post-test too!

Allow more time than you think!

GROUP ACTIVITY PREPARATION DISCUSSION
Now, you will adapt components of the example PHE baseline
questionnaire to monitor and evaluate your community-based PHE
program.
If you already have a program/project, think about a mid-way or final
program survey (or you could consider questions to use in focus
groups, interview, etc.)
Determine general study design (what communities, where, how many
people, who (men, women, youth, etc.)
How will you collect the data? Will the survey be self-administered,
interviewer administered, etc.
Can you use skip patterns? If so, which types of questions would you
skip and for whom?
 SMALL GROUP ACTIVITY
Go back to your M&E plan (logic model and framework) - 6 indicators
 What indicators require a household survey? Focus group? Interviews? Records?
 Which indicators are standard indicators [what number from the Guide]?

Look through the PHE baseline example questionnaire

Using the PHE baseline example questionnaire, determine:
 Can you get your 6 indicator information from the existing tool?
 What questions/sections would you keep?
 Which questions/sections would you delete?
 What sections or components would you add?

Each indicator may require more than one question to get the information
 Think through the numerator information
 Consider the denominator information
 Think about your data and indicator needs. Do costs, timing, other constraints make you
rethink your chosen indicators? Can you collect them?
 SUGGESTED READINGS

How to Conduct Surveys by Fink

and Kosecoff, Sage Publications,
1998
Survey Research by Roger
Sapsford, Sage Publications,
1999