Using Survey Research
I. Survey administration
1
Administering Your Questionnaire
■ MAIL SURVEY
■ A questionnaire is mailed directly to participants
■ Mail surveys are very convenient
■ Non-response bias is a serious problem resulting in an
unrepresentative sample
■ INTERNET SURVEY
■ Survey distributed via e-mail or on a Web site
■ Large samples can be acquired quickly
■ Biased samples are possible because of uneven computer
ownership across demographic groups
2
Administering Your Questionnaire
■ TELEPHONE SURVEY
■ Participants are contacted by telephone and asked questions
directly
■ Questions must be asked carefully
■ The plethora of “junk calls” may make participants
suspicious
■ GROUP ADMINISTRATION
■ A questionnaire is distributed to a group of participants
at once (e.g., a class)
■ Completed by participants at the same time
■ Ensuring anonymity may be a problem
3
Administering Your Questionnaire
■ INTERVIEW
■ Participants are asked questions in a face-to-face structured
or unstructured format
■ Characteristics or behavior of the interviewer may affect the
participants’ responses
4
Administering Your Questionnaire
■ In general
■ Personal techniques (interview, phone) provide
higher response rates, but are more expensive and
may suffer from bias problems.
5
2. Overview of Questionnaire
Construction
6
Parts of a Questionnaire
■ In any study you normally want to collect
demographics – usually done through
questionnaire
■ Single items
■ Composite items
7
Questionnaire Construction
■ Items can be optional. Flow often depicted
verbally and/or pictorially.
14. Have you ever participated in the
Model Cities program?
[ ] Yes
[ ] No
If Yes: When did you last attend
a meeting?
_________________
8
Questionnaire Construction
■ Many heuristics for ordering questions, length
of surveys, etc. For example:
■ Put interesting questions first
■ Demonstrate relevance to what you’ve told
participants
■ Group questions in to coherent groups
9
3. Types of Questionnaire Items
• Restricted (close-ended)
– Respondents are given a list of alternatives and
check the desired alternative
• Open-Ended
– Respondents are asked to answer a question in
their own words
• Partially Open-Ended
– An “Other” alternative is added to a restricted
item, allowing the respondent to write in an
alternative
10
Types of Questionnaire Items
• Rating Scale
– Respondents circle a number on a scale (e.g., 0 to
10) or check a point on a line that best reflects their
opinions
– Two factors need to be considered
• Number of points on the scale
• How to label (“anchor”) the scale (e.g., endpoints only or
each point)
11
Types of Questionnaire Items
– A Likert Scale is a scale used to assess attitudes
• Respondents indicate the degree of agreement or
disagreement to a series of statements
• I am happy.
Disagree 1 2 3 4 5 6 7 Agree
– A Semantic Differential Scale allows
participate to provide a rating within a bipolar
space
• How are you feeling right now?
Sad 1 2 3 4 5 6 7 Happy
12
Writing Good Items
■ Use simple words
■ Avoid vague questions
■ Don’t ask for too much information in one question
■ Avoid “check all that apply” items
■ Avoid questions that ask for more than one thing
■ Soften impact of sensitive questions
■ Avoid negative statements (usually)
13
Two Most Important Rules in
Designing Questionnaires?
■ Use an existing validated questionnaire if you
can find one.
■ If you must develop your own questionnaire,
pilot test it!
14
Acquiring A Survey Sample
■ You should obtain a representative sample
■ The sample closely matches the characteristics of
the population
■ A biased sample occurs when your sample
characteristics don’t match population
characteristics
■ Biased samples often produce misleading or
inaccurate results
■ Usually stem from inadequate sampling procedures
15