DATA COLLECTION TOOLS

AND INSTRUMENTS

AUTHORS

Jyotikumarie Juggernath: African Vision Research Institute (AVRI), Durban, South Africa
Kovin S Naidoo: Brien Holden Vision Institute (BHVI), African Vision Research Institute (AVRI), Durban, South Africa
Urmilla Bob: University of KwaZulu Natal (UKZN), Durban, South Africa
Vadivelu Moodley: African Vision Research Institute (AVRI), Durban, South Africa

PEER REVIEWER

James Loughman: Dublin Institute of Technology (DIT), Dublin, Ireland

DATA COLLECTION METHOD

The data collection method or research instrument influences a number of factors. Different data collection methods
allow for very different sample sizes, and the number of respondents determines the degree of reliability with which
the results may be generalised to various population segments. Each method has its own strengths and limitations;
however, time and resource constraints are deterministic issues in the final choice. The choice of data collection
method is not an isolated decision in survey design, as it influences the whole statistical survey process. For instance,
the choice of data collection method or instrument used has an impact on the design and preparation of the
questionnaire, on the quantity and quality of the data that are to be collected, and on the cost and the timetable of the
survey. Research instruments range from interviews, observational techniques such as participant observation and
fieldwork, through to document or desk based research. Written data sources can include published and unpublished
documents, company reports, memos, letters, reports, email messages, faxes, newspaper articles and so forth.

QUESTIONNAIRES
A questionnaire survey is a process of conducting a study of specific groups or populations by using a set of written
questions that is asked to the targeted population to obtain reliable and valid measures in the form of verbal responses
from one or more respondents. It is a mode of eliciting the feelings, beliefs, experiences, perceptions, or attitudes of a
particular sample of individuals. As a data collecting instrument, it could be structured or unstructured. There are a
variety of questionnaire styles: closed-ended questionnaires, structured interviews and observation using data
recording sheets. Sample range and size are critical factors in deciding to use a questionnaire to collect data. This
method will give a rapid picture of overall trends. It is the process of conducting a study of specific groups or
populations. Interviews are among the most challenging and rewarding forms of measurement. They require personal
sensitivity and adaptability as well as the ability to stay within the bounds of the designed protocol.

Feb 2013, Version 1-1 Research Methodology and Project Design

Chapter 6-1
 Data Collection Tools and Instruments

TYPES OF QUESTIONNAIRE INTERVIEWS

Face-to-face interviews are a direct communication, primary research collection technique

that is used during questionnaire surveys and is generally considered to be the most
expensive data collection method. The benefits of this method, in terms of fulfilling quota
requirements speedily, far outweigh the financial costs, particularly if the study population
is limited. Additional benefits include the fact that interviewing affords the opportunity of
ensuring that the respondent is the intended one, and allows for further explanation of
questions, where necessary. A questionnaire is most frequently a very concise, pre-
FACE TO FACE planned set of questions designed to yield specific information to meet a particular need for
research information about a pertinent topic.
A possible disadvantage of the face-to face interviews is the possibility that the interviewer
might influence the outcome of the questionnaire directly through his / her influence on
respondent answers through interviewer bias or other issues. Also, where multiple
interviewers are involved, the possibilities of variable effects on the study outcome are
critically dependent on interviewer skill and training and avoidance of variable interviewer
bias effects.

Telephonic interviews are a cost efficient and time saving method of gathering data. The
information you need can be gathered quickly with a direct communication over the
telephone, avoiding the hassles of paying for transport to the location of the respondent.
The interview responses can be directly entered onto the electronic survey, thereby saving
printing costs. A major advantage of doing telephonic interviews is that people who are
busy, travelling or reside in other parts of the world can be interviewed directly over their
mobile lines. Furthermore, people in inaccessible locations such as prisons or hospitals
TELEPHONIC can also be telephoned. The likelihood of respondents answering sensitive issues over the
SURVEYS phone is increased since there is no direct face-to-face contact made with the interviewee.
Despite the advantages of the telephonic interview method, it is extremely difficult to
observe the respondent’s expressions and reactions to the questions and to gauge
responses. Additionally, many people do not like to be disturbed at home or have lengthy
discussions over the telephone. Some people are very reluctant to give honest/ true
answers over the phone because they are unaware of the interviewer’s background and
identity. It is also important to consider that people in inaccessible locations often do not
have access to telephones.

Feb 2013, Version 1-1 Research Methodology and Project Design

Chapter 6-2
 Data Collection Tools and Instruments

The electronic survey method has many advantages worth considering. The surveys are
generally sent out via the internet or through email and therefore the respondent can
directly enter the responses onto the survey template and send back to the interviewer.
This eliminates the costs of entering and editing data, as the received surveys can be
directly converted and analysed using a statistical package. The data cleaning and editing
can be done easily on the electronic format. In addition the costs for administering the
questionnaires are also saved because there are no postage or fieldwork expenses. There
ELECTRONIC
are no risks of interviewer bias using this method as surveys are sent out randomly to a
SURVEYS
large percentage of the population and the interviewer cannot influence people’s
responses. Because there is no printing of questionnaires, there is no space constraint in
developing the questionnaire. Questions can be lengthy and elaborative and responses
can be as detailed as possible. Electronic surveys can also include pictures, videos or links
to audio clips to help the respondent better understand the situation through visualization.
As with the telephonic interview, people are more open to answering sensitive questions
when there is no face-to-face contact with the interviewee.

However, it is important that one considers that many people may not have access to
computers and that there is a risk of information loss, since the information can be ‘lost’ in
bits or fully, in transit and viruses can delete information from the system.

Mail surveys are generally directed at very large samples whereby the questionnaires are
sent by postage to the respondents contact address. This method can prove to be less
expensive where the sample numbers are too large because there are no fieldwork costs.
Pictures, samples and small tapes or CDs/ DVDs can be included in the postage.
Respondents have the ability to answer the questions at their leisure and are not pressured
to answer the questions immediately. This way, the questions are thought about and the
answers are more relevant.

MAIL IN SURVEYS However, mail surveys have some disadvantages. These include the high printing and
postage costs and longer waiting return periods because people often forget or are too
busy to return the surveys. Often the surveys may not be returned at all and follow up
reminders can be costly. A ten percent response rate to a sample of 1000, for example,
means that you will need to mail 10 000 questionnaires to possibly have 1000 returned.

Feb 2013, Version 1-1 Research Methodology and Project Design

Chapter 6-3
 Data Collection Tools and Instruments

ADVANTAGES AND DISADVANTAGES OF QUESTIONNAIRE INTERVIEWS

The advantages of using the questionnaire method, according to Bailey (1994), are:
• Flexibility when the respondent indicates misunderstanding, interviews can be used to
probe for more specific answers and can repeat a question.
• Interviews tend to have a better response rate than the mailed questionnaire. People
that are unable to read and write can still answer questions in an interview, and others
who are unwilling to expand their energy to write out their answers may be glad to talk.
ADVANTAGES • Control over the question order is held by the interviewer which ensures that the
respondent does not answer the questions out of order, or in any other way other than
the structure of the questionnaire.
• The interviewer can ensure that all of the questions are answered.
• The interviewer can record the exact time, date, and place of interview.
• They are more scientific in nature than unstructured interviews.
• They introduce controls that permit the formulation of scientific generalisations.
Some disadvantages of using a questionnaire are:
• Unstructured interviews often yield data too difficult to summarise or evaluate.
• Training interviewers, sending them to meet and interview their informants, and
evaluating their effectiveness all add to the cost of the study.
• Structured interviews are rigidly standardised and formal.
• The same questions are presented in the same manner and order to each subject.
• The choice of alternative answers is restricted to a predetermined list.
DISADVANTAGES • The same introductory and concluding remarks are used.
• Possibility of interviewer bias affecting the outcome
• The design of the questionnaire is highly important. Inappropriate or faulty design issues
can influence the study outcomes e.g. questions can be misunderstood if not designed
appropriately.
• The design can influence the respondent’s willingness to answer – e.g. sensitive
questions may be more honestly answered using anonymous, non-face to face
methods.

THINGS TO CONSIDER IN DESIGNING A QUESTIONNAIRE

• Avoid long/ complex questions
• Ask questions in the language that is most understandable to the respondent
• Avoid questions / words that can have multiple meanings or be interpreted differently
• Avoid asking dual or double barrel questions - e.g. do you think students should eat less and exercise more?
• Avoid the use of highly scientific or technical terms and abbreviations / acronyms
• Avoid making assumptions
• Avoid troublesome questions
• Avoid long lists of options
• Use specific time and place frames in question
• Question your questions – if your question does not make sense to you, then it is likely that it will not make
sense to any other person.
• Factors that contribute to bias – such as limiting responses to questions around the responses that you are
looking for.
• Define concepts and terms to respondent if he / she does not understand

Feb 2013, Version 1-1 Research Methodology and Project Design

Chapter 6-4
 Data Collection Tools and Instruments

QUESTIONNAIRE DESIGN

QUESTION SEQUENCE / PLACEMENT OF QUESTIONS

A well-developed questionnaire should be able to meet the research objectives. It is important to make certain that
provision is made on the questionnaire to introduce the research, explain why the respondent’s participation is
needed and how the respondent’s answers will be used. This includes explaining to the respondent his / her rights as
a participant and the level of participant confidentiality.

The questionnaire should be well structured and have a logical flow of questions. For example, if the first question
asks about diabetic retinopathy, the questions that follow in this section should be related to diabetic retinopathy and
not jump to cataracts. Cataracts should be a different section in the questionnaire, including questions relating to
cataracts.

When structuring the questions, there should always be a few warm-up / general questions placed before those
questions that attempt to answer the research objectives. For example:

1. Are you aware of any eye condition / problem that you may have? Yes/ No
2. How did you become aware of the condition/ problem?
3. What is the name of the eye condition / problem that you have (tick below)?

Condition Tick Response

Cataract
Refractive error
Diabetic retinopathy
Glaucoma
Trachoma
Other (specify)

Following the warm up questions you can then move on to asking the following:

4. How does your eye condition impact on your ability to work for a living?
5. Do you think that you could be more productive in your work if you did not have cataract? Yes / No. Why?
6. Have you considered cataract surgery? Yes / No. Why?

Questions should be classified into themes. For example, questions that relates to productivity should be in a
Chapter called “productivity” and not scattered throughout the questionnaire.

For example:

Section 1: Demographic profile of respondents

Section 2: General awareness of refractive errors
Section 3: Refractive errors impact on respondents
Section 4: Feeling about refractive errors

Below each section, there should be a list of questions that relate to the section theme.

QUESTION LAYOUT
The questionnaire should be well presented with a strong structure, and questions should have an appropriate, neat
layout. Lengthy questionnaires should be divided into themes and numbered appropriately in sync with the sub
headings. The numbering in the questionnaire is extremely important as it serves as a guide to which question
should be asked next. Questions that include a skip provide a guide to the next question and if the numbering is
incorrect, relevant questions could not be asked and left unfilled.

Feb 2013, Version 1-1 Research Methodology and Project Design

Chapter 6-5
 Data Collection Tools and Instruments

CLOSED AND OPEN-ENDED QUESTIONS

Closed-ended questions limit respondents’ answers to the question and allow the
respondent to choose from an already developed set of answers, such as yes / no,
true / false, or multiple choices. Closed-ended questions also allow for an option for “other-
specify” or ranking scale response options. The most common of the ranking scale
questions is called the Likert scale question. This type of question asks the respondents to
look at a statement (such as "It is difficult to read close up if you are short sighted") and
then "rank" this statement according to the degree to which they agree (I strongly agree, I
somewhat agree, I somewhat disagree, I strongly disagree). Scale levels should be
consistent and not change from question to question.

For example:

If the scale was set in the first question as:

I strongly agree, I somewhat agree, I somewhat disagree, I strongly disagree

Do not ask another question using a different scale:

I disagree, I agree, I strongly disagree, I strongly agree

TYPES OF Multiple choice responses allow the respondent to choose more than one response from a
CLOSED-ENDED list of options. When designing this type of question, it is essential to number the items in
QUESTIONS the category box correctly, to check that responses are not duplicated and to allow for the
“don't know / no opinion / none of the above” option. Avoid order bias according to the
things that you are aware of that are more important.

Example of a Closed-Ended Question

Indicate the main problems people with poor eyesight face (multiple responses):

Problems Please Tick

Unable to read the newspaper, books, etc
Cannot carry out household chores, e.g. cooking ü
Cannot work to bring in income ü
Cannot take care of children
All of the above
Other (specify)

Open-ended questions do not give respondents answers to choose from. Questions are
phrased in a way that the respondent explains their answers and reactions with a sentence
or in more detail. Questions include "What do you think?”, “In your opinion”, “justify /
TYPES OF substantiate your response”, “how is / does”. These questions elicit the respondent’s own
OPEN-ENDED answers.
QUESTIONS
Example of Open-Ended Question

What are the main problems resulting from your poor eyesight?

Feb 2013, Version 1-1 Research Methodology and Project Design

Chapter 6-6
 Data Collection Tools and Instruments

RESPONSE TYPES

Precise responses are exact information that is required. Questions are closed-ended and
direct. These responses can be re-coded by a computer into groups and can be analysed
using descriptive statistics (e.g. mean and standard deviation). However, questions that
require precise answers are often too intrusive on the respondent’s privacy because
respondents are not allowed to give explanations. For example: “Does your husband beat
PRECISE you? Yes / No” may not be answered candidly.
RESPONSES
Examples:
st
• Give your age (in years) on the 1 Dec 2007
• What is the size of your house?
• What is the total amount of household income per month (in the local currency)?

Questions asking respondents to rank several items in order of importance are commonly
found in questionnaires. Questions are generally phrased as “Rank a set of options in order
of importance by numbering them from 1 to ____”.

For example:

Rank the three main challenges to eye care service delivery:

RANKED Challenges Ranking

RESPONSES
Lack of Human Resources 2
Poor infrastructure and facilities 3
Financial Constraints 1
Inadequate delivery mechanisms

LIKERT SCALE
The Likert Scale is an ordered, one-dimensional, psychometric scale from which
respondents choose one option that best aligns with their view. It is the most widely used
scale in survey research. Respondents indicate their level of agreement with a question. A
RATED RESPONSES
common form is an assertion, with which the person may agree or disagree to varying
degrees.

For example: Likert scale:

Strongly Somewhat Somewhat Strongly

Neutral
agree agree disagree disagree
People aged >40 get presbyopia

Good Neutral Bad

When I can’t cook for my family because I can’t see properly, I feel:

Almost
Never Rarely Frequently Always
always
I feel sad because I cannot read
due to my poor eyesight

Feb 2013, Version 1-1 Research Methodology and Project Design

Chapter 6-7
 Data Collection Tools and Instruments

RESPONSE TYPES (cont.)

A benefit is that questions used are usually easy to understand and so lead to consistent
answers. A disadvantage is that only a few options are offered, with which respondents
RATED RESPONSES
may not fully agree.
(cont.)
Measurements with ordinal scales are ordered in the sense that higher numbers represent
higher values. However, the intervals between the numbers are not necessarily equal.
Be careful of:

o Undecided responses
YES / NO o Not applicable
QUESTIONS o Do not know
o No response

These are totally different responses and do not indicate Yes or No!

QUESTIONNAIRE IMPLEMENTATION
Before implementing the questionnaire, make certain that the questionnaire includes the following:

• A covering letter detailing the purpose of the study.

• Ethical Consent and Information Documents

o Indicating the confidentiality and anonymity of the responses
o Reassuring that the questionnaire does not ask offensive, insensitive and uncomfortable questions
o Reassuring that the questionnaire does not ask for data that is not essential / needed for the study
o Be truthful – Tell the participant about the benefits and risks associated with the study, including:
- The purpose of the survey
- How the information will be used
- How long it will take to complete the questionnaire.

• Decide on the method that will be used to administer the questionnaire

o Face-to-face interviewing
o Telephone surveys
o Electronic surveys
o Mail in surveys
o Self-administered questionnaires.

• The questionnaire needs to be pre-tested / piloted which includes peer evaluations, practice on a group of
people with similar characteristics that do not form part of the study sample, and the revision of the
questionnaire as per the findings of the pre-test.

• Questions should not be unanswered and the interviewer needs to make certain that where there is no
response, it should be stated as “no response”.

Feb 2013, Version 1-1 Research Methodology and Project Design

Chapter 6-8
 Data Collection Tools and Instruments

RASCH ANALYSIS
Rasch models are used for analysing data from assessments to measure variables such as abilities, attitudes, and
personality traits. For example, they may be used to estimate a student's reading ability from answers to questions
on a reading assessment, or the extremity of a person's attitude to capital punishment from responses on a
questionnaire.

A researcher who is developing items of a test or questionnaire intending to sum the scores on the items can use a
Rasch model analysis to check the degree to which this scoring and summing is defensible in the data collected. For
example, if two groups are to be compared on the variable of interest (e.g. males and females), it is important to
demonstrate that the workings of the items is the same in the two groups. Working in the same way permits
interpreting the total score as meaning the same in the two groups.

In checking how well the data fit the model, it is important to be able to diagnose very quickly where the misfit is the
worst, and then proceed to try to understand this misfit in terms of the construction of the items and the
understanding of the variable in terms of its theoretical development.

A very important part of the Rasch analysis from this perspective is to be in dynamic and interactive control of an
analysis and to be able to follow the evidence to see where the responses may be invalid.

Feb 2013, Version 1-1 Research Methodology and Project Design

Chapter 6-9