Street Mobility Survey Guide for Communities
Street Mobility Survey Guide for Communities
How to do a survey
A guide for local authorities,
voluntary organisations and
community groups
CONTENTS
CONTENTS....................................................................................................................... I
1. GATHERING INFORMATION ................................................................................... 1
Using questionnaires to gather information ................................................................................. 1
How can you administer questions? ............................................................................................... 1
How to administer questionnaires .................................................................................................. 2
Things to consider when deciding how to administer questionnaires ...................................... 2
Things to consider when designing your questionnaire .............................................................. 3
Informed consent ............................................................................................................................... 6
Sample (the group of people answering a questionnaire) ....................................................... 7
Additional information ..................................................................................................................... 7
Practical issues ................................................................................................................................... 7
Sensitive topics .................................................................................................................................. 8
Analysis ............................................................................................................................................... 8
2. ANALYSING AND INTERPRETING RESULTS ............................................................... 9
By the end of this section you should: ........................................................................................... 9
Analysis ............................................................................................................................................... 9
Interpretation .................................................................................................................................. 11
Making meaningful comparisons ................................................................................................. 12
Conclusion: results that are ‘fit-for-purpose’ ............................................................................. 12
3. USING THE RESULTS OF ANALYSIS .......................................................................... 13
Making use of the findings ........................................................................................................... 13
Disseminating the findings ............................................................................................................ 13
4. CHECKLIST ................................................................................................................ 14
5. SOURCES OF EXISTING, VALIDATED QUESTIONS .................................................... 14
Measuring well-being ................................................................................................................... 14
Physical activity .............................................................................................................................. 15
i
STREET MOBILITY PROJECT How to do a survey
1. GATHERING INFORMATION
By the end of this section you should:
Understand why questionnaires are a good way of gathering information
Be familiar with a range of additional questions you may wish to ask your respondents.
Most of these methods provide information in the form of words (“qualitative data”) and can be used
alongside questionnaire data to help explore findings in more detail. This handbook focuses on
gathering information using questionnaires.
1
STREET MOBILITY PROJECT How to do a survey
Self-completion questionnaire
This is where the respondent fills in the questionnaire by themselves. This tends to be in one of
these three ways:
o pen and paper
o email
o on the web
Interview
This is where an interviewer asks the respondent the questions. This tends to be either:
in person (face-to-face)
by telephone
It is important to note that there is no ‘right’ way to go about administering a questionnaire – you will
need to make an informed judgement about the best way to do it, taking into consideration factors
such as the resources you have at your disposal, your data needs, and the respondents you will be
surveying.
2
STREET MOBILITY PROJECT How to do a survey
Content
You will be conducting a survey because you want to know what the people you are interested in
think, feel, believe, and/or do about an issue that is important to you.
Where possible, it is good practice to use existing questions that have been developed carefully and
have been tested. Writing questions is a skilled job. It is too easy to ask questions where what you
mean and what the respondents think you mean are different!
Most questionnaires will focus on a particular topic. There is such a large range of potential topics that
this guide does not provide sample questions (except for background information, below). In Section 5,
we provide a list of sources for validated questions on ell-being and on physical activity.
Whatever the topic, it is important to remember that the shorter the survey, the more likely people are
to answer your questions. When designing a questionnaire, you always need to balance the amount of
detail you would like to collect against how many questions you can ask before potential respondents
feel the survey is taking too much of their time.
Background information
It is often useful to collect some basic information about the people answering the questionnaire in
addition to the main questions on the special topic. This can allow you to explore, for example, how
what people do or feel or problems they have vary among your target group according to gender,
age, ethnicity, working status, postcode etc.
The form on the next page is intended as an example only. You will need to think about what you
want to know and how you will analyse your data, and ask the relevant additional questions. We
have also included some questions on disability that you may find useful.
3
STREET MOBILITY PROJECT How to do a survey
Full time Education / Part Time Education (less than 16 hours) / Apprenticeship / Working full time /
Working Part time (less than 16 hours) / Internship / Stay at Home Parent / Volunteering / Other
(please specify)
4
STREET MOBILITY PROJECT How to do a survey
The next questions ask about difficulties you may have doing certain activities because of a HEALTH
PROBLEM.1
2. Do you have difficulty hearing, even if using a hearing aid? (Please circle)
a. No- no difficulty
b. Yes – some difficulty
c. Yes – a lot of difficulty
d. Cannot do at all
5. Do you have difficulty (with self-care such as) washing all over or dressing? (Please circle)
a. No – no difficulty
b. Yes – some difficulty
c. Yes – a lot of difficulty
d. Cannot do at all
6. Using your usual (customary) language, do you have difficulty communicating, for example
understanding or being understood? (Please circle)
a. No – no difficulty
b. Yes – some difficulty
c. Yes – a lot of difficulty
d. Cannot do at all
1 These are the internationally agreed core set of questions on disability, called ‘The Washington Six’ which you may wish
to include.
5
STREET MOBILITY PROJECT How to do a survey
Question order
While it is common for answers to survey questions to be affected by earlier questions in the same
survey, these effects have been found to be particularly strong for some measures such as subjective
well-being. If you are asking well-being questions as part of a larger questionnaire, they should be
asked as early as possible, although ideally after a few basic introductory questions to allow
respondents to feel relaxed about answering the survey.
Frequency of measurement
You will need to think about how often to measure your sample. You may want to do a survey once, to
find out how common certain issues are in your area. You may decide to do two surveys, for a simple
‘before and after’ measurement, if you want to find out whether what people think, feel or do has
changed. More frequent measurement will allow more detailed tracking of these issues, for example,
picking up shorter-term variation in response to particular events and seasons. This may not be
necessary for overall monitoring. It is important to repeat the same contact methodology and question
ordering to be able to compare survey data collected at different times.
Informed consent
You will need to think about obtaining informed consent from your survey participants. First, this means
that you should inform potential participants fully about the purpose of the exercise and what you are
going to do with the data that is collected about them. Secondly, it means you should seek consent to
collect data from them, and only collect such data if they have consented to it. You may wish people
to sign a short consent form as a record that consent has been given – examples of such forms can
readily be found on the internet. Verbal consent to take part is generally sufficient for answering
questionnaires. If you make it clear that people can choose whether or not they take part in the survey
and that they do not have to answer all the questions if they prefer not to, you can assume that
people who answer the questions are consenting to take part.
It is not usually necessary to collect information that can identify individual respondents to your survey:
you don’t need names, but using ID numbers to identify individuals (especially if doing before and
after measurement) can be useful. In some settings you will be able to assure respondents that their
answers will be confidential. At the very least, you should confirm that the information people provide
will only be used for the purposes that they have agreed to (see informed consent above).
6
STREET MOBILITY PROJECT How to do a survey
It may not be feasible to get data from everyone you would like. The most important thing to consider
in your sample is that you will want the responses that you receive to represent as closely as possible
the group of people (the ‘population’) you are seeking to understand.
It is always good practice to ask more people than you strictly need, because not everyone you ask is
likely to complete a questionnaire. For example, if you were measuring the self-reported health of
new mothers through parenting classes, and half of the participants had English as a second language
(ESL), you would ideally hope that the sample of questionnaires you received back included about
half from mothers with ESL. It may be that you need to ask more mothers with ESL than mothers with
English as a first language, in order to receive equal responses from both, as it is possible that if you
are doing a questionnaire in English, the rate of response may be lower among those with ESL.
Additional information
What other information might you need to know alongside the main data that is the focus of your
survey? Some examples are:
Demographic information like age, gender, ethnicity, marital status, residential postcode of
respondents (see part iii)
To what extent have respondents been affected by something changing locally?
Collecting such pieces of information can allow for more detail when you analyse data (see Part III
below), and help put your results in context. For example, if the sample of responses you receive is not
a good representation of the population (group of people) you are trying to understand, then you can
at least know which people within a population have been adequately covered by your sample.
Physical activity
Many surveys would like to ask questions on physical activity. Several different physical activity
questionnaires have been developed and validated.2 Some information about three of these
questionnaires are given in section 5.
Practical issues
Language issues
Are there any language barriers in terms of literacy or non-English speakers? Is there a need to re-
phrase questions if using them with children, or use more creative techniques to try and engage
people?
2 Bull F, Maslin T, Armstrong T. Journal of Physical Activity and Health 2009; 6: 790-804.
7
STREET MOBILITY PROJECT How to do a survey
Cost
Are there costs involved in the measurement process (e.g. to pay for external research
support/community researchers)? How will these costs be covered?
Sensitive topics
You may wish to think about whether any of the issues you are measuring are likely to be sensitive for
people. This may particularly be the case if, for example, you are working with a group of people
who have experienced mental illness. This may affect how you administer the questionnaire, for
example you may decide to ask people to complete it themselves. You should ensure that you can
signpost people towards appropriate support in the event that they disclose anything particularly
sensitive to you.
Response bias
Sometimes people respond to questions in terms of how they think they should respond rather than
how they actually feel. Technically, this is known as “social desirability bias”. You may want to consider
whether this is likely to be a significant issue in your measurement work.
If you are considering other methods of gathering data such as group discussions, you may wish to
consider whether group settings are more likely to encourage people to give responses they think they
should give, or that are socially acceptable.
Analysis
You will need to think about how you are going to analyse your data. Have you identified someone
with experience of handling quantitative data? What information do you want to know? Who will do
the analysis, and do they understand statistics? What analysis software is available? In the following
section we explore in more detail some of the issues around analysis and interpretation.
In general, you will not need a full analysis of all the data you can collect in the survey. You will need
‘indicative’ data, that is, enough information to take to the local authority, or other organisation, to
show that something is a problem, what proportion of the community is affected or concerned, or who
is affected.
8
STREET MOBILITY PROJECT How to do a survey
Analysis
Analysis is the process of turning lots of individual responses into an aggregate picture of the data.
You might start with a pile of completed paper questionnaires, administered before and after an
intervention, and process them to produce a summary of the responses. For example, on average,
people’s well-being in your study may have gone from 6.8 to 7.5 on a ten point scale. Or it may be
that the proportion of people who have difficulties walking around their local area because of traffic
has changed from 42% to 19%. The main analysis task is to find average scores across the sample of
individuals. You could have a look online to learn more about data analysis.
To analyse a large amount of data, it will be easiest to manage this electronically, using a computer
and software. There are free online survey tools which will be sufficient for data entry and basic
analysis of your questions which we recommend – such as Survey Monkey ([Link]).
Spreadsheet software such as Microsoft Excel is useful for holding data. There are free spreadsheet
programmes which can be downloaded from the internet as part of packages such as Open Office
([Link]/download/) and Google Docs ([Link]
If you are reading this guide because you plan to use UCL's Health and Neighbourhood Mobility
survey questionnaire, you can use the Microsoft Excel tool they have prepared for data entry and
simple analysis. You can download the Excel tool from [Link]/street-mobility/toolkit.
9
STREET MOBILITY PROJECT How to do a survey
Let’s consider an analogy: researchers are trying to find out if free toothbrushes made a difference to
the dental health of 200 children at a primary school over the course of one year. This is measured by
dental check-ups every three months throughout the year.
The results of dental checks for one child would not tell you meaningfully whether this is an effective
intervention. There are likely to be many other factors which affect this child’s dental health. In fact we
wouldn’t expect the dental health of all children to improve. Some will have eaten more chocolate –
others might have lost their toothbrush on the way home from school.
However, overall, across 200 children, we could expect this intervention to have a positive impact. We
might expect that although there will be a background of many different factors for many different
children, the average level of dental health will improve because of the toothbrushes.
Other outcomes are similar. There is unlikely to be any programme, activity or intervention which can
improve the self-rated health, for example, of 100% of participants. If the self-rated health of an
individual has improved, or got worse, it may very well be that there were other factors which were
an influence. This is why it is important to reach as large a number of participants as possible, as the
greater the numbers, the less chance that average scores are the result of specific, individual factors.
You will want to know if the data that you have received back from your sample and analysed is
reliable enough to make a meaningful. There are two basic processes you can go through to build
confidence in the reliability of your data.
Firstly, it may be that you have surprising results, in aggregate. After an intervention, the change may
have been smaller or larger than you expected. It may be that this is because the sample of people
who responded does not represent the population well. It may be possible to “weight” the sample,
which is a process of undertaking a mathematical calculation which balances the sample of responses
you have to reflect the population you want to understand. For example, if you want to understand the
dental health of the children at the primary school, and you know the school has 100 boys and 100
girls, you would want your sample to have 50% boys and 50% girls. If in reality you achieved 20
responses from girls and 60 responses from boys, you could count each of the responses from each of
the girls three times (give each girl respondent a weighting of 3), to make the sample better reflect
the known gender split among the whole school.
10
STREET MOBILITY PROJECT How to do a survey
Secondly, it may become obvious through your analysis that underneath the aggregate changes shown
through your overall analysis, the data actually shows different things for different types of people
within the sample. It should be possible for you to see how changes may be different for different age
ranges or gender groups, for example; that is, to look at how the scores vary between different types
of people. This might be one of the most interesting results, although to know this you will need to have
collected demographic information alongside asking questions on the main issue of interest. Even if
you have adjusted your sample to reflect your population characteristics (weighting, described
above), you may find that scores are higher for boys than for girls, or higher for older people than
for younger people, for example. You can then go on, in your interpretation, to explore why this might
be the case.
Finally, it is worth remembering that data which aims to capture changes in levels over time are just
one indication of the true, messy reality of change in people’s lives. Don’t worry too much about the
exact scores and numbers. What is important to be sure of is that you understand the scores in context
(are they generally high or low), the direction of change (getting bigger, smaller, better or worse over
time), and the size of any change (a big change or a small change over time).
Interpretation
If you are confident that your data is reliable and you are aware of the ways in which it might not be
reliable, interpretation of the data is the next step. Interpreting the results of your data analysis
means developing ideas about what you think the data mean: what do the results tell us and why this
might be the case?
We have come up with a list of three simple questions which can help start a useful conversation
about interpretation. It is best if this conversation is with colleagues and others who work with the
people you are surveying. People with experience of research in a range of different fields could
also be helpful.
11
STREET MOBILITY PROJECT How to do a survey
It is important to remember that findings can vary by chance. This is more likely to be the explanation
when the number of participants in a survey is small. Using statistical tests when analysing the data
can help you decide how likely it is that a difference is only by chance.
A rule of thumb is that a bigger sample size means that we can be more confident that the difference
in scores between two groups of respondents is statistically significant.3 If we had only received
responses from 10 boys, and 25 girls, the sample size is too small to rely on to understand whether
gender was a factor in the effectiveness of the intervention. Interpreting a 10% difference in dental
health between boys and girls, and assuming each gender had an adequate sample response; we
might then try and identify factors which could help to explain the difference.
Once results are collected, organisations are often keen to understand their results in context. Are the
results good or bad? What can they be compared against? Evaluating the effectiveness of the dental
intervention, we could seek to compare our results about dental health improvement in a number of
ways. We could investigate dental health indicators for a similar population (e.g. primary school aged
children), or look at another evaluation that had considered a similar or contrasting type of dental
health intervention.
The key question you need to make a judgement on is this: are the analysed results ‘fit for purpose’?
Are they useful enough to use in improving a service, or evidencing impact? Does the set of findings
feel sensible against other information you have about what is happening “on the ground”? If this is a
second survey, for example after an intervention, does the direction of change over time (up or down)
and the magnitude of change (big or small) seem sensible against what else you know? If the answer
to these questions is yes, then the exercise of collecting data has been worthwhile: the subjects of the
intervention are validating the judgements of the organisers, staff and evaluators.
3 The statistical significance of a result is the probability that the observed relationship (e.g., between variables) or a
difference (e.g., between means) in a sample occurred by pure chance ("luck of the draw"), rather than because there is a
real difference or relationship in the population. Findings are “statistically significant” when we have calculated that the
probability of getting a different result, if we repeated the data collection under the same conditions again, but with
different respondents making up the sample, is very low (typically, a 5% chance is used as a threshold): see
[Link] is "statistical significance" (p-level)
12
STREET MOBILITY PROJECT How to do a survey
How you use your findings will depend on what you explored, and on the results obtained. However, it
might be possible to think about using them in order to:
There are many ways in which you can share and disseminate the results. Select whatever methods are
best suited to your needs and resources.
4. CHECKLIST
Congratulations! You have nearly reached the end of our handbook for conducting a survey. By now
we hope you are:
Familiar with measurement issues
Familiar with some useful steps to meaningfully analyse and interpret your results
Familiar with some of the things to think about when deciding how to use results
Self-reported health (page 27), health behaviours (e.g. smoking) (page 44), social care (page 49)
Health Survey for England
[Link]
14
STREET MOBILITY PROJECT How to do a survey
Physical activity
The International Physical Activity Questionnaires (IPAQ)4 provides a set of well-developed
instruments that can be used internationally to obtain comparable estimates of physical activity
(Hagstromer et al 2006). It was designed specifically for adults (18-65 years) and consists of four
domains: (1) transportation; (2) occupational; (3) household and gardening tasks; and (4) leisure-time
activities. There are two versions of the questionnaire. The short version (nine items) is suitable for use
in national and regional surveillance systems and the long version (31 items) provide more detailed
information often required in research work or for evaluation purposes. The long- and short-forms of
the IPAQ questionnaire are available here: [Link]
The Global Physical Activity Questionnaire (GPAQ)5 covers several components of physical activity
and consists of three domains: occupational; transport-related; and leisure activities (Armstrong and
Bull 2006; Bull, Maslin, and Armstrong 2009). It comprises 19 questions. Within the occupational and
leisure-activities domains, questions assess the frequency and duration of two different intensity
categories (vigorous- or moderate-intensity). In the transport domain, the frequency and duration of
all walking and cycling for transport is captured but no attempt is made to differentiate between
these activities. One additional question asks about time spent in sedentary activities. The GPAQ
questionnaire is available here: [Link]/chp/steps/resources/GPAQ_Analysis_Guide.pdf.
The Neighborhood Physical Activity Questionnaire (NPAQ)6 measures walking, including that
undertaken within the neighbourhood, and overall physical activity (Giles-Corti et al, 2006). It is
based on the short form of the IPAQ and the Active Australia survey. It can be downloaded here:
[Link]/research/cbeh/projects/?a=382545
Original document on Measuring well-being written by Juliet Michaelson, Sorcha Mahony and
Jonathan Schifferes and published by the New Economics Foundation.7
Document revised to cover all surveys by: Jennifer Mindell and Jemima Stockton
4 Hagstromer M, Oja P, Sjostrom M. The International Physical Activity Questionnaire (IPAQ): a study of concurrent and
construct validity. Public Health Nutrition. 2006; 9(6): 755-762.
5 Armstrong T, Bull F. Development of the World Health Organization Global Physical Activity Questionnaire (GPAQ).
neighborhood: RESIDE’s Neighborhood Physical Activity Questionnaire. Preventive Medicine. 2006; 42(6): 455–9.
7 Michaelson J, Mahoney S. Measuring well-being. A short handbook for voluntary organisations and community groups.