Approaches to Qualitative Research

Methods

Ashley Cole and Paul Copoc

The Research Shop
October 2022
Table of Contents
1. Background
2. Questionnaires
3. Unstructured Interviews
4. Semi-Structured Interviews
5. Focus Groups
6. Community/Large Meetings
7. Comment Card/Book
8. Document Review
9. Observation
10. Case Scenario or Vignette
11. Art-based Approaches
12. Graffiti Walls
13. Photographs/Images
14. References
Background
Hope House is a nonprofit offering services and programs that challenge the stigmas surrounding
poverty focused on dignity, choice, and skill development for long term self-sufficiency
The team at Hope House regularly collects quantitative data but are seeking methods to capture
more contextual data using qualitative tools. They are interested in developing a more structured
approach to collecting qualitative data in the evaluation of their programs.
Therefore, the purpose of this project is to develop a list of qualitative research methods that
Hope House can use internally to evaluate their programs.
Questionnaires
What is it?
• A tool with a series of questions and prompts to collect
people’s thoughts and feelings1,2
• Participants can complete surveys online with programs
like Qualtrics or Survey Monkey. Or they can write out or
draw their answers on paper for hard copy forms1
• Questions are pre-determined beforehand to allow for
standardization1
When would you use it?
• To understand the needs of a community and identify the
issues most important to them1
• For easy distribution to geographically disperse
participants with high anonymity, when used
electronically.
o Hard copy allows for easy distribution to participants who
might experience barriers online2
Considerations for Questionnaires
E xpertise & Skills
• Survey design background may be helpful as the wording of questions and
question flow can bias responses1
• Hard copy questionnaires require data entry2
• Online questionnaires require compatibility across browsers and platforms2
Costs & Time
• Can collect a large sample of data in a short time frame and with little costs2
• For hard copy questionnaires, there might be additional costs for materials2
• Large datasets may require a lot of time to analyze data2
Limitations
• Online questionnaires require access to computers and internet2
• Online questionnaires require respondents to have computer and literacy
skills2
• Respondents might also not take online questionnaires seriously and/or
might drop out2
Unstructured Interviews
What is it?
• A method where researchers collect data by asking
open-ended questions3
• Typically, there are no pre-determined set of questions4
o Useful for gathering stories and/or comparing responses
• You can collect participants’ perspectives, ideas,
opinions, and experiences regarding a specific research
topic, usually in verbal and/or written formats5
When would you use it?
• Useful to investigate a research question in depth6
• When you are looking to gain insight on “why”
participants are behaving in a certain way, by allowing
participants to speak in depth in their own words4
• When you find the rigid nature of the structured
interviews (e.g., Hope House Intake Form) limiting6
Considerations for Unstructured Interviews
E xpertise & Skills
• It is highly recommended that facilitators are thoroughly trained in this
method because unstructured interviews can be difficult to facilitate1
• Facilitators should have skills such as, building rapport with interviewees,
knowing when and how to probe, knowing how to keep the interview on
topic, and understanding how to ask questions to diverse populations that
may have various barriers, such as language, technology, hearing, or vision2
Costs & Time
• Interviews are time consuming, from contacting participants, to interviewing
them, to analyzing the large amount of data3
• Training facilitators can be time-consuming and expensive1
Limitations
• This method can provide a large amount of qualitative data, which may be
difficult to analyze with qualitative methods such as a thematic analysis3
Semi-Structured Interviews
What is it?
• This method is a mix of the structured interview (e.g., Hope
House Intake form) and the unstructured interview6
• You can collect participants’ perspectives, ideas, opinions,
and experiences regarding a specific research topic, usually
in verbal and/or written formats5
• Researchers pre-plan a set of questions, however, they may
not ask them all and let the interviewee elaborate by asking
them open-ended questions6
When would you use it?
• Useful to investigate a research question in depth6
• When you are looking to gain insight on “why” participants
are behaving in a certain way, by allowing participants to
speak in depth in their own words4
• This method allows for the rich data of the unstructured
interview without the confines of the structured method6
Considerations for Semi-Structured Interviews
E xpertise & Skills
• Facilitators should be thoroughly trained because semi-structured interviews
can be difficult to facilitate5
• Facilitators should have skills such as, building rapport with interviewees,
knowing when and how to probe, knowing how to keep the interview on topic,
and understanding how to ask questions to diverse populations that may have
various barriers, such as language, technology, hearing, or vision5
Costs & Time
• Interviews are time consuming, from contacting participants, to interviewing
them (which can last 15 minutes to an hour and a half), to analyzing the large
amount of data6
• Training facilitators can be time-consuming and expensive4
Limitations
• Considering the open-ended nature of the questions, it is best to pilot this type
of interview in advance to ensure you are getting the data that you want6
• This method can provide a large amount of qualitative data, which may be
difficult to analyze with qualitative methods such as a thematic analysis6
Focus Groups
What is it?
• Structured, semi-structured, or unstructured interviews where
groups of, typically, 4 to 20 participants are interviewed
together to discuss a common experience3,5,6
• Researchers must prepare questions in advance and be
prepared to facilitate the group discussion (i.e., keep the
group on topic, keep good time, know when to probe for
clarity and more detailed responses7
When would you use it?
• This method can be used for marketing research, education
research, program planning, and formative and summative
evaluations7
• What distinguishes this format from other methods is that
participants can discuss and debate the questions which can
lead to rich data8
Considerations for Focus Groups
E xpertise & Skills
• The role of the moderator is to create a welcoming environment where participants
feel comfortable sharing their ideas7
• The moderator should ask probing questions but keep interjections to a minimum,
knowing when to move on to another topic7
• The moderator should be aware of non-verbal utterances such as laughter,
coughing, etc. Non-verbal utterances should be transcribed along with verbal
responses7
Costs & Time
• Group interviews can be more productive and less time consuming than conducting
several single interviews7
• It can be costly and time-consuming to train moderators, and transcribe responses3
• It can be a complex and time-consuming task to analyze the large amount of data5
Limitations
• Due to the group aspect, confidentiality is not possible3
• This method of data collection may not be useful when collecting data on sensitive
topics as participants might not be comfortable sharing sensitive information3
Community/Large Meetings
What is it?
• An open forum for community members to attend and voice their
thoughts on issues9,10
• Led by (a) facilitator(s) who follows a semi-structured agenda10
• Community members work together to prioritize the issues and
identify solutions10
• Notes or a transcript can be analyzed to create an action plan9,10
When would you use it?
• When there are pressing issues affecting a large portion of the
community10
• When you want to involve community members in creating an action
plan10
Considerations for Large Meetings
E xpertise & Skills
• Will need (a) facilitator(s) with facilitation skills (I.e., ability to create a welcoming
and non-judgmental space, remain neutral and respectful, and engage in active
listening skills)9
• Will also need (a) note-taker(s) with note-taking skills (I.e., write legibly)9
Costs & Time
• Data collection process can be quick (from 1 to a few forums)10
• Will need a space and materials (such as easel and markers and/or recorder)9
• May want to consider providing accommodations (such as childcare) and
incentives (such as food, raffles, or gift certificates)10
Limitations
• Can be difficult to find a day and time that works for everyone10
• Hold meetings in an accessible space (I.e., AODA compliant and accessible by
public transportation)10
• May want to use language translators10
• If have large turnout (i.e., more then 30 community members attend), may want to
break up into smaller groups to allow for everyone to participate10
Comment Card/Book
What is it?
• To provide feedback after an encounter (such as a program or
service), in the form of comments, criticisms, and suggestions11
• Collected on a one-page card or in a guest book on an on-going
basis11,12
• Comment cards often have a set of pre-determined questions11,12
while comment books often don’t13
When would you use it?
• To explore community members changing needs over time11,12
• To quickly flag and address issues11,12
• To assess a program at any given time and/or overtime11,12
Considerations for Comment Card/Book
E xpertise & Skills
• Some background on survey design may help improve accessibility and rigour11
Costs & Time
• Little to no cost, other than providing materials such as paper and pens11
• Likely requires minimal time and effort to design, collect, and analyze data11
Limitations
• This method may not be appropriate for sensitive topics, as community members
may feel uncomfortable disclosing certain experiences in front of staff or other
community members. Can improve confidentiality in environment, such as having a
collection box11
• May be difficult to understand and answer questions due to language and literacy
barriers. Can use simple, familiar, and unambiguous language and pictures or a
translator11
Document Review
What is it?
• A way of collecting data through reviewing existing documents14,15
• Documents may be internal or external to the organization (I.e., from
ministries, government agencies, and NGO’s)14,15
• Documents may be in hard copy or electronic format and may
include reports, logs, data files, proposals, and meeting minutes14,15
When would you use it?
• To better understand the background context of an organization
and/or program unobtrusively (I.e., the history, philosophy, and
operation)14
• To better understand the needs of the community you are serving
unobtrusively (I.e., demographics, history, health, housing,
economy)15
• To help develop research questions for future projects (such as
evaluating a program)14
Considerations for Document Review
E xpertise & Skills
• Can be done independently without additional expertise or input from others15
Costs & Time
• Can be inexpensive14,15
• Depending on accessibility of documents and data, could be time consuming to
collect, review, and analyze14,15
Limitations
• May need permission to review certain documents14
• May need to consider ethical guidelines when working with documents with
sensitive information (I.e., how going to maintain confidentiality)14
• Documents may be of poor quality (I.e., disorganized, inapplicable; inaccurate or
incomplete)14,15
 Observation
What is it?
• Watch and document behaviours and/or events in their natural setting14,17
• Can document behaviours and events through
o Behavioural checklists: behaviours are checked off or rated on a scale16,17
o Observation guides: open-ended questions with space for observers to write in
their answers16,17
o Field notes: open space for observers to record any notes they have16,17
When would you use it?
• To gather data on people’s behaviours and/or interactions (rather than self-
reported behaviours, which may be inaccurate due to biases)14,17
• To collect data on an event at is happens (rather than after it happens)14,16,17
This Photo by Unknown
• To better understand the physical environment14,16,17 Author is licensed under
CC BY-SA-NC

• When data collection from participants is not possible (people are not able to
provide information themselves)14,16,17
Considerations for Observation
E xpertise & Skills
• Some training on recording observations might be necessary if have multiple
observers to assure that notes between observers are consistent16
• Behavioural checklists and observation guides are easier to collect and analyze
because they are structured, but do not allow for flexibility when unexpected events
occur. Field notes allow for flexibility, but can be more difficult to analyze16
Costs & Time
• Can be a time-consuming depending on the number of observations needed and
type of analysis conducted14
Limitations
• Observers may carry biases of people and events that influence their observations14
• Can reduce some biases by observing multiple events and having multiple
observers17
• People may act differently if they know they are being observed. Researchers may
be able to collect data without telling participants, but there are ethical concerns
with doing so14
Case Scenarios or Vignettes
What is it?
• A hypothetical scenario prompts participants to reflect on the
scenario and how they might respond18
• The scenarios may be written, spoken, pictures, or videos18
• The presentation of the scenario may be followed up with structured
questions or an open discussion18
When would you use it?
• When you want to understand people's attitudes, feelings, or
behaviours if put in a situation18 This Photo by Unknown Author is licensed under CC BY-
SA

• Allows insights into topics that are complex or sensitive in nature,

while removing people from the situation18
• Also, can be used as a tool for evaluating participant’s knowledge on
a topic19
Considerations for Case Scenarios or Vignettes
E xpertise & Skills
• To ensure vignettes are realistic and appropriate, experience within the context of
study is needed
• Some background on creating hypothetical scenarios would be helpful18
Costs & Time
• Can vary, depending on number of scenarios, participants, and collection method
for responses
Limitations
• Even though participants may be removed from the situation, vignettes may still
cause some emotional distress for participants18
• How people respond to vignettes may be different then how they would respond in
real life18
Art-Based Approaches
What is it?
• A creative and engaging method in which participants use art to
communicate their experiences20,21
• Can involve different mediums, such as visual (carvings, quilting, painting,
sculpting), narrative (poetry or novels), or performative (theatre and dance)20
• Participants are often involved in the research process and therefore often
help shape research questions and interpret their artwork20
• Can also be used to disseminate findings, through exhibits and/or
performances20
When would you use it?
• To better understand the needs and experiences of community members,
especially when certain topics might be difficult to verbalize for community
members (such as those who experience language barriers or major
traumas)20
• To publicize a problem in order to stimulate social action or change, as it can
foster reflection and dialogue20,21
Considerations for Art
E xpertise & Skills
• Participants and researchers might need some training on the medium for data
collection and analysis20
• Depending on the amount and types of data collected, the data can be
overwhelming to analyze and may need someone with some skills in data analysis20
Costs & Time
• Can be a time-consuming method to carry out, as it may be difficult to find
participants20
• There might be costs associated with the method, such as for supplies, for
collaborating with artists, and for incentives20
Limitations
• Some participants may find art to be challenging and therefore may not wish to
participate20
• There may be ethical issues around protecting privacy of participants and who gets
ownership of artwork20
• Artwork can involve a lot of emotional labour for participants20
Graffiti Walls
What is it?
• Interactive medium
• Participants can freely write or draw their informal opinions on an
open canvas displayed in public22
• Can read others’ comments and encouraged to add their own
• Asks for community members’ thoughts and/or feelings on an issue
relevant to the space they are in22
When would you use it?
• To collect baseline information that will help inform additional
research22
• When there may be challenges collecting data in spaces through
other methods (such as observation, interviews, or surveys)22
Considerations for Graffiti Walls
E xpertise & Skills
• N/A
Costs & Time
• Quick and easy collection of data, as method is casual and readily available to
community members22
• Method is of low to no cost, as everyday supplies can be used (such as, post-it
notes and pens)22
Limitations
• Have little control and knowledge over who participates22
• Unable to ask follow-up questions22
• Can increase participation through putting in common and accessible spaces and
providing materials22
• Graffiti walls may deteriorate over time or be mistaken for vandalism. Could take
on-going photos and maintain over time to overcome this22
Photographs/Images (aka Photovoice)
What is it?
• Participants take photos/videos and notes that represent their
experiences and/or environment2,23
• Can use photographs and/or visual recordings for analysis. Participants
can also explain the meaning behind their photos in notes/interviews or
focus groups22
• Participants often shape the research questions themselves23
• Can display findings through other means then a report, such as through
an exhibit or film2,23
When would you use it?
• Participants can record and share their voice through digital storytelling
when other methods might be inaccessible (such as for individuals who
may experience language barriers) 2,23
• To assess community member’s needs as can get better insight into their
lived experiences2,23
• Photos/videos can be used to publicize a problem in order to stimulate
social action or policy change2,23
Considerations for Photographs/Images
E xpertise & Skills
• Participants may need some instruction on camera operations and ethics around
photography (such as getting consent for subjects who might be recorded)2
• If conducting additional interviews and/or focus groups, may require facilitators
and data analysts2,23
Costs & Time
• Participants will need access to equipment (cameras) to participate2,23
• Can be time consuming for participants. But can offer compensation for their time23
• Method can be costly (depends on lengths of time, number of participants, cameras
offered, and incentives)2,23
• Depending on output can be time-consuming for researchers as well2,23
Limitations
• May be difficult to participate if have limited experiences with cameras, access to
cameras, time, and energy. See suggestions above to address2,23
References
1. Terry, G., & Braun, V. (2016). Short but often sweet. In V. Braun, V. Clarke & D. Gray (Eds).
Collecting Qualitative Data: A Practical Guide to Textual, Media and Virtual Techniques.
Cambridge University Press.
2. Center for Community Health and Development. (n.d.). Chapter 3, Section 13: Conducting
Surveys. University of Kansas. [Link]
contents/assessment/assessing-community-needs-and-resources/conduct-surveys/main
3. Mcleod, S. (2014). The Interview Research Method. [Link]. Retrieved 29
September 2022, from [Link]
4. Centre for Studies on Poverty and Social Citizenship. (2019). Community-Based Research
Toolkit: A Project Development Checklist. Carleton University.
[Link]
5. Alsaawi, A. (2014). A critical review of qualitative interviews. European Journal of Business and
Social Sciences, 3(4).
6. McLafferty, I. (2004). Focus group interviews as a data collecting strategy. Journal of advanced
nursing, 48(2), 187-194.
7. Basch, C. E. (1987). Focus group interview: an underutilized research technique for improving
theory and practice in health education. Health education quarterly, 14(4), 411-448.
References
8. Kahn, R. L. (1991). The Focused Interview: A Manual of Problems and Procedures.
9. Center for Community Health and Development. (n.d.). Chapter 3, Section 3: Public Forums
and Listening Sessions. University of Kansas. [Link]
contents/assessment/assessing-community-needs-and-resources/conduct-public-
forums/main
10. UCLA Center for Health Policy Research. (n.d.) Community Forums.
[Link]
11. Wisner, J. D., & Corney, W. J. (1997). An empirical study of customer comment card quality and
design characteristics. International Journal of Contemporary Hospitality Management, 9(3),
110-115.
12. Orlikowski, W. J., & Scott, S. V. (2015). Exploring material‐discursive practices. Journal of
Management Studies, 52(5), 697-705.
13. Macdonald, S. (2005). Accessing audiences: visiting visitor books. Museum and Society, 3(3),
119-136.
14. CDC. (2018). Data collection methods for evaluation: Document review.
[Link]
References
15. WBI Evaluation Group. (2007). Document Review.
[Link]
16. Taylor-Powell & Steele (1996). Collecting evaluation data: Direct observation. University of
Wisconsin-Extension. [Link]
17. Cottage Health Evaluation Toolkit (n.d.). How to plan and conduct direct observation. Cottage
Population Health. [Link]
fdb0186810f1/Collect_Data_Plan_Conduct_Observations_Cottage_Health_Evaluation_Toolkit.p
df
18. Sood, S., Cronin, C., & Kostizak, K. (2018). Participatory research toolkit. Rain Barrel
Communications.
[Link]
8389431/1583728710133/Participatory+Research+Toolkit+Rain+Barrel+[Link]
19. Veal, W. R. (2002). Content specific vignettes as tools for research and teaching. Electronic
Journal of Science Education, 6(4), 1-37.
20. Coemans, S., & Hannes, K. (2017). Researchers under the spell of the arts: Two decades of
using arts-based methods in community-based inquiry with vulnerable populations.
Educational Research Review, 22, 34-49.
References
21. CIHR. (2022). Meet the Methods Series: Arts-Based Methods for Illuminating Gender in
Community-Based Research. [Link]
22. Martin, B., Hanington, B., & Hanington, B. M. (2012). Graffiti Walls. In Universal methods of
design: 100 ways to research complex problems. Rockport Publishers.
23. Sutton-Brown, C. A. (2014). Photovoice: A methodological guide. Photography and culture,
7(2). 169-185.