COMMON RESEARCH DESIGNS AND DATA COLLECTION METHODS
This module discusses the common experimental designs and data collection methods.
Specific topics include field interview, and meta-analysis.
Field interview, (structured and unstructured)
This lesson focuses on the definition of structured interview and the detailed discussion
on unstructured interview.
Objectives:
1. Describe structured and unstructured interview.
2. Discuss the steps in conducting unstructured interview.
3. Explain the role of the interviewer.
INTRODUCTION
Interviews are a widely used tool to access people’s experiences and their inner
perceptions, attitudes, and feelings of reality. Based on the degree of structuring, interviews can
be divided into three categories: structured interviews, semi-structured interviews, and
unstructured interviews (Fontana & Frey, 2005). A structured interview is an interview that has a
set of predefined questions and the questions would be asked in the same order for all respondents.
This standardization is intended to minimize the effects of the instrument and the interviewer on
the research results. Structured interviews are similar to surveys (see the later chapter on survey
research), except that they are administered orally rather than in writing. Semi-structured
interviews (see the later chapter on this method) are more flexible. An interview guide, usually
including both closed-ended and open-ended questions, is prepared; but in the course of the
interview, the interviewer has a certain amount of room to adjust the sequence of the questions to
be asked and to add questions based on the context of the participants’ responses.
UNSTRUCTURED INTERVIEW
This chapter will focus on unstructured interviews as a qualitative research method for
data collection. The unstructured interview technique was developed in the disciplines of
anthropology and sociology as a method to elicit people’s social realities. In the literature, the term
is used interchangeably with the terms, informal conversational interview, in-depth interview,
nonstandardized interview, and ethnographic interview. The definitions of an unstructured
interview are various. Minichiello et al. (1990) defined them as interviews in which neither the
question nor the answer categories are predetermined. Instead, they rely on social interaction
between the researcher and the informant. Punch (1998) described unstructured interviews as a
way to understand the complex behavior of people without imposing any a priori categorization,
which might limit the field of inquiry. Patton (2002) described unstructured interviews as a natural
extension of participant observation, because they so often occur as part of ongoing participant
observation fieldwork. He argued that they rely entirely on the spontaneous generation of questions
in the natural flow of an interaction. While the definitions are not the same, there is more agreement
about the basic characteristics of unstructured interviews. The researcher comes to the interview
with no predefined theoretical framework, and thus no hypotheses and questions about the social
realities under investigation. Rather, the researcher has conversations with interviewees and
generates questions in response to the interviewees’ narration. As a consequence, each unstructured
interview might generate data with different structures and patterns. The intention of an
unstructured interview is to expose the researcher to unanticipated themes and to help him or her
to develop a better understanding of the interviewees’ social reality from the interviewees’
perspectives. While unstructured interviews can be used as the primary data collection method (as
in the two example studies discussed later in this chapter), it is also very common to incorporate
unstructured interviews into a study primarily based on participant observation (see the chapter on
that topic). Just because unstructured interviews don’t use predefined questions doesn’t mean that
they are random and non-directive.
Unstructured interviews cannot be started without detailed knowledge and preparation, if
you hope to achieve deep insights into people’s lives (Patton, 2002). The researcher will keep in
mind the study’s purpose and the general scope of the issues that he or she would like to discuss
in the interview (Fife, 2005). The researcher’s control over the conversation is intended to be
minimal, but nevertheless the researcher will try to encourage the interviewees to relate
experiences and perspectives that are relevant to the problems of interest to the researcher
(Burgess, 1982). The decision to use unstructured interviews as a data collection method is
governed by both the researcher’s epistemology1 and the study’s objectives. Researchers making
use of unstructured interviews often hold a constructivist point of view of social reality and
correspondingly design studies within an interpretive research paradigm. They believe that, to
make sense of a study participant’s world, researchers must approach it through the participant’s
own perspective and in the participant’s own terms (Denzin, 1989; Robertson & Boyle, 1984). No
hypothesis should be made beforehand and the purpose of inquiry is theory development rather
than theory testing.
In an ideal unstructured interview, the interviewer follows the interviewees’ narration and
generates questions spontaneously based on his or her reflections on that narration. It is accepted,
however, that the structure of the interview can be loosely guided by a list of questions, called an
aide memoire or agenda (Minichiello et al., 1990; Briggs, 2000; McCann & Clark, 2005).
Conducting an Unstructured Interview
There are no official and agreed-upon guidelines for how to conduct an unstructured
interview. But in practice, many researchers comply with the steps listed below (Punch, 1998;
Fontana & Frey, 2005) when planning and conducting unstructured interviews.
Step 1: Getting in: accessing the setting. Various difficulties in gaining access to research
settings have been documented, especially when the researcher is an “outsider” in the environment.
Negotiation techniques and tactics are required in this situation. The researcher also has to take
into consideration the possible political, legal, and bureaucratic barriers that may arise during the
process of gaining access to the setting (Lofland et al., 2006).
Step 2: Understanding the language and culture of the interviewees. A primary focus of an
unstructured interview is to understand the meaning of human experiences from the interviewees’
perspectives. Thus, unstructured interviews are governed by the cultural conventions of the
research setting. This requires that the researcher can understand the interviewees’ language and,
further, its meanings in the specific cultural context of the research setting (Minichiello et al., 1990;
Fife, 2005).
Step 3: Deciding on how to present oneself. An unstructured interview is a two-way
conversation. The quality of the conversation is influenced, to a great extent, by how the
interviewer represents him- or herself. The interviewer’s self representation will depend on the
context he or she is in, but in all cases, the interviewer is a “learner” in the conversation, trying to
make sense of the interviewee’s experiences from his or her point of view.
Step 4: Locating an informant. Not every person in the research setting will make a good
informant. The informant (i.e., the interviewee) will be an insider who is willing to talk with you,
of course. But even more importantly, the informant must be knowledgeable enough to serve as a
guide and interpreter of the setting’s unfamiliar language and culture (Fontana & Frey, 2005).
Step 5: Gaining trust and establishing rapport. Gaining trust and establishing rapport is
essential to the success of unstructured interviews. Only when a trustful and harmonious
relationship is established will the interviewee share his or her experience with the interviewer,
especially if the topic of the conversation is sensitive. When endeavoring to cultivate rapport, the
interviewer might need to be careful: it’s easy to become so involved with your informants’ lives
that you can no longer achieve your research purposes (Fontana and Frey, 2005).
Step 6: Capturing the data. Note-taking is a traditional method for capturing interview data.
But in an unstructured interview, note-taking is likely to disrupt the natural flow of the
conversation. Thus, when possible, it is preferable to audio record the interviews by tape or digital
recorder. In situations where only note-taking is possible, you will need to take brief notes during
the interview, writing up more detailed notes immediately after each interview (Fontana and Frey,
2005, Lofland, et al., 2006). As you develop your interviewing skills, you also will want to practice
a variety of memory techniques, to be able to capture as much detail as possible from each
interview.
The Role of the Interviewer
The interviewer has a unique position in an unstructured interview. He or she is an integral
part of the research instrument, in that there are no predefined frameworks and questions that can
be used to structure the inquiry. To a great extent, the success of the interview depends on the
interviewer’s ability to generate questions in response to the context and to move the conversation
in a direction of interest to the researcher. Thus, an unstructured interview is more open to
interviewer effects than its structured and semistructured counterparts. To become a skillful
interviewer takes knowledge and experience (Minichiello et al., 1990). The role that an interviewer
adopts is critical to the success of an unstructured interview. The choice of roles is constrained by
many characteristics of the interviewer, such as gender, age, social status, race and ethnicity. Even
so, it is generally preferable that the interviewer present him- or herself as a learner, a friend, and
a member of the interviewee’s group, who has sympathetic interest in the interviewee’s life and is
willing to understand it (Burgess, 1984). Adopting this kind of role makes building rapport
between the interviewer and interviewees possible; it further makes in-depth understanding of the
interviewees’ lives possible. The merit of an unstructured interview lies in its conversational
nature, which allows the interviewer to be highly responsive to individual differences and
situational changes (Patton, 2002). This characteristic of unstructured interviews requires
interviewers to have a rich set of skills. First, the interviewer should be able to listen carefully
during the conversation. The interviewer often starts the interview with a very broad and open
question, such as, “How do you feel about the …?”. The interviewee then can take over and lead
the conversation. In such conversations, the interviewer usually listens and reflects more than he
or she talks. Second, in order to adjust the interview direction in response to the individual
interview context, the interviewer has to be able to “generate rapid insights [and] formulate
questions quickly and smoothly”
Most importantly, interviewers should be good at questioning, probing, and adjusting the
flow of conversations at an appropriate level. This skill is reflected in three aspects of the
interviewer’s questioning tactics. First, interviewers should be adept at using the appropriate type
of question, based on the specific interview context. The kinds of questions posed are crucial to
the unstructured interview (Burgess, 1984). Spradley (1979) identified three main types of
questions: descriptive questions, which allow interviewees to provide descriptions about their
activities; structural questions, which attempt to find out how interviewees organize their
knowledge; and contrast questions, which allow interviewees to discuss the meanings of situations
and make comparisons across different situations. Each type of question is used at different points
in the interview to encourage interviewees to talk or to probe for more details. Second, interviewers
should be able to monitor and control the directiveness of their questions, comments, and even
gestures and actions (Burgess, 1984). It is important for interviewers not to ask directive questions
when initiating the interview because directive questions may bias the data by leading interviewees
to respond in a way that they thought was expected or desired by the researcher. Patton (2002)
cautioned that interviewers should “guard against asking questions that impose interpretations on
the situation” (p.343). Denzin (1989) also pointed out that a “sympathetic identification” with
interviewees’ points of view is necessary, but the interviewer should avoid giving advice and/or
passing judgments on respondents (Denzin, 1989, p.109). Whyte (1960) provided a six-level scale
to evaluate the degree of directiveness in any question or statement made by the interviewer by
examining it in the context of what immediately preceded it during the interview. Controlling and
adjusting the directiveness of questions and statements is a big challenge for interviewers,
especially for those with little interviewing experience. Third, interviewers should be able to
maintain control of the pace and direction of the conversation. While the interviewer allows the
interviewee to raise new topics or move the conversation in directions that the interviewee believes
are important, it is the interviewer’s responsibility to engage the interviewee in the conversation
and keep the conversation focused on the researcher’s concerns. As Minichiello et al. (1990) note,
an unstructured interview is “always a controlled conversation, which is geared to the interviewer’s
research interests” (p.93). A productive conversation is possible when a balance of control is
achieved.
Meta-analysis
This lesson discusses meta-analysis, a quantitative form of literature review. It focuses on
the decisions a meta analyst makes in conducting such a review and how these decisions affect the
conclusions reached through meta-analysis. It explains the importance of meta-analysis by making
clear the sources of bias and subjectivity in meta-analysis, its problems, ambiguities and ways of
dealing with them, and the usefulness of the technique vis-à-vis less-quantitative forms of literature
review.
Objectives:
[Link] importance of literature review in meta-analysis.
2. Explain continuum of quantification in meta-analysis.
3. Discuss the status of meta-analysis.
INTRODUCTION
When scientists want to get a clear picture of whether a drug works or a particular behavior
is linked to a poor health they need to combine all of the evidence available however this will
generally not be contained in a single scientific study meta-analysis is the science of combining
evidence across multiple studies to address a research question as to finish early as possible for
example we might want to ask does aspirin effectively reduce pain for patients after an operation
to do this we would need to collect the results of all the clinical trials that have assessed this
treatment and combine their results to get an overall picture
Understanding Meta-Analysis
It is a systematic approach to combine findings from various studies, enabling researchers
to find a consensus on treatment effectiveness and health behaviors. This is crucial for evidence-
based medicine.
Weighting Studies
In a meta-analysis, studies with more participants carry more weight, providing a more
accurate estimate of treatment effectiveness. This helps prioritize findings from larger,
more credible studies over smaller ones.
Impact of Publication Bias
Not all studies are published equally; those showing positive results are more likely to be
shared. This bias can distort the true picture of treatment efficacy, leading to skewed
healthcare recommendations.
Correcting Bias
Researchers can use statistical methods to adjust for biases in the data, aiming to provide a
more balanced view of effectiveness. This involves analyzing data asymmetries to restore
integrity to the findings.
Influence on Healthcare Policies
Government organizations rely heavily on meta-analyses as a gold standard for evidence.
Their findings guide clinical practices and healthcare policies, ensuring patients receive the
best care possible.
Foundation of Evidence-Based Medicine
Meta-analysis serves as a cornerstone for evidence-based medicine, integrating diverse
research findings to inform effective treatment protocols and improve patient outcomes.
Importance of Meta-Analysis
Meta-analysis has gained importance due to the increasing amount of research on similar
topics, often with varying results. Rather than treating each study as standalone, meta-analysis
offers a way to resolve discrepancies and provide a unified conclusion. It’s a valuable tool in fields
like medicine, psychology, education, and social sciences, where researchers often deal with
complex human behaviors and varying experimental conditions.
Statistical Models of Meta-Analysis
Fixed Effect-Model
Assumes that all studies are estimating the same true effect size.
Random-Effect Models
Assumes that the studies estimate different but related effect sizes
REFERENCES
Zhang, Y., & Wildemuth, B. M. (2009). Unstructured interviews. Applications of social research
methods to questions in information and library science, 222-231.
Guzzo, R. A., Jackson, S. E., & Katzell, R. A. (1987). Meta-analysis analysis. Research in
organizational behavior, 9(1), 407-442.
John Lester L. Lim