TYPES OF MIXED METHODS DESIGNS
There have been several typologies for classifying and identifying types of mixed methods strategies
that proposal developers might use in their proposed mixed methods study. Creswell and Plano Clark
(2011) identified several classification systems drawn from the fields of evaluation, nursing, public
health, education policy and research, and social and behavioral research. In these classifications,
authors used diverse terms for their types of designs, and a substantial amount of overlap of types
existed in the typologies. For purposes of this discussion, I will identify and discuss the three basic
mixed methods designs (as shown in Figures 10.1 and 10.2) and then briefly mention more advanced
strategies that incorporate these three basic forms. Each approach will be discussed in terms of a
description of the design, the forms of data collection and data analysis, interpretation, and validity
challenges.
Convergent Parallel Mixed Methods Design
Description of the design. The convergent mixed methods approach is probably the most familiar of
the basic and advanced mixed methods strategies. Researchers new to mixed methods typically first
think of this approach because they feel that mixed methods only consists of combining the
quantitative and qualitative data. In this approach, a researcher collects both quantitative and
qualitative data, analyzes them separately, and then compares the results to see if the findings confirm
or disconfirm each other (see Figure 10.1). The key assumption of this approach is that both
qualitative and quantitative data provide different types of information—often detailed views of
participants qualitatively and scores on instruments quantitatively—and together they yield results
that should be the same. It builds off the historic concept of the multimethod, multitrait idea from
Campbell and Fiske (1959), who felt that a psychological trait could best be understood by gathering
different forms of data. Although the Campbell and Fiske conceptualization included only quantitative
data, the mixed methods researchers extended the idea to include the collection of both quantitative
and qualitative data.
Data collection. The qualitative data can assume any of the forms discussed in Chapter 8, such as
interviews, observations, documents, and records. The qualitative data can be instrument data,
observational checklists, or numeric records, such as census data, as discussed in Chapter 7. The key
idea with this design is to collect both forms of data using the same or parallel variables, constructs,
or concepts. In other words, if the concept of self-esteem is being measured quantitatively, the same
concept is asked during the qualitative data collection process, such as in an open-ended interview.
Another data collection issue is the sample size for both the qualitative and quantitative data
collection process. Unquestionably, the data for the qualitative data collection will be smaller than
that for the quantitative data collection. This is because the intent of data collection for qualitative
data is to locate and obtain information from a small sample but to gather extensive information from
this sample; whereas, in quantitative research, a large N is needed in order to conduct meaningful
statistical tests. How is this inequality resolved in a convergent mixed methods design? Sometimes
mixed methods researches will collect information from the same number of individuals on both the
qualitative and quantitative database. This means that the qualitative sample will be increased, and it
will limit the amount of data collected from any one individual. Another approach would be to weight
the qualitative cases so that they equal the N in the quantitative database. One other approach taken by
some mixed methods researchers is not to consider the unequal sample sizes a problem. They would
argue that the intent of qualitative and quantitative research differ (one to gain an in-depth perspective
and the other, to generalize to a population) and that each provides an adequate count. Another issue
in sampling is whether the individuals for the sample of qualitative participants should also be
individuals in the quantitative sample. Typically, mixed methods researchers would include the
sample of qualitative participants in the larger quantitative sample, because ultimately researchers
make a comparison between the two databases and the more they are similar, the better the
comparison.
Figure 10.1 Three Basic Mixed Methods Designs
Figure 10.2 Advanced Mixed Methods Designs
merge the data. We know from a description of this design that the two databases are analyzed
separately and then brought together. There are several ways to merge the two databases. The first
approach is called a side-by-side comparison. These comparisons can be seen in the discussion
sections of mixed methods studies. The researcher will first report the quantitative statistical results
and then discuss the qualitative findings (e.g., themes) that either confirm or disconfirm the statistical
results. Alternatively, the researcher might start with the qualitative findings and then compare them
to the quantitative results. Mixed methods writers call this a side-by-side approach because the
researcher makes the comparison within a discussion, presenting first one set of findings and then the
other. A good example of this can be seen in the Classen and colleagues’ (2007) study.
Researchers can also merge the two databases by changing qualitative codes or themes into
quantitative variables and then combining the two quantitative databases—a procedure called data
transformation. The researcher takes the qualitative themes or codes and counts them (and possibly
groups them) to form quantitative measures. Some useful procedures that mixed methods researchers
have used can be found in Onwuegbuzie and Leech (2006). A final procedure is to merge the two
forms of data in a table or a graph. This is called a joint display of data, and it can take many different
forms. It might be a table that arrays the themes on the horizontal axis and a categorical variable (e.g.,
different types of providers such as nurses, physician assistants, and doctors) on the vertical axis. It
might be a table with key questions or concepts on the vertical axis and then two columns on the
horizontal axis indicating qualitative responses and quantitative results to the concepts (Li, Marquart,
& Zercher, 2000). The basic idea is for the researcher to jointly display both forms of data—
effectively merging them—in a single visual.
Interpretation. The interpretation in the convergent approach is typically written into a discussion
section of the study. Whereas the results section report on the findings from the analysis of both the
quantitative and qualitative databases, the discussion section includes a report comparing the results
from the two databases and notes whether there is convergence or divergence between the two
sources of information. Typically the comparison does not yield a clean convergent or divergent
situation, and the differences exist on a few concepts, themes, or scales. When divergence occurs,
steps for follow-up exist. The researcher can state divergence as a limitation in the study without
further follow-up. This approach represents a weak solution. Alternatively, mixed methods
researchers can return to the analyses and further explore the databases, collect additional
information to resolve the differences, or discuss the results from one of the databases as possibly
limited (e.g., the constructs were not valid quantitatively or the qualitative themes did not match the
open-ended questions).
Validity. Validity using the convergent approach should be based on establishing both quantitative
validity (e.g., construct) and qualitative validity (e.g., triangulation) for each database. Is there a
special form of mixed methods validity that needs to be addressed? There are certainly some
potential threats to validity in using the convergent approach, and several of these have already been
mentioned. Unequal sample sizes may provide less of a picture on the qualitative side than the larger
N on the quantitative side. The use of different concepts or variables on both sides, quantitative and
qualitative, may yield incomparable and difficult to merge findings. A lack of follow-up on
conclusions when the scores and themes diverge also represents an invalid strategy of inquiry.
Explanatory Sequential Mixed Methods Design
Description of the design. The explanatory sequential mixed methods approach is a design in mixed
methods that appeals to individuals with a strong quantitative background or from fields relatively
new to qualitative approaches. It involves a two-phase project in which the researcher collects
quantitative data in the first phase, analyzes the results, and then uses the results to plan (or build on
to) the second, qualitative phase. The quantitative results typically inform the types of participants to
be purposefully selected for the qualitative phase and the types of questions that will be asked of the
participants. The overall intent of this design is to have the qualitative data help explain in more
detail the initial quantitative results. A typical procedure might involve collecting survey data in the
first phase, analyzing the data, and then following up with qualitative interviews to help explain the
survey responses.
Data collection. The data collection proceeds in two distinct phases with rigorous quantitative
sampling in the first phase and with purposeful sampling in the second, qualitative phase. One
challenge in this strategy is to plan adequately what quantitative results to follow up on and what
participants to gather qualitative data from in the second phase. The key idea is that the qualitative
data collection builds directly on the quantitative results. The quantitative results that then are built on
may be extreme or outlier cases, significant predictors, significant results relating variables,
insignificant results, or even demographics. For example, when using demographics, the researcher
could find in the initial quantitative phase that individuals in different socioeconomic levels respond
differently to the dependent variables. Thus, the follow-up qualitatively may group respondents to the
quantitative phase into different categories and conduct qualitative data collection with individuals
representing each of the categories. Another challenge is whether the qualitative sample should be
individuals that are in the initial quantitative sample. The answer to this question should be that they
are the same individuals, because the intent of the design is to follow up the quantitative results and
explore the results in more depth. The idea of explaining the mechanism—how the variables interact
—in more depth through the qualitative follow-up is a key strength of this design.
Data analysis. The quantitative and the qualitative databases are analyzed separately in this
approach. The quantitative results are then used to plan the qualitative follow-up. One important area
is that the quantitative results cannot only inform the sampling procedure but it can also point toward
the types of qualitative questions to ask participants in the second phase. These questions, like all
good qualitative research questions, are general and open-ended. Because analysis proceeds
independently for each phase, this design is useful for student research and perhaps easier to
accomplish (than the convergent design) because one database builds on the other and the data
collection can be spaced out over time.
Interpretation. The mixed methods researcher interprets the follow up results in a discussion section
of the study. This interpretation follows the form of first reporting the quantitative, first-phase results
and then the qualitative, second phase results. However, this design then employs a third form of
interpretation: how the qualitative findings help to explain the quantitative results. A common misstep
at this point by beginning researchers is to merge the two databases. While this approach may be
helpful, the intent of the design is to have the qualitative data help to provide more depth, more insight
into the quantitative results. Accordingly, in the interpretation section, after the researcher presents
the general quantitative and then qualitative results, a discussion should follow that specifies how the
qualitative results help to expand or explain the quantitative results. Because the qualitative database
questions narrows the scope of the quantitative questions, a direct comparison of the two databases
(as in the convergent design) means an inadequate comparison of variables or concepts.
Validity. As with all mixed methods studies, the researcher needs to establish the validity of the
scores from the quantitative measures and to discuss the validity of the qualitative findings. In the
explanatory sequential mixed methods approach, additional validity concerns arise. The accuracy of
the overall findings may be compromised because the researcher does not consider and weigh all of
the options for following up on the quantitative results. Attention may focus only on personal
demographics and overlook important explanations that need further understanding. The researcher
may also contribute to invalidate results by drawing on different samples for each phase of the study.
This minimizes the important of one phase building on the other. The sample size may also be
inadequate on either quantitative side of the study or the qualitative side. These are a few of the
challenges than need to be built into the planning process for a good explanatory sequential mixed
methods study.
Exploratory Sequential Mixed Methods Design
Description of the design. If we reverse the explanatory sequential approach and start with a
qualitative phase first followed by a quantitative phase, we have an exploratory sequential approach.
An exploratory sequential mixed methods is a design in which the researcher first begins by exploring
with qualitative data and analysis and then uses the findings in a second quantitative phase. Like the
explanatory sequential approach, the second database builds on the results of the initial database. The
intent of the strategy is to develop better measurements with specific samples of populations and to
see if data from a few individuals (in qualitative phase) can be generalized to a large sample of a
population (in quantitative phase). For example, the researcher would first collect focus group data,
analyze the results, develop an instrument based on the results, and then administer it to a sample of a
population. In this case, there may not be adequate instruments measuring the concepts with the
sample that the investigator wishes to study. In effect, the researcher employs a three-phase procedure
with the first phase as exploratory, the second as instrument development, and the third as
administering the instrument to a sample of a population.
Data collection. In this strategy, the data collection would occur in two phases with the initial
qualitative data collection followed by the second quantitative data collection. The challenge is how
to use the information from the initial phase in the second phase. Several options exist. The
qualitative data analysis can be used to develop an instrument with good psychometric properties
(i.e., validity, reliability). The qualitative data analysis will yield quotes, codes, and themes (see
Chapter 8). The development of an instrument can proceed by using the quotes to write items for an
instrument, the codes to develop variables that group the items, and themes that that group the codes
into scales. This is a useful procedure for moving from the qualitative data analysis to scale
development. Scale development also needs to follow good procedures for instrument design, and the
steps for this including such ideas as item discrimination, construct validity, and reliability estimates
(see DeVellis, 2012). Developing a good psychometric instrument that fits the sample and population
under study is not the only use of this design. A researcher can analyze the qualitative data to develop
new variables, to identify the types of scales that might exist in current instruments or to form
categories of information that will be explored further in a quantitative phase. The question arises if
the sample for the qualitative phase is the same for the quantitative phase. This cannot be, because the
qualitative sample is typically much smaller than a quantitative sample needed to generalize from a
sample to a population. Sometimes mixed methods researchers will use entirely different samples for
the qualitative and quantitative components of the study (unlike the explanatory sequential design).
However, a good procedure is to draw both samples from the same population but make sure that the
individuals for both samples are not the same. To have individuals help develop an instrument and
then to survey them in the quantitative phase would introduce confounding factors into the study.
Data analysis. In this strategy the researcher analyzes the two databases separately and uses the
findings from the initial exploratory database to build into quantitative measures. This means that the
researcher needs to pay careful attention to the qualitative data analysis steps and determine what
findings to build on. If, for example, the researcher uses grounded theory (see Chapter 9), the
theoretical model generated may provide a model to be tested in the second, quantitative phase. A
qualitative case study can yield different cases that become the focus on important variables in the
second quantitative phase.
Interpretation. Researchers interpret the mixed methods results in a discussion section of a study.
The order of interpretation is to first report the qualitative findings, the use of the qualitative results
(e.g., the development of an instrument, the development of new quantitative measures), and then the
quantitative results of the final phase of the study. It does not make sense to compare the two
databases, because they are typically drawn from different samples (as noted above in the data
collection discussion) and the intent of the strategy is to determine if the qualitative themes can be
generalized to a larger sample.
Validity. Researchers using this strategy need to check for the validity of the qualitative data as well
as the validity of the quantitative scores. Special validity concerns arise, however, in using this
design that need to be anticipated by the proposal developer. One concern is that the researcher may
not use appropriate steps to develop a good psychometric instrument. Developing a good instrument
is not easy, and adequate steps need to be put in place. Another concern is that a researcher may
develop an instrument or measures that do not take advantage of the richness of the qualitative
findings. This occurs when the qualitative data lacks rigor or occurs simply at the theme level without
the further data analysis steps associated with using one of the qualitative design-types, such as
ethnography, grounded theory, or case study procedures. Finally, as previously mentioned, the sample
in the qualitative phase should not be included in the quantitative phase as this will introduce undue
duplication of responses. It is best to have the qualitative participants provide information for scale,
instrument, or variable design but not to also be the individuals completing the follow-up instruments.