Quality in Qualitative Research 1

Hello! I want to welcome everyone to this presentation on Quality and Rigor in Qualitative
Research. I’m Dr. Elizabeth Pope, an Assistant Professor of Educational Research and I
specialize in qualitative research. I’m going to talk with you in this screencast about how to
assess quality in qualitative research.

Learning Outcomes

I have several goals for us in this presentation. I’ve listed them here as learning outcomes. By the
end of this presentation, you should:
- Make the paradigm shift necessary to understand quality and rigor in qualitative research.
- Know how quantitative approaches to validity and reliability do not apply to qualitative
research.
- Understand the different approaches to establishing quality in qualitative research.
- Recognize what methods are needed to have a rigorous qualitative study.

Assessing Quality in Qualitative Research

**As Lincoln & Guba (2011) explain, “Nowhere can the conversation about paradigm
differences be more fertile than in the extended controversy about validity” (p. 120). What this
means is that how researchers answer the question “What makes good qualitative research?”
varies across paradigm, methodology, and focus. In this presentation I will talk with you about
the primary viewpoints and ways of assessing quality and rigor. **A major difference you may
notice throughout this presentation is the use of different terms to assess quality. For example:
• **Lincoln and Guba explain in the 2011 handbook of qualitative research that the
paradigm you follow will determine the terminology you will use to assess the quality of
your research project.
o They discuss how the positivist and postpositivist paradigms use the term “rigor”
to assess external validity, reliability, and objectivity of their studies.
o Critical theorists will focus on historical situatedness, erosion of ignorance and
misapprehensions, and action stimulus to assess quality
o Constructivist paradigms will attempt to provide examples of trustworthiness and
authenticity
• **In their book Research Design: Qualitative, Quantitative, and Mixed Methods
Approaches, Creswell & Creswell explain quality for qualitative research using the terms
validity and reliability, two very quantitative terms.
• **Finally, Tracy’s 2010 article on validity develops 8 “Big-Tent” criteria that can be used
to assess quality of qualitative research regardless of which paradigm you follow.

Validity: An Extended Agenda

Let’s begin by discussing what validity means to Lincoln and Guba, two premier scholars in
qualitative research. **The use of the term “validity” is contentious in qualitative research. Many
researchers are moving away from the term because of the quantitative overtones. I myself do
not use it and require that my students work to establish the quality and rigor of their study,
rather than the validity of it. However, for the purposes of this portion of the presentation, we

Elizabeth M. Pope (2019) – for teaching purposes only; do not cite or redistribute
Quality in Qualitative Research 2

will review the writings from Lincoln & Guba, who discuss what “validity” is from a qualitative
stance.

**They write that validity can be an irritating concept to grasp for qualitative researchers, but it
cannot be dismissed because researchers must be able to answer the question of if their findings
are sufficiently authentic that they can trust themselves in acting upon the implications of their
findings.

2 Simultaneous Arguments

Lincoln and Guba explain that there are 2 simultaneous arguments on validity going on.

**Validity through research methods: The first, typically found in the positivist paradigm, argues
for rigor through the application of the research method. This means there are methodological
criteria to be followed during the construction and application of your research methods to help
establish the “validity” of your study. Translated out of this term, this argument tells us that there
are methodological criteria for quality and rigor. For example – following good formatting for
interviews, being serious about your prolonged time in the field and participant observations,
developing rich data, etc.

**Validity through interpretation: The second argument is the one that has been taken up for
more discussion in recent qualitative research. This form of establishing “validity” requires the
researcher to be able to answer the questions: “Are we interpretively rigorous? Can our co-
created constructions be trusted to provide some purchase on some important human
phenomenon? Do our findings point to action that can be taken on the part of research
participants to benefit themselves or their particular social contexts?” This argument for validity
is more commonly taken up in constructivist and critical lenses of theory. In translating this away
from the term “validity,” this means that the rigor of your study should be evident through your
interpretation of your findings. This is what leads to good quality research.

Shifting our Thinking from Validity to Trustworthiness

As I mentioned previously, validity is a very quantitative term with very quantitative criteria. In
qualitative research, these criteria don’t apply because qualitative researchers conduct
naturalistic inquiry (research within the natural world) and generally do not attempt to conduct
research from an objectivist standpoint. Instead, they approach research and data collection with
the understanding that in order to understand human experience and the natural world, we must
include human understandings and experiences within our data, and these are typically more
subjective than objective in nature. As explained by Erlandson et al. (1993), “The naturalistic
paradigm, valuing as it does the separate realities that have been created by individuals, must
also value the way these realities are responded to and the ways in which they enable individuals
to respond productively to their environments” (p. 132).

What this means is that in order to conduct and assess quality in qualitative research, a paradigm
shift is necessary. Validity and reliability are terms not typically used. New terminology

Elizabeth M. Pope (2019) – for teaching purposes only; do not cite or redistribute
Quality in Qualitative Research 3

represents this shift in understanding of how to establish that your research is of high quality and
rigorous (i.e. is it good?).

**In qualitative research, many researchers will attempt to establish the quality and rigor of their
study through “trustworthiness” rather than “validity.” The standards of trustworthiness are a
way that researchers can be responsive to the multiple realities within the naturalistic paradigm
and still establish the quality and rigor of their study (Erlandson et al., 1993, p. 132).
Trustworthiness is closely connected with rigor. The general idea is that the more rigorous your
study is the more trustworthy your findings are. So, what is trustworthiness?

**Trustworthiness is a term that was developed by Lincoln and Guba in 1985 to replace the
concept of validity in qualitative research.

**Trustworthiness is important because “it allows researchers to describe the virtues of

qualitative terms outside of the parameters that are typically applied in quantitative research”
(Given & Saumure, 2008, p. 895)
• To do this, Lincoln and Guba translated commonly used quantitative terms of validity
into more appropriate terms to be used for related concepts in qualitative trustworthiness.

Establishing Trustworthiness: A Translation of Terms

Erlandson et. al., explain that “establishing trustworthiness enables a naturalistic study to make a
reasonable claim to methodological soundness” (p. 131). The table you see here is from
Erlandson et al.’s 1993 book titled Doing Naturalistic Inquiry: A Guide to Methods. Setting
specific criteria of good research, the table takes the “conventional” or quantitative term and
translates the concept into appropriate concepts for qualitative researchers in confirming the
trustworthiness of their study. The final column includes techniques or methods qualitative
researchers use to establish these criteria.

**The term credibility is used for the criteria of the truth value of a study and replaces the
quantitative concept of internal validity. “A credible study is one where the researchers have
accurately and richly described the phenomenon in question” (Given & Saumure, p. 895). This
ensures that the data is accurately represented. Credibility is assessed by answering the question
of if the researcher’s presentation of the phenomenon under study is deemed correct by the
participants who experienced the phenomenon (i.e. the participants whom the researcher worked
with to generate their data).

**To establish the applicability of a study, transferability replaces generalizability or external

validity. Researchers focus on generating detailed data sets and describing them richly so that
other researchers can ascertain the findings’ ability to be applicable, or transferred, to other
research contexts. This determines the study’s worthiness not by the generalizability of its results
to broader contexts but by how well other researchers can transfer the findings to alternative
contexts. Going back to the writings of Erlandson et al., think about things this way, “every
context shifts over time as the persons in the context, their constructions of reality, and the
relationships among them also shift (even if the individuals are the same)… [thus] the
naturalistic researcher maintains that no true generalization is really possible; all observations are

Elizabeth M. Pope (2019) – for teaching purposes only; do not cite or redistribute
Quality in Qualitative Research 4

defined by the specific contexts in which they occur” (Erlandson et al., p. 32). However, this
doesn’t mean the knowledge one researcher obtains from a particular context isn’t relevant to
another, the authors continue:
Rather than attempting to select isolated variables that are equivalent across contexts, the
naturalistic researcher attempts to describe in great detail the interrelationships and
intricacies of the context being studied. Thus, the result of the study is a description that
will not be replicated anywhere. The “thick description” that has been generated,
however, enables observers of other contexts to make tentative judgments about
applicability of certain observations for their contexts and to form “working hypotheses”
to guide empirical inquiry in those contexts. (Erlandson et al., pp. 32-33)

**Dependability replaces reliability in confirming the consistency of a study. Being able to

reliably reproduce findings in the exact same context or using the exact same procedures is
exceedingly difficult for qualitative researchers. Through dependability “the researcher lays out
his or her procedure and research instruments in such a way that others can attempt to collect
data in similar conditions. The idea here is that if these similar conditions are applied, a similar
explanation for the phenomenon should be found” (Given & Samure, p. 895). So, if researchers
attempt to conduct a similar study in a similar context, dependability indicates that findings
would be repeated in the new study.

In quantitative research, reliability looks at the predictability, stability, or accuracy of findings

based on a researcher’s ability to replicate the study. However, as Erlandson et al. explain:
The naturalistic researcher believes that observed instability may be attributed not only to
error but also to reality shifts. Thus, the quest is not for invariance but for “trackable
variance” (Guba, 1981), variabilities that can be ascribed to particular sources (error,
reality shifts, better insights, etc.). (p. 34)

**Confirmability replaces objectivity in an attempt to establish the neutrality of a researcher.

Qualitative researchers do not attempt to declare the objectivity of their findings or observations
because they recognize that no methodology can be completely separated from the researcher.
So, researchers rely on the confirmability of the data within the study. Confirmability “reflects
the need to ensure that the interpretations and findings match the data. That is, no claims are
made that cannot be supported by the data” (Given & Saumure, p. 895).

Additional Versions of Quality (No Slide)

Establishing trustworthiness through the concepts of credibility, transferability, dependability,

and confirmability has become the most traditional form of quality and rigor in qualitative
research. However, there are other methods used that are based on differing paradigms within
qualitative research. These methods are equally valid and researchers may find them to be more
appropriate depending on the type of study they’re conducting. While I will not review them in
this presentation, you can read about authenticity and ethical relationships in the script. There are
additional views such as crystallization and transgression that researchers use as well.

Authenticity

Elizabeth M. Pope (2019) – for teaching purposes only; do not cite or redistribute
Quality in Qualitative Research 5

The first attempts to establish quality through authenticity of a study. In this process, researchers
typically are working within a constructivist paradigm. Researchers attempt to establish that the
conduct of research and the evaluation of the data are both “genuine and credible” with respect to
the participants, political implications, and the social implications of the research (James, 2008,
p. 45). There are 5 areas of quality as authenticity.
1. **Fairness attempts to establish balance between stakeholders and the participants whose
voices are included in the data and write-up of a study. “This is a deliberate attempt to
prevent marginalization, to act affirmatively with respect to inclusion, and to act with
energy to ensure that all voices in the inquiry effort had a chance to be represented in any
texts to have their stories treated fairly and with balance” (Lincoln & Guba, 2011, p. 122)
2. **Ontological and **educative authenticity focus on quality and rigor being constructed
through relationships with your participants and the people who your participants have
relationships with. The researcher in this case should raise these people’s awareness of
the study purpose, methods and findings so the researcher doesn’t leave them in the dark.
In this way they really are participants and not just subjects of research.
3. **Catalytic and **tactical authenticities assess quality by the actions taken by
participants because of the research. These actions include research participants acting for
social change “in the forms of emancipatory community action” (p. 122) or the researcher
training the participants in specific forms of political or social actions.

Ethical Relationships

Establishing quality through ethical relationships refers to the fact that there are connections
between how researchers know what they know and the relationships they have with their
participants. This means there’s a relationship between epistemology and ethics that researchers
need to attend to in order to establish validity. **There are seven standards for this (p. 123):
1. Positionality or standpoint judgments
2. Specific discourse communities and research sites as arbiters of quality
3. Voice, or the extent to which a text has the quality of polyvocality
4. Critical subjectivity
5. Reciprocity, or the extent to which the research relationship becomes reciprocal rather
than hierarchal
6. Sacredness, or the profound regard for how science can (and does) contribute to human
flourishing
7. And sharing of the perquisites of privilege that accrue to our positions as academics with
university positions

8 “Big Tent” Criteria

Finally, Sarah Tracy (2010) approaches quality from another perspective. She provides her
viewpoint on the benefits of criteria of quality specific to certain paradigms, theories, or
communities vs. general criteria. For instance, narrative studies generally have less than 10
participants. While this is good quality for narrative research, it may be an indication of poor
quality in other methodologies. Tracy’s hope is that a set of “universal” markers of quality will
discriminate between “qualitative ends [and] means” (p. 839). She explains that her goal in
presenting these universal markers of quality and that this “provides an expansive or ‘big tent’

Elizabeth M. Pope (2019) – for teaching purposes only; do not cite or redistribute
Quality in Qualitative Research 6

structure for qualitative quality while still celebrating the complex differences amongst various
paradigms” (p. 839).

Her criteria (p. 840) are:

1. **A worthy topic is a topic that is relevant, timely, significant and interesting
2. **A study follows rich rigor if it uses sufficient, abundant, appropriate, and complex
theoretical constructs, data and time in the field, sample(s), context(s), and data collection
and analysis processes
3. **Sincerity in a study is characterized by self-reflexivity about subjective values, biases,
and inclinations of the researcher as well as transparency about the methods and
challenges
4. **Credibility is a study is marked by thick description, concrete detail, explication of
tacit (nontextual) knowledge and showing rather than telling; triangulation or
crystallization; multivocality; and member reflections
5. **Resonance in a study means that the research must influence, affect, or move particular
readers or a variety of audiences through aesthetic, evocative representation; naturalistic
generalizations; and transferable findings.
6. **Studies that have significant contributions provide contributions to the literature
conceptually/theoretically, practically, morally, methodologically, or heuristically
7. **Researchers are ethical when they consider procedural ethics (such as human subjects
guidelines), situational and culturally specific ethics, relational ethics, and existing ethics
(such as leaving the scene and sharing the research)
8. **A study has meaningful coherence if it achieves what it purports to be about, uses
methods and procedures that fit its stated goals, and meaningfully interconnects literature,
research questions/foci, findings, and interpretations with each other.

Strategies to Establish Quality

As you may have noticed, there are several areas of overlap in establishing quality in qualitative
research. Here are some areas that are common to many, if not all, qualitative researchers in
distinguishing quality. These methods have different strengths and establish various aspects of
quality in research. If you develop a qualitative study, you will likely use many, if not all, of
these methods to confirm the quality and rigor of your research.

Resources

Establishing quality in qualitative research can be complex but it is essential to proving the
credibility and trustworthiness of your study.

Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed
methods approaches. Thousand Oaks, CA: SAGE.

Erlandson, D. A., Harris, E. L., Skipper, B. L., & Allen, S. D. (1993). Doing naturalistic inquiry:
A guide to methods. Newbury Park, CA: SAGE.

Elizabeth M. Pope (2019) – for teaching purposes only; do not cite or redistribute
Quality in Qualitative Research 7

Given, L. M., & Saumure, K. (2008). Trustworthiness. In L. M. Given (Ed.), The SAGE
encyclopedia of qualitative research methods (pp. 895-896). doi:
[Link]

James, N. (2008). Authenticity. In L. M. Given (Ed.), The SAGE encyclopedia of qualitative

research methods (p. 45). doi: [Link]

Lincoln, Y. S., & Lynham, S. A., & Guba, E. G. (2011). Paradigmatic controversies,
contradictions, and emerging confluences, revisited. In Y. S. Lincoln & E. G. Guba
(Eds.), The SAGE handbook of qualitative research (pp. 87-128).

Tracy, S. J. (2010). Qualitative quality: Eight “big-tent” criteria for excellent qualitative
research. Qualitative Inquiry 16(10), 837-851. doi: 10.1177/1077800410383121

Elizabeth M. Pope (2019) – for teaching purposes only; do not cite or redistribute