Sandelowski 03 Classifying Qual Findings
Sandelowski 03 Classifying Qual Findings
[Link]
Published by:
[Link]
Additional services and information for Qualitative Health Research can be found at:
Subscriptions: [Link]
Reprints: [Link]
Permissions: [Link]
Citations: [Link]
What is This?
Margarete Sandelowski
Julie Barroso
AUTHORS’ NOTE: The study referred to in this article is titled “Analytic Techniques for Qualitative
Metasynthesis” and is supported by grant # NR04907 from the National Institute of Nursing Research,
National Institutes of Health. We thank the members of our expert panel, including Cheryl Tatano Beck,
Louise Jensen, Margaret Kearney, George Noblit, Gail Powell-Cope, and Sally Thorne. We also thank
Miriam Jicha for the design of Figure 1.
QUALITATIVE HEALTH RESEARCH, Vol. 13 No. 7, September 2003 905-923
DOI: 10.1177/1049732303253488
© 2003 Sage Publications
905
where a harder line is typically drawn between results and discussion (Sandelowski
& Barroso, 2002a, 2002b).
In our own ongoing efforts to analyze studies for a qualitative metasynthesis
method project, we found that no matter what the stated or implied frame of refer-
ence or method, there was a remarkable uniformity to the way the findings were
produced and presented. Virtually all the study findings were based exclusively on
individual and/or group interviews that were analytically treated in the same way,
that is, as authentic or true reports or indexes of facts and feelings (Sandelowski,
2001). Twenty-four of the 62 journal article reports we read were presented as
grounded theory, phenomenologic, ethnographic, narrative, and/or feminist
works. However, despite these method claims, the findings in 12 of these 24 reports
were produced from some form of relatively manifest content analysis. Thirty-eight
of these reports had no explicit methodological orientation at all, had few or no cita-
tions to method literature, and/or assumed a generic or noncategorical research
stance toward inquiry, their authors referring to their work as “qualitative,”
“descriptive/exploratory,” and/or as “focus group study.”
Accordingly, we found it more useful—for our efforts to extract the findings
from these reports for analysis and synthesis—to classify them by type of findings.
We describe this typology in this article. We anticipate that such a typology will not
only allow readers/reviewers to understand better what was likely actually done in
a study, as opposed to what investigators stated they did or hoped to do
(Sandelowski & Barroso, 2002b), but also provide a basis for selecting appropriate
strategies for the analysis and synthesis of findings in these reports. The typology
allows for the inclusion of studies in metasynthesis projects that might be rejected
because reviewers believe the method to have been described inaccurately. For
example, it allows reviewers to read study findings that, they believe, the authors
misrepresented as the result of a hermeneutic analysis instead as the result of a con-
tent analysis, which is what the reviewers judge it to be. An error that might inter-
fere with the validity of a qualitative metasynthesis is the exclusion of reports of
studies in which the findings do not fit the stated method, both because such a mis-
fit might not affect the credibility of the findings and because what constitutes
adherence to method is still a subject hotly debated by qualitative researchers
(Sandelowski & Barroso, 2002b). What constitutes a grounded theory to one scholar
might be nothing more than a content analysis to another. Although many qualita-
tive researchers will likely find this to be a troubling state of affairs, the rejection of a
study presented as grounded theory does not by itself justify the rejection of its find-
ings and their exclusion from a metasynthesis. Such findings can still be coherent
and well supported by the data. Indeed, we propose here that a claim to having used
a method that is not supported by the content and form of the findings ostensibly
produced by using that method does not by itself leave the findings without
support.
We have already introduced one of the categories in this typology in a previous
article describing the difficulties of finding the findings in reports of qualitative
studies (Sandelowski & Barroso, 2002a). In this article, we present the full typology
and our efforts to evaluate its stability, relevance, and utility. For the sake of clarity
and to avoid redundancy, we describe first the typology in its present and latest
form and then our efforts to develop and refine it.
METHOD
Our intention was to create a typology that will be analytically useful in discerning
the similarities and differences between sets of findings, no matter what the stated
methodological orientation. Because we so often found a discrepancy between the
stated and the actual method used in reports, or the ostensible use of methods to cre-
ate findings not generally entailed by the method (e.g., the stated use of phenomen-
ology to produce a content analysis of findings), we wanted to offer a tool with
which readers/reviewers might be better able to locate and compare findings. Find-
ings from completed studies are the raw data for any research integration project.
The typology we describe here emphasizes differences in kind between qualita-
tive findings, not differences in quality between qualitative studies. Although dif-
ferences in kind are not always easy to separate from differences in quality and can
even entail each other, we do not intend for this typology to be used as a tool for
evaluating the scholarly merits of a study, for determining whether it is “good”
enough to be included in a research integration study. A study might exemplify a
Closest Farthest
to data from data
Conceptual/
No finding Thematic survey Interpretive
Topical survey thematic
explanation
description
Borderline studies
of what they presented as the findings in their reports of the studies they conducted.
We see content and form as inseparable and inevitably entailing each other
(Sandelowski & Barroso, 2002b). A focus on the findings themselves reveals the
kind of analytic and interpretive work actually performed, no matter what the
research rhetoric. What we found as a result of this focus was that although method
might matter in some general sense, the method named in the reports we read often
mattered very little to the findings in those reports. Such factors as the presence,
amount, and absence of verbal text, metaphoric language, quotes, numbers (e.g.,
frequencies, means, percentages), tables, figures, and other textual and visual dis-
plays revealed more about the methodological orientation of a study than any state-
ments of method in the method sections of reports.
In classifying the findings in the studies we read, we sought to maintain a dis-
tinction between the level of abstraction of analysis and interpretation of data dis-
cernible in the findings from the level of abstraction of the phenomena, events, or
cases that constituted the subjects/objects of analysis and interpretation. For exam-
ple, findings about taking zidovudine (AZT) might seem more concrete and tangi-
ble than findings about spirituality simply because spirituality is a phenomenon
more difficult to articulate and less tangible than drug use. However, a set of find-
ings about AZT use might actually be more transformed than a set of findings about
spirituality that does nothing more than offer a list of respondents’ definitions of
spirituality as they were given to the researcher.
Our typology of findings was developed independently of the typology Kear-
ney (2001a) created but is similar to it; we note these similarities in the description
that follows. (Kearney is also a member of our expert panel, the activities of which
we describe later in this article.) However, our typology differs from hers in several
ways. In contrast to her typology, ours does not assume a “gold standard” (p. 149) in
qualitative research, nor does it address “discovery” (p. 146) as a way to classify
findings. We see “complexity” as residing primarily in the degree of transformation
of data, as opposed to the linking of findings in webs of interaction (p. 146). In addi-
tion, although her typology leans toward a view of a priori or, as we call them,
imported theoretical frameworks as restricting the evidentiary value of qualitative
research for practice, ours leans toward a view of such frameworks as enhancing the
evidentiary value of qualitative research. Finally, whereas Kearney’s typology is
intended primarily to be a guide to clinicians in the application of qualitative evi-
dence, ours is intended primarily to bypass the problem of discrepancies between
research claims and research behavior as discernible in a research report. We hope
thereby to salvage qualitative findings that might be lost to practice because they
were excluded from research integration studies only for misrepresenting the
method used. Emphasizing the form of findings in addition to their content, we also
anticipate that our typology will be useful in matching analysis and synthesis tech-
niques to the actual nature of findings. We will address this use of the typology in
future articles.
No-Finding Report
We begin our description of our typology of findings with reports of studies that we
concluded had no findings. We defined findings as the data-driven and integrated
discoveries, judgments, and/or pronouncements researchers offer about the phe-
Topical Survey
Located in the domain of research, but not in the domain of qualitative research,
were reports of qualitative studies offering a topical survey of findings. The defin-
ing feature of the topical surveys in our sample was the reduction of qualitative data
in ways that remained close to those data as given. This type of finding was charac-
terized by the emphasis on nominal or categorical data, or lists and inventories of
topics covered by research participants in interviews and focus groups. This kind of
work can be compared to the “table of contents” of a book Kearney (2001a, p. 147).
We classified 30 of the 62 journal articles in our bibliographic sample as topical sur-
veys. Seventeen of the investigators or groups of investigators of these studies vari-
ously referred to their work as focus group and/or descriptive/exploratory and/or
qualitative study. Five referred to their work as grounded theory, phenomenological/
hermeneutic, or ethnographic (this last reference appeared only in the title), and in
eight cases, the investigators did not cite any methodological orientation. The pri-
mary disciplinary affiliation in 21 of these reports is nursing. Nine of these studies
are in the fields of social work, public health, psychology, marriage and family ther-
apy, and spiritual care.
The topical surveys in our sample emphasized inventories, frequencies, and
percentages of women stating a topic, or enumerations of the topics themselves.
These topics were often introduced by investigators in their interview questions
and/or derived from a manifest content analysis. They were then briefly summa-
rized in words and/or tables. Indeed, many of the topical surveys in our sample
offered little more information about an experience or event in the lives of HIV-posi-
tive women in the verbal text than in the tables of numbers accompanying it. Like
topical anesthesia, the topical surveys we read remained on the surfaces of words,
and they treated individual and group interview data as uncomplicated reports of
facts and feelings (Sandelowski, 2001). Typically, data were derived from moder-
ately to highly forestructured, albeit open-ended, individual or focus group inter-
views in which the horizon of expectations was largely preset. The findings were, in
turn, generally restricted to the countable and codable remark and were usually
organized by the research or interview questions asked, by the prevalence of topics
raised, or by some other a priori but always surface classification system, such as
physical, psychological, social, and spiritual, or individual, family, and community.
Topical surveys appear to be distinguished from traditional quantitative sur-
veys by the absence of an effort or intention to obtain a statistically representative
sample and by the presence of some effort to qualify results, usually with a sprin-
kling of quotes. However, these quotes function like numbers, as they were used to
itemize topics rather than to edify readers about the experience to which the topic
referred. Indeed, the usual format of the topical survey is to name a topic (some-
times called a theme), briefly define it, count it, and then illustrate it with one or two
quotes or references to events. The intent of data collection is to gather information
about a range of responses and/or experiences, such as women’s responses to hav-
ing HIV/AIDS, or a range of responses to one or more specific experiences, such as
HIV-related symptoms or stigma. These studies are typically variable- as opposed
to case-oriented in that their effect was to identify variables of interest and their fre-
quency of occurrence across cases, not to determine the confluence of variables
within cases.
As shown in the shaded areas in Figure 1, topical surveys can border on no-find-
ing reports when the only interpretive move evident is the presentation of data by
some surface ordering system. They border on thematic surveys—the next item in
our typology—when there is more of an effort to qualify individual elements of
those data. Although topical surveys can be highly informative, especially in a new
domain of inquiry, we propose here that they do not constitute qualitative research,
as they are studies that merely use qualitative data obtained by what are commonly
(and, arguably, wrongly) viewed as data collection techniques signaling qualitative
research—namely, interviews and focus groups—and analyzed by techniques com-
monly (and also wrongly) signaling qualitative research, most notably content anal-
ysis. These studies are valuable, albeit noncategorical, descriptive works, as they
typically meet neither the probability sampling or psychometric requirements of
quantitative surveys nor the purposeful sampling or interpretive requirements of
qualitative research.
Thematic Survey
In contrast to the topical survey, the thematic survey is a type of finding that reflects
a greater degree of data transformation. Truer to the interpretive meaning of theme,
thematic surveys convey an underlying or more latent pattern or repetition dis-
cerned in the data: These reports revealed more of a discernible effort to move away
from merely listing topics (or subjects brought up by participants) toward describ-
ing themes (or the patterned responses researchers discerned from the topics
raised). Although there is still an apparent interest in cataloging or enumerating
findings, a greater emphasis is placed on qualifying them. In the dictionary, topic
and theme are synonymous. Moreover, in qualitative research, these terms are not
used in any consistent way (DeSantis & Ugarriza, 2000; Fredericks & Miller, 1997).
As depicted in Figure 1, although the topical survey merely contains qualitative
data, the thematic survey is a form of qualitative research, but it is a form at the low-
est level of abstraction in that it offers data more transformed than topical surveys
but less transformed than other kinds of qualitative findings (to be described
below). We classified 16 of the 62 journal articles in our bibliographic sample as the-
matic surveys. In 3 of these cases, the investigators themselves referred to their
work as exploratory/descriptive or qualitative. In 4 of these cases, they referred to
their work as grounded theory or feminist. In 2 cases, investigators named their
work as secondary analysis, and in 7 cases, there was no explicit reference to a
method. Five of these reports are in the field of nursing, and 11 are in the fields of
social work, public health, psychiatry, or sociology.
The thematic surveys we read tended to move farther from the data as given (in
the language used) and further into those data (in the analytic treatment) to grab
more of the subtleties of experience. Exploring a range of themes within and across
participants, the authors of these works offered more penetrating or nuanced
descriptions of experience, using either in vivo or everyday language (e.g., reasons
for having a baby), or themes or concepts from existing empirical or theoretical liter-
ature to label and/or organize their data (e.g., the concepts of “passing” and “cover-
ing” in the stigma literature). Although thematic surveys tended to offer more ver-
bal text than topical surveys did, the mere presence of more words as opposed to
more numbers was not what led us to classify a set of findings as a thematic as
opposed to a topical survey. Rather, what distinguished the one from the other for
us was the extent to which the authors of these works had transformed the data they
had collected or detailed the experiences the data represented.
A high-quality example of a thematic survey is the Siegel and Schrimshaw
(2001) study of the reasons and justifications a group of HIV-positive women gave
for having a baby. This report is organized by section headers that stay close to the
Conceptual/Thematic Description
Moving to the right in Figure 1 on the continuum of data transformation are find-
ings rendered in the form of one or more concepts or themes either developed in situ
from the data or imported from existing theories or literature outside the study. We
use the terms concept and theme here to differentiate between researchers’ theoretical
renderings of phenomena, events, or cases commonly associated with the social sci-
ences and/or grounded theory work and their narrative or phenomenological ren-
derings (van Manen, 1990) of them. Both concept and theme, as opposed to topic, con-
vey a latent, as opposed to a manifest, pattern in the data. Conceptual or thematic
descriptions move beyond surveying the topical or thematic landscape of events,
phenomena, or cases toward interpretively integrating portions of data. We classi-
fied 10 of the reports in our sample as conceptual/thematic descriptions. Seven of
these works collectively represent the single- or coauthored work of three investiga-
tors. In 7 of these cases, the investigators referred to their work as grounded theory.
In 2 of these cases, the investigators referred to their work as feminist narrative and,
in one case, as ethnographic. Seven of these reports are in nursing, one in social
work, one in marriage and the family, and one in anthropology.
In contrast to thematic surveys, in which authors often used concepts or themes
from existing empirical or theoretical literature largely to organize the presentation
of their findings, in conceptual/thematic descriptions, authors imported concepts
or themes to reframe a phenomenon, event, or case. We differentiate here between
the nominal use of concepts or themes, where they are used only to label and order
portions of data, and the interpretive use of concepts or themes, where concepts are
actually used conceptually or themes are actually used thematically to recast por-
tions of data. Such presentations have the effect not simply of serving as a reason-
able way to group data but of either extending the theoretical or other intellectual
tradition from which they were imported and/or illuminating an experience. Con-
ceptual/thematic descriptions border on thematic surveys when concepts or
themes serve more as a labeling or presentation device than as an interpretive one.
Examples of high-quality conceptual/thematic descriptions are the Ingram
and Hutchinson (1999, 2000) studies of mothering by HIV-positive women. In the
2000 report, they used the psychiatric concept of the “double bind” to describe the
“basic psychosocial problem” these women have by virtue of meeting the cultural
norm for women of having children while also—because they are HIV positive—
failing to meet the norm of being “good” women. In the 1999 report, they described
HIV-positive women’s resolution to this double-bind problem as “defensive moth-
ering.” Each of these reports of a grounded theory study offers one half of a
grounded theory of mothering in the context of HIV infection, the one report
conceptually rendering the problem and the second report conceptually rendering
the solution. Read together, these reports comprise an interpretive explanation—
the final category in our typology—of mothering in the context of HIV.2
Interpretive Explanation
At the extreme right of the continuum of findings in Figure 1 is the interpretive
explanation. The defining feature of such findings is the transformation of data to
produce grounded theories, ethnographies, or otherwise fully integrated explana-
tions of some phenomenon, event, or case. (We had no examples of a phenomenol-
ogy in this category in our sample, but such forms also belong in this category.) Such
explanations are composed of a science- or narrative-informed clarification or eluci-
dation of conceptual or thematic linkages that re-present the target phenomenon in
a new way. We classified three of the reports in our sample as interpretive explana-
tions. The authors of two of these reports referred to their works as feminist
grounded theory and as fieldwork. The third work was referred to as ethnographic
only in the abstract. The primary disciplinary affiliations of these authors are social
work, sociology, and anthropology, and the reports appeared in Qualitative Sociol-
ogy, Journal of Contemporary Ethnography, and Anthropology & Medicine.
In contrast to findings that survey topics and themes without linking them, or
that conceptually or thematically describe elements of experience without explain-
ing them, interpretive explanations offer a coherent model of some phenomenon, or
a single thesis or line of argument that addresses causality or essence. Moreover,
these explanations fully attend to relevant variations in both sample and data.
Because findings in this category emphasize explanation and variation, they are
most akin to findings Kearney (2001a) described as “dense explanatory descrip-
tion” (p. 149), with features of her “shared pathway” and “depiction of experiential
variation” categories (p. 148). These findings come to a point and take a specific
view in explaining an experience. Whereas a topical or thematic survey might list or
detail a set of concerns HIV-positive women expressed, and a conceptual/thematic
description reframe these concerns as rationalizations, an interpretive explanation
might show how each of these concerns was a condition for distinctively different
defensive strategies, only some of which succeeded in allaying those concerns.
An example of a high-quality report in this category is Stanley’s (1999) study
describing how middle- and upper-class White HIV-positive women manage the
stigma of HIV infection. Drawing from social science literature on stigma and iden-
tity, Stanley argued that these women were able to transform HIV/AIDS from a
stigmatized entity to a calling and themselves as HIV-positive women from bad to
good. She showed how these women mobilized spiritually informed rhetorical
strategies emphasizing redemption and self-transformation to manage the stigma
of HIV and to reconstruct an identity “spoiled” by HIV. Stanley used existing litera-
ture and extant theory not just to organize findings but also to explain them. Closely
related to Stanley’s study is the Grove, Kelly, and Liu (1997) study of another group
of middle-class HIV-positive women. These investigators used the concept of social
capital to show that these women’s social position allowed them to escape some of
the stigma of HIV, that is, to remain “nice girls.” However, they also showed how
this nice-girl image could act to prevent these women from being diagnosed with
HIV in a timely manner.
Percent
Rater (N = 6-7) Author A B C D E F Agreement
Study Number
1 4 5 4 4 5 4 X 67
5 4 4 4(3) 4 5 4 71
2 3 3 2 2 3 2 X 50 even split
3 3(2) 2 2 2 2(3) 2 71
3 4 2 3 3 2 2 X 50
3 3(2) 3(4) 2(3) 3 3(2) 3 86
4 2 2 2 2 2 2 X 100
2 2 2 2 2 2 2 100
5 3 3 4 3 4 2 X 50
3 2 4 3(2) 4 4(3) 5 43
6 1 3 3 4 4 3 X 50
1 3 3 3(2) 5 4(3) 3 57
7 2 3 2 2 1 2 X 67
2 3 2 2 2(3) 3(2) 3 57
8 5 3 4 4 5 5 X 50
5 2(3) 4 2(3) 5 5 4 43
9 2 4 3 3 3 2 X 50
2 3 4(3) 2 2 2 4(3) 57
10 4 4 5 4 5 4 X 67
4 1(3) 5 4 5 1(5) 5 43
11 4 5 4 4 4 5 X 67
4 5 4 4(3) 5(4) 5 4 57
12 4 5 5 3 4 1 X 33
4 5 5 3 3(4) 4(3) 5 43
13 2 3 3 3 3 2 X 67
2 3 4(3) 3 2(3) 3 3 57
14 2 4 3 3 3 2 X 50
2 4 3 2 2 3(2) 3 43 split
15 2 3 2 2 2 2 X 83
2 3 2 2 2 2 3 71
16 3 3 3 3 2 2 X 67
3 3 3 2 3(2) 3(2) 3 86
NOTE: For each study, the first line of data represents the first round of testing and the second line repre-
sents the second round. Instances of agreement are bolded. In the case of split votes, numerals are bolded
and italicized. X = no rating; 1 = no finding; 2 = topical survey; 3 = thematic survey; 4 = conceptual/the-
matic description; 5 = explanation.
The typology we just described is the latest outcome of an iterative process of devel-
opment, evaluation, and refinement. To evaluate this typology of findings, we
asked the six members of our expert panel to categorize 16 studies, or an approxi-
mately 20% stratified (by type of finding) random sample of the 62 articles and 4
other published works in our bibliographic sample. (We later decided to concen-
trate on only one medium, namely, the journal article. On random draw, none of the
4 nonarticle works was selected anyway.) We electronically mailed each member a
scanned copy of each study and a description of each category in the typology. We
asked them to read and then to rate each one: 1–no finding; 2–topical survey; 3–
studies could be explained by the fact that the results section contained only data
(making it look like a 1, or a no-finding report), whereas the conceptual model
developed from these data was shown in the discussion section (which would make
it a 4, or conceptual description).
In short, our hope was that our five types would be mutually exclusive enough
that readers would see overlap as occurring in only one direction, that is, we
worked to maximize the likelihood that a set of findings viewed as borderline
would be classified as bordering on the same two categories. We were also inter-
ested in ascertaining the extent to which each member would rate studies the same
way they had in round 1, taking into account the changes in names, definitions, and
instructions between rounds 1 and 2. We asked them not to refer back to their ratings
from round 1. In both rounds, we emphasized that we were trying to capture differ-
ences in kind between studies, not differences in quality, but asked members to
advise us if they could not suspend judgments of quality in categorizing the studies.
The results of the second round of testing with seven raters (again including the
two authors of this article as one rater) are shown in the second line in each of the 16
blocks shown in Table 1. As indicated there, the overall level of agreement was
about the same as in the first round (with one additional rater), at 62%. Study 4 again
elicited 100% agreement. If rating a study in the same conceptual space (that is,
either as or between the same two types) is defined as agreement, then an additional
nine studies (1, 2, 3, 7, 11, 13, 14, 15, 16) can be viewed as eliciting agreement among
at least six of the seven raters. Reports of single cases apparently evoked the most
disparate judgments as to what species of finding they represented. Study 10, a
report of a single case, elicited two ratings in nonadjacent categories. Another sin-
gle-case report, Study 6, elicited every possible rating.
tions and to “think/talk aloud” (Fonteyn, Kuipers, & Grobe, 1993) our reasoning to
make the often inchoate work of qualitative research more communicable and
transparent.
We reached consensus on ratings and reasons for rating relatively easily on all
but the two single-case studies. In these instances, we had conflicting views of what
differentiates data from findings, and findings from discussion, and of what consti-
tutes a nominal versus interpretive use of concepts. As a result of our discussion
with the members of our expert panel, we also discerned several factors that influ-
enced the ratings of studies. Several of our panel members stated explicitly that they
tended to rate findings to the left, or—as they put it—“lower,” on the typology con-
tinuum if the findings did not conform to the expectations they had for them, which
they derived from the method researchers stated they were using. For these mem-
bers, a disappointment factor operated to lower ratings if, for example, a study was
presented as a grounded theory study but the findings were conceptually “thin.”
These reviewers had difficulty not treating the typology as a quality evaluation tool
rather than as a means to differentiate types of findings. If there was something they
perceived as lacking in the presentation and execution of a stated method, they
judged the findings as less credible, which, in turn, caused them to move to the left
on the continuum. Closely related to the stated-versus-actual-method factor operat-
ing here was the effort-performance factor. One reviewer sometimes rated a study
“higher” on the typology if the researchers had discernibly tried to create, for exam-
ple, a grounded theory product, even though they might have failed to produce one.
Several panel members were also influenced by what they referred to as the
“level of the research questions or purpose.” If these were stated “quantitatively” as
opposed to “qualitatively”—for example, as in the question “What are the barriers
and facilitators to obtaining health services?” as opposed to the question “What is
the experience of living with HIV infection?”—then their ratings of findings would
lean toward topical or, at best, thematic survey. One member stated that if she felt a
study was on the border between two categories, she would then look back at the
stated research purpose, questions, and method to decide on its type. Yet another
factor influencing ratings was whether the findings were judged to be novel, with
studies perceived as offering nothing new rated “lower” on the typology.
In short, although the panel members found our typology useful, relevant, and
innovative in its intent to bypass judgments of quality, they sometimes found it dif-
ficult to suspend their judgments of quality in trying to type certain studies. In
another article, we have argued that a combination of personal preferences and aes-
thetic and communal judgments shape what readers will read into or expect from a
study report (Sandelowski & Barroso, 2002b) and will, therefore, affect how they
will evaluate that study. We saw in the preferences and judgments operating here
further confirmation of the varied “reader responses” (Beach, 1993), which resist
efforts to create a common and conventionally “reliable” set of criteria for apprais-
ing qualitative studies. Moreover, those scholars who consider themselves to be
qualitative researchers or to be doing qualitative research actually have little in
common except their dislike of “mainstream” research (e.g., Schwandt, 2000,
p. 190). The term qualitative research masks the profound differences in this method-
ological domain that, in turn, lead to highly idiosyncratic approaches to reading
and evaluating qualitative research.
CONCLUSIONS
We began this article with the question of what matters in the collection of data from
individual study reports in research integration studies, and concluded that stated
method did not seem to matter much in the majority of the journal articles we
reviewed in our metasynthesis project. We developed a typology of qualitative
findings that, we hope, can be used to bypass the stated-versus-actual method prob-
lem and thereby minimize the likelihood that a study containing information criti-
cal to health care will be excluded from consideration solely because of this prob-
lem. The mere fact that a study presented as a grounded theory study contains no
coherent conceptual rendering of data but, rather, a thematic survey of them does
not by itself undermine the credibility of these findings. Although the accurate rep-
resentation of the method used in a study is an indicator of methodological compe-
tence, it is not an indicator of the overall value of a study. Researchers who promise
to produce a grounded theory of an event (or, in our typology, an interpretive expla-
nation) but instead produce a survey of data organized by invented concepts (or a
thematic survey) might have failed to produce a certain kind of product but they
have not necessarily failed to produce a valuable product of a different kind.
The typology we described here arises from the view expressed in a previous
article that research reports constitute a literary technology mediating between
reader/reviewer and writer/researcher, and that readers/reviewers of qualitative
studies should develop “literary competence” (Culler, 1980).5 A competent reader
of qualitative research will understand the report as an after-the-fact reconstruction
of a study and, therefore, be able to read reports for what they represent about what
was likely done as opposed to what was claimed or intended. Our typology is pri-
marily in the service of achieving that competence, as having it is among the most
important factors influencing the utility of qualitative findings and what health care
practitioners will accept as valid knowledge for practice.
NOTES
1. Of the 99 reports, 42 are studies involving the same investigator, and/or one or more of the same
investigators in a research team, and/or the same sample of women. Included in this group of studies are
six individual investigators who had completed six dissertations and had subsequently published from
these dissertations 10 articles and one book, which was identical to the dissertation.
2. Ingram’s (1996) dissertation contains all the findings—that is, the entire grounded theory—in
these two reports and could easily be classified as interpretive explanation. Ingram’s work—in addition
to other works in our bibliographic sample—raises three issues. First, there is the challenge of determin-
ing how to cut findings for journal articles in ways that best represent the findings. Second, although the
findings in a single article might be classified one way, the findings in the entire work from which it was
drawn might be classified another way. Third, when findings from articles and dissertations overlap, a
decision must be made concerning which set of findings ought to be included in the bibliographic sam-
ples of research integration studies. We hope to address these issues in future articles.
3. One panel member was on sabbatical and therefore did not participate in this round of testing.
4. The same panel member referred to in Note 3 could not attend this meeting. This member did par-
ticipate in the second round of testing, as indicated in Table 1.
5. We thank Lioness Ayres for referring us to the citation on “literary competence.”
REFERENCES
Alvesson, M., & Skoldberg, K. (2000). Reflexive methodology: New vistas for qualitative research. London:
Sage.
Beach, R. (1993). A teacher’s introduction to reader-response theories. Urbana, IL: National Council of
Teachers of English.
Belgrave, L. L., & Smith, K. J. (1995). Negotiated validity in collaborative ethnography. Qualitative
Inquiry, 1, 69-86.
Clarke, M., & Oxman, A. D. (Eds.). (2001, October). Cochrane reviewers’ handbook 4.1.4. Cochrane
Library, 1. Retrieved February 13, 2002, from [Link]/cochrane/[Link]
Cooper, H. (1998). Synthesizing research: A guide for literature reviews (3rd ed.). Thousand Oaks, CA: Sage.
Culler, J. (1980). Literary competence. In J. Tompkins (Ed.), Reader-response criticism: From formalism to
post-structuralism (pp. 101-117). Baltimore: Johns Hopkins University Press.
DeMarco, R. F., Miller, K. H., Patsdaughter, C. A., Chisholm, M., & Grindel, C. G. (1998). From silencing
the self to action: Experiences of women living with HIV/AIDS. Health Care for Women International,
19, 539-552.
DeSantis, L., & Ugarriza, D. N. (2000). The concept of theme as used in qualitative nursing research. West-
ern Journal of Nursing Research, 22, 351-372.
Eisner, E. W. (1991). The enlightened eye: Qualitative inquiry and the enhancement of educational practice. New
York: Macmillan.
Fonteyn, M. E., Kuipers, B., & Grobe, S. J. (1993). A description of think-aloud method and protocol anal-
ysis. Qualitative Health Research, 3, 430-441.
Fredericks, M., & Miller, S. I. (1997). Some brief notes on the “unfinished business” of qualitative inquiry.
Quality & Quantity, 31, 1-13.
Geertz, C. (1973). Thick description: Toward an interpretive theory of culture. In C. Geertz, The interpreta-
tion of cultures: Selected essays (pp. 37-126). New York: Basic Books.
Grove, K. A., Kelly, D. P., & Liu, J. (1997). “But nice girls don’t get it”: Women, symbolic capital, and the
social construction of AIDS. Journal of Contemporary Ethnography, 26, 317-337.
Hak, T., & Bernts, T. (1996). Coder training: Theoretical training or practical socialization? Qualitative
Sociology, 19, 235-247.
Ingram, D. A. (1996). HIV-positive women: Double binds and defensive mothering. Unpublished doctoral dis-
sertation, University of Florida, Gainesville.
Ingram, D., & Hutchinson, S. A. (1999). Defensive mothering in HIV-positive mothers. Qualitative Health
Research, 9, 243-258.
Ingram, D., & Hutchinson, S. A. (2000). Double binds and the reproductive and mothering experiences of
HIV-positive women. Qualitative Health Research, 10, 117-132.
Jack, D. C. (1991). Silencing the self: Women and depression. Cambridge, MA: Harvard University Press.
Jensen, L. A., & Allen, M. N. (1994). A synthesis of qualitative research on wellness-illness. Qualitative
Health Research, 4, 349-369.
Kearney, M. H. (2001a). Levels and applications of qualitative research evidence. Research in Nursing &
Health, 24, 145-153.
Kearney, M. H. (2001b). New directions in grounded formal theory. In R. Schreiber & P. N. Stern (Eds.),
Using grounded theory in nursing (pp. 227-246). New York: Springer.
Maxwell, J. A. (1992). Understanding and validity in qualitative research. Harvard Educational Review, 62,
279-299.
Morse, J. M. (1997). “Perfectly healthy, but dead”: The myth of inter-rater reliability. Qualitative Health
Research, 7, 445-447.
Nijhof, G. (1997). “Response work”: Approaching answers to open interviews as readings. Qualitative
Inquiry, 3, 169-187.
Noblit, G. W., & Hare, R. D. (1988). Meta-ethnography: Synthesizing qualitative studies. Newbury Park, CA:
Sage.
Sandelowski, M. (2001). Reembodying qualitative inquiry. Qualitative Health Research, 12, 104-115.
Sandelowski, M., & Barroso, J. (2002a). Finding the findings in qualitative studies. Journal of Nursing
Scholarship, 34, 213-219.
Sandelowski, M., & Barroso, J. (2002b). Reading qualitative studies. International Journal of Qualitative
Methods, 1(1), Article 5. Retrieved January 31, 2002, from [Link]/~ijqm/
Schwandt, T. A. (2000). Three epistemological stances for qualitative inquiry: Interpretivism, hermeneu-
tics, and social constructionism. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research
(2nd ed., pp. 189-213). Thousand Oaks, CA: Sage.
Seale, C. F. (2002). Computer-assisted analysis of qualitative interview data. In J. F. Gubrium & J. A. Hol-
stein (Eds.), Handbook of interview research: Context & method (pp. 651-670). Thousand Oaks, CA:
Sage.
Siegel, K., & Schrimshaw, E. W. (2001). Reasons and justifications for considering pregnancy among
women living with HIV/AIDS. Psychology of Women Quarterly, 25, 112-123.
Stanley, L. D. (1999). Transforming AIDS: The moral management of stigmatized identity. Anthropology &
Medicine, 6, 103-120.
Thorne, S., Joachim, G., Paterson, B., Canam, C. (2002). Influence of the research frame on qualitatively
derived health science knowledge. International Journal of Qualitative Methods, 1(1), Article 1.
Retrieved January 31, 2002, from [Link]/~ijqm/
Thorne, S., Kirkham, S. R., & MacDonald-Emes, J. (1997). Interpretive description: Anoncategorical qual-
itative alternative for developing nursing knowledge. Research in Nursing & Health, 20, 169-177.
Thorne, S., Paterson, B., Acorn, S., Canam, C., Joachim, G., & Jillings, C. (2002). Chronic illness experi-
ence: Insights from a metastudy. Qualitative Health Research, 12, 437-452.
Valsiner, J. (2000). Data as representations: Contextualizing qualitative and quantitative research strate-
gies. Social Science Information, 39, 99-113.
van Manen, M. (1990). Researching lived experience: Human science for an action sensitive pedagogy. Albany:
State University of New York Press.
Wolcott, H. F. (1994). Transforming qualitative data: Description, analysis, and interpretation. Thousand Oaks,
CA: Sage.
Gramling, L., Boyle, J. S., McCain, N., Ferrell, J., Hodnicki, D., & Muller, R. (1996). Reconstructing a
woman’s experiences with AIDS. Family and Community Health, 19, 49-56.
Gray, J. J. (1999). The difficulties of women living with HIV infection. Journal of Psychosocial Nursing and
Mental Health Services, 37, 39-43.
Grove, K. A., Kelly, D. P., & Liu, J. (1997). “But nice girls don’t get it”: Women, symbolic capital, and the
social construction of AIDS. Journal of Contemporary Ethnography, 26, 317-337.
Guillory, J. A., Sowell, R., Moneyham, L., & Seals, B. (1997). An exploration of the meaning and use of
spirituality among women with HIV/AIDS. Alternative Therapies in Health and Medicine, 3, 55-60.
Hackl, K. L., Somlai, A. M., Kelly, J. A., & Kalichman, S. C. (1997). Women living with HIV/AIDS: The
dual challenge of being a patient and caregiver. Health and Social Work, 22, 53-62.
Hassin, J. (1994). Living a responsible life: The impact of AIDS on the social identity of intravenous drug
users. Social Science & Medicine, 39, 391-400.
Hutchison, M., & Kurth, A. (1991). “I need to know that I have a choice”: A study of women, HIV, and
reproductive decision-making. AIDS Patient Care, 5, 17-25.
Ingram, D., & Hutchinson, S. A. (1999a). Defensive mothering in HIV-positive mothers. Qualitative Health
Research, 9, 243-258.
Ingram, D., & Hutchinson, S. A. (1999b). HIV-positive mothers and stigma. Health Care for Women Interna-
tional, 20, 93-103.
Ingram, D., & Hutchinson, S. A. (2000). Double binds and the reproductive and mothering experiences of
HIV-positive women. Qualitative Health Research, 10, 117-132.
Kimberly, J. A., Serovich, J. M., & Greene, K. (1995). Disclosure of HIV-positive state: Five women’s sto-
ries. Family Relations, 44, 316-322.
Leenerts, M. H. (1998). Barriers to self-care in a cohort of low-income White women living with HIV/
AIDS. Journal of the Association of Nurses in AIDS Care, 9, 22-36.
Leenerts, M. H. (1999). The disconnected self: Consequences of abuse in a cohort of low-income White
women living with HIV/AIDS. Health Care for Women International, 20, 381-400.
Leenerts, M. H., Flaskerud, J. H., & Saunders, J. (1999). Lingering images: Symbols and meanings of self-
images in HIV-positive women with abuse histories. National Academies of Practice Forum: Issues in
Interdisciplinary Care, 1, 209-218.
Leenerts, M. H., & Magilvy, J. K. (2000). Investing in self-care: A midrange theory of self-care grounded in
the lived experience of low-income HIV-positive White women. Advances in Nursing Science, 22, 58-
75.
Marcenko, M. O., & Samost, L. (1999). Living with HIV/AIDS: The voices of HIV-positive mothers. Social
Work: Journal of the National Association of Social Workers, 44, 36-45.
Misener, T. R., & Sowell, R. L. (1998). HIV-infected women’s decisions to take antiretrovirals. Western
Journal of Nursing Research, 20, 431-447.
Moneyham, L., Seals, B., Demi, A., Sowell, R., Cohen, L., & Guillory, J. (1996a). Experiences of disclosure
in women infected with HIV. Health Care for Women International, 17, 209-221.
Moneyham, L., Seals, B., Demi, A., Sowell, R., Cohen, L., & Guillory, J. (1996b). Perceptions of stigma in
women infected with HIV. AIDS Patient Care & STDs, 10, 162-167.
Moser, K. M., Sowell, R. L., & Phillips, K. D. (2001). Issues of women dually diagnosed with HIV infection
and substance use problems in the Carolinas. Issues in Mental Health Nursing, 22, 23-49.
Murdaugh, C., Russell, R. B., & Sowell, R. (2000). Using focus groups to develop a culturally sensitive
videotape intervention for HIV-positive women. Journal of Advanced Nursing, 32, 1507-1513.
Napravnik, S., Royce, R., Walter, E., & Lim, W. (2000). HIV-1 infected women and prenatal care utiliza-
tion: Barriers and facilitators. AIDS Patient Care & STDs, 14, 411-420.
Raveis, V. H., Siegel, K., & Gorey, E. (1998). Factors associated with HIV-infected women’s delay in seek-
ing medical care. AIDS Care, 10, 549-562.
Regan-Kubinski, M. J., & Sharts Hopko, N. (1995). Illness cognition of HIV-infected mothers. Issues in
Mental Health Nursing, 16, 327-344.
Rose, M. A. (1993). Health concerns of women with HIV/AIDS. Journal of the Association of Nurses in AIDS
Care, 4, 39-45.
Russell, J. M., & Smith, K. V. (1999). A holistic life view of human immunodeficiency virus-infected Afri-
can American women. Journal of Holistic Nursing, 17, 331-345.
Seals, B. F., Sowell, R. L., Demi, A., Moneyham, L., Cohen, L., & Guillory, J. (1995). Falling through the
cracks: Social service concerns of women infected with HIV. Qualitative Health Research, 5, 496-515.
Semple, S. J., Patterson, T. L., Temoshok, L. R., McCutchan, J. A., Straits-Troster, K. A., Chandler, J. L., et al.
(1993). Identification of psychobiological stressors among HIV-positive women. Women and Health,
20, 15-36.
Serovich, J. M., Kimberly, J. A., & Greene, K. (1998). Perceived family member reaction to women’s disclo-
sure of HIV-positive information. Family Relations: Interdisciplinary Journal of Applied Family Studies,
47, 15-22.
Siegel, K., & Gorey, E. (1997). HIV-infected women: Barriers to AZT use. Social Science & Medicine, 45, 15-
22.
Siegel, K., Lekas, H. M., Schrimshaw, E. W., & Johnson, J. K. (2001). Factors associated with HIV-infected
women’s use or intention to use AZT during pregnancy. AIDS Education and Prevention, 13, 189-206.
Siegel, K., Raveis, V. H., & Gorey, E. (1998). Barriers and pathways to testing among HIV-infected women.
AIDS Education and Prevention, 10, 114-127.
Siegel, K., & Schrimshaw, E. W. (2000). Perceiving benefits in adversity: Stress-related growth in women
living with HIV/AIDS. Social Science & Medicine, 51, 1543-1554.
Siegel, K., & Schrimshaw, E. W. (2001). Reasons and justifications for considering pregnancy among
women living with HIV/AIDS. Psychology of Women Quarterly, 25, 112-123.
Smith, K. V., & Russell, J. (1997). Ethical issues experienced by HIV-infected African-American women.
Nursing Ethics, 4, 394-402.
Sowell, R. L., & Misener, T. R. (1997). Decisions to have a baby by HIV-infected women. Western Journal of
Nursing Research, 19, 56-70.
Sowell, R. L., Moneyham, L., Guillory, J., Seals, B., Cohen, L., & Demi, A. (1997). Self-care activities of
women infected with human immunodeficiency virus. Holistic Nursing Practice, 11, 18-26.
Sowell, R. L., Seals, B., Moneyham, L., Guillory, J., Demi, A., & Cohen, L. (1996). Barriers to health-seeking
behaviors for women infected with HIV. Nursing Connections, 9, 5-17.
Stanley, L. D. (1999). Transforming AIDS: The moral management of stigmatized identity. Anthropology &
Medicine, 6, 103-120.
Stevens, P. E. (1996). Struggles with symptoms: Women’s narratives of managing HIV illness. Journal of
Holistic Nursing, 14, 142-160.
Stevens, P. E., & Doerr, B. T. (1997). Trauma of discovery: Women’s narratives of being informed they are
HIV-infected. AIDS Care, 9, 523-538.
Stevens, P. E., & Richards, D. J. (1998). Narrative case analysis of HIV infection in a battered woman.
Health Care for Women International, 19, 9-22.
Tangenberg, K. (2000). Marginalized epistemologies: A feminist approach to understanding the experi-
ences of mothers with HIV. AFFILIA, 15, 31-48.
Tuchel, T. L., & Feldman, D. A. (1993). A preliminary ethnography of HIV-positive women in Dade
County jails. Practicing Anthropology, 15, 52-55.
Van Loon, R. A. (2000). Redefining motherhood: Adaptation to role change for women with AIDS. Fam-
ilies in Society: The Journal of Contemporary Human Services, 81, 152-161.
Van Servellen, G., Sarna, L., & Jablonski, K. J. (1998). Women with HIV: Living with symptoms. Western
Journal of Nursing Research, 20, 448-464.
Weitz, R. (1993). Powerlessness, invisibility, and the lives of women with HIV disease. Advances in Medi-
cal Sociology, 3, 101-121.
Wesley, Y., Smeltzer, S. C., Redeker, N., Walker, S., Palumbo, P., & Whipple, B. (2000). Reproductive deci-
sion making in mothers with HIV-1. Health Care for Women International, 21, 291-304.
Margarete Sandelowski, Ph.D., R.N., F.A.A.N., is Cary C. Boshamer professor of nursing at the Uni-
versity of North Carolina at Chapel Hill.
Julie Barroso, Ph.D., A.N.P., C.S., is an assistant professor of nursing at the University of North
Carolina at Chapel Hill.