Synthesis objective Method Questions to guide analysis Anticipated outputs

We will categorize or ‘chart’ [47] the

· To identify ‘x’ articles that report a
synthesis methodology reported in
1: To characterize the synthesis 1. What is a general description of the knowledge synthesis method and of
each of the included studies using
methodologies knowledge synthesis method? these, ‘y’ articles used the subjective
specific questions to guide the
idealism approach in ‘z’ discipline.
analysis
· A taxonomy of knowledge synthesis
methods across multidisciplinary
2. What is the purpose of the knowledge fields
   
synthesis method? o Categorization of the synthesis
methods to reveal what research is
available within specific disciplines
[Link] is the epistemological approach of the
method? Is it subjective idealism (i.e., there is
no shared reality independent of multiple
     
alternative human constructions) or objective
idealism (i.e., there is a world of collectively
shared understandings)?
4. Which discipline is the knowledge synthesis
     
associated with (e.g., health, philosophy)?
5. What type of evidence can be synthesized by
the knowledge synthesis method – quantitative,
     
qualitative or mixed quantitative and
qualitative?
o Additional categories may be
identified iteratively through
6. How has the method been used to answer
    completion of the search and in
healthcare topics?
consultation with the team members
including the knowledge users
We will categorize articles that
specifically address the similarities
2: To identify the similarities and and differences between the
1. What are the similarities and differences · An in-depth comparison of the
differences between these knowledge synthesis methods by
among the knowledge synthesis methods? review methods in a table including:
methods comparing the synthesis methodology
reported in each of the included
studies
i. The specific features of the method
2. How does the method differ from ‘traditional’
    that make it more appropriate to
systematic review methods?
answer a question
    3. What is the minimum expertise required to i. The facilitators and barriers to using
implement the knowledge synthesis method? one synthesis method over another
 Synthesis objective Method Questions to guide analysis Anticipated outputs
(especially if more than one synthesis
Are particular skills required? Is a particular
method may be appropriate to answer
disciplinary background recommended?
the same research question)
4. What are the advantages and disadvantages of
     
each knowledge synthesis method?
5. How comprehensive is the knowledge
synthesis method? Can it be used for the entire
     
synthesis or only for a part of the synthesis (e.g.,
the analysis)?
6. How applicable is the method and how can it
     
be applied to healthcare interventions?
3: To map out a process for
conducting different synthesis We will categorize key articles that · An algorithm to guide synthesis
methods and to provide an explicitly explain the specific 1. What are the specific steps to conducting the methodology (informed by findings
approach for matching the methodology of the knowledge knowledge synthesis method? from objective 2 and consultation with
appropriate method to answer synthesis method. knowledge users)
research questions
· The mapping of specific steps to
2. Was the method empirically derived (i.e., conducting the review
    through experiment and observation) or · A bibliography of articles that
theoretically derived? describe how to conduct the different
knowledge synthesis methods
3. Are the steps operationalized (i.e., reported in
     
a reproducible manner)?
4. In what disciplinary fields and contexts are
    the steps operationalizable? Can they feasibly be  
applied to other contexts?
 Knowledge synthesis method Type of evidence Description
A method used in meta-analysis to offer flexibility in handling data from diverse study types
(i.e., the integration qualitative and quantitative forms of evidence). It allows qualitative
Bayesian meta-analysis evidence to contribute to meta-analysis by identifying variables to be included and providing
Mixed (qualitative
(Sutton, AJ, 2001; Roberts KA, evidence about effect sizes (qualitative evidence gets converted into quantitative form); and
and quantitative)
2002) [18, 19] helps to ensure that meta-analyses more properly reflect the diversity of evidence at primary
level – it recognizes the fact that evidence from multiple sources usually needs to be combined to
inform policy decisions.
A technique for categorising data and determining the frequencies of these categories. It differs
from more ‘qualitative’ methods in that it requires categorization to be sufficiently precise to
Content analysis
Qualitative allow multiple coders to achieve the same results, it relies on the systematic application of rules,
(Stemler S, 2001) [20]
and it tends to draw on the concepts of validity and reliability. Text is condensed into fewer
content-related categories.
Developed from meta-ethnography, it is an approach to the entire process of a review rather than
Critical interpretive synthesis just the synthesis component. It uses an iterative approach to refining the research question, the
Mixed
(Dixon-Woods, 2006) [21] searching and selection of articles from the literature, and defining and applying codes and
categories.
A form of meta-analysis, which allows the mixing of different quantitative research designs (e.g.
Cross-design synthesis randomized controlled trials and observational studies) and the pooling of evidence using
Mixed
(Droitcour J, 1993) [22] modeling to estimate a ‘true’ effect of a policy or programme, conditional on both the design of
the study and the characteristics of the relevant population
Uses the concept of triangulation, in which phenomena are studied from a variety of vantage
points. The method ‘unpicks’ the mutually interdependent relationships between behaviour,
Ecological triangulation
Qualitative persons, and environments, and requires ‘ecological sentences’ to be formulated during
(Banning, date unknown) [23]
synthesis: “With this intervention, these outcomes occur with these population foci and within
these ages with these genders… and these ethnicities in these settings”.
Offers a highly structured approach to organizing and analysing data (i.e., indexing using
numerical codes, rearranging data into charts, etc) to handle the large volume of information
Framework synthesis resulting from qualitative research. It’s distinct from other methods in that it utilises an ‘a priori’
(Pope, 2000; Brunton, 2006) [24, Qualitative framework informed by background material and team discussions to extract and synthesize
25] findings (i.e., a deductive approach). The ‘synthetic’ product may be expressed in the form of a
chart for each key dimension, which can be used to map the nature and range of the concept
under study.
A primary research approach used as a method for qualitative sampling, data collection and
Grounded theory analysis. It offers the ‘constant comparative method’ (the most widely used element of grounded
Qualitative
(Strauss & Corbin, 1998) [26] theory) to be used to identify patterns and iterations in primary data. It is an inductive approach
to analysis, allowing the theory to emerge from the data.
Interpretive Qualitative Noblit and Hare (1988) distinguish between the approaches of ‘interpretive’
 Knowledge synthesis method Type of evidence Description
and ‘integrative’ forms of synthesis which can be described as exploring the nature of the
Synthesis/Integrative synthesis
synthesis rather than its application. Interpretive synthesis combines evidence with an intent to
(Noblit and Hare (1988) [27]
develop new concepts and theories (interpretations).
A novel synthesis method aimed to uncover a new theory to explain the range of research
findings encountered. It is a way of re-analysing and comparing the texts of published studies
(rather than the original data of each) to produce a new interpretation. The approach involves
Meta-ethnography induction and interpretation in which separate parts are brought together to forma a “whole” (i.e.,
Qualitative
(Noblit G & Hare R., 1988) [27] looking for new theory or ‘line of argument’ to explain all the studies) so that the result is greater
than the sum of its parts. The product is the translation of studies into one another, which
encourages the researcher to understand and transfer ideas, concepts and metaphors across
different studies.
A method that follows an ideographic rather than pre-determined approach to the development of
Meta-interpretation the following components: exclusion criteria, a focus on meaning in context, interpretations as
Qualitative
(Weed, 2005) [28] raw data for synthesis, an iterative approach to the theoretical sampling of studies for synthesis,
and a transparent audit trail demonstrating the trustworthiness of the synthesis
A method developed from the need to synthesize evidence to inform complex policy-making
Meta-narrative questions, and involves looking across different paradigms/research traditions to uncover their
Qualitative
(Greenhalgh,2005) [29] ‘unfolding storyline” resulting in maps of ‘meta-narratives’ from which dimensions or themes
can be revealed and distilled for the synthesis phase of the review.
A multi-faceted, interpretive approach to synthesis developed to study the experiences of adults
Meta-study living with a chronic illness, and consists of 3 components to be done prior to synthesis: meta-
Qualitative
(Paterson BL, 2001) [30] data-analysis, meta-method, and meta-theory. Collectively, these create a new interpretation
accounting for the results of all three elements of analysis.
A quantitatively oriented summary of qualitative findings (as opposed to data being transformed)
developed to accommodate the distinctive features of qualitative surveys. The approach includes
Meta-summary the extraction, grouping, and formatting of findings, and the calculation of frequency and
Qualitative
(Sandelowski M, 2003) [31] intensity effect sizes, which can be used to produce mixed research syntheses and to conduct
‘posteriori’ analyses of the relationship between reports and findings. Meta-summaries can serve
as a basis for a further synthesis.
A method developed in response to concerns about the relevance and utility of qualitative
research, and involves combining separate elements to form a coherent whole using a process of
Meta-synthesis logical deduction. Its aims are to portray an accurate interpretation of a phenomenon and to
Qualitative
(Sandelowski M, 1997) [32] compare and contrast the constructs of individual studies to reach consensus on a new
construction of that phenomenon. It involves: identifying findings, grouping findings into
categories and grouping categories into synthesised findings.
Mixed studies review Mixed A literature review that simultaneously examines qualitative, quantitative, and mixed methods
(Pluye, 2005, Pluye 2009; primary studies to provide a greater understanding of a health issue than one type of research
 Knowledge synthesis method Type of evidence Description
Sandelowski M, 2003 book) [33–
approach alone (including the process of searching, analysis and study quality appraisal).
35]
An informal approach used to describe the selection, chronicling, and ordering of primary
evidence to produce an account of the evidence with commentary and interpretation. It can
Narrative review / Narrative
‘integrate’ qualitative and quantitative evidence through narrative juxtaposition (discussing
summary Mixed
diverse forms of evidence side by side). It is less concerned with assessing evidence quality and
(Dixon-Woods M, 2005) [36]
more focused on gathering relevant information that provides both context and substance to the
authors’ overall argument.
Similar to “Narrative review”, it involves an approach to evidence review but includes a formal
analytical process of synthesis to generate new insights or knowledge by seeking to be
Narrative synthesis systematic and transparent. It involves the ‘simple’ juxtaposition of findings from the studies
Qualitative
(Popay J, 2006) [37] included in the review and some element of integration or interpretation. There are 3 main
elements to the process: developing a preliminary synthesis of the findings of included studies;
exploring relationships in the data; and assessing the robustness of the synthesis product.
Qualitative cross-case analysis Case studies are used to understand complex social phenomena. Research using a case study
(Miles & Huberman. 1994; Yin Mixed approach may be based on a single or multiple cases, and can include a mixture of qualitative
R. 2003) [38, 39] and quantitative evidence.
Meta-synthesis attempts to integrate results from a number of different but inter-related
qualitative studies. The technique has an interpretive, rather than aggregating, intent, in contrast
Qualitative meta- synthesis
Qualitative to meta-analysis of quantitative studies. Qualitative meta- synthesis defined as theories, grand
(Jensen & Allen 1996) [40]
narratives, generalizations, or interpretive translations produced from the integration or
comparison of findings from qualitative studies.
Method for integrating or comparing findings from qualitative research. The method helps
Qualitative systematic review /
identify themes or constructs that lie in or across individual studies. The resulting accumulated
Qualitative evidence synthesis Qualitative
knowledge may lead to the development of a new theory, an overarching “narrative” a wider
(Grant 2009) [41]
generalization or “interpretative translation”.
A formal process for systematically coding data from a number of qualitative cases sufficient for
Quantitative case survey quantitative analysis. A set of structured questions is used to extract data from individual case
(Yin R and Heald K. 1975; Pelz Mixed studies, which are then treated as observations within a single dataset. Data are then converted to
D. 1981) [42, 43] quantitative form for statistical analysis. It is a way of turning qualitative studies into quantitative
data for analysis, allowing an integrated qualitative-quantitative synthesis to be undertaken.
Rooted in philosophy, this is a method used to investigate ‘what works for whom, under what
circumstances, and why’. Primary focus is on the causal mechanisms or “theories” that underlie
Realist review / synthesis
Mixed types of interventions or programmes and aims to build explanations across interventions or
(Pawson 2005) [10]
programmes which share similar underlying “theories of change” as to why they work (or not)
for particular groups in particular contexts.
Textual Narrative synthesis Mixed An approach that arranges studies into more homogeneous groups, and useful for synthesizing
 Knowledge synthesis method Type of evidence Description
different types of evidence (quantitative, qualitative, economic, etc). Study characteristics,
(Lucas, 2007) [44] context, quality and findings are reported according to a standard format, and similarities and
differences are compared across studies
The most common method adopted within ‘Narrative reviews” to produce a relatively
rudimentary synthesis of findings across the included studies. It involves identifying prominent
Thematic analysis
Mixed or recurring themes in the literature (largely shaped by research questions), and summarizing the
(Mays, 2005) [45]
findings of different studies under thematic headings using summary tables, which can inform a
description of key points.
Qualitative This approach combines and adapts approaches from both meta-ethnography and grounded
Thematic synthesis
theory. Free codes of findings are organized into ‘descriptive’ themes, which are then further
(Thomas, 2008) [46]
interpreted to yield ‘analytical’ themes (comparable to 3rd order interpretations from meta-

Product Audience Method

Publish in relevant journals; present at relevant academic meetings (e.g. Cochrane
Taxonomy for review Health services researchers,
Colloquium); provide the taxonomy online through the Knowledge Synthesis Network,
methods trainees
KT Canada, Cochrane Collaboration, CIHR.
 Product Audience Method
Algorithm for matching the
Health services researchers, Prepare summary document describing the algorithm that will be disseminated through
synthesis question to the
trainees, publishers, journal publication in relevant journal(s). Provide the algorithm online through the Knowledge
appropriate review
editors, funders Synthesis Network, KT Canada, Cochrane Collaboration, CIHR.
method(s)
Develop online methods manual outlining the different review methods to be available as
Health services researchers,
a series of articles, a set of powerpoint slides, and podcasts. We will also explore making
Methods Manual funders, publishers, and
these available as a book and have had preliminary discussions with Wiley Blackwell
policy makers, and trainees
about this topic. Create an online systematic review course.
Create an online systematic review course to provide instruction in the methods for
Review Course Researchers and trainees
completing less traditional knowledge synthesis.