Damschroder et al.

Implementation Science (2022) 17:7

[Link]

DEBATE Open Access

Conceptualizing outcomes for use

with the Consolidated Framework
for Implementation Research (CFIR): the CFIR
Outcomes Addendum
Laura J. Damschroder* , Caitlin M. Reardon, Marilla A. Opra Widerquist and Julie Lowery

Abstract
Background: The challenges of implementing evidence-based innovations (EBIs) are widely recognized among
practitioners and researchers. Context, broadly defined as everything outside the EBI, includes the dynamic and
diverse array of forces working for or against implementation efforts. The Consolidated Framework for Implementa-
tion Research (CFIR) is one of the most widely used frameworks to guide assessment of contextual determinants of
implementation. The original 2009 article invited critique in recognition for the need for the framework to evolve. As
implementation science has matured, gaps in the CFIR have been identified and updates are needed. Our team is
developing the CFIR 2.0 based on a literature review and follow-up survey with authors. We propose an Outcomes
Addendum to the CFIR to address recommendations from these sources to include outcomes in the framework.
Main text: We conducted a literature review and surveyed corresponding authors of included articles to identify
recommendations for the CFIR. There were recommendations to add both implementation and innovation outcomes
from these sources. Based on these recommendations, we make conceptual distinctions between (1) anticipated
implementation outcomes and actual implementation outcomes, (2) implementation outcomes and innovation
outcomes, and (3) CFIR-based implementation determinants and innovation determinants.
Conclusion: An Outcomes Addendum to the CFIR is proposed. Our goal is to offer clear conceptual distinctions
between types of outcomes for use with the CFIR, and perhaps other determinant implementation frameworks as
well. These distinctions can help bring clarity as researchers consider which outcomes are most appropriate to evalu-
ate in their research. We hope that sharing this in advance will generate feedback and debate about the merits of our
proposed addendum.
Keywords: Implementation science, Antecedent assessments, Implementation outcomes, Anticipated outcomes,
Actual outcomes, Innovation outcomes, Implementation framework, Evaluation methods, Theory, Consolidated
Framework for Implementation Research

*Correspondence: [Link]@[Link]
VA Center for Clinical Management Research, VA Ann Arbor Healthcare
System, 2215 Fuller Road, Ann Arbor, MI 48105, USA

© The Author(s) 2022. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which
permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the
original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or
other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line
to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory
regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this
licence, visit [Link] The Creative Commons Public Domain Dedication waiver ([Link]
mmons.​org/​publi​cdoma​in/​zero/1.​0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Damschroder et al. Implementation Science (2022) 17:7 Page 2 of 10

recommendations in both the literature review and sur-

Contributions to the literature
vey responses to add outcomes.
The CFIR Outcomes Addendum:
Although the CFIR is a determinant framework, users
must develop, explore, and test theories of change that link
• Conceptualizes types of outcomes for use with the
determinants to implementation outcomes [8]. For exam-
Consolidated Framework for Implementation Research
ple, Damschroder et al. identified seven CFIR determi-
(CFIR), one of the most widely used implementation
nants that were correlated with implementation outcomes
science frameworks.
using a mixed methods approach [9]; other regression- or
• Clarifies conceptual distinctions between (1) antici-
Boolean-based analyses can be used to identify subsets
pated implementation outcomes versus actual imple-
of determinants that drive implementation outcomes
mentation outcomes, (2) implementation outcomes
[11]. In our trainings and consultations with new CFIR
versus innovation outcomes, and (3) CFIR-based
users, we have found that additional clarification and
implementation determinants versus innovation deter-
guidance is needed about which outcomes CFIR deter-
minants.
minants influence and how to delineate determinants
• Guides researchers to choose and describe which types
versus outcomes during coding and analysis. Currently
of outcomes their studies are proposing to address,
published frameworks that define outcomes can be com-
carefully consider the determinants that can affect
plicated to apply. For example, the Reach, Effectiveness,
those outcomes, and in turn, design studies that collect
Adoption, Implementation, and Maintenance (RE-AIM)
the best data for assessing outcomes and their determi-
framework defines Maintenance outcomes at both the
nants.
setting- and individual-level. As a result, users must be
careful to delineate these levels because the determinants
influencing Maintenance are different depending on how
Background it is defined [12]. Furthermore, definitions are inconsist-
The challenges of implementing evidence-based innova- ent across sources. For example, the RE-AIM framework
tions (EBIs) are widely recognized among practitioners defines Adoption as “the absolute number, proportion,
and researchers. Context, broadly defined as everything and representativeness of: a) settings; and b) intervention
outside the EBI [1], includes the dynamic and diverse agents (people who deliver the program) who are willing
array of forces working for or against implementation to initiate a program” [12]. Proctor et al.’s Implementation
efforts [2]. As a result, implementation scientists have Outcomes Framework (IOF) defines Adoption as “the
prioritized developing methods to understand and meas- intention, initial decision, or action to try or employ an
ure facets of context, which is necessary for all projects innovation” [13]. CFIR users will benefit from more clarity
that involve planning, executing, or evaluating imple- about (1) types of implementation outcomes, (2) imple-
mentation efforts [3]. mentation vs. innovation outcomes, and (3) determinants
Theories that guide conceptualization of context of implementation outcomes versus determinants of
abound and are often encapsulated within determinant innovation outcomes.
frameworks [4, 5]; these frameworks delineate deter- We propose an Outcomes Addendum to the CFIR
minants (i.e., barriers or facilitators) that influence the to address these issues. Our goal is not to create a new
outcome of implementation efforts. Knowledge of con- framework, but to help implementation researchers
textual barriers and facilitators is used to adapt EBIs articulate which outcomes their studies are proposing
[6], select and tailor implementation strategies [3, 7], to address, carefully consider the determinants that can
and predict and/or explain implementation outcomes affect those outcomes, and in turn, design studies that
[8, 9]. Ultimately, the goal of this work is to increase can collect the best data for measuring both outcomes
knowledge about what works where and why to acceler- and their determinants. The aim of this debate article is
ate sustained integration of EBIs into routine practice. to describe the rationale for and conceptualization of the
The Consolidated Framework for Implementation CFIR Outcomes Addendum, which draws on findings
Research (CFIR) is one of the most widely used frame- from the literature review and survey we conducted as
works within and outside implementation science [8, 10]. part of our work on the CFIR 2.0 as well as the RE-AIM
The original 2009 article invited critique in recognition framework and the IOF [12, 13].
of the need for the framework to evolve [2]. As imple-
mentation science has matured, gaps in the CFIR have Methods
been identified and updates are needed. Our team is We completed a literature review to identify recommen-
developing the CFIR 2.0 based on a literature review and dations from the published literature. More details will
follow-up survey with authors. We encountered many be provided in the future CFIR 2.0 manuscript. Briefly,
Damschroder et al. Implementation Science (2022) 17:7 Page 3 of 10

we searched SCOPUS and Web of Science from 2009 (the Implementation outcomes
year the CFIR was published) to July 6, 2020; we included The CFIR Outcomes Addendum broadly conceptualizes
all articles that mentioned the CFIR in the title and/ implementation outcomes as the success or failure of
or abstract. We identified 376 articles total; 16 articles implementation. Anticipated implementation outcomes
included recommendations related to adding outcomes. are based on perceptions or measures of the likelihood
In addition to completing the literature review, we sur- of future implementation success or failure, i.e., imple-
veyed corresponding authors of included articles; there mentation outcomes that have not yet occurred. These
were 337 unique corresponding authors, but only 334 outcomes are forward-looking; constellations of CFIR
with contact information. Of the 334 contacted authors, determinants across domains predict these outcomes.
157 (47%) responded. The survey asked for recommen- The concept of anticipated outcomes is well-established
dations to improve the CFIR including adding, remov- within the Sociology of Science and Technology field.
ing, or modifying constructs and/or domains. Thirteen Borup et al. assert that “[…] expectations can be seen to
respondents recommended adding outcomes and three be fundamentally ‘generative,’ they guide activities, pro-
additional recommendations were related to outcomes, vide structure and legitimation, attract interest and fos-
though they were not explicitly identified as such. The VA ter investment. They give definition to role, clarify duties,
Ann Arbor Healthcare System IRB declared this study offer some shape of what to expect and how to prepare
exempt from the requirements of 38 CFR 16 based on for opportunities and risks” [17]. The terms anticipated
category 2. and expected are used interchangeably and we chose the
former term.
Proposed CFIR Outcomes Addendum Actual implementation outcomes are based on percep-
Overview tions or measures of current (or past) implementation
There were recommendations to add both implemen- success or failure, i.e., implementation outcomes that
tation and innovation outcomes to the CFIR from the have occurred. These outcomes are backward-looking;
literature review and survey. Hung et al. recommended constellations of CFIR determinants across domains
inclusion of both types of outcomes because it would explain these outcomes. Both anticipated and actual
focus “the researcher’s attention squarely on the way implementation outcomes can be assessed quantitatively
that context shapes intermediate results and condi- or qualitatively.
tions, such as user acceptance, which in turn influence Anticipated and Actual Implementation Outcomes
classic measures of an intervention’s ultimate aims or include three broadly conceptualized outcomes based
outcomes” [14]. Some authors addressed this gap by on our own work and the RE-AIM framework. While
linking the CFIR with another framework: nineteen implementation research has tended to focus on initial
used the RE-AIM framework and eight used the IOF implementation success, the importance of shifting from
[12, 13]. Other authors addressed this issue by adapt- near-term implementation goals to long-term sustain-
ing the CFIR to incorporate outcomes from both the ment is increasingly clear. Several recommendations
RE-AIM framework and the IOF, including the CFIR- from both our literature review and survey responses
Process Redesign [14, 15] and the Care Transitions highlighted the importance of capturing concepts of
Framework [16]. implementability and implementation, while even more
The RE-AIM framework and the IOF were used to help discussed the importance of assessing Sustainability
inform broad categories of implementation outcomes and Sustainment [14, 15, 18–26]. One survey respond-
and innovation outcomes included in the CFIR Out- ent explained, “We added sustainability [sustainment] to
comes Addendum. In addition, within implementation the framework in our study. Planning for sustainability
outcomes, we draw a distinction between anticipated [sustainment] should begin at the earliest stages of the
implementation outcomes and actual implementation implementation process.” In a critique by Ilot et al. they
outcomes. Finally, we highlight contextual determinants recognized that when EBIs are not sustained, the result
(as described by CFIR constructs) as potential mod- is a “waste of time, financial resources and leadership
erators of implementation outcomes versus innovation effort at a time of economic austerity” [18]. As a result,
determinants (outside the scope of the CFIR) as potential anticipated outcomes include adoptability, implementa-
moderators of innovation outcomes. Our goal is not to bility, and sustainability, while actual outcomes include
develop a new framework, but rather to clarify relation- adoption, implementation, and sustainment. These major
ships between determinants and outcomes and to pro- categories of implementation outcomes focus on the ulti-
vide broad definitions that users can apply while using mate goals of implementation efforts: first, whether the
other frameworks. decision is made to deliver the innovation (adoption);
Damschroder et al. Implementation Science (2022) 17:7 Page 4 of 10

Table 1 Implementation outcomes definitions

Anticipated outcomes Actual outcomes

Representativeness Adoptability: The likelihood key decision-makers will decide Adoption: The extent key decision-makers decide to put the
to put the innovation in place/innovation deliverers will innovation in place/innovation deliverers decide to deliver the
decide to deliver the innovation. innovation.
Implementability: The likelihood the innovation will be put Implementation: The extent the innovation is in place or being
in place or delivered. delivered.
Sustainability: The likelihood the innovation will be put in Sustainment: The extent the innovation is in place or being
place or delivered over the long-term. delivered over the long-term.

second, whether delivery of the innovation occurs comprising multiple determinants. Since publication
(implementation); and third, whether the delivery of the of the CFIR, there has been continued conceptual and
innovation continues in the long-term. Table 1 lists defi- measurement development for these concepts as poten-
nitions for each implementation outcome. tial predictors of implementation outcomes, but there
As a result, although the IOF lists acceptability, appro- is little consensus on their role within implementation
priateness, and feasibility as implementation outcomes, theories [29–31]. Thus, we also place implementation
these are not included as implementation outcomes in readiness and implementation climate into the ante-
the CFIR Outcomes Addendum. These measures can be cedent assessment category, which lies between CFIR
used to predict any anticipated or actual implementation determinants and implementation outcomes in Fig. 1.
outcome; for example, Weiner et al. developed meas- See Table 2 for a full mapping of RE-AIM Framework
ures for acceptability, appropriateness, and feasibility and IOF outcomes to the CFIR Outcomes Addendum.
of an innovation and highlighted their role as potential Given our goal to provide broad conceptualization of
predictors of adoption or implementation [27]. Thus, outcomes, many of the specific outcomes in existing
like Reilly et al., we classify these measures as “Ante- frameworks map to broader concepts in the CFIR Out-
cedent Assessments” [28]. Additionally, the CFIR lists comes Addendum.
implementation climate and implementation readiness A note on terminology: the terms Sustainability and
as higher-order constructs within the framework—each Sustainment are commonly used colloquially and there is

Fig. 1 CFIR Outcomes Addendum diagram

Table 2 RE-AIM Framework and IOF outcomes mapped to the CFIR Outcomes Addendum
RE-AIM Framework [12] (except where noted) Implementation Outcomes Framework [13] CFIR Outcomes Addendum

N/A: Not explicitly included Acceptability: The extent to which an innovation is perceived as Antecedent assessments [28]
“agreeable, palatable, or satisfactory.”
Appropriateness: The “perceived fit, relevance, or compatibility
of the innovation […] for a given practice setting, provider, or
consumer; and/or perceived fit of the innovation to address a
particular issue or problem.”
Feasibility: The extent to which an innovation “can be success-
fully used or carried out within a given agency or setting.”
Adoption: “The absolute number, proportion, and representa- Adoption: “The intention, initial decision, or action to try or Actual Implementation Outcomes: Adoption
tiveness of: a) settings; and b) intervention agents (people who employ an innovation.”
Damschroder et al. Implementation Science

deliver the program) who are willing to initiate a program.”

a b
Implementation (setting-level): “The intervention agents’ Penetration (setting-level): “The integration of a practice Actual Implementation Outcomes: Implementation
fidelity to the various elements of an intervention’s protocol, within a service setting and its subsystems” which “can be calcu-
including consistency of delivery as intended and the time lated in terms of the number of providers who deliver a given
required. Also includes adaptations made and the costs of imple- service or treatment, divided by the total number of providers
mentation. trained in or expected to deliver the service.”
(2022) 17:7

Fidelity: “The extent to which the program is implemented Fidelity: “The degree to which an intervention was imple-
consistently across different settings, staff, and patients.” mented as it was prescribed in the original protocol or as it was
intended by the program developer.”
Cost: Costs of “replicating a program or policy in different Cost: “The cost impact of an implementation effort,” based on N/A: Depending on how cost is defined, it may represent an imple-
settings;” “costs at the patient-, staff-, clinic-, and organizational “the costs of the particular intervention, the implementation mentation or innovation determinant or outcome.
levels;” “costs to deliver programs.” strategy used, and the location of service delivery.”
Adaptation: Adaptations made “prior to, during, and after N/A: Not explicitly included N/A: Adaption is conceptualized as an implementation determinant
program implementation.” in the CFIR Process Domain.
a c
Maintenance (setting-level): The extent to which “a program Sustainability: “The extent to which a newly implemented Actual Implementation Outcomes: Sustainment
or policy becomes institutionalized or part of the routine treatment is maintained or institutionalized within a service set-
organizational practices and policies. Includes proportion and ting’s ongoing, stable operations.”
representativeness of settings that continue the intervention
and reasons for maintenance, discontinuance or adaptation.”
Setting Impact (setting-level): Adoption X Implementation N/A: Not explicitly included N/A: Not explicitly included
[28, 32]
Page 5 of 10
Table 2 (continued)
RE-AIM Framework [12] (except where noted) Implementation Outcomes Framework [13] CFIR Outcomes Addendum
Damschroder et al. Implementation Science

Reach (recipient-level): “The absolute number, proportion, and bPenetration (recipient-level): “The number of eligible persons Innovation Outcomes: Innovation Impact on Recipients, Deliver-
representativeness of individuals who are willing to participate in who use a service, divided by the total number of persons ers, and Key Decision-Makers
a given initiative, intervention, or program.” eligible for the services.”
a
Implementation (recipient-level) “Clients’ use of the interven-
tion and implementation strategies.”
(2022) 17:7

e
Effectiveness: “The impact of an intervention on important Client Outcomes: “Satisfaction, Function, and Symptomology.”
outcomes, including potential negative effects, quality of life,
and economic outcomes.”
a
Maintenance (recipient-level): The extent to which “behavior
is sustained 6 months or more after treatment or intervention.”
d
Recipient Impact (recipient-level): Reach X Effectiveness [28, N/A: Not explicitly included
32]
e
N/A: Not explicitly included Service Outcomes: “The extent to which services are safe,
effective, patient-centered, timely, efficient, and equitable.”
a
Implementation and Maintenance: The RE-AIM framework includes definitions for Implementation and Maintenance at (1) the setting-level, which map to our Implementation Outcomes and at (2) the innovation
recipient-level, which map to our Innovation Outcomes
b
Penetration: The IOF provides a definition for Penetration at (1) the deliverer-level, which maps to our Implementation Outcomes and at (2) the recipient-level, which maps to our Innovation Outcomes
c
Sustainability: Though the IOF uses the word Sustainability, the definition of this outcome maps to Sustainment in the CFIR Outcomes Addendum
d
The CFIR Outcomes Addendum conceptualizes Recipient Impact for all constituents, which can be measured via Reach × Effectiveness [32] and Reach × Maintenance together [32–34]
e
The CFIR Outcomes Addendum conceptualizes Client and Service Outcomes as potentially relevant to all constituents, e.g., patient-centeredness may be a priority for innovation deliverers and recipients, satisfaction
may be a priority for key decision-makers and deliverers
Page 6 of 10
Damschroder et al. Implementation Science (2022) 17:7 Page 7 of 10

an entire “science of sustainability” that has much to offer key decision-makers, but also innovation deliverers, e.g.,
to the “science of implementation” [17]. As a result, we reducing burnout, improving work experience.
chose these terms over Maintenance from the RE-AIM Sustainment of outcomes may be strengthened when
Framework. goals are aligned between these three key constituen-
cies, each of whom are likely to have different priori-
Innovation outcomes ties and interests [37–40]. For example, an innovation
The CFIR Outcomes Addendum broadly conceptualizes that improves patient function (an important outcome
innovation outcomes as the success or failure of the inno- to patient recipients) is unlikely to be sustained if it
vation, based on the impact of the innovation on three increases burnout for clinicians (an important out-
important constituents: innovation recipients, innova- come to clinician deliverers) and/or increases system
tion deliverers, and key decision-makers. costs (an important outcome to key decision-makers).
Because CFIR users are focused on achieving and sus-
• Recipients are the human-beings for whom the inno- taining implementation, it is important to consider
vation is designed to benefit, e.g., patients receiving which outcomes are most important to which peo-
treatment, students receiving a learning activity, or ple. We believe that by highlighting the human-beings
citizens receiving a city service. impacted by Innovation Outcomes, the CFIR Out-
• Deliverers are the human-beings who are directly comes Addendum will help researchers and organiza-
or indirectly involved with delivering the innovation tions orient to values of humanism and equity. Figure 1
to recipients, e.g., clinicians delivering treatment to illustrates the components of the CFIR Outcomes
patients, teachers delivering a learning activity to stu- Addendum.
dents, or city employees delivering a city service to In Fig. 1, right facing arrows at the top of the fig-
citizens. ure illustrate the temporal nature of (1) anticipated and
• Key decision-makers are the human-beings who have actual implementation outcomes and (2) implementa-
authority within the implementing setting, whether it tion outcomes and innovation outcomes. The right facing
is a formal system or broader community, e.g., a hos- arrow between anticipated and actual implementation
pital director deciding what treatment to deliver, a outcomes illustrates the generative nature of anticipated
school superintendent deciding what learning activ- outcomes (see the “Implementation outcomes” section
ity to deliver, or a city mayor deciding what city ser- above). The right facing arrow between implementation
vice to deliver. and innovation outcomes illustrates the foundational
premise within implementation science that successful
It is important to note that types of recipients and types implementation is a necessary pre-condition to achiev-
of deliverers may overlap, e.g., when implementing a vac- ing maximum innovation benefits [41]. For example, an
cination program for hospital employees, all employees effective innovation will fail to produce expected out-
are potential recipients while the specific employees deliv- comes if it is poorly implemented; this may result in
ering the vaccine (e.g., nurses who work within Employee a “Type III” error, when evaluators conclude that the
Health) are also deliverers. These broad constituencies are innovation is ineffective, when in fact that same inno-
based on feedback from CFIR users, who use the CFIR to vation may have met or exceeded expectations if it had
plan and evaluate implementation of diverse innovations, been properly implemented [42]. In addition, left facing
both within and outside of healthcare. arrows across the bottom of the figure illustrate the rein-
While the outcomes important to innovation recipients forcing loop that can emerge when the positive impact of
(e.g., patients) and key decision-makers (e.g., hospital an innovation inspires continued commitment to imple-
directors) are frequently prioritized in other frameworks mentation and sustainment [43].
(e.g., as reflected by the list of Client and Service Out- A note on terminology: We have opted to use the term
comes within the IOF), outcomes important to innova- Innovation to be broadly inclusive of other terms. Rog-
tion deliverers (e.g., clinicians) are often not prioritized. ers’ classic Diffusion of Innovation theory defines inno-
Consideration of clinicians (and other employees) moti- vation as an idea, practice, or object that is perceived as
vated evolution of the “Triple Aim” (enhancing patient new by an individual or other unit of adoption. If an idea
experience, improving population health, and reducing seems new within a setting or for an individual, it is an
costs) [35] to the “Quadruple Aim,” which added an aim innovation [44]. This is a broad definition and includes
of improving the work-life and well-being of clinicians any “thing” that is being implemented [45]: Innovations
and staff [36]. Ideally, implementation of innovations will can include, e.g., medications, medical devices, behav-
produce benefit for not only innovation recipients and ior change interventions, technology, and more—or any
Damschroder et al. Implementation Science (2022) 17:7 Page 8 of 10

combination. An innovation is ideally supported by a data collection from key decision-makers and individu-
“strong evidence-base” before it is implemented. How- als implementing and/or delivering the innovation about
ever, we also recognize there is lack of agreement on their perceptions of recipients (e.g., recipient characteris-
what types of evidence warrant implementation [46–48] tics and needs), and how those perceptions encourage (or
and there is a compelling need to dismantle knowledge- discourage) completing implementation, informs Imple-
building silos (e.g., clinical trialists versus implementa- mentation Outcomes. Although the CFIR is often not
tion scientists) to translate innovations more quickly into appropriate for use with recipients (because they rarely
practice [49]. Thus, we chose the term “innovation” to hold roles as key decision-makers or innovation imple-
acknowledge that implementation can occur with inno- menters/deliverers), we hope that will change. Recipi-
vations that are supported by diverse sources and types ents should have greater influence, authority, and power
of evidence. in healthcare systems; the CFIR 2.0 will highlight the
importance of implementation teams including innova-
CFIR implementation determinants vs. innovation tion recipients (and innovation deliverers) as members.
determinants When recipients serve in that role, we strongly encour-
When collecting data, researchers must be clear about age using the CFIR to collect data about implementation
the goal of data collection: (1) to predict and/or explain determinants from them—because they are also imple-
implementation outcomes based on implementation mentation team members. Ultimately, equitable popula-
determinants (this is within the scope of the CFIR) or (2) tion impact is only possible when recipients are integrally
to predict and/or explain innovation outcomes based on involved in implementation and all key constituencies
innovation determinants (this is outside the scope of the share power and make decisions together.
CFIR). The following section explores the roles of imple-
mentation versus innovation determinants. Innovation determinants
Innovation determinants capture recipient-level character-
Implementation determinants istics and/or experiences with the innovation that predict
CFIR implementation determinants capture setting-level and/or explain innovation outcomes. These determinants
barriers and facilitators that predict and/or explain ante- are denoted by the gray arrow in Fig. 1 labeled Innova-
cedent assessments and/or anticipated or actual imple- tion determinants. Data (qualitative and/or quantitative)
mentation outcomes. These determinants are denoted on these determinants is best collected from recipients.
by the gray arrow in Fig. 1 labeled CFIR implementation Innovation determinants include constructs or measures
determinants. Data (qualitative and/or quantitative) on that are based on the theoretical framework underlying
these determinants is best collected from individuals who the innovation. For example, in a “small change” weight
have influence and/or authority related to implemen- loss intervention designed for patients, innovation deter-
tation (usually within the implementing setting); these minants included patient-level demographics, motivation
typically include the key decision-makers and individuals and intention, and self-efficacy because the intervention
implementing and/or delivering the innovation. was guided by social-psychological and goal-conflict theo-
Although over 20 users recommended adding a domain ries [54]. This innovation was tested within a randomized
and/or constructs to collect data directly from recipients, clinical trial [55] and a subset of patient characteristics
the CFIR is not the appropriate framework to use for this (innovation determinants) were explored in secondary
purpose unless recipients are also helping to implement analyses to help explain Innovation Outcomes [56–59].
and/or deliver the innovation. As reflected by Orlando The CFIR is not designed to capture these theory-derived
et al., it is disappointing to note that “… while patients are determinants of Innovation Outcomes.
part of the health-care organization and are essential to
assessing intervention [innovation] effectiveness, they are
Conclusion
a less influential component of implementation success in
As implementation science matures as a discipline,
health-care settings than administrators and physicians”
frameworks must mature too [60, 61]. In this debate arti-
(emphases added) [50]. Although hospital systems are
cle, we propose the inclusion of an Outcomes Adden-
increasingly prioritizing patient-centered care, conven-
dum to the CFIR. Our goal is to offer clear conceptual
ing patient advisory boards, and involving patients in
distinctions between the types of outcomes for use with
co-design of initiatives [51, 52], these efforts have not yet
the CFIR, and perhaps other determinant implemen-
resulted in true power-sharing between innovation recip-
tation frameworks as well. These distinctions can help
ients and key decision-makers [53].
bring clarity as researchers consider which outcomes
As a result, direct data collection from recipients does
are most appropriate to evaluate in their research and to
not usually inform implementation outcomes. Instead,
Damschroder et al. Implementation Science (2022) 17:7 Page 9 of 10

help center those outcomes on multiple key constituen- 7. Waltz TJ, Powell BJ, Fernández ME, Abadie B, Damschroder LJ. Choosing
implementation strategies to address contextual barriers: diversity in
cies for sustained outcomes. We hope that sharing this recommendations and future directions. Implement Sci. 2019;14:1–15.
in advance will generate feedback and debate about the 8. Kirk MA, Kelley C, Yankey N, Birken SA, Abadie B, Damschroder L. A
merits of our proposed addendum. systematic review of the use of the consolidated framework for imple-
mentation research. Implement Sci. 2015;11:72. [Link]
s13012-​016-​0437-z.
9. Damschroder LJ, Reardon CM, Sperber N, Robinson CH, Fickel JJ, Oddone
Abbreviations
EZ. Implementation evaluation of the Telephone Lifestyle Coaching (TLC)
CFIR: Consolidated Framework for Implementation Research; EBI: Evidence-
program: organizational factors associated with successful implementa-
based innovation; IOF: Implementation Outcomes Framework; RE-AIM Frame-
tion. Behav Med Pract Policy Res. 2017;7:233–41. [Link]
work: Reach, Effectiveness, Adoption, Implementation, and Maintenance
s13142-​016-​0424-6.
Framework.
10. Skolarus TA, Lehmann T, Tabak RG, Harris J, Lecy J, Sales AE. Assess-
ing citation networks for dissemination and implementation research
Acknowledgements
frameworks. Implement Sci. 2017;12:97. [Link]
We want to express our sincere gratitude to the authors who completed our
s13012-​017-​0628-2.
survey and made this work possible.
11. Whitaker RG, Sperber N, Baumgartner M, Thiem A, Cragun D, Dam-
schroder L, et al. Coincidence analysis: a new method for causal inference
Authors’ contributions
in implementation science. Implement Sci. 2020;15:108. [Link]
MW, CR, and LD developed the literature review search criteria and created
10.​1186/​s13012-​020-​01070-3.
the survey. MW conducted the literature review and fielded the survey. CR
12. Glasgow RE, Harden SM, Gaglio B, Rabin B, Smith ML, Porter GC, et al. RE-
and MW analyzed the survey data. JL, LD, and CR drafted the manuscript; MW
AIM planning and evaluation framework: adapting to new science and
provided survey data in relevant sections. All authors read and approved the
practice with a 20-year review. Front Public Health. 2019;7:64. [Link]
final manuscript.
org/​10.​3389/​fpubh.​2019.​00064.
13. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A,
Funding
et al. Outcomes for implementation research: conceptual distinctions,
This work was funded by the Veterans Affairs (VA) Quality Enhancement
measurement challenges, and research agenda. Adm Policy Ment Health.
Research Initiative (QUE 15-286) and VA Health Services Research and Devel-
2011;38:65–76. [Link]
opment (LIP 20-116).
14. Hung D, Gray C, Martinez M, Schmittdiel J, Harrison MI. Acceptance of
lean redesigns in primary care: a contextual analysis. Health Care Manage
Availability of data and materials
Rev. 2017;42:203–12. [Link]
The datasets used and/or analyzed during the current study are available from
15. Ashok M, Hung D, Rojas-Smith L, Halpern MT, Harrison M. Framework for
the corresponding author on reasonable request.
research on implementation of process redesigns. Qual Manag Health
Care. 2018;27:17–23. [Link]
Declarations 16. Dy SM, Ashok M, Wines RC, Rojas Smith L. A framework to guide imple-
mentation research for care transitions interventions. J Healthc Qual.
Ethics approval and consent to participate 2015;37:41–54. [Link]
The VA Ann Arbor Healthcare System IRB approved this study, declaring it 17. Borup M, Brown N, Konrad K, Van Lente H. The sociology of expectations
exempt from the requirements of 38 CFR 16 based on category 2. in science and technology. Technol Anal Strateg Manag. 2006;18:285–98.
[Link]
Consent for publication 18. Ilott I, Gerrish K, Booth A, Field B. Testing the consolidated framework for
Not applicable. implementation research on health care innovations from South York-
shire: testing the CFIR on health care innovations. J Eval Clin Pract. 2012.
Competing interests [Link]
The authors declare that they have no competing interests. 19. Tinc PJ, Gadomski A, Sorensen JA, Weinehall L, Jenkins P, Lindvall K. Apply-
ing the Consolidated Framework for implementation research to agricul-
Received: 23 August 2021 Accepted: 14 December 2021 tural safety and health: barriers, facilitators, and evaluation opportunities.
Saf Sci. 2018;107:99–108. [Link]
20. Serhal E, Arena A, Sockalingam S, Mohri L, Crawford A. Adapting the
Consolidated Framework for Implementation Research to create organi-
zational readiness and implementation tools for project ECHO. J Contin
References Educ Health Prof. 2018;38:145–51. [Link]
1. McDonald KM. Considering context in quality improvement interven- 00000​000195.
tions and implementation: concepts, frameworks, and application. Acad 21. Ament SMC, Gillissen F, Moser A, Maessen JMC, Dirksen CD, von
Pediatr. 2013;13:S45–53. [Link] Meyenfeldt MF, et al. Factors associated with sustainability of 2 quality
2. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. improvement programs after achieving early implementation success. A
Fostering implementation of health services research findings into prac- qualitative case study. J Eval Clin Pract. 2017;23:1135–43. [Link]
tice: a consolidated framework for advancing implementation science. 10.​1111/​jep.​12735.
Implement Sci. 2009;4:1–15. 22. Callaghan-Koru JA, Islam M, Khan M, Sowe A, Islam J, Mannan II, et al.
3. Fernandez ME, Ten Hoor GA, van Lieshout S, Rodriguez SA, Beidas RS, Factors that influence the scale up of new interventions in low-income
Parcel G, et al. Implementation mapping: using intervention mapping to settings: a qualitative case study of the introduction of chlorhexidine
develop implementation strategies. Front Public Health. 2019;7:158. cleansing of the umbilical cord in Bangladesh. Health Policy Plann.
4. Nilsen P. Making sense of implementation theories, models and frame- 2020;35:440–51. [Link]
works. Implement Sci. 2015;10:53. 23. Hill JN, Locatelli SM, Bokhour BG, Fix GM, Solomon J, Mueller N, et al. Eval-
5. Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research uating broad-scale system change using the Consolidated Framework for
and practice. Am J Prev Med. 2012;43:337–50. [Link] Implementation Research: challenges and strategies to overcome them.
amepre.​2012.​05.​024. BMC Res Notes. 2018;11:560. [Link]
6. Stirman SW, Baumann AA, Miller CJ. The FRAME: an expanded framework 24. Morgan D, Kosteniuk J, O’Connell ME, Kirk A, Stewart NJ, Seitz D,
for reporting adaptations and modifications to evidence-based interven- et al. Barriers and facilitators to development and implementation
tions. Implement Sci. 2019;14:1–10. of a rural primary health care intervention for dementia: a process
Damschroder et al. Implementation Science (2022) 17:7 Page 10 of 10

evaluation. BMC Health Serv Res. 2019;19:709. [Link] 45. Curran GM. Implementation science made too simple: a teaching
s12913-​019-​4548-5. tool. Implement Sci Commun. 2020;1:27. [Link]
25. Vidgen HA, Love PV, Wutzke SE, Daniels LA, Rissel CE, Innes-Hughes C, s43058-​020-​00001-z.
et al. A description of health care system factors in the implementation of 46. Petit-McClure SH, Stinson C. Disrupting dis/abilization: a critical explora-
universal weight management services for children with overweight or tion of research methods to combat white supremacy and ableism in
obesity: case studies from Queensland and New South Wales, Australia. education. Intersect Cri Issues Educ. 2019;3:4.
Implement Sci. 2018;13:109. [Link] 47. Hall BL, Tandon R. Decolonization of knowledge, epistemicide, participa-
26. Breimaier HE, Heckemann B, Halfens RJG, Lohrmann C. The Consolidated tory research and higher education. Res All. 2017;1:6–19. [Link]
Framework for Implementation Research (CFIR): a useful theoretical 10.​18546/​RFA.​01.1.​02.
framework for guiding and evaluating a guideline implementation pro- 48. Althaus C. Different paradigms of evidence and knowledge: recognising,
cess in a hospital-based nursing practice. BMC Nurs. 2015;14:43. [Link] honouring, and celebrating Indigenous ways of knowing and being.
doi.​org/​10.​1186/​s12912-​015-​0088-4. Aust J Public Adm. 2020;79:187–207. [Link]
27. Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. 12400.
Psychometric assessment of three newly developed implementation 49. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-imple-
outcome measures. Implement Sci. 2017;12:108. [Link] mentation hybrid designs: combining elements of clinical effectiveness
s13012-​017-​0635-3. and implementation research to enhance public health impact. Med
28. Reilly KL, Kennedy S, Porter G, Estabrooks P. Comparing, contrasting, and Care. 2012;50:217–26. [Link]
integrating dissemination and implementation outcomes included in the 50. Orlando LA, Sperber NR, Voils C, Nichols M, Myers RA, Wu RR, et al.
RE-AIM and implementation outcomes frameworks. Front Public Health. Developing a common framework for evaluating the implementation
2020;8:430. [Link] of genomic medicine interventions in clinical care: the IGNITE Network’s
29. Weiner BJ, Mettert KD, Dorsey CN, Nolen EA, Stanick C, Powell BJ, Common Measures Working Group. Genet Med. 2018;20:655–63. [Link]
et al. Measuring readiness for implementation: a systematic review of doi.​org/​10.​1038/​gim.​2017.​144.
measures’ psychometric and pragmatic properties. Implement Res Pract. 51. Lyon AR, Whitaker K, Locke J, Cook CR, King KM, Duong M, et al. The
2020;1:263348952093389. [Link] impact of inter-organizational alignment (IOA) on implementation
30. Miake-Lye IM, Delevan DM, Ganz DA, Mittman BS, Finley EP. Unpack- outcomes: evaluating unique and shared organizational influences in
ing organizational readiness for change: an updated systematic review education sector mental health. Implement Sci. 2018;13:24.
and content analysis of assessments. BMC Health Serv Res. 2020;20:106. 52. Dopp AR, Parisi KE, Munson SA, Lyon AR. Integrating implementation and
[Link] user-centred design strategies to enhance the impact of health services:
31. Weiner BJ, Belden CM, Bergmire DM, Johnston M. The meaning and protocol from a concept mapping study. Health Res Policy Sys. 2019;17:1.
measurement of implementation climate. Implement Sci. 2011;6:78. [Link]
[Link] 53. Trofino J. Power sharing. A transformational strategy for nurse retention,
32. Glasgow RE. Evaluating the impact of health promotion programs: using effectiveness, and extra effort. Nurs Leadersh Forum. 2003;8:64–71.
the RE-AIM framework to form summary measures for decision making 54. Lutes LD, DiNatale E, Goodrich DE, Ronis DL, Gillon L, Kirsh S, et al. A
involving complex issues. Health Educ Res. 2006;21:688–94. [Link] randomized trial of a small changes approach for weight loss in veterans:
org/​10.​1093/​her/​cyl081. design, rationale, and baseline characteristics of the ASPIRE-VA trial.
33. Abildso CG, Zizzi SJ, Reger-Nash B. Evaluating an insurance-sponsored Contemp Clin Trials. 2013;34:161–72. [Link]
weight management program with the RE-AIM Model, West Virginia, 09.​007.
2004-2008. Prev Chronic Dis. 2010;7:A46. 55. Damschroder LJ, Lutes LD, Kirsh S, Kim HM, Gillon L, Holleman RG, et al.
34. Feldstein AC, Glasgow RE. A practical, robust implementation and sus- Small-changes obesity treatment among veterans. Am J Prev Med.
tainability model (PRISM) for integrating research findings into practice. Jt 2014;47:541–53. [Link]
Comm J Qual Patient Saf. 2008;34:228–43. [Link] 56. Masheb RM, Lutes LD, Kim HM, Holleman RG, Goodrich DE, Janney CA,
7250(08)​34030-6. et al. Weight loss outcomes in patients with pain: weight loss and pain.
35. Berwick DM, Nolan TW, Whittington J. The triple aim: care, health, and Obesity. 2015;23:1778–84. [Link]
cost. Health Aff. 2008;27:759–69. [Link] 57. Masheb RM, Lutes LD, Myra Kim H, Holleman RG, Goodrich DE, Janney
36. Bodenheimer T, Sinsky C. From triple to quadruple aim: care of the CA, et al. High-frequency binge eating predicts weight gain among vet-
patient requires care of the provider. Ann Fam Med. 2014;12:573–6. erans receiving behavioral weight loss treatments: high-frequency binge
[Link] eating and weight gain. Obesity. 2015;23:54–61. [Link]
37. Jackson GL, Damschroder LJ, White BS, Henderson B, Vega RJ, Kilbourne oby.​20931.
AM, et al. Balancing reality in embedded research and evaluation: low 58. Vimalananda V, Damschroder L, Janney CA, Goodrich D, Kim HM, Hol-
vs high embeddedness. Learn Health Sys. 2021. [Link] leman R, et al. Weight loss among women and men in the ASPIRE-VA
lrh2.​10294. behavioral weight loss intervention trial: sex-specific weight loss results in
38. Damschroder LJ, Knighton AJ, Griese E, Greene SM, Lozano P, Kilbourne ASPIRE-VA. Obesity. 2016;24:1884–91. [Link]
AM, et al. Recommendations for strengthening the role of embedded 59. Janney CA, Masheb RM, Lutes LD, Holleman RG, Kim HM, Gillon LR, et al.
researchers to accelerate implementation in health systems: findings Mental health and behavioral weight loss: 24-month outcomes in Veter-
from a state-of-the-art (SOTA) conference workgroup. Healthcare. ans. J Affect Disord. 2017;215:197–204. [Link]
2021;8:100455. [Link] 03.​003.
39. Lennox L, Maher L, Reed J. Navigating the sustainability landscape: a 60. Kislov R, Pope C, Martin GP, Wilson PM. Harnessing the power of theoris-
systematic review of sustainability approaches in healthcare. Implement ing in implementation science. Implement Sci. 2019;14:103. [Link]
Sci. 2018;13:27. [Link] org/​10.​1186/​s13012-​019-​0957-4.
40. Scheirer MA, Dearing JW. An agenda for research on the sustainability of 61. Glasgow RE, Estabrooks PA, Ory MG. Characterizing evolving frameworks:
public health programs. Am J Public Health. 2011;101:2059–67. [Link] issues from Esmail et al. (2020) review. Implement Sci. 2020;15:53. [Link]
doi.​org/​10.​2105/​AJPH.​2011.​300193. doi.​org/​10.​1186/​s13012-​020-​01009-8.
41. Damschroder LJ. Clarity out of chaos: use of theory in implementation
research. Psychiatry Res. 2020;283:112461.
42. Dobson D, Cook TJ. Avoiding type III error in program evaluation. Eval Publisher’s Note
Program Plann. 1980;3:269–76. [Link] Springer Nature remains neutral with regard to jurisdictional claims in pub-
90042-7. lished maps and institutional affiliations.
43. Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability frame-
work: addressing the paradox of sustainment amid ongoing change.
Implement Sci. 2013;8:117.
44. Rogers E. Diffusion of innovations. 5th ed. New York: Free Press; 2003.