N I ST

T r ustworthy and Responsible AI

NIST AI 600
Art ificial Intelligenc Ris k Management
FrameworkGenerat ive Art ificial
Int elligence Profile
This publicationis available free of charge from:
https ://[Link] g/10.6028 [Link].600
N I ST
T r ustworthy and Responsible AI
NIST AI 600
Art ificial Intelligenc Ris k Management
Generat ive Art ificial
Int elligence Profile
https ://[Link] g/10.6028 [Link].600
July 2024
U.S. Departmenof Comm erce
Gina M. Raim ondo, Secretary
Laurie E. Locas cio, NIST Director and Under Secretary of
Com m erce for Standards and Technology
About AI at NIST
Stanley, and Elham Tabassi.
NIST Technical Series Policies
National Ins titute of Standards and Technology
Additional informatioabout this publicationand other NIST AI publication
https ://[Link] [Link]/Home
are available at
Disclaim er
ute of Standards and Technologynor is it
s is
Tab le of Contents
Introdu ction
Overview of Risk s Uniqu e to or Exacerb ated
by GAI
Su ggested Actions to Manage GAI Risk s
Appendix A. Primary GAI Considerations
Ap p endix B. References
This document
is a
cros s
s ectoral
companionres ource
is an
le. Cros s
s uch as the us e of
large language models
c onte nt .fl
d e rive d f ro m
. Th e
A I m o de l t h at is
t ra ine d on broa d data ; ge
ne ra lly us e s s e lf
s upe r v is io n; c onta ins
at l ea st te ns of
d ire c t s t h e
de ve lop a c om pa nio n
p ro c e s s
fa c il it ate d v ia
i n t his doc um e nt
p rim a rily
a ddr e s s
a s t he
s pa c e
e vo l ve
a nd e m pi ric a l e v
ide nc e
D e pt h Glo ssa r y o f
Te r m s
from s e veral
Overview of Risks Unique to or ExacerbateGAI
of occurring and the
or degree of the cons equences of the corres ponding
event.
r m ay be
can vary along many dimens ions :
Stage of the AI lifecycle:
des ign
developm ent
decom m is s ion
m ay
m odel
or system
at the
system or
im pacts on
access to
Source of ris k:
em erge from factors relate d to the
and in other cas es
fiA lgo r it h m ic m o n
o c ult ure s fl refe rs to t
h e
syste m s
to c o rre l ate d fa i lu
re s
s ame algorit hm.
proj e c te d t he im pa c
t of AI on
indust r y
sur veys
i n d i c at e t h at
t hat bot h
e mp
lo ye e s a nd
e m p lo ye rs a re
others res ult
s cale:
m ay
abruptly or
due to
im ages
trust in public
mechanis ms
and libraries
data types
us ed for
m eas ure
bas ed on
. They may
direct and m aterial impacts
harm s
m odel
or system
us e
cas e or
s cope
or evaluate
state of the s cience of AI m eas urement
and safety
This document focus es on
is an
evidence bas e
system s are not cons idered
Future updates may
the developmeand us e of
labeled according to
the outcom e
s ubject
dom ain
and others are
fioffl or
fifrom fl
an is s ue or
them e
2) M isuse by hum a ns ( or m
a licious use) :
D ata P riva c y ;
3) Ecosy stem / societa l r isks ( or
system ic r isks)
P ro p e rt y
as ed
acces s to
or synthes is of
materially nefarious
m aterials
or agents
colloquiall as
us ers
may be
mis led or
Dange rous
harm or
Includes d
dis paraging
or stereotypi content
Data Privacy:
Impacts due to
us e
dis clos ure
or de
or other
Environmen
Impacts:
ystem ic
bias es
perform ance
s ub
groups or languages
poss ibly
due to
training data
; undes ired homogeneity
syste m or m odel
lead to
founded decis ion
bias es
Arrangem ent
s of
human and
AI system
the human
algorithmicavers ion
system s
s cale dis
and mis
autom ated
S om e c omm e nte rs have note
d t hat
t h e te r m s
hum a n
c a n be hig hly
o n t he n at ure o f t
he
t he
m o st a p p ro p ria te
d is p a rit y
been in t he abs ence of any AI system
outc om e una m big uo us ly a
harm of t he syste m
m ay
com prom is ea
integrity of
IntellectualProperty:
of alleged
trade s ecrets ;
Obs cen
bus ive
im agery
can caus e harm
of adults
transparenor untraceable
upstream third
proces s ed and
the AI lifecycle
m ay
enable malicious
actors to m ore eas ily
acces s
m aterials
or agents
acces s ible
could facilitate its
analys is or synthes is
as s istance
phys ical synthe s is
us e of chem ical or
m aterials and
malicious actors
one s uch barrier
can help
addres s thos e barriers
Furtherm ore
chem ical and
biological des ign tools
trained on
that aid in chemical and biological des ign
m ay
and biology
. As thes e m odels becom e m ore
chemical or biological
While s om e of thes e
des cribed
lie beyond the
the ability of
AI tools
acces s to relevant data and tools
TrustworthAI
erroneous or fals e content
in res pons e to prompts
diverge from the
prom pts
or other
contradict previous ly generated statementsin the s ame
es e
als o
fl or fi
occur acros s
are a natural
re s ult
des igned
ir training data
in a s entence or phras e
ly accurate
and cons istent
als o
outputs that are factually
or internally inconsistent
it come s to
ended prompts
form res ponses
and in
of the respons e
leading us ers to act upon
or prom ote
fals e
This poses a
m any real
incorrect diagnos es
and/or recommend
im portant to m onitor
include confabulated
mis lead
humans into inappropriately
fals ely
that it
is human
has human
em erging
areas of study
and Interpretable
content that is
greate r
eas e and s cale
other technologies
reported to generate
of fa ls e hoo ds
a re
fa c t ua l con te nt c a
n be
a de s ired be hav io r.
i n c u r r e n t s
tate
t he
create im ages
that could be us ed to
promote dangerousor violent mes s age
are pres ent for other
video and audio.
als o
produce content that recommens elf
any current systems
restrict m odel outputs
to limit
certain content or
but this approach may
in res pons e to other les s
Data Privacy
can be
Studies have obs erved that
users may
dis clos e mental health
and that
thes e
or hateful
s uch as fueling dangerousor violent behaviors. The spread
of
stere otypical
Data Privacy
s everal
s om e cas es
m ay
accepted privacy principles
ost m odel developers
do not dis clos e
included in their training data. This problem has been referred
to as
for data pres ent
in a
s mall number of training s amples
infe r
II or
Thes e i
even if the
that is
us ed to
dis advantage or harm
s uch as
inferences of PII can contribute to
or s econdary
im pacts
an contribute to
advers e decis ions
or groups
m aint
bas ed on
the m odality
of the
conte nt
Curre nt e
raining a s ingle
transform er
emit as much carbon
m ore energy
and carbon
ntens ive th
training and tuning s uch models
m ay
contribute to
their environmen
impacts
Currently there
no agreed upon method
environm ental
im pacts
in many forms
and can become ingrained in automatedsystems . AI systems
can increas e the s peed and s cale
m anifest
and amplify
s ociety
im age m odels
underrepreent
Im age generator m odels
als o
bias ed or stereotypeoutput for
various demographic
im age features that
Harm ful b
may stem from
their training data
can als o
bias bas ed on
Harmful b
als o
lead to harm s via
m odel perform s for
m ay
certain dialects
dis criminator decis ion
to perform s imilarly
acros s all
s ubgroups
Dis parate or r
educed perform ance for
res ource
als o
pres ents challenges
endangerelanguages
becom e em bedded in
processes that
to us e the
undes ired
that are overly uniform
content divers ity
lead to
unreliable decis ion
Thes e phenomena
s ing
points of failure.
contribute to
model collaps e
m odel
training over
odel collaps e
could lead to
from the model us ed to
us e
Humans bring their
dom ain
unneces s arily
avers e
system s
and thus
deprive them s elves
or others
Convers ely
hum an
m ay
or m ay
produced by
s ources
This phenomenon
i s
to autom ated
system s
There may als o be
im pacts
des cribes the
appropriatevidence. High
202 2
Whi te H o use
eas e the
conte nt
at s cale
the content
can als o
or dis
an actor
s ubtle changes
enable a
for m alicious actors to produce
Current and e
m odels
m ay
erode public trust
Pentagon blast
in more reach and
bas ed system s
auto m ated
imultaneouly
prom pt
or data
pois oning.
s om e
for us e in s everal parts of
the integrity
and model
and s ecur
AI system s
of the AI
Se e also
value chain
environm ents
m ay need to
adapt or evolve
For instance
input is provided to a
s o that it
input them
a variety of
cons equences to
interconnesystem s .
code rem otely
on a machine.
previous ly undis clos ed
data pois oning
an advers ary
com prom ises
a training
datas et used by a model to manipulateits
s or
data or parts
of the
IntellectualProperty
the us e of
is not a fair
us e under the fair us e doctrine.
dis playing
instances of
could infringe on
the status of
content that is s im ilar to but
does not
strictly copy
being debated in legal
dis cus sions are
re gard
us e
Obs cene,
cons ens ual
im agery
of adult
abus ive
conte nt can
harm s
including children.
can have
of s uch images c
s ources from
im pact
ubs equent
s uicidal thoughts
Data us ed for training
m odels
m ay
ecent report
increas ingly capable
m oved
value chains
involve m any
Thes e componentmight be improp
system s
and s ome other digital technologies
s cale of
limited numbers of
m odels ; an
in tegrat
devices and s ervices .
com ponents
and data s ources
thes e s ources .
com ponents
can als o
on accuracy and robustness
test datas ets
commonlyus ed
label errors
Inaccuraciein thes e labels
m any
cons ider
during the
proces s .
Sugges ted
to Manage GAI Ris ks
s et forth in the AI
for managing
are encourageto
s uggested
system s
vary dependingon
stage of
Suggested a
AI RMF subcategor
to stream line thes e
every subcategorof the AI RMF is included in this
document.
for only s ome s ubcategori.
A s t his doc um e nt
s ubc ate gorie s
n ot
a ddre s s e d he re
m ay b e
a dde d
l at e r
N ot
s uggested
is relevant
s uggested
s hould be
bas ed
their unique
uses of
system s
of s uggest
s uggested
002 corres ponds to the
s econd s uggested
or AI actor
GAI Risks:
AI Actor
each s ubcategory
AI developm ent and
s uggested
G OV ER N 1.1:
S u g gested
G A I
R is k s
0 01
de ve lopm e nt a n d
in c lu d in g
t hos e re late d to data pr iva
cy
D ata P riva c y
I n te l le c t u a l
P ro p e rt y
A I A c to r Ta s k s
: Gover na nce a nd O versig ht
AI Actors
G OV ER N 1.2:
S u g gested
G A I
R is k s
a nd
h i s to r y
to a dva nce
d ig ita l
c onte nt t ra ns pa re nc y
a pproa c he s
D at a
P riva c y
I n te g rit y
I n te lle c t u a l P
ro p e rty
AI Actor Tasks: Gover na nce a nd O versig
ht
G OV ER N 1.3:
S u g gested
G A I
R is k s
0 01
a nd
im p a c t s
data syste ms ; Ha rm
t o fu nda m e ntal rig ht s
pub li c
s a fe t y
or
d is c r im i n a
inva l id or unt rut hfu lo ut put
;
Po s s ib i lit y fo r m
; Whe t her t he
syste m introduc e s
a lic i o u s u s e
impact on
s om e g rou p s c om p
a re d t o ot h e rs
C onte nt
H a rm f u l
D a nge rous
C onte nt
o r a s s u ra n c e
c ri te ria
p a rt o f
D a nge rous
Obtain inp ut from stakeholdecommunitito identify unacceptabuse
accordanc with activities in the AI RMF Map function
A bus ive C onte nt
h ie ra rc h
m ode l a dvan c e m e nt
a nd
s u c h a s m o d e
m o n o c u lt u re .
l c o lla p s e a n d
alg orit h m ic
la rge
D a nge rous
I n te g rit y
AI Actor Tasks: Gover na nce a nd O versig
ht
G OV E R N
G A I
R is k s
0 01
A bus ive C onte nt
C onte nt
C onte nt
D ata P rivacy
; C iv il
AI Ac to r Ta s k s: AI D
Gover na nce a nd O versig ht
e ve lo p m e nt, AI D e
p lo y m e nt,
G OV ER N 1.5:
a n d
a re
S u g gested
G A I
R is k s
0 01
c onte nt prove na nc e
a nd i nc ide nt
m o n it o r in g
inc i de nt re s pons e a nd
i nc i dent dis c l os ure proc
e s s e s a s re quire d.
a nd
dig ita l cont e nt t ra ns pa
re ncy
m e t hods
fo r
I n te lle c t u a l
P ro p e rt y
A I A c to r Ta s k s
:
G OV ER N 1.6:
S u g gested
G A I
R is k s
0 01
a nd
0 02
A I inc ident
data ba s e
o r
i nc ide nt mo nito r
D ata P riva c y
I n te g rit y
I n te lle c t u a l P
ro p e rty
A I A c to r Ta s k s
:
Gover na nce a nd O versig ht
G OV ER N 1.7:
P roc e s s e s a nd proc
e d ure s are in p la c e
for de c om m is s ion ing a
n d pha s i ng out A I syste
m s s a fe ly and in a m
a nne r t hat
S u g gested
G A I
R is k s
0 01
ro to c o ls a re put in p
e ns ure
la c e to
are a b le
neces s ar y
U s e of
op e n
s o u rc e
d ata or m od e ls ;
A I A c to r Ta s k s
:
G OV ER N 2.1:
S u g gested
G A I
R is k s
in c lu d in g
A I in c id e n t
data ba s e
o r
AI Actors
0 04
s e c u r it y
C onte nt
C reate
me chanis ms
to p
or p os e s a
to pub li c
s afe t y
D a nge rous
C onte nt
A I A c to r Ta s k s
:
Gover na nce a nd O versig ht
G OV ER N 3.2:
a nd ove rsig ht of A I systems .
S u g gested
G A I
R is k s
0 01
Po lic ie s a re in p la c
e to
o r assessments
0 02
a nd re d
nc l udi ng
c r ite r ia fo r
fo r
A I A c to rs : A I D
e s i g n
G OV ER N 4.1:
S u g gested
G A I
R is k s
fo r
c a de nc e s .
i n c lu d in g in te rn
al
a nd C om pone nt
A I A c to r Ta s k s
:
G OV ER N 4.2:
S u g gested
G A I
R is k s
us e a n d te rm s of s
e r v ic e
I n te lle c t u a l P
ro p e rty
D a nge rous
A bus ive C onte nt
p a rt y
A I A c to r Ta s k s
:
G OV ER N 4.3:
G A I
R is k s
empl oyed
c onte nt
e tc .
gene rated
A I syste m s
D at a
P riva c y
G OV ER N 5.1:
to A I
G A I
R is k s
0 01
system developme.
H a rm ful
0 02
oc um e nt
G OV ER N 6.1:
infr inge m ent of a t hird
S u g gested
G A I
R is k s
0 01
t h ird
p a rt y
g ht s
D ata P riva c y
I n te lle c t u a l
P ro p e rt y
to p rom ote
D e ve lop a nd va li date a
prove na nc e
pproac he s for m e a s ur ing
t h e s uc c e s s of
c onte nt
t hat s pec if y
S e c u rit y
I n te l le c t u a l
P ro p e rty
I mple me nt a
us e
c a s e d based
m o n it o r t h ird
va l ue c ha in
a nd
le gal compl iance
D ata P riva c y
I n te g rit y
I n te lle c t u a l P
ro p e rty
I nc lude
c la us e s in c ont ra c
t s
a n
e va lu ate t h ird
p a rt y
proc e s s es a nd sta nda rds
.
I nve n to r y a ll t h
ird
C om p on e n t
c onte nt
I n te lle c t u a l P
ro p e rty
roc esses
m o n ito ri n g
to o ls fo r m o n ito rin
g t h ird
op e n
prov ide rs a ga inst inc i de nt
or v u lne ra bi lit y da ta
ba s e s
D ata P riva c y
S e c u rit y
I n te l le c t u a l
P ro p e rty
t h ird
p a rt y
pe rs onne l.
I n te lle c t u a l P
ro p e rty
A I A c to r Ta s k s
:
G OV ER N 6.2:
pa rt y data or A I syste m s
h ig h
de e m e d to be
S u g gested
G A I
R is k s
syste m
va l ue c ha in
re lia n c e
t h ird
D o c u m e n t in c
id e n t s inv olv in g t
h ird
data a n d
o pe n
s o u rc e
I n te lle c t u a l P
ro p e rty
p a rt y
AI Actors
D ata P riva c y
S e c u rit y
H a rm f u l
syste m s in de p loy m e nt
.
m ay inc l ude m a nua l
pro cessing.
stan dard te rms t hat may amp lif y
o r
/o r c o n t r i b u
te to u
C o n s id e r : C le
a r
s e rio u s i n c id e
t h ird
n t s a r isin g f ro m
pa rt y data a nd syste m
Ser v ice
a nd C om pone nt
A I A c to r Ta s k s
:
M A P 1.1:
a lo n g
o r
S u g gested
G A I
R is k s
0 01
inte nde d
ons i de r fac tors s uc h a
s inte rna l vs .
t un ing
a nd
a ug m e nte d
D ata P riva c y
I n te lle c t u a l
P ro p e rt y
a ddre s s
AI Actors
fa ilu re m o d e s ; I
n
la b e l u s e ;
a nd st ruct ure d hum a n f
H a rm ful
D a nge rous
C onte nt
D a nge rous
C onte nt
a nd/o r Abus ive C onte nt
AI Ac to r Ta s k s: AI D
e p lo y m e nt
M A P 1.2:
I n te rd is c ip li n a
AI Actors
r y
in te rd is c ip li n a r
y
S u g gested
G A I
R is k s
H a rm ful
H a rm ful
A I A c to r Ta s k s
:
AI D e p lo y m e nt
M A P 2.1:
S u g gested
G A I
R is k s
pu rpos e s .
te st a nd
d e c is io n
I n te lle c t u a l P
ro p e rty
D ata P riva c y
A I A c to r Ta s k s
:
T EV V
M A P 2.2:
AI Actors
S u g gested
G A I
R is k s
inc l udi ng
for c onte nt prove na nc e
a nd if it s e r ve s a
o t h e r syste m s .
s a n upst re a m de pe nde
nc y for
prove na nc e m ig ht be c om
prom is e d.
A I A c to r Ta s k s
:
End U sers
M A P 2.3:
t ru c t
S u g gested
G A I
R is k s
0 01
out put
c r y ptog ra phic te c hn ique
s
of c ontent inp ut s
0 02
I n te lle c t u a l P
ro p e rty
D e ploy and d oc um e nt fa
c t
so urces.
ge ne rate d c onte nt .
o r m is u s e .
A I A c to r Ta s k s
:
AI D evelo p m ent, D o m a
in Ex p er ts , T EV V
M A P 3.4:
a nd re leva nt
S u g gested
G A I
R is k s
us e rs c an a c c urate ly
c onte nt line a ge a n d o
unde rsta nd
rig i n.
d ig i ta l
c onte nt
t ra ns pa re nc y.
I nvo l ve t h e e n d
A I syste m
protot y pi ng
va rio u s s c e n a r
io s
such as c risis
D a nge rous
AI Ac to r Ta s k s: AI D
Ex p e r t s , En d
e s ig n , AI D e ve lo
p m e nt, D o m a in
M A P 4.1:
inc lud ing t he u s e of t
hird
p a rt y d ata o r
t h ird
rig h t s .
S u g gested
G A I
R is k s
C onduc t perio dic m on ito r
in g o f
pos s ible insta nc e s o f
PII or
D ata P riva c y
I m ple m e nt proc e s s
e s for re s pondi
c la im s
o r o t h e r r ig h
t s
I n te lle c t u a l P
ro p e rty
D ata P riva c y
a nd a c c ordi ng to
I n te lle c t u a l P
ro p e rty
D ata P riva c y
A bus ive C onte nt
i na ppropr iate
U s e of Ille ga l or da nge
rous c onte nt ;
Tra i ning data
im b a la n c e s t h
at c o u ld g ive rise t o
h a rm f u l b ia s e
s
I n te lle c t u a l P
ro p e rty
S e c u rit y
D a nge rous
D at a
P riva c y
pa rt y inte lle c t ua l prope
rt y a nd
I n te lle c t u a l P
ro p e rty
t une d
or e nha nced
o n to p of t hird
p a rt y
m o d e ls .
c on t
of us e
o r m a p p
s uc h a s
I n te lle c t u a l P
ro p e rty
D a nge rous
D at a
P riva c y
a pproa c he s to de te c t
t he pre s e nc e o f
PII or
D ata P riva c y
C onduc t ap propr iate di lige nce on
t ra i ning data
us e
a nd
I n te lle c t u a l P
ro p e rty
D ata P riva c y
P ro c u re m e nt , T
h i rd
M A P 5.1:
e p l o ye d
S u g gested
G A I
R is k s
0 01
S e c u rit y
o f
D a nge rous
A bus ive C onte nt
C o n s id e r d
is c lo s
in g
of t he
d is c lo s u re s .
D a nge rous
C onte nt
C onduc t
a d ve rs a ria l ro le
M A P 5.2:
AI Actors
S u g gested
G A I
R is k s
0 01
AI Actors
0 02
AI Actors
inc l udi ng t hird
impact s .
A I A c to r Ta s k s
:
M EA S U R E 1.1:
o r c a nnot
m e a s u re d a re p
pe rly d ocum e nte d.
ro
S u g gested
G A I
R is k s
me t
H a rm f u l
D evelop a s ui te of met rics to
publ ic
evaluate st ruct ured
AI Actors
H a rm ful
va luate nove l m e t hods a nd
te c hn ol og ie s for t he
m e a s ure m e nt of
re late d
c lu d in g in
c onte nt prove na n c e
out put s .
A bus ive C onte nt
out put s a re e quita ble a c
ross va r ious s ub
te a m ing to m on ito r a
nd im prove out put s .
va luate the qua lit y a nd inte g
r it y of d ata us e d in
t ra in ing a nd t he prove na
nce of
generated content
m p lo y in g
e ng ine e ring a n d
us e .
t h at c a nnot be
A I A c to r Ta s k s
:
AI D evelo p m ent, D o m a
in Ex p er ts , T EV V
M EA S U R E 1.3:
li ne de ve l ope rs for t he
syste m a nd /or i nd e pende nt
a s s e ss ors a re
AI Actors
S u g gested
G A I
R is k s
H a rm ful
n ga ge i n
inte r na l and
A I re d
AI Actors
H a rm ful
D at a
P riva c y
A I A c to r Ta s k s
:
En d
M EA S U R E 2.2:
S u g gested
G A I
R is k s
te c hni que s s uc h a s
re
S e c u rit y
d at a
d at a
i n t e ra c t
p r iva c y a n d s e
. C ons i de r
c u rit y
in g
data t o pro te c t t he p
hum a n s ubje c t s ;
riva c y o f
in g
in g
a ny p e rs o n a lly
D ata P riva c y
Human AI
I n te g rit y
D a nge rous
C onte nt
c on s e n t for
p re s e nt o r
D ata P riva c y
I n te g rit y
o r o t h e r
p riva c y
e nha nc i ng te c hn olog ie
s
D ata P riva c y
A I A c to r Ta s k s
:
M EA S U R E 2.3:
S u g gested
G A I
R is k s
C ons ide r ba s e l ine
m e t hods .
S e c u rit y
S h a re re s u lt s o
f p re
AI Actors
D ata P riva c y
S e c u rit y
D a nge rous
H ate ful C onte nt
A I A c to r Ta s k s
:
A I D e p l o y m e
nt, T EV V
M EA S U R E 2.5:
S u g gested
R is k s
Av o i d
re t rie va l
t uni ng
o r re t rie va l
data is ground e d.
D a nge rous
C onte nt
A I A c to r Ta s k s
:
D o m a i n Ex p e r
ts , T EV V
M EA S U R E 2.6:
S u g gested
G A I
R is k s
Assess adverse impacts
va lu e c h a in
o r o t h e r
AI Act ors
d u rin g
C onte nt
D a nge rous
C onte nt
le ve ls o f
h a rm f u l b ia s
syste m tra ini ng data .
D ata P riva c y
I n te lle c t u a l P
ro p e rty
A bus ive C onte nt
D a nge rous
D a nge rous
C onte nt
D a nge rous
H ate ful C onte nt
impact s are detected.
I n te g rit y
c y be r
p o s s ib le
on of
s a fe t y
me as ure s
A I A c to r Ta s k s
:
M EA S U R E 2.7:
AI system s ecur it y and res il ience
a re e va lu ate d a nd do
c um e nted.
S u g gested
G A I
R is k s
Apply estab lished secu rity measures to: Assess
u d e of
t h r e at
s uc h a s
t he
ot he r basel ine s e c ur it
y c onc e rns
D ata P riva c y
a ga inst ind ust r y sta nda rds
and be st
fe at ure s and c onte nt prove nanc
e m e t ho ds a ga inst ind
ust r y state
t he
a rt .
S e c u rit y
generated content
a nd
c o n c e rn s a n d
/o r c u rre n t li tera c
y le ve ls re l ate d to c
o n te n t p ro ve n a
n c e
a nd
unde rsta ndi ng of la be ls on c
onte nt
S e c u rit y
d ig ita l
a nd
t he
a n d inc i dent s
S e c u rit y
Pe rfo rm A I re d
D a nge rous
t un ing d oe s not
c o m p ro m is e s a
fe t y a nd s ec urit y c
o n t ro ls .
S e c u rit y
D a nge rous
H ate ful C onte nt
re m a in
be e n c omprom is e d.
A I A c to r Ta s k s
:
M EA S U R E 2.8:
doc um e nte d.
S u g gested
G A I
R is k s
C o m p ile
act ual
a nd
in te lle c t u a l
p ro p e rt y in f rin ge
m e n t
I n te lle c t u a l P
ro p e rty
te ame rs.
Us e
es across the AI
A I A c to r Ta s k s
:
M EA S U R E 2.9:
t o info rm re s pons ible us e
a nd g ove rna n c e .
S u g gested
G A I
R is k s
A pply a nd
a pproa c he s; F i ne
t un ing
o r re t rie va l
a pproa c he s;
A I A c to r
Ta sks:
A I D e p l o y m e
nt, A I Im pa c t A s s
e s s m e nt, D o m a
i n Ex p e r ts , En d
M EA S U R E 2.10:
S u g gested
G A I
R is k s
C onduc t AI re d
team ing to assess
I n te lle c t u a l
P ro p e rt y
g uide t he de s ig n of provena
nc e
d at a
te c h nique s
t rain ing d ata s ample s
d at a
A I A c to r Ta s k s
:
A I D e p l o y m e
nt, A I Im pa c t A s s
e s s m e nt, D o m a
i n Ex p e r ts , En d
M EA S U R E 2.11:
Fa i rne s s and b ia s
a re e va lu ate d a nd re
s ult s a re doc um e nte d.
S u g gested
G A I
R is k s
00 1
A pply use
H ate ful or H a rm fu l
Win oge nde r
Sche ma s
h atef ul conte nt
pos s ible tra ini ng / te st data
c ros s
de ploy m e nt e nv ironm e nt
.
g ro
c o m e s t h at re ly
c o lla b o
D a nge rous
C onte nt
d at a ; Fo
or c on te n t
Wi noge nde r S c he m a s
is a s ample s e t of
t ra ini ng data
is n ot ove r ly h om oge n
ou s or
pro duc e d
m o d e l c o lla p s
e .
A I A c to r Ta s k s
:
U sers,
M EA S U R E 2.12:
a re a s s es s e d a
nd doc um e nted.
S u g gested
G A I
R is k s
00 1
D a nge rous
C onte nt
00 2
t ra i ning an d
c onc e rns
A I A c to r Ta s k s
:
M EA S U R E 2.13:
doc um e nte d.
S u g gested
G A I
R is k s
00 1
C re ate m e a s u re m
e n t e rror mod e ls f o
r p re
de ploy m e nt m e t ric s
to de m onst rate
va ria n c e in a p p l
ie
ha tef ul
c onte nt
I n te g rit y
A I A c to r Ta s k s
:
nd M onitor ing , T EV V
M EA S U R E 3.2:
S u g gested
G A I
R is k s
inc l udi ng
AI Actors
A I A c to r Ta s k s
:
M EA S U R E 3.3:
S u g gested
G A I
R is k s
C onduc t impa c t
c onte nt
a nd a c c ompa ny i ng
systems related t o
d ig ita l
c onte nt t ra ns pa re nc y
fo r
AI Actors
A I A c to r
Ta sks:
M EA S U R E 4.2:
AI Actors
S u g gested
G A I
R is k s
at a re g u la r c a d
e n c e
t o
map an d me as ure
inc l udi ng te st s to a d
dre s s
d e c e ive o r m a n
ip u late
prove na nc e te c h niq ue s
o r o t h e r m is u
s e s
S e c u rit y
p e rfo rm a n c e
in re a l
S e c u rit y
I m ple m e nt
syste m s
re s u lt s o f
st ru c t u re d
pu bl ic
go fl
A I
Ac to r Ta sks:
AI D ep loy m ent, D om a in
Exp er ts, En d
M A N A G E 1.3:
S u g gested
G A I
R is k s
D oc um e nt tra de
t o l e ra n ce
C o n s id e r
a pproa c he s for
leverag i ng
sta ge d r e l e a s e
a pproa c h.
C o n s id e r
ro bust
e s s a nd
p la n s
re d
A I A c to r Ta s k s
:
A I
D evelo p
m e nt ,
M A N A G E 2.2:
S u g gested
G A I
R is k s
p re
g uide li ne s
a nd pr inc iple s
te s t
ge ne rate d c onte nt a ga inst
t he s e g uide line s .
A bus ive C onte nt
D a nge rous
D oc um e nt tra in ing da ta
s ourc e s to t ra c e t
he o rig i n a nd prove na nc
e o f A I
generated content .
te c hni que s s uc h a s
re
bia s e s in t he ge ne rate
d conte nt .
re late d o r N C I I c
o n te n t
A bus ive C onte nt
D a nge rous
AI Actors
generated co ntent .
Us e real
g e n e rat e d d at a
.
in put a b out A I
c om m unit y a nd s o c
ie ta l va lues .
H a rm ful
C ons ide r
a nd ot he r pr iva c y
a ppl ic a ble
D ata P riva c y
n te lle c t u a l
ro p e rt y ;
A I A c to r Ta s k s
:
M AN AG E
S u g gested
G A I
R is k s
p o l ic ie s a n d p
ro c e d u res for dete c
va lue c ha in
re c ove r y pla ns a re update
d fo r a n d i nc l ude
ne c e s s a r y de ta
ils to c om m unic a te
Po i n t s
A I A c to r Ta s k s
:
M A N A G E 2.4:
S u g gested
G A I
R is k s
op e n
s o u rc e
i nc lu di ng r
manage ment
A I A c to r Ta s k s
:
M A N A G E 3.1:
doc um e nte d.
S u g gested
G A I
R is k s
t uning
re t rie va l
pa rt y d ata s e t s a
nd ot he r
t une d t hird
p a rt y
h ird
p a rt y m o d e ls
t uni ng third
I n te l le c t u a l
P ro p e rty
Te s t
va lu e c h a in
D ata P riva c y
t un ing
o r re t rie va l
a n d fo r a ny t h ird
m ple m e nt re a s ona ble
lic e n s e d
c onte nt
o r t ra d e s e c re
t m ate ria l
I n te lle c t u a l P
ro p e rty
t h ird
p a rt y m o d e ls .
S e c u rit y
A I A c to r Ta s k s
:
M A N A G E 3.2:
P re
m a inte na nce .
S u g gested
G A I
R is k s
t une d
re t rie va l
o r o t h e r sys te m
upda te s
D ata P riva c y
t uning
o r re t rie va l
a ug m e nte d
proc e s s e s a pplie d.
I n te lle c t u a l
P ro p e rt y
i n to s y s te m
upda te s .
D a nge rous
C onte nt
i nc l udi ng
fo r
D a nge rous
A bus ive C onte nt
I mple me nt re al
sta nda rds a nd t rig ge r a
le rts fo r hum a n
t h ird
p a rt y p re
t ra ine d mode ls .
pe rform a nce
m e t ric s a n d d e
c o m m is s ion or re t
ra in p re
t ra ine d mode ls t hat pe rform
out s ide o f
A I A c to r Ta s k s
:
M A N A G E 4.1:
P o st
input from us e rs a nd ot he r
AI Actors
re le va nt
manage ment .
S u g gested
G A I
R is k s
te c h n o lo g ie s in
proc e dure s
for p ost
f or
S e c u rit y
AI Actors
A I system t o e nha nc e t
a c c ounta bilit y.
ra nspa re nc y a nd
H a rm ful
c o n te n t o rig in s
.
AI Actors
in c lu d in g
c onte nt
prove na nc e
A I A c to r Ta s k s
:
M o n ito r in g
M A N A G E 4.2:
AI Actors
S u g gested
G A I
R is k s
C onduc t reg ula r
m o n ito ri n g
pre ve nt fut ure oc c urre nc e
s . C ond uc t pos t
re le va n t
AI Actors
me as ure s.
D a nge rous
C onte nt
or ot he r m e t ho ds
to repres ent
n on
M on itor in g , T EV V
M A N A G E 4.3:
I n c id e n t s a n
AI Actors
d e rro rs a re co mm un ic
a te d to re le va n t
S u g gested
G A I
R is k s
le ga l a n d re g u l
ato r y b o d ies .
S e c u rit y
I n te g rit y
D ata P riva c y
A I A c to r Ta s k s
:
M o n ito r in
Appendi x A.
are relevant
for volun
tary use
highlights the most relevant topics derived from the
Governa nce
Ove r vie w
t to revis
the AI value chain.
Governa nce
than non
AI Actors
AI technologycan produce varied outputs
interfaces . This leads to a broader s et of
AI Actors
lans and
Acces s ibility and reas onable
m anagementcontrols
Com m ercial us e
Data provenance
Decommissioning
Dis couraging anonymouus e
Impact assessments
Incident res pons e
bas ed controls
labeling tools and techniques
interdis ciplinary teams
T hi rd
s ource or proprietar third
including but not limited to
re gardl
addres s ing third
for trans parency
s ource
Ove rview
Πa
nd documented
in early stages of the AI
AI Actors
recom m ended fipre
deploymenTest Approaches
Currently available pre
precis ely
St ruct ured Publ i c Feedba ck
but are not lim ited to:
s ens e of AI
AI Red
com m onl
y us ed in early stages of AI or product
development.
AI Red
AI red
conductedin a controlled
behavior or
outcom es of
stress test safeguardsfl
teaming in pre
The quality of AI red
teaming outputs
its elf. Demographi
and
divers e AI red
teams s hould demonstradomain
teaming res ults
team ing
. Thes e individualsmay
have been
teamers could modify or verify the
s pecialist or non
s pecialist
team s .
led red
human red
teamers alone
led AI
Cont ent Provena nce
Ove rview
generated content from AI
acces s
digital content trans parency
and history
of conte nt
to as s ist in
m etad
intellectualproperty rights
conte nt
m etadata
among others
ProvenancData Tracking Approaches
of data
overt and covert
AI A ctors
control ove r the various trade
robustnes s
an AI actor may inadvertently
s uch as
and corres ponding detectors
m etadata
recording techniquesand metadata
tools and repos itories
to trace content
Enha nci ng Cont ent Provena nce t hrou gh
St ruct ured Publ i c Feedba ck
approachedes cribed in
to capture
as through AI red
and post
models and
from outputs
and digital
trans parency approaches
are deployed
to gain ins ights about
on techniques
on us ers and
Inci dent Di s cl os ure
Ove rview
AI incidents can be
as an
injury or
to the health of
a pers on
or group
of people
and intellectualproperty rights
or the environm ent.
AI incidents can
State of AI Incident Tracking and Disclosure
available databas es
am ount of m edia cove rage
AI A ctors
tracing impacts to their s ource
m anagementacros s the AI ecosystem.
AI Actors
AI Actors
regardles s of role
party inputs and
AI Actors
inputs and content delivered through thes e
acces s control.
AI Actors
AI Actors
res ponding to and
managing AI incidents .
Appendi x B.
AI Incide nt Database .
s afe ty/
Em brace The Red
Deploym ent.
Center for Security and Em erging Technology
s urpris ing
m odels
Canadian Centre for Cyber Security
IT SAP.00.041
its ap00041
ICLR 2023
bus iness
chines [Link]
Term Impacts of AI in Cybers ecurity.
cybers ecurity/
Busines s School
vis ion
system s
. et al
perceived controllability.
on Algorithm Aversion.
ECIS 2020
cons ens ual
cons umer
analys is
illum inate s
re vealing
pois oning/
Bloom berg
m anagem ent
s ec
, Chapter 6
s ec
Standards and Technology
s ec
Glossary of Term s
4 System Card
Partners hip on AI
Disclosu re
m edia
m ethods
dis clos ure/
com pres s ion.
and biological des ign tools .
White Hous e
The White Hous e
s afe
s ecure
developm ent
us e
The White Hous e
Developm ent.
Stanford Cyber
im age
m odels
abus e
Nature Machine
Council of Europe.
dis order
interdis ciplinary
The Verge
im ages
pos es
analys es .
Bloom berg
disadvantalanguages .
decis ion
avers ion
pers uas ive
conte nt