Introduction 1

Generative AI Overview 2
What is Generative AI? 2
How is Generative AI Different from Traditional AI? 2
Generative AI in Procurement 3
Overall Benefits 3
Types of use Cases / Examples 3

Generative AI Architecture 5
Overview 5
Users 6
Applications 6
Orchestration 6
Information 6
LLMs 7

Success Criteria 8
Maximizing Adoption 8
Communication 9
User Access 10
Trust 11
Optimizing Output Quality 14
Information 14
Orchestration 15
Large Language Models (LLMs) 16
Ensuring Agility 18
LLMs 18
Use Case Flexibility 19
Mitigating Security & Compliance Risk 18

Conclusion 20

Summary Checklist 21

Glossary of Terms 22
INTRODUCTION
The tremendous excitement surrounding The questions we hear over and over from
Generative AI (Gen AI) is permeating both our procurement leaders involve how to get started.
personal lives and nearly every business function, What do I need to consider when purchasing and
including procurement. As is often the case with deploying technology? What use cases really
groundbreaking innovations, the initial excitement work? How do I avoid making a mistake by rushing
surrounding them has yet to translate into a in carelessly? With so much hype and vendors
proportional business impact. However, unlike promising different approaches and benefits,
some past hyped technologies in procurement getting started can seem daunting.
(i.e. blockchain, RPA…), we strongly believe the
reality will match, possibly even exceed, the hype.
And while it will take some time to realize the full The purpose of this guide is to address these
potential benefits, real value is possible and being questions, provide you with an overview of
achieved today. There may not yet be truly mature the technology, explain how it can help safely
procurement organizations when it comes to supercharge (or at least drive some tangible benefits
Generative AI, but many have rolled it out to some for) your team, and offer strategic recommendations
level. for planning a roadmap for Generative AI in
procurement.

1
GENERATIVE AI OVERVIEW
WHAT IS GENERATIVE AI? Generative AI, on the other hand, generates outputs
that are not explicitly programmed, but rather
Artificial Intelligence (AI) involves computer systems
produced based on patterns learned from training
that mimic human cognitive functions such as
data. The models are creating something new.
learning, problem-solving, and pattern recognition,
Generative AI is also better suited for non-numerical
enabling them to perform tasks that normally require
output such as images, text, sounds (ex. voice or
human intelligence.
music), etc. This means the range of potential use
Generative AI refers to a class of AI systems cases is far broader for a model.
designed to generate content, often in the form of
text, images, music, or other media, that is original It is this distinction that greatly widens the potential
and not directly copied from existing data. These use cases, but also what contributes to a common
systems are capable of creating new content based risk. In creating new content, Generative AI can
on patterns and examples learned from training also make up things that do not exist and provide a
data. For example, Generative AI systems can write convincing sounding but incorrect response. This
the lyrics to a new song on a topic you decide is what is commonly referred to as hallucinations.
considering any other criteria you specify (length, There are ways to control this risk, and improving
topic, style of music, tone, etc.). models are likely to further decrease hallucinations,
but it is something to watch for.

HOW IS GENERATIVE AI DIFFERENT Generative AI is actually not new. Today’s models

FROM TRADITIONAL AI? draw on research and computational advances
dating more than 50 years. The reason for the recent
A common question is how this differs from past
surge is that the amount of data available and used
versions of AI (a.k.a. traditional AI). It is important
to train such models, and the complexity and quality
to understand the difference, as that is what drives
of output, has skyrocketed recently.
the huge potential behind today’s excitement. While
there are several differences, the critical area to Traditional AI Generative AI
understand regards output.

Traditional AI focuses on analyzing historical data Predictions based on

Creates new content
existing data
and making predictions based on that data, while
Generative AI allows computers to produce brand-
new outputs. Traditional AI’s outputs are typically Best at non-numerical
Best at numerical output output (text, images,
predetermined based on input data or defined rules
sound…)
or algorithms. As such, they tend to be designed for
specific use cases and more applicable to numbers
Specialized use cases Broad use cases
(though not exclusively so).

2
GENERATIVE AI IN PROCUREMENT
OVERALL BENEFITS value given procurement’s growing list of objectives
and growing set of information needed to achieve
There is tremendous excitement about the potential
those objectives.
of AI in procurement, with lofty expectations. A
June 2024 study by Ardent Partners of nearly 400
procurement leaders found 62% believing the
TYPES OF USE CASES / EXAMPLES
impact of AI on procurement in the next 2-3 years
will be Transformational or Significant The specific use cases that Generative AI can
be used for seem limited only by your business
The specific benefits and overall value realized is
processes and imagination. Attempting to list them
highly correlated to the level of usage and the types
all would not only be fruitless, but make this one
of use cases, so giving a specific value is very difficult.
excessively long and quickly obsolete document.
Furthermore, as the capabilities improve and use
More useful is to understand the types of use cases
cases expand, so will the ROI. The most notable
Generative AI is appropriate for, with some real
benefits being realized today can be grouped into
examples for each. You can then look at your own
two core categories:
processes, assess which are most time-consuming
G
 reater Efficiency. The greatest benefits being or flawed, and look for capabilities to automate
achieved today come from the efficiencies obtained them. The following categories lend themselves
from streamlining and automating processes. This very well to Generative AI and are therefore a good
greater automation can free employee time for way to think about where the technology can create
more strategic activities and enable more work value in your procurement organization.
to be done without adding additional headcount.  ontent Creation. When you look at the typical
C
And yes there is the potential for headcount procurement professional’s day, much of it is
reductions. Generative AI can automate a wide often consumed by creating or editing content,
range of processes, with various examples in the whether RFxs, questionnaires or other forms.
next section. Given Generative AI’s particular strength creating
new, non-numerical content, this is the most
Improved Decision-making. Generative AI is very
natural category of use cases that can benefit
good at finding information and increasingly
you. If creating some type of content consumes
good at analyzing it to make recommendations.
a meaningful amount of your procurement
Decision-makers can more quickly access relevant
department’s time, consider automating it with
information, some of which they otherwise may not
Generative AI.
have found. This allows them to make faster, more
informed decisions which can be of tremendous

3
Examples that are actually available today include:  esearch/Analysis. The amount of information
R
> Contract clause drafting. Generative AI can needed by procurement to make informed
draft or modify contract clauses based on decisions continues to grow, as do the sources. As
instructions and the nature of the contract. the pace of business and the level of disruptions
> Mass communication assistant. Generative AI continues to grow, quickly accessing the right
can draft emails for review and then send to information can be a tremendous challenge.
relevant suppliers. For example, if you require Generative AI’s ability to quickly search and digest
new information from existing suppliers (ex. mass volumes of sources, whether internal or
Carbon emissions) to meet a new company external (on the web), makes it particularly valuable
policy or industry regulation, it can draft the here. This is an area where the benefits can extend
message. beyond efficiency to better performance against
objectives, whether that involves cost reduction,
D
 ocument Analysis. Over time a huge volume
risk mitigation, sustainability or other priorities.
of documents are accumulated through the
Some real examples available include:
procurement process, such as contracts, supplier
performance evaluations, invoices, and many > Category market research. Generative AI can
others. Finding necessary information, especially search the web and create a detailed category
when spread across more than one document, can intelligence report addressing price trends,
take many hours. It is also frustrating work that adds new or pending regulations to prepare for,
and risk considerations, and even provide
no value until the details are obtained. Generative
recommended actions.
AI use cases available today include:
> Supplier discovery. A common challenge
> Creating supplier performance improvement is identifying new suppliers to consider,
plans. Building a solid plan can take many hours especially during a sourcing event. Generative
of work, reviewing internal feedback forms AI can research suppliers based on your criteria
and performance results, assessing focus (products/services, size, location, credit rating,
areas, and then properly crafting the language. etc.) and suggest new suppliers not already in
Generative AI can quickly scour details such as your database. Unlike network-based supplier
supplier questionnaire responses, draft a plan discovery offered by some vendors, Generative
for review and send to suppliers after validated AI can search the whole web, match against
/ edited. As use cases evolve, they will be able your own suppliers to omit existing ones, and
to analyze multiple sources of information filter based on much broader criteria, delivering
when drafting a plan. Technically, this use far superior recommendations.
case bridges content creation and document
analysis.
> Contract analysis. Reading through contracts
can consume a large amount of time, especially
if in a different language. Generative AI can
provide a summary (in any desired language) of
key terms of a specific contract.

4
GENERATIVE AI ARCHITECTURE
OVERVIEW
While we often think purely of the Large Language a properly structured architecture offers a range of
Models (LLMs) driving Generative AI, implementing benefits which will be discussed in this guide, greatly
it in procurement, or any function for that matter, reduces risk, and is therefore highly recommended.
really involves multiple elements. Each should be The following diagram illustrates the 5 key elements
considered when building a strategic roadmap to of a proper Generative AI architecture:
avoid making suboptimal decisions that negatively
impact adoption, quality, agility or security. While
you could technically allow users to simply access a
LLM such as ChatGPT directly, doing so through

5
USERS necessary information, routes the final structured
prompt to the correct LLM(s), checks the response
The most obvious element of the architecture, and returns it to the user.
users are the employees that have access and
permission to use Generative AI capabilities in your Lengthy prompts may need to be broken into
organization. Other parties, such as suppliers or separate ones before being sent to one or more
contractors, may also be users if you decide to give LLMs. Orchestration will assess what information
them access. How these parties access Generative needs to be provided to the LLM(s) besides the user
AI and which use cases they can use it for determine input and will find that information (i.e. Requiring
the potential business impact that can be achieved, a web search). Various types of information can
and also influence the risks involved. be useful, or even essential, to Generative AI in
procurement.

APPLICATIONS If more than one LLM may be used, orchestration

will determine the optimal one(s) and route the
Applications are the software products accessed query accordingly. It can then check the response,
by users where Generative AI capabilities are performing quality control checks to minimize the
made available. It is where users log in to conduct risk of hallucinations or other erroneous responses.
their work activities. For procurement, these are
typically products that address the Source-to-Pay
(S2P) process, either individual products such as INFORMATION
Sourcing, Contract Management or eProcurement,
For many Generative AI use cases, the models
or S2P suites or platforms. cannot themselves provide an effective, accurate
response. They need to be fed or access various
ORCHESTRATION types of information. Several types of information
may be needed in procurement.
While LLMs have garnered the bulk of attention,
another important component of any organizational Internal S2P Data. Data captured within your S2P
Generative AI architecture is orchestration. It is not activities, such as who your suppliers are, order
actually essential - you could technically have a details, etc. are essential for many use cases.
simple API that allows users to directly access and
Internal Documents. Documents related to that
prompt ChatGPT or another LLM. Yet that doesn’t
activity, such as contracts, questionnaires,
mean it should be omitted.
certifications, etc. may also be necessary.
Orchestration serves as the glue between the
 he Internet / world wide web. There is a wide range
T
various aspects of Generative AI. It is increasingly
of information available on the internet which can be
embedded in S2P technology that offers Generative
valuable or essential to a particular use case, such
AI capabilities. Effective orchestration technology
intelligently processes prompts or requests, pulls all as news on disruptions, supplier profile information
and much more.

6
Depending on the specific use case, you may need  losed source: If the average person is asked to
C
information from one, two or all of these source name a couple of LLMs they are familiar with, odds
types to generate an effective response. For are they would name closed source models. These
example, a use case that summarizes the key terms are proprietary LLMs, such as OpenAI’s Chat GPT
in a contract needs to only access the relevant models, Microsoft Azure’s Cognitive Services,
contract. But one that researches and recommends Google’s Gemini or Anthropic’s Claude. They can
new suppliers to invite to a sourcing event must
be thought of as more of a black box, with users
access the internet to identify and assess potential
unable to audit behavior. This can be an issue in
suppliers and your S2P data to filter out suppliers
highly regulated industries or for more sensitive
you already work with.
use cases. Closed-source LLMs typically come
as-is, with limited scope for modifications to suit
LLMS specific needs or preferences.
Last but certainly not least, you must consider  roprietary: While technically the same as closed
P
LLMs. LLMs have been the focus of discussions source LLMs, I believe it is worth calling out a
related to Generative AI, and for good reason. It is subset based on who develops them. Today’s
the development of and advances in these models closed source LLMs have primarily been developed
that have unlocked the value potential of Generative for commercial use by, or in partnership with,
AI, and the current excitement. They are also what technology companies (Microsoft, Google, etc.). In
individuals have directly accessed, most commonly
the future, we expect more large organizations to
via ChatGPT, and seen the impressive responses.
invest in building their own LLMs for their own use.
If orchestration is the glue in your Generative AI
These may be fully developed in house or based on
architecture, LLMs are the engine.
open source LLMs that have been tailored for an
There are various categories of models: organization’s needs. While costly to build and train,
they can be tailored to an organization’s needs and
O
 pen source: With an open-source LLM, any person
security and can be more tightly controlled.
or business can use it for their means without
having to pay licensing fees. This includes deploying The different models and many LLM examples within
the LLM to their own infrastructure and fine- each are an important element to understand when
tuning it to fit their own needs. You can tweak the we discuss agility later on.
model to better align with your needs, objectives,
or regulatory requirements. The nature of open
source LLMs also leads to greater transparency in
how Generative AI works to produce a response.
Meta’s Llama models, Stability AI’s models, and
Mistral’s models are examples available today.

7
SUCCESS CRITERIA
When considering each element of your Generative
AI architecture, it is useful to do so in context of the MAXIMIZING ADOPTION
key criteria that will ultimately determine whether
you realize the expected and maximum value. In Value realization from any technology is highly
particular, adoption, output quality, agility and dependent upon user adoption. Regardless of how
security should be effectively planned for to greatly well a technology works, you will always leave value
increase the odds of success. The following sections on the table if users do not broadly adopt it. When it
cover each of these success criteria, providing an comes to Generative AI, there are particular risks to
overview, goals, factors to consider in your plans, consider. You should ensure you have a plan for the
and recommendations. following aspects to ensure rapid, broad adoption.

Note that the steps and recommendations listed

here are intended for planning your overall strategy
and making decisions on specific aspects thereof.
They are not intended as steps required for the
rollout of each specific use case, which would
generally be overkill and slow value realization. The
point is to develop a clear strategy and then move
quickly, piloting and deploying use cases rapidly.

8
 Communication
Overview: While effective communication is
important to the roll-out of any new technology,
it is especially so when it comes to Generative AI.
Generative AI raises both concerns and excitement,
and how you communicate greatly influences which
materializes among employees. People are naturally
resistant to changing how they work, plus many are
concerned that Generative AI is going to eliminate
their position. A Dec 2023 Forrester survey found
36% of respondents fear losing their jobs to AI within
10 years. That said, the excitement and momentum
behind AI should ease, and hopefully overshadow,
any concerns or pushback.

Goal:
Have your employees eager to start using AI

Recommendations:

1. G
 ather user feedback during planning. Feedback point out that employees will be able to quickly
on use cases, timing of broader communication see a summary of contract details rather than
and roll-out, etc. This is typically best done by having to read through full contracts (if that is a
working with representative users from distinct use case being deployed).
teams. These users can be champions later,
c If reducing headcount / eliminating roles is not
especially if their feedback contributed to the
part of the business case, be explicit about this
ultimate solution deployed. Their feedback is
to allay anxiety and hopefully cut off rumors.
also useful for tailoring broad communication
If some redundancies are expected, this is a
messages to better resonate with employees.
trickier situation where the message clearly
2. Communicate well in advance of roll-out. must be carefully assessed together with HR
based on the expected scope.
a C
 learly articulate the targeted business
objectives and the benefits for employees. 3. Provide a channel for questions to address
concerns or issues, both in advance and after
b P
 rovide specific examples of benefits. For
roll-out.
example, rather than simply saying employee
productivity will be improved (sounds like fluff
and may be interpreted as pending layoffs),

9
 User Access
A key factor to how well Generative AI is adopted is users describe that they would like the contract
how users access the new capabilities. The simpler summarized. Users may not and need not even know
and more natural it is, the greater adoption will be. AI is being used in such UI-based access.
This involves where users access Generative AI,
whether in the systems they already are using or Conversational GenAI Access
externally through different systems.
IVA

It also involves how users initiate Generative AI Hi there - Feel free to ask me anything
and i will try to assist you.
Can you find suppliers of plexiglas
windows in the Bay Area, Ca?

capabilities. Specifically, Generative AI can be Category Monitor

IVA

Certainly! Here are a few suppliers of

initiated by clicking action buttons embedded in the
IVA
plexiglass windows in the Bay Area, CA:

1. Roxy Glass
UI. Alternately, access can be conversational, most 2. Acme Plastics
3. Livermore Glass Company

commonly via chatbots. These are just a few options to get you
started. it’s always a good idea to contact
Hello, I can search the internet to build the suppliers directly to inquire about the
market intelligence for a particular specific products, pricing, and availability.
commodity, including price fluctuations,

UI-based AI Access
supply dirsruption, regulatory changes, I can also offer a few related skills that
compliance risk. Just select a commodity might be helpful:
and let me do the rest!

Type a few letters to select a

commodity Can you compare those suppliers based
on reviews available from the internet?
MA_MI216 - Aluminium
IVA IVA

Commodity: Aluminium ...

Market intelligence generated on 10/22/2023

Price Trends
Aluminium prices hit their highest in
nearly four months on February 8, 2022,
boosted by supply concerns heightened
by coronavirus curbs...

For use cases where user input is needed to

UI-based access is typically fixed to a specific use
achieve the optimal response, conversational
case. Use cases with fixed input and outputs where
access is typically a better fit. In such cases, most
instructions are predetermined and can pull data
users will understand that they are leveraging AI.
from the application, the internet and communicate
Conversational approaches can still be limited to
with the LLM lend themselves well to UI-based
specific use cases or they can be open ended,
access. It is the simplest approach for a user and
supporting any question a user wishes to pose to
avoids output quality potentially being adversely
the LLM (just like when you directly access ChatGPT
impacted by how the user prompts the system. In
and simply type a request). Allowing more or
such cases it may not even be apparent to users
open-ended use cases naturally supports broader
that Generative AI is being leveraged.
adoption, but it can negatively impact trust and
An example could be a button labeled “Summarize thereby turn users against it. This last aspect is
Key Terms” or similar that appears next to specific discussed in the next section.
contracts. Clicking it prompts a model to process
the contract and generate a concise summary of key
terms for the user. This is a more efficient process
and much simpler user experience than having

10
 Goal: Trust
Make Generative AI a seamless part of
how users work Having users keen to try Generative AI and able
to easily access it can drive early adoption, but
Recommendations: if the output is of poor quality, adoption will
quickly deteriorate and be hard to recover. Errors
1. P
 rovide direct access to Generative AI in the core
or hallucinations, where LLMs fabricate realistic
S2P application(s). Requiring users to access
sounding but inaccurate responses, are a real
different systems to leverage Generative AI will
risk recognized by many people. According to the
curtail adoption significantly. The experience is far
2023 McKinsey Global Survey on AI, the top risk
superior with access available where they work,
identified by 913 respondents whose organizations
without new logins, a different user interface or
have adopted Generative AI was inaccurate results,
separate window to open.
specified by 56% of respondents.
a B
 est: Systems with embedded Generative AI Taking steps to ensure your systems provide high
offer the simplest approach quality output is critical to building confidence, and
b A
 lternate: Separate Generative AI solutions the focus of the next section. But several factors
integrated into core S2P applications with single must also be considered on the user side. First, keep
sign on (SSO) in mind that how users prompt Generative AI when
accessed conversationally greatly impacts output
c W
 orst: Distinct Generative AI tools accessed quality. Particularly in these early days, few users
outside of S2P. Most disruptive if distinct tools will be prompt experts. While you can take steps
for different use cases to improve their skills, there will always be variation
and gaps in prompt engineering skills, particularly
2. Provide chat and UI (buttons) access. Each type
among less frequent and less tech savvy users.
of access is optimal for specific use cases so
selecting applications that offer both options, Second, the risk involved will vary greatly by use
based on what is ideal for each use case, delivers case, as will the value delivered. The broader the
the best experience and output. set of use cases available, the greater variety in
output quality. At one extreme, with only a few very
specific use cases deployed, organizations can
better assess and control quality. Confidence will be
higher but less potential value can be achieved. At
the other extreme, with users given the ability to ask
anything of Generative AI, quality control is limited
but there is far greater potential for value realization.
Organizations must assess and decide on the right
balance.

11
 Goal:
Build employee confidence in the responses
and actions of Generative AI

Recommendations:

1. P
 lan a structured user training program that Such guided experiences mitigate the risk of user
includes: caused output deficiencies.

a P
 rompt engineering training. Be sure to explain 3. Look for systems that specify external sources,
to users how to most effectively prompt ideally with links, when leveraged to provide
Generative AI. Besides general best practices, responses. Being able to see and review the
be sure to include specific examples for the source increases user confidence, and also
use cases rolled out at your organization. You streamlines the identification of incorrect
may have internal resources, most likely within responses.
IT, qualified to conduct this training or feel you
4. Plan a use case level roll-out. Assessing use
must leverage external experts.
cases, piloting the most promising ones and
b E
 xpectation setting. Users should understand only rolling out those that prove themselves may
that AI generated content should be considered be a slower approach but it will support growing
a good draft. But they should not expect confidence and enthusiasm for Generative AI.
perfection and should always review output Certain use cases may not offer significant value
before sending or otherwise using. Having a solid for a particular organization. Pilots can reveal
draft to start work can save the bulk of effort but shortcomings in the models or your own data
the expectation is not 100% automation. that need to be addressed. In the below real
use case of a global manufacturer, 9 use cases
c C
 ontext understanding. Users should
were assessed with 6 deemed worthy of a pilot
understand that output quality will vary more
due to the potential value. Of those, 3 were
when submitting freeform conversational
found to be ready for immediate deployment
questions.
whereas 3 needed further work. In one of these
2. Leverage intelligent applications to mitigate the cases, the organization’s data was a constraint
need for expert prompting. The more open-ended as many contracts were uploaded as images
prompts are, the greater the expertise required and not yet been digitized, rendering the
of users and the more likely output quality will contract summarization impossible (until further
suffer. Look for applications that can guide users embedding of OCR capabilities addressed this
as much as possible. For example, intelligent issue). In the category intelligence report use
systems will first identify the use case (or even case, the results did not use the specific source
present a list of options when limited cases of benchmark data deemed acceptable by
are deployed), and then ask a series of precise category managers. Hence, the model needed
questions for the required prompt elements, further refinement to access specific sources
ensuring users provide all necessary information. before it would deliver value at that organization.

12
 USE CASE ASSESSMENT APPROACH

GLOBAL INDUSTRIAL MANUFACTURER CASE STUDY

GEN AI USE CASE STRATEGY

13
 is a common challenge for organizations and
OPTIMIZING OUTPUT QUALITY limits the ability of LLMs to provide quality output
in many use cases.
Generative AI provides no value if the responses
provided by models are not of high quality. Hence, any
effective roadmap must consider how to optimize Goal:
Ensure all relevant information is of sufficient
output quality. Several elements of the architecture
quality, and accessible for queries
have a major influence on quality and each must be
addressed to avoid results being degraded by the
Recommendations:
lowest common denominator.
Establish a single source of truth for S2P data. The
Information simplest way to do so is with a single, core platform
for S2P processes. If you opt for this route, be sure
Data is the lifeblood of AI, with huge volumes used that data in that platform is in fact unified. Most S2P
to train models so they can effectively respond platforms and suites were either built in silos or via
to prompts. It is also critical to input the right acquisition and in reality have separate data tables
information to a LLM to obtain an effective response. for different solutions. They may have added a data
Data issues were identified as the single biggest layer on top but those separate records must be
obstacle to AI projects in a 2024 Procurement matched to varying degrees of accuracy and often
Leaders study. There are two critical elements that records added or deleted in one process are not
must be considered: modified across all tables. If that is the case, or if you
opt for a point solution approach, you should work to
1. D
 ata Quality. Accessing particular information
unify the data in a separate data layer / lake. Key is
is only useful if that information is readable,
that Generative AI is able to access that centralized
sufficiently complete and accurate. In the supplier
data.
discovery example from above, if your supplier
records have misspelled the supplier name or
lack details on the supplier such as address or
identifiers such as tax IDs or DUNS numbers,
Generative AI is more likely to struggle to know
whether a potential supplier it has identified is the
same as who you work with.

2. Data Access. Data access is greatly complicated by

having data dispersed across multiple systems. If
data is dispersed, systems will struggle to access
it and match relevant information from various
sources. It also tends to exacerbate quality
problems with duplicate records and partial
information, often entered in different ways. This

14
 Orchestration
Orchestration’s core value is in driving output quality
and that is why it is such an important part of the
overall architecture. It processes prompts, pulls
relevant information, feeds it to the LLM and checks
responses before sending them back to the user.
Each of these steps has a significant impact on
quality.

Goal:
Ensure prompts are optimally processed & routed,
access all relevant information & that responses
are checked for accuracy

Recommendation:

Carefully assess orchestration capabilities of any  rovision of sources. If responses are based on
P
technology being evaluated. Don’t overlook this specific 3rd party sources, having those sources
element as differences in orchestration can greatly listed in responses, with links to research and verify,
impact the output quality from one solution to both allows users to check responses and better
another even when both leverage the same LLM. Be assess whether the source is reliable
sure to look at the capabilities to process prompts,
access information and perform quality checks on F lexibility on confidence requirements. There is a
responses. Many teams will lack the expertise to balance between the depth and level of responses
effectively assess orchestration so may want to you would like from LLMs and the likelihood of a
leverage 3rd party experts. As a minimum, ensure mistake or hallucination. In certain use cases, you
orchestration is included and supports: are likely to want to give the model more leeway
than in others. More advanced orchestration
A
 ccessing your internal S2P data, documents and engines allow you to set the confidence level at the
the internet for individual requests, as appropriate use case level
for the use case
P
 rompt optimization, breaking up large prompts
based on LLM capability and routing to the best
approved LLM for that request
Q
 uality control checks. For example, automatically
checking responses and asking the same question
in different ways, cross checking responses, to
try to filter out hallucinations before providing the
response to a user

15
 Large Language Models (LLMs)
The excitement about Generative AI is
fundamentally driven by the remarkable advances
in LLM capabilities. LLM processing of user prompts
processed by orchestration layers is where the
“rubber meets the road” when it comes to output
quality.

Given the rapid expansion of new LLMs and versions

of existing ones, it would not make sense to
recommend specific LLMs as they may no longer be
the optimal choice when you read this. Furthermore,
greater specialization will likely result in different
LLMs performing better at specific tasks but not
others. Lastly, specific company policies may
exclude certain LLMs or mandate others.

Goal:
The optimal, policy compliant LLM(s) is used for
each query

Recommendations:

A
 lign w/CIO on company policies. Your company
may have already defined policies on which specific
LLM(s) or at least which type of LLM can be used.
Ensure any architecture complies with any such
policy.
O
 ptimize. Look for solutions that are able to leverage
multiple LLMs, routing requests optimally based on
which LLM is best suited to a specific prompt and
are acceptable based on any company policies you
have.

16
ENSURING AGILITY LLMs
The LLM you are required to use or that works best
Given market uncertainty and regularly evolving
for any particular use case is very likely going to
business priorities and requirements, procurement
evolve. Company policies will be defined or refined,
must stay agile. Technology plays a key role but is
new LLMs are being developed and existing LLM
often an obstacle. In fact, a 2023 Forrester survey
relative strengths will shift as well. As a result, any
of over 400 procurement and supply chain leaders
architecture must provide flexibility on the LLM(s)
globally found that technology was one of the top
used at any point, for any use case.
3 obstacles limiting procurement agility, often being
too rigid to adjust to new ways of working. This is
Goal:
mainly driven by a rarely discussed downside of many
Ensure flexibility to change the LLM(s) used
cloud-based solutions. The multi-tenant approach
without having to replace your procurement
that predominates generally offers limited flexibility technology
when requirements don’t fit the pre-packaged
definitions. Recommendations:

When it comes to Generative AI, this is an even 1. C

 hoose LLM agnostic S2P technology. You don’t
greater concern due to the early stage and rapid want to have to change your S2P technology or
evolution of the technology. Simply waiting for a more complicate the user experience by accessing
mature stage in the technology lifecycle is a poor Generative AI outside the application due to
option as it will delay value realization for years and changes in the LLM(s) strategy. Hence, it is
set your organization back relative to competitors. essential that the S2P technology you leverage is
A smarter approach is to proceed thoughtfully, in a not tied to a specific LLM and has the flexibility to
manner that ensures you can benefit from advances leverage different LLM(s). That flexibility should
in Generative AI without having to add to or make be at the individual customer level since different
costly, time consuming changes to your overall companies will have different policies.
architecture. Two aspects in particular must be
2. Ensure orchestration is independent from
taken into account.
LLMs. If orchestration is embedded in your S2P
applications, the first recommendation addresses
this simultaneously. But if orchestration is
supported outside of the S2P applications, be
sure it too is independent of specific LLMs and
can adjust as per your requirements.

17
 Use Case Flexibility case without vendor dependencies. It should be
both possible and also practical in that you don’t
Use cases are being rapidly developed by vendors. require coding skills but can refine or create use
However, regardless of how robust your vendor’s pre- cases through the interface via configuration.
packaged use cases are, there will always be useful
options available from other vendors. Furthermore, 2. Define a process for defining / refining use cases.
your own staff is most knowledgeable about Having this flexibility can be extremely valuable
which of your processes would best benefit from but that doesn’t mean any user should be able
Generative AI. As adoption increases, ideas are sure to do so. You want to ensure proper vetting of
to blossom and it is unlikely any vendor will support the benefits and risks of new use cases. Defining
all your ideas “out of the box.” You want to be able to a clear approval process is therefore highly
deploy useful ideas in a reasonable amount of time recommended and those that actually create
and without having to license distinct technologies, or refine use cases should be limited to select
which would create integration complexity and/or administrators.
disrupt the user experience.

Therefore, it is imperative that the technology you

MITIGATING SECURITY & COMPLIANCE
select provides flexibility for you to define new
RISK
use cases, as well as refine existing ones for your
purposes. For example, a use case that generates a As beneficial as Generative AI will be within
category market intelligence report may need to be procurement, it does not come without risks. There
refined to leverage a specific external source given are multiple categories of risk that organizations
your policies or category manager preferences. should consider and the relative importance varies
If you are unable to refine the use case for your by department (ex. Ethical / bias risks are particularly
specific use, it won’t be valuable to you. important in a HR context). This brief summary is
not intended to provide a comprehensive overview
Procurement leaders seem to recognize this for organizations but call out specific aspects
requirement already. A recent survey of 110 senior procurement should focus on. Within procurement,
procurement leaders by WBR found that the the following are likely to be most relevant:
ability to tailor unique use cases based on company
needs was considered the #1 consideration  ata privacy and cybersecurity risk. Use of a 3rd
D
for successfully integrating Generative AI into party LLM could potentially expose sensitive
procurement. information about individuals (employees,
contractors, supplier contacts…) or your
Goal:
organization (contracts, new products…). If users
You are able to create and refine use cases on provide such detail in prompts, or information
your own
pulled from your internal documents and data
Recommendations: is stored by that LLM or used to train models,
it is conceivable that the information is hacked
1. E
 nsure the technology you select offers you or even exposed to external users of that LLM
the capability to fine tune or create a new use
in responses. There have been many cases of

18
 regulated data, intellectual property and user > Used to train models, which could result in it
passwords being leaked by users through careless being shared in responses
prompting. And LLM providers have had issues on  ssess the security conditions of LLM agreements.
A
their side as well, such as ChatGPT’s self reported It is not sufficient to simply look at which LLMs are
data breach, which exposed some customer data. being used. LLMs often have different security
C
 ompliance risk. Governments are increasingly conditions based on the agreement level. For
starting to regulate Generative AI, such as the example, ChatGPT enterprise does not store,
March 2024 EU AI Act. Expect more regulations to share or use any information from prompts to train
follow, in more countries. While the focus of current its models, but the free version does. Ensure you
regulations is not on procurement use cases, understand the agreement you or your application
that may change in the future and remember that provider have in place and make sure employees
employees can deviate from expected use cases, only access the approved LLM(s) in a way where
especially with conversational Generative AI tools. those apply (rather than from personal accounts).

In response to such concerns, organizations  rain employees on acceptable use of and how
T
are increasingly defining policies and limits on to access it properly on company systems.
Generative AI usage. In 2023, Samsung completely Every employee should understand the types of
banned usage of Generative AI after careless information they should not include in prompts.
employee leaks of sensitive information. But most That should be a prerequisite before any employees
organizations, even in highly regulated industries are given access.
such as finance, are likely to take less extreme
 onsider risk when assessing use cases. The type
C
positions given the benefits.
of information shared with LLMs will vary widely
Recommendations: The first step is clearly to check based on the use case. In some cases (i.e. market
with your CISO or CIO for any company policies on
research for a category), only external data is
Generative AI and ensure compliance with those. At
needed to receive an effective response. Risk is low
organizations with clear, comprehensive policies,
in such cases. In others, internal data (suppliers,
that is likely to suffice. For those lacking policies or
products…) must be shared. The threshold on
when they are very basic, consider the following
when planning your roadmap and roll-out to ensure where the cost/benefit balance needs to lie will
reasonable precautions: vary across organizations.

E
 nsure that with the systems used your user inputs
and information accessed by prompts are never:
> Shared with other organizations
> Stored outside your S2P applications. Data is
already captured and stored in your applications
– you want to avoid having it accessible to
hackers in additional systems, creating new
failure points

19
CONCLUSION
Generative AI has generated tremendous excitement Hopefully this guide has provided you with the
and anxiety. For procurement professionals, the fundamental understanding of Generative AI and
potential to improve efficiency and decision-making factors to consider so you can build your own
is significant. While there are legitimate concerns, strategic roadmap for Generative AI in procurement.
they should not hold procurement leaders back Naturally, we have only scratched the surface here.
at the large majority of organizations that have There is much more detail involved and, as always,
not and will not ban Generative AI usage. Doing so one must consider your own organization – your
would not only delay value realization, but also put policies, processes and team.
your organization at a disadvantage relative to
your competitors. Instead, a strategic approach
should be taken to minimize risk while maximizing To discuss in greater depth, or to learn how Ivalua
value. Such an approach should plan for the various can help you safely supercharge procurement with
elements of the overall architecture when selecting Generative AI, visit our website or contact us at
and deploying technology. sales@[Link].

20
SUMMARY CHECKLIST
The following is a summary checklist of success criteria and recommendations:

Success Criteria Recommendations

Maximizing Adoption  ommunicate well:
C
> Gather user feedback during planning
> Communicate well in advance of roll-out
> Provide a simple channel for user feedback

Simplify user access:

> Provide direct access to Generative AI in the core S2P application(s)
> Provide chat and UI level access based on the use case

Build trust:
> Plan a structured user training program
> Leverage intelligent applications to mitigate the need for expert prompting
> Look for systems that specify response sources
> Plan a use case level roll-out

Optimizing Output Quality Establish a single source of truth for your spend / supplier data
 arefully assess technology orchestration capabilities, considering key
C
factors such as:
> Prompt optimization
> Accessing various categories of information
> Specification of response sources
> Quality control checks
Select technology that can optimize across different/multiple LLMs

Ensuring Agility Choose LLM agnostic S2P technology

Ensure orchestration is LLM independent
 elect technology that enables you to refine or define new use cases without
S
software vendor dependencies

Mitigating Security & Compliance

Risk
Verify and comply with company AI policies
 nsure 3rd party LLMs do not share, store your information, nor use it to train
E
models
Train employees on Generative AI acceptable usage
Assess risk at a use case level
Assess the security policies within LLM agreements (own or 3rd party)

21
GLOSSARY OF TERMS
The following is a list of terms that are important to understand in the context of Generative AI :

Term Definition
Artificial Intelligence (AI) Computer systems able to perform tasks that normally require human
intelligence, such as visual perception, speech recognition, decision-making,
and translation between languages.

Generative AI (Gen AI) A type of AI that uses machine learning models to create new content in res-
ponse to prompts. This content can include images, text, videos, audio, code,
and simulations. models use neural networks to identify patterns in existing
data, such as documents and artifacts online, and then generate new content
based on what they’ve learned.

Machine Learning Computer systems that are able to learn and adapt without following explicit
instructions, by using algorithms and statistical models to analyze and draw
inferences from patterns in data
Neural Network A method in artificial intelligence that teaches computers to process data in a
way that is inspired by the human brain. It is a type of machine learning process,
called deep learning, that uses interconnected nodes or neurons in a layered
structure that resembles the human brain.

Deep Learning A type of machine learning based on artificial neural networks in which multiple
layers of processing are used to extract progressively higher level features from
data.

Large Language Model (LLM) A type of machine learning model leveraging deep learning that can perform a
variety of natural language processing (NLP) tasks, such as generating and
classifying text, answering questions in a conversational manner, and
translating text from one language to another.

The label “large” refers to the number of values (parameters) the language
model can change autonomously as it learns. Some of the most successful LLMs
have hundreds of billions of parameters.

AI Orchestration The process of managing the interaction, deployment, and integration of various
AI models, data sources and software applications. It involves coordinating mul-
tiple tasks and processes to work together seamlessly across different environ-
ments such as cloud platforms, data centers and software applications within a
workflow or system.
Key roles of the orchestration layers include:
 cting as an integration layer between LLMs, the enterprise data assets, and
A
applications
Retaining memory during user’s conversational sessions because foundation
models can be stateless
Linking multiple LLMs in a chain for more complex operations
Functioning as a user’s proxy, devising intricate strategies for executing
complicated tasks

22
Term Definition

Hallucinations Inaccurate, biased, or unintended results generated by LLMs. These errors can
occur for a number of reasons, including: Insufficient training data, incorrect
assumptions, biased data, and faulty recall

Prompt Natural language text describing the task that an AI system should perform

Prompt engineering The art of crafting input messages or queries (prompts), to effectively
communicate with Generative AI models

Open source LLM LLMs that are free and available for anyone to access, use for any purpose,
modify and distribute. The LLM code and underlying architecture is accessible
to the public, with developers and researchers free to use, improve or otherwise
modify the model

Closed source LLM (Proprietary LLMs that are owned by a company and can only be used by customers that
LLM) purchase a license. The license may restrict how the LLM can be used. The LLM
code and underlying architecture is not accessible to the public and may not be
modified

About Ivalua
Ivalua is a leading provider of cloud-based, AI-powered Spend Management software. Our unified
Source-to-Pay platform empowers businesses to effectively manage all categories of spend and
all suppliers, increasing profitability, improving sustainability, lowering risk and boosting employee
productivity. We are trusted by hundreds of the world’s most admired brands and recognized as a
leader by Gartner and other analysts. Learn more at [Link] Follow us on LinkedIn and X.

Contact: info@[Link] [Link]

P.
P. 35
P. 21
29
USA Canada France UK Germany Italy Sweden Singapore India Australia