Generative AI

A guide for building your own AI applications

Contents
Introduction 03
Intended audience
Executive summary

Chapter 1 
Comparing approaches: Generative AI-enabled software devel-
opment vs standard software development 05

Chapter 2 
Five common applications of building with generative AI 07

Chapter 3 
Select the right model for your use case 09

Chapter 4 
Building a generative AI development team 13
AI engineer
Data professional
Domain SME
Data scientist/machine learning (ML) professional

Chapter 5 
Azure AI Foundry: A comprehensive platform 17
Deploy generative AI responsibly with Azure AI Foundry
Integrated data security and privacy

Chapter 6 
Key insights and next steps 25
 Generative AI: A guide for building your own AI applications 3

Introduction
Executive Summary
Generative AI is transforming organisations and the people who drive them. It
allows software to be more intuitive and helpful, raising employee productivity
and user satisfaction. At the same time, generative AI poses a new set of
challenges and risks.

This e-book delves into the strategic considerations for those who want to build
their own AI applications using foundation and generative AI models, or those
that want to add generative AI to their existing applications. It covers details on
generative AI, offering guidance tailored for ITDMs on harnessing this technology
to fulfil specific business needs and achieve competitive advantages.

Intended audience
This e-book is specifically geared towards IT decision-makers (ITDMs), such as
Chief Technology Officers (CTOs) or Chief Information Officers (CIOs) at companies
of all sizes, including Independent Software Vendors (ISVs), who are interested in
building an AI application using a foundation model.

The approximate reading time for this e-book is 30 to 45 minutes.

 Generative AI: A guide for building your own AI applications 4

Key topics covered in this eBook

include: What’s the difference between
1. Understanding generative AI customised machine-learning
Differentiating generative AI from traditional (ML) models and generative
software approaches, highlighting the ability to AI models?
create dynamic and contextually relevant responses
beyond predetermined outputs. Custom ML models
Built for a specific purpose
2. Building a generative AI team
You’re probably the model builder
Detailing the roles within a generative AI
Useful to make predictions on future
development team to optimise the deployment
outcomes based on pre-existing data
processes.
(past outcomes)
3. Developing with responsible AI practices
Generative AI models
Learn about the tools and best practices that
Built as 'general-purpose'
mitigate risk and support AI safety, quality and
You’re probably not the model builder
compliance.
Useful when you want to create new
4. Using a comprehensive AI development platform content or data that resembles/mimics
Guidelines when selecting generative AI platforms patterns from pre-existing data
and tools that align with specific business objectives.
This includes learning about Azure AI Foundry,
an essential tool for developing, deploying and
managing generative AI applications. The platform
supports the entire AI lifecycle with advanced tools Microsoft provides a variety of courses
for model selection, data integration and enterprise- designed to help data and development
grade production at scale. professionals enhance their AI skills.

These can be found in our AI learning hub.

This eBook serves as an introductory guide to
navigating generative AI and facilitating informed
decisions that drive efficiency and innovation.

Learn the basics of generative AI with

Microsoft Azure AI Fundamentals: Generative AI.
 Generative AI: A guide for building your own AI applications 5

Comparing approaches: Generative

AI-enabled software development vs
standard software development
Integrating generative AI into applications, when compared to standard
software development, is different in three key ways.

1. Ownership and data control

Standard software development: Microsoft Azure Security
In standard software development, data control and Regulated industries still require risk
ownership are clearly defined. Developers maintain managers and regulators to demonstrate
complete control over their data, which remains that data remains private throughout the
within their own data estate. This ownership provides development process. This is why security
straightforward management and compliance with and compliance are so important when
data regulations, ensuring security and privacy without choosing a platform for the responsible
the need for third-party involvement. development and deployment of generative
AI applications at scale.
Generative AI development:
Alternatively, generative AI development involves With Microsoft Azure Security, gain multi-
less control over data, as it often requires sending layered security across physical data centres,
data outside one’s own data estate to interact with infrastructure and operations. Azure cloud
AI models. This could potentially raise security and is built with customised hardware with
regulatory concerns. Although the data still legally integrated security controls and firmware
belongs to the developer, the dependency on external components, as well as added protections
AI models and services managed by third parties against threats such as Distributed Denial-of-
introduces complexity in ensuring data privacy and Service (DDoS) attacks.
compliance.
Learn more about Azure Security.
 Generative AI: A guide for building your own AI applications 6

2. Development process and problem 3. Evaluation

solving Standard software development:
Standard software development: The developer uses unit tests to determine if the
Developers write explicit instructions to solve explicit application is returning the correct answer and
problems, leading to predictable and consistent working as intended.
outputs.
Generative AI development:
Generative AI development: Generative AI requires a deeper, continuous
In contrast, generative AI operates on probabilistic involvement of domain subject matter experts (SMEs)
principles, using patterns and contextual data to throughout the development process considering the
generate outputs that may vary with the same inputs. multiple answer outputs that need to be evaluated if
This uncertainty makes the development process more they are all correct answers.
complex and less predictable. It requires extensive
These experts not only contribute to defining the
testing and iteration to achieve a production-ready
scope and specifications of applications, but they
state. The variability in outputs often necessitates
also play a decisive role in continually assessing the
a significant involvement from domain-specific
AI-generated outputs for metrics such as accuracy
experts who assess the relevance and accuracy of the
and relevance. The iterative nature of this process,
responses from AI.
along with the need for frequent evaluations and
adjustments, underscores the importance of the
human-in-the-loop development process.

One way to evaluate applications is for

human experts to rate individual answers.
However, this is time-consuming, error-
prone and not feasible at scale or for the
ongoing monitoring of applications in
production.

To help expedite and remediate this

process, Microsoft researchers are
developing tools for automating
assessment by using models to evaluate the
output of other models.

Human experts remain necessary even

in this automated assessment to provide
some ground-truth answers, which can be a
significant investment in time.
 Generative AI: A guide for building your own AI applications 7

Five common applications

of building with generative AI
Generative AI-enabled applications can be developed from scratch, but often a faster way to gain generative
AI benefits is integrating it into existing applications. This can allow users to interact with applications in new,
more intuitive ways, contextualise assistance and provide more relevant information. Generative AI is, as a result,
improving applications across the value chain from customer contact to financial and operational analysis.
 Generative AI: A guide for building your own AI applications 8

Below are five types of applications that organisations are building with generative AI. They combine intuitive
interfaces, links to proprietary knowledge bases and enterprise systems:

Chat with your own data

Many organisations possess a wealth of proprietary knowledge in documents and websites. To
unlock this value, they are developing chat applications that allow employees and customers
to query this data in plain language and receive direct answers instead of just links to source
documents. Generative AI tools should be equipped with built-in responsible AI capabilities
so that responses are derived solely from internal data, safeguarding against off-brand
experiences and any exposure to malware through untrusted data sources. In doing so,
information remains secure and compliant with company standards.

Create personalised AI agents

Software developers use generative AI to integrate advanced helpers into their systems,
aimed at reducing the monotony of repetitive tasks and minimising disruptions that hinder
professional focus. These AI agents can offer enhanced in-context support, automatically
draft responses tailored to the requester’s identity and the context of their enquiry and
compile comprehensive end-of-day or weekly summaries detailing activities and statuses.
This automation aids professionals in maintaining productivity without sacrificing accuracy or
attention to detail.

Add hyper-personalisation
Tailor interactions to individual users at scale. Hyper-personalisation can range from
customising marketing messages and adjusting website interfaces in real-time based on user
actions, to personalising product recommendations at an individual level and enhancing
customer engagement and satisfaction.

Build customer service avatars

Generative AI and speech AI are allowing enterprises to create sophisticated, multilingual
brand ambassador avatars where they can engage with the public in the brand’s look and tone
via website, kiosk or smartphone. While early versions provided general information, avatars
are now increasingly able to interact with business systems to provide specific information,
make informed recommendations, check inventory and add items to an e-commerce cart,
making them an exciting new way to create high-value branded experiences.

Access insights
When important information is scattered across incompatible systems, analysts can spend
hours just assembling the data for routine reports, such as end-of-quarter performance and
prospect profiles. With new agent architecture, organisations are now using generative AI to pull
information from multiple systems, assemble it in standard visualisations and draft a narrative.
This enables analysts to spend their time building and adding value since the computer has
already done the rote work.
 Generative AI: A guide for building your own AI applications 9

Select the right model for

your use case
The number and diversity of generative AI models is growing rapidly, creating
confusion for would-be users. Here are some considerations for selecting the
right model for your application:

Model power
Evaluate the model’s computational power and
Small Language Models (SLMs)
sophistication. This determines its capability to handle
complex datasets and produce nuanced outputs. With SLMs, which are trained on smaller
Higher power might be required for intricate tasks, amounts of data with fewer parameters,
impacting operational demands and associated costs. you get the benefit of a model that can work
with limited computing resources, while
Time efficiency/latency allowing for increased specificity. SLMs can
The model’s speed is vital, especially for tasks that be run locally – an advantage for regulated
need fast data processing or instant decision-making. industries and in cases where lag time is
Efficiency not only affects performance but also extremely important – and can be fine-tuned to
influences user satisfaction and operational efficiency. specific tasks and work with narrow context. This
can be an excellent choice for specific industries
looking to complete a specific task. For instance,
Cost effectiveness
a financial services organisation can benefit
Analyse all associated costs, including initial set-up, from SLMs when processing claims but wants to
ongoing operations and maintenance. It’s essential to determine the precise training to use.
strike a balance between the model’s capabilities and
your budget. Learn more about Microsoft’s recent
advancements with SLMs.
 Generative AI: A guide for building your own AI applications 10

Fine-tuneability and extensibility Generality versus specificity

Take into consideration the ability to tweak the Choose between general-purpose and specialised
model to your specific data, as this can be important models based on the breadth or specificity of your
for the extensibility and modifications necessary for needs. General models offer flexibility, while specialised
your model and the intended purposes. However, if models provide high efficiency in specific contexts. For
your use case requires playing with the weights, then example, some models specialise in specific tasks, such
choosing an open model is more appropriate. Keep as chat completion or summarisation, or are purpose-
in mind that not all open models will have permissive built for specific data types such as code, images, video
licence to build commercial applications. or text, or a specific industry such as healthcare.

Licensing and availability of LLMs

To use LLMs for commercial purposes, consider
the licence of a particular model. Remember that
availability is not always straightforward, considering
that some models are closed source.

Another availability consideration is that models are

dependent on the data centre and not all models are
available in all data centres, creating challenges around
restrictions with data crossing certain boundaries.
 Generative AI: A guide for building your own AI applications 11

Microsoft evaluation and

monitoring metrics for d
te
era
generative AI Genwers
Dat
a Ans
ce
Automated metrics for output evaluation S our

To enable evaluation at scale, Microsoft is

developing tools to have LLMs evaluate the
output of generative AI applications.

Groundedness
Azure AI Content Safety-based Prompt-only-based
groundedness groundedness
The model’s generated answers are evaluated Measures how well the model’s generated
based on their alignment with information from answers align with information from the
the source data, such as retrieved documents in source data (user-defined context).
RAG, question and answering, or documents used
for summarisation. The evaluation process flags
output that lacks grounding.

Relevance Coherence
The extent to which the model’s How well the language model can
generated responses are pertinent produce
pr output that flows smoothly,
and directly related to the given reads
r naturally and resembles
questions. human-like language.

Fluency GPT-similarity
How well written and easy to How close the answer is to a user-
understand the answer is. provided ground truth and only
applies when you’ve supplied ground
truth and are using a generative AI
model to compare them.
 Generative AI: A guide for building your own AI applications 12

By incorporating these considerations, organisations can more effectively align generative AI

capabilities with strategic goals. The nuanced approach required for selecting and managing
generative AI highlights the importance of expert involvement and iterative testing, ensuring
the technology not only performs well, but also integrates seamlessly into organisational
processes to meet business needs.

Further details on the roles of domain SMEs and other team members will be explored later in
this e-book, providing deeper insights into the collaborative and dynamic nature of generative
AI development.
 Generative AI: A guide for building your own AI applications 13

Building a generative AI
development team
Developing with generative AI requires a hybrid approach – standard software
development blended with AI expertise. Roles like the AI engineer have
emerged to provide this connection between software development and AI.

At the same time, there is a partnership between

technical and business teams, with business needs
driving development. Non-technical business leaders Preparing your data for
and other stakeholders play focal roles in determining generative AI development
the utility and trustworthiness of AI applications. Their
Organisations can effectively prepare for
decisions are vital in deciding whether to continue
generative AI projects by ensuring that their
using and investing in the AI system.
data is ready. Generative AI runs on data,
so it’s essential to ensure its quality so the
output is more accurate.

Prepare your relevant data

Your organisation needs a robust data
infrastructure to handle large volumes of
data and high-performance computing
tasks. Data must be accessible, well
organised and secure to facilitate effective
AI training and operations.

Ensure data quality

Clean and prepare your data to ensure that
it is trusted and suitable for using prebuilt
foundation models. In addition to data
cleaning, consider if data is normalised and
if biases are mitigated. Quality is essential,
as it affects how precise and dependable
the AI models’ results are.
 Generative AI: A guide for building your own AI applications 14

Building a generative AI development team

From the visual below, we can see that the AI engineer leads the development team, while time investment may
be greater for the domain SMEs.

Overall project Picks the tech Develops the Iterates the Hardens for
lead stack app prompt(s) deployment

AI Engineer

a gene
ing
,
Iterates process-
Builds data r ing, chunking and
pipelines
ld

at
indexing
Bui

Data
ive

ro
Data Pro Science
Validates data Pro Leads evaluation (in
inputs and conjunction with the
m
AI

outputs domain SME)

ev
ea

elopment t
d

Building a generative

AI development
Domain SME team

Partners to identify Gathers Evaluates iterative Leads UAT,

business goals documents versions of the output roll out
 Generative AI: A guide for building your own AI applications 15

AI engineer
AI engineers are typically central, orchestrating the AI engineer learning resources
development initiative. Often, they are software
Looking for specific AI engineer educational
developers who have been informally upskilled to fill
resources? Get started with:
this role on the team. Consequently, the AI engineer
assumes the dual role of a leader and an integrator, AI engineer certification path
bridging the gap between AI capabilities and business Azure AI Fundamentals
needs without the necessity to delve deeply into data
Introduction to Azure AI Foundry
science, which is typically handled by data science
professionals. The AI engineer also takes information
from the domain SME and adjusts the application
based on their testing.

Data professional
A critical component of building with generative AI is pulling data from business systems and getting it to the
model. The data professional is the one who builds the pipelines that make this happen safely and efficiently.
Therefore, expertise in the nuances of data sources can be critical to create quality generative AI applications. The
data professional must also navigate the complexities of data privacy regulations to ensure all data collection and
usage comply with legal standards.

Domain SME
The domain subject matter expert (SME) collaborates closely with the AI engineer to ensure precise data collection.
They also continually monitor key performance metrics set by the AI engineer. In generative AI development,
domain SMEs tend to play a bigger role in than in standard software or machine learning because the system
outputs are not easily validated.

Domain SMEs take on the role of evaluating the correctness of responses, which can be time-consuming. Their
role is pivotal in bridging the technical and business aspects of AI implementations. To streamline their input,
domain SMEs can work with the rest of the team to create test prompts and ground truth answers. AI engineers
can then use Microsoft’s automated evaluation metrics to assess the application’s response quality over time and
compare performance across different models.
 Generative AI: A guide for building your own AI applications 16

Data scientist/machine-learning
professional Looking for specific AI data
Not every generative AI team has a data scientist, science educational resources?
but they are often present on complex and business-
Get started with:
critical projects. The data scientist plays an advisory
role, responsible for developing formalised test plans Microsoft Certified: Azure Data
and setting evaluation criteria, which alleviates some of Scientist Associate
the domain SME’s workload.

A common approach is Retrieval Augmented

Generation (RAG), which combines the reasoning
capabilities of models with customised data to
As the need for generative AI within
generate responses. This involves testing various
organisations grows, so does the demand
document chunking and annotation techniques,
for a centralised platform to develop and
assessing retrieval quality and evaluating responses
deploy these applications responsibly. This
across different prompts.
underscores the increasing significance
of the data science professional’s role
in ensuring these technologies are
implemented effectively and ethically.
 Generative AI: A guide for building your own AI applications 17

Azure AI Foundry:
A comprehensive platform
Organisations need tools that simplify AI development,
allowing them more time to focus on big-picture
business needs. Azure AI Foundry, Microsoft’s McDonald’s China transforms
generative AI platform, is designed to democratise its operations, elevates service
the AI development process for developers, bringing levels with Azure AI
together the models, tools, services and integrations
necessary to begin quickly and efficiently developing What was their goal?
your own AI applications. McDonald’s China needed to level up their
customer service, quality and operations
to accommodate the growing number of
locations and the rapid pace of innovation.
87% They needed a way to adhere to their
brand mission through increased digital
of organisations believe that AI will give
transformation.
them a competitive edge1
How did they achieve it?
66% With the support of Microsoft, McDonald’s
of surveyed board members report pressure China established the AI Lab, allowing
to accelerate AI adoption2 them to integrate AI into their existing
operations, predominately by leveraging
large language models (LLMs).

Through the AI Lab, AI has proliferated

their entire operations, from supply chain
operations to even supporting marketing
campaigns, as it affects how precise and
dependable the AI models’ results are.

Learn more

¹Expanding AI’s Impact with Organisational Learning

2
CEO decision-making in the age of AI
 Generative AI: A guide for building your own AI applications 18

As a comprehensive platform for developing and

deploying generative AI applications, Azure AI Foundry
emerges as a game-changing tool for both seasoned Siemens and Microsoft partner
developers and those new to AI. With drag-and-drop to drive cross-industry AI
functionality, visual programming environments and adoption
prebuilt templates, Azure AI Foundry makes it easier
for users to prototype, build and refine AI applications What was their goal?
without deep technical knowledge of the underlying Multinational technology conglomerate
algorithms. Siemens needed to simplify the virtual
collaboration of design engineers, front-line
This accessibility accelerates the development process workers and other teams across key business
and helps users to quickly translate their creative and functions. They were looking to utilise
business ideas into fully operational AI solutions. generative AI to rapidly generate, optimise and
debug complex automation code.

How did they achieve it?

With the introduction of Siemens Industrial
Copilot, a generative AI-powered assistant,
Siemens enhanced human-machine
collaboration and boosted productivity. They
also significantly shortened simulation times,
reducing task time from weeks to minutes.
Maintenance staff could now use the power
of natural language and gain assistance
with detailed repair instructions and provide
engineers with quick access to simulation tools.

Learn more
 Generative AI: A guide for building your own AI applications 19

Key features of Azure AI Foundry that Enterprise-grade production:

provide substantial value to users The platform facilitates the scalable deployment of
models, flows and apps, incorporating continuous
include:
monitoring and fine-tuning capabilities within a secure
Responsible AI tools and best practices: and governed environment. Organisations can scale
Azure AI Foundry empowers developers to safely, their AI solutions as needed without compromising
securely and responsibly innovate and shape the future on security or performance, providing developers with
with AI. The comprehensive platform accelerates the a flexible and robust infrastructure and ensuring that
development of production-ready agents to support users enjoy consistent, reliable AI services.
enterprise chat, content generation, data analysis and
more. Developers use their protected data to build
custom models and solutions with collaborative, The centralised nature of the Azure AI
responsible AI tools and best practices. Foundry platform provides a collaborative
development environment so generative
API and model choice:
AI teams can efficiently work together
Users can access and deploy the latest models as and stay in sync. Azure AI Foundry helps
APIs, facilitating rapid, serverless and fine-tuned users build AI solutions faster with prebuilt
model deployment. This reduces development time capabilities and templates to ultimately
and resource costs, accelerating time to market for accelerate solution development.
developers and providing end-users with state-of-the-
art AI capabilities quickly. The Assistants API makes it
easier for developers to create automated applications
with sophisticated, agent-like experiences that sift
through data, suggest solutions and automate tasks
using advanced tools like code.

Complete AI toolchain:
Azure AI Foundry offers tools to ground models on
specific data, orchestrate complex AI workflows and
evaluate model outputs for quality and safety, ensuring
robust end-to-end management. Developers find it
easier to integrate and manage AI projects, enhancing
productivity and operational efficiency, while users
experience more reliable and effective AI applications.
 Generative AI: A guide for building your own AI applications 20

Deploy responsible AI with Azure AI

Foundry ASOS uses Azure AI Foundry
Responsible generative AI refers to the development to surprise and delight young
and deployment of generative AI systems in a manner
fashion lovers
that is safe, transparent and accountable. Microsoft
offers numerous tools and controls that help with the What was their goal?
responsible deployment of generative AI. With the prevalence of generative AI
platforms, ASOS, a United Kingdom-based
Azure AI services: Developers can utilise pre-built and
fashion and cosmetic retailer, saw an
customisable APIs and models to rapidly build cutting-
opportunity to expand its business model,
edge, responsible applications. These services provide
enrich its technology infrastructure and
detailed Transparency Notes and fairness assessments,
meet customers’ modern tech expectations.
such as Face and Speech, to support customer choice
and transparency. How did they achieve it?
Responsible use of AI with Azure AI services ASOS used Azure OpenAI Service and
prompt flow, part of Azure AI Foundry, to
Azure AI Content Safety: Developers can get support
quickly streamline their development and
in detecting and mitigating risky content, including
testing cycles, helping the customer and the
prompt attacks, and the generation of ungrounded or
solution to interact effectively. ASOS helped
copyright material. By using Azure AI Content Safety
developers onboard quickly with built-in
as a built-in safety system, develops can build more
prompt flow resources, limiting the need
trustworthy applications.
for custom code.
Build AI applications responsibly with Azure AI
Content Safety Learn more

Evaluation and Monitoring in Azure AI Foundry: In addition to mitigating problematic inputs and outputs,
developers can continuously measure the effectiveness of their mitigations during development and in production
environments using pre-built evaluation and monitoring metrics for risks and safety. As data and user behaviour
change over time, these measurement tools help developers and domain SMEs understand their application’s
behaviour and intervene quickly when performance degrades or an end user is attempting to manipulate the
application to behave outside its prescribed purpose.

Evaluation and monitoring metrics for generative AI

Security: Microsoft provides the multi-layered security backbone of your applications, using built-in security
controls and unique threat intelligence to help identify and secure against evolving threats.

A focal point of Microsoft security is Microsoft Defender for Cloud – Microsoft’s cloud-native application
protection platform (CNAPP) – which provides comprehensive security to proactively mitigate risks and identify
threats to AI applications from code to cloud.

Learn more about Microsoft Defender for Cloud

 Generative AI: A guide for building your own AI applications 21

Mitigation layers in Azure AI Foundry

Using a layered mitigation plan can help developers
identify and remediate potential risks earlier in the
development process. Here is a brief overview of Azure System Message
AI Foundry’s robust safety mechanisms, designed to
secure and optimise your generative AI deployments:

System message layer: This layer provides hidden

instructions to your model with every user prompt, so Safety System
you can guide the model’s behaviour and data retrieval
to generate higher-quality responses by default.

Safety system layer: This added layer goes beyond

the basic safety fine-tuning that is part of the model. Model
Azure AI Content Safety provides this extra layer by
running both the prompt and completion of your
model through classification models. These models are
designed to detect and prevent the output of harmful
content across a range of categories and severity
levels.

Model layer: With Azure AI Foundry’s model

catalogue, you can explore benchmarks and model
cards for Azure OpenAI Service, Meta, Hugging
Face and other model developers, all organised by
collection and task.
 Generative AI: A guide for building your own AI applications 22

Safety feature highlights in Azure AI Foundry

Azure AI Content Filters

Prompt shields: Groundedness detection:

Detect and block Detect 'hallucinations' in
prompt injection attacks, model outputs, for better
including direct and accuracy and reliability of
indirect prompt attacks. the content generated by AI.

Configurable harmful content filters:

Detect four categories of harmful content (violence, hate, sexual and self-harm) at four
severity levels respectively (safe, low, medium and high), and optional binary classifiers
for detecting jailbreak risk, existing text and code in public repositories.

Safety system message: Steer your model’s behaviour towards safe, responsible outputs using system message
templates developed by Microsoft Research.

Safety evaluations: Assess an application’s vulnerability to jailbreak attacks and content risks using your own test
dataset or a test dataset generated with AI assistance.

Risk and Safety Monitoring: Understand what model inputs, outputs and end users are triggering content filters
to inform mitigations.

System message and grounding layer: This is where retrieval augmented generation (RAG) comes in. Instead
of using the model as a source of information, the model serves as a reasoning engine over data sources that
are relevant to the query. The system message helps guide the model to use grounding data effectively and
can help steer overall behaviour for more predictable, responsible model outputs.

User experience layer: At this level, there is a litany of user-centred interventions, guidance and best practices
that can be provided to users to ensure that the system is used as intended.

These enhancements to Azure AI Foundry’s safety features demonstrate Microsoft’s commitment to responsible
AI, ensuring that your applications are not only effective but also align with the highest standards of data integrity
and security.

Read in more detail: Harms mitigation strategies with Azure AI

 Generative AI: A guide for building your own AI applications 23

Another key component of responsible AI is the

evaluation and monitoring of your application. We’ve
discussed the generation of quality metrics that Azure Trailblazing AI answer
AI Foundry provides, but there are three overarching engine Perplexity. AI doubles
topics for consideration: throughput, cuts cost with
Manual evaluations: Azure AI Foundry
This is the form of evaluation that we recommend What was their goal?
starting with before moving to automated evaluations. Start-up [Link] is the creator of
This can be a useful way of tracking progress on a Perplexity Ask, a revolutionary AI-based
small set of priority issues. conversational answer engine that combines
Automated evaluations: large language models with a robust semantic
search engine. To better support Perplexity
On the other hand, automated evaluations can be
Ask, they needed a platform that would
more useful for measuring quality and safety on a
support faster time to market, serve as a force
larger scale and can gain more comprehensive results.
multiplier for their lean staff, scale to support
Monitoring: millions of users and deliver security and
Monitoring models that are deployed in production reliability at a cost-effective price.
is an essential part of the generative AI application How did they achieve it?
lifecycle. Changes in data and consumer behaviour
[Link] used Azure AI Foundry to
can change your applications performance over time.
develop their first prototype in several hours.
Azure AI Foundry makes it easy for you to monitor
They were able to try out large language
your applications in production for ongoing safety and
models available with Azure OpenAI Service,
quality.
getting going ‘with just a few clicks.’ Overall,
Learn more: Evaluation of generative AI applications this resulted in running twice as many
experiments, in parallel, before they adopted
Azure, enabling them to retrain the new
version of the model twice as fast.

Learn more
 Generative AI: A guide for building your own AI applications 24

Integrated data security and privacy

To help ensure data security and privacy, Microsoft Microsoft is obligated to defend
makes several commitments to its customers. For customers against certain third-party
one, Microsoft Azure is built on security, privacy intellectual property claims relating to
and compliance. When data is stored in Azure, your Output Content. For Azure OpenAI Service
data remains yours – it is never used for marketing, and any Configurable Generative AI
advertising or foundation model-training purposes Service, Customer also must implement all
without your permission. The prompts and the mitigations required by the Azure OpenAI
completions back are still your data. Service documentation in the offering that
delivered the Output Content that is the
Microsoft Fabric, Microsoft’s unified AI-powered subject of the claim.
analytics platform, can help reshape and improve
Learn more: Introducing the Microsoft
how you access, manage and act on data. On a single
Copilot Copyright Commitment
platform, Fabric unites data and services to simplify
data integration into a single, multicloud data lake for
your organisation to work from the same data across
analytics engines and languages.

In addition, Azure AI Foundry contains enterprise

security configurations, including:

Azure Policy integrations

Role-based access control (RBAC)

Network isolation and security

Data protection and encryption

Vulnerability management

To tie it all together, when managing data security and

privacy, Fabric has a native integration with Microsoft
Purview, which enables organisations to govern,
protect and manage their data across their entire
data estate. When integrated together, Fabric and
Purview enhances AI capabilities through secure data
integration.
 Generative AI: A guide for building your own AI applications 25

Key insights and next steps

As the demand for generative AI within organisations
grows, the need for a centralised platform to develop and
deploy these technologies responsibly and effectively
becomes critical.
Azure AI Foundry provides a robust solution, offering a comprehensive platform
that caters to developers at all levels, including those with no data science expertise.
This platform not only simplifies the development process by providing access to
a comprehensive model catalogue from leading providers, but also supports the
full lifecycle of AI development with tools for advanced data integration, workflow
orchestration and interactive apps

Get started with Azure AI Foundry

Microsoft Certified: Azure AI Engineer Associate

Microsoft Certified: Azure AI Fundamentals

GitHub repo for building a RAG application

Azure AI Foundry model catalogue

Azure AI Foundry Overview

Introduction to Azure AI Foundry – Training | Microsoft Learn

© 2024 Microsoft Corporation. All rights reserved. This document is

provided ’as is’. Information and views expressed in this document,
including URL and other Internet website references, may change
without notice. You bear the risk of using it. This document does not
provide you with any legal rights to any intellectual property in any
Microsoft product. You may copy and use this document for your
internal, reference purposes.