Module Six

Oracle Cloud Infrastructure (OCI) AI Services

1. Introduction

Artificial Intelligence (AI) is no longer just for researchers or tech giants—it’s

becoming an essential tool for businesses. Oracle Cloud Infrastructure (OCI)
provides a collection of AI services designed to help organizations take
advantage of their data and build smarter applications without needing deep
expertise in machine learning.

At the heart of OCI AI services is a simple idea:

 Data → feeds AI services

 AI services → are consumed by business applications

 Applications → use AI to deliver value (like chatbots, image

recognition, or document processing)

Oracle has invested heavily in this, moving from infrastructure (servers,

storage, networks) to software (applications powered by AI) and now into
Generative AI with large-scale models.

2. What Are OCI AI Services?

OCI AI Services are prebuilt machine learning models that developers

can easily use in business applications.

 Some models are pretrained (ready to use immediately).

 Others can be custom-trained with your own business data for better
accuracy.

 You don’t need to manage servers or complex infrastructure—just call

the service via an API, pass in your data, and get results.

👉 This makes AI accessible, even if you’re not an AI expert.

3. How to Access OCI AI Services

Oracle provides different ways to work with AI services depending on your

skill level:
 1. OCI Console

o A web-based interface (easy for beginners).

o Lets you access notebooks, data science tools, and AI services in

your browser.

2. REST API

o Gives programmatic access to AI services.

o Requires programming knowledge.

o Documentation includes API references.

3. Language SDKs (Software Development Kits)

o Prebuilt libraries for popular languages like Python, Java,

TypeScript/JavaScript, .NET, Go, and Ruby.

o Makes coding with AI services easier.

4. Command Line Interface (CLI)

o Allows quick access to services directly from your terminal.

o Useful for automation without writing full programs.

4. Types of OCI AI Services

Oracle offers a collection of AI services, each focused on solving different

business problems. Let’s explore them one by one:

(a) OCI Language Service

Helps businesses understand and process text.

 Pretrained Models (ready to use):

o Language detection (English, Spanish, etc.)

o Sentiment analysis (positive, negative, neutral)

o Key phrase extraction (important words/phrases)

o Text classification (e.g., spam vs. not spam)

o Named entity recognition (find people, places, organizations)

o PII detection (find sensitive data like phone numbers, emails)

  Custom Models (train with your data):

o Custom text classification

o Custom named entity recognition

 Text Translation:

o Uses neural machine translation to translate text across

many languages.

(b) OCI Vision Service

Helps businesses process and analyze images.

 Pretrained Models:

o Object detection (e.g., car, dog, table in a picture)

o Image classification (e.g., categorize image as "landscape,"

"food," etc.)

o OCR (Optical Character Recognition: extract text from images)

 Custom Models:

o Custom object detection (e.g., detect your company’s specific

product in photos)

o Custom image classification (train models to recognize unique

features of your images)

(c) OCI Speech Service

Converts speech to text.

 Upload audio or video files.

 Service returns transcripts in formats like JSON or SRT (subtitle

format).

 Useful for generating captions, searchable transcripts, or analyzing

conversations.

(d) OCI Document Understanding Service

Helps extract structured information from documents (PDFs, receipts,
forms, IDs, etc.).

 Text Extraction: Detects words and their positions in documents.

 Key Value Extraction: Pulls out important info like "Invoice Number,"
"Date," "Amount."

 Table Extraction: Extracts rows and columns while keeping structure.

 Document Classification: Automatically sorts documents into types

(e.g., invoice, passport, resume).

(e) Oracle Digital Assistant

An AI-powered chatbot platform that helps you build conversational

interfaces.

 Greets users and explains available actions.

 Routes conversations to the right "skills" (predefined tasks).

 Handles interruptions, clarifies ambiguous inputs, and exits gracefully

when asked.

 Useful for customer service, HR helpdesks, and IT support bots.

5. Why Use OCI AI Services?

 No AI expertise needed – prebuilt models do the heavy lifting.

 Customizable – you can fine-tune with your own data.

 Scalable – runs on Oracle’s cloud infrastructure.

 Multi-access options – console, APIs, SDKs, CLI.

 Business-focused – designed to solve real-world problems

(documents, speech, language, vision, assistants).

Flashcard 1

Q: What does OCI stand for in cloud computing?

A: OCI stands for Oracle Cloud Infrastructure.
Flashcard 2

Q: What is the foundation of AI and ML services in OCI?

A: Data is the foundation—AI services consume data to generate results.

Flashcard 3

Q: Do you need to manage servers when using OCI AI services?

A: No, there is no infrastructure to manage—you just call the service API.

Flashcard 4

Q: What is the most common way to access OCI AI services?

A: Through the OCI Console, a web-based interface.

Flashcard 5

Q: Which method gives developers programmatic access to OCI AI services?

A: The REST API.

Flashcard 6

Q: Name three programming languages supported by OCI SDKs.

A: Python, Java, JavaScript/TypeScript (others include .NET, Go, Ruby).

Flashcard 7

Q: What does the OCI Command Line Interface (CLI) provide?

A: Quick access and full functionality without writing full scripts.

Flashcard 8

Q: What are OCI AI Services in simple terms?

A: A collection of prebuilt machine learning models for business
applications.
Flashcard 9

Q: Can OCI AI models be customized with business-specific data?

A: Yes, some models can be custom-trained with your own datasets.

Flashcard 10

Q: What is the purpose of the OCI Language service?

A: To perform text analysis at scale, such as sentiment analysis and
classification.

Flashcard 11

Q: Give two examples of pretrained models in the OCI Language service.

A: Sentiment analysis and language detection.

Flashcard 12

Q: What does “text translation” in OCI Language service use?

A: Neural Machine Translation (NMT).

Flashcard 13

Q: What does the OCI Vision service do?

A: Analyzes images with tasks like object detection, image classification,
and OCR.

Flashcard 14

Q: What is the difference between pretrained and custom models in Vision?

A: Pretrained models detect general objects, while custom models detect
business-specific objects.

Flashcard 15
Q: What does the OCI Speech service provide?
A: Converts audio/video speech into text transcripts (JSON or SRT).

Flashcard 16

Q: What type of documents can OCI Document Understanding process?

A: Receipts, invoices, passports, driver IDs, and more.

Flashcard 17

Q: What does Key Value Extraction do in Document Understanding?

A: Extracts important pairs of information (e.g., "Invoice Number: 12345").

Flashcard 18

Q: What is Table Extraction in Document Understanding?

A: Extracting tabular data while keeping row and column structure.

Flashcard 19

Q: What is the Oracle Digital Assistant used for?

A: To build AI-powered chatbots for natural language conversations.

Flashcard 20

Q: Give one real-world use case of Oracle Digital Assistant.

A: Customer service bots that answer user questions and route tasks.
Oracle Cloud Infrastructure (OCI) Machine Learning Services

1. Introduction

Artificial Intelligence (AI) and Machine Learning (ML) are revolutionizing the
way businesses use data. To make this easier for organizations, Oracle
provides a portfolio of cloud services under Oracle Cloud Infrastructure
(OCI).

At the heart of these services is data.

 Data layer → provides the foundation.

 AI & ML services layer → processes the data.

 Applications layer → consumes AI/ML results (e.g., apps, analytics

systems, business workflows).

In this lesson, the focus is on OCI Data Science, which belongs to the
machine learning services group.

2. What is OCI Data Science?

OCI Data Science is a cloud service designed to support data scientists

through the entire machine learning lifecycle, from building and training
models to deploying and managing them.

 ✅ Built for Python and supports open-source tools.

 ✅ Provides a familiar JupyterLab Notebook environment.

 ✅ Eliminates infrastructure headaches (compute, storage,

maintenance).

 ✅ Designed for both individual data scientists and collaborative

teams.

3. Core Principles of OCI Data Science

OCI Data Science is built on three guiding principles:

1. Accelerated

 Boosts individual productivity.

  Provides open-source libraries and Oracle’s own Accelerated Data
Science (ADS) library.

 Easy access to powerful computing resources (CPU/GPU) without

managing infrastructure.

2. Collaborative

 Encourages teamwork among data scientists.

 Assets (projects, notebooks, models) can be shared to avoid

duplication.

 Ensures reproducibility (others can replicate your results) and

auditability (track what was done).

3. Enterprise-Grade

 Secure and fully managed.

 Integrated with OCI’s security and access protocols.

 Automatically handles provisioning, patching, and upgrades.

 Lets users focus on solving business problems instead of maintaining

infrastructure.

4. Features of OCI Data Science

Here are the main features that make OCI Data Science powerful and easy to
use:

🔹 Projects

 Collaborative workspaces for organizing data science assets.

 Can contain notebooks, models, and documentation.

 No limit—each OCI tenancy can have as many projects as needed.

🔹 Notebook Sessions

 Provide an interactive JupyterLab environment.

 Preinstalled with popular open-source Python libraries.

 Users can add their own libraries.

  Run on managed infrastructure where you choose CPU/GPU and
storage.

 Ideal for building and training ML models.

🔹 Conda Environments

 A package management system for Python.

 Lets you easily install, update, and manage dependencies.

 Allows switching between different environments for different projects.

🔹 Accelerated Data Science (ADS) SDK

 Oracle’s Python library for simplifying data science workflows.

 Automates common steps such as:

o Connecting to data

o Exploring and visualizing data

o Training models with AutoML

o Evaluating and explaining models

 Provides a simple interface to OCI services (Data Science, Object

Storage, Model Catalog).

🔹 Models

 Mathematical representations of data and business processes.

 Created in notebook sessions as part of projects.

 Stored and shared for reuse.

🔹 Model Catalog

 A centralized repository to store, track, share, and manage

models.

 Each model includes metadata (e.g., Git info, training scripts).

 Models can be pulled back into notebooks or shared with team

members.

🔹 Model Deployments
  Allows models stored in the catalog to be deployed as HTTP
endpoints.

 Enables real-time predictions via APIs.

 This is how ML models are operationalized for business use.

🔹 Jobs

 Define and run repeatable ML tasks on fully managed infrastructure.

 Useful for scheduled tasks such as batch predictions or retraining

models.

5. The Machine Learning Lifecycle with OCI Data Science

Here’s how a typical workflow looks in OCI Data Science:

1. Create a project → organize assets.

2. Start a notebook session → write Python code and experiment.

3. Set up Conda environments → manage dependencies.

4. Use ADS SDK → automate and streamline ML tasks.

5. Build and train models → create ML solutions.

6. Save models in Model Catalog → store and share securely.

7. Deploy models → expose as APIs for real-time use.

8. Run jobs → automate repeatable tasks like retraining or batch

predictions.

6. Why Use OCI Data Science?

 Productivity: Accelerates the work of individual data scientists.

 Collaboration: Makes teamwork easy and efficient.

 Enterprise readiness: Secure, scalable, and fully managed.

 Flexibility: Works with open-source tools and Python.

 Integration: Connects seamlessly with other OCI services.

Flashcard 1

Q: What is the foundation of AI and ML services in OCI?

A: Data is the foundation—AI and ML depend on it.

Flashcard 2

Q: What are the two main groups between applications and data in OCI’s
architecture?
A: AI Services (prebuilt models) and Machine Learning Services (custom
ML with OCI Data Science).

Flashcard 3

Q: What is OCI Data Science?

A: A cloud service that supports data scientists throughout the full
machine learning lifecycle.

Flashcard 4

Q: Which programming language is primarily supported by OCI Data

Science?
A: Python (with open-source support).

Flashcard 5

Q: What familiar interface does OCI Data Science provide for coding?
A: JupyterLab Notebook.

Flashcard 6

Q: What are the three core principles of OCI Data Science?

A: Accelerated, Collaborative, and Enterprise-Grade.

Flashcard 7
Q: What does “accelerated” mean in OCI Data Science?
A: It speeds up individual data scientists’ work with open-source libraries,
ADS SDK, and managed compute power.

Flashcard 8

Q: How does OCI Data Science support collaboration?

A: By allowing teams to share assets, avoid duplication, and ensure
reproducibility and auditability.

Flashcard 9

Q: What does “enterprise-grade” mean for OCI Data Science?

A: It integrates with OCI security, provides fully managed infrastructure,
and handles upgrades/patches.

Flashcard 10

Q: What is a “project” in OCI Data Science?

A: A container/workspace for organizing notebooks, models, and other
assets.

Flashcard 11

Q: Is there a limit to how many projects you can create in OCI Data Science?
A: No, a tenancy can have unlimited projects.

Flashcard 12

Q: What is a “notebook session”?

A: An interactive JupyterLab environment for coding, training, and
experimenting with ML models.

Flashcard 13
Q: What resources can you choose for notebook sessions?
A: CPU or GPU compute shapes and storage, all on managed
infrastructure.

Flashcard 14

Q: What is a Conda environment used for in OCI Data Science?

A: To manage Python packages and dependencies across projects.

Flashcard 15

Q: What is ADS SDK?

A: Oracle’s Accelerated Data Science SDK, a Python library that
automates ML workflows.

Flashcard 16

Q: Name two tasks the ADS SDK can automate.

A: Exploring/visualizing data and training models with AutoML (also:
evaluation, explanation, deployment).

Flashcard 17

Q: What is the Model Catalog?

A: A central repository for storing, sharing, and managing ML models with
metadata.

Flashcard 18

Q: What are model deployments in OCI Data Science?

A: Deploying models as HTTP endpoints/APIs to serve real-time
predictions.

Flashcard 19
Q: What are “jobs” in OCI Data Science?
A: Repeatable ML tasks (like retraining or batch processing) run on
managed infrastructure.

Flashcard 20

Q: Why should organizations use OCI Data Science?

A: It provides productivity, collaboration, enterprise security,
flexibility with open-source, and seamless integration with other OCI
services.
GPUs in AI Infrastructure

A Graphics Processing Unit (GPU) is a critical hardware component for

accelerating artificial intelligence (AI) and machine learning (ML) workloads.
GPUs are specifically designed to perform many repetitive calculations
simultaneously, which is essential for the large-scale data processing
required in training and running AI models.

Why are GPUs Important for AI?

 High Volume Calculations: AI and ML workloads involve performing

billions of calculations repeatedly on huge datasets, especially during
model training and inference (making predictions).

 Parallel Computing: Unlike traditional CPUs that have a few powerful

cores, GPUs consist of thousands of lightweight cores working
simultaneously on different pieces of data. This parallelism allows GPUs
to process large datasets at very high speed.

 Improved Throughput: GPUs can handle multiple inference tasks in

parallel, making them highly efficient for batch processing or serving
many prediction requests at the same time — something CPUs struggle
to do efficiently.

Modern GPU Optimization for Deep Learning

 GPUs are fine-tuned to support popular deep learning frameworks such

as TensorFlow, PyTorch, and ONNX Runtime.

 These frameworks use GPU-specific libraries to speed up compute-

intensive operations common in neural network training and inference.

NVIDIA GPU Architectures for AI

1. A100 (Ampere Architecture, 2020)

 Uses specialized units called Tensor Cores.

 Tensor Cores perform fused multiply-accumulate operations in a

single clock cycle, greatly accelerating deep learning workloads.

2. H100 (Hopper Architecture, 2022)

 Introduces a dedicated Transformer Engine optimized for

transformer models, which are foundational for natural language
processing (NLP) and large language models (LLMs).
 3. H200 (2024)

 Builds on H100 with increased memory to handle larger AI

models and datasets.

4. Blackwell Architecture (2025)

 Designed for accelerating very large-scale AI models, such as

advanced LLMs.

 Supports cutting-edge AI workloads focusing on scalability and

performance.

NVIDIA Grace CPU and Blackwell Chips

 Grace CPU: A new processor designed for high-performance

computing (HPC) and AI cloud data centers.

 GB200, Grace Blackwell, and L40 Superchip: State-of-the-art

components combining CPU and GPU capabilities to unleash powerful
AI performance, introduced in 2025.

OCI GPU Offerings

Oracle Cloud Infrastructure (OCI) offers a broad range of GPU compute

options to support various AI workloads, from small to large scale:

 GPUs like H100N and L40 are currently available.

 Orders are open for upcoming GPUs like H200, B200, and
the GB200 superchips, anticipated to be generally available in 2025.

 OCI supports superclusters built with these advanced GPUs, offering

very high computing power.

Performance Advancements

 Starting with H200, OCI GPU superclusters deliver 4 times the

performance compared to H100 clusters.

 The B200 and GB200 chips provide APEX-level performance for

demanding AI workloads.

Using OCI AI Infrastructure for Large Language Models (LLMs)

 OCI Data Science enables deployment of popular large language

models directly to virtual machines (VMs) or bare metal servers
powered by GPUs.
  Users can fine-tune base models and deploy customized versions for
inference using OCI Data Science AI Quick Actions.

 The platform supports registration and deployment of models

compatible with virtual LLM containers and next-generation inference
containers.

🎴 Flashcards: GPU in AI Infrastructure

Basics of GPU in AI

Q1: What does GPU stand for?

A1: GPU stands for Graphics Processing Unit.

Q2: Why are GPUs important in AI and ML?

A2: Because they handle massive repetitive calculations needed for
training and inference much faster than CPUs.

Q3: What is the main difference between CPU and GPU?

A3: CPUs have few powerful cores, while GPUs have thousands of
smaller cores that work in parallel.

GPU Functionality

Q4: What is parallel computing in GPUs?

A4: It’s the ability of GPUs to run many processes at the same time,
enabling high-speed data processing.

Q5: Why are GPUs better than CPUs for inference tasks?
A5: GPUs can handle many inference tasks in parallel, leading to higher
throughput.

Q6: Which AI tasks benefit most from GPUs?

A6: Model training and model inference (predictions).

Deep Learning & Frameworks

Q7: Name 3 popular deep learning frameworks optimized for GPUs.

A7: TensorFlow, PyTorch, ONNX Runtime.
Q8: What do these frameworks use to accelerate computations on GPUs?
A8: They use GPU-specific libraries.

NVIDIA GPU Architectures

Q9: Which GPU architecture introduced Tensor Cores?

A9: A100 (Ampere architecture, 2020).

Q10: What do Tensor Cores do?

A10: They perform fused multiply-accumulate operations in a single
clock cycle, boosting deep learning performance.

Q11: Which architecture introduced the Transformer Engine?

A11: H100 (Hopper architecture, 2022).

Q12: What makes H200 different from H100?

A12: H200 has more memory, allowing larger models to be trained.

Q13: Which NVIDIA architecture was designed for large-scale AI models

and LLMs?
A13: Blackwell architecture (2025).

Q14: What is the NVIDIA Grace CPU designed for?

A14: For AI cloud and high-performance computing (HPC) in modern
data centers.

Q15: What is the GB200 Superchip?

A15: A combination of Grace CPU + Blackwell GPU, delivering massive
performance for AI workloads.

OCI (Oracle Cloud Infrastructure) and GPUs

Q16: What GPU options does OCI currently offer?

A16: L40 GPUs and H100N superclusters with 10,800 GPUs.

Q17: Which GPUs will OCI make generally available in 2025?

A17: H200, B200, and GB200 Superchips.

Q18: What performance improvement does H200 bring compared to H100?

A18: 4X performance of H100 superclusters.

Q19: What feature in OCI Data Science allows quick deployment of LLMs?
A19: AI Quick Actions.
Q20: What can users do with AI Quick Actions?
A20: Deploy LLMs directly, fine-tune base models, and deploy
custom models for inference.
RDMA and OCI Superclusters

1. Introduction: Performance at Scale

Oracle Cloud Infrastructure (OCI) focuses on delivering maximum

performance at the lowest possible cost.
One of the key technologies enabling this is RDMA (Remote Direct
Memory Access).

 RDMA = a networking technology that allows data transfer directly

between machines’ memory without involving the CPU.

 This reduces CPU overhead and enables high bandwidth, ultra-low

latency communication.

👉 In simple terms: RDMA lets machines (and GPUs) talk directly, very
fast, without disturbing the CPU.

2. Why RDMA Matters in OCI

 Used in database services (e.g., Exadata Cloud Service, Autonomous

Database).

 Essential for HPC (High-Performance Computing) workloads.

 Critical for GPU workloads (AI/ML, LLMs, deep learning).

OCI made a strategic bet on RoCE (RDMA over Converged Ethernet),

which allows RDMA to run over standard Ethernet fabric.

3. Superclusters with RDMA

As AI workloads grow, customers and partners (like NVIDIA) need very large
GPU clusters—spanning thousands or tens of thousands of GPUs.

To meet this need, OCI designed the RDMA Supercluster:

 A massive cluster supporting GPUs at scale.

 Built with special networking architecture to keep performance

high while scaling.
4. Inside the GPU Node

Each GPU node typically contains:

 8 NVIDIA A100 GPUs, interconnected using NVLink (fast GPU-to-GPU

communication).

 These nodes connect to the network fabric at 1.6 terabits per

second (Tbps).

👉 That means each GPU gets 200 Gbps bandwidth (1.6 Tbps ÷ 8 GPUs).

5. The Network Fabric

 The network fabric is a non-blocking interconnect, meaning any

GPU can talk to any other GPU at the same time without slowdown.

 To scale, OCI uses a three-tier Clos network (a well-known design in

networking).

 This enables scaling to tens of thousands of GPUs and potentially

100,000+ GPUs in the future.

6. Latency in Superclusters

 Within a block: round-trip latency ≈ 6.5 microseconds.

 Across blocks: round-trip latency ≈ 20 microseconds (still much

lower than typical cloud networking).

OCI manages this by:

1. Using QoS-enabled networks with enough buffering in switches.

2. Ensuring lossless networking → packets are never dropped.

3. Designing congestion controls so traffic flows smoothly.

7. Balancing Scale vs. Latency

 Not all workloads need tens of thousands of GPUs.

 Some workloads (e.g., databases or smaller HPC jobs) are more

sensitive to latency than scale.
OCI handles this with intelligent placement:

 Smaller workloads are deployed within a single block to enjoy the

lowest latency (6.5 µs).

 Large-scale workloads can span multiple blocks, accepting slightly

higher latency (20 µs).

8. Network Locality Hints

OCI introduces network locality hints to help customers optimize

performance:

 Information is provided about how GPUs are connected.

 Customers can design their ML model topologies to keep most

communication local.

👉 Example:

 85% of GPU traffic stays within the same block (ultra-low latency).

 Only a small fraction crosses blocks (slightly higher latency).

 This reduces flow collisions (two data streams competing for the
same path), leading to higher throughput.

9. Key Optimizations in OCI Superclusters

OCI’s RDMA Superclusters are not just big networks—they’re optimized

networks.
Three major innovations:

1. Tuned Buffers: Switches and silicon designed with enough buffering

for worst-case latency → ensures lossless communication.

2. Smart Placement: Workloads are intelligently placed in blocks or

towers to minimize latency and avoid unnecessary traffic.

3. Locality Hints: Customers get hints about network topology, so GPU

workloads can be structured to keep traffic local → lowers latency and
increases throughput.
10. Why It Matters

 For AI/LLM training: Tens of thousands of GPUs can work together

efficiently.

 For HPC workloads: Scientists get predictable, ultra-fast networking.

 For databases: Oracle services run faster and more reliably.

In short:
✅ RDMA Superclusters allow OCI to deliver scale + speed → enabling the
next generation of AI, HPC, and database services.

⚡ Simplified Analogy:
Think of RDMA Superclusters like a highway system:

 Within a block = local roads (fast, short trips, 6.5 µs).

 Across blocks = highways (slightly longer trips, 20 µs).

 Traffic rules (QoS, buffers, lossless design) ensure no cars

(packets) crash or get lost.

 Smart city planning (placement + locality hints) ensures most

cars stay local, keeping traffic smooth.

1. Q: What does RDMA stand for?

A: RDMA stands for Remote Direct Memory Access.

2. Q: What is the main advantage of RDMA?

A: It allows direct memory-to-memory data transfer between machines

without involving the CPU, reducing latency and CPU overhead.

3. Q: Why is RDMA important for AI workloads?

A: AI workloads need fast communication between GPUs. RDMA provides
ultra-low latency and high bandwidth networking, making large-scale
training efficient.

4. Q: What is RoCE in OCI networking?

A: RoCE (RDMA over Converged Ethernet) enables RDMA to work over

standard Ethernet networks.

5. Q: Which OCI services use RDMA?

A: Exadata Cloud Service (ExaCS), Autonomous Database, HPC

workloads, and GPU workloads all rely on RDMA.

6. Q: What is an RDMA Supercluster?

A: A large-scale GPU cluster in OCI designed with RDMA-enabled

networking to support tens of thousands of GPUs with high performance.

7. Q: How many GPUs are typically in one GPU node?

A: Each node usually has 8 NVIDIA A100 GPUs.

8. Q: How are GPUs within a node interconnected?

A: They are connected using NVIDIA NVLink, enabling fast GPU-to-GPU

communication.

9. Q: What is the bandwidth from each GPU node to the fabric?

A: Each node connects at 1.6 terabits per second (Tbps).

10. Q: How much bandwidth does each GPU get?

A: Each GPU gets about 200 gigabits per second (Gbps).

11. Q: What kind of network design does the Supercluster use?

A: A three-tier Clos network, a scalable architecture for large data

centers.

12. Q: How many GPUs can the Supercluster scale to?

A: It can scale to tens of thousands of GPUs and potentially over

100,000 GPUs.

13. Q: What is the latency within a single block?

A: About 6.5 microseconds round trip.

14. Q: What is the latency across blocks?

A: About 20 microseconds round trip.

15. Q: What does “lossless networking” mean in RDMA?

A: It means no packets are dropped, thanks to buffering, congestion

control, and QoS mechanisms.

16. Q: How does OCI handle workloads that need very low latency?

A: Smaller workloads are placed within a single block, so they only

experience 6.5 µs latency.

17. Q: What are “network locality hints”?

A: Information provided to customers about GPU placement so workloads

can keep most communication local, reducing latency and flow collisions.

18. Q: What percentage of GPU traffic usually stays within the same
block?

A: Around 85% of the traffic remains local to the block.

19. Q: What side benefit comes from keeping traffic local?

A: It reduces flow collisions, leading to higher throughput.

20. Q: What are the three key optimizations in OCI’s Superclusters?

A:

1. Tuned buffers to handle latency while staying lossless.

2. Smart placement of workloads to reduce latency.

3. Locality hints to optimize traffic flow and throughput.

Responsible AI

1. Introduction: Why Do We Need Responsible AI?

Artificial Intelligence (AI) is becoming part of our daily lives — from self-
driving cars to AI-assisted medical diagnosis.
But an important question arises: Can we fully trust AI?

For example:

 Would you sit in a self-driving car without keeping an eye on the road?

 Would you accept an AI’s medical diagnosis without consulting a

doctor?

The truth is, AI can be powerful, but it must also be trustworthy. That’s
where Responsible AI comes in.

2. Guiding Principles of Trustworthy AI

To be considered trustworthy, AI should follow three pillars:

1. Lawful – AI must comply with existing laws and regulations.

o Example: In healthcare, medical devices must follow strict legal

rules before being used on patients.

2. Ethical – AI should align with human values and ethical principles

such as fairness, privacy, and respect.

3. Robust – AI must be safe, secure, and reliable, both technically

(performing correctly) and socially (not harming individuals or
society).

3. The Role of Laws in AI

AI does not operate in a vacuum. Different national and international

rules govern its use.

 What laws do:

o Prohibit harmful actions (e.g., discrimination, surveillance abuse).

o Enable positive actions (e.g., protecting minority rights,

safeguarding the environment).
  Domain-specific regulations:

o Example: In healthcare, medical device regulations govern

how AI-driven diagnostic tools are tested and approved.

4. Human Ethics and Fundamental Rights

Responsible AI must respect human dignity and freedom.

 Human dignity → Every human being has inherent worth. AI should

never compromise people’s physical or mental integrity.

 Freedom & privacy → AI must protect freedom of expression and the

right to a private life.

 Democracy → AI must support, not undermine, democratic processes

like voting.

 Equality → AI should avoid unfair bias and protect citizens’ rights

equally.

5. Core Ethical Principles for AI

From these human values, three main ethical principles emerge:

1. Human-centered – AI should assist humans, allow for oversight, and

leave meaningful choices to people.

2. Do no harm – AI must avoid both physical harm (e.g., unsafe robots)

and social harm (e.g., discrimination).

3. Transparency and fairness – AI decisions should be explainable

and fair to everyone.

6. Mapping Ethics to Responsible AI Requirements

To put ethics into practice, Responsible AI requires:

 Human oversight → Systems must be designed with human-in-the-

loop decision-making.

 Safety & security → AI must be robust, secure, and protected

from malicious use.
  Fairness → Benefits and risks should be distributed equitably across
all groups.

 Explainability → AI decisions should be understandable to those

affected.

7. Responsible AI Implementation Process

How do organizations actually implement Responsible AI?

1. Governance → Set up frameworks for accountability.

2. Policies & Procedures → Define rules to ensure ethical, legal, and

robust AI use.

3. Monitoring & Evaluation → Continuously check if AI is following

these rules.

 Who is involved?

o Developers (build AI systems),

o Deployers (implement AI into real-world systems),

o End users (use and are affected by AI decisions).

8. Key Challenges in Responsible AI

AI is powerful but comes with challenges, especially in healthcare:

1. Bias in AI systems

o AI is only as good as the data it learns from.

o If training data is biased (e.g., mostly male or one racial group),

AI may give unfair results for other groups.

2. Transparency & Explainability

o Many AI models (like deep learning) are “black boxes.”

o Doctors and patients may struggle to understand how the AI

reached its conclusion.

o This makes trust difficult without explainability.

3. Ongoing Evaluation
 o AI must be regularly checked to ensure it is performing correctly
and not causing harm.

Flashcard 1

Q: What is Responsible AI?

A: Responsible AI ensures that AI systems are lawful, ethical, and robust,
making them safe, fair, and trustworthy.

Flashcard 2

Q: Why do we need Responsible AI?

A: Because AI is increasingly used in daily life (e.g., self-driving cars,
healthcare), and it must be trustworthy to avoid harm.

Flashcard 3

Q: What are the three pillars of trustworthy AI?

A: Lawful, Ethical, and Robust.

Flashcard 4

Q: What does “Lawful AI” mean?

A: AI that complies with all relevant national and international laws.

Flashcard 5

Q: Give an example of domain-specific AI laws.

A: In healthcare, medical device regulations govern AI-based diagnostic
tools.

Flashcard 6

Q: What does “Ethical AI” mean?

A: AI that respects human values such as dignity, freedom, equality, and
fairness.
Flashcard 7

Q: What does “Robust AI” mean?

A: AI that is technically reliable, safe, secure, and protected from
malicious use.

Flashcard 8

Q: What is human dignity in AI ethics?

A: Every person has intrinsic worth that AI must never compromise.

Flashcard 9

Q: How should AI support democracy?

A: AI should foster democratic processes and never undermine voting or
decision-making systems.

Flashcard 10

Q: What is equality in Responsible AI?

A: AI systems must not produce unfairly biased outputs and must treat all
groups fairly.

Flashcard 11

Q: What are the three core ethical principles for AI?

A: (1) Human-centered with oversight, (2) Do no harm, (3) Transparent and
fair.

Flashcard 12

Q: What does “human oversight” in AI mean?

A: AI should leave meaningful opportunities for human choice and
control.
Flashcard 13

Q: Why is fairness important in AI?

A: To ensure that benefits and risks are equally shared and not biased
toward one group.

Flashcard 14

Q: What does “explainability” in AI mean?

A: AI decisions should be understandable to those directly or indirectly
affected.

Flashcard 15

Q: What are the three steps of Responsible AI implementation?

A: (1) Governance, (2) Policies and procedures, (3) Monitoring and
evaluation.

Flashcard 16

Q: Who are the main roles in Responsible AI implementation?

A: Developers, Deployers, and End Users.

Flashcard 17

Q: What is one challenge of AI in healthcare?

A: Bias – if training data is unbalanced (e.g., mostly one gender or race), AI
may perform poorly for others.

Flashcard 18

Q: Why is transparency a challenge in AI?

A: Many AI systems are “black boxes,” making their decisions hard to
explain or trust.

Flashcard 19
Q: Why must AI be continuously monitored?
A: To ensure it works as intended and does not unintentionally cause
harm.

Flashcard 20

Q: What is the ultimate goal of Responsible AI?

A: To make AI that is trustworthy, fair, safe, and designed to help
humans.