0% found this document useful (0 votes)
2 views41 pages

Module 5 Understanding Vector Databases

Module 5 discusses the advantages of vector databases over traditional keyword searches, emphasizing that AI can understand meaning and context through embeddings and semantic search. It explains how vectors represent data and how similarity searches find results based on meaning rather than exact matches. The module also highlights real-world applications of vector databases in AI chatbots, recommendations, and search engines.

Uploaded by

Masood Alam
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
2 views41 pages

Module 5 Understanding Vector Databases

Module 5 discusses the advantages of vector databases over traditional keyword searches, emphasizing that AI can understand meaning and context through embeddings and semantic search. It explains how vectors represent data and how similarity searches find results based on meaning rather than exact matches. The module also highlights real-world applications of vector databases in AI chatbots, recommendations, and search engines.

Uploaded by

Masood Alam
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Module 5

Why AI searches by meaning, not just keywords

Understanding
Vector Databases
AI & Machine Learning Series
Module 5: Vector Databases Exact Match Only

Only finds pages with the exact words


typed — no flexibility or variation allowed.

Keyword
Search Limits Misses Synonyms

Searching "car" won't find "automobile" —


same meaning, but keyword search fails.

No Context Understanding

"How does photosynthesis work?" may


miss great articles that don't use those
exact words.

@Reallygreatsite
Module 5 · Vector Databases

What is a
Vector?
Vectors = Numbers with Direction

A vector is a list of numbers that describes a point in


space — capturing meaning, direction, and distance.
• Like coordinates on a map (x, y, z...)
• Represents data: words, images, sounds
• Direction + length = meaning in AI

Analogy: GPS coordinates


locate a place; vectors
locate meaning!
Module 5: Vector Databases

Recap:
Embe
ddings
What are Embeddings?

• Text becomes numbers in vector form


• Embeddings capture the semantic meaning of words and
sentences

• Similar meanings similar numbers
• Visual: a sentence mapped to a point in space
Meaning Over Words
Module 5: Vector Databases
Instead of matching exact words, AI finds
results with the closest meaning in vector

Similarity Search space.

Nearest Meaning Wins

The search returns results closest in


meaning, even if different words are used.

Context is Everything

Similarity search understands context, so


"happy" and "joyful" are treated as close
matches.

@Reallygreatsite
Module 5: Vector Databases

Semantic
Search

Meaning-Based Search

Semantic search finds results by MEANING, not just exact


words.
🔍 Keyword Search: "car"→ finds only "car"
🧠 →
Semantic Search: "car" finds "vehicle", "automobile",
"sedan", "transport"
AI understands that these words share the same concept!

@reallygreatsite
Module 5 · Vector Databases

Cosine
Similarity
Measuring How Close Two Meanings Are

📐 Angle = Similarity: Smaller angle between two vectors → more


similar meaning!
🧲 Like Friends: Words with similar meanings "stand close" in
vector space — just like friends who share interests hang out
together.
🔵 Clusters: Similar words form groups (clusters) in space — "dog",
"puppy", "pet" all cluster near each other!
✅ No math needed — just think: closer angle = closer meaning!

Class 5–12 · AI Education


Module 5: Vector Databases

Nearest
Neighbors

Finding Closest Points

In vector space, similar items cluster together. The


"nearest neighbors" of a point are the closest surrounding
points — meaning the most similar items! AI finds
answers by locating the nearest neighbors to your query.
Module 5 – Vector Databases Store Embeddings

Saves vector embeddings from text, images,


or audio for AI to access quickly.

Vector
Database Semantic Search

Finds results by meaning, not just exact


words — perfect for AI-powered search.

Fast Retrieval

Uses special indexing to locate nearest


neighbors in vector space super fast!

@Reallygreatsite
Module 5 – Vector Databases

Traditional
vs Vector
Databases

Side-by-Side Comparison

🔵 Traditional Database
→ Exact keyword match only
→ Returns exact record or nothing
→ Misses synonyms & context
🟡 Vector Database
→ Semantic query by meaning
→ Returns closest meaning matches
→ Understands context & synonyms
@reallygreatsite
Module 5: Vector Databases

Real-World
Uses
AI Chatbots & Search Recommendations & Docs

AI chatbots use vector databases to Recommendation systems suggest


find relevant answers, and image movies or products, and document
@reallygreatsite search engines find visually similar search finds the most relevant files by
pictures instantly. meaning.
Module 5 – Vector Databases

Popular
Vector
Databases
Pinecone & Qdrant Weaviate & ChromaDB
Pinecone: Cloud-native, fast semantic Weaviate: Open-source, built-in ML
search, great for production AI apps. models, best for knowledge graphs.
@reallygreatsite Qdrant: Open-source, high-speed, ChromaDB: Lightweight, easy setup,
ideal for recommendation engines. perfect for local AI projects.
Module 5 – Vector Databases

Activity!

Which Database Would You Choose?

Traditional DB = exact match | Vector DB = meaning-based search


🔍 "Find all files named [Link]"
🤔 "Find documents about climate change"
🛒 "Show products with SKU #4521"
💬 "Recommend songs similar to this one"
👥 Discuss: Which needs a Vector DB? Why?
University Presentation

How AI uses external knowledge to answer questions

Module 6: Retrieval-
Augmented
Generation (RAG)
RAG & External Knowledge
General Knowledge Only
Module 6: RAG

AI is trained on public data — it cannot read your private


PDFs, notes, or uploaded files.

The Problem
Missing Your Files

Your personal documents, class notes, and reports are


invisible to the AI model by default.

No Access to Context

Without extra tools, AI cannot answer questions about


content it has never seen before.

@Reallygreatsite
Module 6: RAG

How Can AI Answer


Questions About
Unknown Documents?

Think About It...


AI knows a lot — but what if you ask it
about YOUR notes, YOUR files, or a brand
new document it has never seen before?
Module 6: RAG

RAG Pipeline
Overview
The Solution: How RAG Works

📄 PDF/Doc → ✂️ Chunking → 🔢 Embeddings →


🗄️ Vector DB → 🔍 Retrieval → 🤖 LLM → 💬 Answer
Each step transforms your document into a
smart, searchable AI response!

@AI_EduClass
Module 6: RAG Pipeline

Step 1:
Chunk
ing
Document Chunking
• Split large documents into smaller, manageable pieces
• Each chunk = a focused section of text
• Smaller chunks are easier for AI to process and retrieve
accurately
Module 6: RAG Pipeline

Step 2:
Embeddings

Chunks → Vectors
Each text chunk is passed through an embedding
model, which converts it into a list of numbers called a
vector. These vectors capture the meaning of the text
so similar ideas end up close together in vector space.
Module 5: Vector Databases

Step
3
Storing in a Vector Database

Once embeddings are created, they are saved into a Vector


Database for fast retrieval.
• Each embedding is stored as a point in vector space
• The database is optimized to find nearest neighbors quickly
• Enables lightning-fast semantic search at scale
Module 6: RAG Pipeline

Step 4: Query
& Retrieval
How the System Finds Answers

🔍 User types a question → it becomes an embedding


📡 System searches the vector database for closest
✅ Most relevant chunks are retrieved and highlighted
chunks

💡 Only the best-matching pieces are passed to the AI!

RAG Step 4 of 5
Module 6: RAG Pipeline

Step 5: LLM Generates


Answer

How AI Crafts the Final Response

The LLM receives retrieved chunks + your question, then


generates a clear, accurate answer using only the relevant
information found — like a student reading notes before
answering!

@reallygreatsite
Module 6: RAG

RAG
Applications

School & Work Chatbots Legal & Medical Search

AI chatbots answer student RAG helps search legal contracts,


questions, help employees find medical records, and research
@reallygreatsite company info, and support papers for precise, relevant
customer service teams instantly. answers.
Module 6: RAG

Open-
Book
Exam
Analogy
RAG = AI with an Open Book

Just like a student looks up notes before answering exam questions,

📖
RAG lets AI search its knowledge base before responding.
Without RAG: AI answers from memory only — may be wrong or

📚
outdated.
With RAG: AI finds the right notes first, then gives an accurate,
grounded answer!
Module 6: RAG – Classroom Activity

Classro
om
Activity
When to Use RAG?

📌
Should AI answer from memory or use RAG? Discuss each scenario:

📌
Q1: "Summarise the French Revolution" — Memory or RAG?

📌
Q2: "Answer from my uploaded PDF notes" — Memory or RAG?

📌
Q3: "What did our teacher say last week?" — Memory or RAG?
Q4: "Explain gravity" — Memory or RAG?
University Presentation

Module 7 – Structured Outputs

Structured
Outputs
Why AI needs structured data,
not just plain text
Module 7: Structured Outputs

Plain Text is Hard


for Computers

The Problem
Imagine this plain text:
Name: Riya
Age: 14
Job: Student
Can a computer easily read this? 🤔 Plain text has no structure —
machines can't tell what is a name, what is a number, or what is a
job!

Module 7 • Page 27
Module 7: Structured Outputs

JSON
Format

Structured Format: JSON

Keys and values make data clear and easy for


computers to read. Each piece of information has a
label (key) and its content (value), making it perfectly
organized!
Module 7: Structured Outputs

Comparing Data
Formats
Plain Text · Markdown · JSON · XML · CSV

📄 Plain Text: General writing | Simple, readable | Notes,


📝 Markdown: Formatted docs | Easy styling | README,
drafts

🔷 JSON: Data exchange | Structured, key-value | APIs,


blogs

🔶 XML: Complex data | Hierarchical, flexible | Enterprise,


configs

📊 CSV: Tabular data | Compact, spreadsheet-ready |


web services

Databases, imports
Module 7: Structured Outputs

Real-World
Examples
Invoice & Resume Parsing Chatbots & Database Updates

AI extracts structured data from Chatbot responses use structured


invoices automatically and parses JSON; AI updates databases and
resumes into neat fields like name, imports spreadsheets automatically.
@reallygreatsite
skills, and experience.
Module 7: Structured Outputs

Output Parsers
as Translators

What is an Output Parser?

AI speaks in plain text — but computers need structured

🔄
data!
Output parsers act like translators:
• Take messy AI text as input
• Convert it into clean, structured format (JSON, CSV, etc.)
• Make data usable by other programs instantly

→ →

AI Text Output Parser
Structured Data
Module 7 – Structured Outputs

Activity!

🎯 Which Format is Best?


Look at these 3 AI outputs below. Which is easiest for a
computer program to use?
① Plain Text② Markdown ③ JSON
Discuss with your group and share your answer!
University Presentation

How apps talk to each other

Module 8:
Understanding
APIs
Connecting the Digital World
Module 8: Understanding APIs
Rules for Communication

APIs define a set of rules that apps follow to request and share
data with each other.

What is an API?
Connects Two Apps

Like a bridge, an API links two different apps so they can


exchange information seamlessly.

Standardized Interface

Any app can use an API without knowing how the other app
works internally — just follow the rules!

@Reallygreatsite
Module 8: APIs

Key API Terms

Client & Server Request, Response & Endpoint

Client: Your device sending a request. Server: Request: question sent. Response: answer
The service provider that receives and received. Endpoint: the address where the API
responds to it. lives.
@reallygreatsite
Module 8: APIs

Restaurant
Analogy

How It All Maps Together

🍽️ Customer = Client (makes a request)


👨‍🍳 Waiter = API (carries the request)
🏠 Kitchen = Server (processes & prepares)
🍕 Food = Response (delivered back to you)
Just like ordering food, apps use APIs to request and receive
data!

@reallygreatsite
Module 8: Understanding APIs

Request-Response
Cycle

How the Flow Works


[Link] sends a Request from their device
[Link] forwards it to the API Endpoint
[Link] passes it to the Server
[Link] processes and sends Response back
[Link] displays the result to the User

@reallygreatsite
Module 8: APIs Why Identify?
APIs need to know who is asking to
keep data safe and prevent misuse
Authentication &
API Keys
Types of Auth
Username & password, API keys,
OAuth tokens — each proves your
identity

API Keys & Tokens


API keys are secret codes passed
with every request to unlock access

@Reallygreatsite
Module 8: APIs

Rate
Limits

The Restaurant Analogy

🍽️ Limited tables = limited API calls


⏳ Too many requests? You must wait!
🚦 Rate limits prevent server overload
✅ Keeps the service fair & fast for all
Module 8: APIs

Real-World APIs & Activity


Where do we use APIs every
day?

🌦 Weather API – live forecasts


🗺 Google Maps API – navigation
💳 Payment API – online checkout
🤖 OpenAI API – AI responses
🌐 Translation API – language tools
📝 Activity: Spot the API!
Food ordering? Ride booking? Google login? Weather
check? WhatsApp? — Which app uses which API?
Modules 5–8 Complete!

Thank You &


Next Steps!
Explore: AI Automation & AI
Agents

Try workflows with n8n

Stay curious — next lecture coming soon!

You might also like