Beginner’s Guide to RAG
AI by Hand ✍
Prof. Tom Yeh
Hosted by:
Download Slides
[Link]
2
Roadmap
Q/A
RAG Equation
Load
Prompt Advanced
Split
Index Retrieval
Embed
Store
RAG Equation
Beginner’s Guide to RAG - AI by Hand ✍
Download: [Link]
RAG = Query + Prompt + Context + LLM
How much does [INST] <<SYS>> You are a _______, respectful, ______
Tom make? assistant. Always answer as helpfully as possible,
while being safe. Your answers should ___ include
Search any harmful, _____________, racist, sexist, toxic,
dangerous, or illegal content. Please ensure that
your responses are socially _________ and positive
in nature. <</SYS>>
Please answer the following question by using
Database information from the provided context information!
CONTEXT_INFORMATION:
LLM
Amy, 100, CEO
Tom, 50, Prof
Sam, 120, CTO QUESTION:
[/INST]
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
RAG = Query + Prompt + Context + LLM
How much does [INST]
Tom make? Please answer the following question by using
information from the provided context information!
Search CONTEXT_INFORMATION:
QUESTION:
[/INST]
Database LLM
Amy, 100, CEO
Tom, 50, Prof
Sam, 120, CTO
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
RAG = Query + Prompt + Context + LLM
How much does [INST]
Tom make? QUESTION:
Search [/INST]
LLM
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Prompt
Beginner’s Guide to RAG - AI by Hand ✍
Download: [Link]
Summarize
Context information from ________ sources is below.
---------------------
{context_str}
---------------------
Given the information from multiple sources and not
__________________, answer the query.
Query: {query_str}
Answer:
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Single Choice
Some choices are given below. It is provided in a numbered list
(1 to {num_chunks}),
where each item in the list corresponds to a summary.
---------------------
{context_list}
---------------------
Using only the choices above and not prior knowledge, return the
_________ that is most relevant to the question:
{query_str}
Provide choice in the following format: 'ANSWER: <number>' and
____________ why this choice was selected in relation to the
question.
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Refine
The original query is as follows: {query_str}
We have provided an existing answer: {existing_answer}
We have the opportunity to _________ the existing answer
with some more context below.
------------
{context_str}
------------
Given the new context, refine the original answer to better answer
the query. If the context isn't useful, return the _________ answer.
Refined Answer:
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Index
Beginner’s Guide to RAG - AI by Hand ✍
Download: [Link]
id: 1 id: 2 id: 2
Load
date: 5/10 date: 5/15 date: 5/17
大人吃大包 每天都吃一 高山，大海
子，小孩吃 個蘋果，可 ，森林，動
小包子 以保持健康 物，大自然
Split 每天都
吃一個
蘋果可
高山大
大海森
林動物
以保持 大自然
Embed 1 1 0 1 1 -2
1 0 1 1 1 1
0 0 1 0 0 1
0 1 0 0 1 1
Store Chunk Embed Meta
{id:_, date: _____}
{id:_, date: _____}
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Load
Beginner’s Guide to RAG - AI by Hand ✍
Download: [Link]
LlamaHub Data Loaders
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Which loader to use to read CSV data?
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Which loader to use to read Word documents?
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Split
Beginner’s Guide to RAG - AI by Hand ✍
Download: [Link]
Split by Tokens
How are you doing today? I hope you're doing
TokenTextSplitter( well. Whether your day is busy or calm, I hope
chunk_size=4, you find a moment of joy. Life moves fast, with
chunk_overlap=0 many ups and downs. It's important to stop and
) enjoy the little things. Maybe today brought a
good chat, a tasty meal, or a beautiful sunset.
Cherish these happy moments. Each day is a
chance to make good memories and connect
with others. Take time to appreciate your
surroundings and the people in your life. Even
on challenging days, there are opportunities to
learn and grow. Let's embrace each experience
and look forward to tomorrow with optimism.
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Split by Tokens
How are you doing today? I hope you're doing
TokenTextSplitter( well. Whether your day is busy or calm, I hope
chunk_size=4, you find a moment of joy. Life moves fast, with
chunk_overlap=1 many ups and downs. It's important to stop and
) enjoy the little things. Maybe today brought a
good chat, a tasty meal, or a beautiful sunset.
Cherish these happy moments. Each day is a
chance to make good memories and connect
with others. Take time to appreciate your
surroundings and the people in your life. Even
on challenging days, there are opportunities to
learn and grow. Let's embrace each experience
and look forward to tomorrow with optimism.
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Embed
Beginner’s Guide to RAG - AI by Hand ✍
Download: [Link]
Massive Text Embedding Benchmark
[Link]
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Which models have 7B parameters?
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Which model has the most parameters?
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Which model supports the longest context
window?
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Which model has the best classification
performance?
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Which model has the best clustering
performance?
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Store
Beginner’s Guide to RAG - AI by Hand ✍
Download: [Link]
Choose a Vector Database
[Link]
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Which database can fit 3072 dimensions?
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Retrieval
Beginner’s Guide to RAG - AI by Hand ✍
Download: [Link]
SQL – Retrieve by Similarity
________ comment, emb<___>[0, 2, 1, 1] AS score
FROM posts
ORDER BY ______ ASC | DESC ;
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
High-level API: Superlinked
query = Query(post_index)
._________(post)
._________(relevance_space.text, Param("________"))
[Link](query, _________ = "how are you?" )
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Advanced RAGs
Beginner’s Guide to RAG - AI by Hand ✍
Download: [Link]
RAG
DB
Prompt
doc-
doc-
doc-
Query
LLM
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Multiple Embedding Spaces
DB
Prompt
doc-
doc-
doc-
Query
LLM
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Superlinked Vector Compute
Memory
Search
ID Description Headline Recency
Output 1
2
0 1 1 0 0 1 0 1 0
1 1 0 0 1 0 0 1 0
3
df = result._________() 0 1 1 1 0 0 1 0 1
__________(df)
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
RankGPT
DB
Prompt
doc-21
doc-42
doc-39
Query
LLM
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
RankGPT
Search Query: {query}.
Rank the {num} passages above based on their _____________ to the
search query. The passages should be listed in _____________ order
using identifiers.
The most relevant passages should be listed first.
The output format should be [] > [], e.g., [1] > [2].
Only response the ranking results,
do not ___________.
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Multi-Query Retrieval
DB
Prompt
doc-21
doc-42
doc-39
Query
LLM
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Multi-Query Retrieval
You are an AI language model assistant. Your task is to generate ____
different versions of the given user question to retrieve relevant
documents from a vector database.
By generating multiple _____________ on the user question, your goal
is to help the user overcome some of the limitations of _____________
search.
Provide these alternative questions seperated by newlines.
Original question: {question}
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Contextual Compression
DB
Prompt
doc-21
doc-42
doc-39
Query
LLM
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Hypothetical Document Embeddings (HyDE)
DB
Prompt
doc-
doc-
doc-
Query
LLM
AI by Hand ✍ 2024 © Tom Yeh Download: [Link]
Q/A
Beginner’s Guide to RAG - AI by Hand ✍
Download: [Link]