0% found this document useful (0 votes)
2 views13 pages

quick_reference_guide.md

The document serves as a quick reference guide for understanding key concepts related to Large Language Models (LLMs), including their architecture, tokenization, embeddings, and Retrieval-Augmented Generation (RAG). It also covers AI agents, temperature and sampling techniques, and provides code patterns for implementing LLMs and RAG systems. Additionally, it addresses common challenges in LLM usage and offers solutions, along with essential terminology and performance metrics.

Uploaded by

preetham93m
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
2 views13 pages

quick_reference_guide.md

The document serves as a quick reference guide for understanding key concepts related to Large Language Models (LLMs), including their architecture, tokenization, embeddings, and Retrieval-Augmented Generation (RAG). It also covers AI agents, temperature and sampling techniques, and provides code patterns for implementing LLMs and RAG systems. Additionally, it addresses common challenges in LLM usage and offers solutions, along with essential terminology and performance metrics.

Uploaded by

preetham93m
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

# Aiotrix Interview - Quick Reference Guide

## Essential Concepts at a Glance

### LLMs (Large Language Models)

**What:** Neural networks trained on massive text to predict next word


**Size:** Billions of parameters (e.g., GPT-4 has ~1.7 trillion)
**Training:** Transformer architecture + next-token prediction
**Strength:** Pattern recognition, text generation, reasoning
**Weakness:** Hallucinations, outdated info, no real reasoning

**Key Models:**
- GPT-4-turbo: Most capable, $0.01/1K input
- Claude 3.5 Sonnet: Good reasoning, $3/1M input
- Llama 3: Open source, free to use

---

### Tokens & Context

**Token:** ~4 characters or 1 word (varies by tokenizer)


**Context Window:** How much text LLM can see at once
- GPT-4: 128K tokens
- Claude: 200K tokens
- Llama: 4K-128K tokens

**Cost Formula:** `tokens / 1000 * price_per_1k`

Example: 1000 tokens at $0.01 per 1K tokens = $0.01

---

### Embeddings

**What:** Convert text to vector (list of numbers)


**Similarity:** Similar texts → similar vectors
**Uses:** Search, clustering, RAG retrieval

```
"I love pizza" → [0.23, 0.45, -0.12, 0.89, ...]
"Pizza is great" → [0.25, 0.43, -0.10, 0.87, ...] (similar!)
"The weather is nice" → [0.01, 0.02, 0.03, ...] (different)
```

**Common Sizes:** 384, 768, 1536 dimensions

---

### RAG (Retrieval-Augmented Generation)

**Problem Solved:** LLM outdated knowledge, hallucinations

**Flow:**
```
User Question

Convert to embedding

Search vector database

Retrieve top 3-5 relevant chunks

Add to LLM prompt: "Using these documents: [chunks]"

LLM generates grounded answer
```

**Why it matters:** Production systems need accurate, current info

---

### AI Agents

**What:** AI that can use tools to solve complex tasks

**Cycle:**
```
1. Understand goal
2. Decide which tools needed
3. Execute tools
4. Analyze results
5. Decide next action (repeat)
6. When done → return answer
```

**Example:**
```
User: "What was the weather yesterday in Bengaluru and book a flight?"

Agent thinking:
→ Need: weather_api, flights_api
→ Call both
→ Got: Sunny 28°C, flights available
→ Summary: "Yesterday was sunny. Found 5 flights for tomorrow."
```

---

### Temperature & Sampling

**Temperature (0-2):**
- 0: Always same answer (deterministic)
- 0.7: Good balance (default)
- 1.5+: Very creative/random

**When to use:**
- Facts/Code: temp = 0-0.3
- Creative: temp = 1.0-1.5

**Top-P (Nucleus sampling, 0-1):**


- 0.9: Consider top 90% probability tokens
- Lower = more focused
- Default: 0.9

---
## Code Patterns Cheat Sheet

### 1. Basic LLM Call

```python
from openai import OpenAI

client = OpenAI()
response = [Link](
model="gpt-4-turbo",
messages=[
{"role": "system", "content": "You are helpful."},
{"role": "user", "content": "Explain RAG"}
],
temperature=0.7
)
print([Link][0].[Link])
```

### 2. Prompt Template

```python
prompt = f"""
You are a {role}.
Topic: {topic}

Question: {user_question}

Answer:
"""
```

### 3. Error Handling with Retries

```python
import time

def call_with_retry(func, max_retries=3):


for attempt in range(max_retries):
try:
return func()
except Exception as e:
if attempt < max_retries - 1:
wait = 2 ** attempt
print(f"Error: {e}. Retry in {wait}s...")
[Link](wait)
else:
raise
```

### 4. Text Chunking

```python
def chunk_text(text, chunk_size=500, overlap=100):
chunks = []
for i in range(0, len(text), chunk_size - overlap):
chunk = text[i:i + chunk_size]
[Link](chunk)
return chunks
```

### 5. Vector Similarity

```python
import numpy as np

def cosine_similarity(vec1, vec2):


return [Link](vec1, vec2) / ([Link](vec1) *
[Link](vec2))

# Returns: 1.0 (identical), 0.0 (orthogonal), -1.0 (opposite)


```

### 6. Basic RAG Function

```python
def simple_rag(query, documents, client):
# 1. Get query embedding
query_vec = get_embedding(query, client)

# 2. Find similar docs


similarities = [cosine_similarity(query_vec, get_embedding(doc,
client))
for doc in documents]
top_docs = [doc for doc, _ in sorted(zip(documents, similarities),
key=lambda x: x[1], reverse=True)[:3]]

# 3. Create prompt
context = "\n".join(top_docs)
prompt = f"Using these docs:\n{context}\n\nAnswer: {query}"

# 4. Generate answer
response = [Link](
model="gpt-4-turbo",
messages=[{"role": "user", "content": prompt}]
)
return [Link][0].[Link]
```

### 7. Function Calling (Tool Use)

```python
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get weather for location",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string"}
},
"required": ["location"]
}
}
}
]

response = [Link](
model="gpt-4-turbo",
messages=[{"role": "user", "content": "Weather in Bengaluru?"}],
tools=tools
)

# Check if tool was called


if [Link][0].message.tool_calls:
for call in [Link][0].message.tool_calls:
print(f"Tool: {[Link]}")
print(f"Args: {[Link]}")
```

### 8. Conversation Memory

```python
class ChatBot:
def __init__(self):
[Link] = []
[Link] = OpenAI()

def chat(self, user_message):


[Link]({
"role": "user",
"content": user_message
})

response = [Link](
model="gpt-4-turbo",
messages=[Link]
)

answer = [Link][0].[Link]
[Link]({
"role": "assistant",
"content": answer
})

return answer

# Usage
bot = ChatBot()
[Link]("My name is Preetham")
[Link]("What's my name?") # Remembers!
```

### 9. API with FastAPI

```python
from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()
class QueryRequest(BaseModel):
question: str

@[Link]("/ask")
async def ask(request: QueryRequest):
answer = [Link]([Link])
return {"answer": answer}

# Run: uvicorn main:app --reload


# Test: curl -X POST [Link] -H "Content-Type:
application/json" -d '{"question":"What is RAG?"}'
```

### 10. Chroma Vector Store

```python
import chromadb

client = [Link]()
collection = client.create_collection(name="docs")

# Add documents
[Link](
ids=["1", "2", "3"],
documents=[
"RAG improves LLM accuracy",
"Embeddings convert text to vectors",
"Agents can use tools"
]
)

# Search
results = [Link](
query_texts=["How does RAG work?"],
n_results=2
)

print(results["documents"][0]) # Most relevant docs


```

---

## Key Interview Questions & Quick Answers

### Q: "Explain Transformers"

**30-second version:**
"Transformers use self-attention to process text in parallel. Self-
attention lets each word 'look at' other words to understand context. The
model learns which words matter for which tasks through training."

**2-minute version:**
"Transformers have:
1. **Embedding layer** - converts words to vectors
2. **Self-attention** - each position looks at all other positions
3. **Feed-forward layers** - additional processing
4. **Multiple layers** - stack of these blocks
5. **Output layer** - predicts next token (for LLMs)
The key insight: **Self-attention** lets the model weigh importance of
different words contextually. Unlike RNNs, it processes all words in
parallel, making it fast."

---

### Q: "What are Tokens?"

"Tokens are chunks of text. OpenAI's tokenizer typically counts:


- 1 token ≈ 4 characters
- 1 token ≈ 0.75 words

Example:
'Hello world!' = 3 tokens: 'Hello' (1), 'world' (1), '!' (1)

**Why it matters:**
- Costs calculated per token
- Context windows limited by tokens
- Affects API pricing and latency"

---

### Q: "Why does RAG matter?"

"RAG solves three core problems with LLMs:

1. **Outdated knowledge** - LLMs trained until cutoff date


2. **Hallucinations** - LLM confidently invents facts
3. **Private data** - LLM doesn't know your company docs

**Solution**: Retrieve relevant documents before prompting LLM. This


grounds responses in real data.

**Real use case**: Customer service chatbot needs current product info
and company policies. RAG ensures it pulls correct docs before
answering."

---

### Q: "How would you build a RAG system?"

**Step-by-step:**

1. **Ingest** - Read and parse documents


2. **Chunk** - Split into 500-1000 char chunks with overlap
3. **Embed** - Convert chunks to vectors using embedding model
4. **Store** - Save vectors in DB (Chroma, Pinecone, etc.)
5. **Query** - Convert user query to vector
6. **Retrieve** - Find top 3-5 similar chunks
7. **Prompt** - Add chunks to LLM prompt: "Using these docs: [chunks]
Answer: [question]"
8. **Generate** - LLM generates grounded answer

**Optimization considerations:**
- Chunk size (too small = loss of context, too large = noise)
- Embedding model (smaller = faster, larger = better quality)
- Retrieval count (3-5 usually good)
- Metadata filtering (only search relevant documents)
- Reranking (re-score retrieved docs)

---

### Q: "What are AI Agents?"

"Agents are AI systems that can:


1. **Understand goals** - Parse user request
2. **Plan** - Decide what tools needed
3. **Execute** - Call tools/APIs
4. **Observe** - Get results
5. **Reason** - Decide next steps
6. **Repeat** - Until goal achieved

**Example flow:**
```
User: "Book me a flight to Delhi tomorrow"
Agent: "Need weather + flights. Calling both APIs..."
Agent: "Got data. Delhi is rainy. Found 5 flights."
Agent: "Recommending evening flight (avoids rain delay)"
```

**vs Basic Chatbot:**


- Chatbot: Just generates text
- Agent: Generates text AND takes actions via tools"

---

### Q: "Explain Function Calling / Tool Use"

"It's how LLMs call external functions/APIs:

1. You define available tools (schema)


2. LLM decides which tool(s) to use
3. LLM returns function name + arguments
4. You execute function
5. Pass result back to LLM
6. LLM generates final answer

**Example:**
```
Tools: [calculator, weather_api]
User: "What's 15 * 8? And weather in Delhi?"

LLM response:
- Call calculator(15, 8) → 120
- Call weather_api(Delhi) → 28°C, Sunny

LLM: "15 * 8 = 120. Delhi is 28°C and sunny."


```

**Key advantage**: LLM doesn't hallucinate answers; it fetches real


data."

---

### Q: "What's the difference between Temperature and Top-P?"


**Temperature:**
- 0 = Always pick highest probability token (deterministic)
- 1 = Normal distribution
- 2 = Very random

Use: Lower for facts, higher for creative writing

**Top-P (Nucleus Sampling):**


- 0.9 = Consider top 90% probability tokens (ignore bottom 10%)
- 0.5 = More focused
- 1.0 = Consider all tokens

**Together**: Usually use either temperature OR top_p, not both. Default


is usually fine (temp=0.7, top_p=1.0).

---

## System Design Patterns

### Pattern 1: Simple Chatbot

```
User Input

Chat History (add user message)

LLM (with system prompt)

Response + Save to history

Display
```

### Pattern 2: RAG-Based Q&A

```
Document Upload

Chunk + Embed

Store in Vector DB

User Question

Retrieve Similar Chunks

Add to LLM Prompt

Generate Answer
```

### Pattern 3: Agent with Tools

```
User Goal

LLM Decides Tools

Execute Tools (parallel possible)

Observation

LLM Decides Next Action

Loop until done

Final Answer
```

---

## Performance Metrics

### LLM Quality Metrics

| Metric | How | When |


|--------|-----|------|
| **BLEU** | Compare generated text to reference | Translation,
summarization |
| **ROUGE** | Overlap of n-grams | Summarization |
| **Exact Match** | Generated == Reference | Q&A |
| **Human Eval** | Humans rate quality | Most accurate but slow/expensive
|
| **Latency** | Response time | Real-time apps |
| **Token/sec** | Tokens per second | Throughput |

### RAG Metrics

| Metric | Meaning | Good Values |


|--------|---------|------------|
| **Recall** | % of relevant docs retrieved | > 0.7 |
| **Precision** | % retrieved docs are relevant | > 0.5 |
| **MRR** | Rank of first relevant doc | > 0.7 |

---

## Common Challenges & Solutions

### Challenge: Poor RAG Retrieval

**Causes:**
- Chunks too long/short
- Query doesn't match doc language
- Low-quality embedding model
- No metadata filtering

**Solutions:**
- Test different chunk sizes
- Hybrid search (keyword + semantic)
- Use better embedding model
- Add metadata filters
- Query expansion (rephrase with LLM)

---
### Challenge: LLM Hallucinations

**Causes:**
- LLM trained to generate plausible text
- Topic outside training data
- Incomplete context

**Solutions:**
- Use RAG (ground in documents)
- Add "If you don't know, say so" to prompt
- Use lower temperature
- Fact-check with tools
- Use Claude/GPT-4 (less hallucination than smaller models)

---

### Challenge: High API Costs

**Causes:**
- Many API calls
- Large prompts
- Long conversations

**Solutions:**
- Use smaller models for simple tasks
- Reduce context (summarize old messages)
- Batch similar queries
- Cache prompts (if using newer models)
- Use open-source models locally

---

### Challenge: Slow Responses

**Causes:**
- Large embedding model
- Many documents to retrieve
- Multiple API calls
- Synchronous processing

**Solutions:**
- Use faster embedding model
- Limit retrieval to fewer docs
- Parallelize API calls (async)
- Cache embeddings
- Reduce prompt length

---

## Must-Know Terminology

| Term | Meaning |
|------|---------|
| **LLM** | Large Language Model (GPT-4, Claude, etc.) |
| **Embedding** | Vector representation of text |
| **Vector DB** | Database storing vectors (Chroma, Pinecone) |
| **Token** | ~4 characters of text |
| **Context Window** | Max tokens model can see |
| **RAG** | Retrieval-Augmented Generation |
| **Agent** | AI that uses tools to solve tasks |
| **Function Calling** | LLM calling external APIs |
| **Prompt Engineering** | Writing better prompts |
| **Few-Shot** | Giving examples in prompt |
| **Chain-of-Thought** | Asking to think step-by-step |
| **Temperature** | Randomness in responses (0-2) |
| **Hallucination** | LLM inventing false facts |
| **Fine-tuning** | Retraining LLM on custom data |
| **Transformer** | Architecture behind LLMs |
| **Self-Attention** | Mechanism for context understanding |
| **Tokenizer** | Converts text to tokens |

---

## Quick Wins for Interview

**Do these before interview:**

1. **Run a working RAG system** (5 min)


```bash
# Clone example
git clone [Link]
python examples/[Link]
```

2. **Practice explaining concepts** (10 min)


- Record yourself explaining RAG, agents, embeddings
- Notice where you stumble
- Practice those parts

3. **Review your projects** (15 min)


- Know your code by heart
- Can explain every decision
- Prepared for "why did you do X?"

4. **Prepare examples** (10 min)


- Have 2-3 real examples ready
- "In my RAG project, I..."
- Specific metrics/results

5. **Think about questions** (5 min)


- 3 thoughtful questions for them
- Shows you researched the role

---

## Final Checklist

Before interview:

- [ ] Can explain all 5 core concepts (LLM, RAG, Agent, Embedding, Token)
- [ ] Can code a basic LLM call from memory
- [ ] Can discuss trade-offs (cost vs quality, speed vs accuracy)
- [ ] Have 3 working projects
- [ ] Can describe your biggest challenge & how you solved it
- [ ] Know Aiotrix's mission & products
- [ ] Have 3 thoughtful questions for them
- [ ] Practiced talking through your work

---

Good luck!

You might also like