Modern AI Application Development
Learning Notes on RAG, LangChain, LangGraph & LangFuse
A beginner-friendly conceptual guide — what these technologies are, why they exist, and how they fit
together in a real AI application.
Prepared as concise, audio-friendly learning notes
Contents
• 1. AI Foundations
• 2. Retrieval-Augmented Generation (RAG)
• 3. LangChain
• 4. LangGraph
• 5. LangFuse
• 6. How Everything Works Together
• 7. Glossary
• 8. Cheat Sheet
1. AI Foundations
Before diving into RAG and the LangChain ecosystem, it helps to have a shared vocabulary. This section moves
quickly through the foundational ideas — just enough to understand the rest of the document.
Artificial Intelligence (AI)
Simple explanation: AI is the broad field of building systems that perform tasks normally requiring human
intelligence — understanding language, recognizing images, making decisions.
Why it exists / problem it solves: Many real-world tasks are too complex or variable for hand-written rules. AI lets
systems learn patterns from data instead of being explicitly programmed for every case.
Where it fits: AI is the umbrella term. Everything else in this document — ML, deep learning, LLMs — is a subfield
inside it.
Machine Learning (ML)
Simple explanation: ML is a branch of AI where a system learns patterns from data rather than following hard-
coded rules.
How it works (high level): You feed the system examples (data). It adjusts its internal parameters to reduce errors.
Over time it generalizes — it can handle new, unseen examples reasonably well.
Common terms: training, model, dataset, prediction.
Deep Learning
Simple explanation: Deep learning is a type of ML that uses artificial neural networks with many layers to learn very
complex patterns — especially useful for language, images, and audio.
Why it exists: Traditional ML struggled with unstructured data like raw text or pixels. Deep neural networks,
especially the 'transformer' architecture, made it possible to model language at scale.
Where it fits: Deep learning is the engine behind modern LLMs.
Large Language Models (LLMs)
Simple explanation: An LLM is a deep learning model trained on massive amounts of text that can understand and
generate human-like language.
Why it exists / problem it solves: Earlier language models were narrow and brittle. LLMs, trained on huge and
diverse datasets, generalize across many tasks — writing, summarizing, coding, reasoning — without task-specific
retraining.
How it works (high level): An LLM predicts the next most likely piece of text given everything before it, one step at
a time. Trained on enough data, this simple mechanism produces coherent, useful responses.
Where it fits: The LLM is usually the 'brain' at the center of an AI application — it's what LangChain, LangGraph,
RAG, and LangFuse are all built around.
Key limitation: LLMs don't 'know' things the way databases do — they generate plausible text based on patterns,
which is why they can be wrong confidently (see Hallucinations below).
Tokens
Simple explanation: A token is a small chunk of text — often a word, part of a word, or punctuation — that an LLM
processes as its basic unit.
Why it matters: LLMs don't read letters or full words directly; they read sequences of tokens. Cost, speed, and
limits are usually measured in tokens, not words or characters.
Common misconception: 1 token ≠ 1 word. On average, 1 token is roughly 3–4 characters of English text, so a
paragraph might be 40 words but 55+ tokens.
Context Window
Simple explanation: The context window is the maximum amount of text (measured in tokens) an LLM can 'see' at
once — including the prompt, conversation history, and any retrieved documents.
Why it exists / problem it solves: Models have a limited working memory. The context window defines how much
information you can feed in before older information gets dropped or ignored.
Where it fits: This limit is a core reason RAG exists — you can't just paste your entire company's documents into a
prompt, so you retrieve only the most relevant pieces to fit the window.
Embeddings
Simple explanation: An embedding is a way of turning text (a word, sentence, or document) into a list of numbers
(a vector) that captures its meaning.
Why it exists / problem it solves: Computers can't compare 'meaning' directly. Embeddings place similar meanings
close together in a mathematical space, so 'dog' and 'puppy' end up near each other, while 'dog' and 'stock market'
end up far apart.
How it works (high level): A specialized embedding model reads text and outputs a fixed-length vector (e.g., 768 or
1536 numbers). Texts with similar meaning produce vectors that are mathematically close together.
Where it fits: Embeddings are the foundation of semantic search and RAG retrieval.
Vector Databases
Simple explanation: A vector database is a specialized database built to store embeddings and quickly find the ones
most similar to a given query vector.
Why it exists / problem it solves: Traditional databases are built for exact matches (SQL WHERE clauses) or
keyword search. They're not efficient at answering 'which of these million vectors is closest to this one?' Vector
databases are optimized exactly for that.
Where it fits: Vector databases store the knowledge base used in RAG — chunks of documents converted into
embeddings, ready to be searched at query time.
Semantic Search
Simple explanation: Semantic search finds results based on meaning, not just matching keywords.
Why it exists / problem it solves: Keyword search fails when the user's words don't match the document's exact
words (e.g., searching 'car' won't find a document that only says 'automobile'). Semantic search solves this by
comparing meaning via embeddings.
How it works (high level): Convert the query into an embedding, then find the closest stored embeddings in the
vector database.
Hallucinations
Simple explanation: A hallucination is when an LLM generates text that sounds confident and fluent but is factually
wrong or fabricated.
Why it happens: LLMs generate the statistically likely next words, not verified facts. If they lack real knowledge on
a topic, they may still produce a plausible-sounding but incorrect answer.
Why it matters: This is one of the biggest reasons RAG exists — grounding the model in real, retrieved documents
reduces (but doesn't fully eliminate) hallucinations.
Common mistake: Assuming RAG completely eliminates hallucinations. It significantly reduces them by giving the
model real context, but a model can still misread or misuse the retrieved information.
2. Retrieval-Augmented Generation (RAG)
What RAG Is
Simple explanation: RAG is a technique that gives an LLM access to external, up-to-date, or private information at
the moment it answers a question — instead of relying only on what it memorized during training.
In one line: Retrieve relevant information first, then generate an answer using that information.
Why RAG Exists
LLMs have three core limitations that RAG addresses:
• Knowledge cutoff: LLMs only know what existed in their training data up to a certain date.
• No private data: LLMs have never seen your company's internal documents, databases, or personal files.
• Hallucination risk: Without grounding, LLMs may confidently invent facts.
RAG solves all three by fetching real, relevant, current information and handing it to the model as context before it
answers.
Complete RAG Architecture (High-Level Flow)
A RAG system has two distinct phases: an offline 'indexing' phase (done once, or periodically) and an online 'query'
phase (done every time a user asks something).
Phase 1 — Indexing (offline, done ahead of time)
1. Collect source documents (PDFs, web pages, tickets, wikis, etc.).
2. Chunking: split documents into smaller pieces.
3. Embeddings: convert each chunk into a vector.
4. Store the vectors (plus original text) in a vector database.
Phase 2 — Query (online, happens per user question)
1. User asks a question.
2. Convert the question into an embedding.
3. Vector search: find the most similar chunks in the vector database (retrieval).
4. Prompt augmentation: insert those chunks into the prompt sent to the LLM.
5. Generation: the LLM produces an answer grounded in the retrieved chunks.
Chunking
Simple explanation: Chunking is splitting large documents into smaller, manageable pieces before embedding
them.
Why it matters: Embedding an entire 50-page document as one vector loses detail and precision. Smaller chunks
let retrieval pinpoint the exact relevant passage instead of a whole document.
Common approaches: fixed-size chunks (e.g., 500 tokens), sentence/paragraph-based chunks, or 'semantic'
chunking that splits at natural topic boundaries. Overlapping chunks slightly (so context isn't cut mid-thought) is a
common practice.
Common mistake: Chunks too large lose retrieval precision; chunks too small lose context. Finding the right size is
mostly trial and error for your specific content.
Embeddings (in RAG context)
Each chunk is passed through an embedding model to produce a vector. This happens once per chunk during
indexing, and once per user query during search. The embedding model used for the documents and the one used
for the query must be the same (or compatible), otherwise similarity comparisons won't be meaningful.
Retrieval
Simple explanation: Retrieval is the process of fetching the most relevant chunks for a given query.
How it works: The query is embedded, then compared against all stored chunk embeddings using a similarity
metric (commonly cosine similarity). The top-k most similar chunks are returned (e.g., the 3–5 best matches).
Vector Search
Simple explanation: Vector search is the underlying algorithm that efficiently finds the closest vectors in a huge
collection, without comparing against every single one (which would be too slow at scale).
Key term: ANN (Approximate Nearest Neighbor) search — most vector databases use ANN algorithms (like HNSW)
to search millions of vectors in milliseconds, trading a small amount of accuracy for large speed gains.
Prompt Augmentation
Simple explanation: This is the step where retrieved chunks are inserted into the prompt sent to the LLM, usually
with instructions like 'Answer the question using only the following context.'
Why it matters: This is literally where 'retrieval' meets 'generation' — the augmented prompt is what makes the
answer grounded rather than purely memorized.
Generation
Simple explanation: The final step where the LLM reads the augmented prompt (question + retrieved context) and
produces a natural-language answer.
Good practice: Instructing the model to cite which chunk supported each claim, or to say 'I don't know' when the
retrieved context doesn't contain the answer, meaningfully reduces hallucination.
Basic vs. Advanced RAG
Basic RAG Behavior
Query handling Uses the raw user question as-is for retrieval
Basic RAG Behavior
Retrieval Single retrieval pass, fixed top-k chunks
Ranking Relies purely on vector similarity
Reasoning One-shot: retrieve once, then answer
Advanced RAG Behavior
Query handling Query rewriting/expansion for better retrieval (e.g., generating multiple
query variants)
Retrieval Multi-step or iterative retrieval; may re-retrieve based on intermediate
answers
Ranking Adds a re-ranking model on top of vector search to reorder results by true
relevance
Reasoning Can loop: retrieve → reason → retrieve again → answer (this is where
LangGraph becomes useful)
Common Vector Databases
• Pinecone — popular fully-managed cloud vector database.
• Weaviate — open-source, supports hybrid (keyword + vector) search.
• Chroma — lightweight, popular for local development and prototyping.
• Qdrant — open-source, performance-focused.
• Milvus — open-source, built for very large-scale deployments.
• pgvector — a vector extension for PostgreSQL, useful if you already use Postgres.
Key Advantages of RAG
• Keeps answers grounded in real, current, or private data.
• Reduces hallucinations compared to relying on the model's memorized knowledge alone.
• No need to retrain the model when your data changes — just update the vector database.
• More cost-effective than fine-tuning for most 'knowledge injection' use cases.
Key Limitations of RAG
• Retrieval quality caps answer quality — if the right chunk isn't retrieved, the model can't use it.
• Doesn't eliminate hallucination entirely; the model can still misinterpret retrieved text.
• Adds latency (retrieval takes time) and infrastructure complexity (a vector database to maintain).
• Chunking strategy strongly affects quality and requires tuning per use case.
Common Misconceptions
• 'RAG is the same as fine-tuning.' No — fine-tuning changes the model's internal weights; RAG feeds the model
relevant text at query time without changing the model at all.
• 'More retrieved chunks = better answers.' Not always — too much context can dilute relevance or exceed the
context window; quality of retrieval matters more than quantity.
3. LangChain
What It Is
Simple explanation: LangChain is an open-source framework that provides reusable building blocks for creating
LLM-powered applications — prompts, model connections, memory, tools, and more — so developers don't have
to build everything from scratch.
Why It Exists
Problem it solves: Building an LLM application from raw API calls means repeatedly writing boilerplate code for
prompt formatting, connecting to different LLM providers, chaining steps together, managing conversation history,
and connecting to external tools or data. LangChain standardizes these patterns into a consistent, swappable set of
components.
Core Components
LLMs / Chat Models
A standardized interface to call different LLM providers (OpenAI, Anthropic, etc.) using the same code pattern, so
switching providers doesn't require rewriting your application.
Prompts (Prompt Templates)
Reusable templates with placeholders (e.g., 'Summarize the following text: {text}') so you don't hard-code prompts
as raw strings scattered through your code.
Chains
A sequence of steps linked together — e.g., take user input → format a prompt → call the LLM → parse the output.
A chain is the original way LangChain composed multi-step logic.
Runnables (LCEL — LangChain Expression Language)
The modern, unified interface for composing steps using a pipe-like syntax (step1 | step2 | step3). Runnables
replaced many older chain classes with a more flexible, composable approach, and support streaming, batching,
and async execution out of the box.
Documents
A standard object representing a piece of text plus metadata (e.g., source file, page number) — used throughout
LangChain's data-loading and retrieval components.
Retrievers
A standard interface for fetching relevant Documents given a query — this is the component that plugs a vector
database into the rest of a LangChain application, powering the RAG retrieval step.
Tools
Functions the LLM can choose to call — like a web search, calculator, or API request — extending the model
beyond pure text generation into taking actions.
Memory
Mechanisms for retaining conversation history or state across multiple turns, so the application can maintain
context in an ongoing conversation.
Agents
A pattern where the LLM decides, step by step, which tool to use and when, rather than following a fixed
sequence. The model acts as a reasoning engine that plans its own actions.
How It Integrates With RAG
LangChain provides ready-made components for nearly every RAG step: document loaders (for ingesting PDFs,
websites, etc.), text splitters (for chunking), embedding model integrations, vector store integrations (for
storing/searching embeddings), and retrievers (for the retrieval step). This means a developer can build a working
RAG pipeline by connecting existing LangChain components rather than writing custom integration code for every
provider.
Key Advantages
• Huge ecosystem of pre-built integrations (LLM providers, vector databases, tools, document loaders).
• Consistent interfaces make it easy to swap one component (e.g., LLM provider) without rewriting the whole
app.
• Large community, extensive examples, and rapid iteration.
Key Limitations
• Can feel like 'a lot of abstraction' for simple use cases — sometimes a direct API call is simpler.
• Best suited to linear or lightly-branching flows; complex, cyclic, multi-agent logic is where LangGraph (next
section) takes over.
• Rapid framework evolution means APIs and best practices have changed significantly over time, so
documentation/tutorials can go stale quickly.
Common Misconceptions
• 'LangChain is required to use an LLM.' No — you can call an LLM API directly. LangChain is a convenience
layer, not a requirement.
• 'LangChain is a vector database.' No — LangChain connects to vector databases; it doesn't store vectors itself.
4. LangGraph
What It Is
Simple explanation: LangGraph is a framework (built by the LangChain team) for building applications as a graph of
steps — with explicit state, branching, loops, and control — rather than a simple straight-line chain.
Why It Was Created
Problem it solves: Real-world AI applications often aren't linear. An agent might need to retry a failed step, loop
until it gets a good answer, branch based on a decision, hand off between multiple specialized agents, or pause and
wait for human approval. Plain LangChain chains are designed for straight-line or lightly-branching flows; they get
awkward once you need loops, explicit state tracking, or multi-agent coordination. LangGraph was built specifically
to model this kind of complex, stateful, and sometimes cyclic control flow.
Core Concepts
Nodes
A node is a single step or unit of work in the graph — e.g., 'call the LLM', 'retrieve documents', 'run a tool'. Each
node is typically a function that takes the current state and returns an update to it.
Edges
Edges define the path from one node to another. A regular edge always moves to the next node; a conditional
edge decides which node to go to next based on the current state (e.g., 'if the answer is incomplete, go back to
retrieval; otherwise, finish').
State
State is a shared, structured object that flows through the graph, holding everything relevant to the current run
(e.g., the conversation so far, retrieved documents, intermediate results). Every node can read from and write to
this state, which is explicitly defined and typed — unlike LangChain agent loops, where state tends to be implicit
and scattered across the code.
Multi-Agent Workflows
Simple explanation: LangGraph makes it straightforward to have multiple specialized 'agents' (each essentially its
own node or sub-graph) collaborate — e.g., a 'router' agent decides which specialist (a coding agent, a research
agent, a writing agent) should handle a task, and results are passed along the graph.
Why this matters: Splitting a complex task across specialized agents tends to produce more reliable results than
asking one generalist agent to do everything at once.
Human-in-the-Loop
Simple explanation: LangGraph supports pausing execution at a specific node to wait for human review or approval
before continuing — critical for high-stakes actions like sending an email, executing code, or making a payment.
How it works (high level): The graph is paused ('interrupted') before a sensitive node; the current state is saved
(checkpointed); a human reviews and approves or edits; execution then resumes exactly where it left off.
How It Differs From LangChain
LangChain Characteristic
Structure Mostly linear chains / pipe-style composition
State handling Implicit, often passed along in the runnable chain
Best for Straightforward pipelines: prompt → LLM → parse; simple RAG
Control flow Limited native support for loops/branches
LangGraph Characteristic
Structure Explicit graph of nodes and edges, supports branching and loops
State handling Explicit, typed, shared state object passed through the graph
Best for Complex agents: multi-step reasoning, multi-agent systems, workflows
needing retries or approval
Control flow Native support for conditionals, loops, and pausing (human-in-the-loop)
In practice: LangGraph is built on top of LangChain concepts and often used alongside it — LangChain provides the
individual building blocks (LLM calls, retrievers, tools), while LangGraph orchestrates how those blocks are wired
together into a controllable, stateful flow.
Key Advantages
• Handles loops, retries, and branching naturally — impossible or awkward in a simple chain.
• Explicit, inspectable state makes complex agent behavior easier to debug and reason about.
• Built-in checkpointing enables persistence, resuming after failure, and 'time-travel' debugging (replaying from
an earlier state).
• Native human-in-the-loop support for safety-critical or high-stakes actions.
Key Limitations
• Steeper learning curve than basic LangChain chains.
• Overkill for simple, linear use cases — adds complexity you may not need.
• Debugging a graph with many branches can still be non-trivial without visualization tooling.
Common Misconceptions
• 'LangGraph replaces LangChain.' No — LangGraph typically uses LangChain components inside its nodes;
they're complementary, not competitors.
• 'LangGraph is only for multi-agent systems.' It's also very useful for single-agent workflows that simply need
loops, retries, or approval steps.
5. LangFuse
What It Is
Simple explanation: Langfuse is an open-source observability and evaluation platform for LLM applications — it lets
you see exactly what happened inside your AI application: which prompts ran, what the model returned, how long
each step took, and how much it cost.
Why Observability Matters
Problem it solves: LLM applications are hard to debug with traditional tools. When an agent gives a bad answer, is
it because retrieval failed, the prompt was poorly worded, the model misunderstood, or a tool call broke? Without
visibility into each step, you're guessing. Observability tools like Langfuse capture every step of execution so you
can pinpoint exactly where something went wrong — and track quality, cost, and performance over time as your
application evolves.
Tracing
Simple explanation: A trace is a complete record of everything that happened during one run of your application —
every LLM call, every tool call, every retrieval step, nested in the order they occurred.
Why it matters: When a LangGraph agent has 10 steps and the final answer is wrong, tracing lets you open that
specific run and see exactly which of the 10 steps produced the bad result, instead of re-running the whole thing
blindly.
Logging
Simple explanation: Beyond full traces, Langfuse logs individual events — inputs, outputs, metadata, errors — that
make up each trace. These logs are the raw building blocks that traces are assembled from.
Prompt Versioning
Simple explanation: Langfuse lets you manage prompts centrally and track versions over time — similar to how
code is version-controlled — so you can see how a prompt changed, roll back to a previous version, and know
exactly which prompt version produced which output.
Why it matters: Prompts are frequently tweaked as an application improves. Without versioning, it's hard to know
which prompt was actually used for a given past result, or to safely test a new prompt against the old one.
Evaluation
Simple explanation: Evaluation is the process of scoring how good your application's outputs are — using
automated methods (another LLM acting as a judge, or rule-based checks) or human review/annotation.
Why it matters: 'It works on my test question' isn't enough for production. Evaluation gives a repeatable,
measurable way to check quality across many examples, and to catch regressions when you change a prompt or
model.
Monitoring
Simple explanation: Monitoring is the ongoing tracking of your application's health and quality in production —
dashboards showing trends in error rates, latency, and evaluation scores over time, often with alerts.
Cost and Latency Tracking
Simple explanation: Langfuse automatically tracks how many tokens each call used, what it cost, and how long
each step took.
Why it matters: LLM costs and response times can spiral unexpectedly, especially with RAG (extra retrieval steps)
and multi-agent workflows (many LLM calls per request). Visibility into cost and latency per step helps identify
expensive bottlenecks — for example, discovering that one particular node in a LangGraph workflow accounts for
most of the cost or delay.
Where It Fits
Langfuse doesn't build or run your application logic — it observes it. It typically integrates via a lightweight SDK or
callback handler added to your LangChain or LangGraph code, capturing traces automatically as your application
runs, without requiring you to rewrite your core logic.
Key Advantages
• Open-source and self-hostable, giving full control over sensitive data — important for regulated industries like
healthcare or finance.
• Framework-agnostic — works with LangChain and LangGraph, but also with other stacks.
• Combines tracing, evaluation, prompt management, and cost/latency tracking in one platform.
Key Limitations
• Adds a bit of integration and operational overhead (another system to configure and, if self-hosting,
maintain).
• Automated evaluation (LLM-as-judge) is helpful but imperfect — it can still miss subtle quality issues that
need human review.
Common Misconceptions
• 'Observability is just logging errors.' It's much broader — quality evaluation, cost tracking, and prompt history
are equally important parts of it.
• 'You only need this after something breaks.' Observability set up from the start makes it far easier to catch
problems early and to safely iterate on prompts and models.
6. How Everything Works Together
Each of the four technologies plays a distinct role in a production AI application. Here is a typical end-to-end flow
for a RAG-based assistant:
Typical Production Flow
User → Application → LangChain → LangGraph → RAG → Vector Database → LLM → Langfuse → Response
Responsibility of Each Component
• User: Asks a question or gives an instruction through some interface (chat UI, app, API).
• Application: The overall product — the front-end and backend that receive the request and return a
response.
• LangChain: Supplies the building blocks: the standardized LLM connection, prompt templates, document
loaders, the retriever interface, and any tools the agent can call.
• LangGraph: Orchestrates the control flow — decides the sequence of steps, handles branching (e.g., 'is more
information needed?'), loops (e.g., 'retry retrieval with a reworded query'), and pauses for human approval if
configured.
• RAG (as a process): Within the graph, one or more nodes perform the RAG steps: turning the query into an
embedding, retrieving relevant chunks, and augmenting the prompt.
• Vector Database: Stores the embedded knowledge base and returns the most relevant chunks when queried
during retrieval.
• LLM: Generates the final natural-language answer, grounded in the retrieved context and shaped by the
prompt template.
• Langfuse: Observes everything happening across LangChain and LangGraph — capturing a full trace of the
request, tracking cost and latency per step, enabling prompt versioning, and feeding evaluation data back to
the team so they can improve the system.
• Response: The final, grounded answer is returned to the user — while Langfuse continues logging the full run
in the background for later review.
A Simple Mental Model
• LangChain = the toolbox (individual reusable parts: LLM calls, prompts, retrievers, tools).
• LangGraph = the blueprint/orchestrator (decides the order of steps, handles loops, branches, and approvals).
• RAG = the technique (a pattern for grounding answers in retrieved data, implemented using
LangChain/LangGraph parts).
• Vector Database = the memory (where the knowledge base lives, ready to be searched).
• Langfuse = the observer (watches, measures, and helps improve everything happening across the other
three).
Why This Combination Is Common in Practice
A team rarely needs just one of these. RAG is the technique; LangChain provides the parts to build it; LangGraph
provides the control flow for anything beyond a simple, straight-line pipeline (retries, multi-step reasoning, multi-
agent collaboration, human approval); and Langfuse provides the visibility needed to trust, debug, and improve the
system once it's running in front of real users. Together, they cover the full lifecycle: build → orchestrate →
observe → improve.
7. Glossary
AI Broad field of building systems that perform tasks requiring human-like intelligence.
Agent An LLM-driven system that decides its own sequence of actions/tool calls rather than following a fixed
script.
ANN (Approximate Nearest A fast search algorithm used by vector databases to find similar vectors without checking every one.
Neighbor)
Chain A fixed sequence of steps in LangChain (e.g., prompt → LLM → parse output).
Chunking Splitting large documents into smaller pieces before embedding them for retrieval.
Context Window The maximum amount of text (in tokens) an LLM can process at once.
Embedding A numeric vector representation of text that captures its meaning.
Generation The step where an LLM produces the final natural-language answer.
Hallucination Confident but factually incorrect output from an LLM.
Human-in-the-loop Pausing an automated workflow to get human review or approval before continuing.
LCEL / Runnables LangChain's modern pipe-style syntax for composing steps.
LangChain A framework providing reusable building blocks for LLM applications.
Langfuse An open-source observability and evaluation platform for LLM applications.
LangGraph A framework for building stateful, graph-based agent workflows with branching and loops.
LLM Large Language Model — a deep learning model trained on massive text data to understand and
generate language.
Memory (LangChain) Mechanism for retaining conversation history/state across turns.
Multi-agent workflow Multiple specialized agents collaborating, coordinated via a graph or router.
Node (LangGraph) A single step/unit of work in a LangGraph graph.
Edge (LangGraph) A connection defining which node runs next, sometimes conditionally.
Prompt Augmentation Inserting retrieved context into the prompt sent to the LLM.
Prompt Template A reusable prompt structure with placeholders for dynamic content.
RAG Retrieval-Augmented Generation — retrieving relevant data before generating an answer.
Retriever A LangChain interface for fetching relevant documents given a query.
Semantic Search Search based on meaning (via embeddings) rather than exact keyword matching.
State (LangGraph) A shared, structured object that flows through a graph, updated by each node.
Token A small chunk of text (word, sub-word, or punctuation) that is the basic unit an LLM processes.
Tool A function an LLM/agent can call to take an action beyond text generation.
Tracing A complete recorded timeline of every step in a single run of an application.
Vector Database A database optimized for storing embeddings and finding similar ones quickly.
Vector Search Finding the closest matching vectors to a query vector.
8. Cheat Sheet
The Four Pillars, in One Line Each
Technology One-line summary
RAG The technique: retrieve relevant data, then generate a grounded answer.
LangChain The toolbox: reusable building blocks for LLM apps (LLMs, prompts,
retrievers, tools).
LangGraph The orchestrator: models complex, stateful, branching/looping agent
workflows.
Technology One-line summary
Langfuse The observer: traces, evaluates, and tracks cost/latency of everything
running.
The RAG Pipeline in Order
Chunk → Embed → Store (vector DB) → [Query] → Embed query → Retrieve top-k chunks → Augment prompt →
Generate answer.
When to Reach for Each Tool
• Need to connect to an LLM, format prompts, or plug in a vector database? → LangChain.
• Need branching, loops, retries, multiple agents, or human approval steps? → LangGraph.
• Need to see what happened, measure quality, or track cost/latency in production? → Langfuse.
• Need the model to answer using your own or current data? → RAG (built using the tools above).
Key Numbers & Facts to Remember
• 1 token ≈ 3–4 characters of English text (not 1 word).
• Context window = the model's total 'working memory' for one request, shared by prompt + history + retrieved
chunks.
• Retrieval quality is usually the biggest lever on RAG answer quality — better than just adding more chunks.
• LangGraph state is explicit and typed; plain LangChain agent state tends to be implicit.
Common Pitfalls to Avoid
• Assuming RAG eliminates hallucinations entirely — it reduces, not removes, the risk.
• Confusing RAG with fine-tuning — RAG doesn't change model weights.
• Over-engineering with LangGraph when a simple linear LangChain flow would do.
• Skipping observability until something breaks in production — set up tracing early.
Final Mental Model
Build with LangChain. Orchestrate with LangGraph. Ground answers with RAG and a vector database. Observe and
improve with Langfuse.