LOCAL LLM MASTERY

Context, Memory, MCP & Agentic AI

From Core Knowledge to Career-Ready Skills

A Consolidated Reference for Eduardo

March 2026

Part I — Core Knowledge | Part II — Career Learning Path | Part III — Inflight Lab
 Local LLM Mastery — Core Knowledge

PART I
Core Knowledge
This section covers all the foundational knowledge you need to understand and implement local
LLMs, memory systems, context engineering, MCP, and agentic tooling. No timeline — just
structured concepts and recommended learning resources.

1. Local LLM Fundamentals

1.1 How Local Inference Works

Running an LLM locally means loading the model’s weights into your GPU (or CPU) memory and
performing inference — the forward pass through the transformer architecture — on your own
hardware. There is no cloud API involved; the model file lives on your disk and all computation
happens on-device.
The three critical resources that determine what you can run are VRAM (GPU memory), system
RAM (for overflow and KV cache), and compute throughput (tokens per second). On Apple Silicon,
unified memory means GPU and CPU share the same pool, which simplifies things but also means
the OS competes for the same memory.

Key Concepts
• Quantization: Compressing model weights from 16-bit floats to 4-bit integers (Q4). This
reduces file size by ~75% with minimal quality loss for conversation and reasoning. A 9B
parameter model goes from ~18GB (FP16) to ~5–6GB (Q4). The tradeoff: precision on
structured outputs (JSON, function calling) degrades because the probability distributions
become “fuzzier”.
• KV Cache: The Key-Value cache stores attention state for all tokens in the context window.
This is what grows as conversations get longer. On a 16GB Mac with a 9B model (~6.6GB),
you have ~6–7GB left for KV cache, which supports roughly 16K–32K tokens of context.
Setting OLLAMA_KV_CACHE_TYPE=q8_0 compresses this cache to fit ~2x more tokens.
• MoE (Mixture of Experts): Models like Qwen 3.5 35B-A3B have 35B total parameters but
only activate 3B per token. This means 35B-class knowledge with 3B-class compute cost.
The catch: all 35B parameters must still be loaded into memory even though only 3B are
“active” at any time.
• Context Window: The model’s “working memory”. Everything the model can “see”: system
prompt, user messages, assistant replies, and any injected text. Once the window fills, old
messages are truncated and the model literally forgets them. Qwen 3.5 9B supports 256K
theoretically but 16K–32K practically on 16GB hardware.

1.2 Models Worth Knowing

Model Size @ Q4 Strengths Best For

Page 2
 Local LLM Mastery — Core Knowledge

Qwen 3.5 9B ~6.6 GB 81.7 GPQA Diamond, Daily driver on 16GB

multimodal, thinking Mac — coding + chat
mode, 256K ctx

Qwen 3.5 4B ~2.5 GB Blazing fast, multimodal, Quick chats, worker

surprisingly strong model for delegation
reasoning

Qwen 3.5 27B ~16-17 GB Q3 72.4 SWE-bench, dense Serious coding on

reasoning OMEN (tight VRAM fit)

Qwen 3.5 35B-A3B ~22 GB Q4 MoE — 35B knowledge, General reasoning on

3B compute OMEN with RAM spill

GLM-4.7-Flash ~15 GB Q4 73.8% SWE-bench, Best local coding model

preserved thinking, 25.9 on OMEN
Coding Index

Hardware-Model Fit Summary

MacBook Pro M1 Pro 16GB: Qwen 3.5 9B (MLX 4-bit) as main, 4B as fast secondary.
HP OMEN RTX 5060 Ti 16GB + 32GB RAM: GLM-4.7-Flash for coding, Qwen 3.5 9B for general,
27B for deep coding sessions.

1.3 Inference Engines

Three backends exist for running models on Apple Silicon, each a different layer of the same
ecosystem:

Backend Speed (M1 Pro) What It Is

MLX (via LM Studio) ~35–50 tok/s Apple’s native ML framework.

Speaks directly to Metal hardware.
Fastest on Mac.

[Link] (direct) ~30–40 tok/s Raw C++ inference engine. Uses

Metal shaders. Maximum control,
compile yourself.

Ollama (wraps [Link]) ~20–25 tok/s Model manager + server +

[Link]. Adds convenience but
overhead. Note: Qwen 3.5 GGUF
has compatibility issues with
Ollama as of March 2026.

Recommendation: LM Studio with MLX backend for the best speed-to-usability ratio on Mac. On the
OMEN (Linux + NVIDIA), Ollama with CUDA is the natural choice since MLX is Apple-only.

1.4 Chat Interfaces

• LM Studio: Most polished local LLM GUI. Built-in model browser, MLX backend on Mac,
local API server, MCP support since v0.3.17. Recommended for Mac.

Page 3
 Local LLM Mastery — Core Knowledge

• Open WebUI: Self-hosted web UI (123K GitHub stars). Runs on top of Ollama at
localhost:3000. Full conversation management, RAG support, model presets. More powerful
than LM Studio but requires Docker or pip install.
• Jan: Full offline ChatGPT-style app. Simplest option, zero config. Open source (Apache 2.0).
Best for pure conversation, less extensible.

Page 4
 Local LLM Mastery — Core Knowledge

2. Context Engineering
The 2026 industry framing shifts from “prompt engineering” to “context engineering.” The key insight:
the most important skill is not crafting clever prompts but designing the entire information pipeline
that feeds into the context window. This includes what memories to retrieve, how to compress them,
what to evict, and how to structure the system prompt.

2.1 The Context Window as Working Memory

The context window is the LLM’s working memory — analogous to human working memory (Miller’s
7±2 items). Everything the model can reason about must fit inside it. The math for your 16GB Mac:

Component Memory

macOS + background apps ~3 GB

Qwen 3.5 9B (Q4) ~6.6 GB

KV Cache (16K context, q8_0) ~2 GB

Framework overhead ~0.5 GB

TOTAL ~12.1 GB (comfortable)

2.2 Context Management Patterns

• Summarize-and-Replace: After a tool call returns a large result, your orchestration code
replaces the raw output with a summary before the next turn. 3000 tokens of passages
become ~40 tokens of key points.
• Ephemeral Tool Results: Show the model the result once for generating a response, then
drop the tool output from persistent history. The model “forgets” the raw data but its response
already incorporated it.
• Sliding Window with Pinning: Keep the last N messages but maintain a “pinned” section
(core memories, system instructions) that never gets evicted. Old tool calls get dropped first.
• Sub-Agent Delegation: Use a smaller model (4B) to handle tool calls and data processing.
It receives the raw output, summarizes it, and returns only the concise result to the main
model (9B). The main model’s context stays clean.

The Core Principle

The LLM is not the orchestrator. Your code is. The LLM is a reasoning engine you feed carefully
curated context to. The art is in what you include, what you exclude, what you delegate, and what
you summarize.

Page 5
 Local LLM Mastery — Core Knowledge

2.3 Why LLMs Cannot Selectively Forget

Once tokens are in the context window, they are there until the conversation ends or they get
truncated off the top. Transformers have no mechanism to selectively erase parts of context mid-
conversation. This is a fundamental architectural limitation. The solution is always external: your
Python code controls what enters the messages array, simulating “forgetting” by omitting or
summarizing content before sending it to the model.

Page 6
 Local LLM Mastery — Core Knowledge

3. LLM Memory Systems

3.1 Human Memory → LLM Memory (CoALA Framework)

The CoALA paper (Cognitive Architectures for Language Agents, 2023) is the most influential
framework connecting human cognition to LLM memory. It draws from decades of cognitive science
to define four memory types:

Memory Type Human Analogy LLM Implementation

Working Memory What you are actively thinking The context window — limited by
about (7±2 items) token count

Episodic Memory Autobiographical events: “Last Conversation logs, timestamped

Tuesday I had coffee with Maria” interaction history stored externally

Semantic Memory General knowledge: “Madrid is in Knowledge bases, vector

Spain” databases, RAG collections

Procedural Memory How to ride a bike — skills Model’s trained weights + stored
executed without conscious thought code/functions/tools

The critical parallel: human memory is reconstructive, not like a video recording. Each recall partially
reconstructs and can introduce errors. Similarly, when an LLM retrieves and reasons over stored
information, it can hallucinate or distort. Understanding this shared weakness is fundamental to
designing robust memory systems.

3.2 Industry Memory Models (2026)

MemGPT / Letta: The LLM-as-Operating-System
The most influential academic-to-production memory system, created at UC Berkeley (now Letta,
$10M funded). It treats memory management like an OS manages RAM and disk. Three tiers: Core
Memory (always in context, editable by the model via tool calls), Recall Storage (searchable
conversation logs), and Archival Storage (vector DB for long-term knowledge). The revolutionary
aspect: the LLM itself decides what to remember and what to forget using tool calls.

Claude’s Memory (Anthropic)

Emphasizes user control and data isolation. Memories are derived from conversations and stored as
factual summaries. The user can view, edit, and delete them. Scoped per-project or global, stored as
plain text summaries rather than raw logs. Full transparency by design.

OpenAI’s Memory
More automated extraction from conversations, stored as short statements. Less transparent than
Claude’s approach. Memory is global across all conversations.

Knowledge Graphs for Memory

An emerging approach where memories are stored as nodes and edges: “Eduardo” → “works at” →
“[company]” → “on team” → “SBD”. Relationships are explicit and queryable. Tools like Cognee and

Page 7
 Local LLM Mastery — Core Knowledge

Neo4j are being used. The industry is moving toward graph-based memory as the next evolution
beyond simple RAG.

Context Engineering (2026 Trend)

The latest framing: the most important skill is designing the entire information pipeline feeding the
context window. Letta calls this “programming the context window.” This includes retrieval strategy,
compression, eviction policy, and system prompt architecture.

Page 8
 Local LLM Mastery — Core Knowledge

4. RAG (Retrieval-Augmented Generation)

4.1 How RAG Works

RAG is NOT permanent and does NOT modify the model. Think of it as handing the model a
reference book opened to the right page before asking a question. The pipeline has three steps:
• Indexing: A book or document is split into chunks (~500 tokens each). Each chunk is
converted into a numerical vector (embedding) using a small embedding model (all-MiniLM-
L6-v2, ~80MB). These vectors are stored in a vector database (ChromaDB) on disk.
• Retrieval: When you ask a question, the question is also embedded. The vector DB finds
the 3–5 chunks whose embeddings are most similar (cosine similarity). This takes
milliseconds.
• Generation: The retrieved passages are injected into the system prompt alongside the
user’s question. The LLM reasons over them and generates an answer. A 100K-token book
works within a 16K context window because only ~2500 tokens of relevant passages are
injected per question.

The Amnesia Analogy

Imagine a brilliant person with amnesia (the LLM). Every morning they wake up knowing nothing
about yesterday.
But they have a librarian friend (RAG) who hands them exactly the right pages before each
conversation.
They can reason brilliantly about those pages, but tomorrow they won’t remember them unless the
librarian hands them over again.

4.2 Multiple “Instances” Without Copying the Model

You do not need separate model copies. The model is one binary file. What varies is the data
pipeline feeding into it:
• Modelfiles / Presets: Different system prompts and temperatures. Same 6.6GB weights,
different personas. Switching is instant.
• RAG Collections: Different ChromaDB collections (one per book). Your script selects which
collection to query. Same model, different knowledge sources.
• Multi-book Expert: Query across ALL collections simultaneously and merge results. Ask
“What should I prioritize for health at 40?” and get Attia’s medical protocols, Burkeman’s
philosophical framing, and Aurelius’s stoic perspective in one answer.

4.3 MarkItDown for Content Conversion

Microsoft’s MarkItDown converts DOCX, XLSX, PPTX, PDF, EPUB, HTML, CSV, JSON, images,
audio, and YouTube URLs into Markdown suitable for LLM consumption. Install with: pip install

Page 9
 Local LLM Mastery — Core Knowledge

'markitdown[all]'. Convert a PDF book to markdown, then index it into ChromaDB. YouTube
conversion extracts transcripts — do this while you have Wi-Fi.

Page 10
 Local LLM Mastery — Core Knowledge

5. MCP, CLI Tools & Agentic Patterns

5.1 Model Context Protocol (MCP)

MCP has three core concepts: Tools (functions the model can call, like POST endpoints), Resources
(read-only data, like GET endpoints), and Prompts (reusable templates). The model sees a tool’s
name, description, and parameter types, decides when to call it, and the host (LM Studio, Claude
Desktop, Cursor) shows a confirmation dialog before executing.
Building your own MCP server is simple with FastMCP (part of the official MCP Python SDK). Install
with: pip install "mcp[cli]". A complete working MCP server is about 20 lines of Python — decorate
functions with @[Link]() and the schema, validation, and documentation are generated
automatically.

5.2 The MCP vs CLI Debate

This is the hottest debate in the agent space (early 2026). The core argument for CLI: a typical MCP
server dumps an entire schema into the agent’s context window. One example: the full GitHub MCP
server ships with 93 tools costing ~55,000 tokens before you’ve asked a single question. CLI tools
are context-free — LLMs already know bash, git, and common CLIs from training data.

Dimension MCP CLI

Context cost High — all schemas loaded upfront Low — only tool docs in system
prompt (~100–200 tokens)

Structured output Requires precise JSON (quantized Natural language — models’

models struggle) strongest mode

Error recovery Silent failures (invalid JSON = Informative errors the model can
crashed call) read and self-correct from

Discovery All tools visible at once Progressive: model drills into --help
only when needed

Enterprise fit Strong — audit trails, structured Weaker — less standardized

permissions

Why Quantized Models Struggle with MCP

Function calling requires the model to output precise JSON: exact field names, correct types,
properly escaped values.
Quantization compresses probability distributions, losing the sharp peaks needed for exact-character
precision.
A Q4 model might output {name: search_memories} instead of {"name": "search_memories"} —
looks right to a human, invalid JSON.
CLI sidesteps this entirely: the model outputs bash commands, which are natural language text —
the thing models are optimized for.

Page 11
 Local LLM Mastery — Core Knowledge

5.3 Teaching a Local LLM About Your CLI

Three approaches, from simplest to most sophisticated:
• System Prompt Documentation: List available commands in the system prompt (~100–200
tokens). The model outputs <cmd>...</cmd> tags, your Python script parses and executes
them via subprocess. Total context cost: minimal.
• [Link] / [Link] Pattern: A markdown file describing your tools that the model
reads once. Each “command” is a tiny Python script in ~/llm-lab/bin/. This is what Claude
Code and OpenClaw use under the hood.
• Progressive Discovery (--help): Instead of dumping all tool definitions, tell the model: “Run
tool --help to learn what it can do.” The model only loads what it needs, when it needs it. Best
for large CLIs with many subcommands.

5.4 Multi-Agent Orchestration

LM Studio (and similar tools) do not natively orchestrate multiple agents. Your Python code is the
orchestrator. The pattern:
• Main Model (9B): Handles conversation and reasoning. Its context stays clean and focused.
• Worker Model (4B): Handles tool calls, data parsing, memory extraction. Receives tasks,
returns concise summaries. Its context is disposable — created fresh per task.
This is essentially what Letta/MemGPT does architecturally: an orchestration loop around the LLM,
not inside it. The LLM is stateless; your code maintains state.

Orchestration Frameworks
• CrewAI: 44K+ GitHub stars. Role-playing AI agents in Python. Define agents with roles, give
them tools, they collaborate. Works with Ollama. Best for structured multi-agent QA
workflows.
• LangGraph: Graph-based agent architecture from LangChain. Most efficient state
management. More code-heavy but more control.
• AutoGen (Microsoft): Conversational multi-agent setups. Consistent coordination behavior.

Page 12
 Local LLM Mastery — Core Knowledge

6. Recommended Courses & Resources

6.1 Essential Papers

Paper Why It Matters

MemGPT: Towards LLMs as THE foundational paper on LLM memory. 16 pages. Introduces
Operating Systems hierarchical memory tiers and self-editing memory.
([Link]/abs/2310.08560)

CoALA: Cognitive Architectures for The theoretical framework connecting human cognition to LLM
Language Agents agent design. Defines the four memory types.
([Link]/abs/2309.02427)

Generative Agents: Interactive Stanford’s AI town. 25 agents with episodic + semantic memory.
Simulacra ([Link]/abs/2304.03442) Memory retrieval using recency, importance, relevance scores.

6.2 Free Courses

Course Platform Details

LLMs as Operating Systems [Link] FREE. ~1.5 hours. Python

(with Letta) notebooks building a MemGPT-
style agent from scratch. Highest
priority.

Prompt Engineering Anthropic Academy Free. Context management

sections directly relevant to memory
design.

Introduction to LangGraph LangChain Academy Free. Stateful agent design with

memory. LangGraph approach to
conversation persistence.

AI Agents in LangGraph [Link] FREE. Agent loops, tool use, state

management in graph-based
architectures.

Building Agentic RAG [Link] FREE. RAG as an agentic tool —

the model decides when to search.

Google AI Essentials Coursera / Google Skills Foundation-level AI literacy. Part of

your 19-week curriculum.

MCP Quickstart [Link] Official spec and tutorials. Short

and readable.

6.3 Key Blog Posts & Articles

• Making Sense of Memory in AI Agents (Leonie Monigatti) — Connects CoALA, Letta, and
practical implementation.

Page 13
 Local LLM Mastery — Core Knowledge

• Design Patterns for Long-Term Memory in LLM Architectures (Serokell) — Compares

MemGPT, OpenAI, Claude, and toolkit approaches side by side.
• LangGraph Memory Documentation ([Link]) — Memory types with
implementation patterns.
• Cognee: CoALA Explained ([Link]/blog) — Readable breakdown of the CoALA paper.

6.4 Books for Deeper Understanding

Book Why

Thinking, Fast and Slow (Daniel Dual-process theory (System 1/2) maps directly to thinking/non-
Kahneman) thinking modes in models like Qwen 3.5.

The Design of Everyday Things (Don Mental models and feedback loops apply directly to designing
Norman) memory interfaces for agents.

Grokking Artificial Intelligence Visual intro to search, optimization, and learning algorithms.
Algorithms (Rishal Hurbans) Foundation for understanding embeddings.

Page 14
 Local LLM Mastery — Core Knowledge

PART II
Learning Path → Agentic AI QA Engineer
How the core knowledge maps to the SIXT Agentic AI QA Engineer role and your career transition.
This section connects every topic to a practical skill the position requires.

7. The SIXT Role: What They Want

The Agentic AI QA Engineer role requires someone who can design, build, and maintain AI-powered
testing agents. Based on the role description and industry trends, the key competencies are:

Competency What You’re Building

Prompt & Context Engineering Your memory systems, system prompt design, context
management patterns — all from Part I Sections 2–3.

Agent Orchestration Multi-agent patterns (main + worker models), CrewAI/LangGraph

knowledge, MCP vs CLI evaluation — Part I Section 5.

Tool Integration (MCP/CLI) Building custom MCP servers, CLI-based tool interfaces, function
calling reliability assessment — Part I Section 5.

RAG Engineering Document indexing, embedding models, vector DBs, multi-source

retrieval — Part I Section 4.

Local LLM Evaluation Model selection, quantization tradeoffs, benchmark interpretation

— Part I Section 1.

QA Domain Knowledge Test automation patterns, framework design, multi-platform testing

— your existing 8+ years of expertise.

7.1 Your Unique Advantage

You bring something most AI engineers lack: deep QA domain expertise. You understand test
design, failure modes, multi-platform complexity (Web, WAP, Android, iOS), and framework
architecture. The gap to close is the AI/agent layer on top. Everything in this document targets
exactly that gap.

7.2 Portfolio Pieces from This Journey

• CoALA-based Memory System: A four-tier memory architecture (working, episodic,
semantic, procedural) built from scratch. Demonstrates cognitive architecture knowledge
applied to real implementation.
• MCP vs CLI Comparison: A practical evaluation showing token costs, reliability, and
performance differences. Demonstrates critical evaluation skills.
• RAG-Powered Book Expert: Multi-collection retrieval system with ChromaDB. Adaptable to
test documentation indexing.

Page 15
 Local LLM Mastery — Core Knowledge

• Local Orchestrator: Python orchestration layer with main + worker model delegation. The
exact pattern production agentic systems use.

8. Structured Learning Path

This learning path builds incrementally. Each phase adds a layer of capability. No strict timeline —
move at your pace, but maintain the sequence.

Phase 1: Foundations (Weeks 1–4)

• Anthropic Academy: Prompt Engineering — Context management, system prompts,
structured outputs. Direct relevance to how you’ll instruct test agents.
• [Link]: LLMs as Operating Systems (Letta) — Build a MemGPT-style agent.
Understand memory tiers, self-editing memory, the OS analogy.
• Read: MemGPT and CoALA papers — The theoretical foundation for everything that
follows.
• Hands-on: Complete the Inflight Lab experiments (context monitoring, memory systems,
RAG). This is your practical foundation.

Phase 2: Agent Architecture (Weeks 5–8)

• LangChain Academy: Introduction to LangGraph — Graph-based agent design, state
management, memory persistence.
• [Link]: AI Agents in LangGraph — Agent loops, tool use patterns, multi-step
reasoning.
• MCP Quickstart ([Link]) — Build your own MCP server. Understand
tools/resources/prompts.
• Hands-on: Build the MCP vs CLI comparison project. Create a custom MCP server
wrapping your ChromaDB RAG setup.

Phase 3: Applied QA (Weeks 9–12)

• [Link]: Building Agentic RAG — RAG as an agentic tool. The model decides
when to search.
• CrewAI documentation + tutorials — Multi-agent QA workflows: test designer agent, code
generator agent, reviewer agent.
• Hands-on: Index your FRAMEWORK_GUIDE.md and test suite docs in ChromaDB. Build a
test-writing agent that follows your exact patterns by retrieving relevant examples. Expose
your framework via MCP.

Phase 4: Integration & Portfolio (Weeks 13–16)

• Build a memory-augmented QA agent: Remembers which tests failed last sprint, what
bugs were found in similar features, what strategies worked. Episodic memory = test

Page 16
 Local LLM Mastery — Core Knowledge

execution history, semantic memory = domain knowledge (BetBuilder rules, LFB logic),
procedural memory = test patterns.
• Document everything on GitHub: Architecture decisions, README files, code examples.
This becomes your portfolio for the SIXT application.

Phase 5: Advanced Topics (Weeks 17–19+)

• Knowledge graph memory — Nodes and edges instead of flat text. Relationships are
explicit and queryable.
• Appium MCP servers — official appium-mcp and @mobilenext/mobile-mcp for mobile test
automation.
• Custom MCP server wrapping your existing framework API — The bridge between your
QA expertise and agentic AI.

Page 17
 Local LLM Mastery — Core Knowledge

PART III
Inflight Lab & Travel Experiments
Everything you need to learn by doing during your Madrid → Dallas → Guadalajara trip and back.
Structured around your travel segments: bus rides, airport waits, flights, layovers, and Guadalajara
evenings. Includes a complete pre-flight installation checklist.

9. Pre-Flight Setup Checklist

CRITICAL: Complete ALL of this while you have Wi-Fi

Everything after this section works fully offline. If any step fails, fix it now. Test every component
before boarding.

9.1 Install LM Studio + Models

• Download LM Studio from [Link]. Install (drag to Applications).
• Download models (Discover tab → search “Qwen3.5” → filter MLX):
◦ mlx-community/Qwen3.5-9B-MLX-4bit (~5–6 GB) ← main model
◦ mlx-community/Qwen3.5-4B-MLX-4bit (~2.5 GB) ← fast comparison / worker
◦ mlx-community/Qwen3.5-0.8B-MLX-4bit (~0.8 GB) ← tiny for context experiments
• Enable the local server: Developer tab → toggle server ON (localhost:1234).
• Test: Select the 9B model, send a message, confirm ~35+ tok/s in the status bar.

9.2 Install Python Dependencies

# Create a virtual environment for the lab
python3 -m venv ~/llm-lab/venv
source ~/llm-lab/venv/bin/activate

# Core RAG dependencies

pip install chromadb sentence-transformers langchain
pip install langchain-community langchain-text-splitters pypdf

# MarkItDown for content conversion

pip install 'markitdown[all]'

# MCP SDK (for building your own MCP server)

pip install 'mcp[cli]'

# Pre-download the embedding model (runs offline after this)

python3 -c "from sentence_transformers import SentenceTransformer;
SentenceTransformer('all-MiniLM-L6-v2')"

Page 18
 Local LLM Mastery — Core Knowledge

9.3 Install Filesystem MCP Server

npm install -g @modelcontextprotocol/server-filesystem

Then add to LM Studio’s [Link] (Program tab → Install → Edit [Link]):

{
"mcpServers": {
"my-files": {
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-filesystem",
"/Users/eduardo/llm-lab"]
}
}
}

9.4 Download Books & Content

mkdir -p ~/llm-lab/books

# Free books from Project Gutenberg

curl -o ~/llm-lab/books/[Link] \
[Link]
curl -o ~/llm-lab/books/art_of_war.txt \
[Link]

# Convert any PDFs/EPUBs you have

markitdown "[Link]" > ~/llm-lab/books/[Link]
markitdown "[Link]" > ~/llm-lab/books/four_thousand_weeks.md

# Convert YouTube videos (while on Wi-Fi!)

markitdown "[Link] >
~/llm-lab/books/attia_podcast.md

9.5 Download Papers & Courses for Offline Reading

• MemGPT paper: [Link]/abs/2310.08560 → Download PDF
• CoALA paper: [Link]/abs/2309.02427 → Download PDF
• [Link] Letta course: Download notebooks and videos for offline access
• Blog posts: Save as PDF: Leonie Monigatti’s memory article, Serokell design patterns,
Cognee CoALA explainer

9.6 Create the Lab Directory Structure

mkdir -p ~/llm-lab/{books,memory,scripts,bin,chroma_db}
mkdir -p ~/llm-lab/memory/{working,episodic,semantic,procedural}

9.7 Final Verification

• LM Studio: Open app → select 9B → send a message → confirm response
• API server: Developer tab ON → run: curl [Link]
• Python: source ~/llm-lab/venv/bin/activate → python3 -c "import chromadb; print('OK')"

Page 19
 Local LLM Mastery — Core Knowledge

• Embedding model: python3 -c "from sentence_transformers import SentenceTransformer;

m = SentenceTransformer('all-MiniLM-L6-v2'); print([Link]('test').shape)"
• MCP server: python3 ~/llm-lab/my_mcp_server.py (should start without error, Ctrl+C to stop)
• Run caffeinate -i in a terminal to prevent Mac from sleeping. Close Safari/Chrome to free
memory for the model.

Page 20
 Local LLM Mastery — Core Knowledge

10. Travel-Segment Lab Schedule

Segment 1: Bus to Madrid Airport (~3 hours)

• READ: MemGPT paper. Read thoroughly. Mark sections about the three memory tiers. By
the airport, you should be able to explain the OS analogy from memory.
• READ: Leonie Monigatti’s “Making Sense of Memory in AI Agents” blog post.

Segment 2: Madrid Airport Wait (~2 hours)

• READ: CoALA paper, focusing on Sections 4 (the framework) and 5 (mapping existing
agents).
• THINK: Map your memory_chat.py to the CoALA framework. Which memory types does it
implement? Which does it lack? Write your analysis in a text file.

Segment 3: Madrid → Dallas Flight (~10 hours)

Hours 1–2: Context Window Experiments
• Experiment 1: Monitor context with “ollama ps” (or LM Studio status bar). Watch token count
grow per exchange. Find your effective context boundary.
• Experiment 2: Use the 0.8B model with a tiny context (2048 tokens). Tell it your name,
profession, hobbies across 5–6 messages. Then ask it to recall. Watch it forget. Repeat with
9B at 16K — feel the difference.
• Experiment 3: Run context_monitor.py. Use /stats to see exact token counts. Use /clear to
reset and observe the fresh start.

Hours 3–4: Memory System from Scratch

• Build memory_chat.py: Text files as memory nodes ([Link], [Link],
[Link], [Link], [Link]). Injected into system prompt each turn. Commands:
/remember, /memories, /forget, /stats, /clear.
• Build auto_memory_chat.py: After each exchange, the model itself decides what’s worth
remembering. Extracts facts as JSON, saves them automatically.

Hours 5–6: RAG Book Expert

• Run index_book.py: Index your downloaded book(s) into ChromaDB. Chunks of ~500
tokens, embedded with all-MiniLM-L6-v2.
• Run book_expert.py: Ask questions about the book. Use /passages to see exactly what
RAG retrieved. A 100K-token book works in your 16K window because only ~2500 tokens
are injected per question.
• Multi-book experiment: Index multiple sources into separate collections. Build the multi-
book expert that queries across all collections and merges results.

Hours 7–8: CoALA Memory Architecture

Page 21
 Local LLM Mastery — Core Knowledge

• Extend memory_chat.py to four tiers: working/current_goals.md, episodic/2026-03-

[Link] (timestamped events), semantic/[Link] (extracted knowledge),
procedural/[Link] (effective prompt templates).
• Add reflection mechanism: Every N turns, the model reviews episodic memories and
generates semantic summaries. This mirrors human sleep-based memory consolidation.

Hours 9–10: MCP vs CLI Comparison

• Build your MCP server: my_mcp_server.py with search_memories, save_memory,
list_books tools. Connect via LM Studio’s [Link].
• Build the CLI equivalent: Same functions as tiny scripts in ~/llm-lab/bin/. Document in
system prompt. Parse <cmd> tags from model output.
• Compare: Same conversation with each approach. Which uses fewer tokens? Which
produces better results? Which keeps context cleaner?

Segment 4: Dallas Layover (~2 hours)

• If Wi-Fi available: Download [Link] Letta course notebooks. Browse Letta GitHub
repo.
• If no Wi-Fi: Read Serokell article on Design Patterns for Long-Term Memory. Write notes
comparing MemGPT, OpenAI, and Claude’s approaches.

Segment 5: Dallas → Guadalajara (~3 hours)

• Build a forgetting mechanism: Add decay to episodic memory. Entries older than N hours
get summarized into semantic memory, then originals are deleted. Mirrors human memory
consolidation.
• Implement sub-agent delegation: Use the 4B model as a worker. It handles tool calls and
returns summaries to the 9B main model. Observe how the 9B’s context stays clean.

Segment 6: Guadalajara Evenings (3 days)

Day 1 Evening (2–3 hours)
Work through the [Link] Letta course. Adapt exercises to use your local LM Studio model
instead of OpenAI. Theory meets implementation.

Day 2 Evening (2–3 hours)

Build a knowledge graph memory. Instead of flat text files, store memories as nodes and edges in a
JSON structure. When the model needs context about “stoicism,” traverse the graph to find related
nodes.

Day 3 Evening (2–3 hours)

Synthesis project: combine everything into one system. User message → retrieve from knowledge
graph + RAG + episodic memory + core memory → assemble context → model generates → post-
processing (log episodic, auto-extract facts, periodic consolidation). This is a simplified Letta built
from scratch.

Page 22
 Local LLM Mastery — Core Knowledge

Segment 7: Return Journey (~23 hours)

• Bus to GDL airport: Read Thinking, Fast and Slow chapters 1–4. Map System 1/System 2
to thinking/non-thinking modes in Qwen 3.5.
• GDL → Dallas: Refine your synthesis project. Add error handling, better retrieval, test with
different books. Write a README documenting architecture decisions.
• Dallas → Madrid: The big experiment: use your full system as a genuine 8-hour
conversation companion. Discuss books, life, career, music. Let memory accumulate across
multiple /clear cycles. By landing, the system should know you as well as a friend you’ve
been talking to for a week.
• Bus home: Reflect and write: “What I Learned About LLM Memory.” Plan how this applies to
your Agentic AI QA curriculum.

Page 23
 Local LLM Mastery — Core Knowledge

11. LM Studio Quick Reference

11.1 Two Presets

Setting Coding Preset Chat/Books Preset

Temperature 0.3 (deterministic) 0.7 (creative)

Thinking mode OFF (/no_think) — faster ON (/think) — deeper reasoning

Max tokens 4096 (longer code) 2048 (standard)

Top K 20 20

Top P 0.9 0.95

Context length 16384 (safe) or 32768 (tight) 16384 (safe) or 32768 (tight)

11.2 Essential Tips

• Thinking mode: Add /think at the start of a message to enable, /no_think to disable. Or set
in system prompt.
• Context usage: Watch the bottom status bar for token counts and tok/s. If speed drops
significantly, your context is getting long.
• Long conversations: Start new threads by topic (“Book discussion — Borges”, “QA career
strategy”). Each gets clean context. Switch freely.
• API + GUI simultaneously: The Developer tab server and the Chat tab share the same
loaded model. Chat in the GUI and run Python scripts at the same time.
• Prevent sleep: Run caffeinate -i in Terminal. Close browsers to free memory.

11.3 Unified Python Client

Create ~/llm-lab/llm_client.py so all scripts work with any backend:
import requests, os

BACKEND = [Link]('LLM_BACKEND', 'lmstudio')

def chat(messages, model=None, temperature=0.7, max_tokens=2048):

if BACKEND == 'lmstudio':
url = '[Link]
model = model or 'mlx-community/Qwen3.5-9B-MLX-4bit'
r = [Link](url, json={
'model': model, 'messages': messages,
'temperature': temperature, 'max_tokens': max_tokens,
'stream': False
})
data = [Link]()
return {
'content': data['choices'][0]['message']['content'],

Page 24
 Local LLM Mastery — Core Knowledge

'prompt_tokens': [Link]('usage',{}).get('prompt_tokens',0),
'completion_tokens': [Link]('usage',{}).get('completion_tokens',0),
}
else: # ollama
url = '[Link]
model = model or 'qwen3.5:9b'
r = [Link](url, json={
'model': model, 'messages': messages, 'stream': False
})
data = [Link]()
return {
'content': data['message']['content'],
'prompt_tokens': [Link]('prompt_eval_count', 0),
'completion_tokens': [Link]('eval_count', 0),
}

The LLM is not the orchestrator. Your code is.

The LLM is a reasoning engine you feed carefully curated context to. The art is in what you include, what
you exclude, what you delegate, and what you summarize. That’s context engineering — and it’s the
most valuable skill in the agentic AI space right now.

Page 25