Contents

• LLM Engineering: Beginner’s Notes

◦ Part 1 — What Is a Large Language Model?

▪ What “LLM” actually means

▪ What “Frontier model” means
◦ Part 2 — The Transformer: How We Got Here

▪ What “GPT” stands for

▪ A one-minute history of neural networks
▪ “Attention Is All You Need”
▪ From GPT-1 to today
▪ The surprise nobody fully explains
◦ Part 3 — Tokens: How LLMs See Text

▪ Text is broken into tokens

▪ Seeing tokenization in code
▪ Why tokens matter to you
▪ Why “tokens” and not letters or words?
▪ A useful rule of thumb
▪ Try it yourself
▪ Tokens are not vectors
◦ Part 4 — Parameters and the Size of a Model

▪ What a parameter is
▪ The numbers have exploded
▪ Why models come in sizes (nano, mini, haiku, sonnet…)
▪ Inference, and two ways to make a model “smarter”
◦ Part 5 — LLMs Have No Memory (The Stateless Truth)

▪ Every request starts from zero

▪ How we fake a memory
▪ The key takeaways
◦ Part 6 — The Context Window

▪ What it is

1 / 52
 ▪ Sizes vary a lot
▪ Why it matters
◦ Part 7 — Talking to an LLM: Prompts and Messages

▪ The two kinds of prompt

▪ The messages structure
▪ Building messages with a small helper function
◦ Part 8 — Types of Models: Base, Chat, Reasoning, and
Hybrid

▪ 1. Base model
▪ 2. Chat (or “instruct”) model
▪ 3. Reasoning (or “thinking”) model
▪ 4. Hybrid model
▪ Which type should you use?
▪ A note on terminology
◦ Part 9 — The Chat Completions API and Endpoints

▪ What an API and an endpoint are

▪ The Chat Completions API
▪ Calling the endpoint directly with raw HTTP
◦ Part 10 — The OpenAI Client Library

▪ What the openai package really is

▪ The same request, made simple
▪ Setting up API keys safely
◦ Part 11 — The Development Environment (The Tools You’ll
Use)

▪ Git and GitHub (they’re not the same)

▪ An editor (IDE): Cursor or VS Code
▪ uv: managing Python and packages
▪ Jupyter notebooks: where the experimenting happens
▪ Getting an API key (and the cost side)

2 / 52
◦ Part 12 — Using Other Providers and Running Models Locally

▪ OpenAI-compatible endpoints
▪ Running models locally with Ollama
▪ Frontier vs. local: the trade-off
▪ Chatting with Ollama straight from the command line
◦ Part 13 — The Landscape of Models (Who Makes What)

▪ Closed source vs. open source

▪ The major closed-source models
▪ The major open-source models
▪ Distillation (how the small DeepSeek variants were
made)
◦ Part 14 — Three Ways to Use a Model

▪ 1. Packaged products
▪ 2. Cloud APIs
▪ 3. Running a model yourself (locally)
◦ Part 15 — Your First Real Project: Summarize a Web Page

▪ Step 1 — Get the page’s text

▪ Step 2 — Write the prompts
▪ Step 3 — Put it all together
▪ A note on the limits of simple scraping
◦ Part 16 — Guiding the Model’s Output: One-Shot Prompting
and JSON

▪ One-shot prompting
▪ Asking for structured JSON
▪ One-shot vs. multi-shot prompting
▪ Why JSON works so well
▪ What response_format actually does
◦ Part 17 — Combining Multiple Calls: A Multi-Step Solution

▪ Example: a company brochure generator

▪ Controlling tone

3 / 52
 ▪ Keeping input to a reasonable size
◦ Part 18 — Streaming: The Typewriter Effect
◦ Part 19 — What Frontier Models Are Great and Bad At

▪ Strengths
▪ Weaknesses
▪ The takeaway: use them under supervision
◦ Part 20 — Understanding API Costs

▪ Subscriptions vs. paying per use

▪ What you pay for
▪ The scale of pricing
▪ A couple of extra wrinkles
◦ Part 21 — Advanced Applications: Prompt Engineering,
Context Engineering, and Agentic AI

▪ Prompt engineering
▪ Context engineering
▪ Tools
▪ Agentic AI
◦ Part 22 — Where This Is Useful (Business Applications)
◦ Quick Glossary

LLM Engineering: Beginner’s

Notes
A friendly, from-scratch guide to the core ideas behind working
with Large Language Models (LLMs). These notes gather everything
from the lab exercises and reorder it so that each idea builds on
the one before. You don’t need to have read anything else first —
every section explains itself.

4 / 52
Part 1 — What Is a Large Language
Model?

What “LLM” actually means

LLM stands for Large Language Model. It is a computer program
that has been trained on an enormous amount of text (books,
websites, articles, code, and more) so that it can understand and
produce human-like language.

The single most important thing to understand is what an LLM

actually does under the hood:

An LLM predicts the next chunk of text, one small piece at a

time.

That’s it. When you give an LLM some text, it looks at everything
so far and asks itself: “Given all of this, what is the most likely
next piece of text?” Then it adds that piece, looks again, and
predicts the next one, and so on. What feels like conversation,
reasoning, or writing is really this prediction happening over and
over, very fast.

What “Frontier model” means

A Frontier model is one of the most powerful, cutting-edge LLMs
available — the ones “at the frontier” of what AI can currently do.
Examples include models from OpenAI (the GPT family), Google
(Gemini), and Anthropic (Claude). These are usually accessed over
the internet through a paid service.

There are also smaller, open-source models you can download

and run on your own computer for free (more on those later). They
are less powerful than frontier models but private and cost nothing
to run.

5 / 52
Part 2 — The Transformer: How We Got
Here
You’ll hear the word transformer constantly. It’s worth knowing
where LLMs came from, because it explains a lot of the vocabulary
you’ll meet later.

What “GPT” stands for

GPT is short for Generative Pre-trained Transformer. Each word
tells you something:

• Generative — it generates (produces) text, one token at a

time.
• Pre-trained — it was already trained ahead of time on huge
amounts of text scraped from the internet, books, and code.
• Transformer — the type of design used to build the model
(explained below).

A one-minute history of neural networks

Long before LLMs, data scientists used neural networks:
programs loosely inspired by the brain. A neural network is made
of many tiny maths functions called artificial neurons, all wired
together. By showing the network lots of examples, it learns to spot
patterns and make predictions. Stacking many layers of these
neurons on top of each other is called deep learning (the “deep”
just means “many layers”). More layers generally means the
network can learn deeper, subtler patterns.

6 / 52
“Attention Is All You Need”
In 2017, researchers at Google published a paper called Attention
Is All You Need. It introduced the transformer — a new way of
wiring up a neural network that is especially good at handling
sequences (like a sentence, where the order of words matters). Its
key ingredient is a step called self-attention: a layer that figures
out which earlier words in the input matter most when predicting
what comes next.

The word architecture just means “the way the pieces are
connected together.” The transformer is one architecture; there are
others (older ones like RNNs and LSTMs, and newer experimental
ones). None has yet clearly beaten the transformer, so it remains
the standard design for LLMs today.

An important perspective: the transformer isn’t magic or

fundamental. It’s mainly an efficiency breakthrough — it lets us
train much bigger models on much more data, faster and cheaper,
largely because its calculations can be run in parallel. Without it we
might have gotten here anyway, just more slowly and expensively.

From GPT-1 to today

Once the transformer existed, models scaled up fast: GPT-1 (2018),
GPT-2, GPT-3, then ChatGPT in late 2022 (built on GPT-3.5), GPT-4
(2023), the multimodal GPT-4o, and so on. Each jump mostly meant
“bigger model, more data.”

The surprise nobody fully explains

Here’s the genuinely strange part. It’s not surprising that a next-
token predictor produces text that sounds plausible — that’s
exactly what it was trained to do. What surprised everyone, even
the experts, is that the plausible next tokens are so often actually
correct. Give it a maths problem, and it doesn’t just produce

7 / 52
answer-shaped text — it frequently produces the right answer. This
tendency for real capability to appear once a model is large
enough is called emergent intelligence, and it’s still not fully
understood.

(Early critics captured the skeptical view in a famous paper on

“stochastic parrots,” arguing these models just parrot likely word
patterns without understanding. The debate continues, but the
practical capabilities turned out to be far greater than that view
expected.)

Part 3 — Tokens: How LLMs See Text

Text is broken into tokens

LLMs do not read text letter by letter or word by word. They break
text into pieces called tokens. A token is often a whole word, but
it can also be part of a word, a single character, or a punctuation
mark. Roughly speaking, one token is about 3–4 characters of
English text.

Every token is represented internally by a number called a token

ID. The model only ever works with these numbers; “converting
text to token IDs” is called tokenizing.

Seeing tokenization in code

The tiktoken library (made by OpenAI) lets you see exactly how
text gets split into tokens.

8 / 52
 import tiktoken

# Get the tokenizer used by a specific model

encoding = tiktoken.encoding_for_model("gpt-4.1-mini")

# Turn text into a list of token ID numbers

tokens = [Link]("Hi my name is Ed and I like banoffee
pie")
print(tokens)
# e.g. [13347, 856, 836, 374, 3279, ...] (a list of numbers)

You can also go the other way and turn a token ID back into text:

# Turn each token ID back into the text it represents

for token_id in tokens:
token_text = [Link]([token_id])
print(f"{token_id} = {token_text}")

# Decode a single token on its own

print([Link]([326]))

Why tokens matter to you

• Cost and limits: Paid LLM services charge by the number of
tokens processed, and every model has a maximum number of
tokens it can handle at once. Knowing that text becomes tokens
helps you understand pricing and size limits.
• Everything is tokens: Your instructions, the conversation, and
the model’s reply are all just tokens being predicted one after
another.

Why “tokens” and not letters or words?

Early models tried working letter by letter. That kept the list of
possible inputs tiny (about 100 characters), but it forced the model
to learn too much from scratch — how to build every word out of
letters and what those words mean.

9 / 52
The next attempt was word by word. But there are a huge
number of possible words (plus every name and place), so the list
exploded and rare words had to be dropped.

Tokens are the happy middle ground. A token is a chunk of

text — often a whole common word, sometimes a fragment of a
longer or rarer word, occasionally even two words that appear
together a lot. The full set of tokens a model knows is its
vocabulary (or vocab). This approach is compact, trains fast, and
lets the model recognize meaningful word-pieces. Like the
transformer, there’s nothing sacred about it — it just works well in
practice.

A nice side effect: tokens capture word structure. A common word

like “important” is one token, but a rarer word gets split into pieces
(e.g. “unimportant” → “un” + “important”). Tokens also distinguish
the start of a word from a mid-word fragment (the leading space is
part of the token), which helps the model handle things like plurals
and prefixes.

A useful rule of thumb

• Roughly 1 token ≈ 4 characters of English.
• Roughly 1,000 tokens ≈ 750 words.
• The complete works of Shakespeare (~900,000 words) is about
1.2 million tokens — which is why you’ll see prices quoted
“per million tokens.”

Code, maths, and scientific terms use up more tokens per word
(closer to one token per character), because they contain unusual
symbols and names.

10 / 52
Try it yourself
Besides tiktoken in code, OpenAI has a web page at
[Link]/tokenizer where you can paste text and
see it split into colored tokens. It’s a great way to build intuition.

Tokens are not vectors

One point of confusion for people who’ve heard of “vectors”: the
very first thing fed into the model is just the token ID (a number
identifying which token it is). Vectors are a different, later concept
inside the network — set them aside for now; they’re not the same
as tokens.

Part 4 — Parameters and the Size of a

Model
When people say a model is “big” or “small,” they’re usually
talking about its number of parameters.

What a parameter is
A parameter is a single adjustable number inside the model —
think of it as one tiny dial. Training the model means tuning all
these dials, by showing it example after example, until the whole
collection of dials produces good predictions. A model can have
millions, billions, or even trillions of these dials.

The numbers have exploded

• GPT-1 had about 117 million parameters.
• GPT-2: about 1.5 billion.
• GPT-3: about 175 billion.
• GPT-4: reportedly around 1.76 trillion.

11 / 52
 • The newest frontier models don’t publish their counts, but
they’re believed to be in the tens of trillions.

As a rough rule: more parameters generally means a more

capable model, because it has absorbed more knowledge and can
capture finer patterns. But it’s not the whole story — newer
techniques pack more ability into fewer parameters, so a modern
small model can easily beat an old large one.

A note on reading these numbers: charts of model sizes are

often drawn on a logarithmic scale, where each step means
10× bigger (1 billion → 10 billion → 100 billion), not just “one
more.” It’s the only way to fit such enormous ranges on one
picture.

Why models come in sizes (nano, mini, haiku,

sonnet…)
Most model families ship in several sizes — for example GPT-5
comes as nano, mini, and full, and Claude comes as haiku
(small), sonnet (medium), and opus (large). These are the
same model family at different parameter counts. Bigger versions
are smarter but slower and more expensive to run; smaller versions
are faster and cheaper. A small language model (SLM) is just a
deliberately small LLM (though even a “small” 3-billion-parameter
model is still large by older standards).

Many of the biggest models use a design called mixture of

experts (MoE): instead of one giant model, they contain many
smaller specialist sub-models, and route each question to the ones
best suited to it. This keeps them powerful without every
parameter having to run on every request.

12 / 52
Inference, and two ways to make a model
“smarter”
Inference is simply the technical word for running a trained model
— feeding it an input and getting an output. (Training is building
the model; inference is using it.)

There are two different ways to get better results, and you can use
both:

• Training-time scaling — build a bigger model and train it on

more data. (This is what “more parameters” refers to. A rough
historical guideline called the Chinchilla scaling laws said the
amount of training data a model can absorb grows roughly in
step with its parameter count.)
• Inference-time scaling — get more out of a model when you
run it, without retraining. Two common tricks: telling the model
to reason step by step before answering (covered later), and
stuffing more useful information into the input so it has more to
draw on (the idea behind techniques like RAG).

Over the last couple of years, inference-time scaling has become

just as important as making models bigger.

Part 5 — LLMs Have No Memory (The

Stateless Truth)
This is a beginner “aha” moment that surprises almost everyone.

13 / 52
Every request starts from zero
Each time you send a message to an LLM, it is a brand-new,
independent request. The model remembers nothing from your
previous messages. In technical terms, the LLM is stateless — it
holds no state, no memory, between calls.

Here’s the surprise in action. Imagine two separate requests:

# Request 1: we tell it our name

messages = [
{"role": "system", "content": "You are a helpful assistant"},
{"role": "user", "content": "Hi! I'm Ed!"}
]
# (model replies: "Hi Ed! How can I help?")

# Request 2 (a fresh, separate call): we ask it our name

messages = [
{"role": "system", "content": "You are a helpful assistant"},
{"role": "user", "content": "What's my name?"}
]
# The model does NOT know — we never told it in *this* request!

The second request has no idea our name is Ed, because the first
request is gone.

How we fake a memory

If the model forgets everything, how does ChatGPT seem to
remember your whole conversation? The trick is simple: we send
the entire conversation back every single time.

14 / 52
 # We include the earlier turns ourselves, so the model can "see"
them
messages = [
{"role": "system", "content": "You are a helpful assistant"},
{"role": "user", "content": "Hi! I'm Ed!"},
{"role": "assistant", "content": "Hi Ed! How can I assist you
today?"},
{"role": "user", "content": "What's my name?"}
]
# Now the model can predict "Ed" — because the answer is right
there in the text.

The key takeaways

1. Every call to an LLM is stateless — it starts fresh each time.
2. To create the illusion of memory, we resend the whole
conversation with each new message.
3. Because the model just predicts the next tokens, when the text
contains “My name is Ed” earlier and “What’s my name?” later,
the most likely next token is naturally… Ed.
4. This is why longer conversations cost more: you are re-sending
(and paying to process) all the earlier text every time. That’s
expected and exactly what we want — we’re paying for the
model to “think” across the whole conversation.

Part 6 — The Context Window

We just saw that to fake a memory, we resend the whole
conversation every time. That raises a natural question: is there a
limit to how much we can send? Yes — it’s called the context
window.

15 / 52
What it is
The context window is the maximum number of tokens a model
can look at in one go. It’s the model’s “working memory” for a
single request. Try to send more than fits, and the request simply
fails.

Crucially, everything has to fit inside this window together:

• the system prompt,

• the entire conversation history you’re resending (the fake
memory),
• any extra information you’ve added (examples, documents,
data),
• and the tokens the model generates as its reply.

The model produces its answer one token at a time, feeding its
own growing output back in as it goes — so a long answer also
eats into the window.

Sizes vary a lot

Different models offer very different context windows. As a
snapshot of typical sizes:

• GPT-5: around 400,000 tokens

• Claude models: around 200,000 tokens
• GPT-OSS (open source): around 130,000 tokens
• Gemini: up to 1,000,000 tokens

For perspective, the complete works of Shakespeare is roughly 1.2

million tokens — so only a million-token window could hold all of it
in a single prompt.

16 / 52
Why it matters
The context window sets a hard ceiling on how much background
the model can consider at once. It becomes especially important
for techniques that deliberately fill the input with helpful material
— like multi-shot prompting (giving several worked examples)
and RAG (feeding in relevant documents). If you’ve ever used a
coding assistant and noticed it “forgetting” earlier details in a very
long session, that’s the context window filling up.

Part 7 — Talking to an LLM: Prompts

and Messages

The two kinds of prompt

Models like GPT are trained to expect instructions in a particular
shape. You give them:

• A system prompt — background instructions that set the

model’s role, task, and tone. Think of it as: “You are a , and
you should behave like .” The user usually never sees this.
• A user prompt — the actual question or content you want a
reply to. This is the “conversation starter.”

Example of a system prompt that shapes the model’s personality

and job:

system_prompt = """
You are a snarky assistant that analyzes the contents of a website
and provides a short, humorous summary, ignoring navigation text.
Respond in markdown. Do not wrap the markdown in a code block.
"""

17 / 52
Simply changing the wording of the system prompt changes how
the model behaves — for example, adding “Respond in Spanish” or
“Be witty and humorous” instantly changes the output. This is one
of the easiest and most powerful ways to control an LLM.

The messages structure

Requests to the LLM are sent as a list of messages. Each
message is a small dictionary with two keys: a role and the
content .

messages = [
{"role": "system", "content": "system instructions go here"},
{"role": "user", "content": "the user's message goes here"}
]

There are three roles you’ll use:

• system — the background instructions (task and tone).

• user — something the human said.
• assistant — something the model previously replied (used
when resending conversation history, as we saw in Part 3).

This message format was created by OpenAI but is now used by

almost every LLM provider, so learning it once applies nearly
everywhere.

Building messages with a small helper function

Because the structure is always the same, it’s common to write a
tiny function that assembles it for you:

18 / 52
 def messages_for(website_text):
return [
{"role": "system", "content": system_prompt},
{"role": "user", "content": "Summarize this website:\n\n"
+ website_text}
]

Part 8 — Types of Models: Base, Chat,

Reasoning, and Hybrid
Not all LLMs behave the same way, even within one family. The
differences come from how they were trained. There are four
flavors worth knowing.

1. Base model
A base model does one raw thing: given a sequence of text, it
predicts what comes next. Nothing more. It hasn’t been taught how
to “chat” or follow instructions — it just continues the text.

You already use a base model every day: the predictive text on
your phone keyboard. Type “See you” and it suggests “later” —
that’s next-token prediction with no notion of conversation. Early
models like GPT-3 were base models, and people had to trick them
into answering questions by writing prompts in a “Q: … A: …”
pattern.

Base models are mostly useful when you plan to train the model
further yourself to give it a new skill, because you want a blank-ish
slate rather than one already shaped for chatting.

19 / 52
2. Chat (or “instruct”) model
OpenAI’s breakthrough was to train a base model further so it
works as a back-and-forth assistant: a system message, then user/
assistant turns. The result is a chat model (also called an instruct
model) — the kind you actually talk to.

The training technique behind this is RLHF (Reinforcement

Learning from Human Feedback): humans rate the model’s
responses, and the model is nudged toward the kinds of answers
people prefer. RLHF is what turned plain GPT into ChatGPT.

3. Reasoning (or “thinking”) model

People noticed a simple prompting trick: if you add “think step by
step” to your request, the model often gets harder problems right,
because it works through them methodically. This is called chain-
of-thought prompting.

That inspired the next step: train a model to always write out its
reasoning before giving a final answer. The result is a reasoning
model (or thinking model). It first produces its thought process,
then its conclusion. (You may have seen a model show its
“thinking” in a different color before answering — that’s this.)

• How much thinking a reasoning model does is called its

reasoning budget or reasoning effort.
• Budget forcing means making it think longer. Amusingly, one
well-known trick (from a 2025 paper called S1) is literally to
insert the word “wait” into the model’s thinking — which
nudges it to reconsider and reason more deeply. The tricks here
are often surprisingly low-tech.

20 / 52
4. Hybrid model
A hybrid model decides for itself how much to reason based on
the question. A simple “hi there” gets almost no reasoning (fast,
like a chat model); a hard puzzle triggers lots of it. Most of the
newest flagship models (e.g. GPT-5, Gemini 2.5 Pro) are hybrids.

Which type should you use?

• Reasoning models are best for hard problem-solving, puzzles,
and anything needing careful multi-step logic. They score
highest on difficult tests — but they’re slower and cost more
(you pay for all those thinking tokens).
• Chat models are faster, cheaper, and better for interactive
back-and-forth. Many people find them better for creative, free-
flowing writing, where reasoning models can feel over-analytical.
• Base models are the right starting point only when you intend
to train the model yourself.

A note on terminology
You’ll also hear foundation model, used much like “frontier
model”: it means a large, general-purpose model that many other
products are built on top of. The terms are fuzzy and often used
interchangeably.

21 / 52
Part 9 — The Chat Completions API and
Endpoints

What an API and an endpoint are

An API (Application Programming Interface) is simply a way for
your code to talk to someone else’s service over the internet. An
endpoint is the specific web address (a URL) you send your
request to. You send data to the endpoint, and it sends data back.

The Chat Completions API

The most common way to call an LLM is the Chat Completions
API. The name describes what it does: you give it a conversation
(a list of messages), and it “completes” it by predicting what
should come next — the assistant’s reply.

This API was invented by OpenAI, but it became so popular that

nearly everyone adopted the same design.

Calling the endpoint directly with raw HTTP

You can talk to the endpoint using plain web requests, with no
special LLM library at all. This shows what’s really happening
beneath the surface:

22 / 52
 import requests

# The endpoint address and your credentials

url = "[Link]
headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "
application/json"}

# The request body: which model, and the messages

payload = {
"model": "gpt-5-nano",
"messages": [{"role": "user", "content": "Tell me a fun fact"}
]
}

# Send it and read the reply (which comes back as JSON)

response = [Link](url, headers=headers, json=payload)

# Dig into the JSON to find the actual text

print([Link]()["choices"][0]["message"]["content"])

Notice how the reply is a nested structure — the useful text lives
inside choices[0]["message"]["content"] . Working with raw JSON like
this is fiddly, which is exactly why the client library (next) exists.

Part 10 — The OpenAI Client Library

What the openai package really is

The openai Python package is a client library — a convenient
wrapper around the exact same HTTP call you saw above. It lets
you write clean Python instead of hand-building JSON.

A common misconception: the openai package does not contain

the GPT model itself. The model lives on OpenAI’s servers. The
library is small, open-source, and only helps you call the endpoint.
No AI model is downloaded to your machine.

23 / 52
The same request, made simple

from openai import OpenAI

# Create a client (it reads your API key automatically)

openai = OpenAI()

# Make the call — much cleaner than raw HTTP

response = [Link](
model="gpt-5-nano",
messages=[{"role": "user", "content": "Tell me a fun fact"}]
)

# The text you want is here:

print([Link][0].[Link])

Setting up API keys safely

To use a paid service like OpenAI, you need an API key — a secret
password that identifies your account. You should never paste keys
directly into your code. Instead, store them in a file named .env
and load them at runtime:

import os
from dotenv import load_dotenv

# Read the secret values from the .env file

load_dotenv(override=True)
api_key = [Link]('OPENAI_API_KEY')

# A quick sanity check that the key was found and looks right
if not api_key:
print("No API key found — check your .env file.")
elif not api_key.startswith("sk-proj-"):
print("A key was found, but it doesn't look like an OpenAI
key.")
else:
print("API key found and looks good!")

Your .env file simply contains lines like:

24 / 52
 OPENAI_API_KEY=sk-proj-//.your-key-here//.

Part 11 — The Development

Environment (The Tools You’ll Use)
Before writing LLM code, it helps to know the handful of tools you’ll
work in. You don’t need to master them — just recognize what each
one is for.

Git and GitHub (they’re not the same)

• Git is a tool on your computer for tracking changes to code.
• GitHub is a website that stores code projects (called
repositories, or repos) online.

To get a copy of an online project onto your machine, you clone it:

git clone [Link]

An editor (IDE): Cursor or VS Code

You write and run code in an IDE (Integrated Development
Environment) — an editor with helpful features. Popular choices are
Cursor (an AI-assisted editor that can autocomplete and explain
code) and VS Code (which Cursor is based on). Any editor works;
the code is the same.

25 / 52
uv: managing Python and packages
Real projects depend on many Python packages (reusable code
libraries). Keeping the right versions of everything installed and
consistent is handled by an environment manager. uv is a fast,
popular one. A single command sets up everything a project needs:

uv sync

This builds a virtual environment — a self-contained folder (often

named .venv ) holding the exact Python version and packages for
this project, isolated from everything else on your computer.

Jupyter notebooks: where the experimenting

happens
Most learning labs are Jupyter notebooks (files ending in .ipynb ,
also called “labs”). A notebook mixes formatted text and code in a
single document, split into blocks called cells. You run a cell
(typically with Shift + Enter) and see its output right below it.
This makes notebooks perfect for experimenting: change
something, run it, see what happens, repeat.

A notebook runs on a kernel — the background Python process

that actually executes your cells. You pick the kernel that points to
your project’s virtual environment, or nothing will run correctly.

This experimental, tweak-and-rerun mindset is itself an

important skill. Getting good results from LLMs is largely about
iterating: try a prompt, see the output, refine it, try again.

26 / 52
Getting an API key (and the cost side)
To call a paid service like OpenAI, you sign up on its platform (the
developer side, separate from the ChatGPT product) and create an
API key — a secret password linking your code to your account.
Paid APIs are usually pay-as-you-go: you add a small balance (e.g.
a $5 minimum) and draw it down as you use the service. Everyday
experimenting costs tiny fractions of a cent, and keeping “auto-
recharge” off means you stay in full control of spending. As
covered earlier, the key goes in a .env file, never directly in your
code.

Part 12 — Using Other Providers and

Running Models Locally

OpenAI-compatible endpoints
Because the Chat Completions API became a de-facto standard,
other providers (like Google) built endpoints that behave
identically. These are called OpenAI-compatible endpoints.

The friendly result: you can reuse the very same openai client
library to talk to a different provider — you just point it at a
different URL and give it that provider’s key. Again, no OpenAI
model is involved; you’re only borrowing the lightweight library.

27 / 52
 # Using Google Gemini through the OpenAI-compatible endpoint
gemini = OpenAI(
base_url="[Link]
openai/",
api_key=google_api_key # a Google key, starts with "AIz..."
)

response = [Link](
model="gemini-2.5-flash-lite",
messages=[{"role": "user", "content": "Tell me a fun fact"}]
)
print([Link][0].[Link])

Running models locally with Ollama

Ollama is a free tool that runs open-source LLMs directly on your
own computer. It also provides an OpenAI-compatible endpoint —
but on your local machine instead of the internet.

After installing Ollama from [Link], the server runs at http://

localhost:11434 . You download a model once with a terminal
command:

ollama pull llama3.2 # a small Meta model (use

llama3.2:1b if your computer is limited)
ollama pull deepseek-r1:1.5b # a small reasoning-focused model

Then you call it with the same familiar code, just pointed at your
local address:

28 / 52
 # Point the OpenAI client at your local Ollama server
ollama = OpenAI(base_url="[Link] api_key="oll
ama")

response = [Link](
model="llama3.2",
messages=[{"role": "user", "content": "Tell me a fun fact"}]
)
print([Link][0].[Link])

Frontier vs. local: the trade-off

Frontier models (e.g. Local open-source models

GPT, Gemini) (via Ollama)

Power Most capable Noticeably less capable

Cost Pay per token Free to run

Your text goes to the Your data never leaves your

Privacy
provider machine

Download and run the model

Setup Just an API key
yourself

A common pattern for learners: use free local models while

experimenting, and switch to a frontier model when you need the
best possible quality.

Chatting with Ollama straight from the command

line
Before calling Ollama from Python, you can talk to a local model
directly in a terminal — a quick way to try one out:

29 / 52
 ollama run gemma3:270m
# download (if needed) and start chatting with a tiny model
ollama run phi3 # a larger, more capable model (needs
more disk/RAM)

Type a message and the model replies right there. Press Ctrl-D to
end the chat. Bigger models give more capable conversations but
need more disk space and memory. Running a model like this —
feeding it input and getting output — is what’s technically called
inference.

Part 13 — The Landscape of Models

(Who Makes What)
There are many LLMs out there. Here’s a map so the names stop
being a blur. The big dividing line is closed source vs. open
source.

Closed source vs. open source

• Closed source models are owned by a company. You can’t
download them; you use them by paying to call their service.
Training them costs hundreds of millions of dollars, so the
companies charge for access. These are what people usually
mean by frontier models, made by the big frontier labs.
• Open source models are released so anyone can download and
run them for free. (Purists point out these are often more
precisely open weight: the trained model is shared, but not
always the full training data or recipe. In everyday speech
people still say “open source.”)

30 / 52
The major closed-source models
• GPT — from OpenAI, the most famous lab. GPT-5 is a hybrid
chat-and-reasoning model; the earlier o-series (o1, o3, o4-mini)
were pure reasoning models, and GPT-4.1 is a fast pure chat
model many people still like. The chat product is ChatGPT.
• Claude — from Anthropic (founded by ex-OpenAI staff). Comes
in haiku / sonnet / opus sizes; sonnet is often the sweet spot.
It’s the model behind the Claude Code coding tool and is a
community favorite.
• Gemini — from Google. Started behind (remember “Bard”?) but
caught up fast; strong on multimodal and huge context
windows.
• Grok — from xAI (Elon Musk’s company). Spelled with a k.
(Don’t confuse it with Groq spelled with a q — that’s a different
company, mentioned below.)
• Others in the next tier include Mistral (French; has both closed
and open models), Cohere, and the model behind Perplexity
(known for real-time search).

The major open-source models

• Llama — from Meta; the model that really launched open-
source LLMs. Llama 3.2 comes in tiny 1B and 3B sizes that run
well on a laptop; Llama 4 is the larger, newer line.
• Mistral — from the French company Mistral; uses the mixture-
of-experts design.
• Qwen — from Alibaba Cloud (China); powerful and underrated.
• Gemma — Google’s open-source cousin of Gemini, famous for
an extremely tiny 270-million-parameter version.
• Phi — Microsoft’s small, capable models, good at tool use and
commercial tasks.

31 / 52
 • DeepSeek — from the Chinese lab DeepSeek AI; caused a stir
not for beating the top models but for reaching near-frontier
quality at a fraction of the training cost (reportedly a few million
dollars versus $100M+). Its full model is 671B parameters, too
big for a laptop.
• GPT-OSS — OpenAI’s own open-source model, released in 20B
and 120B sizes.

Distillation (how the small DeepSeek variants

were made)
The small “DeepSeek” models you can run locally aren’t shrunk-
down DeepSeeks at all. Instead, the big DeepSeek generated lots
of synthetic data (AI-generated training examples), which was
then used to train small Llama and Qwen models to imitate it.
Teaching a small model to copy a big one this way is called
distillation.

Part 14 — Three Ways to Use a Model

There are three broadly different ways to actually use an LLM.
You’ll meet all three.

1. Packaged products
These are finished apps with a user interface, like ChatGPT,
Claude, or Gemini’s chat site. A key distinction: you’re using a
product, not a model directly. The product wraps a model with
extra features — a chat interface, memory, web search, file
uploads — that were built by engineers on top of the underlying
model. When ChatGPT searches the web, that search isn’t part of
the LLM; it’s added functionality glued around it.

32 / 52
2. Cloud APIs
Here your own code calls a model running on a server somewhere.
This is what most of this material focuses on. There are a few
varieties:

• Direct APIs — calling a provider straight, like OpenAI’s

endpoint (what we’ve been doing).
• Managed cloud services — big platforms that host and run
models for you: Amazon Bedrock, Google Vertex AI, and
Azure ML. You call the service and it calls the model.
• Fast-inference services — e.g. Groq (spelled with a q), which
runs open-source models very quickly using specialized
hardware. (Again: not the same as Elon Musk’s Grok with a k.)
• Routers — e.g. OpenRouter, a single connection point that
forwards your request to whichever of many providers you
choose.

You can reach both closed and open models this way.

3. Running a model yourself (locally)

If a model is open source, you can download and run it on your
own computer — no cloud, no API charges, and your data never
leaves your machine. Two main tools do this, and they’re different:

• Hugging Face Transformers library — you literally run the

model’s own code (a neural network) inside your Python
program. It’s the most direct approach: take the open-source
code and the billions of trained weights, and execute them
yourself.
• Ollama — a product that packages selected open-source
models to run easily and efficiently on your machine. It
compresses the model into an optimized file (a GGUF file) and,

33 / 52
 once running, gives you a local endpoint to call — like an API,
but pointed at your own computer.

So Ollama is the friendly, packaged, fast option for a curated set of

models; Hugging Face Transformers is the more general, do-it-
yourself option for running open-source model code directly.

Part 15 — Your First Real Project:

Summarize a Web Page
Now we combine the pieces into something genuinely useful: give
the program a URL, and it returns a short summary — like a
“Reader’s Digest” of any web page.

Step 1 — Get the page’s text

You first fetch the visible text from the website (a technique called
web scraping). Here we assume a helper,
fetch_website_contents(url) , that returns the page text.

from scraper import fetch_website_contents

website_text = fetch_website_contents("[Link]

34 / 52
Step 2 — Write the prompts

system_prompt = """
You are an assistant that summarizes the contents of a website.
Respond in markdown. Ignore navigation-related text.
"""

user_prompt_prefix = """
Here are the contents of a website. Provide a short summary.
If it includes news or announcements, summarize those too.

"""

Step 3 — Put it all together

from openai import OpenAI

from [Link] import Markdown, display

openai = OpenAI()

def summarize(url):
website_text = fetch_website_contents(url)
messages = [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_prompt_prefix + website_t
ext}
]
response = [Link](
model="gpt-4.1-mini",
messages=messages
)
return [Link][0].[Link]

def display_summary(url):
summary = summarize(url)
display(Markdown(summary)) # show it nicely formatted

display_summary("[Link]

35 / 52
A note on the limits of simple scraping
This basic approach reads the text that’s already in the page’s
HTML. It will not work on sites that build their content with
JavaScript after loading (many modern apps do this) — those pages
come back empty. Some sites also block automated requests and
return errors. Tools like Selenium or Playwright, which run a real
browser behind the scenes, can handle those harder cases.

Part 16 — Guiding the Model’s Output:

One-Shot Prompting and JSON
As projects grow, you often need the model to respond in a
precise, predictable format so your code can use the answer
automatically.

One-shot prompting
One-shot prompting means giving the model one worked
example of what you want inside the prompt, so it copies the
pattern. (Zero-shot = no example; few-shot = several examples.)
Showing an example is one of the simplest ways to get reliable
results.

Asking for structured JSON

JSON is a simple text format for structured data that code can
easily read. Many LLM APIs let you require the reply to be valid
JSON by setting response_format={"type": "json_object"} . Combined
with a one-shot example, this makes the output dependable.

36 / 52
 # System prompt includes ONE example of the exact JSON shape we
want
link_system_prompt = """
You are given a list of links from a webpage.
Decide which links are relevant for a company brochure
(such as About or Careers pages), and respond in JSON like this:

{
"links": [
{"type": "about page", "url": "[Link]
{"type": "careers page", "url": "[Link]
careers"}
]
}
"""

import json

def select_relevant_links(url):
response = [Link](
model="gpt-5-nano",
messages=[
{"role": "system", "content": link_system_prompt},
{"role": "user", "content": get_links_user_prompt(url)
}
],
response_format={"type": "json_object"} # force valid
JSON
)
result = [Link][0].[Link]
return [Link](result)
# turn the JSON text into a Python dictionary

This is a great example of a task that would be painfully hard to

code by hand (deciding which links “look relevant” needs real
understanding of language), but is easy for an LLM.

One-shot vs. multi-shot prompting

• One-shot prompting gives the model one example to copy
(what we did above).

37 / 52
 • Multi-shot (or few-shot) prompting gives several examples.
Strictly, this means a series of example question-and-answer
pairs, so the model sees the pattern of good responses. Adding
more good examples — and even counter-examples (“don’t do it
like this”) — is one of the most reliable ways to improve output
quality. This is a practical use of the context window.

Why JSON works so well

LLMs are trained on three kinds of data more than anything else:
natural language, markdown (a simple text-formatting shorthand
for web content), and JSON (structured data). Because they’ve
seen so much JSON, they’re very good at both reading it and
producing it. So when you want structured output, describing it as
a clean JSON shape is far more reliable than inventing your own
“put it in bullet points like this” format.

What response_format actually does

Setting response_format={"type": "json_object"} isn’t just a polite
request. When a model generates text, it doesn’t pick one definite
next token — it produces a probability for every possible next
token. This setting constrains the choice at inference time so
that only tokens which keep the output as valid JSON can be
selected. In effect, the model is forced to produce well-formed
JSON, no matter what it was “leaning” toward.

A more advanced version, called structured outputs, lets

you require the JSON to match an exact specification (a
schema). It’s used in more advanced agentic projects.

38 / 52
Part 17 — Combining Multiple Calls: A
Multi-Step Solution
Real business tools rarely rely on a single LLM call. Chaining
several calls together — where the output of one feeds the next —
is the first taste of Agentic AI (systems where LLMs work through
a task in multiple coordinated steps).

Example: a company brochure generator

The goal: given a company name and website, produce a polished
brochure. This takes several steps:

1. Fetch the links on the company’s landing page.

2. First LLM call: decide which links are relevant (using JSON
output, from Part 9).
3. Fetch the text of each relevant page and combine it.
4. Second LLM call: turn all that content into a brochure.

39 / 52
 def fetch_page_and_all_relevant_links(url):

# Combine the landing page text with the text of each relevant
sub-page
contents = fetch_website_contents(url)
relevant = select_relevant_links(url) # LLM call #1
result = f"/# Landing Page:\n\n{contents}\n/# Relevant Links:\
n"
for link in relevant['links']:
result += f"\n\n//# {link['type']}\n"
result += fetch_website_contents(link["url"])
return result

brochure_system_prompt = """
You are an assistant that analyzes several pages from a company
website
and creates a short brochure for customers, investors, and
recruits.
Respond in markdown. Include company culture, customers, and
careers if available.
"""

def create_brochure(company_name, url):

combined_text = fetch_page_and_all_relevant_links(url)
combined_text = combined_text[:5000] # keep the input to a
sensible size
response = [Link]( # LLM call #2
model="gpt-4.1-mini",
messages=[
{"role": "system", "content": brochure_system_prompt},
{"role": "user", "content": f"Company: {company_name}\
n\n{combined_text}"}
]
)
display(Markdown([Link][0].[Link]))

Controlling tone
Notice that switching from a professional brochure to a humorous
one takes nothing more than rewording the system prompt (e.g.,
“create a witty, entertaining brochure”). Tone is controlled entirely
through the prompt — no code changes needed.

40 / 52
Keeping input to a reasonable size
The line combined_text[:5000] truncates the text to the first 5,000
characters. Because you pay per token and models have size limits,
trimming overly long input keeps calls cheap and reliable.

Part 18 — Streaming: The Typewriter

Effect
By default, you wait for the whole reply and then see it all at once.
Streaming instead sends the response back piece by piece as
it’s generated, producing the familiar typewriter animation you
see in ChatGPT.

You turn it on with stream=True and loop over the incoming chunks:

def stream_brochure(company_name, url):

stream = [Link](
model="gpt-4.1-mini",
messages=[
{"role": "system", "content": brochure_system_prompt},
{"role": "user", "content": get_brochure_user_prompt(c
ompany_name, url)}
],
stream=True # ask for the reply in pieces
)

response = ""
display_handle = display(Markdown(""), display_id=True)
for chunk in stream:
# Each chunk carries a little more text in .[Link]
response += [Link][0].[Link] or ''
update_display(Markdown(response), display_id=display_hand
le.display_id)

Streaming doesn’t change the answer — it only changes how it

arrives, making the app feel faster and more responsive.

41 / 52
Streaming also reflects what’s really happening inside the model: it
generates the reply one token at a time, so it can hand each token
to you the moment it’s produced. Each small piece that arrives is
called a chunk, and the new bit of text in a chunk lives in
.[Link] (a “delta” is just “the extra bit since last time”).
Under the hood this uses a web technology called SSE (Server-Sent
Events) to push the pieces to you as they’re ready.

Part 19 — What Frontier Models Are

Great and Bad At
It’s worth having a realistic picture of where these models shine
and where they trip up.

Strengths
• Synthesizing information — reading a lot of material and
producing a clear, well-structured, balanced summary or a pros-
and-cons analysis. This is one of their most reliable talents.
• Generating content — drafting emails, presentations, outlines,
and first versions of almost anything. They’re excellent for
fleshing out a rough idea into a starting skeleton.
• Coding — writing, explaining, and debugging code, often
iterating in a loop of write-test-fix. They’ve largely replaced
older resources like Stack Overflow as the first place engineers
turn for help.

Weaknesses
• Knowledge cutoff — a model only knows what existed up to
when it was trained. Ask about newer events and it simply

42 / 52
 doesn’t know (unless the product adds web search on top). It
can even confidently insist a real, recent thing “doesn’t exist.”
• Knowledge gaps — expertise is uneven. A model may be
above PhD level in one area and shaky in another.
• Confident mistakes (hallucination) — because a model only
predicts plausible next tokens, it can produce something that
sounds authoritative but is wrong, stated with complete
confidence. (It’s arguably surprising they’re right as often as
they are.)
• Jumping to conclusions — especially in coding, a model tends
to slap a patch on the obvious symptom and push forward,
rather than stepping back to find the real root cause. A classic
trap: a beginner gets pages of increasingly elaborate AI-
generated code fixing the wrong problem, and doesn’t realize
they’ve been led astray.

The takeaway: use them under supervision

Think of a frontier model as a tireless, brilliant junior analyst. It
can do an enormous amount of work quickly — but it’s your job to
check that work, keep it on track, and challenge it when it heads
down a wrong path. These models perform best with a
knowledgeable human supervising them, which is one reason they
can be more useful to an experienced person (who can catch
mistakes) than to a total beginner (who can’t).

Part 20 — Understanding API Costs

We touched on cost when talking about memory. Here’s the fuller
picture, because it’s most people’s biggest worry.

43 / 52
Subscriptions vs. paying per use
There are two separate ways money changes hands, and they’re
unrelated:

• Chat product subscriptions (like ChatGPT Plus) — a flat

monthly fee, roughly $20 to $200/month, for using the product
with some usage limits.
• API usage — when your code calls the model, you pay per
use, whether or not you also have a subscription. This is what
you’d pay to build your own product on top of a model.

What you pay for

API cost is based on the number of tokens, split into two kinds:

• Input tokens — everything you send in. Remember this

includes the entire conversation history (the fake memory) plus
any extra material you’ve added. This is why longer
conversations cost more.
• Output tokens — everything the model generates. For
reasoning models, this includes the model’s thinking steps —
and with some models you’re billed for that thinking even
though you never see it.

Both of these are the price of the actual computation (“the

electricity bill”) for those trillions of calculations — plus a bit
toward recouping the model’s training cost.

The scale of pricing

Prices are usually quoted per million tokens. As a snapshot:

• GPT-5: about $1.25 per million input tokens and $10 per
million output tokens.

44 / 52
 • GPT-5 nano (the tiny version): about $0.05 per million input
and $0.40 per million output.

For perspective, generating the entire complete works of

Shakespeare (~1.2M tokens) would cost roughly $10 on the big
model — or under a dollar on the nano. For a single “hi, my name
is Ed” style request, the cost is a fraction of a fraction of a cent.
Costs only become a real concern when you run many
conversations at scale or use agent loops that churn through lots of
tokens.

A couple of extra wrinkles

• Caching — if you send the same input again within a short
window, some providers charge less because part of it is
cached. With OpenAI this can be automatic.
• Leaderboards — sites like the Vellum leaderboard compare
models side by side, including their context window sizes and
per-million token prices — handy for choosing a model that
balances intelligence, speed, and cost for your needs.

Part 21 — Advanced Applications:

Prompt Engineering, Context
Engineering, and Agentic AI
A few ideas keep coming up as you move beyond single, simple
calls. Here they are in plain terms.

45 / 52
Prompt engineering
Prompt engineering is the craft of writing prompts that get good
results — being clear about the task, giving context, specifying the
style and format, and showing examples. It was briefly a
standalone, high-paying job title; now it’s just a basic skill everyone
working with LLMs picks up.

Context engineering
Context engineering is the newer, broader idea: thinking
carefully about all the information you place into the model’s input
to set it up for success. That includes the prompt, but also
business-specific data you stitch in, and tools the model can use
(see below). It sounds fancy, but at heart it’s simple — give the
model the right input so its predicted output matches your goal. If
you want it to quote correct ticket prices, those prices need to be
in the input.

Tools
A tool is a piece of functionality you give the model access to —
like searching the web, running code, or looking something up in a
database. Rather than answering only from memory, the model can
decide to use a tool and incorporate the result. Tools are a big part
of what makes modern assistants so capable.

Agentic AI
Agentic AI is currently the hottest topic in the field. Two common
definitions:

1. A system where an LLM controls the workflow — it decides

what happens next, possibly calling other LLMs or tools.

46 / 52
2. An LLM running in a loop with tools — it’s called repeatedly,
and on each pass it can take actions (use tools) and decide its
next step.

People often describe agents with the word autonomy: the model
appears to choose its own next moves. Under the hood it’s still just
predicting tokens — but when those predicted tokens describe
actions to take next, the system behaves autonomously. That
repeated call-act-decide cycle is an agent loop.

You’ve likely already seen agentic AI in action:

• Deep research — a feature where the assistant goes off, reads

many sources over several minutes, and returns a written
report. It’s really many chained LLM calls working together.
• Agent mode — the assistant driving a real browser to carry out
a task (e.g. finding and booking a restaurant), clicking and
typing on your behalf.
• Coding agents / copilots — tools like Claude Code, cursor
agents, and GitHub/Microsoft Copilot that read your project, plan
a to-do list, and work through it step by step. Watching one tick
off tasks in a loop is agentic AI made visible.

These are the building blocks the rest of an LLM-engineering

journey builds toward — including related techniques you’ll meet
later such as RAG (feeding a model relevant documents so it can
answer about them) and fine-tuning (training a model further for
a specific job).

47 / 52
Part 22 — Where This Is Useful
(Business Applications)
The small projects above map directly onto real, valuable use
cases:

• Summarization — condense news, financial reports, resumes,

long documents, or web pages into short summaries. One of the
most common and immediately useful LLM applications.
• Content generation — produce brochures, marketing copy,
product tutorials from a spec, personalized emails, and more.
• Structured extraction — pull clean, structured data (like JSON)
out of messy text, so other software can use it.
• Multi-step / agentic workflows — chain several LLM calls
together to handle tasks too complex for a single prompt, the
foundation of more advanced autonomous AI systems.

The underlying skills — prompting, the messages format, calling

APIs, controlling output, and combining calls — apply across nearly
every provider and almost any industry.

Quick Glossary
• LLM (Large Language Model): a program trained on huge
amounts of text that predicts the next chunk of text.
• Frontier model: one of the most powerful, state-of-the-art
LLMs (e.g., GPT, Gemini, Claude), usually accessed as a paid
online service.
• Token / token ID: the small pieces (and their numeric codes)
that text is broken into for the model.
• Stateless: having no memory between requests; each call
starts fresh.

48 / 52
• Prompt: the text you send the model. A system prompt sets
role and tone; a user prompt is the actual question.
• Message: a {"role": //., "content": //.} dictionary. Roles are
system , user , and assistant .

• API / endpoint: a way for your code to call a service, and the
specific URL it calls.
• Chat Completions API: the standard way to ask an LLM to
continue a conversation.
• Client library: a package (like openai ) that wraps the API in
convenient code; it does not contain the model.
• OpenAI-compatible endpoint: another provider’s endpoint
that works with the same code and format.
• Ollama: a free tool for running open-source LLMs locally on
your own machine.
• API key: a secret password identifying your account; kept out
of code in a .env file.
• Web scraping: programmatically fetching the text content of a
web page.
• One-shot prompting: including one example in the prompt so
the model copies the pattern.
• JSON: a structured text format for data that code can easily
read.
• Streaming: receiving the model’s reply piece by piece as it’s
generated (the typewriter effect).
• Agentic AI: systems that chain multiple LLM calls to work
through complex tasks.
• Transformer: the neural-network design (from the 2017 paper
Attention Is All You Need) behind modern LLMs; the “T” in GPT.
• GPT: Generative Pre-trained Transformer — generative
(produces text), pre-trained (trained in advance), transformer
(its design).

49 / 52
• Neural network / deep learning: a model built from many
connected “neurons”; “deep” means many stacked layers.
• Self-attention: the transformer step that decides which earlier
parts of the input matter most for predicting the next token.
• Emergent intelligence: the surprising way real capability
appears once a model is large enough.
• Parameter: one adjustable number (“dial”) inside a model;
models have millions to trillions of them.
• Inference: running a trained model to get an output (as
opposed to training it).
• Training-time vs. inference-time scaling: getting more out
of a model by making it bigger/training more vs. by using tricks
when you run it.
• Mixture of experts (MoE): a big model made of many
specialist sub-models, each used only when relevant.
• Small language model (SLM): a deliberately small LLM.
• Context window: the maximum number of tokens a model can
consider at once (history + input + generated reply).
• Base model: a raw next-token predictor, not trained to chat
(like phone predictive text).
• Chat / instruct model: a model trained (via RLHF) to work as
a back-and-forth assistant.
• RLHF (Reinforcement Learning from Human Feedback):
training that nudges a model toward responses people prefer;
turned GPT into ChatGPT.
• Chain-of-thought prompting: asking a model to “think step
by step” to improve hard answers.
• Reasoning / thinking model: a model trained to write out its
reasoning before answering.
• Hybrid model: a model that decides for itself how much to
reason.

50 / 52
• Reasoning budget / budget forcing: how much a reasoning
model thinks, and techniques to make it think longer.
• Foundation model: a large general-purpose model others build
on; used much like “frontier model.”
• Closed source vs. open source (open weight): models you
can only pay to call vs. models you can download and run
yourself.
• Frontier lab: a leading company that trains top-tier models
(OpenAI, Anthropic, Google, etc.).
• Distillation: training a small model to imitate a big one, often
using AI-generated (synthetic) data.
• Managed cloud service: a platform (Bedrock, Vertex AI, Azure
ML) that hosts and runs models you call.
• Groq (with a q): a service that runs open-source models very
fast on special hardware — not Grok (with a k), which is xAI’s
model.
• Hugging Face Transformers: a library for running open-source
model code directly on your machine.
• GGUF: the compressed file format Ollama uses to store a
model.
• Hallucination: a confident but incorrect answer from a model.
• Knowledge cutoff: the date beyond which a model has no
built-in training knowledge.
• Multi-shot / few-shot prompting: giving several examples in
the prompt (one example = one-shot).
• Structured outputs: forcing a model’s output to match an
exact data specification (schema).
• Prompt engineering: the craft of writing effective prompts.
• Context engineering: thoughtfully assembling all the input
(prompts, data, tools) that a model receives.

51 / 52
• Tool: functionality (web search, code execution, database
lookup) a model can choose to use.
• Agent loop / autonomy: an LLM called repeatedly with tools,
deciding its own next steps.
• RAG (Retrieval-Augmented Generation): feeding a model
relevant documents so it can answer about them.
• Fine-tuning: training an existing model further to specialize it
for a task.
• API costs: pay-per-use charges for API calls, based on input
and output tokens (usually priced per million).
• Chunk / delta: a small streamed piece of a reply, and the new
text it adds.
• Git / GitHub / repo: a change-tracking tool, the website that
hosts code, and a stored project.
• IDE (Cursor / VS Code): the editor you write and run code in.
• uv / virtual environment: a tool that installs a project’s exact
Python and packages into an isolated folder.
• Jupyter notebook / cell / kernel: an interactive code-and-text
document, its runnable blocks, and the Python process that runs
them.

52 / 52