Patterns of Intelligence
Explaining Large Language Models
1948 — 2026
CHAPTER I: STATISTICS
The Counting Era
1948 O(Vⁿ)
Shannon's N-gram Proposal The Memory Bottleneck
Language modeling began with counting words. For half a
century, models were tables of conditional probabilities
estimated by counting occurrences in a corpus.
n≤5 0.31
Practical Context Limit Typical High Probability
Shannon proposed this formulation in 1948, powering early
autocomplete and speech recognition.
CHAPTER II: GEOMETRY
2013
Word2vec: Meaning as Space
Mikolov demonstrated that dense vectors, trained to
predict neighbors, capture analogies as arithmetic.
High-dimensional semantic vectors mapping relationships.
Embeddings became the lingua franca of NLP, moving
from discrete symbols to continuous geometry.
CHAPTER III: ARCHITECTURE
Attention Over Recurrence
The 2017 paper "Attention Is All You Need" replaced
sequential RNNs with a fully parallel architecture.
Parallelization: No longer processing word by word.
Global Context: Every token "sees" every other token.
Scalability: Scales almost linearly with compute.
SELF-ATTENTION MECHANISM
How a model understands context: Visualizing attention weights for "the cat sat on the mat".
Token the cat sat on the mat
the 0.60 0.10 0.10 0.05 0.10 0.05
cat 0.10 0.50 0.25 0.05 0.05 0.05
sat 0.05 0.30 0.40 0.10 0.05 0.10
mat 0.05 0.35 0.10 0.05 0.10 0.35
Note: High weights (shaded) show the model linking related concepts across the sequence simultaneously.
CHAPTER IV: SCALE
175B
GPT-3 Parameters
Emergent Abilities
Scale unlocked "In-context learning"—solving
tasks from examples with no gradient updates.
Kaplan's scaling laws: Loss falls as a power law in
parameters, data, and compute.
CHAPTER V: PRODUCT
RLHF Chat UI Growth
Reinforcement Learning from ChatGPT wrapped complexity in 100M users in 2 months—the
Human Feedback teaches a simple interface, making LLMs fastest growing consumer
models to follow instructions. consumer-ready. product in history.
The Shift: LLMs evolved from mere "Completion Engines" to helpful "Assistants" through human alignment.
CHAPTER VI: DISTRIBUTION
The Open Weights Era
Meta's LLaMA series and others narrowed the gap between closed and open frontiers from years to months.
Feb 2023 Sep 2023 Apr 2024 2025
LLaMA Mistral 7B LLaMA 3 DeepSeek R1
Open Era Kicked Off Efficiency Breakthrough Frontier Weights Reasoning at Scale
CHAPTER VII: MODALITIES
Beyond Text
The input stream widened. The same transformer backbone
now accepts tokens from all modalities.
Image Audio
(GPT-4V) (Whisper)
Video Action (Tool
(Sora) Use)
CHAPTER VIII:
REASONING
System 2 Thinking
Reasoning is now a first-class training objective.
Chain-of-Thought showed models think better when
allowed to write steps.
o1 and DeepSeek-R1 use Reinforcement Learning on long
internal reasoning traces, trading test-time compute for
accuracy.
THINKING: Finding prime factors of 2024...
Answer: 11
CHAPTER IX: AGENCY
Models as Actors
Agents turn words into actions. A model that can call tools,
observe results, and iterate is an agent.
Tool use and function calling (2023)
Browser and Computer control (2024-25)
Unsupervised long-horizon tasks
CODA: THE ROAD AHEAD
Pre-training Scale Test-Time Compute
Is data finite? Frontier runs now cost $100M+. How elastic is reasoning compute at inference?
Agency Reliability Weight Distribution
Can agents generalize over long bottlenecks? Open vs Closed: Nations vs Labs.
Questions?
Thank you for your attention.
IMAGE SOURCES
[Link]
esents-artifi[Link]
Source: [Link]
[Link]
Source: [Link]
[Link]
Source: [Link]
[Link]
Source: [Link]
[Link]
Source: [Link]