Developing AI from Zero

Developing AI from Zero

A beginner's guide to machine learning, in Python

This guide builds a real, working neural network from scratch, in plain Python, so the math is never
hidden behind a black box. Each chapter adds one piece: data, a model, a way to measure error, and
a way to learn from it. No prior AI experience assumed.

Machine Learning Edition Page 1

Developing AI from Zero

01 What Is AI, Really?

"Artificial intelligence" is a broad label. The part you can actually build from zero is machine learning:
instead of writing explicit rules, you give a program examples, and it works out its own rules by adjusting
internal numbers until its guesses match the examples well.

The core loop, in plain language

• Guess — the model makes a prediction using its current internal numbers.

• Measure — compare the guess to the real answer; the difference is the "error" or "loss".

• Adjust — nudge the internal numbers slightly to make that error smaller next time.

• Repeat — thousands of times, over many examples, until the guesses become useful.

That four-step loop, called training, is the entire idea behind every AI model in this guide — and behind
every large model in production today.

Machine Learning Edition Page 2

Developing AI from Zero

02 Setting Up
You need Python and one library, numpy, which handles the number-crunching. Nothing else is required to
build and train a real model from scratch.

pip install numpy

Create a file [Link], and confirm everything works:

import numpy as np

x = [Link]([1, 2, 3])
print(x * 2) # [2 4 6]
print("Setup complete.")

Run it with python [Link]. If you see [2 4 6] printed, numpy is working and you are ready to build the first
model.

Machine Learning Edition Page 3

Developing AI from Zero

03 The Simplest Model: Linear Regression

The smallest useful model is a straight line: y = w*x + b. Given enough example points, the model learns the
slope w and offset b that fit them best. This is the same shape of problem as every larger model — just with
one number instead of millions.

import numpy as np

# Example data: hours studied -> exam score

x = [Link]([1, 2, 3, 4, 5], dtype=float)
y = [Link]([52, 60, 68, 76, 84], dtype=float)

w = 0.0 # slope, starts as a guess

b = 0.0 # offset, starts as a guess

def predict(x, w, b):

return w * x + b

Right now w and b are both zero, so every prediction is zero — clearly wrong. Chapter 4 fixes that by
teaching the model to adjust them.

Machine Learning Edition Page 4

Developing AI from Zero

04 Measuring Error and Learning

First, define how wrong a prediction is. A common choice is mean squared error: the average of (prediction
− real answer) squared.

def loss(x, y, w, b):

predictions = predict(x, w, b)
return [Link]((predictions - y) ** 2)

Next, gradient descent: nudge w and b a small step in the direction that reduces the error, and repeat.

learning_rate = 0.01

for step in range(1000):

predictions = predict(x, w, b)
error = predictions - y

# how much each parameter contributed to the error

dw = [Link](2 * error * x)
db = [Link](2 * error)

w -= learning_rate * dw
b -= learning_rate * db

if step % 200 == 0:
print(f"step {step}, loss={loss(x, y, w, b):.2f}")

print(f"Learned: w={w:.2f}, b={b:.2f}")

Run this and watch the loss shrink toward zero — that shrinking is, in a very real sense, the model learning.

Machine Learning Edition Page 5

Developing AI from Zero

05 Beyond Straight Lines: Why We Need Networks

A straight line cannot capture curved or complex patterns. The fix is to stack many small linear pieces
together, with a twist added between them called an activation function — this combination is a neural
network.

The building blocks

Piece Role

Neuron One weighted sum plus a bias, like Chapter 3's line

Layer A group of neurons processed together

Activation A non-linear twist (e.g. ReLU) so layers can't collapse into one line

Weights The learnable numbers, adjusted during training

ReLU is simple on purpose: it returns the input unchanged if positive, and zero otherwise (max(0, x)). That
tiny non-linearity is enough to let a network approximate almost any pattern.

Machine Learning Edition Page 6

Developing AI from Zero

06 Building a Neural Network from Scratch

Here is a minimal two-layer network for a simple task: predicting whether a point (x, y) is inside or outside a
small region. No libraries beyond numpy.

import numpy as np

[Link](0)

# 2 inputs -> 4 hidden neurons -> 1 output

W1 = [Link](2, 4) * 0.5
b1 = [Link](4)
W2 = [Link](4, 1) * 0.5
b2 = [Link](1)

def relu(z):
return [Link](0, z)

def sigmoid(z):
return 1 / (1 + [Link](-z))

def forward(x):
h = relu(x @ W1 + b1) # hidden layer
out = sigmoid(h @ W2 + b2) # output: 0 to 1
return h, out

forward() is the "guess" step from Chapter 1's loop — data flows through the layers, each one transforming
it, until a final prediction comes out.

Machine Learning Edition Page 7

Developing AI from Zero

07 Training the Network

Training repeats the same loop as the straight-line model: predict, measure error, adjust every weight a little,
repeat. With more layers, the adjustment step is called backpropagation — the chain rule applied layer by
layer.

X = [Link]([[0.2,0.2],[0.8,0.8],[0.1,0.9],[0.9,0.1]])
y = [Link]([[1],[1],[0],[0]]) # 1 = inside region, 0 = outside

lr = 0.1
for epoch in range(2000):
h, out = forward(X)
error = out - y

# backpropagation: push the error backward through the network

d_out = error * out * (1 - out)
d_W2 = h.T @ d_out
d_b2 = d_out.sum(axis=0)

d_h = (d_out @ W2.T) * (h > 0)

d_W1 = X.T @ d_h
d_b1 = d_h.sum(axis=0)

W2 -= lr * d_W2; b2 -= lr * d_b2
W1 -= lr * d_W1; b1 -= lr * d_b1

if epoch % 500 == 0:
loss = [Link](error ** 2)
print(f"epoch {epoch}, loss={loss:.4f}")

Machine Learning Edition Page 8

Developing AI from Zero

08 Overfitting and Generalization

A model that memorizes its training examples perfectly but fails on new, unseen ones has overfit. The point
of training was never to memorize — it was to learn a pattern general enough to work on data the model has
never seen.

Practical signs and fixes

• Split your data — train on most of it, test on a held-out portion the model never sees during training.

• Watch both losses — if training loss keeps falling but test loss rises, that is overfitting.

• Get more data or simplify the model — both reduce the model's ability to just memorize.

• Stop training earlier — often the simplest fix once test loss starts climbing.

A model that never sees the same overfitting pitfall is not "better," it is usually just undertrained — the goal is the point
where test performance peaks, not where training loss hits zero.

Machine Learning Edition Page 9

Developing AI from Zero

09 Mini Project: Classify a Handwritten-Style Digit

This ties every idea together: real-shaped input, a network with a hidden layer, and a prediction turned into a
decision. Here, a tiny 3x3 pixel grid stands in for a miniature image, kept small so it runs instantly with no
dataset download required.

import numpy as np
[Link](1)

# 3x3 'images' flattened to 9 values: two tiny patterns

plus = [Link]([0,1,0, 1,1,1, 0,1,0], dtype=float) # a plus shape
cross = [Link]([1,0,1, 0,1,0, 1,0,1], dtype=float) # an X shape

X = [Link]([plus, cross])
y = [Link]([[1,0],[0,1]]) # one-hot: [is_plus, is_cross]

W1 = [Link](9, 6) * 0.3
W2 = [Link](6, 2) * 0.3

def relu(z): return [Link](0, z)

def softmax(z):
e = [Link](z - [Link](axis=1, keepdims=True))
return e / [Link](axis=1, keepdims=True)

for epoch in range(3000):

h = relu(X @ W1)
out = softmax(h @ W2)
error = out - y

d_W2 = h.T @ error

d_h = (error @ W2.T) * (h > 0)
d_W1 = X.T @ d_h

W1 -= 0.1 * d_W1
W2 -= 0.1 * d_W2

print("Prediction for 'plus':", softmax(relu(plus @ W1).reshape(1,-1) @ W2))

The printed values are probabilities. After training, the first number should be near 1.0 — the network
correctly recognizing the plus shape.

Machine Learning Edition Page 10

Developing AI from Zero

10 Where to Go From Here

You have now built, by hand, the same core mechanism — layers, weights, loss, gradients, backpropagation
— that every larger AI model uses, just at a much bigger scale, with more layers and far more data.

Suggested next topics

• PyTorch or TensorFlow — libraries that automate the gradient math you just wrote by hand.

• Real datasets (e.g. MNIST digits) — apply this same network shape to real, larger data.

• Convolutional networks — a layer type specialized for images.

• Transformers — the architecture behind modern language models, built from these same core ideas.

The fastest way to solidify this: rebuild Chapter 9's network for a slightly bigger pattern set by hand, before
reaching for a library that hides the math.

Machine Learning Edition Page 11