Module 3: Mathematics for AI
Linear Algebra | Probability |
Calculus | Optimization | Lab
Introduction
– Mathematics forms the foundation of Artificial
Intelligence.
– In this module, we cover Linear Algebra,
Probability, Calculus, and Optimization.
– We also implement Gradient Descent in Python.
Vectors
– Definition: An ordered list of numbers.
– Used to represent data points or directions in
space.
– Example: v = [1, 2, 3].
Vector Operations
– Addition and Subtraction.
– Scalar Multiplication.
– Dot Product (measures similarity).
Dot Product Example
– v1 = [1, 2, 3], v2 = [4, 5, 6].
– Dot = 1*4 + 2*5 + 3*6 = 32.
– Python: [Link](v1, v2) → 32.
Matrices
– Definition: 2D array of numbers.
– Matrix multiplication represents transformations.
– Example: rotation, scaling in ML.
Eigenvalues & Eigenvectors
– Definition: Av = λv.
– Show direction (eigenvector) and scale
(eigenvalue).
– Important for PCA (dimensionality reduction).
Probability Basics
– Probability quantifies uncertainty.
– P(E) = Favorable outcomes / Total outcomes.
– Used in Bayesian models, generative AI.
Statistics Basics
– Mean: Average of values.
– Variance: Measure of spread.
– Standard Deviation: Square root of variance.
Distributions
– Normal Distribution (Gaussian).
– Binomial Distribution.
– Bernoulli Distribution.
Statistics Example
– Data = [2,4,4,4,5,5,7,9].
– Mean = 5.0, Variance = 4.0.
– Python: [Link](data), [Link](data).
Calculus – Derivatives
– Derivative = slope of a function.
– In AI, used in optimization (backpropagation).
– Example: d/dx (x^2 + 3x) = 2x + 3.
Gradients
– Generalization of derivatives to multiple variables.
– Vector of partial derivatives.
– Used in ML to minimize loss functions.
Chain Rule
– Used to differentiate composite functions.
– Essential in neural network backpropagation.
– Example: d(f(g(x)))/dx = f'(g(x))*g'(x).
Optimization
– Goal: minimize loss or maximize accuracy.
– Methods: Gradient Descent, Stochastic Gradient
Descent.
– Learning rate controls step size.
Gradient Descent Concept
– Iteratively update parameters to minimize loss.
– θ_new = θ_old - η * gradient.
– η = learning rate.
Convex vs Non-Convex Functions
– Convex: One global minimum.
– Non-convex: Many local minima.
– Neural networks often involve non-convex
optimization.
Lab: Gradient Descent
– We implement gradient descent for linear
regression.
– y = mx + b.
– Goal: learn m and b from data.
Gradient Descent Python Example
– X = [1,2,3,4,5], y = [2,4,6,8,10].
– Initialize m, b = 0.
– Update iteratively using gradients.
Output of Gradient Descent
– After training: m ≈ 2, b ≈ 0.
– This matches the equation y = 2x.
– Demonstrates convergence.
Summary
– We studied Linear Algebra, Probability, Calculus,
Optimization.
– Implemented Gradient Descent in Python.
– These concepts are essential in AI and ML.