[Link].

ng

Raising a new Generation of Leaders

GET312: MATHEMATICAL
FOUNDATIONS FOR AI

BY

Omoruyi O.
Module 7
OUTLINE
This Module will detail the following topics :
• Linear Algebra
- Define linear algebra and its importance in AI
- Introduce vectors and matrices, and their basic operations
• Vector Operations
- Explain vector addition, scalar multiplication, and dot product
- Provide examples and practice problems
• Matrix Operations
- Introduce matrix addition, multiplication, and inverse
- Explain the concept of determinants and eigenvalues
• Eigenvalues and their Application
- Introduce the concept of eigenvalues and eigenvectors
- Explain their application in AI, such as in principal component analysis (PCA)

2
 Outline II
• Introduction to Probability and Statistics
- Define probability and statistics, and their importance in AI
- Introduce Bayes' theorem and its significance
• Bayes' Theorem
- Explain the concept of Bayes' theorem and its application
- Provide examples and practice problems
• Distributions
- Introduce different types of distributions, such as Gaussian, Bernoulli, and
Poisson
- Explain their importance in AI and provide examples
• Statistical Inference
- Introduce the concept of statistical inference and its application in AI
- Explain the importance of hypothesis testing and confidence intervals
3
Outline III
• Introduction to Calculus
- Define calculus and its importance in AI
- Introduce the concept of gradients and optimization
• Gradients
- Explain the concept of gradients and their application in optimization
- Provide examples and practice problems
• Optimization Basics
- Introduce the concept of optimization and its application in AI
- Explain the importance of convex optimization and local minima
• Applications of Calculus in AI
- Explain the application of calculus in AI, such as in deep learning and neural
networks
- Provide examples and case studies
4
 Linear Algebra
Linear algebra is the study of linear maps on finite-
dimensional vector spaces.
Vector Space is a concept central to Linear Algebra. From

of the set 𝐑 of real numbers.

elementary algebra you already know the basic properties

The real numbers, denoted R, encompass all rational and

irrational numbers, forming a complete ordered field.
In elementary algebra, these numbers are studied for their
behavior under basic operations, particularly addition and
multiplication.
5
 Applications
• Systems of linear equations
• Geometric transformations
• Data analysis and statistics
• Computer graphics
• Machine learning algorithms
• Physics simulations

6
 Properties of Real Numbers
Closure Property
• The closure property means that when you add or multiply any two real
numbers, the result is always another real number. For example, 3 + 5 = 8
(both 3, 5, and 8 are real numbers), and 4 × 6 = 24 (all are real numbers).
Commutative Property
• This property says the order doesn't matter when adding or multiplying. For
addition, 2 + 3 is the same as 3 + 2, both equal 5. For multiplication, 2 × 3 is
the same as 3 × 2, both equal 6.
Associative Property
• The associative property means grouping doesn't affect the result when
adding or multiplying multiple numbers. For addition, (1 + 2) + 3 = 6 is the
same as 1 + (2 + 3) = 6. For multiplication, (2 × 3) × 4 = 24 is the same as 2
× (3 × 4) = 24.
7
 Properties of Real Number
• Distributive Property
This property links addition and multiplication, stating that multiplying a
number by a sum is the same as multiplying by each part and adding. For
example, 2 × (3 + 4) = 2 × 3 + 2 × 4, which is 14 in both cases.
• Identity Property
The identity property involves special numbers: 0 for addition (adding 0
doesn't change the number, like 5 + 0 = 5) and 1 for multiplication
(multiplying by 1 doesn't change the number, like 5 × 1 = 5).
• Inverse Property
The inverse property means every real number has an opposite for addition
(e.g., 5 and -5 add to 0, as 5 + (-5) = 0) and, for non-zero numbers, a
reciprocal for multiplication (e.g., 2 and 1/2 multiply to 1, as 2 × 1/2 = 1).

8
Set Notations

9
 Vector Space
A vector space (V) over a field F (typically R or C) is defined as a set equipped with two

vector space are called vectors or points. A vector space is also defined as a set 𝑉 along
operations: addition and scalar multiplication, satisfying eight axioms. Elements of a

with an addition on 𝑉 and a scalar multiplication on 𝑉 such that the following properties
hold:

• 𝑢 + 𝑣 = 𝑣 + 𝑢 for all 𝑢, 𝑣 ∈ 𝑉.
commutativity

• (𝑢 + 𝑣) + 𝑤 = 𝑢 + (𝑣 + 𝑤) and (𝑎𝑏)𝑣 = 𝑎(𝑏𝑣) for all 𝑢, 𝑣, 𝑤 ∈ 𝑉 and for all 𝑎, 𝑏 ∈ 𝐅.

associativity

• There exists an element 0 ∈ 𝑉 such that 𝑣 + 0 = 𝑣 for all 𝑣 ∈ 𝑉.

additive identity

• For every 𝑣 ∈ 𝑉, there exists 𝑤 ∈ 𝑉 such that 𝑣 + 𝑤 = 0.

additive inverse

10
Vector Space properties

• 1𝑣 = 𝑣 for all 𝑣 ∈ 𝑉.
multiplicative identity

• 𝑎(𝑢 + 𝑣) = 𝑎𝑢 + 𝑎𝑣 and (𝑎 + 𝑏)𝑣 = 𝑎𝑣 +

distributive properties

𝑏𝑣 for all 𝑎, 𝑏 ∈ 𝐅 and all 𝑢, 𝑣 ∈ 𝑉.

associativity of scalar multiplication
• (ab)v = a(bv) for all a, b ∈ F and v ∈ V ,
11
 Eight Axioms of a Vector
Space
A vector space V over a field F must satisfy the following axioms for its addition (where
u, v, w ∈ V) and scalar multiplication (where a, b ∈ F, and v ∈ V):
Commutativity of Addition: For all u, v ∈ V, u + v = v + u. This means the order of
addition doesn't matter.
• For example, in R2, (1, 2) + (3, 4) = (3, 4) + (1, 2) = (4, 6).
Associativity of Addition: For all u, v, w ∈ V, (u + v) + w = u + (v + w). This ensures
that grouping doesn't affect addition.
• In R2, ((1, 2) + (3, 4)) + (5, 6) = (1, 2) + ((3, 4) + (5, 6)) = (9, 12).
Additive Identity: There exists an element 0 ∈ V, called the zero vector, such that for
all v ∈ V, v + 0 = v. This guarantees a "neutral" element for addition.
• In R2, the zero vector is (0, 0), so (1, 2) + (0, 0) = (1, 2).
Additive Inverses: For each v ∈ V, there exists an element -v ∈ V such that v + (-v) =
0. Every vector has an "opposite" that sums to the zero vector.
• In R2, for (1, 2), the inverse is (-1, -2), since (1, 2) + (-1, -2) = (0, 0).
12
 Eight Axioms of a Vector
Space Cont’d
Distributivity of Scalar Multiplication over Vector Addition: For all a ∈ F and u, v ∈
V, a(u + v) = au + av. This means you can distribute a scalar across a sum of vectors. I
• In R2, if a = 2, then 2((1, 2) + (3, 4)) = 2(4, 6) = (8, 12), and 2(1, 2) + 2(3, 4) = (2, 4) +
(6, 8) = (8, 12).
Distributivity of Scalar Addition over Scalar Multiplication: For all a, b ∈ F and v ∈ V,
(a + b)v = av + bv. This allows you to distribute a sum of scalars over a vector.
• In R2, if a = 2, b = 3, then (2 + 3)(1, 2) = 5(1, 2) = (5, 10), and 2(1, 2) + 3(1, 2) = (2, 4)
+ (3, 6) = (5, 10).
Compatibility of Scalar Multiplication: For all a, b ∈ F and v ∈ V, a(bv) = (ab)v. This
ensures that scalar multiplication is consistent when applying multiple scalars.
• In R2, if a = 2, b = 3, then 2(3(1, 2)) = 2(3, 6) = (6, 12), and (2 · 3)(1, 2) = 6(1, 2) = (6,
12).
Multiplicative Identity for Scalar Multiplication: The scalar 1 ∈ F acts as the identity,
so for all v ∈ V, 1v = v. Multiplying by 1 leaves the vector unchanged.
• In R2, 1(1, 2) = (1, 2).
13
 Real and Complex numbers
as Fields
A complex number is a pair (a, b) where a, b ∈ R. We usually
write a + bi instead of (a, b). The set of all complex numbers is

A vector space over 𝐑 is called a real vector space. A vector

denoted by C: C = {a + bi | a, b ∈ R}.

space over 𝐂 is called a complex vector space. The letter 𝐅 is

used because 𝐑 and 𝐂 are examples of what are called fields.
Also, The set C is equipped with operations addition and
multiplication:
• (a + bi) + (c + di) = (a + c) + (b + d)i
• (a + bi)(c + di) = (ac − bd) + (ad + bc)i.
14
 Properties of Complex
Arithmetic
• 𝛼 + 𝛽 = 𝛽 + 𝛼 and 𝛼𝛽 = 𝛽𝛼 for all 𝛼, 𝛽 ∈ 𝐂.
commutativity

• (𝛼 + 𝛽) + 𝜆 = 𝛼 + (𝛽 + 𝜆) and (𝛼𝛽) 𝜆 = 𝛼(𝛽𝜆) for all 𝛼, 𝛽, 𝜆 ∈ 𝐂.

associativity

• 𝜆 + 0 = 𝜆 and 𝜆1 = 𝜆 for all 𝜆 ∈ 𝐂.

identities

• For every 𝛼 ∈ 𝐂, there exists a unique 𝛽 ∈ 𝐂 such that 𝛼 + 𝛽 = 0.

additive inverse

• For every 𝛼 ∈ 𝐂 with 𝛼 ≠ 0, there exists a unique 𝛽 ∈ 𝐂 such that 𝛼𝛽 = 1.

multiplicative inverse

• 𝜆(𝛼 + 𝛽) = 𝜆𝛼 + 𝜆𝛽 for all 𝜆, 𝛼, 𝛽 ∈ 𝐂.

distributive property

15
 Fields
Elements of 𝐅 are called scalars. Remember that F
can either be R or C. The set 𝐑2, which you can think
of as a plane, is the set of all ordered pairs of real

• 𝐑2 = {(𝑥, 𝑦) ∶ 𝑥, 𝑦 ∈ 𝐑}.
numbers:

The set 𝐑3, which you can think of as ordinary space,

• 𝐑3 = {(𝑥, 𝑦, 𝑧) ∶ 𝑥, 𝑦, 𝑧 ∈ 𝐑}.
is the set of all ordered triples of real numbers:

16
Lists and Sets
𝐅𝑛 is the set of all lists of length 𝑛 of
elements of 𝐅:
• 𝐅𝑛 = {(𝑥1, …, 𝑥𝑛) ∶ 𝑥𝑘 ∈ 𝐅 for 𝑘 = 1, …, 𝑛}.
For (𝑥1, … , 𝑥𝑛) ∈ 𝐅𝑛 and 𝑘 ∈ {1, … , 𝑛}, we
say that 𝑥𝑘 is the 𝑘th coordinate of (𝑥1, … ,
𝑥𝑛).

17
 Vectors
• A vector is an ordered collection of
numbers or mathematical objects.
• In ℝn, a vector is represented as
an ordered n-tuple: v = (v₁, v₂, ...,
vₙ)
• Vectors can be visualized as
arrows in space with magnitude
(length) and direction.
• We typically denote vectors using
bold lowercase letters (v, u, w) or
with arrows (→)
18
Vector Addition
Given vectors u = (u₁,
u₂, ..., uₙ) and v = (v₁,
v₂, ..., vₙ)
• u + v = (u₁ + v₁, u₂ +
v₂, ..., uₙ + vₙ)
Geometrically: place
vectors head-to-tail and
connect them to form a
resultant vector
19
Scalar Multiplication
Given a scalar c and
vector v = (v₁, v₂, ...,
vₙ)
• cv = (cv₁, cv₂, ..., cvₙ)
Geometrically:
stretching or shrinking
a vector (and reversing
direction if c < 0)
20
 Dot Product
Given vectors u = (u₁, u₂, ..., uₙ) and v = (v₁,
v₂, ..., vₙ)
• u · v = u₁v₁ + u₂v₂ + ... + uₙvₙ
Properties:
 Commutative: u · v = v · u
 Distributive: u · (v + w) = u · v + u · w
 If u · v = 0, the vectors are orthogonal
(perpendicular)
21
 Vector Norm (Magnitude)
For vector v = (v₁, v₂, ...,
vₙ)
• ||v|| = √(v₁² + v₂² + ...
+ vₙ²)
Represents the length of
the vector

22
 Matrices
• A matrix is a rectangular
array of numbers arranged A = [a₁₁ a₁₂ ... a₁ₙ]
[a₂₁ a₂₂ ... a₂ₙ]
in rows and columns [ ⋮ ⋮ ... ⋮ ]
[aₘ₁ aₘ₂ ... aₘₙ]
• An m×n matrix has m rows
and n columns
• Matrices are typically
denoted by uppercase
letters (A, B, C)

23
 Matrix Addition
For matrices A and B of
the same dimensions
(m×n)
• C = A + B, where cᵢⱼ = aᵢⱼ
+ bᵢⱼ
Addition is performed
element-wise
24
Scalar Multiplication
For a scalar c and
matrix A. cA is a
matrix where each
element is multiplied
by c:
• (cA)ᵢⱼ = c·aᵢⱼ

25
 Matrix Multiplication
For matrices A (m×p) and B (p×n)
• C = AB is an m×n matrix
• cᵢⱼ = aᵢ₁·b₁ⱼ + aᵢ₂·b₂ⱼ + ... + aᵢₚ·bₚⱼ = Σₖ aᵢₖ·bₖⱼ
Properties:
Not commutative: generally, AB ≠ BA
Associative: (AB)C = A(BC)
Distributive: A(B + C) = AB + AC
26
Matrix Transpose
For matrix A, the transpose AT is obtained by
flipping entries across the main diagonal:
• (AT)ᵢⱼ = aⱼᵢ
Properties:
(AT)T = A
(A + B)T = AT + BT
(AB)T = BT·AT
27
 Special Cases
Identity Matrix (I): Has 1s on the main diagonal and
0s else
• Where AI = IA = A, for any matrix A of appropriate
size
Zero Matrix (0): All elements are zero
Diagonal Matrix: All off-diagonal elements are zero
Symmetric Matrix: A = AT
Invertible Matrix: Has an inverse A⁻¹ such that AA⁻¹
= A⁻¹A = I
28
Comparing Vectors to
Matrices
•A vector can be
represented as an n×1
matrix (column vector) Or
as a 1×n matrix (row
vector)
• Vector operations are
special cases of matrix
operations
29
 Determinant of a Matrix
The determinant of a square matrix A is a scalar value that provides information about the matrix's
properties.
• det(A) or |A| for a 2×2 matrix: ad – bc, for larger matrices, it’s computed using cofactor
expansion or other methods.
Properties
• det(AB) = det(A) · det(B)
• det(AT) = det(A)
• If A is invertible, det(A) ≠ 0
• If det(A) = 0, then A is singular (non-invertible)
• det(λA) = λn · det(A); for an n×n matrix

In AI, determinants are used for testing for invertibility of matrices, used in calculating matrix
inverses, measures volume transformation in linear mappings and component in many machine
learning algorithms.
30
 Eigenvalues and
Eigenvectors
• For a square matrix A, a non-zero vector v is an
eigenvector, if Av = λv for some scalar λ.
• The scalar λ is called the eigenvalue corresponding to v.
• In other words, applying A to v only changes its scale,
not its direction.
• For eigenvalues: Solve the characteristic equation det(A
- λI) = 0
• For each eigenvalue λ, find eigenvectors by solving (A -
λI)v = 0
31
 Properties
• An n×n matrix has at most n distinct
eigenvalues
• Eigenvalues of a triangular matrix are its
diagonal entries
• Eigenvectors corresponding to distinct
eigenvalues are linearly independent
• Eigenvalues of AT are the same as eigenvalues
of A
32
Eigen Decomposition
If A has n linearly independent eigenvectors,
it can be decomposed as A = PDP(-1)
• P is the matrix of eigenvectors
• D is a diagonal matrix of eigenvalues
This decomposition allows us to easily
compute powers of A:
• Ak = PDkP(-1)
33
Principal Component Analysis
PCA is a dimensionality reduction technique widely used in AI and data
science. Transforms data into a new coordinate system where the
greatest variance lies on the first coordinate (principal component).

• Start with data matrix X (each row is a data point, each column a
feature)
• Compute the covariance matrix Σ = XT·X (after centering)
• Find eigenvalues and eigenvectors of Σ
• Eigenvectors with largest eigenvalues are the principal components
• Project data onto these eigenvectors to reduce dimensionality

34
 PCA cont’d
• Eigenvalues represent the amount of variance explained by
each principal component
• Larger eigenvalues indicate more important components
• The ratio of an eigenvalue to the sum of all eigenvalues gives
the proportion of variance explained
PCA applications include: Image recognition: Reducing
dimensionality of image data (e.g., eigenfaces); Data
visualization: Projecting high-dimensional data to 2D or 3D; Noise
reduction: Discarding components with small eigenvalues;
Feature extraction: Creating uncorrelated features from
correlated ones;
35
 Conclusion
• Determinants provide critical information about matrix
properties and transformations
• Eigenvalues and eigenvectors reveal intrinsic properties of
linear transformations
• PCA uses eigendecomposition to find directions of maximum
variance in data
• These concepts underpin many fundamental AI and machine
learning algorithms
• Understanding these mathematical tools enables more
effective AI model design and interpretation.
36
References
• Grok – [Link]/i/Grok

37