[Link].

ng

Raising a new Generation of Leaders

GET312: MATHEMATICAL
FOUNDATIONS FOR AI

BY
Omoruyi O.
Module 7
OUTLINE
This Module will detail the following topics :
• Linear Algebra
- Define linear algebra and its importance in AI
- Introduce vectors and matrices, and their basic operations
• Vector Operations
- Explain vector addition, scalar multiplication, and dot product
- Provide examples and practice problems
• Matrix Operations
- Introduce matrix addition, multiplication, and inverse
- Explain the concept of determinants and eigenvalues
• Eigenvalues and their Application
- Introduce the concept of eigenvalues and eigenvectors
- Explain their application in AI, such as in principal component analysis (PCA)

2
 Outline II
• Introduction to Probability and Statistics
- Define probability and statistics, and their importance in AI
- Introduce Bayes' theorem and its significance
• Bayes' Theorem
- Explain the concept of Bayes' theorem and its application
- Provide examples and practice problems
• Distributions
- Introduce different types of distributions, such as Gaussian, Bernoulli, and Poisson
- Explain their importance in AI and provide examples
• Statistical Inference
- Introduce the concept of statistical inference and its application in AI
- Explain the importance of hypothesis testing and confidence intervals

3
Outline III
• Introduction to Calculus
- Define calculus and its importance in AI
- Introduce the concept of gradients and optimization
• Gradients
- Explain the concept of gradients and their application in optimization
- Provide examples and practice problems
• Optimization Basics
- Introduce the concept of optimization and its application in AI
- Explain the importance of convex optimization and local minima
• Applications of Calculus in AI
- Explain the application of calculus in AI, such as in deep learning and neural networks
- Provide examples and case studies

4
 Linear Algebra
Linear algebra is the study of linear maps on finite-
dimensional vector spaces.
Vector Space is a concept central to Linear Algebra. From
elementary algebra you already know the basic
properties of the set 𝐑 of real numbers.
The real numbers, denoted R, encompass all rational and
irrational numbers, forming a complete ordered field.
In elementary algebra, these numbers are studied for
their behavior under basic operations, particularly
addition and multiplication.
5
 Applications
• Systems of linear equations
• Geometric transformations
• Data analysis and statistics
• Computer graphics
• Machine learning algorithms
• Physics simulations

6
 Properties of Real Numbers
Closure Property
• The closure property means that when you add or multiply any two real
numbers, the result is always another real number. For example, 3 + 5 = 8
(both 3, 5, and 8 are real numbers), and 4 × 6 = 24 (all are real numbers).
Commutative Property
• This property says the order doesn't matter when adding or multiplying. For
addition, 2 + 3 is the same as 3 + 2, both equal 5. For multiplication, 2 × 3 is
the same as 3 × 2, both equal 6.
Associative Property
• The associative property means grouping doesn't affect the result when
adding or multiplying multiple numbers. For addition, (1 + 2) + 3 = 6 is the
same as 1 + (2 + 3) = 6. For multiplication, (2 × 3) × 4 = 24 is the same as 2 ×
(3 × 4) = 24.

7
 Properties of Real Number
• Distributive Property
This property links addition and multiplication, stating that multiplying a
number by a sum is the same as multiplying by each part and adding. For
example, 2 × (3 + 4) = 2 × 3 + 2 × 4, which is 14 in both cases.
• Identity Property
The identity property involves special numbers: 0 for addition (adding 0
doesn't change the number, like 5 + 0 = 5) and 1 for multiplication (multiplying
by 1 doesn't change the number, like 5 × 1 = 5).
• Inverse Property
The inverse property means every real number has an opposite for addition
(e.g., 5 and -5 add to 0, as 5 + (-5) = 0) and, for non-zero numbers, a reciprocal
for multiplication (e.g., 2 and 1/2 multiply to 1, as 2 × 1/2 = 1).

8
Set Notations

9
 Vector Space
A vector space (V) over a field F (typically R or C) is defined as a set equipped with two
operations: addition and scalar multiplication, satisfying eight axioms. Elements of a
vector space are called vectors or points. A vector space is also defined as a set 𝑉 along
with an addition on 𝑉 and a scalar multiplication on 𝑉 such that the following properties
hold:
commutativity
• 𝑢 + 𝑣 = 𝑣 + 𝑢 for all 𝑢, 𝑣 ∈ 𝑉.
associativity
• (𝑢 + 𝑣) + 𝑤 = 𝑢 + (𝑣 + 𝑤) and (𝑎𝑏)𝑣 = 𝑎(𝑏𝑣) for all 𝑢, 𝑣, 𝑤 ∈ 𝑉 and for all 𝑎, 𝑏 ∈ 𝐅.
additive identity
• There exists an element 0 ∈ 𝑉 such that 𝑣 + 0 = 𝑣 for all 𝑣 ∈ 𝑉.
additive inverse
• For every 𝑣 ∈ 𝑉, there exists 𝑤 ∈ 𝑉 such that 𝑣 + 𝑤 = 0.

10
 Vector Space properties
multiplicative identity
• 1𝑣 = 𝑣 for all 𝑣 ∈ 𝑉.
distributive properties
• 𝑎(𝑢 + 𝑣) = 𝑎𝑢 + 𝑎𝑣 and (𝑎 + 𝑏)𝑣 = 𝑎𝑣 + 𝑏𝑣 for all
𝑎, 𝑏 ∈ 𝐅 and all 𝑢, 𝑣 ∈ 𝑉.
associativity of scalar multiplication
• (ab)v = a(bv) for all a, b ∈ F and v ∈ V ,
11
 Eight Axioms of a Vector Space
A vector space V over a field F must satisfy the following axioms for its addition (where u, v, w ∈ V) and
scalar multiplication (where a, b ∈ F, and v ∈ V):
Commutativity of Addition: For all u, v ∈ V, u + v = v + u. This means the order of addition doesn't
matter.
• For example, in R2, (1, 2) + (3, 4) = (3, 4) + (1, 2) = (4, 6).
Associativity of Addition: For all u, v, w ∈ V, (u + v) + w = u + (v + w). This ensures that grouping doesn't
affect addition.
• In R2, ((1, 2) + (3, 4)) + (5, 6) = (1, 2) + ((3, 4) + (5, 6)) = (9, 12).
Additive Identity: There exists an element 0 ∈ V, called the zero vector, such that for all v ∈ V, v + 0 = v.
This guarantees a "neutral" element for addition.
• In R2, the zero vector is (0, 0), so (1, 2) + (0, 0) = (1, 2).
Additive Inverses: For each v ∈ V, there exists an element -v ∈ V such that v + (-v) = 0. Every vector has
an "opposite" that sums to the zero vector.
• In R2, for (1, 2), the inverse is (-1, -2), since (1, 2) + (-1, -2) = (0, 0).

12
 Eight Axioms of a Vector Space Cont’d
Distributivity of Scalar Multiplication over Vector Addition: For all a ∈ F and u, v ∈ V,
a(u + v) = au + av. This means you can distribute a scalar across a sum of vectors. I
• In R2, if a = 2, then 2((1, 2) + (3, 4)) = 2(4, 6) = (8, 12), and 2(1, 2) + 2(3, 4) = (2, 4) + (6, 8)
= (8, 12).
Distributivity of Scalar Addition over Scalar Multiplication: For all a, b ∈ F and v ∈ V, (a
+ b)v = av + bv. This allows you to distribute a sum of scalars over a vector.
• In R2, if a = 2, b = 3, then (2 + 3)(1, 2) = 5(1, 2) = (5, 10), and 2(1, 2) + 3(1, 2) = (2, 4) + (3,
6) = (5, 10).
Compatibility of Scalar Multiplication: For all a, b ∈ F and v ∈ V, a(bv) = (ab)v. This
ensures that scalar multiplication is consistent when applying multiple scalars.
• In R2, if a = 2, b = 3, then 2(3(1, 2)) = 2(3, 6) = (6, 12), and (2 · 3)(1, 2) = 6(1, 2) = (6, 12).
Multiplicative Identity for Scalar Multiplication: The scalar 1 ∈ F acts as the identity, so
for all v ∈ V, 1v = v. Multiplying by 1 leaves the vector unchanged.
• In R2, 1(1, 2) = (1, 2).

13
 Real and Complex numbers as Fields
A complex number is a pair (a, b) where a, b ∈ R. We usually
write a + bi instead of (a, b). The set of all complex numbers is
denoted by C: C = {a + bi | a, b ∈ R}.
A vector space over 𝐑 is called a real vector space. A vector
space over 𝐂 is called a complex vector space. The letter 𝐅 is
used because 𝐑 and 𝐂 are examples of what are called fields.
Also, The set C is equipped with operations addition and
multiplication:
• (a + bi) + (c + di) = (a + c) + (b + d)i
• (a + bi)(c + di) = (ac − bd) + (ad + bc)i.

14
 Properties of Complex Arithmetic
commutativity
• 𝛼 + 𝛽 = 𝛽 + 𝛼 and 𝛼𝛽 = 𝛽𝛼 for all 𝛼, 𝛽 ∈ 𝐂.
associativity
• (𝛼 + 𝛽) + 𝜆 = 𝛼 + (𝛽 + 𝜆) and (𝛼𝛽) 𝜆 = 𝛼(𝛽𝜆) for all 𝛼, 𝛽, 𝜆 ∈ 𝐂.
identities
• 𝜆 + 0 = 𝜆 and 𝜆1 = 𝜆 for all 𝜆 ∈ 𝐂.
additive inverse
• For every 𝛼 ∈ 𝐂, there exists a unique 𝛽 ∈ 𝐂 such that 𝛼 + 𝛽 = 0.
multiplicative inverse
• For every 𝛼 ∈ 𝐂 with 𝛼 ≠ 0, there exists a unique 𝛽 ∈ 𝐂 such that 𝛼𝛽 = 1.
distributive property
• 𝜆(𝛼 + 𝛽) = 𝜆𝛼 + 𝜆𝛽 for all 𝜆, 𝛼, 𝛽 ∈ 𝐂.

15
 Fields
Elements of 𝐅 are called scalars. Remember that F
can either be R or C. The set 𝐑2, which you can think
of as a plane, is the set of all ordered pairs of real
numbers:
• 𝐑2 = {(𝑥, 𝑦) ∶ 𝑥, 𝑦 ∈ 𝐑}.
The set 𝐑3, which you can think of as ordinary space,
is the set of all ordered triples of real numbers:
• 𝐑3 = {(𝑥, 𝑦, 𝑧) ∶ 𝑥, 𝑦, 𝑧 ∈ 𝐑}.

16
 Lists and Sets
𝐅𝑛 is the set of all lists of length 𝑛 of elements of 𝐅:
• 𝐅𝑛 = {(𝑥1, …, 𝑥𝑛) ∶ 𝑥𝑘 ∈ 𝐅 for 𝑘 = 1, …, 𝑛}.
For (𝑥1, … , 𝑥𝑛) ∈ 𝐅𝑛 and 𝑘 ∈ {1, … , 𝑛}, we say that
𝑥𝑘 is the 𝑘th coordinate of (𝑥1, … , 𝑥𝑛).

17
 Vectors
• A vector is an ordered collection of
numbers or mathematical objects.
• In ℝn, a vector is represented as an
ordered n-tuple: v = (v₁, v₂, ..., vₙ)
• Vectors can be visualized as arrows
in space with magnitude (length) and
direction.
• We typically denote vectors using
bold lowercase letters (v, u, w) or
with arrows (→)

18
Vector Addition
Given vectors u = (u₁, u₂, ...,
uₙ) and v = (v₁, v₂, ..., vₙ)
• u + v = (u₁ + v₁, u₂ + v₂, ...,
uₙ + vₙ)
Geometrically: place
vectors head-to-tail and
connect them to form a
resultant vector

19
Scalar Multiplication
Given a scalar c and
vector v = (v₁, v₂, ..., vₙ)
• cv = (cv₁, cv₂, ..., cvₙ)
Geometrically: stretching
or shrinking a vector
(and reversing direction
if c < 0)
20
 Dot Product
Given vectors u = (u₁, u₂, ..., uₙ) and v = (v₁, v₂, ..., vₙ)
• u · v = u₁v₁ + u₂v₂ + ... + uₙvₙ
Properties:
 Commutative: u · v = v · u
 Distributive: u · (v + w) = u · v + u · w
 If u · v = 0, the vectors are orthogonal (perpendicular)

21
 Vector Norm (Magnitude)
For vector v = (v₁, v₂, ..., vₙ)
• ||v|| = √(v₁² + v₂² + ... +
vₙ²)
Represents the length of
the vector

22
 Matrices
• A matrix is a rectangular
array of numbers arranged A = [a₁₁ a₁₂ ... a₁ₙ]

in rows and columns [a₂₁ a₂₂ ... a₂ₙ]

23
 Matrix Addition
For matrices A and B of the
same dimensions (m×n)
• C = A + B, where cᵢⱼ = aᵢⱼ +
bᵢⱼ
Addition is performed
element-wise

24
 Scalar Multiplication
For a scalar c and matrix
A. cA is a matrix where
each element is
multiplied by c:
• (cA)ᵢⱼ = c·aᵢⱼ

25
 Matrix Multiplication
For matrices A (m×p) and B (p×n)
• C = AB is an m×n matrix
• cᵢⱼ = aᵢ₁·b₁ⱼ + aᵢ₂·b₂ⱼ + ... + aᵢₚ·bₚⱼ = Σₖ aᵢₖ·bₖⱼ
Properties:
Not commutative: generally, AB ≠ BA
Associative: (AB)C = A(BC)
Distributive: A(B + C) = AB + AC
26
 Matrix Transpose
For matrix A, the transpose AT is obtained by
flipping entries across the main diagonal:
• (AT)ᵢⱼ = aⱼᵢ
Properties:
(AT)T = A
(A + B)T = AT + BT
(AB)T = BT·AT
27
 Special Cases
Identity Matrix (I): Has 1s on the main diagonal and
0s else
• Where AI = IA = A, for any matrix A of appropriate
size
Zero Matrix (0): All elements are zero
Diagonal Matrix: All off-diagonal elements are zero
Symmetric Matrix: A = AT
Invertible Matrix: Has an inverse A⁻¹ such that AA⁻¹
= A⁻¹A = I
28
 Comparing Vectors to Matrices
• A vector can be represented
as an n×1 matrix (column
vector) Or as a 1×n matrix
(row vector)
• Vector operations are special
cases of matrix operations

29
 Determinant of a Matrix
The determinant of a square matrix A is a scalar value that provides information about the matrix's properties.
• det(A) or |A| for a 2×2 matrix: ad – bc, for larger matrices, it’s computed using cofactor expansion or
other methods.
Properties
• det(AB) = det(A) · det(B)
• det(AT) = det(A)
• If A is invertible, det(A) ≠ 0
• If det(A) = 0, then A is singular (non-invertible)
• det(λA) = λn · det(A); for an n×n matrix

In AI, determinants are used for testing for invertibility of matrices, used in calculating matrix inverses,
measures volume transformation in linear mappings and component in many machine learning algorithms.

30
 Eigenvalues and Eigenvectors
• For a square matrix A, a non-zero vector v is an
eigenvector, if Av = λv for some scalar λ.
• The scalar λ is called the eigenvalue corresponding to v.
• In other words, applying A to v only changes its scale,
not its direction.
• For eigenvalues: Solve the characteristic equation det(A
- λI) = 0
• For each eigenvalue λ, find eigenvectors by solving (A -
λI)v = 0
31
 Properties
• An n×n matrix has at most n distinct eigenvalues
• Eigenvalues of a triangular matrix are its diagonal
entries
• Eigenvectors corresponding to distinct eigenvalues
are linearly independent
• Eigenvalues of AT are the same as eigenvalues of A

32
 Eigen Decomposition
If A has n linearly independent eigenvectors, it
can be decomposed as A = PDP(-1)
• P is the matrix of eigenvectors
• D is a diagonal matrix of eigenvalues
This decomposition allows us to easily compute
powers of A:
• Ak = PDkP(-1)

33
Principal Component Analysis
PCA is a dimensionality reduction technique widely used in AI and
data science. Transforms data into a new coordinate system where the
greatest variance lies on the first coordinate (principal component).

• Start with data matrix X (each row is a data point, each column a
feature)
• Compute the covariance matrix Σ = XT·X (after centering)
• Find eigenvalues and eigenvectors of Σ
• Eigenvectors with largest eigenvalues are the principal components
• Project data onto these eigenvectors to reduce dimensionality

34
 PCA cont’d
• Eigenvalues represent the amount of variance explained by
each principal component
• Larger eigenvalues indicate more important components
• The ratio of an eigenvalue to the sum of all eigenvalues gives
the proportion of variance explained
PCA applications include: Image recognition: Reducing
dimensionality of image data (e.g., eigenfaces); Data
visualization: Projecting high-dimensional data to 2D or 3D;
Noise reduction: Discarding components with small
eigenvalues; Feature extraction: Creating uncorrelated features
from correlated ones;

35
 Conclusion
• Determinants provide critical information about matrix
properties and transformations
• Eigenvalues and eigenvectors reveal intrinsic properties
of linear transformations
• PCA uses eigendecomposition to find directions of
maximum variance in data
• These concepts underpin many fundamental AI and
machine learning algorithms
• Understanding these mathematical tools enables more
effective AI model design and interpretation.
36
References
• Grok – [Link]/i/Grok

37