AI-501 Mathematics for AI
Vectors – Notation and Basic Operations
Zubair Khalid
School of Science and Engineering
Outline
• Vectors notation
• Applications Chapter 2
• Basic vector operations
Chapter 1 and 2
Vectors
Definition:
A vector is an ordered finite list of numbers (real or complex).
Notation:
Usually denoted by a letter symbol; stack the list of numbers in an ordered form.
For example, consider a vector of 4 real numbers given by
Size of a vector: Number of elements the vector contains (also referred to as length or dimension).
Entry of a vector:
Vectors
Zero vector: A vector with all elements equal to zero.
One vector: A vector with all elements equal to one.
Unit vector: A standard unit vector is vector with all elements zero except one
element that is equal to one.
k-Sparse vector: A vector with at-most k non-zero entries.
Geometric Interpretation
2-vector:
Examples of Vectors - Applications
Location
Displacement, Velocity, Acceleration
Examples of Vectors - Applications
Color
Each color is represented by 3-vector.
Quantities
An n-vector q can represent the amounts or quantities of n different resources or
products held (or produced, or required) by an entity such as a company.
For example, n-vector represents the quantity of n products stocked in a warehouse.
Values across a population
An n-vector can give the values of some quantity across a population of individuals or entities.
For example, an n-vector a can represent the blood pressure of a collection of n patients, with
ai the blood pressure of patient i, for i = 1, 2, ..., n.
Examples of Vectors - Applications
Image
Time series
12-vector can represent the average monthly
temperature, rainfall, pressure etc of Lahore.
30-vector can represent the number of expected COVID-
19 patients in Pakistan cases over the next 30 days.
Other examples include exchange rate, audio, and, in fact,
any quantity that varies over time.
Examples of Vectors - Applications
Audio
Sound (Flute) versus time
8 second sound = 44100x8-vector
Feature or Attribute
In Machine Learning, classification is mostly carried out
collecting features or characteristics derived from the
object.
Such a vector is sometimes called a feature vector, and
its entries are called the features or attributes.
Emotion Recognition
Operations on Vectors
Additivity and Scaling
Operations on Vectors
Linear Combination
Operations on Vectors
Linear Combination
Operations on Vectors
Linear Combination
Operations on Vectors
Inner Product
Operations on Vectors
Inner Product - Applications
Operations on Vectors
Inner Product - Applications
Operations on Vectors
Inner Product – Applications – In class example:
Operations on Vectors
Norm
Operations on Vectors
Norm
Operations on Vectors
Norm – Example:
Operations on Vectors
Distance
Interpretation: Compare two vectors; close or far
Operations on Vectors
Application Examples
Prediction error
Operations on Vectors
Angle
Operations on Vectors
Correlation coefficient
Operations on Vectors
Standard Deviation
Operations on Vectors
Standard Deviation
Relationship between std, avg and rms
Operations on Vectors
Standard Deviation
Relationship between std, avg and rms
Operations on Vectors
Standard Deviation
Relationship between std, avg and rms
Operations on Vectors
Standard Deviation
Relationship between std, avg and rms
Operations on Vectors
Standard Deviation
Concept of Standardization
Span
Span
Linear Independence
Linear Independence
Linear Independence
Examples
Basis
Independence-Dimension Inequality
(Connection with linear independence):
Basis
Independence-Dimension Inequality – Example:
(Connection with linear independence):
Basis
Basis of a set of Vectors:
Basis
Basis of a set of Vectors – Example:
Orthonormal Vectors
Orthonormal Vectors
Example:
Orthonormal Vectors
Linear Combination of Orthonormal Vectors
Orthonormal Vectors
Linear Combination of Orthonormal Vectors
Outline
• Gram-Schmidt Orthogonalization
• Overview
• Algorithm
• Example
Chapter 5
Gram-Schmidt Orthogonalization
Overview:
Gram-Schmidt Orthogonalization
Example:
Gram-Schmidt Orthogonalization
Example:
Gram-Schmidt Orthogonalization
Orthogonalization Normalization
Interpretation
Iteration 1
Iteration 2
Gram-Schmidt Orthogonalization
Algorithm:
Orthogonalization
Normalization
Outline
• Vector Spaces
• Subspaces
• Dimension of subspace
Vector Space
Definition
Subspace
Definition
Alternatively
Non-emptiness
Closure under addition
Closure under multiplication
Subspace
Examples (Subspace or not):
Line through origin Plane through origin
Line not through origin First quadrant Line union plane
Subspace
Examples – In class questions:
Dimension of Subspace
The number of vectors in any basis of V is called the dimension of V.
Expressed as dim(V)
Examples