INTRODUCTION TO ARTIFICIAL

INTELLIGENCE AND DATA SCIENCE

MODULE 2
 What is Data?
 Data refers to raw facts and figures collected from observations,
measurements, or experiments that can be processed to gain information.

 In simple terms, data is any piece of information that can be stored and
analyzed.
1. Numerical (Quantitative) Data

Data that represents numbers — can be measured or counted.

a. Discrete Data
 Only whole numbers, no decimals.
 Example: Number of children, cars, books.

b. Continuous Data
 Can take any value within a range (including decimals).
 Example: Temperature (37.5°C), weight (65.4 kg), speed (30.2 km/h)
2. Categorical (Qualitative) Data

Represents categories or groups, not numbers.

a. Nominal Data
 Categories without a specific order.
 Example:
• Gender: Male / Female / Other
• Blood Group: A, B, AB, O
• City: Kochi, Delhi, Mumbai

b. Ordinal Data
 Categories with a meaningful order, but difference between values is not fixed.
 Example:
• Survey ratings: Poor, Fair, Good, Excellent
• Education level: High School < Undergraduate < Postgraduate
3. Text Data (Free-form or Natural Language)

 Words, sentences, or paragraphs.

 Cannot be directly analyzed like numbers — needs Natural Language
Processing (NLP).
 Example:
Product reviews, tweets, comments, emails.
 Mathematics of Vectors and Matrices

 Linear Algebra focuses on vectors (1D arrays) and matrices (2D arrays) which
are used to represent data, equations, and transformations in a compact,
efficient way.
 Role of Linear Algebra in Data Representation

 When you transform an image in Photoshop...

You’re applying matrix transformations like scaling, rotation, translation — all linear
algebra!

 When you play a video game...

Every 3D object movement, camera shift, and lighting effect uses vectors and matrices to
position and render objects.

 When you take a photo...

The camera sensor captures light as matrices of pixels, and operations like sharpening,
blurring — linear algebra again!
 Role of Linear Algebra in Data Representation

 Linear algebra is a fundamental part of data science, machine

learning, and artificial intelligence.

 It helps in the representation and manipulation of data, which

is crucial when working with complex datasets.

 From representing data in vectors and matrices to performing

operations like matrix multiplication, linear algebra provides
the tools to analyze and interpret relationships between
features.
 Role of Linear Algebra in Data Representation

1. Data Representation: Data sets are often represented as vectors and

matrices, wherein every row corresponds to an observation and every
column represents a function. This matrix illustration permits efficient
manipulation and data analysis

Datasets are often structured as matrices, where:

•Each row = one observation/data point

•Each column = one feature/variable/function
 Role of Linear Algebra in Data Representation
Why Use Vectors and Matrices?

 Enables efficient storage and access

 Supports linear transformations (scaling, rotation, etc.) and statistical analysis
 E.g. 1: Student data is given below:

Height (cm) Weight (kg) Age (yrs)

170 65 25
160 55 30
180 75 22

The above data can be represented as a matrix as given below:

 Role of Linear Algebra in Data Representation

E.g. 2:- Another example is Image and Signal

Representation

 Audio signals = 1D vectors of amplitudes

 An audio signal is a continuous-time wave It is
sampled at regular intervals
 Each sample represents the amplitude
(loudness) of the audio wave at that moment in
time.
 These amplitude values are stored as a 1D
vector or array, usually as integers or floats.

audio_signal = [0, 124, 231, 128, -50, -128, -231, -124, 0, 124, ...]
 Role of Linear Algebra in Data Representation

 Images are stored as pixel intensity matrices

 Example: A 28x28 grayscale image = a matrix with 784 numbers → treated as a

784 D vector in ML as shown below:
Example:- Representation of an image

 In image processing, a matrix is a

fundamental way to represent an image
using numerical values. For a grayscale
image, each pixel corresponds to an
intensity value ranging from 0 (black) to 255
(white), and the image is represented as a
2D matrix where each element holds one of
these values.

 This matrix-based representation allows the

application of various linear algebra
operations such as transformations,
filtering, and compression.
 Role of Linear Algebra in Data Representation

Example 3: Text data

Suppose we have the following 3 sentences:

1."I love cats"
2."I love dogs"
3."Cats and dogs are great“

First, we extract all unique words (ignoring case and punctuation):

Vocabulary = ["i", "love", "cats", "dogs", "and", "are", "great"]
(7 words total)
 Role of Linear Algebra in Data Representation

Sentence` i love cats dogs and are great

"I love
cats" 1 1 1 0 0 0 0
"I love
dogs" 1 1 0 1 0 0 0
"Cats and
dogs are 0 0 1 1 1 1 1
great"
 Role of Linear Algebra in Data Representation

Represent Each Sentence as a Vector

Now each sentence is a 7-dimensional vector:

 Sentence 1: → [1, 1, 1, 0, 0, 0, 0]
 Sentence 2: → [1, 1, 0, 1, 0, 0, 0]
 Sentence 3: → [0, 0, 1, 1, 1, 1, 1]

These vectors can now be used in:

• Similarity search (e.g., document similarity)

•Input for machine learning models

Or the sentence vectors can be combined to a sentence matrix

 Role of Linear Algebra in Data Representation

2. Data Transformation

Scaling, rotating, translating data is need in many data sets including image
datasets

E.g. in image data:

Objects may appear in different positions, sizes, or orientations.
So, we:
 Scale (resize)
 Rotate (align)
 Translate (center)
 Role of Linear Algebra in Data Representation

 Another e.g. Features like: Income (₹1000s), Age (years) and Height (cm) have
different scales. So, we have to scale them.

 All these (scaling, rotation, etc.) can be done using matrix operations
 Role of Linear Algebra in Data Representation

3. Dimensionality Reduction: Singular Value Decomposition (SVD) and Principal

Component Analysis (PCA) methods rely on principles from linear algebra to
decrease the complexity of data while retaining critical information.

Dimensionality reduction is mainly the process of reducing the number of features

(variables) in a dataset while preserving as much important information as possible.

This helps to:

 Reduce computational complexity
 Remove noise and redundancy
 Visualize high-dimensional data in 2D or 3D
 Role of Linear Algebra in Data Representation

Simple Example: Student Performance Dataset

Suppose you have the following data for 3 students with 4 features (Math,
Physics, English, History):
Student Math Physics English History
A 90 85 70 65
B 80 78 75 70
C 85 83 72 68

So, instead of working in 4 dimensions, we can reduce to 2 dimensions (one for

science, one for arts). PCA or SVD are techniques to do that.
 Role of Linear Algebra in Data Analysis

4. Machine Learning Algorithms: Many machine learning algorithms use linear

algebra operations such as dot products, matrix multiplication, vector projections,
and inversions to learn patterns from data and make predictions.
 Role of Linear Algebra in Data Analysis
5. Correlation Analysis:-

 Correlation is a statistical measure that describes the strength and direction of a

linear relationship between two variables. Correlation matrices are calculated
using dot products and matrix multiplication.
 Role of Linear Algebra in Data Analysis
6. Distance and Similarity Calculations:-
 To measure how similar two vectors are in direction, we use cosine similarity,
which is based on the dot product and the norms of vectors in linear algebra.
Some common operations on matrices
 Product of 2 Matrices

NXP PXM NXM

• ‘inner matrix dimensions must agree’

• Note: Matrix multiplication doesn’t (generally) commute, AB  BA
Matrix Transpose
Eigenvectors & eigenvalues
 Definitions

A nonzero vector x is an eigenvector (or characteristic vector) of a

square matrix A if there exists a scalar λ such that Ax = λx. Then λ is
an eigenvalue (or characteristic value) of A.
Are there any special vectors
that only get scaled?

• For this special vector, multiplying

by M is like multiplying by a scalar.
= (3,3) • (1,1) is called an eigenvector of M
• 3 (the scaling factor) is called the
= (1,1) eigenvalue associated with this
eigenvector
 Eigenvalues

Let x be an eigenvector of the matrix A. Then there must exist an eigenvalue λ

such that Ax = λx or, equivalently,
Ax - λx = 0 or
(A – λI)x = 0
det(A – λI) = 0
This is called the characteristic equation of A. Its roots determine the eigenvalues
of A.
 Matrix decomposition

Matrix decomposition (also called matrix factorization) is a process of breaking a

matrix into a product of simpler or more structured matrices. It's a core concept
in linear algebra used widely in:

 Solving systems of linear equations

 Data compression
 Machine learning
 Why decompose a matrix?

Matrix operations (like solving Ax=b) can be computationally expensive.

Decomposition helps by turning a complex matrix into simpler parts that are
easier to work with.
 The Core Idea

You start with a matrix A, and decompose it, A=M1⋅M2⋅M3….

These smaller matrices can be:

Diagonal
Triangular
Orthogonal (rotation/reflection matrices)
Sparse
Simpler in some computational way

These simpler matrices can have special properties that make them faster to
compute with, easier to invert and more interpretable
 Spectral Decomposition (Eigen Decomposition)
 Spectral Decomposition is a way of expressing a real, square, symmetric matrix
using its eigenvalues and eigenvectors.

 Spectral Decomposition (also called Eigen decomposition) is a method used to

break down a square symmetric matrix into its eigenvalues and eigenvectors.
It’s a fundamental concept in linear algebra, especially useful in systems
analysis, quantum mechanics, and data science.
 Singular Value Decomposition (SVD)

Singular Value Decomposition (SVD) is a powerful and universal method of

matrix decomposition. It breaks any matrix into three simpler, structured
matrices that reveal its essential properties.
Singular Value Decomposition (SVD)
 Singular Value Decomposition (SVD)

A matrix A is taken and it is transformed by rotating, stretching, flipping, etc. to

new output matrices.

U:-It can be considered as the rotation of final result, each column in this
matrix tells you a direction in the output space

∑:-The values on the diagonal (singular values) indicates how much to stretch
or compress in each direction
 SVD vs. Spectral Decomposition

 Spectral decomposition is a special case of SVD that works only for square,
symmetric matrices

 SVD works for all matrices

 Spectral decomposition is therefore a simpler version of SVD for special

matrices
Another example
 Applications of Singular Value Decomposition (SVD)

1. Dimensionality Reduction (Principal Component Analysis - PCA)

 SVD helps identify the most important features (principal components) in

data.
 Reduces noise, improves computational efficiency, and helps with data
compression.
 Used in image compression, pattern recognition, and machine learning
preprocessing.
2. Data Noise Filtering

 SVD can separate signal from noise by keeping large singular values (signal) and
discarding small ones (noise).
 Especially useful in:
• EEG/ECG signal denoising
• Sensor data filtering
• Financial time series smoothing
3. Recommender Systems

 Singular Value Decomposition (SVD) is widely used in recommender systems to

predict missing entries. In platforms like Netflix or Amazon, users rate only a few
items, leaving many entries blank.

 SVD decomposes this matrix into three components—user preferences, item

features, and their importance—allowing us to identify hidden patterns between
users and items.

 By reconstructing the matrix using only the most significant features, SVD can
estimate the missing ratings accurately. These predicted ratings help recommend
new movies, products, or songs that a user is likely to enjoy, making SVD a powerful
tool for personalized recommendation systems.
 INTRODUCTION TO ARTIFICIAL
INTELLIGENCE AND DATA SCIENCE
MODULE 2
What is Dimensionality Reduction?

 Dimensionality Reduction is the process of reducing the number of input

variables (features) in a dataset while preserving its essential structure and
information.

 It means simplifying your dataset by keeping the most important

features/variables and discarding the less important or redundant ones.
Why is Dimensionality Reduction Important?

[Link] computational cost

[Link] noise and redundancy
[Link] model performance (less overfitting)
[Link] data visualization (e.g., converting high-dimensional data to 2D or
3D for plotting)
[Link] interpretability
Key Idea:
In high-dimensional datasets, many features may be:

 Correlated (contain overlapping info)

 Irrelevant (don't contribute much)
 Noisy (contain random error)

Dimensionality reduction identifies the core structure of the data using fewer
variables.
Two Main Types of Dimensionality Reduction:

1. Feature Selection
 Keep only a subset of original features, no transformation is applied to the data
— only a selection of existing features.
 Methods: Filter methods, Embedded methods, etc

2. Feature Extraction
 Create new features by combining existing ones.
 These new features capture the most important patterns.
•Methods:
• Principal Component Analysis (PCA)
• Linear Discriminant Analysis (LDA)
• Autoencoders (Neural Networks)
 PCA's Goal
“In which direction is the data most spread out?”

 It finds new axes (directions) that capture the spread (variance) of the data.
These are called principal components.

 PCA aims to project high-dimensional data into lower dimensional space while
preserving as much of the data’s variability as possible. This is through identifying
directions along which data varies the most
 PCA

Assumption: Data lies on or near a low dimensional subspace

Axes of this subspace are effective representation of the data

J. Leskovec, A. Rajaraman, J. Ullman: Mining of Massive Datasets,

76
Simple Example:
 Imagine your dataset has 100 features (columns in Excel) — like height,
weight, age, income, blood pressure, etc.

 Some features are related. For example:

 Height and weight often go together.

 Age and blood pressure may be related.

 This means many features are redundant — repeating the same info.
How PCA Helps:

PCA finds the 10 best new features (called principal components) that combine
your 100 features in a smart way.

 These 10 components:
 Keep the most important patterns.
 Remove repeated or unimportant features.
 Still allow you to do analysis, machine learning, or visualization.

 If those 10 components capture 95% of the variance, it means:

 You're keeping 95% of the total "information" or structure of the original
data.
 And you're throwing away only 5% of the less useful details.
 The Principal Components
Principal component #1 points
in the direction of the largest variance.

Each subsequent principal component

 is orthogonal to the previous ones, and

 points in the directions of the largest variance of the remaining
residual subspace
Dataset 1st PCA axis 2nd PCA axis
If the variables or features have same scale, no need to divide by standard deviation, you just
need to centre the data
 Applications of PCA

1. Data Visualization

Goal: Reduce high-dimensional data to 2D or

3D for easier human interpretation.

How PCA helps:

PCA finds the directions (principal

components) along which the variance of the
data is maximized. By projecting the data
onto the top 2 or 3 principal components, we
can visualize complex data structures.
2. Image Compression

•Goal: Reduce the storage space of

image data without losing significant
visual quality.

•How PCA helps:

Each image can be treated as a
high-dimensional data vector
(e.g., each pixel is a feature).
PCA transforms the data into a new basis (principal components), keeping only the
top few components that capture the most variance (i.e., visual information).
This reduces data dimensionality and redundancy.