MODULE 2

Linear Algebra for Data Science

Linear algebra is a cornerstone of data science, enabling the manipulation,
transformation, and analysis of data. An algebraic view of linear algebra focuses
on understanding the operations, concepts, and structures used to handle and
process data efficiently. In the context of data science, the algebraic view provides
essential insights into how linear equations, vector spaces, matrices, and
transformations play a role in modeling and solving data-driven problems.
Vectors and Vector Spaces
Vectors:
A vector is an ordered set of numbers, which can represent data in multi-
dimensional space. Each component of a vector corresponds to a feature of the
data. Algebraically, a vector is often written as:
v=[v1,v2,…,vn]
where vi represents the value of the i-th feature. In data science, vectors represent
data points in feature space, and their operations (addition, scaling, dot product)
are crucial for tasks such as regression, classification, and similarity analysis.
Vector Spaces:
A vector space is a collection of vectors where you can perform vector addition
and scalar multiplication, and the results remain within the same space. For
example, the space of all 2D vectors R2 forms a vector space where any two 2D
vectors can be added together, and any 2D vector can be multiplied by a scalar.
In data science, vector spaces represent the feature space of datasets, where each
data point corresponds to a vector in this space.
Linear Independence and Basis:
A set of vectors is linearly independent if no vector can be written as a linear
combination of others. These vectors form a basis for the vector space, and the
number of vectors in the basis is called the dimension of the vector space. This
is particularly important for dimensionality reduction techniques in data science,
such as Principal Component Analysis (PCA).
Vectors:
• A vector is an ordered collection of numbers (elements), typically
representing data points or features in a dataset.
• Vectors can be row vectors (horizontal) or column vectors (vertical).
For example:
Operations on Vectors:
• Addition: Adding two vectors element-wise.
• Scalar multiplication: Multiplying a vector by a scalar scales each
element.

Vectors represent data points or features in machine learning models

(e.g., in a dataset with multiple features, each data point is a vector of
values). Matrices are used to represent datasets, where each row is a data
point and each column is a feature.
Data Science Application: When working with large datasets, matrices
help organize the data for algorithms such as linear regression or neural
networks.
Matrices and Matrix Operations
A matrix is a two-dimensional array of numbers, where each element represents
a specific data value. In data science, matrices are used to represent datasets,
where rows correspond to data points (observations) and columns represent
features.
A matrix A with m rows and n columns is denoted as:
Matrices represent linear transformations in data science. For example, in linear
regression, a matrix XXX holds feature values, and the goal is to find the best-
fitting parameters using matrix operations.
Matrix Operations:
• Addition/Subtraction: Can only be done with matrices of the same
dimension.
• Multiplication: Matrix multiplication is defined as the dot product
between rows of the first matrix and columns of the second.
• Transpose: Swapping rows and columns.
• Matrix Addition: Two matrices of the same size can be added element-wise.
• Scalar Multiplication: Each element of a matrix can be multiplied by a scalar.
• Matrix Multiplication: This operation is key in linear algebra and is widely used in
data science algorithms, particularly in machine learning. The product of two matrices
is computed by taking the dot product of the rows of the first matrix with the columns
of the second matrix.

Identity Matrix and Inverse:

An identity matrix is a square matrix where all diagonal elements are 1, and all off-diagonal
elements are 0. It acts like the number 1 in matrix operations, as multiplying any matrix by
the identity matrix leaves the original matrix unchanged.

The inverse of a matrix A, denoted A−1, is the matrix that, when multiplied by A, gives the
identity matrix. In data science, matrix inverses are used in solving linear systems, such as in
linear regression problems.

This operation is fundamental to many machine learning algorithms. For instance, linear
regression involves multiplying the feature matrix XXX with the weight vector www to
make predictions.

Data Science Application: In the prediction phase of linear regression or in neural

networks, matrix-vector products calculate the model’s output.
Rank is important in understanding the linear independence of features in a
dataset, and it can be used in dimensionality reduction (e.g., Principal
Component Analysis, PCA).
Null Space helps identify the features or patterns that do not contribute any new
information, often used in feature selection or sparse models.
Data Science Application: If the data matrix has low rank, it means there are highly
correlated features. This can affect the performance of models like regression or
classification, leading to multicollinearity.
Overdetermined systems (where there are more equations than unknowns) are
common in regression tasks. The least squares method is used to find the best-
fitting model when there are more data points than parameters.
Data Science Application: In linear regression, overdetermined systems are
solved using the least squares method to minimize the error.

• The pseudo inverse is used to solve non-square systems of linear

equations and is a crucial tool in linear regression (when the matrix XXX
is not invertible).
• Data Science Application: The pseudo-inverse is used to compute
solutions in regularized regression models (like ridge regression) and
when the data matrix is not full rank.
 • Geometrically, vectors represent data points in a high-dimensional space.
The dot product and cosine similarity are often used to measure the
similarity between data points, a key step in clustering and
recommendation systems.
• Data Science Application: In algorithms like K-means clustering, the
distance (or similarity) between points plays a key role. Cosine similarity
is used in text mining and recommendation systems.

Distances between vectors are used in various algorithms like K-Nearest

Neighbors (KNN), Clustering, and classification.
Data Science Application: The Euclidean distance or Manhattan distance is
used to compute distances between data points in clustering algorithms like K-
means.
Projections are used in dimensionality reduction techniques like Principal
Component Analysis (PCA), where high-dimensional data is projected onto
lower-dimensional subspaces.
Data Science Application: In PCA, data points are projected onto the principal
components, reducing the number of features while retaining most of the variance.
 Eigenvalues and eigenvectors are central in many dimensionality reduction
techniques like PCA and in the optimization of machine learning models. They
help in understanding the principal directions in which the data varies the most.
 Data Science Application: In PCA, the eigenvectors represent the directions of
maximum variance, and the eigenvalues indicate how much variance is explained
by each principal component.

PROBLEMS