Module 2 Updated
Module 2 Updated
DATA SCIENCE
1
COURSE OBJECTIVES
Ø To understand the mathematical foundations required for data science.
Ø To describe a flow process for data science problems.
Ø To introduce basic data science algorithms and data visualization.
Ø To learn machine tools and techniques.
Ø To learn the ideas and tools for data visualization.
2
UNIT 2 LINEAR ALGEBRA
Algebraic view – vectors 2D, 3D and nD, matrices, product of matrix & vector, rank, null
space, solution of over determined set of equations and pseudo-inverse. Geometric view -
vectors, distance, projections, eigenvalue decomposition, Equations of line, plane, hyperplane,
circle, sphere, Hypersphere.
3
Applications of Data Science
[Link] Algebra
• Linear algebra is one of the most important mathematical and computational tools in
data science.
• It is a branch of mathematics that deals with the theory of systems of linear equations,
matrices, vector spaces, determinants, and linear transformations.
4
Perpendicular vector and orthogonal vector
[Link] a 2D point/Vector:
In 2D space, a point is defined as the (x,y) coordinates as shown above. Here, the x1 coordinate
(x coordinate) is 2, and the x2 coordinate (y coordinate) is 3.
Defining a 3D point/Vector:
5
Figure 6 : 3D vector plot
Extending the 2D concept in 3D space point ‘p’ is defined by (x,y,z) coordinates, where 2 is x1
coordinate(x coordinate), 3 is x2 coordinate (y coordinate), and 5 is x3 coordinate (z coordinate).
a) In 2D
b) In 3D
6
c) In nD:
In n-Dimensional space applying Pythagoras theorem on point ‘p’ we get,
a) In 2D
Consider we have two points say, p and q then the distance d is given by,
b) In 3D
Extending the same concept in 3D space we get the distance d’ for the points p and q as
follows:
7
c) In nD
Extending the above concept in nD space, we get the distance formulae as,
8
[Link] Product of Vectors
• Matrices can be used to represent samples with multiple attributes in a compact form.
• Matrices can also be used to represent linear equations in a compact and simple fashion.
• Linear algebra provides tools to understood and manipulate matrices to derive useful
knowledge from data.
9
• A matrix with m rows and n columns is called an m×n matrix or m-by-n matrix,
where m and n are called the matrix dimensions.
• Matrices can be used to compactly write and work with multiple linear equations, that
is, a system of linear equations.
• A matrix is a 2-D array of shape (m×n) with m rows and n columns is as given below:
Tensor
Generally, an n-dimensional array where n>2 is called a Tensor. But a matrix or a vector is also
a valid tensor. A tensor is an algebraic object that describes a multilinear relationship between
sets of algebraic objects related to a vector space.
• Order of matrix – If a matrix has 3 rows and 4 columns, order of the matrix is 3*4
i.e. row*column.
• Square matrix – The matrix in which the number of rows is equal to the number of
columns.
• Diagonal matrix – A matrix with all the non-diagonal elements equal to 0 is called
a diagonal matrix.
• Upper triangular matrix – Square matrix with all the elements below diagonal equal
to 0.
• Lower triangular matrix – Square matrix with all the elements above the diagonal
equal to 0.
• Scalar matrix – Square matrix with all the diagonal elements equal to some constant
k.
• Identity matrix – Square matrix with all the diagonal elements equal to 1 and all the
non-diagonal elements equal to 0.
• Column matrix – The matrix which consists of only 1 column. Sometimes, it is used
to represent a vector.
10
• Row matrix – A matrix consisting only of row.
• Trace – It is the sum of all the diagonal elements of a square matrix.
• Multiplication – In matrix multiplication the matrices don’t need to be quadric, but the
inner dimension needs to be same. The size of the resulting matrix will be the outer
dimensions.
11
Figure 13 : Matrix multiplication
Eg :
12
Eg :
Determinant of a matrix
13
Inverse of a Matrix
[Link] of a matrix
• Rank of a matrix is equal to the maximum number of linearly independent row
vectors in a matrix.
• A set of vectors is linearly dependent if we can express at least one of the vectors as a
linear combination of remaining vectors in the set.
• Note : Rank is the number of rows with non zero vectors.
• Rank of a matrix – Rank of a matrix is equal to the maximum number of linearly
independent row vectors in a matrix.
• A set of vectors is linearly dependent if we can express at least one of the vectors as a
linear combination of remaining vectors in the set.
• To Calculate Rank of Matrix There are Two Methods:
1. Minor method
2. Echelon form
The maximum number of linearly independent rows in a matrix A is called the row
rank of A, and the maximum number of linearly independent columns in A is called
the column rank of A.
If A is an m by n matrix, that is, if A has m rows and n columns, then it is obvious that
14
• To find the rank of a matrix, we will transform that matrix into its echelon form.
• Then determine the rank by the number of non zero rows.
• Consider the following matrix.
• While observing the rows, we can see that the second row is two times the first row.
Here we have two rows. But it does not count. The rank is considered as 1.
We can see that the rows are independent. Hence the rank of this matrix is 3.
The rank of a unit matrix of order m is m.
If A matrix is of order m×n, then ρ(A ) ≤ min{m, n } = minimum of m, n.
If A is of order n×n and |A| ≠ 0, then the rank of A = n.
If A is of order n×n and |A| = 0, then the rank of A will be less than n
15
(ii) The number of zeros before the first non-zero element in a row is less then the
number of such zeros in the next row.
For example, consider the following matrix.
Here R1 and R2 are non zero rows.
R3 is a zero row.
Note: A non-zero matrix is said to be in a row-echelon form, if all zero rows occur as
bottom rows of the matrix and if the first non-zero element in any lower row occurs to
the right of the first non-zero entry in the higher row.
If a matrix is in row-echelon form, then all elements below the leading diagonal are
zeros.
Consider the following matrix.
Check the rows from the last row of the matrix. The third row is a zero row. The first
non-zero element in the second row occurs in the third column and it lies to the right of
the first non-zero element in the first row which occurs in the second column. Hence
the matrix A is in row echelon form.
16
17
18
19
20
21
22
[Link] Space and Nullity of a Matrix
• Null space is a concept in linear algebra identifies the linear relationship among
attributes.
• The null space of a matrix A consists of all vectors B such that AB -= 0 and B ≠ 0.
• Size of null space of matrix – number of linear relation among attributes
Consider
Size of A is m * n and
23
Size of B is n * 1
24
[Link] - Nullity Theorem
The rank – nullity theorem helps us to relate the nullity of the data matrix to the rank and the
number of attributes in the data
25
Nullity of a matrix =n-r= 3-2=1
• If there are fewer equations than variables, then the system is called underdetermined
and cannot have a unique solution. In this case, there are either infinitely many or no
solutions.
• A system with more equations than variables is called overdetermined.
• If the number of equations equals the number of variables, it is a balanced or square
system.
• A balanced system or an overdetermined system may have a unique solution.
Example 1: This system has infinitely many solutions. You can tell because these two lines
are the same. (The second one is scaled by a factor of 3.)
x+2y=4
3x+6y=12
26
Notice that there is only one pivot column in this row-reduced matrix. The second column is
not a pivot column, so we call y a free variable. A system with free variables is
called dependent. Variables that do correspond to a pivot column are called fixed variables. In
general, we can express the solution of the system as the fixed variables in terms of the free
variables.
x=4−2yx=4−2y
This is a dependent system, and there are infintely many solutions, depending on the value of
the free variable y. Once a value of yy is selected, the value of x is automatically fixed.
We don't need to go any further than this. The last row reads 0x+0y+0z=−10x+0y+0z=−1, that
is, 0=−10=−1. Such a false statement reveals that this system of equations has no solution. It is
inconsistent.
27
Figure 15: Eigen value λ
Characteristic Equation
Note:
1. Solving , we get n roots for and these roots are called
characteristic roots or eigen values or latent values of the matrix A
2. Corresponding to each value of ,the equation AX = has a non-zero
solution vector X.
Fora 3x3matrix:
Method1:
Method2:
Fora2x2matrix
Method1:
28
Method2:
Problems:
Solution:
S3=Determinant of A= A=0
S1 = 3+2=5and =3(2)–1(-1)=7
Steps to find out the eigen value and eigen vector for variable x,
29
Problems:
Solution:
Let A= which is a non-symmetric matrix
---------------(1)
Case1: If From(1)]
i.e.,
i.e., ,
30
Therefore
Case2:If From(1)]
i.e.,
i.e.,
Hence,
Solution:
Let A=
=2(-5)-2(-6)-7(2)=-10+12–14=-12
3 1 0 -13 12
1 3 31 -4 0
Therefore, the eigenvaluesare3,1,and-4
Case1:If
i.e.,
(1)
(2)
(3)
0 2 2 0
x x x x x x
1=2=3Þ1=2=3
4 -16 -4 1 ç ÷
æ1ö
32
-4 -1
Therefore, X1=ç -4÷
ç-1÷
è ø
Case 2:If ,
i.e.,
(1)
33
(2)
(3)
Considering equations (1) and (2) and using method of cross-multiplication, we get,x1
x2 x3
2 -7 -1 2
-2 2 2 -2
x x x x x x
1=2=3Þ1=2=3
-10 -12 -2 5 6 1
Therefore,
Case3:If
(1)
(2)
(3)
Considering equations (1) and (2) and using method of cross-multiplication, we get,x1
x2 x3
2 -7 6 2
5 2 2 5
34
Therefore,
35
1.14. Pseudo Inverse
Pseudo inverse or Moore – Penrose inverse is the generalization of the matrix inverse that may not
be invertible. If the matrix is invertible then its inverse will be equal to pseudo inverse and denoted
by A+.
36
Problems:
37
38
1.15. Eigen Value Decomposition:
⚫ A few applications of eigen values and eigenvectors that are very useful when handing the
data in a matrix form because you could decompose them into matrices that are easy to
manipulate.
⚫ In order for the matrix “A” to be either diagonalized or eigen decomposed, it has to meet the
following criteria:
39
Problem:
• The equation of a line means an equation in x and y whose solution set is a line in the
(x,y) plane.
• Line as a geometrical object and not the graph of a function, it makes sense to treat x and
y more even handedly. The general equation for a line (normal form) is ax + by = c,
y = (-a/b) + c/b,
40
except for the special case b = 0, when the line is parallel to the y-axis.
For any two points P and Q, there is exactly one line PQ through the points. If the coordinates
of P and Q are known, then the coefficients a, b, c of an equation for the line can be found by
solving a system of linear equations.
Example: For P = (1, 2), Q = (-2, 5), find the equation ax + by = c of line PQ.
a(1) + b(2) = c, or a + 2b = c
Or -2 a + 5b = c.
Multiply the first equation by 2 and add to eliminate a from the equation:
4b + 5b = 9b = 2c + c = 3c, so b = (1/3)c.
ax + by + cz = d,
If c is not zero, it is often useful to think of the plane as the graph of a function z of x and y.
The equation can be rearranged like this:
Another useful choice, when d is not zero, is to divide by d so that the constant term = 1.
41
Given points P, Q, R in space, find the equation of the plane through the 3 points.
Example: P = (1, 1, 1), Q = (1, 2, 0), R = (-1, 2, 1). We seek the coefficients of an equation ax + by
+ cz = d, where P, Q and R satisfy the equations, thus:
a+b+c=d
a + 2b + 0c = d
-a + 2b + c = d
Subtracting the first equation from the second and then adding the second equation to the
third, we eliminate a to get
b-c=0
4b + c = 2d
Adding the equations gives 5b = 2d, or b = (2/5)d, then solving for c = b = (2/5)d and then a
= d - b - c = (1/5)d.
So the equation (with a nonzero constant left in to choose) is d(1/5)x + d(2/5)y + d(2/5)z = d,
so one choice of constant gives
x + 2y + 2z = 5
Example 2: P(x1, y1, z1), Q(x2, y2, z2), and R (x3, y3, z3) are three non-collinear points on a
plane. Find equation of plane.
42
We know that: ax + by + cz + d = 0 —————(i)
By plugging in the values of the points P, Q, and R into equation (i), we get the following:
Then, by substituting the values in the above equations, we get the following:
By plugging in the values from (ii) into (i), we end up with the following:
ax + by + cz + d = 0
ax + 3ay + 4az−9a
x + 3y + 4z−9
Therefore, the equation of the plane with the three non-collinear points P, Q, and R is x + 3y + 4z−9.
Example 3: A (3,1,2), B (6,1,2), and C (0,2,0) are three non-collinear points on a plane. Find the
equation of the plane.
Solution:
By plugging in the values of the points A, B, and C into equation (i), we get the following:
43
Solving these equations gives us
By plugging in the values from (ii) into (i), we end up with the following:
ax + by + cz + d = 0
(0)x + (—by) + ½ bz — 2b = 0
x - y + ½ z —2 = 0
2x-2y + z-4 = 0
Therefore, the equation of the plane with the three non-collinear points A, B and C is
2x-2y + z-4 = 0.
Half Space
Example
44
1.19. Equation of circle
A circle is a closed curve that is drawn from the fixed point called the center, in which all the
points on the curve are having the same distance from the center point of the center. The
equation of a circle with (h, k) center and r radius is given by:
(x-h)2 + (y-k)2 = r2
This is the standard form of the [Link], if we know the coordinates of the center of
the circle and its radius as well, we can easily find its equation.
Example:
1. Consider a circle whose center is at the origin and radius is equal to 8 units.
Solution:
2. Find the equation of the circle whose center is (3,5) and the radius is 4 units.
Solution:
45
x2 +y2 -6x -10y + 18 =0 is the equation of circle
3. Equation of a circle is x2+y2−12x−16y+19=0. Find the center and radius of the circle.
Solution:
A sphere is a geometrical object in three-dimensional space that resembles the surface of a ball.
Similar to a circle in two-dimensional space, a sphere can be mathematically defined as the set of all
points that are at the same distance from a given point. This given point is called the center of the
sphere. The distance between the center and any point on the surface of the sphere is called the radius,
represented by r.
x2 + y2 + z2 = r2
This is called the equation of a sphere, also known as the general equation of a sphere or the equation
of a sphere through the circle.
46
1.21. Equation of hypersphere
• The intersection of a sphere with a plane is a circle; the intersection of a hypersphere with a
hyperplane is a sphere. These analogies are reflected in the underlying mathematics.
• The n-hypersphere (often simply called the n-sphere) is a generalization of the circle (called
by geometers the 2-sphere) and usual sphere (called by geometers the 3-sphere) to
dimensions n>=4. The n-sphere is therefore defined (again, to a geometer; see below) as the
set of n-tuples of points (x1, x2, ..., xn) such that
x12+x22+...+xn2=R2,
• The hypersphere has a hypervolume (analogous to the volume of a sphere) of π2r 4/2, and a
surface volume (analogous to the sphere's surface area) of 2π2r 3.
47
TEXT / REFERENCE BOOKS
1. Cathy O’Neil and Rachel Schutt. Doing Data Science, Straight Talk From The Frontline.
O’Reilly. 2014.
2. Introduction to Linear Algebra - By Gilbert Strang, Wellesley-Cambridge Press, 5th
Edition.2016.
3. Avrim Blum, John Hopcroft and Ravindran Kannan. Foundations of Data Science.
48
QUESTION BANK
Part-A
[Link] Questions Competence BT Level
BTL 1
1. Define Data science. Remember
BTL 4
2. Distinguish between between a vector and a scalar point Analysis
BTL 1
3. List out the applications of data science. Remember
BTL 4
4. Differentiate between matrices and tensors. Analysis
BTL 4
5. When two vector will become perpendicular? Analysis
BTL 2
6. Define Linear algebra. Understand
Illustrate relationship diagram between data science and BTL 4
7. Analysis
other branch of studies.
BTL 4
8. Illustrate the geometrical representation of a vector point. Analysis
Is all orthogonal vector are perpendicular vectors? Justify BTL 4
9. Analysis
your answer.
Given two points A and B with coordinates (3,5) and (6,8) BTL 3
10. Apply
respectively. What is the distance between A and B?
. BTL 2
11. Understand
Define 1D vector?
BTL 2
12. Define 2D vector? Understand
BTL 2
13. Enumerate null space. Understand
BTL 2
14. Define Rank of a matrix with an example. Understand
BTL 2
15. Describe Rank-Nullity theorem. Understand
Differentiate between underdetermined and overdetermined BTL 4
16. Analysis
set of equations.
BTL 4
17. When a set of equations can be termed as balanced system? Analysis
BTL 4
18. Compare and contrast hyperplane and halfspace. Analysis
−5 2 BTL 3
19. Find out the Eigen value for the matrix ! ' Apply
−7 4
49
Write the equation of a circle when the center is at the BTL 3
20. Apply
origin?
PART B
50