III. Questions Question 1 (4.5 points).
Investigate in-detail the singular value
decomposition (SVD) of a rectangular matrix:
[Link]/Formulas of singular values:
-Definition : Singular values of a matrix A are the square roots of the eigenvalues
of ATA ( if matrix A is matrix m x n )
-Formulas: the singular values can be computed as follows:
det(ATA−σ2I)=0.
This gives the eigenvalues of A A , which are σ2 where σ are the singular values
T
of A.
2. - At least one proposition related to singular values
The singular values of matrix A are the square roots of the non-negative
eigenvalues of ATA.
σi=√ λi
where λi are the eigenvalues of the matrix ATA.
3. Definition/Formulas of SVD
-Definition: This is a method to decompose a square matrix into other special
matrices.
-Formulas:
A=UΣVT
Where:
• A is the original matrix we want to analyze
• U is an m x m matrix, with columns contain eigenvectors of matrix AAT
•Σ is a diagonal matrix containing the singular values
•V is an n x n matrix, with the eigenvectors of ATA as its columns.
4. At least one theorem related to a SVD finding
5. Steps for finding a SVD
1. Compute ATAA^T AATA: Calculate the matrix ATA
2. Find Eigenvalues and Eigenvectors of ATA: These eigenvalues are the
square of the singular values, and the eigenvectors make up the matrix V.
3. Form matrix VVV: Use the eigenvectors of ATA to create V.
4. Compute AATA A^TAAT: Now calculate AAT.
5. Find Eigenvalues and Eigenvectors of AAT: The eigenvectors of AAT
make up matrix U.
6. Form diagonal matrix Σ: The singular values are the square roots of the
eigenvalues, and they go on the diagonal of Σ.
where:
U and V are orthogonal matrices.
Σ is a diagonal matrix with non-negative singular values.
[Link] applications of SVD
In math:
Matrix Approximation: SVD helps to simplify a matrix by keeping only the
most important parts.
Solving Equations: SVD helps to solve tough equations with unstable or bad
matrices.
Eigenvalue Decomposition: SVD works with non-square matrices, just like
eigenvalue decomposition.
Dimensionality Reduction: SVD is used to reduce the number of variables in
data, keeping the key information.
Matrix Factorization: SVD breaks a matrix into smaller pieces to make
calculations easier.
In real life:
Data Compression: SVD reduces the size of large datasets while keeping key
information.
Image Compression: SVD simplifies images by using fewer values, saving
storage space.
Noise Reduction: SVD removes unnecessary noise from data by eliminating
small values.
7. Take two examples of SVD finding (examples from Internet/books are not
allowed) :
o One with a matrix of size 2 rows x 3 columns and the rank of the matrix is
2
o One with a matrix of size 2 rows x 3 columns and the rank of the matrix is
1
Example 1: Rank 2 Matrix (2x3 Matrix)
1 2 3
Matrix: A = ( )
9 1 7
Step 1: Compute ATA
First, find the transpose of A:
( )
1 9
T
A = 2 1
3 7
After that, find ATA:
( )( )
1 9
T 1 2 3
A A= 2 1
9 1 7
3 7
( )
82 11 70
= 11 5 17
70 17 58
Step 2: Find Eigenvalues and Eigenvectors of ATA
Arcording to: det(ATA−λI)=0
( )
8 2−λ 11 70
T
A A = 11 5−λ 17
70 17 58−λ
1.1 Question 2
( )
1 2 2
Question 2 (1.5 points): Find a matrix P that diagonalizes A= 2 1 1
0 0 1
In the first step we have to find the Eigenvalues of matrix A:
( ) ( )
1 2 2 1 0 0
With matrix A= 2 1 1 and I= 0 1 0
0 0 1 0 0 1
( )
1−λ 2 2
So det(A−λI)= 2 1−λ 1
0 0 1−λ
= (1− λ)