0% found this document useful (0 votes)
2 views9 pages

Module 1 Linear Algebra Teaching Guide

Module 1 covers Linear Algebra, focusing on vectors and matrices, including definitions, operations, and key concepts such as dot product, cross product, and matrix inversion. It provides graded examples to illustrate basic to advanced applications, such as calculating norms, projections, and solving systems of equations. The content is essential for understanding transformations in quantum mechanics, machine learning, and computer graphics.

Uploaded by

Ishani Murkewar
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
2 views9 pages

Module 1 Linear Algebra Teaching Guide

Module 1 covers Linear Algebra, focusing on vectors and matrices, including definitions, operations, and key concepts such as dot product, cross product, and matrix inversion. It provides graded examples to illustrate basic to advanced applications, such as calculating norms, projections, and solving systems of equations. The content is essential for understanding transformations in quantum mechanics, machine learning, and computer graphics.

Uploaded by

Ishani Murkewar
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Module 1: Linear Algebra

1.1 Vectors

Theory & Core Concepts


A vector is a mathematical object possessing both magnitude and direction, foundational to represent states in quantum
mechanics, features in machine learning, and transformations in computer graphics.

 Definition & Types: A matrix with a single column (n × 1) or single row (1 × n). In abstract linear algebra, a column
vector v ∈ ℝⁿ is written as a vertical array of components v = [v₁, v₂, ..., v] ᵀ.
 Zero & Unit Vectors: Standard zero vector 0 = [0, 0, ..., 0]ᵀ with zero magnitude. Unit vector u is any vector with
length ‖u‖ = 1. Normalized vector from v is u = v / ‖v‖.
 Basic Operations: Vector addition v + w = [v₁ + w₁, ..., vₙ + wₙ]ᵀ (head-to-tail rule). Scalar multiplication αv = [αv₁, ...,
αvₙ]ᵀ scales magnitude by |α| and reverses direction if α < 0.
 Dot Product & Geometric Interpretation: For u, v ∈ ℝⁿ, the dot product is defined as u · v = ∑ᵢ uᵢ vᵢ = uᵀ v = ‖u‖ ‖v‖
cos(θ), where θ is the angle between vectors. Geometrically, it measures directional alignment.
 Cross Product (3D): Defined specifically in 3D (u, v ∈ ℝ³), producing a vector orthogonal to both: u × v = [(u₂v₃ - u₃v₂),
(u₃v₁ - u₁v₃), (u₁v₂ - u₂v₁)]ᵀ. Magnitude ‖u × v‖ = ‖u‖ ‖v‖ sin(θ) equals the area of the parallelogram spanned by u and
v.
 Vector Norms: Norms measure vector length/magnitude:
• L₁ Norm (Manhattan): ‖v‖₁ = ∑ |vᵢ|
• L₂ Norm (Euclidean): ‖v‖₂ = √(∑ vᵢ²)
• L∞ Norm (Maximum): ‖v‖∞ = max |vᵢ|
 Orthogonality & Projections: Vectors u, v are orthogonal if u · v = 0 (θ = 90°). Vector projection of v onto u is projᵤ(v)
= ((v · u) / ‖u‖²) u.
 Linear Independence & Span: A set of vectors {v₁, ..., vₖ} is linearly independent if ∑ cᵢ vᵢ = 0 implies all scalar
coefficients cᵢ = 0. Span({v₁,...,vₖ}) is the set of all possible linear combinations.
 Basis & Dimension: A basis for a vector space V is a set of linearly independent vectors that spans V. Dimension
dim(V) is the number of vectors in any basis for V.

Graded Worked Examples (Easy to Hard)


Example 1 (Easy): Basic Operations and Vector Norms
Problem Statement:
Given vectors u = [3, -4, 0]ᵀ and v = [1, 2, 2]ᵀ in ℝ³:
a) Compute 2u - 3v.
b) Calculate the L₁, L₂, and L∞ norms of u.
c) Find the unit vector in the direction of u.
Solution / Step-by-Step Proof:
a) Scalar multiplication and subtraction:
2u = [6, -8, 0]ᵀ, 3v = [3, 6, 6]ᵀ
2u - 3v = [6 - 3, -8 - 6, 0 - 6]ᵀ = [3, -14, -6]ᵀ

b) Calculating Norms for u = [3, -4, 0]ᵀ:


• L₁ Norm: ‖u‖₁ = |3| + |-4| + |0| = 3 + 4 + 0 = 7
• L₂ Norm: ‖u‖₂ = √(3² + (-4)² + 0²) = √(9 + 16 + 0) = √25 = 5
• L∞ Norm: ‖u‖∞ = max(|3|, |-4|, |0|) = 4

c) Unit Vector in direction of u:


û = u / ‖u‖₂ = (1/5)[3, -4, 0]ᵀ = [0.6, -0.8, 0]ᵀ.

Example 2 (Easy-Medium): Dot Product, Angle, and Projections


Problem Statement:
Let a = [2, 1, -2]ᵀ and b = [1, 1, 0]ᵀ.
a) Find the dot product a · b.
b) Find the angle θ between a and b.
c) Calculate the orthogonal projection of a onto b (proj_b(a)).
Solution / Step-by-Step Proof:
a) Dot Product:
a · b = (2)(1) + (1)(1) + (-2)(0) = 2 + 1 + 0 = 3

b) Angle θ:
‖a‖₂ = √(2² + 1² + (-2)²) = √(4 + 1 + 4) = 3
‖b‖₂ = √(1² + 1² + 0²) = √2
cos(θ) = (a · b) / (‖a‖₂ ‖b‖₂) = 3 / (3 √2) = 1 / √2
θ = arccos(1 / √2) = 45° or π/4 radians.

c) Vector Projection proj_b(a):


proj_b(a) = [(a · b) / ‖b‖₂²] b = [3 / (√2)²] [1, 1, 0]ᵀ = (3/2) [1, 1, 0]ᵀ = [1.5, 1.5, 0]ᵀ.

Example 3 (Medium): 3D Cross Product and Geometric Area


Problem Statement:
Find a vector orthogonal to both p = [1, 2, 3]ᵀ and q = [4, 5, 6]ᵀ using the cross product, and calculate the area of the
parallelogram formed by p and q.
Solution / Step-by-Step Proof:
a) Cross Product p × q:
p×q=|i j k|
|1 2 3|
|4 5 6|
= i(2·6 - 3·5) - j(1·6 - 3·4) + k(1·5 - 2·4)
= i(12 - 15) - j(6 - 12) + k(5 - 8)
= -3i + 6j - 3k = [-3, 6, -3]ᵀ.

b) Verification of Orthogonality:
(p × q) · p = (-3)(1) + (6)(2) + (-3)(3) = -3 + 12 - 9 = 0
(p × q) · q = (-3)(4) + (6)(5) + (-3)(6) = -12 + 30 - 18 = 0

c) Area of Parallelogram:
Area = ‖p × q‖₂ = √((-3)² + 6² + (-3)²) = √(9 + 36 + 9) = √54 = 3√6 ≈ 7.348 square units.
Example 4 (Medium-Hard): Testing Linear Independence and Span
Problem Statement:
Determine whether the set of vectors V = {v₁ = [1, 2, 3]ᵀ, v₂ = [0, 1, 2]ᵀ, v₃ = [2, 1, 0]ᵀ} is linearly independent in ℝ³, and
determine if V spans ℝ³.
Solution / Step-by-Step Proof:
Set up the equation c₁ v₁ + c₂ v₂ + c₃ v₃ = 0:
c₁[1, 2, 3]ᵀ + c₂[0, 1, 2]ᵀ + c₃[2, 1, 0]ᵀ = [0, 0, 0]ᵀ
Form the coefficient matrix A with vectors as columns:
A=[1 0 2]
[2 1 1]
[3 2 0]

Compute the determinant det(A):


det(A) = 1·(1·0 - 1·2) - 0 + 2·(2·2 - 1·3)
det(A) = 1(-2) + 2(4 - 3) = -2 + 2 = 0.

Conclusion:
Since det(A) = 0, the matrix columns are linearly dependent. Thus, {v₁, v₂, v₃} is NOT linearly independent.
Because 3 vectors in ℝ³ are linearly dependent, they cannot span ℝ³ (their span forms a 2D plane in ℝ³).

Example 5 (Hard): Gram-Schmidt Orthogonalization (Quantum / ML Basis construction)


Problem Statement:
Using the Gram-Schmidt process, convert the linearly independent set {u₁ = [1, 1, 0]ᵀ, u₂ = [1, 0, 1]ᵀ} into an orthonormal
basis {q₁, q₂}.
Solution / Step-by-Step Proof:
Step 1: Construct first orthogonal vector v₁ and unit vector q₁:
v₁ = u₁ = [1, 1, 0]ᵀ
‖v₁‖ = √(1² + 1² + 0) = √2
q₁ = v₁ / ‖v₁‖ = [1/√2, 1/√2, 0]ᵀ

Step 2: Construct second orthogonal vector v₂ by subtracting projection onto q₁:


proj_q₁(u₂) = (u₂ · q₁) q₁
u₂ · q₁ = (1)(1/√2) + (0)(1/√2) + (1)(0) = 1/√2
proj_q₁(u₂) = (1/√2) [1/√2, 1/√2, 0]ᵀ = [1/2, 1/2, 0]ᵀ
v₂ = u₂ - proj_q₁(u₂) = [1, 0, 1]ᵀ - [1/2, 1/2, 0]ᵀ = [1/2, -1/2, 1]ᵀ

Step 3: Normalize v₂ to get q₂:


‖v₂‖ = √((1/2)² + (-1/2)² + 1²) = √(1/4 + 1/4 + 1) = √(3/2)
q₂ = v₂ / ‖v₂‖ = √(2/3) [1/2, -1/2, 1]ᵀ = [1/√6, -1/√6, √(2/3)]ᵀ

Verification: q₁ · q₂ = (1/√2)(1/√6) + (1/√2)(-1/√6) + 0 = 1/√12 - 1/√12 = 0. Orthonormal basis constructed successfully.


1.2 Matrices

Theory & Core Concepts


Matrices represent linear transformations between vector spaces and serve as the backbone for solving system
equations, graph embeddings, dimensionality reduction (PCA), and quantum operator representations.
 Matrix Types: Square (m=n), Diagonal (a_ij=0 for i≠j), Identity (I), Symmetric (A = Aᵀ), Sparse (majority elements
zero).
 Matrix Operations: Addition A + B (element-wise), Multiplication C = AB where C_ij = ∑_k A_ik B_kj (requires inner
dimensions to match), Transpose Aᵀ where (Aᵀ)_ij = A_ji.
 Inversion & Determinant: Inversion: A⁻¹ exists iff det(A) ≠ 0. Inverse satisfies A A⁻¹ = A⁻¹ A = I. Determinant det(A)
measures volume scaling factor under matrix transformation.
 Rank & Row Echelon Form: Rank r(A) is the maximum number of linearly independent columns (or rows). Nullity = n
- rank(A) (Rank-Nullity Theorem).
 Systems of Equations (Ax = b): Gaussian Elimination reduces [A|b] to Row Echelon Form (REF) using row operations.
Gauss-Jordan Elimination further reduces to Reduced Row Echelon Form (RREF) to solve Ax = b directly.
 Matrix Factorizations: Core Matrix Factorizations:
• LU Decomposition: A = LU (Lower and Upper triangular) for efficient linear system solving.
• QR Decomposition: A = QR (Orthogonal Q, Upper triangular R) for least-squares fitting.
• Singular Value Decomposition (SVD): A = U Σ Vᵀ (U, V orthogonal; Σ singular values) for low-rank approximations
and data compression.
 Eigenvalues & Eigenvectors: Satisfies A v = λ v where λ is eigenvalue and v ≠ 0 eigenvector. Characteristic equation:
det(A - λI) = 0.
 Diagonalization & Spectral Theorem: A = P D P⁻¹ where D is diagonal containing eigenvalues, P contains
eigenvectors. Spectral Theorem: Any real symmetric matrix is orthogonally diagonalizable (A = Q D Qᵀ).

Graded Worked Examples (Easy to Hard)


Example 1 (Easy): Matrix Multiplication and Transpose Properties
Problem Statement:
Given matrices A = [[1, 2], [3, 4]] and B = [[2, 0], [1, 5]], calculate:
a) AB
b) BA
c) Verify that (AB)ᵀ = Bᵀ Aᵀ.
Solution / Step-by-Step Proof:
a) AB = [[1·2 + 2·1, 1·0 + 2·5], [3·2 + 4·1, 3·0 + 4·5]]
= [[2 + 2, 0 + 10], [6 + 4, 0 + 20]] = [[4, 10], [10, 20]]

b) BA = [[2·1 + 0·3, 2·2 + 0·4], [1·1 + 5·3, 1·2 + 5·4]]


= [[2, 4], [16, 22]] (Note: Matrix multiplication is non-commutative, AB ≠ BA).

c) Transpose Verification:
(AB)ᵀ = [[4, 10], [10, 20]]ᵀ = [[4, 10], [10, 20]]
Aᵀ = [[1, 3], [2, 4]], Bᵀ = [[2, 1], [0, 5]]
Bᵀ Aᵀ = [[2·1 + 1·2, 2·3 + 1·4], [0·1 + 5·2, 0·3 + 5·4]] = [[4, 10], [10, 20]].
Thus, (AB)ᵀ = Bᵀ Aᵀ holds true.
Example 2 (Easy-Medium): Matrix Inversion and Systems Ax = b
Problem Statement:
Solve the system of linear equations using matrix inversion A⁻¹:
2x + y = 5
4x + 3y = 11
Solution / Step-by-Step Proof:
Step 1: Write in matrix form Ax = b:
A = [[2, 1], [4, 3]], x = [x, y]ᵀ, b = [5, 11]ᵀ

Step 2: Find determinant det(A):


det(A) = (2)(3) - (1)(4) = 6 - 4 = 2 ≠ 0 (Inverse exists).

Step 3: Compute inverse A⁻¹:


For 2×2 matrix [[a, b], [c, d]], A⁻¹ = (1/det(A)) [[d, -b], [-c, a]]
A⁻¹ = (1/2) [[3, -1], [-4, 2]] = [[1.5, -0.5], [-2, 1]]

Step 4: Solve x = A⁻¹ b:


x = [[1.5, -0.5], [-2, 1]] [5, 11]ᵀ
x = (1.5)(5) + (-0.5)(11) = 7.5 - 5.5 = 2
y = (-2)(5) + (1)(11) = -10 + 11 = 1
Solution: x = 2, y = 1.

Example 3 (Medium): Gauss-Jordan Elimination and Matrix Rank


Problem Statement:
Find the Reduced Row Echelon Form (RREF) and rank of matrix A:
A = [ 1 2 -1 3 ]
[2 4 1 0]
[3 6 0 3]
Solution / Step-by-Step Proof:
Step 1: Row operations to reach REF:
R₂ ← R₂ - 2R₁: [0, 0, 3, -6]
R₃ ← R₃ - 3R₁: [0, 0, 3, -6]

Step 2: Eliminate R₃:


R₃ ← R₃ - R₂: [0, 0, 0, 0]

Step 3: Normalize pivot row 2:


R₂ ← (1/3) R₂: [0, 0, 1, -2]

Step 4: Eliminate above pivot in R₁:


R₁ ← R₁ + R₂: [1, 2, 0, 1]

Final RREF(A):
RREF = [ 1 2 0 1 ]
[ 0 0 1 -2 ]
[0 0 0 0]

Rank Analysis: There are 2 non-zero pivot rows. Therefore, Rank(A) = 2, Nullity(A) = 4 - 2 = 2.

Example 4 (Medium-Hard): Eigenvalues, Eigenvectors, and Diagonalization


Problem Statement:
Find the eigenvalues and corresponding eigenvectors of matrix M = [[4, 1], [2, 3]], and express M in diagonalized form M
= P D P⁻¹.
Solution / Step-by-Step Proof:
Step 1: Characteristic Equation det(M - λI) = 0:
det([[4 - λ, 1], [2, 3 - λ]]) = (4 - λ)(3 - λ) - 2 = λ² - 7λ + 12 - 2 = λ² - 7λ + 10 = 0
(λ - 5)(λ - 2) = 0 => Eigenvalues: λ₁ = 5, λ₂ = 2.

Step 2: Find Eigenvector for λ₁ = 5:


(M - 5I) v = [[-1, 1], [2, -2]] [x, y]ᵀ = 0 => -x + y = 0 => x = y.
Eigenvector v₁ = [1, 1]ᵀ.

Step 3: Find Eigenvector for λ₂ = 2:


(M - 2I) v = [[2, 1], [2, 1]] [x, y]ᵀ = 0 => 2x + y = 0 => y = -2x.
Eigenvector v₂ = [1, -2]ᵀ.

Step 4: Form P, D, P⁻¹:


P = [[1, 1], [1, -2]], D = [[5, 0], [0, 2]]
det(P) = -2 - 1 = -3
P⁻¹ = (-1/3) [[-2, -1], [-1, 1]] = [[2/3, 1/3], [1/3, -1/3]]
Verification: M = P D P⁻¹ holds true.

Example 5 (Hard): Singular Value Decomposition (SVD) and PCA Application


Problem Statement:
Compute the SVD of matrix A = [[3, 0], [0, -2]] and explain how SVD is utilized for Dimensionality Reduction / Principal
Component Analysis (PCA).
Solution / Step-by-Step Proof:
Step 1: SVD formulation A = U Σ Vᵀ:
Aᵀ A = [[9, 0], [0, 4]]
Eigenvalues of Aᵀ A: λ₁ = 9, λ₂ = 4.
Singular values: σ₁ = √9 = 3, σ₂ = √4 = 2.
Diagonal matrix Σ = [[3, 0], [0, 2]].

Step 2: Right Singular Vectors (V):


Eigenvectors of Aᵀ A: v₁ = [1, 0]ᵀ, v₂ = [0, 1]ᵀ => V = [[1, 0], [0, 1]] = I₂.

Step 3: Left Singular Vectors (U):


u₁ = (1/σ₁) A v₁ = (1/3) [3, 0]ᵀ = [1, 0]ᵀ
u₂ = (1/σ₂) A v₂ = (1/2) [0, -2]ᵀ = [0, -1]ᵀ
U = [[1, 0], [0, -1]].
SVD Result: A = [[1, 0], [0, -1]] [[3, 0], [0, 2]] [[1, 0], [0, 1]]ᵀ.

PCA Connection:
In PCA, for zero-centered data matrix X (N × d), the right singular vectors V represent principal directions (eigenvectors
of covariance matrix XᵀX), and singular values σᵢ are proportional to variance explained along each principal axis.
Truncating SVD to top k components minimizes reconstruction MSE.

1.3 Tensor Products

Theory & Core Concepts


Tensors extend scalars, vectors, and matrices to arbitrary dimensions (orders/ranks), forming the mathematical
backbone of quantum entanglement state spaces and multidimensional deep learning representation tensors.

 Tensors & Order (Rank): Order/Rank 0 = Scalar (s ∈ ℝ); Order/Rank 1 = Vector (v ∈ ℝⁿ); Order/Rank 2 = Matrix (M ∈
ℝᵐⁿ); Order/Rank k = k-dimensional array (T ∈ ℝⁿ¹×ⁿ²×...×ⁿᵏ).
 Tensor Product (Kronecker Product): The Kronecker product A ⊗B for A (m × n) and B (p × q) produces a block
matrix of size (mp × nq):
A ⊗B = [[a₁₁B, a₁₂B, ...], [a₂₁B, a₂₂B, ...]]
 Mathematical Properties: Fundamental properties of Tensor Products:
• Bilinearity: A ⊗(B + C) = (A ⊗B) + (A ⊗C)
• Mixed-Product Property: (A ⊗B)(C ⊗D) = (AC) ⊗(BD)
• Transpose & Inverse: (A ⊗B)ᵀ = Aᵀ ⊗Bᵀ, (A ⊗B)⁻¹ = A⁻¹ ⊗B⁻¹
• Determinant & Trace: tr(A ⊗B) = tr(A) tr(B)
 Applications: Primary Applications across Modern Disciplines:
• Quantum Computing: Multi-qubit joint state spaces |Ψ⟩ = |q₁⟩ ⊗|q₂⟩ . Tensor product expands single qubit 2D
state space to 2ⁿ-dimensional Hilbert space.
• Deep Learning: CNN feature maps, multidimensional gradient propagation, attention mechanisms in
Transformers.
• Multilinear Algebra: Higher-order singular value decomposition (HOSVD), tensor factorization.
 Basic Operations: Tensor Operations:
• Contraction: Summing over repeated indices (generalizing matrix trace and dot product).
• Reshaping: Re-arranging tensor dimensions without altering element memory order.
• Broadcasting: Automatically expanding smaller tensor dimensions during element-wise operations.

Graded Worked Examples (Easy to Hard)


Example 1 (Easy): Vector Kronecker Product (Quantum Qubit State Construction)
Problem Statement:
In quantum computing, single qubit states |0⟩ = [1, 0]ᵀ and |1⟩ = [0, 1]ᵀ form the computational basis. Compute the joint
state vector for a 2-qubit system in state |01⟩ = |0⟩ ⊗|1⟩ .
Solution / Step-by-Step Proof:
State |0⟩ = [1, 0]ᵀ, State |1⟩ = [0, 1]ᵀ.

Compute Tensor Product |0⟩ ⊗|1⟩ :


|0⟩ ⊗|1⟩ = 1 · [0, 1]ᵀ
0 · [0, 1]ᵀ
= [ 1·0, 1·1, 0·0, 0·1 ]ᵀ = [0, 1, 0, 0]ᵀ.

Interpretation: The resulting 4D vector represents the quantum state |01⟩ in standard basis {|00⟩ , |01⟩ , |10⟩ , |11⟩ }.

Example 2 (Easy-Medium): Matrix Kronecker Product (Quantum Gate Operations)


Problem Statement:
Given the Pauli-X gate X = [[0, 1], [1, 0]] and Identity gate I = [[1, 0], [0, 1]], compute the composite 2-qubit operator X ⊗
I.
Solution / Step-by-Step Proof:
X ⊗ I = [[ 0·I, 1·I ],
[ 1·I, 0·I ]]

Substituting matrix I = [[1, 0], [0, 1]]:


X ⊗I = [ 0 0 1 0 ]
[0 0 0 1]
[1 0 0 0]
[0 1 0 0]

Interpretation: This 4×4 operator applies the Pauli-X (NOT) gate to the first qubit while leaving the second qubit
unchanged.

Example 3 (Medium): Tensor Properties - Trace and Mixed-Product Property


Problem Statement:
Given A = [[1, 2], [3, 4]] and B = [[2, -1], [0, 3]]:
a) Compute tr(A), tr(B), and verify tr(A ⊗B) = tr(A) tr(B).
b) Verify the mixed-product property (A ⊗I)(I ⊗B) = A ⊗B.
Solution / Step-by-Step Proof:
a) Trace Verification:
tr(A) = 1 + 4 = 5
tr(B) = 2 + 3 = 5
tr(A) tr(B) = 5 × 5 = 25.

Now compute A ⊗ B:
A ⊗B = [[ 1·B, 2·B ],
[ 3·B, 4·B ]]
Diagonal elements of A ⊗B are:
(A ⊗B)₁₁ = 1·b₁₁ = 2
(A ⊗B)₂₂ = 1·b₂₂ = 3
(A ⊗B)₃₃ = 4·b₁₁ = 8
(A ⊗B)₄₄ = 4·b₂₂ = 12
tr(A ⊗B) = 2 + 3 + 8 + 12 = 25. (Verified!)

b) Mixed Product Property:


(A ⊗I)(I ⊗B) = (A·I) ⊗(I·B) = A ⊗B. (Verified!)
Example 4 (Medium-Hard): Deep Learning Tensor Reshaping & Contraction
Problem Statement:
A batch of image features in deep learning is represented as a 3rd-order tensor T ∈ ℝ²ˣ³ˣ⁴ (Batch=2, Channels=3,
Features=4).
a) Compute total elements in T.
b) Perform tensor contraction over the feature dimension (index 3) with weight vector w ∈ ℝ⁴.
c) Reshape T into a 2D matrix M for matrix multiplication in dense layers.
Solution / Step-by-Step Proof:
a) Total elements: N = 2 × 3 × 4 = 24 elements.

b) Tensor Contraction:
Contracting T_ijk with w_k along dimension k=1..4 yields output tensor Y ∈ ℝ²ˣ³:
Y_ij = ∑ₖ₌₁⁴ T_ijk · w_k
This reduces tensor order from 3 to 2 (matrix of dimensions 2 × 3).

c) Reshaping for Linear Layers:


Flatten spatial/channel dimensions: Combine (Channels=3, Features=4) into single feature vector of length 12.
New shape M ∈ ℝ²ˣ¹² where row 1 contains all 12 feature elements of batch item 1, and row 2 contains batch item 2.

Example 5 (Hard): Bell State Entanglement & Tensor Rank in Quantum Systems
Problem Statement:
Consider the Bell state qubit system |Ψ⁺⟩ = (1/√2) [ |00⟩ + |11⟩ ] = [ 1/√2, 0, 0, 1/√2 ]ᵀ.
Show that |Ψ⁺⟩ CANNOT be written as a simple tensor product |ϕa⟩ ⊗|ϕb⟩ of two single-qubit states (i.e., prove that
the Bell state is quantum entangled).
Solution / Step-by-Step Proof:
Let single-qubit states be |ϕa⟩ = [a₁, a₂]ᵀ and |ϕb⟩ = [b₁, b₂]ᵀ.
Their tensor product is:
|ϕa⟩ ⊗|ϕb⟩ = [ a₁b₁, a₁b₂, a₂b₁, a₂b₂ ]ᵀ.

Set this equal to state vector |Ψ⁺⟩ = [ 1/√2, 0, 0, 1/√2 ]ᵀ:


1) a₁b₁ = 1/√2 => a₁ ≠ 0 and b₁ ≠ 0
2) a₁b₂ = 0 => since a₁ ≠ 0, b₂ MUST be 0
3) a₂b₁ = 0 => since b₁ ≠ 0, a₂ MUST be 0
4) a₂b₂ = 1/√2 => substituting a₂=0 and b₂=0 yields (0)(0) = 0 ≠ 1/√2!

Contradiction:
No choice of single-qubit states |ϕa⟩ and |ϕb⟩ can satisfy the set of equations.
Thus, |Ψ⁺⟩ has Tensor Rank > 1 and CANNOT be decomposed into separable state tensor product. This rigorously proves
quantum entanglement!

You might also like