Skip to main content

Matrix Decompositions - Deep Dive

Mathematical Foundations​

Rigorous treatment of SVD and positive definite matrices.

Singular Value Decomposition​

Existence Theorem​

Theorem: Every m×n matrix A has a singular value decomposition A = UΣVᵀ.

Proof: (To be completed)

Properties​

  • Singular values are square roots of eigenvalues of AᵀA
  • Best low-rank approximation theorem

Geometric Interpretation​

(To be completed)

Positive Definite Matrices​

Characterizations​

Theorem: The following are equivalent for symmetric A:

  1. A is positive definite
  2. All eigenvalues are positive
  3. All pivots are positive
  4. xᵀAx > 0 for all x ≠ 0

Proof: (To be completed)

Applications​

(To be completed)

Minimum Principles​

Rayleigh Quotient​

Theorem: For symmetric A with eigenvalues λ₁ ≥ λ₂ ≥ ... ≥ λₙ:

min (xᵀAx)/(xᵀx) = λₙ max (xᵀAx)/(xᵀx) = λ₁

Proof: (To be completed)

Matrix Factorization Theory​

Comparison of Factorizations​

(To be completed)

Exercises​

(Advanced problems to be completed)