Skip to main content

Matrix Decompositions - Overview

Introduction​

Overview of SVD, positive definite matrices, and key matrix factorizations.

Key Concepts​

Singular Value Decomposition (SVD)​

  • A = UΣVᵀ for any matrix A
  • Geometric interpretation
  • Applications to data science and ML

Positive Definite Matrices​

  • Tests for positive definiteness
  • xᵀAx > 0 for all x ≠ 0
  • Connection to eigenvalues
  • Applications to optimization

Minimum Principles​

  • Rayleigh quotient
  • Variational characterization of eigenvalues

Matrix Factorizations Summary​

  • LU, QR, Eigenvalue, SVD
  • When to use each

Applications​

  • Principal Component Analysis (PCA)
  • Low-rank approximations
  • Optimization
  • Numerical stability

References​

  • Strang Chapter 6.2, 6.3, 6.4
  • Appendix C
  • Deep Dive