Skip to main content

Orthogonality - Deep Dive

Mathematical Foundations​

Rigorous treatment of orthogonality with proofs and theory.

Orthogonal Subspaces​

Fundamental Theorem​

Theorem: The four fundamental subspaces come in orthogonal pairs:

  • C(A)⊥ = N(Aᵀ)
  • C(Aᵀ)⊥ = N(A)

Proof: (To be completed)

Projection Theory​

Projection onto Subspace​

Theorem: The projection of b onto C(A) is p = A(AᵀA)⁻¹Aᵀb

Proof: (To be completed)

Properties of Projection Matrices​

Theorem: P is a projection matrix if and only if P² = P and Pᵀ = P

Proof: (To be completed)

Least Squares​

Normal Equations​

Theorem: x̂ minimizes ||Ax - b||² if and only if Ax̂ = p where p is the projection of b onto C(A).

Proof: (To be completed)

Uniqueness​

Theorem: If A has full column rank, then AᵀA is invertible and x̂ is unique.

Proof: (To be completed)

Gram-Schmidt​

Classical Gram-Schmidt​

Algorithm: (To be completed)

Theorem: Gram-Schmidt produces an orthogonal basis

Proof: (To be completed)

QR Decomposition​

Theorem: Every matrix A with full column rank has a QR decomposition A = QR

Proof: (To be completed)

FFT Theory​

(To be completed)

Exercises​

(Advanced problems to be completed)