Skip to main content

Orthogonality - Overview

Introduction​

Overview of orthogonal vectors, projections, least squares, and the Gram-Schmidt process.

Key Concepts​

Orthogonal Vectors and Subspaces​

  • Definition: v · w = 0
  • Orthogonal complements
  • Orthogonal bases
  • Pythagorean theorem

Projections​

  • Projection onto a line
  • Projection onto a subspace
  • Projection matrix P = A(AᵀA)⁻¹Aᵀ
  • Properties: P² = P, Pᵀ = P

Least Squares​

  • Solving Ax = b when no exact solution exists
  • Normal equations: AᵀAx̂ = Aᵀb
  • Applications to data fitting

Gram-Schmidt Process​

  • Orthogonalizing a basis
  • QR decomposition
  • Modified Gram-Schmidt for stability

Fast Fourier Transform​

  • Orthogonal basis of complex exponentials
  • FFT algorithm
  • Applications to signal processing

Applications​

  • Linear regression
  • Data fitting
  • Computer graphics
  • Signal processing

References​