Skip to main content

Section 2: Multivariate Distributions and Estimation

Overview​

Focus on joint distributions, the multivariate Gaussian, and the two major paradigms for parameter estimation: maximum likelihood and Bayesian inference.

Topics Covered​

Chapter 1: Joint, Marginal, and Conditional Distributions​

  • Joint distributions and the sum/product rules
  • Marginalization
  • Conditional distributions and conditioning

Chapter 2: The Multivariate Gaussian​

  • MVN definition and parameterization
  • Conditioning and marginalization formulas
  • Mahalanobis distance and affine transformations

Chapter 3: Maximum Likelihood Estimation​

  • MLE derivation and properties
  • MAP estimation
  • Fisher information and the Cramer-Rao bound

Chapter 4: Bayesian Inference​

  • Prior, posterior, and likelihood
  • Conjugate priors
  • Posterior predictive distributions
  • Beta-Binomial and Normal-Normal examples

Learning Objectives​

  • Compute marginal and conditional distributions from joint distributions
  • Apply the multivariate Gaussian conditioning and marginalization formulas
  • Derive maximum likelihood estimators for common distributions
  • Set up and solve Bayesian inference problems with conjugate priors