Singular Value Decomposition
The Universal Matrix Factorization
Master the SVD — the most powerful factorization in linear algebra. Understand how any matrix decomposes into rotations and scalings, how it generalizes the spectral theorem, and how it underpins PCA, low-rank approximation, and multidimensional scaling.
Sections
Motivation for SVD
8 min · 2 quiz questions
The SVD Theorem
12 min · 2 quiz questions · Interactive diagram
Freedom in the SVD
7 min · 2 quiz questions
Immediate Uses of SVD
8 min · 2 quiz questions
Gram and Covariance Matrices
10 min · 2 quiz questions · Interactive diagram
Spectral Decomposition from SVD
8 min · 2 quiz questions · Interactive diagram
Computing SVD via Covariance Matrix
8 min · 2 quiz questions · Interactive diagram
Computing SVD via Gram Matrix
8 min · 2 quiz questions · Interactive diagram
The Affine Reduction Problem
10 min · 2 quiz questions
Linearization by Centering
7 min · 2 quiz questions
The Linear Reduction Problem
8 min · 2 quiz questions
Matrix Formulation of Linear Reduction
8 min · 2 quiz questions
The SVD Solution
10 min · 2 quiz questions · Interactive diagram
Truncated SVD
8 min · 2 quiz questions · Interactive diagram
Eckart-Young Theorem
8 min · 2 quiz questions · Interactive diagram
Principal Components
10 min · 2 quiz questions · Interactive diagram
Classical PCA
10 min · 2 quiz questions
Dual PCA
8 min · 2 quiz questions
Visualizing Low-Dimensional Structure
8 min · 2 quiz questions · Interactive diagram
Principal Coordinates from Gram Matrix
8 min · 2 quiz questions · Interactive diagram
Multidimensional Scaling
10 min · 2 quiz questions