9.158 min read

Eckart-Young Theorem

The Eckart-Young Theorem states: among all matrices of rank at most $k$ , the truncated SVD $A_k$ is closest to $A$ in both the Frobenius norm and the spectral norm. No other rank- $k$ matrix achieves a smaller error in either norm.

Errors: $\|A - A_k\|_F^2 = \sigma_{k+1}^2 + \cdots + \sigma_r^2$ and $\|A - A_k\|_2 = \sigma_{k+1}$ . The truncated SVD is the universally optimal low-rank approximation.

Formal View

Theorem 9.6 (Eckart-Young) — Optimality of Truncated SVD

For any rank-

k

matrix

B

\|A - A_k\|_F \leq \|A - B\|_F

and

\|A - A_k\|_2 \leq \|A - B\|_2

. The errors are

\|A - A_k\|_F^2 = \sum_{i>k} \sigma_i^2

and

\|A - A_k\|_2 = \sigma_{k+1}

Interactive Visualization

Low-Rank Approximation

Why This Matters

Eckart-Young is the theoretical guarantee that SVD-based compression is provably optimal.

Netflix prize: SVD is the best rank- $k$ approximation to the rating matrix.
Eckart-Young guarantees SVD sensor selection is optimal for reconstruction.
Signal processing: truncated SVD provides the best rank- $k$ filter.

Learning Resources

Eckart-Young theorem

MIT OpenCourseWare

Proof and applications of the Eckart-Young theorem.

45 min

Best low-rank approximation

Steve Brunton

Intuitive explanation and applications of Eckart-Young.

20 min

Quiz

Question 1

Eckart-Young states that $A_k$ minimizes the Frobenius-norm error among all rank- $k$ matrices.

Question 2

For $A$ with singular values $5, 3, 2, 1$ , what is $\|A - A_2\|_F^2$ ?

Common Mistakes

Thinking Eckart-Young only applies to Frobenius norm — it holds for both Frobenius and spectral norms.
Confusing $\|A - A_k\|_F^2 = \sum_{i>k} \sigma_i^2$ with $\|A - A_k\|_2 = \sigma_{k+1}$ (spectral norm, not squared).