9.148 min read

Truncated SVD

The truncated SVD keeps only the $k$ largest singular values: $A_k = U_k \Sigma_k V_k^\top = \sum_{i=1}^k \sigma_i \mathbf{u}_i \mathbf{v}_i^\top.$ It is the best rank- $k$ approximation of $A$ in the Frobenius norm. Error: $\|A - A_k\|_F^2 = \sigma_{k+1}^2 + \cdots + \sigma_r^2$ .

As $k$ increases, error decreases and $A_r = A$ (exactly) for $r = \text{rank}(A)$ . The storage cost of $A_k$ is $(m + n + 1)k$ numbers vs $mn$ for the full matrix — a saving when $k \ll \min(m,n)$ .

Formal View

Definition 9.4 — Truncated SVD

The rank-$k$ truncated SVD:

A_k = U_k \Sigma_k V_k^\top = \sum_{i=1}^k \sigma_i \mathbf{u}_i \mathbf{v}_i^\top

where

U_k, V_k

contain the first

k

singular vectors and

\Sigma_k = \operatorname{diag}(\sigma_1, \ldots, \sigma_k)

Interactive Visualization

Low-Rank Approximation

Why This Matters

Truncated SVD is the workhorse of data compression, dimensionality reduction, and noise filtering.

Image compression: store $(m + k + n)k$ numbers instead of $mn$ .
Latent semantic analysis: $k$ -truncated SVD finds topic structure in term-document matrices.
Denoising: small singular values capture noise; truncating cleans the signal.

Learning Resources

Truncated SVD and image compression

Steve Brunton

Visual demonstration of truncated SVD for image compression.

20 min

Low-rank approximation

MIT OpenCourseWare

Theory and applications of rank-$k$ approximations.

45 min

Quiz

Question 1

$A_k$ is the best rank- $k$ approximation in what sense?

Question 2

The rank of $A_k = U_k \Sigma_k V_k^\top$ is exactly $k$ (assuming all $k$ singular values are nonzero).

Common Mistakes

Thinking larger $k$ always better — it reduces error but increases storage and computation.
Confusing truncated (keeps largest) with deflated (removes largest) SVD.