9.68 min read

Spectral Decomposition from SVD

The spectral decomposition $K = AA^\top = U(\Sigma\Sigma^\top)U^\top$ directly gives $U$ and the eigenvalues $\sigma_i^2$ . Singular values are $\sigma_i = \sqrt{\lambda_i(K)}$ .

Once we have $U$ and $\sigma_i$ , we recover right singular vectors: $\mathbf{v}_i = \frac{1}{\sigma_i} A^\top \mathbf{u}_i$ for each nonzero $\sigma_i$ . This follows from $A = U\Sigma V^\top \implies A^\top \mathbf{u}_i = \sigma_i \mathbf{v}_i$ . So the full SVD is computable from the spectral decomposition of $K$ alone.

Formal View

Theorem 9.3 — Computing SVD from Covariance

Let

K = AA^\top = U\Lambda_K U^\top

(spectral decomposition). Then: -

\sigma_i = \sqrt{(\Lambda_K)_{ii}}

\mathbf{v}_i = \frac{1}{\sigma_i} A^\top \mathbf{u}_i

for each

\sigma_i > 0

- These give the full SVD

A = U\Sigma V^\top

Interactive Visualization

Eigenvector Explorer

Why This Matters

This two-step construction shows the SVD is just two spectral decompositions unified.

When $m \ll n$ , eigendecomposing the $m \times m$ matrix $K$ is cheaper than the $n \times n$ matrix $G$ .
Kernel PCA uses this path with a kernel matrix instead of $K$ .
Covariance-based SVD underlies the "economy" SVD used in practice.

Learning Resources

Computing SVD via eigendecomposition

MIT OpenCourseWare

Strang shows the relationship between SVD and spectral decompositions.

45 min

SVD algorithm overview

Steve Brunton

Practical computation of SVD via covariance and Gram matrices.

20 min

Quiz

Question 1

If an eigenvalue of $K = AA^\top$ is 16, what is the corresponding singular value?

Question 2

Given $U$ and singular values $\sigma_i$ , we compute $\mathbf{v}_i = \frac{1}{\sigma_i}A^\top \mathbf{u}_i$ .

Common Mistakes

Forgetting the $1/\sigma_i$ factor when computing right singular vectors.
Trying to compute $\mathbf{v}_i$ for zero singular values using this formula — it is undefined.