8.128 min read

The Spectral Decomposition Gives Eigenvalues

If we know the spectral decomposition $A = U\Lambda U^\top$ , the eigenvalues are simply the diagonal entries of $\Lambda$ , and the eigenvectors are the columns of $U$ . This is worth verifying directly: for the $i$ -th column $\mathbf{u}_i$ of $U$ , we compute $A\mathbf{u}_i = U\Lambda U^\top \mathbf{u}_i$ .

Since $U^\top \mathbf{u}_i = \mathbf{e}_i$ (the $i$ -th standard basis vector, because $U$ is orthogonal and its columns are orthonormal), we get $U\Lambda \mathbf{e}_i = U (\lambda_i \mathbf{e}_i) = \lambda_i U \mathbf{e}_i = \lambda_i \mathbf{u}_i$ . So $A\mathbf{u}_i = \lambda_i \mathbf{u}_i$ — confirming that each column of $U$ is an eigenvector with eigenvalue $\lambda_i$ .

This also shows the spectral decomposition can be written as a sum of outer products: $A = \sum_{i=1}^n \lambda_i \mathbf{u}_i \mathbf{u}_i^\top$ . Each term $\lambda_i \mathbf{u}_i \mathbf{u}_i^\top$ is a rank-1 matrix encoding the action along the $i$ -th principal axis.

Formal View

Theorem 8.3 — Outer Product Form of Spectral Decomposition

A = U\Lambda U^\top

with orthogonal

U = [\mathbf{u}_1 \cdots \mathbf{u}_n]

and

\Lambda = \operatorname{diag}(\lambda_1, \ldots, \lambda_n)

, then

A = \sum_{i=1}^n \lambda_i \mathbf{u}_i \mathbf{u}_i^\top.

Each term

\lambda_i \mathbf{u}_i \mathbf{u}_i^\top

is a rank-1 symmetric matrix.

The verification is: $A\mathbf{u}_j = \left(\sum_i \lambda_i \mathbf{u}_i \mathbf{u}_i^\top\right)\mathbf{u}_j = \sum_i \lambda_i (\mathbf{u}_i^\top \mathbf{u}_j) \mathbf{u}_i = \lambda_j \mathbf{u}_j$ using orthonormality $\mathbf{u}_i^\top \mathbf{u}_j = \delta_{ij}$ .

Why This Matters

The outer-product form of the spectral decomposition is the foundation of low-rank approximation and PCA.

Truncating the sum to $k$ terms gives the best rank- $k$ approximation to $A$ .
Image compression: storing only the largest $k$ eigenvalue-eigenvector pairs captures most of the structure.
Quantum mechanics: density matrices are written as sums of outer products weighted by probabilities.

Learning Resources

Spectral decomposition and outer products

MIT OpenCourseWare

Strang explains the outer product form and its uses in approximation.

45 min

Eigendecomposition

Steve Brunton

Intuitive explanation of eigendecomposition with practical examples.

20 min

Quiz

Question 1

In the outer product form $A = \sum_{i=1}^n \lambda_i \mathbf{u}_i \mathbf{u}_i^\top$ , what is the rank of each term $\lambda_i \mathbf{u}_i \mathbf{u}_i^\top$ ?

Question 2

The step $U^\top \mathbf{u}_i = \mathbf{e}_i$ uses the orthonormality of the columns of $U$ .

Common Mistakes

Confusing $\mathbf{u}_i \mathbf{u}_i^\top$ (an outer product, $n \times n$ matrix) with $\mathbf{u}_i^\top \mathbf{u}_i$ (an inner product, a scalar equaling 1).
Forgetting the $\lambda_i$ factor in front of each outer product.