17.88 min read

Proving Spectral Decomposition

We just showed: if the constrained quadratic problem $\min \frac{1}{2}\mathbf{x}^t A \mathbf{x}$ s.t. $\|\mathbf{x}\|^2 = 2$ has a minimum, then that minimum is an eigenvector of $A$ . But does the minimum exist?

Yes! The constraint set $\{\mathbf{x} : \frac{1}{2}\mathbf{x}^t \mathbf{x} = 1\}$ is a sphere — a compact set. By the extreme value theorem, any continuous function on a compact set attains its minimum. So the minimum exists, and Lagrange guarantees it is an eigenvector $\mathbf{u}_1$ with eigenvalue $\lambda_1$ .

This proves symmetric matrices have at least one eigenvalue. To get more: restrict to the orthogonal complement $\mathbf{u}_1^\perp$ and repeat. The restricted problem is again a quadratic on a compact sphere, so it has a minimum, giving a second eigenvector $\mathbf{u}_2 \perp \mathbf{u}_1$ .

Repeating $n$ times gives an orthonormal set $\{\mathbf{u}_1, \ldots, \mathbf{u}_n\}$ of eigenvectors with eigenvalues $\lambda_1, \ldots, \lambda_n$ . Assembling as columns of $U$ : $A = U \Lambda U^t$ — the spectral decomposition. Lagrange multipliers proved the whole theorem!

Formal View

Theorem 17.5 — Spectral Decomposition (Existence Proof)

Every symmetric

n \times n

matrix

A

has

n

real eigenvalues

\lambda_1, \ldots, \lambda_n

with corresponding orthonormal eigenvectors

\mathbf{u}_1, \ldots, \mathbf{u}_n

, giving the spectral decomposition

A = U \Lambda U^t

where

U = [\mathbf{u}_1 | \cdots | \mathbf{u}_n]

is orthogonal and

\Lambda = \text{diag}(\lambda_1, \ldots, \lambda_n)

Proof sketch: (1) Compact constraint set → minimum exists (EVT). (2) Lagrange → minimum is eigenvector. (3) Restrict to orthogonal complement and repeat. This is the cleanest proof of the spectral theorem.

Why This Matters

The spectral decomposition is one of the most important theorems in linear algebra, and this proof shows it follows naturally from calculus and optimization.

SVD of any matrix follows from spectral decomposition of $A^t A$
Quantum mechanics: observables are symmetric operators, their spectral decomposition gives measurement outcomes
PCA, LDA, kernel methods — all rely on spectral decomposition
Google PageRank and graph algorithms use eigenvectors of symmetric matrices

Learning Resources

Lagrange multipliers introduction

3Blue1Brown

Provides the intuition for the quadratic constrained problem that underlies this proof.

13 min

Eigenvectors and eigenvalues

3Blue1Brown

Provides context for eigenvalue/eigenvector meaning and the spectral decomposition.

14 min

Symmetric matrices and the spectral theorem

MIT OpenCourseWare

Full lecture on the spectral theorem for symmetric matrices.

49 min

Quiz

Question 1

Why is the extreme value theorem (EVT) needed in this proof?

Question 2

After finding first eigenvector $\mathbf{u}_1$ , how is the second obtained?

Question 3

Non-symmetric matrices always have real eigenvalues.

Question 4

The spectral decomposition $A = U\Lambda U^t$ encodes:

Question 5

What makes this proof of the spectral theorem elegant?

Common Mistakes

Forgetting the EVT step — the Lagrange theorem is only useful if a minimum is guaranteed to exist.
Thinking the orthogonal complement restriction is obvious — you need to verify that $A$ preserves $\mathbf{u}_1^\perp$ (which follows from symmetry).
Confusing the proof of existence (this section) with the computation of eigenvalues (characteristic polynomial).