9.212 min read

The SVD Theorem

The Singular Value Decomposition Theorem states: for any real $m \times n$ matrix $A$ , there exist an $m \times m$ orthogonal matrix $U$ , an $m \times n$ matrix $\Sigma$ (with non-negative entries only on the main diagonal), and an $n \times n$ orthogonal matrix $V$ such that $A = U \Sigma V^\top$ .

The diagonal entries $\sigma_1 \geq \sigma_2 \geq \cdots \geq \sigma_{\min(m,n)} \geq 0$ of $\Sigma$ are called the singular values of $A$ . They are always real and non-negative, and they are uniquely determined by $A$ . The columns of $U$ are the left singular vectors and the columns of $V$ are the right singular vectors.

The rank of $A$ equals the number of nonzero singular values. MATLAB computes: [U, Sigma, V] = svd(A).

Formal View

Theorem 9.1 (SVD) — Singular Value Decomposition

For any real

m \times n

matrix

A

, there exist orthogonal matrices

U \in \mathbb{R}^{m \times m}

and

V \in \mathbb{R}^{n \times n}

, and a matrix

\Sigma \in \mathbb{R}^{m \times n}

with

\Sigma_{ii} = \sigma_i \geq 0

and

\Sigma_{ij} = 0

for

i \neq j

, such that

A = U \Sigma V^\top

. The values

\sigma_1 \geq \sigma_2 \geq \cdots \geq 0

are the singular values of

A

$\operatorname{rank}(A) = \#\{i : \sigma_i > 0\}$ . MATLAB: `[U, S, V] = svd(A)`.

Interactive Visualization

SVD as Three Transformations

Why This Matters

The SVD is the most general and stable matrix factorization — it works for any matrix and reveals the true geometric structure of any linear map.

Numerical rank determination: count singular values above a threshold.
Pseudoinverse: $A^+ = V \Sigma^+ U^\top$ where $\Sigma^+$ replaces nonzero $\sigma_i$ with $1/\sigma_i$ .
Condition number: $\kappa(A) = \sigma_1/\sigma_r$ measures numerical sensitivity.

Learning Resources

SVD theorem and computation

MIT OpenCourseWare

Gilbert Strang proves the SVD theorem and explains singular values.

45 min

The singular value decomposition

Steve Brunton

Overview of SVD structure, singular values, and MATLAB computation.

20 min

Quiz

Question 1

The singular values of a matrix are always real and non-negative.

Question 2

For a $3 \times 5$ matrix $A$ of rank 2, how many nonzero singular values does it have?

Common Mistakes

Confusing singular values with eigenvalues — singular values are always non-negative real numbers, while eigenvalues can be negative or complex.
Thinking $U$ and $V$ are the same matrix — $U$ is $m \times m$ and $V$ is $n \times n$ ; they act in different spaces.