9.510 min read

Gram and Covariance Matrices

Two symmetric PSD matrices arise naturally from any matrix $A$ : the Gram matrix $G = A^\top A$ (size $n \times n$ ) and the covariance matrix $K = AA^\top$ (size $m \times m$ ). Both are symmetric and PSD.

If $A = U\Sigma V^\top$ , then $K = AA^\top = U(\Sigma\Sigma^\top)U^\top$ — the spectral decomposition of $K$ , with eigenvalues $\sigma_i^2$ . Similarly, $G = A^\top A = V(\Sigma^\top\Sigma)V^\top$ , also with eigenvalues $\sigma_i^2$ . So $K$ and $G$ share the same nonzero eigenvalues (both equal $\sigma_i^2$ ), but have different sizes and different eigenvectors.

Formal View

Theorem 9.2 — Gram and Covariance via SVD

Let

A = U\Sigma V^\top

. Then: -

K = AA^\top = U(\Sigma\Sigma^\top)U^\top

— spectral decomposition of

K

G = A^\top A = V(\Sigma^\top\Sigma)V^\top

— spectral decomposition of

G

- The nonzero eigenvalues of both

K

and

G

are

\sigma_1^2 \geq \cdots \geq \sigma_r^2 > 0

$K$ and $G$ always share the same nonzero eigenvalues (the squared singular values of $A$ ), even though they may have different sizes.

Interactive Visualization

Transpose Visualizer

Why This Matters

Gram and covariance matrices bridge SVD to the spectral theorem — they let you compute SVD using eigendecomposition.

In statistics, the covariance matrix $K = \frac{1}{n-1}XX^\top$ is the foundation of PCA.
Kernel methods: the Gram matrix encodes pairwise similarities.
Structural mechanics: the stiffness matrix $K = B^\top E B$ is a Gram matrix.

Learning Resources

Gram matrix and covariance

MIT OpenCourseWare

Strang on how $A^\top A$ and $AA^\top$ relate to the SVD.

45 min

Covariance matrices and PCA

Steve Brunton

Connection between covariance matrices and principal component analysis.

16 min

Quiz

Question 1

The nonzero eigenvalues of $AA^\top$ and $A^\top A$ are always equal.

Question 2

If $A$ has singular values $3, 1, 0$ , what are the eigenvalues of $A^\top A$ ?

Common Mistakes

Thinking $AA^\top$ and $A^\top A$ have the same eigenvalues — they share the NONZERO eigenvalues, but the sizes differ.
Confusing eigenvectors of $K$ (columns of $U$ ) with those of $G$ (columns of $V$ ).