Linear Algebra
8.1410 min read

The Principal Form of a Quadratic

Given a quadratic form f(x)=x⊤Axf(\mathbf{x}) = \mathbf{x}^\top A \mathbf{x} with spectral decomposition A=UΛU⊤A = U\Lambda U^\top, we can simplify the formula by changing to principal-axis coordinates. Define xU=U⊤x\mathbf{x}_U = U^\top \mathbf{x} (the coordinates of x\mathbf{x} in the eigenvector basis). Then: f(x)=x⊤UΛU⊤x=(U⊤x)⊤Λ(U⊤x)=xU⊤ΛxU=∑i=1nλi(xU,i)2.f(\mathbf{x}) = \mathbf{x}^\top U\Lambda U^\top \mathbf{x} = (U^\top \mathbf{x})^\top \Lambda (U^\top \mathbf{x}) = \mathbf{x}_U^\top \Lambda \mathbf{x}_U = \sum_{i=1}^n \lambda_i (x_{U,i})^2.

In principal-axis coordinates, the quadratic form diagonalizes — it becomes a pure sum of squares with eigenvalue coefficients, and no cross terms. This is called the principal form. The shapes we described in Section 8.5 are exactly the shapes of this diagonal form: all positive λi\lambda_i gives a bowl, mixed signs give a saddle, etc.

The key idea: all quadratic forms are diagonal at heart — they just look complicated because we are in the wrong coordinate system.

Formal View

Theorem 8.5 — Principal Form (Diagonalization of Quadratics)
Let A=UΛU⊤A = U\Lambda U^\top be the spectral decomposition. Under the coordinate change x=UxU\mathbf{x} = U\mathbf{x}_U, the quadratic form becomes
f(x)=x⊤Ax=xU⊤ΛxU=∑i=1nλi(xU,i)2.f(\mathbf{x}) = \mathbf{x}^\top A \mathbf{x} = \mathbf{x}_U^\top \Lambda \mathbf{x}_U = \sum_{i=1}^n \lambda_i (x_{U,i})^2.
This is the principal form of the quadratic — a diagonal sum of squares with no cross terms.

Interactive Visualization

Diagonalization: A = PDP⁻¹

Why This Matters

Diagonalizing a quadratic form turns a complicated multi-variable problem into nn independent one-variable problems.

  • Principal component analysis: the principal coordinates are uncorrelated (no cross terms in the covariance).
  • Normal mode analysis: in the eigenvector basis, coupled oscillators become independent harmonic oscillators.
  • Convexity analysis: checking all eigenvalues of AA tells you whether ff is convex, concave, or neither.

Quiz

Question 1

After the coordinate change xU=U⊤x\mathbf{x}_U = U^\top \mathbf{x}, the quadratic form x⊤Ax\mathbf{x}^\top A \mathbf{x} becomes:

Question 2

In principal-axis coordinates, a quadratic form has no cross terms (terms involving xU,ixU,jx_{U,i} x_{U,j} for i≠ji \neq j).

Common Mistakes

  • Applying A=UΛU⊤A = U \Lambda U^\top to get x⊤Ax=x⊤Λx\mathbf{x}^\top A \mathbf{x} = \mathbf{x}^\top \Lambda \mathbf{x} — this is wrong! You need to substitute xU=U⊤x\mathbf{x}_U = U^\top \mathbf{x}, not just replace AA with Λ\Lambda.
  • Forgetting that the principal coordinates xU=U⊤x\mathbf{x}_U = U^\top \mathbf{x} are rotated versions of x\mathbf{x} — ∣xU∣=∣x∣|\mathbf{x}_U| = |\mathbf{x}| since UU is orthogonal.