Lagrange Multipliers
Optimizing Under Equality Constraints
When optimization is restricted to a surface defined by equality constraints, gradient descent alone fails. Lagrange multipliers provide the systematic tool, revealing that constrained optima occur where gradients of the objective and constraints align — and even proving that symmetric matrices have eigenvalues.
Sections
Isovectors and the Tangent Space
8 min · 4 quiz questions
The Gradient as Normal Vector
6 min · 4 quiz questions
Constrained Optimization — The Setup
7 min · 4 quiz questions
The Lagrange Condition — One Constraint
9 min · 5 quiz questions · Interactive diagram
Solving with Lagrange — A Worked Example
9 min · 5 quiz questions
Multiple Equality Constraints
8 min · 4 quiz questions
Lagrange Applied to Quadratics — Eigenvalues Appear
9 min · 5 quiz questions
Proving Spectral Decomposition
8 min · 5 quiz questions