Linear Algebra
17.49 min read

The Lagrange Condition — One Constraint

We showed that at a constrained minimum, Null(Dg(x0))⊆Null(Df(x0))\text{Null}(Dg(\mathbf{x}_0)) \subseteq \text{Null}(Df(\mathbf{x}_0)). In most situations (when Df(x0)≠0Df(\mathbf{x}_0) \neq \mathbf{0}), both null spaces have the same dimension n−1n-1, so they must be equal.

Two row vectors with the same null space must be scalar multiples of each other (basic linear algebra). So Df(x0)=λ Dg(x0)Df(\mathbf{x}_0) = \lambda \, Dg(\mathbf{x}_0) for some scalar λ\lambda.

Equivalently: ∇f(x0)=λ∇g(x0)\nabla f(\mathbf{x}_0) = \lambda \nabla g(\mathbf{x}_0). The gradients are parallel. Geometrically: the ff-isosurface and the constraint surface SS are tangent to each other at x0\mathbf{x}_0 — they touch and share a tangent plane.

Intuition: if you were hiking and had to stay on a contour of gg, you reach a highest or lowest ff-contour exactly when the ff-contours become tangent to your allowed path. At that point, you cannot move along gg without crossing ff-contours in both directions.

The scalar λ\lambda is called a Lagrange multiplier. We do not particularly care about its value — we just care that it exists.

Formal View

Theorem 17.2 — Lagrange Theorem (One Constraint)
Let f,g∈C1f, g \in C^1. Suppose x0\mathbf{x}_0 is a minimum of the constrained problem min⁡f\min f s.t. g(x)=kg(\mathbf{x}) = k, and Rank(Dg(x0))=1\text{Rank}(Dg(\mathbf{x}_0)) = 1. Then there exists a scalar λ\lambda (the Lagrange multiplier) such that:
λ Dg(x0)=Df(x0)equivalently:∇f(x0)=λ∇g(x0)\lambda \, Dg(\mathbf{x}_0) = Df(\mathbf{x}_0) \qquad \text{equivalently:} \qquad \nabla f(\mathbf{x}_0) = \lambda \nabla g(\mathbf{x}_0)

This is a necessary condition only — not every point satisfying it is a constrained minimum. Such points are called Lagrange points; constrained optima must be among them.

Interactive Visualization

Lagrange Multiplier Visualizer

Why This Matters

The Lagrange condition converts a constrained optimization problem into a system of equations — which can be solved to find all candidate optima.

  • Finding shortest path on a surface (geodesic problems)
  • Maximizing utility subject to a budget constraint in economics
  • Principal component analysis: finds the eigenvector of the covariance matrix
  • Proving symmetric matrices have eigenvalues (spectral theorem — next section)

Quiz

Question 1

The Lagrange condition ∇f=λ∇g\nabla f = \lambda \nabla g means:

Question 2

How many equations does the Lagrange condition ∇f(x)=λ∇g(x)\nabla f(\mathbf{x}) = \lambda \nabla g(\mathbf{x}) give (for x∈Rn\mathbf{x} \in \mathbb{R}^n)?

Question 3

Every point satisfying the Lagrange condition is a constrained minimum.

Question 4

Geometrically, the Lagrange condition at x0\mathbf{x}_0 means:

Question 5

Why does it not matter what value λ\lambda takes?

Common Mistakes

  • Solving ∇f=λ∇g\nabla f = \lambda \nabla g and reporting those points as the answer without checking them against the constraint g(x)=kg(\mathbf{x}) = k.
  • Treating λ\lambda as meaningful — it is just the proportionality constant, often discarded.
  • Forgetting the necessary vs. sufficient direction: the Lagrange condition is necessary for a minimum, not sufficient.