17.68 min read

Multiple Equality Constraints

What if we have two constraints $g_1(\mathbf{x}) = k_1$ and $g_2(\mathbf{x}) = k_2$ ? The feasible set is the intersection $S_1 \cap S_2$ — a "hyper-hyper-surface" of dimension $n-2$ (a curve when $n=3$ ).

The tangent space of this intersection is $T_{\mathbf{x}_0}(S_1 \cap S_2) = \text{Null}(Dg_1) \cap \text{Null}(Dg_2) = \text{Null}(D\mathbf{g})$ , where $D\mathbf{g}$ is the $2 \times n$ constraint Jacobian.

The necessary condition for a constrained minimum still requires $\text{Null}(D\mathbf{g}) \subseteq \text{Null}(Df)$ . By linear algebra, this now means $Df$ is a linear combination of the rows of $D\mathbf{g}$ : $Df(\mathbf{x}_0) = \lambda_1 Dg_1(\mathbf{x}_0) + \lambda_2 Dg_2(\mathbf{x}_0)$ .

In general with $m$ constraints, the Lagrange condition is $\boldsymbol{\lambda}^t D\mathbf{g}(\mathbf{x}_0) = Df(\mathbf{x}_0)$ where $\boldsymbol{\lambda} \in \mathbb{R}^m$ is a vector of Lagrange multipliers.

Formal View

Theorem 17.3 — Lagrange Theorem (Multiple Constraints)

Let

f, g_1, \ldots, g_m \in C^1

and

\mathbf{g}: \mathbb{R}^n \to \mathbb{R}^m

. Suppose

\mathbf{x}_0

is a constrained minimum (so

\mathbf{g}(\mathbf{x}_0) = \mathbf{k}

) and

\text{Rank}(D\mathbf{g}(\mathbf{x}_0)) = m

. Then there exists

\boldsymbol{\lambda} \in \mathbb{R}^m

such that:

\boldsymbol{\lambda}^t D\mathbf{g}(\mathbf{x}_0) = Df(\mathbf{x}_0)

Why This Matters

Real problems often have multiple constraints. The vector Lagrange condition handles all of them uniformly.

Mechanics: multiple conservation laws (energy, momentum, angular momentum) as constraints
Chemical reactions: multiple stoichiometric constraints
Machine learning: training with multiple fairness or budget constraints
Spectral decomposition proof (next): two constraints (quadratic form + norm)

Learning Resources

Lagrange multipliers with multiple constraints

Khan Academy

Extends Lagrange multipliers to handle two equality constraints.

14 min

Lagrange multipliers introduction

3Blue1Brown

Geometric picture that generalizes naturally to multiple constraints.

13 min

Quiz

Question 1

With two constraints in $\mathbb{R}^3$ , the feasible set has dimension:

Question 2

With two constraints $g_1, g_2$ , the Lagrange condition is:

Question 3

With $m$ constraints, there are $m$ Lagrange multipliers.

Question 4

For multiple constraints $\mathbf{g}(\mathbf{x}) = \mathbf{k}$ , the tangent space $T_{\mathbf{x}_0} C$ equals:

Common Mistakes

Using $\nabla f = \lambda \nabla g_1 = \lambda \nabla g_2$ (same $\lambda$ ) — each constraint gets its own multiplier.
Forgetting that the rank condition $\text{Rank}(D\mathbf{g}) = m$ is needed for the theorem to apply.