Linear Algebra
12.178 min read

Global Minima

A global minimum of ff on a set DD is a point aD\mathbf{a}^* \in D such that f(a)f(x)f(\mathbf{a}^*) \leq f(\mathbf{x}) for all xD\mathbf{x} \in D. Finding the global minimum is generally much harder than finding local minima or critical points.

Every global minimum is a local minimum (and hence a critical point for differentiable ff on an open domain), but not vice versa. For convex functions on convex domains, every local minimum is global — this is why convex optimization problems are tractable.

For non-convex functions, the only guaranteed strategy is to find all critical points and compare their values, or to use global optimization algorithms (branch and bound, simulated annealing, evolutionary methods). In practice, many problems are solved by assuming the objective is approximately convex near the solution.

Formal View

Definition 12.7 — Global Minimum
A point aD\mathbf{a}^* \in D is a global minimum of f:DRf: D \to \mathbb{R} if f(a)f(x)f(\mathbf{a}^*) \leq f(\mathbf{x}) for all xD\mathbf{x} \in D.
Theorem 12.11 — Convex Functions Have Unique Global Minima
If f:DRf: D \to \mathbb{R} is strictly convex and DD is a convex set, then ff has at most one global minimum. Every local minimum is also the unique global minimum.

Why This Matters

Global optimization is the ultimate goal of most applied optimization problems.

  • Convex optimization (e.g., linear programming, SDP, ridge regression) guarantees global solutions
  • Combinatorial optimization (e.g., traveling salesman) seeks global minima over discrete spaces
  • Drug discovery and protein design require finding global energy minima

Quiz

Question 1

For a strictly convex function on Rn\mathbb{R}^n, gradient descent is guaranteed to find the unique global minimum.

Question 2

Which type of function guarantees that every local minimum is also a global minimum?

Common Mistakes

  • Assuming gradient descent finds a global minimum for non-convex objectives.
  • Forgetting to check the boundary of the domain when the domain is bounded.
  • Thinking that having a unique critical point guarantees a global minimum — the critical point could be a saddle point or local max.