Linear Algebra
12.108 min read

Local Minima

A point a\mathbf{a} is a local minimum of ff if f(a)≤f(x)f(\mathbf{a}) \leq f(\mathbf{x}) for all x\mathbf{x} in some neighborhood of a\mathbf{a}. In other words, f(a)f(\mathbf{a}) is the smallest value of ff in a small ball around a\mathbf{a}.

A strict local minimum requires f(a)<f(x)f(\mathbf{a}) < f(\mathbf{x}) for all x≠a\mathbf{x} \neq \mathbf{a} in the neighborhood. A global minimum satisfies f(a)≤f(x)f(\mathbf{a}) \leq f(\mathbf{x}) for all x\mathbf{x} in the entire domain.

The relationship is: global minimum ⇒\Rightarrow local minimum ⇒\Rightarrow critical point (for differentiable ff). Neither implication reverses: local minima need not be global, and critical points need not be local minima.

Formal View

Definition 12.5 — Local Minimum
A point a\mathbf{a} is a local minimum of ff if there exists δ>0\delta > 0 such that f(a)≤f(x)f(\mathbf{a}) \leq f(\mathbf{x}) for all x\mathbf{x} with ∥x−a∥<δ\|\mathbf{x}-\mathbf{a}\| < \delta. It is a strict local minimum if the inequality is strict for x≠a\mathbf{x} \neq \mathbf{a}.

Why This Matters

The distinction between local and global minima determines whether a found solution is optimal.

  • Non-convex loss function landscapes in deep learning have many local minima
  • Protein folding: the native structure is the global minimum of the free energy
  • Portfolio optimization: local minima of risk-adjusted return correspond to suboptimal portfolios

Quiz

Question 1

Which statement is correct?

Question 2

The function f(x)=x4−4x2f(x) = x^4 - 4x^2 has only one local minimum.

Common Mistakes

  • Assuming gradient descent finds the global minimum for non-convex functions.
  • Confusing local minimum (defined by neighborhood comparison) with critical point (defined by zero gradient).
  • Forgetting that differentiability is required for the implication local min ⇒\Rightarrow critical point.