Linear Algebra
12.98 min read

Critical Points

A critical point (or stationary point) of ff is a point a\mathbf{a} where the gradient vanishes: ∇f(a)=0\nabla f(\mathbf{a}) = \mathbf{0}. Equivalently, all partial derivatives are zero at a\mathbf{a}: ∂f/∂xi(a)=0\partial f/\partial x_i(\mathbf{a}) = 0 for all ii.

At a critical point, every directional derivative is zero (Duf(a)=∇f(a)⋅u=0D_\mathbf{u} f(\mathbf{a}) = \nabla f(\mathbf{a}) \cdot \mathbf{u} = 0 for all u\mathbf{u}). The function has no first-order preference for any direction — it is "flat" at the critical point.

Critical points come in three varieties: local minima, local maxima, and saddle points. To distinguish them, we need second-order information (the Hessian matrix). A local minimum in particular requires a positive definite Hessian.

Formal View

Definition 12.4 — Critical Point
A point a∈Rn\mathbf{a} \in \mathbb{R}^n is a critical point of f:D⊆Rn→Rf: D \subseteq \mathbb{R}^n \to \mathbb{R} if ff is differentiable at a\mathbf{a} and ∇f(a)=0\nabla f(\mathbf{a}) = \mathbf{0}.

Critical points are also called stationary points or, in optimization contexts, first-order optimal points.

Why This Matters

Finding critical points is the first step in every calculus-based optimization method.

  • Setting ∇f=0\nabla f = 0 to derive normal equations, MLE conditions, and MAP estimators
  • Equilibrium analysis in dynamical systems
  • Phase transitions in physics: critical points of energy functions

Quiz

Question 1

A critical point of ff is a point where:

Question 2

Every local minimum of a differentiable function is a critical point.

Common Mistakes

  • Thinking every critical point is a local minimum — saddle points and local maxima are also critical.
  • Confusing f(a)=0f(\mathbf{a}) = 0 (function value is zero) with ∇f(a)=0\nabla f(\mathbf{a}) = \mathbf{0} (gradient is zero).
  • Setting individual partial derivatives to zero but forgetting to verify all of them are zero simultaneously.