Linear Algebra
16.69 min read

The Second Derivative Test

We know that critical points are candidates for local minima or maxima, but not every critical point qualifies. The second derivative test uses the Hessian to classify critical points.

The key idea: near a critical point x0\mathbf{x}_0, the function looks like its quadratic approximation 12utHf(x0)u\frac{1}{2}\mathbf{u}^t Hf(\mathbf{x}_0)\mathbf{u} (the linear term vanishes since ∇f(x0)=0\nabla f(\mathbf{x}_0) = 0). The behavior of this quadratic is determined by the definiteness of the Hessian.

If Hf(x0)Hf(\mathbf{x}_0) is positive definite, the quadratic bowl opens upward — so nearby ff also rises in every direction. Local minimum! If it is negative definite, ff falls in every direction. Local maximum! If it is indefinite, ff rises in some directions and falls in others. Saddle point!

The tricky case: if HfHf is only semidefinite (has a zero eigenvalue), the test is inconclusive — the "flat" direction leaves room for higher-order effects to create a min, max, or saddle.

Formal View

Theorem 16.4 — Second Derivative Test
Let x0\mathbf{x}_0 be a critical point of ff (so ∇f(x0)=0\nabla f(\mathbf{x}_0) = \mathbf{0}) with f∈D2f \in D^2 at x0\mathbf{x}_0. Then: \n* If Hf(x0)Hf(\mathbf{x}_0) is positive definite → strict local minimum. \n* If Hf(x0)Hf(\mathbf{x}_0) is negative definite → strict local maximum. \n* If Hf(x0)Hf(\mathbf{x}_0) is indefinite → saddle point. \n* If Hf(x0)Hf(\mathbf{x}_0) is semidefinite → test is inconclusive.

Critical points where HfHf has a zero eigenvalue are called degenerate.

Interactive Visualization

Hessian Surface Explorer

Why This Matters

The second derivative test tells you whether you have found a true minimum, a maximum, or a saddle — without needing to examine the function globally.

  • Verifying that a critical point found by gradient descent is actually a minimum
  • Sharp vs. flat minima in deep learning — related to smallest eigenvalue of the Hessian
  • Economics: checking whether a Nash equilibrium is a local optimum
  • Engineering design: confirming optimal solutions satisfy second-order conditions

Quiz

Question 1

At a critical point with Hf=[5113]Hf = \begin{bmatrix} 5 & 1 \\ 1 & 3 \end{bmatrix}, what is the classification?

Question 2

At a critical point with Hf=[2552]Hf = \begin{bmatrix} 2 & 5 \\ 5 & 2 \end{bmatrix}, what is the classification?

Question 3

The second derivative test is inconclusive when:

Question 4

The second derivative test can be applied at any point, not just critical points.

Question 5

Why does HfHf positive definite imply a local minimum (intuitively)?

Common Mistakes

  • Applying the second derivative test at a non-critical point.
  • Concluding "local minimum" when HfHf is PSD (might be degenerate) — need PD for strict local min.
  • Forgetting that the test only gives *local* information — a local min might not be a global min.