Quadratic Case: Classifying Extrema

For the quadratic $f(\mathbf{x}) = \mathbf{x}^T A \mathbf{x} + \mathbf{b}^T \mathbf{x} + c$ with invertible symmetric $A$, the critical point $\mathbf{x}^* = -A^{-1}\mathbf{b}/2$ can be classified by the eigenvalues of $A$:

• All eigenvalues of $A$ positive ($A$ positive definite): $\mathbf{x}^*$ is a strict global minimum.
• All eigenvalues of $A$ negative ($A$ negative definite): $\mathbf{x}^*$ is a strict global maximum.
• Mixed signs (some positive, some negative): $\mathbf{x}^*$ is a saddle point.

This is because the value of $f$ around $\mathbf{x}^*$ is $f(\mathbf{x}^* + \mathbf{v}) = f(\mathbf{x}^*) + \mathbf{v}^T A \mathbf{v}$. If $A$ is positive definite, $\mathbf{v}^T A \mathbf{v} > 0$ for all $\mathbf{v} \neq \mathbf{0}$, so $f(\mathbf{x}^*+\mathbf{v}) > f(\mathbf{x}^*)$ — a minimum.