16.47 min read

Hessian of a Quadratic

Quadratic functions are the cleanest case. For $f(\mathbf{x}) = \frac{1}{2}\mathbf{x}^t M \mathbf{x}$ with $M$ symmetric, let us compute the Hessian directly.

Expanding: $f = \frac{1}{2}\sum_{k,l} m_{kl} x_k x_l$ . For off-diagonal terms $i \neq j$ : $\frac{\partial^2 f}{\partial x_i \partial x_j} = \frac{1}{2}[m_{ij} + m_{ij}] = m_{ij}$ (using symmetry of $M$ ). For diagonal: $\frac{\partial^2 f}{\partial x_i^2} = m_{ii}$ .

The stunning conclusion: for a quadratic $f(\mathbf{x}) = \frac{1}{2}\mathbf{x}^t M \mathbf{x}$ , the Hessian is constant and equal to $M$ everywhere: $Hf(\mathbf{x}_0) = M$ for all $\mathbf{x}_0$ .

This means we can recover the quadratic from its Hessian: $f(\mathbf{x}) = \frac{1}{2}\mathbf{x}^t [Hf] \mathbf{x}$ . Adding linear or constant terms to $f$ does not change the Hessian.

Formal View

Theorem 16.2 — Hessian of a Quadratic

For

f(\mathbf{x}) = \frac{1}{2}\mathbf{x}^t M \mathbf{x}

with

M

symmetric, the Hessian is constant:

Hf(\mathbf{x}_0) = M \quad \text{for all } \mathbf{x}_0

Adding any linear term $\mathbf{b}^t \mathbf{x}$ or constant $c$ does not change the Hessian — only the quadratic part contributes.

Why This Matters

Quadratics are the canonical second-order approximation to any smooth function. Understanding the quadratic case tells us how to interpret the Hessian at any point.

Any smooth function looks like a quadratic near a point — the Hessian is its defining matrix
Newton's method replaces the function with its local quadratic and minimizes that
Quadratic programming (QP) solves optimization over quadratic objectives — foundational in robotics and control
Understanding loss landscape curvature in neural networks via quadratic approximation

Learning Resources

The Hessian matrix

Khan Academy

Works through computing the Hessian for quadratic and general functions.

10 min

Second derivatives and the Hessian matrix

Steve Brunton

Discusses the quadratic case and its connection to optimization curvature.

14 min

Quiz

Question 1

For $f(\mathbf{x}) = \frac{1}{2}\mathbf{x}^t M \mathbf{x}$ with $M$ symmetric, $Hf(\mathbf{x}_0)$ equals:

Question 2

For $f(\mathbf{x}) = \frac{1}{2}\mathbf{x}^t M \mathbf{x} + \mathbf{b}^t \mathbf{x} + c$ , what is $Hf$ ?

Question 3

For a general smooth function $f$ , the Hessian $Hf(\mathbf{x}_0)$ is constant (same at every point).

Question 4

Given $Hf = M$ for a quadratic $f$ , how do we recover $f$ ?

Common Mistakes

Forgetting the $\frac{1}{2}$ when reconstructing $f$ from $Hf$ — writing $\mathbf{x}^t M \mathbf{x}$ instead of $\frac{1}{2}\mathbf{x}^t M \mathbf{x}$ .
Assuming the Hessian of a non-quadratic is constant — only quadratics have constant Hessians.
Forgetting to require $M$ symmetric when using this theorem.