Linear Algebra
12.57 min read

Sign of the U-Derivative

The sign of the directional derivative Duf(a)=f(a)uD_\mathbf{u} f(\mathbf{a}) = \nabla f(\mathbf{a}) \cdot \mathbf{u} determines whether ff increases or decreases when we move from a\mathbf{a} in direction u\mathbf{u}:

  • Duf>0D_\mathbf{u} f > 0: ff is increasing in direction u\mathbf{u} — moving in direction u\mathbf{u} increases ff.
  • Duf<0D_\mathbf{u} f < 0: ff is decreasing in direction u\mathbf{u} — moving in direction u\mathbf{u} decreases ff.
  • Duf=0D_\mathbf{u} f = 0: ff is neither increasing nor decreasing in direction u\mathbf{u} — moving tangentially along a level curve.

By the Cauchy-Schwarz inequality, f(a)uf(a)u=f(a)|\nabla f(\mathbf{a}) \cdot \mathbf{u}| \leq \|\nabla f(\mathbf{a})\|\|\mathbf{u}\| = \|\nabla f(\mathbf{a})\|. Equality holds when u=±f(a)/f(a)\mathbf{u} = \pm \nabla f(\mathbf{a})/\|\nabla f(\mathbf{a})\| — so the gradient direction gives the maximum directional derivative.

Formal View

Theorem 12.4 — Sign of the Directional Derivative
Let ff be differentiable at a\mathbf{a} with f(a)0\nabla f(\mathbf{a}) \neq \mathbf{0}. For unit vector u\mathbf{u}: - Duf(a)>0D_\mathbf{u} f(\mathbf{a}) > 0 iff the angle between u\mathbf{u} and f(a)\nabla f(\mathbf{a}) is less than 90°90° - Duf(a)=0D_\mathbf{u} f(\mathbf{a}) = 0 iff u\mathbf{u} is perpendicular to f(a)\nabla f(\mathbf{a}) (tangent to level set) - Duf(a)<0D_\mathbf{u} f(\mathbf{a}) < 0 iff the angle exceeds 90°90°

Why This Matters

Understanding when directional derivatives are positive/negative is the basis for all descent algorithms.

  • Gradient descent: moving in the direction f-\nabla f guarantees a decrease in ff (locally)
  • Level set methods: directions tangent to level sets have zero directional derivative
  • Constrained optimization: feasible descent directions must be tangent to constraints and have negative directional derivative

Quiz

Question 1

At a point where f(a)=(1,1)/2\nabla f(\mathbf{a}) = (1, 1)/\sqrt{2} (unit vector), in which direction is DufD_\mathbf{u} f maximized?

Question 2

If Duf(a)<0D_\mathbf{u} f(\mathbf{a}) < 0, then moving from a\mathbf{a} a small step in direction u\mathbf{u} will decrease ff.

Common Mistakes

  • Confusing the gradient direction (steepest ascent) with the negative gradient direction (steepest descent).
  • Assuming Duf=0D_\mathbf{u} f = 0 means ff has a critical point — it only means ff is not changing in direction u\mathbf{u}.