12.57 min read
Sign of the U-Derivative
The sign of the directional derivative determines whether increases or decreases when we move from in direction :
- : is increasing in direction — moving in direction increases .
- : is decreasing in direction — moving in direction decreases .
- : is neither increasing nor decreasing in direction — moving tangentially along a level curve.
By the Cauchy-Schwarz inequality, . Equality holds when — so the gradient direction gives the maximum directional derivative.
Formal View
Theorem 12.4 — Sign of the Directional Derivative
Let be differentiable at with . For unit vector :
- iff the angle between and is less than
- iff is perpendicular to (tangent to level set)
- iff the angle exceeds
Why This Matters
Understanding when directional derivatives are positive/negative is the basis for all descent algorithms.
- Gradient descent: moving in the direction guarantees a decrease in (locally)
- Level set methods: directions tangent to level sets have zero directional derivative
- Constrained optimization: feasible descent directions must be tangent to constraints and have negative directional derivative
Quiz
Question 1
At a point where (unit vector), in which direction is maximized?
Question 2
If , then moving from a small step in direction will decrease .
Common Mistakes
- Confusing the gradient direction (steepest ascent) with the negative gradient direction (steepest descent).
- Assuming means has a critical point — it only means is not changing in direction .