Linear Algebra
12.26 min read

U-Derivative of a Linear Function

For a linear function f(x)=cTxf(\mathbf{x}) = \mathbf{c}^T \mathbf{x}, the directional derivative in any direction u\mathbf{u} is simply Duf(a)=cTuD_\mathbf{u} f(\mathbf{a}) = \mathbf{c}^T \mathbf{u}, regardless of the base point a\mathbf{a}.

This follows directly from the definition: f(a+tu)f(a)t=cT(a+tu)cTat=tcTut=cTu\frac{f(\mathbf{a}+t\mathbf{u}) - f(\mathbf{a})}{t} = \frac{\mathbf{c}^T(\mathbf{a}+t\mathbf{u}) - \mathbf{c}^T\mathbf{a}}{t} = \frac{t\mathbf{c}^T\mathbf{u}}{t} = \mathbf{c}^T\mathbf{u}.

For a linear function, the directional derivative equals the dot product of the coefficient vector with the direction. This is both a check of the formula and a preview of the general result: for any differentiable ff, Duf(a)=f(a)uD_\mathbf{u} f(\mathbf{a}) = \nabla f(\mathbf{a}) \cdot \mathbf{u}.

Formal View

Theorem 12.1 — Directional Derivative of a Linear Function
For f(x)=cTxf(\mathbf{x}) = \mathbf{c}^T\mathbf{x}, the directional derivative in direction u\mathbf{u} at any point is Duf(a)=cTu=fuD_\mathbf{u} f(\mathbf{a}) = \mathbf{c}^T \mathbf{u} = \nabla f \cdot \mathbf{u}.

The result is constant (independent of a\mathbf{a}) because ff is linear and has a constant gradient.

Why This Matters

Understanding directional derivatives for linear functions provides the template for the general formula.

  • Sensitivity coefficients in linear models: effect of changing input by one unit in direction u\mathbf{u}
  • Verifying gradient formulas for linear functions
  • Foundation for deriving the gradient formula via the LLA

Quiz

Question 1

For f(x,y)=3x2yf(x,y) = 3x - 2y, the directional derivative in direction u=(1/2,1/2)\mathbf{u} = (1/\sqrt{2}, 1/\sqrt{2}) is:

Common Mistakes

  • Evaluating DufD_\mathbf{u} f at a specific point for a linear function — the result is constant for linear functions.
  • Using non-normalized direction vectors without adjusting the formula.