Linear Algebra
14.168 min read

Non-Canonical Case: Output is a Scalar

Another non-canonical case: g:Rk→Rg: \mathbb{R}^k \to \mathbb{R} is scalar-valued (as always), but f:R→Rk\mathbf{f}: \mathbb{R} \to \mathbb{R}^k is a trajectory (scalar input). So h(t)=g(f(t))h(t) = g(\mathbf{f}(t)): a scalar function of a trajectory.

This is the "one underlying variable" case with scalar output (which we covered in section 14.10). The gradient formula is h′(t)=∇g(f(t))⋅f′(t)h'(t) = \nabla g(\mathbf{f}(t)) \cdot \mathbf{f}'(t).

Geometrically: h′(t)h'(t) is the rate of change of the scalar field gg along the trajectory f\mathbf{f}. This is the directional derivative of gg along the velocity vector f′(t)\mathbf{f}'(t). So h′(t)=Df′(t)g(f(t))h'(t) = D_{\mathbf{f}'(t)} g(\mathbf{f}(t)) (where the direction vector is f′(t)\mathbf{f}'(t), not necessarily unit).

Formal View

Theorem 14.10 — Rate of Change Along a Trajectory
For a differentiable scalar field g:Rk→Rg: \mathbb{R}^k \to \mathbb{R} and a differentiable trajectory f:R→Rk\mathbf{f}: \mathbb{R} \to \mathbb{R}^k:
ddt[g(f(t))]=∇g(f(t))⋅f′(t)\frac{d}{dt}[g(\mathbf{f}(t))] = \nabla g(\mathbf{f}(t)) \cdot \mathbf{f}'(t)
This is the rate of change of gg along the curve f\mathbf{f}.

Setting g=xig = x_i (coordinate function) gives d(xi∘f)/dt=ei⋅f′(t)=fi′(t)d(x_i \circ \mathbf{f})/dt = e_i \cdot \mathbf{f}'(t) = f_i'(t) — a sanity check.

Why This Matters

The rate of change of a scalar quantity along a trajectory is fundamental to physics and optimization.

  • Physics: rate of change of potential energy along a particle's path
  • Level set methods: when g(f(t))=cg(\mathbf{f}(t)) = c (constant), we get ∇g⋅f′=0\nabla g \cdot \mathbf{f}' = 0 — velocity is tangent to level set
  • Gradient flow: if f′(t)=−∇g(f(t))\mathbf{f}'(t) = -\nabla g(\mathbf{f}(t)), then dg/dt=−∥∇g∥2≤0d g/dt = -\|\nabla g\|^2 \leq 0 — gg always decreases

Quiz

Question 1

If g(f(t))=cg(\mathbf{f}(t)) = c (constant) for all tt, what can we conclude?

Question 2

For gradient flow f′(t)=−∇g(f(t))\mathbf{f}'(t) = -\nabla g(\mathbf{f}(t)), the scalar field gg decreases along the trajectory.

Common Mistakes

  • Forgetting to evaluate ∇g\nabla g at f(t)\mathbf{f}(t), not at tt.
  • Thinking the result is a vector — d[g(f(t))]/dtd[g(\mathbf{f}(t))]/dt is a scalar, since gg is scalar.
  • Confusing f′≠0\mathbf{f}' \neq \mathbf{0} with d(g∘f)/dt≠0d(g\circ\mathbf{f})/dt \neq 0 — the velocity can be nonzero while moving along a level set.