Linear Algebra
10.157 min read

The Derivative as a Function

If ff is differentiable at every point in its domain, we can define the derivative function f′(x)f'(x): for each xx, compute f′(x)f'(x) by the limit definition. The result is a new function that can itself be differentiated, giving the second derivative f′′(x)f''(x), and so on.

For example, if f(x)=x3f(x) = x^3, then f′(x)=3x2f'(x) = 3x^2 (a new function), and f′′(x)=6xf''(x) = 6x, and f′′′(x)=6f'''(x) = 6 (constant). The nn-th derivative is denoted f(n)(x)f^{(n)}(x).

Not all continuous functions have derivatives everywhere — the derivative function may itself fail to be continuous or may not exist at some points. This motivates the hierarchy of function classes in the next two sections.

Formal View

Definition 10.11 — Derivative Function
If ff is differentiable at every xx in its domain DD, the derivative function f′:D→Rf': D \to \mathbb{R} is defined by f′(x)=lim⁡h→0f(x+h)−f(x)hf'(x) = \lim_{h \to 0} \frac{f(x+h) - f(x)}{h}. Higher derivatives: f′′=(f′)′f'' = (f')' , f(n)=(f(n−1))′f^{(n)} = (f^{(n-1)})'.

Why This Matters

The derivative function enables second-order analysis (curvature, convexity) and the Taylor series expansion.

  • Newton's method uses both f′(x)f'(x) and f′′(x)f''(x) (or Jacobian and Hessian in multiple dimensions) for fast convergence.
  • The Hessian matrix Hij=∂2f∂xi∂xjH_{ij} = \frac{\partial^2 f}{\partial x_i \partial x_j} is the matrix of second-order partial derivatives.
  • Taylor series: f(x)≈f(x0)+f′(x0)(x−x0)+f′′(x0)2(x−x0)2+⋯f(x) \approx f(x_0) + f'(x_0)(x-x_0) + \frac{f''(x_0)}{2}(x-x_0)^2 + \cdots

Quiz

Question 1

If f(x)=x4f(x) = x^4, what is f′′(x)f''(x)?

Question 2

The derivative of a differentiable function is always itself differentiable.

Common Mistakes

  • Thinking the derivative of a smooth function is always smooth — regularity can degrade with each differentiation.
  • Confusing the order of derivatives: f(2)f^{(2)} means differentiate twice, not square the derivative.