Approximation Error

How good is the local linear approximation? The approximation error is $e(x) = f(x) - \tilde{f}(x) = f(x) - f(x_0) - f'(x_0)(x - x_0)$. For a differentiable function, this error shrinks faster than linearly as $x \to x_0$.

More precisely, the scaled error $\frac{e(x)}{x - x_0} = \frac{f(x) - f(x_0) - f'(x_0)(x-x_0)}{x - x_0} \to 0$ as $x \to x_0$. This is the precise statement that the linear approximation is "first-order accurate" — the error is $o(x - x_0)$ (little-oh notation, meaning it goes to zero faster than linearly).

This characterization is often taken as the definition of differentiability: $f$ is differentiable at $x_0$ with derivative $f'(x_0)$ iff there exists a number $m$ such that $\frac{f(x) - f(x_0) - m(x-x_0)}{x-x_0} \to 0$ as $x \to x_0$. This formulation generalizes to multiple dimensions cleanly.