Local Linear Approximation for Vector-Valued Functions

Just as scalar functions are locally approximated by affine functions, vector-valued functions are locally approximated by affine maps (linear map plus constant). The LLA of $\mathbf{f}$ at $\mathbf{a}$ is:

$\mathbf{f}(\mathbf{a}+\mathbf{h}) \approx \mathbf{f}(\mathbf{a}) + J\mathbf{f}(\mathbf{a})\,\mathbf{h}$

This means: near the point $\mathbf{a}$, the vector-valued function $\mathbf{f}$ behaves like the linear (actually affine) function $\mathbf{y} \mapsto \mathbf{f}(\mathbf{a}) + J\mathbf{f}(\mathbf{a})(\mathbf{y}-\mathbf{a})$. Geometrically, the Jacobian matrix $J\mathbf{f}(\mathbf{a})$ is the best linear approximation to $\mathbf{f}$ near $\mathbf{a}$ — it captures how infinitesimal input changes map to infinitesimal output changes.

The approximation error $\mathbf{E}(\mathbf{h}) = \mathbf{f}(\mathbf{a}+\mathbf{h}) - \mathbf{f}(\mathbf{a}) - J\mathbf{f}(\mathbf{a})\mathbf{h}$ satisfies $\|\mathbf{E}(\mathbf{h})\|/\|\mathbf{h}\| \to 0$ when $\mathbf{f}$ is differentiable at $\mathbf{a}$.