11.612 min read

Local Linear Approximation (Multivariate)

The single-variable local linear approximation $f(a+h) \approx f(a) + f'(a)h$ generalizes elegantly to multiple variables. Near a point $\mathbf{a}$ , a differentiable function $f: \mathbb{R}^n \to \mathbb{R}$ is approximated by a linear function:

$f(\mathbf{a} + \mathbf{h}) \approx f(\mathbf{a}) + \frac{\partial f}{\partial x_1}(\mathbf{a})\, h_1 + \cdots + \frac{\partial f}{\partial x_n}(\mathbf{a})\, h_n$

This can be written compactly as $f(\mathbf{a} + \mathbf{h}) \approx f(\mathbf{a}) + Df(\mathbf{a})\mathbf{h}$ , where $Df(\mathbf{a}) = \begin{bmatrix} \partial f/\partial x_1(\mathbf{a}) & \cdots & \partial f/\partial x_n(\mathbf{a}) \end{bmatrix}$ is the $1 \times n$ Jacobian matrix (row vector of partials). This linear map is the best linear approximation to $f$ near $\mathbf{a}$ .

Geometrically for $n=2$ : the graph $z = f(x,y)$ is a surface in $\mathbb{R}^3$ , and the local linear approximation gives the tangent plane at the point $(a_1, a_2, f(\mathbf{a}))$ . The equation of the tangent plane is $z = f(\mathbf{a}) + f_x(\mathbf{a})(x-a_1) + f_y(\mathbf{a})(y-a_2)$ .

Formal View

Definition 11.7 — Local Linear Approximation (Multivariate)

The local linear approximation (LLA) of

f: D \subseteq \mathbb{R}^n \to \mathbb{R}

\mathbf{a}

is the affine function

L_{\mathbf{a}}(\mathbf{x}) = f(\mathbf{a}) + Df(\mathbf{a})(\mathbf{x}-\mathbf{a})

where

Df(\mathbf{a}) = \begin{bmatrix} \partial_1 f(\mathbf{a}) & \cdots & \partial_n f(\mathbf{a})\end{bmatrix}

is the Jacobian row vector.

Example 11.2 — Tangent Plane

For

f(x,y) = x^2 + y^2

\mathbf{a} = (1,1)

f(\mathbf{a})=2

f_x(\mathbf{a})=2

f_y(\mathbf{a})=2

. The tangent plane is

z = 2 + 2(x-1) + 2(y-1) = 2x + 2y - 2

Why This Matters

The local linear approximation is the foundation of all first-order optimization methods and sensitivity analysis.

Gradient descent: moving in the direction that decreases the LLA most rapidly
Error propagation: estimating how small input errors translate to output errors
Newton's method in multiple variables uses the LLA to take linearized steps

Learning Resources

Tangent Planes and Linear Approximations

Khan Academy

How the tangent plane generalizes the tangent line to functions of two variables.

12 min

Local Linearization (Multivariable)

3Blue1Brown

Visual explanation of the best linear approximation in multiple variables.

14 min

Quiz

Question 1

For $f(x,y) = e^{x+y}$ at the point $(0,0)$ , the local linear approximation $L_{(0,0)}(x,y)$ is:

Question 2

The local linear approximation of $f$ at $\mathbf{a}$ equals $f(\mathbf{a})$ plus a dot product of the gradient and the displacement vector.

Common Mistakes

Forgetting the constant term $f(\mathbf{a})$ — the LLA is an affine function, not a linear one.
Using global approximation when the LLA is only accurate near $\mathbf{a}$ .
Confusing the LLA (a real-valued affine function) with the Jacobian matrix (a matrix).