11.118 min read

Linear Functions and the Jacobian

For a linear function $\mathbf{f}(\mathbf{x}) = A\mathbf{x}$ (where $A$ is an $m\times n$ matrix), the Jacobian is simply $A$ itself — constant everywhere.

This makes intuitive sense: the local linear approximation to a linear function is the function itself. The LLA $\mathbf{f}(\mathbf{a}+\mathbf{h}) \approx \mathbf{f}(\mathbf{a}) + J\mathbf{f}(\mathbf{a})\mathbf{h}$ becomes $A(\mathbf{a}+\mathbf{h}) = A\mathbf{a} + A\mathbf{h}$ , which is exact (zero error).

More generally, for an affine function $\mathbf{f}(\mathbf{x}) = A\mathbf{x} + \mathbf{b}$ , the Jacobian is again $A$ — the constant term $\mathbf{b}$ disappears because differentiation kills constants. This reflects the fact that the derivative measures rate of change, and constants do not change.

Formal View

Theorem 11.4 — Jacobian of a Linear Function

\mathbf{f}(\mathbf{x}) = A\mathbf{x} + \mathbf{b}

for

A \in \mathbb{R}^{m\times n}

and

\mathbf{b} \in \mathbb{R}^m

, then

J\mathbf{f}(\mathbf{x}) = A

for all

\mathbf{x}

In particular, the Jacobian of the identity function $\mathbf{f}(\mathbf{x}) = \mathbf{x}$ is the identity matrix $I_n$ .

Example 11.4 — Jacobian of a Linear Map

Let

\mathbf{f}(x_1, x_2, x_3) = (2x_1 - x_2, x_2 + 3x_3)

. The Jacobian is

J\mathbf{f} = \begin{bmatrix} 2 & -1 & 0 \\ 0 & 1 & 3 \end{bmatrix}

, a constant

2\times 3

matrix.

Interactive Visualization

Matrix-Vector Multiplication

Why This Matters

The fact that the Jacobian of a linear function is constant is why linear regression has a simple, closed-form solution.

Linear regression gradient: $\nabla_{\mathbf{w}}\|A\mathbf{w} - \mathbf{b}\|^2 = 2A^T(A\mathbf{w}-\mathbf{b})$ — a linear function of $\mathbf{w}$
Sensitivity analysis: for linear systems, the Jacobian gives exact (not approximate) sensitivities
Change of coordinates: the Jacobian of a linear change of coordinates is the transformation matrix itself

Learning Resources

Derivative of Linear Functions

MIT OpenCourseWare

Why the derivative of a linear map is itself.

20 min

Jacobian Examples: Linear and Nonlinear Maps

Khan Academy

Side-by-side examples of Jacobians for linear and nonlinear functions.

12 min

Quiz

Question 1

For $f(\mathbf{x}) = \mathbf{c}^T \mathbf{x}$ (a linear functional), the Jacobian $Df$ is:

Question 2

Adding a constant vector to a function changes its Jacobian.

Common Mistakes

Thinking the Jacobian of an affine function includes the constant offset $\mathbf{b}$ — it does not.
For scalar functions, confusing the $1\times n$ Jacobian row vector with the $n\times 1$ gradient column vector.
Forgetting that linear function LLAs have zero error (not just small error).