14.77 min read

Multivariate Setup

The full multivariate chain rule handles the general case: $\mathbf{f}: \mathbb{R}^n \to \mathbb{R}^k$ and $\mathbf{g}: \mathbb{R}^k \to \mathbb{R}^m$ . The intermediate variables are $u_1, \ldots, u_k$ , and the underlying variables are $x_1, \ldots, x_n$ .

Each output $h_i = g_i(f_1(\mathbf{x}), \ldots, f_k(\mathbf{x}))$ depends on all intermediate variables, which in turn depend on all underlying variables. The total effect of $x_j$ on $h_i$ sums contributions through all intermediate variables $u_k$ .

This "sum of paths" interpretation is the essence of the Leibniz form: $\frac{\partial h_i}{\partial x_j} = \sum_{k=1}^K \frac{\partial g_i}{\partial u_k}\frac{\partial f_k}{\partial x_j}$ . Each term in the sum is the contribution through one intermediate variable.

Formal View

Remark 14.2 — Multivariate Chain Rule: Paths Interpretation

The

(i,j)

entry of

J\mathbf{h} = J\mathbf{g} \cdot J\mathbf{f}

is:

\frac{\partial h_i}{\partial x_j} = \sum_{k=1}^K \frac{\partial g_i}{\partial u_k}\frac{\partial f_k}{\partial x_j}

Each term is a "path" from

x_j

h_i

through intermediate variable

u_k

. This is literally the matrix product formula.

Interactive Visualization

Chain Rule Circuit Diagram

Why This Matters

The "sum over paths" interpretation of the chain rule generalizes naturally to computational graphs and backpropagation.

Computational graphs in deep learning: each path from input to output contributes to the gradient
Dynamic programming: the Bellman equation sums contributions through all next states
Chemical kinetics: rate equations sum contributions through all reaction pathways

Learning Resources

Multivariable Chain Rule

Khan Academy

The multivariate chain rule as a sum over intermediate variables.

9 min

Chain Rule and Computation Graphs

Steve Brunton

Connecting the multivariate chain rule to computation graphs.

20 min

Quiz

Question 1

In the Leibniz chain rule $\frac{\partial h_i}{\partial x_j} = \sum_k \frac{\partial g_i}{\partial u_k}\frac{\partial f_k}{\partial x_j}$ , how many terms are in the sum for $\mathbf{f}: \mathbb{R}^3 \to \mathbb{R}^5$ and $\mathbf{g}: \mathbb{R}^5 \to \mathbb{R}^2$ ?

Common Mistakes

Summing over the wrong index — the sum is over intermediate variables $k$ , not over inputs $j$ or outputs $i$ .
Forgetting to evaluate partial derivatives of $\mathbf{g}$ at $\mathbf{u} = \mathbf{f}(\mathbf{x})$ , not at $\mathbf{x}$ .