Matrix Form Justification

The matrix multiplication form of the chain rule $J\mathbf{h} = J\mathbf{g} \cdot J\mathbf{f}$ can be understood through the LLA perspective: composing two affine maps gives another affine map.

The LLA of $\mathbf{f}$ near $\mathbf{a}$ is $\mathbf{f}(\mathbf{a}+\mathbf{h}) \approx \mathbf{f}(\mathbf{a}) + J\mathbf{f}(\mathbf{a})\mathbf{h}$ — an affine map in $\mathbf{h}$. The LLA of $\mathbf{g}$ near $\mathbf{f}(\mathbf{a})$ is $\mathbf{g}(\mathbf{f}(\mathbf{a})+\boldsymbol{\delta}) \approx \mathbf{g}(\mathbf{f}(\mathbf{a})) + J\mathbf{g}(\mathbf{f}(\mathbf{a}))\boldsymbol{\delta}$.

Substituting $\boldsymbol{\delta} = J\mathbf{f}(\mathbf{a})\mathbf{h}$: $\mathbf{g}(\mathbf{f}(\mathbf{a}+\mathbf{h})) \approx \mathbf{g}(\mathbf{f}(\mathbf{a})) + J\mathbf{g}(\mathbf{f}(\mathbf{a})) \cdot J\mathbf{f}(\mathbf{a}) \mathbf{h}$. The composed LLA has the linear part $J\mathbf{g} \cdot J\mathbf{f}$, which is the Jacobian of the composition.