4.210 min read

Defining Matrix-Matrix Multiplication

The matrix product $C = AB$ is defined so that $C$ represents the composition of the linear maps represented by $A$ and $B$ . Specifically: $C\mathbf{v} = A(B\mathbf{v})$ for all $\mathbf{v}$ .

This definition immediately gives us the size rule: if $A$ is $m \times n$ and $B$ is $n \times p$ , then $C = AB$ is $m \times p$ . The inner dimension $n$ is "consumed" — it represents the intermediate space where $B$ outputs and $A$ inputs.

The product is only defined when the number of columns of $A$ equals the number of rows of $B$ . This is not a convention — it's forced by the composition definition.

Formal View

Definition 4.2 — Matrix Product

For

A \in \mathbb{R}^{m \times n}

and

B \in \mathbb{R}^{n \times p}

, the matrix product

C = AB \in \mathbb{R}^{m \times p}

is the unique matrix satisfying

C\mathbf{v} = A(B\mathbf{v})

for all

\mathbf{v} \in \mathbb{R}^p

Why This Matters

Understanding that AB = "do B first, then A" is the key to using matrix multiplication correctly.

In deep learning, $AB$ where $B$ encodes features and $A$ classifies: compose the two operations
3D transformation matrices: $\text{Rotate} \cdot \text{Scale}$ applies scale first, then rotation

Learning Resources

Matrix multiplication

3Blue1Brown

Why matrix multiplication is defined the way it is.

10 min

Quiz

Question 1

Can we compute $AB$ if $A$ is $3 \times 4$ and $B$ is $3 \times 2$ ?

Common Mistakes

Checking the wrong dimensions — columns of $A$ must match rows of $B$ , not rows of $A$ with rows of $B$ .