3.210 min read

Matrix-Vector Multiplication

To multiply matrix $A$ (size $m \times n$ ) by vector $\mathbf{x}$ (in $\mathbb{R}^n$ ), the result $A\mathbf{x}$ is a vector in $\mathbb{R}^m$ . The dimensions must be compatible: the number of columns of $A$ must equal the length of $\mathbf{x}$ .

The column perspective: $A\mathbf{x} = x_1\mathbf{a}_1 + x_2\mathbf{a}_2 + \cdots + x_n\mathbf{a}_n$ — a linear combination of the columns of $A$ weighted by the entries of $\mathbf{x}$ . This is the geometric view.

The two-finger rule (entry-by-entry): The $i$ -th entry of $A\mathbf{x}$ is the dot product of row $i$ of $A$ with $\mathbf{x}$ : $(A\mathbf{x})_i = \sum_j a_{ij} x_j$ . This is the computational view.

Formal View

Definition 3.2 — Matrix-Vector Multiplication

For

A \in \mathbb{R}^{m \times n}

and

\mathbf{x} \in \mathbb{R}^n

A\mathbf{x} = \begin{pmatrix} \sum_j a_{1j}x_j \\ \vdots \\ \sum_j a_{mj}x_j \end{pmatrix} = x_1\mathbf{a}_1 + \cdots + x_n\mathbf{a}_n

where

\mathbf{a}_j

is the

j

-th column of

A

Interactive Visualization

Matrix-Vector Multiplication

Why This Matters

Matrix-vector multiplication is the core computational primitive of linear algebra and deep learning.

Neural network forward pass: each layer computes $\mathbf{y} = A\mathbf{x} + \mathbf{b}$
Image filtering: applying a kernel is matrix-vector multiplication (vectorized)
PageRank: Google's original algorithm computes repeated matrix-vector products

Learning Resources

Linear transformations and matrices

3Blue1Brown

Matrix-vector multiplication as a linear transformation.

10 min

Quiz

Question 1

The product $A\mathbf{x}$ for $A \in \mathbb{R}^{3 \times 2}$ and $\mathbf{x} \in \mathbb{R}^2$ lives in:

Common Mistakes

Forgetting that $A\mathbf{x}$ requires $n$ (columns of $A$ ) to equal the length of $\mathbf{x}$ .
Computing $A\mathbf{x}$ by rows when the column perspective is more geometric and informative.