Triple Products and the Row Perspective

Given an $m \times n$ matrix $A$, a vector $\mathbf{x} \in \mathbb{R}^n$, and a vector $\mathbf{y} \in \mathbb{R}^m$, the expression $\mathbf{y}^t A \mathbf{x}$ is a product of three matrices: a $1 \times m$, an $m \times n$, and an $n \times 1$. The result is a $1 \times 1$ matrix — just a real number.

Expanding, $\mathbf{y}^t A \mathbf{x} = \sum_{i,j} y_i a_{ij} x_j$. Since transposing a $1 \times 1$ matrix does nothing, we also have $\mathbf{y}^t A \mathbf{x} = \mathbf{x}^t A^t \mathbf{y}$. This symmetry is surprisingly useful.

The expression $A^t \mathbf{y}$ also has a clean row interpretation: the $i$-th entry of $A^t \mathbf{y}$ is the dot product of the $i$-th column of $A$ with $\mathbf{y}$. This bridges the column and row perspectives of a linear map.