This article explores three different interpretations of matrix products with the help of animations. Each of them provides a different perspective on the matrix product and can be useful in different contexts.

A row vector is shown as a horizontal rectangle and a column vector as a vertical rectangle. The scalar numbers are shown as circles.

A circle next to a rectangle represents the scalar multiplication of the number with the vector. The color of the circle and the rectangle are the same to indicate that they are related.

Finally, the outer product of a column vector and a row vector is shown as a vertical rectangle joined with a horizontal rectangle on the top left corner. This visual aid should make sense if you are familiar with the concept of outer products.

That's it! On to the visualizations now.

Scaling a vector by a scalar quantity defined as multiplying each of the vector's components by the scalar. Adding two vectors is defined as adding their corresponding components. This interpretation is quite useful when using matrices in geometry or any visual application. Eg taking a convex sum of n points in a 2D plane (or any higher-dimensional space).

Note the duality between the two interpretations. The first one is useful when you want to think of the matrix as a collection of columns and the second one is useful when you want to think of the matrix as a collection of rows.

This is covered in more detail in Geometry of weighted sums.

A dot product is a function that takes two vectors of equal length and returns a scalar quantity. It is defined as the sum of the products of the corresponding components of the two vectors.

One needs to have some understanding of the dot product to appreciate this interpretation. It is covered in more detail in The Dot Product of Two Vectors (paywalled).

This is a less common interpretation but is quite useful. An outer product is a function that takes two vectors of arbitrary lengths and returns a matrix. There are two ways to look at this matrix:

A matrix-matrix product can be seen as a sum of outer products of the columns of the first matrix with the rows of the second matrix. This interpretation is useful in studying the spectral properties of symmetric matrices. Eg the product of a symmetric matrix with a vector can be seen as a sum of projections of the vector onto the eigenvectors of the matrix (scaled by the corresponding eigenvalues).

This is covered in more detail in Symmetric Mv Product (paywalled).

I find this interpretation quite beautiful. Note that when the number of columns in the second matrix is one, it reduces to the weighted sum of columns interpretation. Similarly when the number of rows in the first matrix is one, it reduces to the weighted sum of rows interpretation.