Lecture 3: Multiplication and Inverse Matrices

Two topics:

Matrix multiplication: five ways to compute $AB$. They all give the same answer, and each one is useful.
Inverses: when $A^{-1}$ exists, and how to compute it with Gauss–Jordan elimination.

Part 1: Five ways to multiply $AB = C$

Shapes. If $A$ is $m \times n$ and $B$ is $n \times p$, then $C$ is $m \times p$. The number of columns of $A$ must equal the number of rows of $B$.

Way 1: One entry at a time (row · column)

Each entry of $C$ is a row of $A$ dotted with a column of $B$:

c_{ij} = (\text{row } i \text{ of } A) \cdot (\text{column } j \text{ of } B) = \sum_{k=1}^n a_{ik}\, b_{kj}

Strang's example is $c_{34} = a_{31}b_{14} + a_{32}b_{24} + \cdots$. This is the standard rule.

Way 2: One column at a time

Treat $B$ as $p$ columns sitting side by side, and multiply $A$ by each one:

C = A\,[\,b_1 \;\cdots\; b_p\,] = [\,Ab_1 \;\cdots\; Ab_p\,]

Takeaway: each column of $C$ is a combination of the columns of $A$. The entries of the matching column of $B$ say which combination.

Way 3: One row at a time

Each row of $C$ is the matching row of $A$ times the whole matrix $B$.

Takeaway: each row of $C$ is a combination of the rows of $B$.

Way 4: Columns times rows (sum of outer products)

A column times a row gives a full matrix. An $m \times 1$ column times a $1 \times p$ row is an $m \times p$ matrix. Example:

\begin{bmatrix} 2\\3\\4 \end{bmatrix}\begin{bmatrix} 1 & 6 \end{bmatrix} = \begin{bmatrix} 2 & 12 \\ 3 & 18 \\ 4 & 24 \end{bmatrix}

This is a very special matrix:

Every column is a multiple of $(2,3,4)$, so the column space is a single line.
Every row is a multiple of $(1,6)$, so the row space is a single line.

Strang previews the terms row space and column space here.

The rule: $AB$ equals the sum of (column $k$ of $A$) times (row $k$ of $B$):

AB = \sum_{k=1}^{n} (\text{col}_k A)(\text{row}_k B)

Example:

\begin{bmatrix} 2 & 7\\3 & 8\\4 & 9 \end{bmatrix}\begin{bmatrix} 1 & 6\\0 & 0 \end{bmatrix} = \begin{bmatrix} 2\\3\\4 \end{bmatrix}\begin{bmatrix} 1 & 6 \end{bmatrix} + \begin{bmatrix} 7\\8\\9 \end{bmatrix}\begin{bmatrix} 0 & 0 \end{bmatrix}

The second term is zero, so the product equals the matrix above.

Way 5: Block multiplication

Cut $A$ and $B$ into blocks whose sizes match. Then multiply block rows by block columns, exactly as if the blocks were numbers:

\begin{bmatrix} A_1 & A_2 \\ A_3 & A_4 \end{bmatrix}\begin{bmatrix} B_1 & B_2 \\ B_3 & B_4 \end{bmatrix} \;\Rightarrow\; \text{top-left block} = A_1B_1 + A_2B_3

Strang's point: all five ways perform the same multiplications, just grouped differently.

Part 2: Inverses of square matrices

Definition:

A^{-1}A = I = AA^{-1}

Left and right inverses agree. For square matrices, a left inverse is automatically a right inverse. This is not obvious to prove, but it is true. For rectangular matrices, left and right inverses differ, and that comes later in the course.

The key question about any square matrix: is it invertible?

The good matrices are called invertible or non-singular.

The singular case: example with no inverse

A = \begin{bmatrix} 1 & 3 \\ 2 & 6 \end{bmatrix}

Here are three reasons $A$ has no inverse:

Determinant. $\det A = 0$. Strang notes this is the reason students reach for first, even though determinants haven't been covered yet.
Column picture. Every product $AX$ has columns that are combinations of $A$'s columns. Both columns, $(1,2)$ and $(3,6)$, lie on the same line. So no product can produce the identity's column $(1,0)$.
The null-vector argument (Strang's favorite). There is a nonzero $x$ with $Ax = 0$. Take $x = (3, -1)$:
$3\begin{bmatrix}1\\2\end{bmatrix} - 1\begin{bmatrix}3\\6\end{bmatrix} = \begin{bmatrix}0\\0\end{bmatrix}$ If $A^{-1}$ existed, multiplying $Ax = 0$ by it would give $x = A^{-1}0 = 0$. That contradicts $x \neq 0$. Once $A$ sends a nonzero vector to zero, nothing can recover it.

Conclusion: $A$ is singular exactly when some combination of its columns gives the zero column, that is, when $Ax = 0$ has a nonzero solution.

Finding the inverse: Gauss–Jordan elimination

Reformulation. Read $AA^{-1} = I$ column by column:

A \cdot (\text{column } j \text{ of } A^{-1}) = (\text{column } j \text{ of } I)

Finding $A^{-1}$ means solving $n$ systems that share the same matrix $A$ but have different right-hand sides. Jordan's idea is to solve them all at once.

Example:

A = \begin{bmatrix} 1 & 3 \\ 2 & 7 \end{bmatrix}

Step 1. Augment with the identity:

\left[\begin{array}{cc|cc} 1 & 3 & 1 & 0 \\ 2 & 7 & 0 & 1 \end{array}\right]

Step 2. Eliminate downward: subtract 2 × row 1 from row 2.

\left[\begin{array}{cc|cc} 1 & 3 & 1 & 0 \\ 0 & 1 & -2 & 1 \end{array}\right]

Gauss would stop here, since the left side is now upper triangular.

Step 3. Jordan keeps going and eliminates upward: subtract 3 × row 2 from row 1.

\left[\begin{array}{cc|cc} 1 & 0 & 7 & -3 \\ 0 & 1 & -2 & 1 \end{array}\right]

Result:

A^{-1} = \begin{bmatrix} 7 & -3 \\ -2 & 1 \end{bmatrix}

Check:

\begin{bmatrix} 7 & -3 \\ -2 & 1 \end{bmatrix}\begin{bmatrix} 1 & 3 \\ 2 & 7 \end{bmatrix} = \begin{bmatrix} 7-6 & 21-21 \\ -2+2 & -6+7 \end{bmatrix} = I

Why Gauss–Jordan works

Each elimination step is a left multiplication by some $E_{ij}$. Combine all the steps into one matrix $E$. Applying it to the augmented matrix gives

E\,[\,A \mid I\,] = [\,EA \mid E\,]

Elimination is carried out until the left half becomes $I$. So $EA = I$, which means $E = A^{-1}$.

The right half started as $I$ and became $EI = E = A^{-1}$. In short, the identity attached on the right records every step and ends up as the inverse.

Big ideas from Lecture 3

Five views of $AB$:
- entries (row · column)
- columns of $C$ as combinations of columns of $A$
- rows of $C$ as combinations of rows of $B$
- a sum of column × row matrices
- blocks
A column times a row gives a matrix whose rows all lie on one line and whose columns all lie on one line. This is the simplest building block, later called a rank-1 matrix.
For square matrices, a left inverse is also a right inverse.
Singular: $A$ is singular if and only if $Ax = 0$ for some nonzero $x$. Then $A^{-1}$ cannot exist, since it would have to send 0 back to a nonzero vector.
Gauss–Jordan: row-reduce $[A \mid I]$ to $[I \mid A^{-1}]$. It works because the combined elimination matrix $E$ satisfies $EA = I$.