Lecture 10: The Four Fundamental Subspaces

Topics:

1. A correction to Lecture 9.
2. The four fundamental subspaces: their dimensions and how to find a basis for each.
3. A new kind of vector space, whose "vectors" are matrices.

1. Correction from Lecture 9

Strang opens by fixing the error noted in the Lecture 9 summary.

The claim. In Lecture 9 he said \((1,1,2)\), \((2,2,5)\), \((3,3,8)\) form a basis for \(\mathbb{R}^3\).

Why it's wrong. After class, a student pointed out that the matrix with these columns has two identical rows, \((1,2,3)\) and \((1,2,3)\). So the matrix is not invertible, and the columns are dependent.

The lesson. The rows tell you something about the columns. The row space is only 2-dimensional, so the rank is 2, so only 2 of the 3 columns can be independent. This "row rank = column rank" connection is the main conclusion of this lecture.

2. The four subspaces

For an \(m \times n\) matrix \(A\):

Why the row space is written \(C(A^\top)\). Strang prefers column vectors. Transposing \(A\) makes its rows "stand up" as columns, so the row space of \(A\) is the column space of \(A^\top\). No new mathematics is involved, only notation.

The picture:

• In \(\mathbb{R}^n\) sit the row space and the nullspace.
• In \(\mathbb{R}^m\) sit the column space and the left nullspace.

3. Dimensions

In \(\mathbb{R}^n\): \(\dim C(A^\top) + \dim N(A) = r + (n - r) = n\). This repeats the count of \(r\) pivot variables plus \(n - r\) free variables.

In \(\mathbb{R}^m\): \(\dim C(A) + \dim N(A^\top) = r + (m - r) = m\). This is the same rule applied to \(A^\top\), which is \(n \times m\) and so has \(m\) columns.

Key fact: the row space and the column space have the same dimension \(r\).

4. Worked example

Row reduction to \(R\)

1. Subtract row 1 from row 2 and from row 3. Row 2 becomes \((0,-1,-1,0)\) and row 3 becomes \((0,0,0,0)\).
2. Multiply row 2 by \(-1\). It becomes \((0,1,1,0)\).
3. Subtract 2 × row 2 from row 1. Row 1 becomes \((1,0,1,1)\).

The rank is \(r = 2\).

What row operations preserve

The column space changes. \(C(R) \neq C(A)\). For example, \((1,1,1)\) is a column of \(A\), so it is in \(C(A)\). It is not in \(C(R)\), because every column of \(R\) has third component 0.

The row space is preserved. \(C(R^\top) = C(A^\top)\).

• Each row operation replaces a row by a combination of rows, so the new rows stay inside the old row space.
• Reversing the steps rebuilds the rows of \(A\) from the rows of \(R\).
• So the two row spaces contain each other, which means they are equal.

Row space basis

The basis. The nonzero rows of \(R\):

Why it is a basis.

• These rows are in the row space.
• They are independent, because of the identity block \(I\).
• There are \(r\) of them, matching the dimension.

The cleanest basis. Strang calls this the "best" basis for the row space. It is the analog of the identity's columns as the best basis for \(\mathbb{R}^n\).

Use rows of \(R\), not rows of \(A\). The rows of \(A\) itself may be dependent, as they are here (row 1 equals row 3).

Left nullspace: why "left"

The definition. \(y\) is in the left nullspace when \(A^\top y = 0\).

Transposing the equation. Take the transpose of both sides:

What it means. A vector in \(N(A^\top)\) gives a combination of the rows of \(A\) that produces the zero row.

Finding \(N(A^\top)\) without transposing: track \(E\)

The idea. Row-reduce the augmented matrix \([A \mid I_m]\):

The matrix \(E\) records every row operation that took \(A\) to \(R\).

Comparison with Chapter 2. For a square invertible \(A\), we had \(R = I\), so \(E = A^{-1}\). For a rectangular \(A\) there is no inverse, but \(E\) still exists.

Applying the same steps to \(I\) gives:

Check. Row 3 of \(EA\) is \(-(\text{row 1 of } A) + (\text{row 3 of } A) = 0\). This row of \(E\) produced the zero row of \(R\).

Basis for \(N(A^\top)\). The last \(m - r\) rows of \(E\), here \(3 - 2 = 1\) row:

Summary for this example

The special solutions for \(N(A)\) come from \(R\). Each one is \(-F\) in the pivot positions, with a 1 in its own free position.

5. A new vector space: matrices as "vectors"

The space \(M\). Take \(M\) to be all \(3 \times 3\) matrices. It is a vector space:

• You can add two matrices.
• You can multiply a matrix by a scalar.
• The zero matrix plays the role of \(0\).
• All eight rules hold.

What is ignored. Matrix multiplication \(AB\) plays no role in the vector-space structure.

Subspaces of \(M\):

• all upper triangular matrices
• all symmetric matrices
• their intersection, which is all diagonal matrices. A symmetric matrix with zeros below the diagonal must also have zeros above it.

Dimension of the diagonal matrices \(D\). It is 3. One basis is three diagonal matrices, each with a single nonzero diagonal entry. Every diagonal matrix is a unique combination of them.

Coming next lecture: the dimensions of the upper triangular and symmetric subspaces.

Big ideas from Lecture 10

1. The four subspaces. \(C(A)\) and \(N(A^\top)\) live in \(\mathbb{R}^m\). \(C(A^\top)\) and \(N(A)\) live in \(\mathbb{R}^n\).
2. Dimensions. They are \(r\), \(m - r\), \(r\), and \(n - r\). Within each ambient space, the dimensions add to \(m\) or to \(n\).
3. Row rank = column rank. This explains the Lecture 9 error: equal rows force dependent columns.
4. What row operations do. They preserve the row space and the nullspace. They change the column space.
5. How to get each basis:
   • \(C(A)\): pivot columns of \(A\)
   • \(C(A^\top)\): nonzero rows of \(R\)
   • \(N(A)\): special solutions
   • \(N(A^\top)\): the bottom \(m - r\) rows of \(E\), from \([A \mid I] \to [R \mid E]\)
6. Vector spaces beyond \(\mathbb{R}^n\). Matrices can serve as "vectors". Subspaces include the symmetric, upper triangular, and diagonal matrices.

Side question: which variables can be free?

You're right that the choice is not unique, but not any set of \(n-r\) variables can be free. There is a precise condition.

What "free" actually means

A set of variables is free if you can assign them any values and the remaining variables are then determined uniquely.

Suppose you choose a set \(S\) of \(n - r\) variables to be free. Move their columns to the right-hand side:

The remaining \(r\) columns \(\{a_j : j \notin S\}\) must solve this system for every choice of the free values. That requires two things:

• Existence: a solution must exist for every right-hand side. So the remaining \(r\) columns must span \(C(A)\).
• Uniqueness: the solution must be unique. So the remaining \(r\) columns must be independent.

Together:

A set of \(n - r\) variables can be free exactly when the other \(r\) columns form a basis for \(C(A)\).

Example (the matrix from Lecture 10)

Check each choice of two free variables by asking whether the other two columns are independent:

Why \(x_2, x_3\) fails. Set \(x_2 = 1\) and \(x_3 = 0\). You then need

A different valid choice gives the same nullspace

Take \(x_1, x_4\) as the free variables instead.

First special solution. Set \((x_1, x_4) = (1, 0)\) and solve \(x_2 a_2 + x_3 a_3 = -a_1\). This gives \((1, 1, -1, 0)\).

Second special solution. Set \((x_1, x_4) = (0, 1)\) and solve \(x_2 a_2 + x_3 a_3 = -a_4\). This gives \((0, 1, -1, 1)\).

Comparison with the standard choice. The pivot convention gave \(s_1 = (-1,-1,1,0)\) and \(s_2 = (-1,0,0,1)\). The new vectors are

What is fixed and what is a convention

Fixed:

• The number of free variables is always \(n - r\).
• The nullspace and its dimension do not depend on the choice.

Convention:

• Which variables are called free is a choice.

Why the pivot convention is used. Elimination scans the columns left to right. Each column is kept if it is independent of the earlier ones, and skipped otherwise. This produces the leftmost possible basis for \(C(A)\), and it gives \(R\) the clean form \([\,I \;\; F\,]\), which requires no column swaps. Any other valid choice is equally legitimate. It just corresponds to reordering the columns before eliminating.