Lecture 6: Column Space and Nullspace

Topics:

Review of subspaces, plus unions vs. intersections.
Column space \(C(A)\), which tells you which \(b\) make \(Ax = b\) solvable.
Nullspace \(N(A)\), which is all \(x\) that solve \(Ax = 0\).
Why the solutions of \(Ax = b\) with \(b \neq 0\) do not form a subspace.

1. Subspace review

Requirement. A subspace must be closed under linear combinations: if \(v, w\) are in it, so is \(cv + dw\).

Subspaces of \(\mathbb{R}^3\): all of \(\mathbb{R}^3\), any plane \(P\) through \(0\), any line \(L\) through \(0\), and \(\{0\}\).

Union vs. intersection

Let \(P\) be a plane and \(L\) a line, both through 0, with \(L\) not inside \(P\).

Set	Subspace?	Reason
\(P \cup L\) (in \(P\) or \(L\))	No	Add a vector from \(P\) to one from \(L\). The sum usually lies in neither.
\(P \cap L\) (in both)	Yes	Here it is just \(\{0\}\), which is a subspace.

General fact. For any subspaces \(S\) and \(T\), the intersection \(S \cap T\) is a subspace.

Proof. Take \(v, w \in S \cap T\).

\(v + w \in S\), because \(S\) is a subspace. Likewise \(v + w \in T\). So \(v + w \in S \cap T\).
\(cv \in S\) and \(cv \in T\) for the same reason. So \(cv \in S \cap T\).

The intersection is usually smaller, but it is always a subspace.

2. The column space \(C(A)\)

Example:

A = \begin{bmatrix} 1 & 1 & 2 \\ 2 & 1 & 3 \\ 3 & 1 & 4 \\ 4 & 1 & 5 \end{bmatrix} \quad (4 \times 3)

Definition. \(C(A)\) is the set of all linear combinations of the columns of \(A\).

Here the columns have 4 components, so \(C(A)\) is a subspace of \(\mathbb{R}^4\).
In general, an \(m \times n\) matrix has \(C(A) \subseteq \mathbb{R}^m\).

Does \(C(A)\) fill all of \(\mathbb{R}^4\)? No. Combinations of 3 vectors cannot fill a 4-dimensional space.

Connection to \(Ax = b\)

x_1\begin{bmatrix}1\\2\\3\\4\end{bmatrix} + x_2\begin{bmatrix}1\\1\\1\\1\end{bmatrix} + x_3\begin{bmatrix}2\\3\\4\\5\end{bmatrix} = \begin{bmatrix}b_1\\b_2\\b_3\\b_4\end{bmatrix}

This is 4 equations in 3 unknowns, so not every \(b\) has a solution. Some \(b\) do:

\(b\)	Solution \(x\)	Why
\((0,0,0,0)\)	\((0,0,0)\)	Always works
\((1,2,3,4)\)	\((1,0,0)\)	\(b\) is column 1
\((1,1,1,1)\)	\((0,1,0)\)	\(b\) is column 2

Strang's tip. Pick any \(x\) first, then compute \(Ax\). That \(Ax\) is automatically a good \(b\).

Key result: \(Ax = b\) is solvable exactly when \(b \in C(A)\).

\(C(A)\) is precisely the set of all products \(Ax\). If \(b\) is a combination of the columns, the weights in that combination give \(x\). If \(b\) is not a combination, no \(x\) exists.

How big is \(C(A)\) here?

Column 3 adds nothing. Column 3 equals column 1 plus column 2. It lies in the plane of the first two and contributes no new direction.

Result. \(C(A)\) is a 2-dimensional subspace (a plane) of \(\mathbb{R}^4\).

Which column to drop. You could drop column 1 instead, since it equals column 3 minus column 2. But the convention is to keep the earliest columns that are independent. Strang previews that these will be called pivot columns, here columns 1 and 2. The words "independent" and "dependent" are previewed but not yet defined.

3. The nullspace \(N(A)\)

Definition. \(N(A)\) is the set of all solutions \(x\) to \(Ax = 0\).

It contains \(x\)'s, not \(b\)'s.
So \(N(A) \subseteq \mathbb{R}^n\), where \(n\) is the number of columns (unknowns).

Subspace	Contains	Lives in (for \(m\times n\) \(A\))	Here
\(C(A)\)	right-hand sides \(b\)	\(\mathbb{R}^m\)	\(\mathbb{R}^4\)
\(N(A)\)	solutions \(x\) of \(Ax=0\)	\(\mathbb{R}^n\)	\(\mathbb{R}^3\)

Finding \(N(A)\) for the example

Always present: \(x = 0\).

One nonzero solution. Since \(\text{col}_1 + \text{col}_2 - \text{col}_3 = 0\), the vector \((1, 1, -1)\) works.

All solutions. Every multiple works:

N(A) = \left\{ c\begin{bmatrix}1\\1\\-1\end{bmatrix} : c \in \mathbb{R} \right\}

So \(N(A)\) is a line through the origin in \(\mathbb{R}^3\).

Why \(N(A)\) is always a subspace

Suppose \(Av = 0\) and \(Aw = 0\).

Sums: \(A(v + w) = Av + Aw = 0 + 0 = 0\). This uses the distributive law.
Multiples: \(A(cv) = c(Av) = c \cdot 0 = 0\).

Both requirements hold for any matrix \(A\).

4. Solutions of \(Ax = b\) with \(b \neq 0\) are not a subspace

Take \(b = (1,2,3,4)\).

Solutions exist. For example:

\((1, 0, 0)\)
\((0, -1, 1)\), because column 3 minus column 2 equals \((1,2,3,4)\)

But they do not form a subspace. The zero vector is not a solution, since \(A \cdot 0 = 0 \neq b\).

What the solution set looks like. Strang describes it as a line or plane that does not pass through the origin. For this example it is the line \((1,0,0) + c(1,1,-1)\): the nullspace line, shifted away from the origin. This form follows from the example; Strang develops it properly in later lectures.

5. Two ways to describe a subspace

Method	Example	How it works
Give vectors, take their combinations	Column space	Start from the columns and fill out all combinations
Give equations that must hold	Nullspace	Describe the \(x\)'s by the condition \(Ax = 0\), then work out what satisfies it

Both always produce subspaces. These are the two standard ways to construct one.

Next lecture: a systematic method, using elimination, to compute \(N(A)\) when the matrix is too big to solve by inspection.

Big ideas from Lecture 6

Intersection vs. union: the intersection of two subspaces is a subspace. The union is generally not.
\(C(A)\) consists of all combinations of the columns, lies in \(\mathbb{R}^m\), and \(Ax = b\) is solvable exactly when \(b \in C(A)\).
Redundant columns: a column that is a combination of earlier columns adds nothing to \(C(A)\).
\(N(A)\) consists of all solutions of \(Ax = 0\), lies in \(\mathbb{R}^n\), and is always a subspace.
\(Ax = b\) with \(b \neq 0\): the solution set is not a subspace, because it does not contain 0. It is a shifted copy of \(N(A)\).