Lecture 9: Independence, Basis, and Dimension

Topic. This lecture gives precise meanings to four words:

• independence
• spanning
• basis
• dimension

What each word applies to. Strang stresses this point:

• A set of vectors can be independent, can span a space, or can be a basis.
• A space has a dimension.
• A matrix has a rank.

1. A key fact to start: more unknowns than equations

Suppose \(A\) is \(m \times n\) with \(n > m\). Then \(Ax = 0\) has a nonzero solution.

Why:

1. Elimination produces at most \(m\) pivots, since each row holds at most one pivot.
2. There are \(n\) variables, so at least \(n - m \ge 1\) variables must be free.
3. Set a free variable to 1 and solve for the pivot variables. This gives a nonzero vector in \(N(A)\).

2. Independence

Definition. Vectors \(x_1, \dots, x_n\) are (linearly) independent if no combination of them gives the zero vector, except the zero combination:

They are dependent if some combination with not all \(c_i = 0\) gives \(0\).

Examples in \(\mathbb{R}^2\)

Three vectors in \(\mathbb{R}^2\). Put them as the columns of a \(2 \times 3\) matrix \(A\). This matrix has more unknowns than equations. By Section 1, some nonzero \(c\) satisfies \(Ac = 0\), and that \(c\) is a dependency among the columns. Strang's example uses columns that look like \((2,1)\), \((1,2)\), and \((2.5,-1)\).

Matrix test for independence

Put \(v_1, \dots, v_n\) in the columns of \(A\). Then \(Ac\) is exactly the combination \(c_1v_1 + \cdots + c_nv_n\).

Relation to earlier lectures: "dependent" is the official name for "some combination of the columns gives the zero column."

3. Spanning

Definition. Vectors \(v_1, \dots, v_\ell\) span a space if the space consists of all their combinations.

Example. The columns of \(A\) span \(C(A)\).

The smallest space. The span of a set of vectors is the smallest space containing them. Any space that contains the vectors must contain all their combinations.

Spanning says nothing about independence. A spanning set may or may not be independent.

4. Basis

Definition. A basis for a space is a sequence of vectors that is:

1. independent, and
2. spans the space.

In Strang's words, it has enough vectors and not too many. Giving a basis tells you everything about the subspace, because the subspace is exactly all combinations of the basis vectors.

Examples in \(\mathbb{R}^3\)

Standard basis. \((1,0,0)\), \((0,1,0)\), \((0,0,1)\). As columns they form \(I\), and \(N(I) = \{0\}\), so they are independent.

Two vectors that don't span. \((1,1,2)\) and \((2,2,5)\) are independent, but they don't span \(\mathbb{R}^3\). They are a basis for the plane they span.

Adding \((3,3,7)\) fails. It is the sum of the first two, so it lies in their plane and the set becomes dependent.

Adding \((3,3,8)\) also fails. Correction to the lecture: Strang offers \((3,3,8)\) as a third vector that should complete a basis. It does not.

• All three vectors, \((1,1,2)\), \((2,2,5)\), and \((3,3,8)\), have equal first and second components. So they all lie in the plane \(x_1 = x_2\).
• As the columns of a \(3\times3\) matrix, they give rows 1 and 2 equal to each other, so the matrix is singular.
• Explicitly: \((3,3,8) = -\tfrac{1}{1}(1,1,2)\cdot 1 + \ldots\) is not needed; the equal-rows argument already settles it. A vector with \(x_1 \neq x_2\), such as \((1,0,0)\), would complete a basis.

Strang's general method below is correct; only this particular numerical choice was wrong.

Test for a basis of \(\mathbb{R}^n\)

\(n\) vectors in \(\mathbb{R}^n\) form a basis exactly when the \(n \times n\) matrix with those columns is invertible.

Bases are not unique. The columns of any invertible matrix form a basis for \(\mathbb{R}^n\).

5. Dimension

Key fact: Every basis for a given space has the same number of vectors.

Definition. That number is the dimension of the space.

For \(\mathbb{R}^n\), the dimension is \(n\). Fewer than \(n\) vectors cannot span. More than \(n\) vectors cannot be independent.

Shortcut. If you know a space has dimension \(d\), then:

• any \(d\) independent vectors in it automatically span it, so they form a basis;
• any \(d\) vectors that span it are automatically independent.

Why the shortcut holds. If \(d\) independent vectors failed to span, you could add another independent vector. That would give \(d+1\) independent vectors in a space of dimension \(d\), which is impossible.

6. Worked example: \(C(A)\) and \(N(A)\)

The columns satisfy two relations: column 3 = column 1 + column 2, and column 4 = column 1.

Column space

Do the 4 columns span \(C(A)\)? Yes, by definition.

Are they independent? No. For example, \((-1,-1,1,0)\) is in \(N(A)\).

The natural basis is the pivot columns: columns 1 and 2. Columns 3 and 4 add nothing new.

Other valid bases:

• columns 1 and 3
• columns 2 and 3
• columns 2 and 4
• vectors not taken from the columns at all, such as \((2,2,2)\) and \((7,5,7)\). Both lie in \(C(A)\) and they are independent. Since \(\dim C(A) = 2\), they must form a basis.

Rank equals the dimension of the column space:

Nullspace

Free variables: \(x_3\) and \(x_4\). Each gives one special solution:

Reading them.

• \(s_1\) says column 3 minus column 1 minus column 2 equals 0.
• \(s_2\) says column 4 minus column 1 equals 0.

Each special solution records one way the columns are dependent.

Basis. The special solutions are independent and span \(N(A)\), so they form a basis for \(N(A)\).

Dimension of the nullspace:

Summary of definitions

Big ideas from Lecture 9

1. \(n > m\) forces dependence. If \(n > m\), then \(Ax = 0\) has nonzero solutions. So more than \(m\) vectors in \(\mathbb{R}^m\) are always dependent.
2. Columns are independent exactly when \(N(A) = \{0\}\), equivalently when \(r = n\).
3. Basis = independent + spanning. A basis is not unique, but every basis of a space has the same size, and that size is the dimension.
4. \(\dim C(A) = r\). The pivot columns form a basis for \(C(A)\).
5. \(\dim N(A) = n - r\). The special solutions form a basis for \(N(A)\).