Ashutosh Patidar · notes

Linear Algebra (source: Gilbert Strang notes) · 8 / 37

Lecture 7: Solving \(Ax = 0\): Pivot Variables, Special Solutions

The OCW 2010 page for this lecture has no transcript, so this summary is based on the same 1999 lecture's transcript from the 18.06SC version on MIT Open Learning Library.

Goal: Turn the definition of the nullspace into an algorithm for computing it, for any matrix, including rectangular ones.

1. Example 1: a 3×4 matrix

\[A = \begin{bmatrix} 1 & 2 & 2 & 2 \\ 2 & 4 & 6 & 8 \\ 3 & 6 & 8 & 10 \end{bmatrix}\]

Before computing, Strang notes two dependencies he expects elimination to discover:

  • Column 2 is 2 × column 1.
  • Row 3 is row 1 plus row 2.

Elimination does not change the nullspace

Row operations change the equations into equivalent ones. So the solutions to \(Ax = 0\), \(Ux = 0\), and \(Rx = 0\) are identical.

Elimination does change the column space, so you have to keep track of which space is preserved.

The right-hand side stays 0 throughout, so it is not written.

Forward elimination to echelon form \(U\)

Column 1. The pivot is 1. Subtract 2 × row 1 from row 2, and 3 × row 1 from row 3:

\[\begin{bmatrix} 1 & 2 & 2 & 2 \\ 0 & 0 & 2 & 4 \\ 0 & 0 & 2 & 4 \end{bmatrix}\]

Column 2. The pivot position holds a 0, and everything below it is also 0. A row exchange cannot help. Don't stop; move on to the next column. Column 2 is dependent on the columns before it.

Column 3. The pivot is 2. Subtract row 2 from row 3:

\[U = \begin{bmatrix} \boxed{1} & 2 & 2 & 2 \\ 0 & 0 & \boxed{2} & 4 \\ 0 & 0 & 0 & 0 \end{bmatrix}\]

Echelon form. \(U\) is in echelon form, meaning staircase form. It is not truly triangular.

The zero row. It appears because row 3 was a combination of rows 1 and 2, and elimination exposed that.

Rank

Rank \(r\) = the number of pivots. Here \(r = 2\).

Pivot variables and free variables

Pivot columns: columns 1 and 3. So \(x_1, x_3\) are pivot variables.

Free columns: columns 2 and 4. So \(x_2, x_4\) are free variables. You may assign them any values. The pivot variables are then determined by back substitution.

The equations \(Ux = 0\):

\[x_1 + 2x_2 + 2x_3 + 2x_4 = 0, \qquad 2x_3 + 4x_4 = 0\]

Special solutions

Rule. Set one free variable to 1 and the others to 0. Then back-substitute.

Special solution 1: \(x_2 = 1, x_4 = 0\).

  • Second equation gives \(x_3 = 0\).
  • First equation gives \(x_1 = -2\).
    \[s_1 = \begin{bmatrix} -2 \\ 1 \\ 0 \\ 0 \end{bmatrix}\]
    This says \(-2\,\text{col}_1 + \text{col}_2 = 0\), which matches column 2 being twice column 1.

Special solution 2: \(x_2 = 0, x_4 = 1\).

  • Second equation gives \(x_3 = -2\).
  • First equation gives \(x_1 = -2(0) - 2(-2) - 2(1) = 2\).
    \[s_2 = \begin{bmatrix} 2 \\ 0 \\ -2 \\ 1 \end{bmatrix}\]
    This says \(2\,\text{col}_1 - 2\,\text{col}_3 + \text{col}_4 = 0\).

The two special solutions are genuinely different. No multiple of \(s_1\) can have a 1 in position 4. So \(s_2\) is new.

The whole nullspace

\[N(A) = \left\{ c\,s_1 + d\,s_2 \right\} = \left\{ c\begin{bmatrix} -2\\1\\0\\0\end{bmatrix} + d\begin{bmatrix} 2\\0\\-2\\1\end{bmatrix} \right\}\]

This is a 2-dimensional subspace inside \(\mathbb{R}^4\).

The counting rule

For an \(m \times n\) matrix with rank \(r\):

  • Pivot variables: \(r\)
  • Free variables: \(n - r\)
  • Special solutions: one per free variable, so \(n - r\)
  • Nullspace: all combinations of the special solutions

Here \(n - r = 4 - 2 = 2\). There appeared to be 3 equations, but only \(r = 2\) were independent.

2. Reduced row echelon form \(R\)

Going one step further. Clean \(U\) up even more:

  1. Eliminate upward, to get zeros above the pivots as well as below.
  2. Divide each pivot row by its pivot, so every pivot equals 1.

Neither step changes the nullspace.

For Example 1:

  • Subtract row 2 from row 1: row 1 becomes \((1, 2, 0, -2)\).
  • Divide row 2 by 2: row 2 becomes \((0, 0, 1, 2)\).
\[R = \begin{bmatrix} \boxed{1} & 2 & 0 & -2 \\ 0 & 0 & \boxed{1} & 2 \\ 0 & 0 & 0 & 0 \end{bmatrix}\]

MATLAB computes this with rref(A).

What \(R\) shows directly

  • Pivot rows and pivot columns. In this example, the pivot rows are 1 and 2, and the pivot columns are 1 and 3.
  • An identity block. The \(r \times r\) identity matrix \(I\) sits in the pivot rows and pivot columns.
  • The free part \(F\). These are the entries of the free columns within the pivot rows:
    \[F = \begin{bmatrix} 2 & -2 \\ 0 & 2 \end{bmatrix}\]
  • Zero rows. They mark rows of \(A\) that were combinations of other rows.

Reading off the special solutions

The pivot entries of the special solutions are \(-F\): the free numbers with their signs flipped. Check against Example 1:

  • In \(s_1\), the pivot entries \((x_1, x_3) = (-2, 0)\) are the negative of \(F\)'s first column, \((2, 0)\).
  • In \(s_2\), the pivot entries \((x_1, x_3) = (2, -2)\) are the negative of \(F\)'s second column, \((-2, 2)\).

3. The general block form and the nullspace matrix \(N\)

Block form of \(R\). Reorder the columns so the pivot columns come first:

\[R = \begin{bmatrix} I & F \\ 0 & 0 \end{bmatrix}\]
Here \(I\) is \(r \times r\), \(F\) is \(r \times (n-r)\), and the zero rows sit at the bottom.

Solving \(Rx = 0\). Split \(x\) into pivot variables \(x_{\text{pivot}}\) and free variables \(x_{\text{free}}\):

\[x_{\text{pivot}} + F\,x_{\text{free}} = 0 \quad\Longrightarrow\quad x_{\text{pivot}} = -F\,x_{\text{free}}\]
This is all back substitution amounts to once \(R\) is fully reduced.

The nullspace matrix. Choose the free variables as the columns of the identity, one special solution at a time. Collecting the special solutions as columns gives the nullspace matrix:

\[N = \begin{bmatrix} -F \\ I \end{bmatrix}, \qquad RN = 0\]
The block order in \(N\) follows the pivot-first reordering. In the original column order, the rows are interleaved.

In software. Strang's teaching code (a MATLAB function, null) computes \(R\), identifies the pivot and free variables, sets the free variables to 1s and 0s, and reads off the pivot variables.

4. Example 2: the transpose of \(A\)

\[A^\top = \begin{bmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \\ 2 & 6 & 8 \\ 2 & 8 & 10 \end{bmatrix}\]

Prediction. Column 3 equals column 1 plus column 2. So expect 2 pivot columns and 1 free column.

Elimination:

  1. Subtract 2 × row 1 from rows 2, 3, and 4. This gives the rows \((0,0,0)\), \((0,2,2)\), and \((0,4,4)\).
  2. Column 2 has a 0 in the pivot position but a 2 below it. Exchange rows to bring the 2 up.
  3. Subtract 2 × the new row 2 from the row below it.
  4. Check column 3 before stopping. Its pivot position holds a 0, and everything below it is 0. Now you may stop.
\[U = \begin{bmatrix} 1 & 2 & 3 \\ 0 & 2 & 2 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}\]

Rank. The rank is again \(r = 2\). Strang calls this an important fact: \(A\) and \(A^\top\) always have the same number of pivots.

Nullspace. There is \(n - r = 3 - 2 = 1\) free variable, \(x_3\). Set \(x_3 = 1\):

  • Second equation, \(2x_2 + 2x_3 = 0\), gives \(x_2 = -1\).
  • First equation, \(x_1 + 2x_2 + 3x_3 = 0\), gives \(x_1 = -1\).
\[N(A^\top) = \left\{ c\begin{bmatrix} -1\\-1\\1\end{bmatrix} \right\}\]

This is a line in \(\mathbb{R}^3\). It checks out: \(-\text{col}_1 - \text{col}_2 + \text{col}_3 = 0\).

Strang's exam note. "Find the nullspace" means the whole line, so include the \(c\). Later, "find a basis for the nullspace" will mean just the vector itself.

Reducing to \(R\). Subtract row 2 from row 1, then divide row 2 by 2:

\[R = \begin{bmatrix} 1 & 0 & 1 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix} = \begin{bmatrix} I & F \\ 0 & 0 \end{bmatrix}, \quad F = \begin{bmatrix} 1 \\ 1 \end{bmatrix}\]

The special solution is \(N = \begin{bmatrix} -F \\ 1 \end{bmatrix} = (-1, -1, 1)\), which matches.

The algorithm for \(N(A)\)

  1. Eliminate to \(U\), or further to \(R\). When a column has no pivot, skip it and continue.
  2. Count the pivots. That count is the rank \(r\).
  3. Classify the variables. Pivot columns give pivot variables. The other columns give free variables, and there are \(n - r\) of them.
  4. Find the special solutions. For each free variable, set it to 1, set the other free variables to 0, and solve for the pivot variables. With \(R\), the pivot variables are simply \(-F\).
  5. Take all combinations. \(N(A)\) is the set of all combinations of the \(n - r\) special solutions.

Big ideas from Lecture 7

  1. Elimination preserves \(N(A)\) but changes \(C(A)\).
  2. Rank \(r\) is the number of pivots. The number of free variables is \(n - r\).
  3. Special solutions: each one sets one free variable to 1 and the rest to 0. Every vector in \(N(A)\) is a combination of them.
  4. \(R = \begin{bmatrix} I & F \\ 0 & 0\end{bmatrix}\), so the nullspace matrix is \(N = \begin{bmatrix} -F \\ I \end{bmatrix}\).
  5. Zero rows in \(U\) or \(R\) show rows of \(A\) that were combinations of other rows.
  6. \(A\) and \(A^\top\) have the same rank. This is previewed here and proved later.