Ashutosh Patidar · notes

Linear Algebra (source: Gilbert Strang notes) · 9 / 37

Lecture 8: Solving \(Ax = b\): Row Reduced Form \(R\)

This summary is based on the lecture's transcript on MIT OCW.

Goal: Solve \(Ax = b\) completely. That means three things:

  1. Decide whether a solution exists.
  2. Find one solution.
  3. Find all solutions.

Then use the rank to classify every case.

1. Example: the same matrix as Lecture 7

\[A = \begin{bmatrix} 1 & 2 & 2 & 2 \\ 2 & 4 & 6 & 8 \\ 3 & 6 & 8 & 10 \end{bmatrix}\]

Row 3 is row 1 plus row 2. So any solvable \(b\) must satisfy the same relation: \(b_3 = b_1 + b_2\). For example, if \(b_1 = 1\) and \(b_2 = 5\), then \(b_3\) must be 6.

Elimination on the augmented matrix \([A \mid b]\)

Treat \(b\) as an extra column and apply the same row operations to it.

Step 1. Subtract 2 × row 1 from row 2, and 3 × row 1 from row 3:

\[\left[\begin{array}{cccc|c} 1 & 2 & 2 & 2 & b_1 \\ 0 & 0 & 2 & 4 & b_2 - 2b_1 \\ 0 & 0 & 2 & 4 & b_3 - 3b_1 \end{array}\right]\]

Step 2. Subtract the new row 2 from row 3:

\[\left[\begin{array}{cccc|c} 1 & 2 & 2 & 2 & b_1 \\ 0 & 0 & 2 & 4 & b_2 - 2b_1 \\ 0 & 0 & 0 & 0 & b_3 - b_2 - b_1 \end{array}\right]\]

The solvability condition. The last row reads \(0 = b_3 - b_2 - b_1\). So a solution exists only when \(b_3 = b_1 + b_2\).

With \(b = (1, 5, 6)\): the right side becomes \((1, 3, 0)\). The last equation is \(0 = 0\), which holds, so this \(b\) is solvable.

Two equivalent ways to state solvability

  1. Column view: \(Ax = b\) is solvable exactly when \(b \in C(A)\).
  2. Row view: if some combination of the rows of \(A\) gives the zero row, then the same combination of the entries of \(b\) must give 0.

These two statements are equivalent. That is not obvious, but both are exactly the condition for solvability.

2. Finding the complete solution

Step 1: One particular solution \(x_p\)

Method. Set all free variables to zero, then solve for the pivot variables by back substitution.

Here the free variables are \(x_2\) and \(x_4\). Setting both to zero leaves only the pivot columns:

\[x_1 + 2x_3 = 1, \qquad 2x_3 = 3\]

Back substitution gives \(x_3 = \tfrac32\), then \(x_1 = 1 - 2\cdot\tfrac32 = -2\):

\[x_p = \begin{bmatrix} -2 \\ 0 \\ 3/2 \\ 0 \end{bmatrix}\]

Strang advises checking \(x_p\) in the original equations, for instance on a quiz. Here: \(-2 + 3 = 1\), \(-4 + 9 = 5\), \(-6 + 12 = 6\).

Step 2: Add the whole nullspace

\[\boxed{x_{\text{complete}} = x_p + x_n}\]

Why this works:

\[A(x_p + x_n) = Ax_p + Ax_n = b + 0 = b\]

Strang notes this same pattern appears throughout mathematics, wherever equations are linear.

For this example, using the special solutions from Lecture 7:

\[x = \begin{bmatrix} -2 \\ 0 \\ 3/2 \\ 0 \end{bmatrix} + c_1\begin{bmatrix} -2 \\ 1 \\ 0 \\ 0 \end{bmatrix} + c_2\begin{bmatrix} 2 \\ 0 \\ -2 \\ 1 \end{bmatrix}\]

No constant multiplies \(x_p\). Scaling it would change the right side: \(A(3x_p) = 3b \neq b\). Only the nullspace part can be scaled and combined freely.

Geometry

The solution set is not a subspace. It does not contain 0.

What it is. It is the 2-dimensional nullspace inside \(\mathbb{R}^4\), shifted so that it passes through \(x_p\) instead of the origin. It is a flat plane that misses the origin.

The term dimension gets its formal definition next lecture. Intuitively, it is the number of free constants: 2 here.

3. The big picture: rank decides everything

For an \(m \times n\) matrix of rank \(r\) (the number of pivots), always:

\[r \le m \qquad \text{and} \qquad r \le n\]
A row cannot hold more than one pivot, and neither can a column.

Case A: full column rank, \(r = n\) (tall, thin matrix)

Structure.

  • Every column has a pivot.
  • There are no free variables.
  • \(N(A) = \{0\}\).

Solutions. \(x = x_p\) only. The solution is unique if it exists, so there are 0 or 1 solutions.

Example:

\[A = \begin{bmatrix} 1 & 3 \\ 2 & 1 \\ 6 & 1 \\ 5 & 1 \end{bmatrix} \quad\Longrightarrow\quad R = \begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 0 & 0 \\ 0 & 0 \end{bmatrix}\]

  • This is 4 equations in 2 unknowns. A random \(b\) has no solution.
  • \(b = (4, 3, 7, 6)\), the sum of the two columns, has exactly one solution: \(x = (1, 1)\).

Strang notes this case is common in applications, where the columns are independent.

Case B: full row rank, \(r = m\) (short, wide matrix)

Structure.

  • Every row has a pivot.
  • Elimination produces no zero rows, so there is no condition on \(b\).

Solutions. \(Ax = b\) is solvable for every \(b\). There are \(n - r = n - m\) free variables.

Example (the transpose of the matrix in Case A):

\[A = \begin{bmatrix} 1 & 2 & 6 & 5 \\ 3 & 1 & 1 & 1 \end{bmatrix}\]

  • The rank is 2, with pivots in columns 1 and 2.
  • \(R\) has the \(2\times2\) identity in the pivot columns and a free part \(F\) in the other columns.
  • There are no zero rows.

Case C: square and full rank, \(r = m = n\)

Structure.

  • The matrix is invertible.
  • \(R = I\).
  • \(N(A) = \{0\}\).
  • There is no condition on \(b\).

Solutions. There is exactly one solution for every \(b\).

Example:

\[\begin{bmatrix} 1 & 2 \\ 3 & 1 \end{bmatrix} \quad\Longrightarrow\quad R = I\]

Case D: \(r < m\) and \(r < n\)

Structure.

  • There are zero rows, which impose conditions on \(b\).
  • There are free variables, so the nullspace is nonzero.

Solutions. Either 0 or infinitely many solutions.

Summary table

Rank Form of \(R\) Number of solutions to \(Ax = b\)
\(r = m = n\) \(I\) exactly 1
\(r = n < m\) \(\begin{bmatrix} I \\ 0 \end{bmatrix}\) 0 or 1
\(r = m < n\) \(\begin{bmatrix} I & F \end{bmatrix}\) \(\infty\) (always solvable)
\(r < m,\; r < n\) \(\begin{bmatrix} I & F \\ 0 & 0 \end{bmatrix}\) 0 or \(\infty\)

Two notes on the table:

  • In \(R\), the \(I\) and \(F\) columns can be interleaved if the pivot columns are not the first ones.
  • While filling in this table on the board, Strang briefly mislabeled one row and then corrected himself. The table above is the corrected version.

Strang's closing point: the rank \(r\) tells you everything about how many solutions exist. Only the actual entries of the solutions require working with the matrix itself.

Big ideas from Lecture 8

  1. Augmented matrix. Eliminate on \([A \mid b]\). Each zero row of \(A\) produces a condition on \(b\).
  2. Solvability. \(b \in C(A)\). Equivalently, any combination of rows that gives the zero row must also give 0 when applied to \(b\).
  3. Particular solution. Set the free variables to 0 and solve for the pivot variables.
  4. Complete solution. \(x = x_p + x_n\): one fixed \(x_p\) plus all of \(N(A)\). Geometrically, a shifted copy of the nullspace.
  5. Rank classification.
    • \(r = n\): at most one solution.
    • \(r = m\): at least one solution.
    • Both (\(r = m = n\)): exactly one solution.