Lecture 1: The Geometry of Linear Equations

Core problem: Solve \(n\) linear equations in \(n\) unknowns, written as \(Ax = b\).

Strang presents three ways to view this problem:

1. Row picture: one equation at a time.
2. Column picture: one column at a time. He marks this as the most important view.
3. Matrix form: \(Ax = b\).

1. The 2×2 example

System:

Matrix form:

Row picture

Each equation is a line in the \(xy\)-plane.

• First line, \(2x - y = 0\): It passes through the origin, because the right side is 0. It also passes through \((1, 2)\) and \((\tfrac12, 1)\).
• Second line, \(-x + 2y = 3\): It does not pass through the origin. It goes through \((-3, 0)\) and \((-1, 1)\).
• Solution: The two lines meet at \((x, y) = (1, 2)\). Check in the second equation: \(-1 + 4 = 3\).

Strang notes the word "linear" contains "line": a linear equation in two unknowns graphs as a line.

Column picture (the key idea)

Rewrite the system as a combination of the columns of \(A\):

The problem now asks: which linear combination of the columns produces \(b\)? Strang calls linear combination the most fundamental operation of the whole course.

• Drawing it: Draw column 1, which is \((2, -1)\). Attach two copies of column 2, \((-1, 2)\), head to tail. You land on \((0, 3)\).
• Checking it: First component: \(2 - 2 = 0\). Second component: \(-1 + 4 = 3\).

Preview question: What do all combinations of these two columns produce? For this matrix, they fill the whole plane, so every right-hand side \(b\) is reachable.

2. The 3×3 example

System:

Row picture

Each equation is now a plane in 3D.

• Equation 2 as a plane: It does not pass through the origin. Points on it include \((1,0,0)\), \((0,0,1)\), and \((0,-\tfrac12,0)\).
• Equation 1 as a plane: It has no \(z\) term, so \(z\) can be anything and the plane is vertical.
• How the planes combine: Two planes meet in a line. The third plane cuts that line at a single point, which is the solution.

Strang admits the row picture becomes hard to see in 3D, and harder still in higher dimensions. This is one reason he prefers the column picture.

Column picture

Strang chose \(b\) on purpose to equal column 3. So the solution is visible at once:

This is the point where the three planes meet. It was hard to find from the row picture.

Changing the right-hand side

Now take \(b = \text{col}_1 + \text{col}_2 = (1, 1, -3)\). The solution becomes \((x, y, z) = (1, 1, 0)\).

• Row picture: There are three new planes, because each equation's right side changed.
• Column picture: The same three columns are combined differently.

This shows an advantage of the column view: the columns stay fixed when only \(b\) changes.

Systematic solution method

In general, you won't spot the answer by inspection. Elimination is the systematic method used by all serious software, and it is the topic of Lecture 2.

3. The central question: can we solve \(Ax = b\) for every \(b\)?

In column language: Do the linear combinations of the columns fill all of 3D space?

For this \(A\): yes. Strang calls such a matrix non-singular or invertible, and these are the "good" matrices.

When the answer is no:

• Columns in a plane: If the three columns lie in the same plane, every combination also lies in that plane. Example: column 3 equals column 1 plus column 2.
• Effect on \(b\): Only right-hand sides \(b\) inside that plane can be reached. Most \(b\) cannot.
• Name: This is the singular case. The matrix is not invertible.

Higher dimensions (9×9):

• Setup: A 9×9 matrix has nine columns, each a vector in 9-dimensional space. Solving \(Ax=b\) means finding the right combination of those columns.
• Random matrices: A randomly chosen matrix, such as one from MATLAB's random command, is almost surely invertible, so its columns fill all of 9-dimensional space.
• Dependent columns: If column 9 duplicates column 8, it adds nothing new. The combinations then fill only an 8-dimensional "plane" inside 9-dimensional space.
• Strang's point: You can't truly visualize this, but with practice you learn to reason about it. Such lower-dimensional planes become central later in the course.

4. Matrix times vector: two ways

Example:

By columns (Strang's preferred way): Take 1 of column 1 plus 2 of column 2.

By rows: Take the dot product of each row with the vector.

• Row 1: \((2,5)\cdot(1,2) = 2 + 10 = 12\)
• Row 2: \((1,3)\cdot(1,2) = 1 + 6 = 7\)

Key takeaway: \(Ax\) is a linear combination of the columns of \(A\). Strang wants this to be the default way you think about matrix–vector multiplication.

As a consequence, if \(b = (12, 7)\), the solution of \(Ax = b\) is \(x = (1, 2)\).

Big ideas from Lecture 1

1. Three views: Row picture (lines or planes intersecting), column picture (combining columns), and matrix form.
2. The column picture matters most: Solving \(Ax = b\) means finding the combination of columns that equals \(b\).
3. Solvable for every \(b\): This holds exactly when the columns' combinations fill the whole space. Such a matrix is non-singular or invertible.
4. Singular case: If the columns are dependent (one lies in the span of the others), the combinations fill only a lower-dimensional subspace, and most \(b\) are unreachable.
5. Matrix–vector multiplication: \(Ax\) equals a combination of the columns of \(A\), weighted by the entries of \(x\).