Lecture 5: Transposes, Permutations, Vector Spaces $\mathbb{R}^n$

Two parts:

Finishing Chapter 2: row exchanges ($PA = LU$), transposes, and symmetric matrices.
Starting Chapter 3: vector spaces and subspaces. Strang calls this the beginning of "real" linear algebra.

Part 1: Finishing elimination

$PA = LU$: elimination with row exchanges

When row exchanges are needed. $A = LU$ assumes no row exchanges. If a zero appears in a pivot position, rows must be swapped.

The general factorization. Do all the needed row exchanges first, using a permutation matrix $P$. After that, elimination runs cleanly:

\[PA = LU\]

Scope. This holds for every invertible $A$. Most matrices need no exchanges, so $P = I$. A few do need them.

What MATLAB does in practice. It exchanges rows not only for a zero pivot but also for a small pivot, because tiny pivots cause numerical error. Algebra doesn't require these exchanges, but accuracy does. (This is called partial pivoting.)

Permutation matrices: recap

Definition: $P$ is the identity matrix with its rows reordered. The identity itself counts.
Count: there are $n!$ permutation matrices of size $n \times n$. That is 6 for $n=3$, 24 for $n=4$, and 120 for $n=5$.
Inverse: $P^{-1} = P^\top$, which is the same statement as $P^\top P = I$.
Preview: matrices with $Q^\top Q = I$ will be important later (orthogonal matrices). Permutations are a small special family inside that group.

Transposes

Definition. Flip the matrix across its main diagonal, so rows become columns:

(A^\top)_{ij} = A_{ji}

Example. A $3\times 2$ matrix becomes $2 \times 3$:

\begin{bmatrix} 1 & 3 \\ 2 & 3 \\ 4 & 1 \end{bmatrix}^\top = \begin{bmatrix} 1 & 2 & 4 \\ 3 & 3 & 1 \end{bmatrix}

MATLAB writes the transpose with a prime: A'.

Symmetric matrices

Definition. $A^\top = A$, so entry $a_{ij}$ equals entry $a_{ji}$. The matrix is a mirror image across its diagonal. For example, entries 1, 7, 9 above the diagonal must appear again below it in the mirrored positions:

\begin{bmatrix} 3 & 1 & 7 \\ 1 & 2 & 9 \\ 7 & 9 & 4 \end{bmatrix}

Where symmetric matrices come from. For any rectangular $R$, the product $R^\top R$ is always symmetric. So is $RR^\top$.

Numerical example. With the $3\times 2$ matrix $R$ above, Strang multiplies the $3\times 2$ matrix by its $2\times 3$ transpose:

\begin{bmatrix} 1 & 3 \\ 2 & 3 \\ 4 & 1 \end{bmatrix}\begin{bmatrix} 1 & 2 & 4 \\ 3 & 3 & 1 \end{bmatrix} = \begin{bmatrix} 10 & 11 & 7 \\ 11 & 13 & 11 \\ 7 & 11 & 17 \end{bmatrix}

The 11 appears in both (1,2) and (2,1), because $(1,3)\cdot(2,3) = (2,3)\cdot(1,3)$. Strang loosely calls this product "$R^\top R$". Strictly, with this $R$ it is $RR^\top$, but the symmetry argument is identical for both.

Proof in two lines. Transpose the product and check that nothing changes:

(R^\top R)^\top = R^\top (R^\top)^\top = R^\top R

Strang notes that this is where symmetric matrices come from in many applications.

Part 2: Vector spaces and subspaces

What a vector space is

The two operations. You can do two things with vectors:

Add them: $v + w$.
Multiply them by scalars: $cv$.

Definition. A vector space is a set of vectors that is closed under linear combinations. Any combination $cv + dw$ of vectors in the set stays in the set.

The eight rules. The book lists eight rules that addition and scalar multiplication must obey, such as commutativity and distributivity. In practice these are rarely in doubt. Closure is the real test.

The examples $\mathbb{R}^n$

$\mathbb{R}^2$: all vectors with 2 real components, such as $(3,2)$, $(0,0)$, $(\pi, e)$. It is pictured as the $xy$-plane.

$\mathbb{R}^3$: all vectors with 3 real components. For example, $(3, 2, 0)$ is in $\mathbb{R}^3$, not $\mathbb{R}^2$. Having a zero component doesn't change which space a vector lives in.

$\mathbb{R}^n$: all column vectors with $n$ real components. Complex vectors come much later in the course.

Every vector space contains the zero vector. Multiplying any vector by 0 must stay in the space. So must adding $v$ and $-v$.

A non-example. Removing the origin from $\mathbb{R}^2$ breaks closure.

Non-example: the first quadrant

Take all vectors in $\mathbb{R}^2$ with both components $\geq 0$.

Addition works: the sum of two such vectors stays in the quadrant.
Scalar multiplication fails: $-5 \cdot (3,2) = (-15,-10)$ falls outside.

So this set is not closed under multiplication by all real numbers, and it is not a vector space.

Subspaces

Definition. A subspace is a vector space that sits inside a larger one. It is closed under addition and scalar multiplication.

Building one from a single vector. If a subspace contains a nonzero vector $v$, it must contain every multiple $cv$, including $2v$, $\tfrac12 v$, $-v$, and $0$. So it contains the whole line through $v$.

Lines must pass through the origin. A line that misses the origin is not a subspace, because $0 \cdot v$ would fall off the line.

Every subspace contains $0$.

All subspaces of $\mathbb{R}^2$

Subspace	Description
$\mathbb{R}^2$ itself	The whole plane
Any line $L$ through $(0,0)$	One-dimensional
$Z = \{(0,0)\}$	The zero vector alone, the smallest subspace

A line in $\mathbb{R}^2$ is not $\mathbb{R}^1$. It looks similar, but its vectors have 2 components, not 1.

Why $\{0\}$ qualifies. $0 + 0 = 0$ and $17 \cdot 0 = 0$, so the operations never leave it.

All subspaces of $\mathbb{R}^3$

All of $\mathbb{R}^3$
Any plane through the origin
Any line through the origin
$\{0\}$ alone

The column space $C(A)$

This is the first important subspace that comes from a matrix.

Example:

A = \begin{bmatrix} 1 & 3 \\ 2 & 3 \\ 4 & 1 \end{bmatrix}

The columns $(1,2,4)$ and $(3,3,1)$ are vectors in $\mathbb{R}^3$.

Why the columns alone are not enough. Two vectors by themselves do not form a subspace. For closure, you must include:

$0$
their sum $(4,5,5)$
every combination $c_1\,\text{col}_1 + c_2\,\text{col}_2$

Definition. The column space $C(A)$ is the set of all linear combinations of the columns of $A$. It is automatically a subspace.

Geometry.

All combinations of two independent vectors in $\mathbb{R}^3$ fill a plane through the origin. Both lines fill out, and so does everything between them.
If the two columns had been on the same line, $C(A)$ would only be that line.

Higher dimensions. Take 5 vectors in $\mathbb{R}^{10}$. Their combinations form some subspace of $\mathbb{R}^{10}$, at most five-dimensional and passing through the origin. It is not $\mathbb{R}^5$, because the vectors have 10 components. Its actual dimension depends on the vectors. For example, if all 5 lie on one line, you get only that line.

Preview for Lecture 6: understanding $Ax = b$ in terms of column spaces, and meeting other subspaces such as the null space.

Big ideas from Lecture 5

$PA = LU$ is elimination in its general form, covering any invertible $A$. $P$ puts the rows in an order where no zero pivots appear.
Permutations: there are $n!$ of them, and $P^{-1} = P^\top$.
Transpose: $(A^\top)_{ij} = A_{ji}$, and $(AB)^\top = B^\top A^\top$.
Symmetric: $A^\top = A$. The product $R^\top R$ is always symmetric.
Vector space: a set closed under addition and scalar multiplication. It always contains $0$.
Subspace: a vector space inside another one. In $\mathbb{R}^3$ these are the whole space, planes through 0, lines through 0, and $\{0\}$.
Column space $C(A)$: all combinations of the columns of $A$, which is a subspace of $\mathbb{R}^m$.

Lecture 5: Transposes, Permutations, Vector Spaces \(\mathbb{R}^n\)

Part 1: Finishing elimination

\(PA = LU\): elimination with row exchanges

Permutation matrices: recap

Transposes

Symmetric matrices

Part 2: Vector spaces and subspaces

What a vector space is

The examples \(\mathbb{R}^n\)

Non-example: the first quadrant

Subspaces

All subspaces of \(\mathbb{R}^2\)

All subspaces of \(\mathbb{R}^3\)

The column space \(C(A)\)

Big ideas from Lecture 5