Lecture 11: Matrix Spaces; Rank 1; Small World Graphs

The OCW 2010 page has no transcript for this lecture, so this summary is based on the same lecture's transcript from an OCW mirror.

Topics:

Vector spaces whose "vectors" are matrices or functions.
Rank-1 matrices.
A complete four-subspace example.
A preview of graphs and "small world" networks.

1. Matrix spaces

The space \(M\). \(M\) is the set of all \(3\times3\) matrices.

You can add matrices and multiply them by scalars, so \(M\) is a vector space.
Matrix multiplication is not part of the vector-space structure.

Basis and dimension of \(M\)

The standard basis. It consists of 9 matrices, each with a single 1 and zeros elsewhere.

Dimension. \(\dim M = 9\).

Strang notes that \(M\) is essentially \(\mathbb{R}^9\) with its nine entries arranged in a square instead of a column.

Subspaces

Subspace	Dimension	Why
\(S\) = symmetric \(3\times3\)	6	Choose 3 diagonal entries and 3 entries above the diagonal. The entries below are then determined.
\(U\) = upper triangular \(3\times3\)	6	3 diagonal entries and 3 entries above the diagonal
\(D = S \cap U\) = diagonal	3	A symmetric matrix with zeros below the diagonal must also have zeros above it

\(S\) is closed under addition but not multiplication. The sum of two symmetric matrices is symmetric, so \(S\) is a subspace. The product of two symmetric matrices is generally not symmetric, but that doesn't matter, since products play no role here.

A basis for \(M\) need not contain a basis for a subspace

For \(U\), it happens to work. Six of the nine standard basis matrices are upper triangular, and together they form a basis for \(U\).

For \(S\), it does not. Only 3 of the 9 standard basis matrices are symmetric: the three with a single 1 on the diagonal. A basis for \(S\) also needs matrices like

\begin{bmatrix} 0&1&0\\1&0&0\\0&0&0\end{bmatrix}

which is not in the standard basis for \(M\).

In general, you have to construct a basis for a subspace from scratch.

Union vs. sum

The union \(S \cup U\) is not a subspace. Adding a symmetric matrix to an upper triangular one usually gives a matrix in neither set. Strang compares it to taking two lines in the plane and stopping there.

The sum fills in the gap:

S + U = \{\, s + u : s \in S,\; u \in U \,\}

Here this gives all of \(M\). Every \(3\times3\) matrix is a symmetric matrix plus an upper triangular one.

Dimension formula

For any two subspaces \(S\) and \(U\):

\dim S + \dim U = \dim(S \cap U) + \dim(S + U)

Here: \(6 + 6 = 3 + 9\).

2. Function spaces: solutions of a differential equation

Consider the differential equation

\frac{d^2y}{dx^2} + y = 0

The solutions. All solutions have the form

y = c_1 \cos x + c_2 \sin x

The solution set is a vector space. In effect it is the nullspace of the operator \(\frac{d^2}{dx^2} + 1\).
Basis: \(\{\cos x,\; \sin x\}\). These play the role of special solutions.
Dimension: 2. This matches the order of the equation.

Bases are not unique here either. \(\{e^{ix},\; e^{-ix}\}\) is another basis.

Why functions count as vectors. You can add them and multiply them by constants. Strang's remark: a course on linear differential equations is largely about finding a basis for a solution space.

3. Rank-1 matrices

Example:

A = \begin{bmatrix} 1 & 4 & 5 \\ 2 & 8 & 10 \end{bmatrix}

Row 2 is 2 × row 1.
A basis for the row space is \((1,4,5)\).
A basis for the column space is \((1,2)\).
Row space and column space both have dimension \(r = 1\).

Every rank-1 matrix is a column times a row:

A = \begin{bmatrix} 1 \\ 2 \end{bmatrix}\begin{bmatrix} 1 & 4 & 5 \end{bmatrix} = uv^\top

Building blocks. Rank-1 matrices are the building blocks of all matrices. A matrix of rank \(r\) can be written as a sum of \(r\) rank-1 matrices. For example, a \(5\times17\) matrix of rank 4 is a sum of 4 rank-1 matrices.

Coming later. Determinants and eigenvalues of rank-1 matrices turn out to be easy and interesting.

Matrices of a fixed rank do not form a subspace

Take \(M\) to be all \(5 \times 17\) matrices.

Rank-1 matrices. The sum of two rank-1 matrices usually has rank 2. So the rank-1 matrices are not a subspace.

Rank-4 matrices. Same problem. The sum of two of them can have rank 5. Note that 5 is the maximum possible rank for a \(5\times17\) matrix.

General inequality:

\text{rank}(A + B) \le \text{rank}(A) + \text{rank}(B)

4. Worked example: all four subspaces

The subspace. In \(\mathbb{R}^4\), let

S = \{v : v_1 + v_2 + v_3 + v_4 = 0\}

\(S\) is a subspace. Scaling or adding vectors whose components sum to zero gives another vector whose components sum to zero.

\(S\) is a nullspace. It is \(N(A)\) for the \(1 \times 4\) matrix

A = \begin{bmatrix} 1 & 1 & 1 & 1 \end{bmatrix}

Dimension. \(A\) has \(m = 1\), \(n = 4\), and \(r = 1\). So

\dim S = n - r = 4 - 1 = 3

Basis for \(S\) (the special solutions). The pivot variable is \(v_1\), and the free variables are \(v_2, v_3, v_4\). Setting each free variable to 1 in turn gives

\begin{bmatrix}-1\\1\\0\\0\end{bmatrix},\; \begin{bmatrix}-1\\0\\1\\0\end{bmatrix},\; \begin{bmatrix}-1\\0\\0\\1\end{bmatrix}

The four subspaces of \(A = [1\;1\;1\;1]\)

Subspace	Lives in	Dimension	Basis
Row space \(C(A^\top)\)	\(\mathbb{R}^4\)	1	\((1,1,1,1)\)
Nullspace \(N(A)\)	\(\mathbb{R}^4\)	3	the three special solutions above
Column space \(C(A)\)	\(\mathbb{R}^1\)	1	\((1)\), so \(C(A)\) is all of \(\mathbb{R}^1\)
Left nullspace \(N(A^\top)\)	\(\mathbb{R}^1\)	0	empty set

Dimension checks:

In \(\mathbb{R}^4\): \(1 + 3 = 4 = n\).
In \(\mathbb{R}^1\): \(1 + 0 = 1 = m\).

The zero subspace. \(\{0\}\) has dimension 0, and its basis is the empty set. A basis with zero vectors fits a space of dimension zero.

5. Preview: graphs and the small-world question

What a graph is. A graph is a set of nodes together with edges that connect pairs of nodes. This is not a graph of a function. Strang sketches a graph with 5 nodes and 6 edges. In the next lecture, a \(6 \times 5\) matrix will describe it completely.

The friendship graph.

Nodes: people.
Edges: an edge joins two people who are friends.

The question. What is the largest distance between two people, where distance means the number of steps along edges?

"Six degrees of separation." This phrase reflects the observation that most people are connected through only a few intermediaries. The familiar remark "it's a small world" comes from discovering such connections.

The mathematical point. Adding a few shortcut edges reduces distances dramatically. The same questions arise for the web graph, where websites are nodes and links are edges.

Next lecture: linear algebra applied to graphs.

Big ideas from Lecture 11

Vectors don't have to be columns of numbers. Matrices and functions qualify, because you can add them and multiply them by scalars.
Matrix spaces. \(\dim M = 9\), \(\dim S = \dim U = 6\), and \(\dim D = 3\).
Sum vs. union. \(S + U\) is a subspace, while \(S \cup U\) is not. Their dimensions satisfy \(\dim S + \dim U = \dim(S\cap U) + \dim(S+U)\).
Differential equations. The solutions of \(y'' + y = 0\) form a 2-dimensional space with basis \(\{\cos x, \sin x\}\).
Rank 1. A rank-1 matrix equals \(uv^\top\). Rank-\(r\) matrices are sums of \(r\) such matrices. Matrices of a fixed rank do not form a subspace.
The zero subspace has dimension 0 and an empty basis.