Lecture 24b: Quiz 2 Review · Ashutosh Patidar

This summary is based on the same lecture's transcript from an OCW mirror. Where Strang stops short of a final number, I've finished the computation and marked it as mine.

What the quiz covers.

Chapter 4, orthogonality: \(Q^\top Q = I\), projections, least squares, Gram–Schmidt.
Chapter 5, determinants: Properties 1–10, the big formula, cofactors, the inverse formula.
Sections 6.1–6.2: eigenvalues and eigenvectors, \(A = S\Lambda S^{-1}\), powers \(A^k\).
Not covered: differential equations (Section 6.3), which go on Quiz 3.

Format. As in Lecture 13, Strang works through problems from old exams, mainly 1984 and 1992.

Problem 1: Projection onto a line, then iterating it (1984)

Setup. \(a = (2, 1, 2)\). Find \(P\), the projection matrix onto the line through \(a\).

(a) The projection matrix

For a single column, the general formula \(A(A^\top A)^{-1}A^\top\) reduces to \(\dfrac{aa^\top}{a^\top a}\). Here \(a^\top a = 9\):

P = \frac{aa^\top}{a^\top a} = \frac19\begin{bmatrix} 4 & 2 & 4 \\ 2 & 1 & 2 \\ 4 & 2 & 4 \end{bmatrix}

(b) Rank, column space, eigenvalues

Rank: 1.
Column space: the line through \(a\).
\(\lambda = 0\), repeated twice. \(P\) has rank 1, so its nullspace is 2-dimensional. That gives two independent eigenvectors with \(\lambda = 0\).
\(\lambda = 1\). Two ways to see it:
- The trace is \(9/9 = 1\), and the other two eigenvalues are 0.
- \(Pa = a\), because the projection leaves \(a\) in place. So \(a\) is the eigenvector for \(\lambda = 1\).

(c) Solve \(u_{k+1} = Pu_k\) with \(u_0 = (9, 9, 0)\)

First step:

u_1 = Pu_0 = \frac{a^\top u_0}{a^\top a}\,a = \frac{27}{9}\,a = 3a = (6, 3, 6)

All later steps. \(u_1\) already lies on the line, and \(P^2 = P\), so projecting again changes nothing:

u_k = (6, 3, 6) \quad\text{for all } k \geq 1

For a general matrix, not a projection, use the full method:

u_0 = c_1x_1 + c_2x_2 + c_3x_3, \qquad u_k = c_1\lambda_1^kx_1 + c_2\lambda_2^kx_2 + c_3\lambda_3^kx_3

That means finding all the \(\lambda\)'s, all the \(x\)'s, and the \(c\)'s.

For this \(P\), two eigenvalues are 0, so those terms vanish after one step. The remaining term is \(c_3(1)^kx_3\), which stays constant.

Problem 2: Least squares and Gram–Schmidt (1984)

(a) Best line through the origin

Data. \((t, y) = (1, 4),\ (2, 5),\ (3, 8)\). The model is \(y = Dt\), with one unknown \(D\).

The equations we'd like to solve:

\begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix} D = \begin{bmatrix} 4 \\ 5 \\ 8 \end{bmatrix}

Normal equation. \(A^\top A\hat{D} = A^\top b\), with \(A^\top A = 14\) and \(A^\top b = 4 + 10 + 24 = 38\):

\hat{D} = \frac{38}{14} = \frac{19}{7}

(b) Which vector is projected onto which line?

Strang stresses that least squares has two different pictures.

Picture 1: the \(t\)–\(y\) plane. It shows the three data points and the fitted line. There is no projection in this picture. The points are not being projected onto the line.

Picture 2: \(\mathbb{R}^3\). The vector \(b = (4, 5, 8)\) is projected onto the line through \((1, 2, 3)\), which is \(C(A)\). \(\hat{D}\) is the multiple of \((1,2,3)\) that gives the projection.

(c) Gram–Schmidt

Setup. Make \(a_1 = (1, 2, 3)\) and \(a_2 = (1, 1, 1)\) orthogonal, staying in the plane they span.

Keep \(a_1\). Then subtract from \(a_2\) its projection onto \(a_1\):

B = a_2 - \frac{a_1^\top a_2}{a_1^\top a_1}\,a_1 = (1,1,1) - \frac{6}{14}(1,2,3)

Finishing the arithmetic (my computation):

B = \left(\tfrac47,\; \tfrac17,\; -\tfrac27\right)

Check: \(a_1^\top B = \tfrac{4 + 2 - 6}{7} = 0\).

No square roots. Only orthogonality was asked for, so there is no need to normalize.

Not unique. Any rotation of this orthogonal pair within the same plane would also work.

Problem 3: Eigenvalue facts (1984)

\(A\) is \(4\times4\) with eigenvalues \(\lambda_1, \lambda_2, \lambda_3, \lambda_4\).

Question	Answer	Reason
When is \(A\) invertible?	No \(\lambda_i = 0\)	\(\lambda = 0\) means a nonzero vector in the nullspace
\(\det(A^{-1})\)?	\(\dfrac{1}{\lambda_1\lambda_2\lambda_3\lambda_4}\)	The eigenvalues of \(A^{-1}\) are \(1/\lambda_i\), and det = product of eigenvalues
\(\text{trace}(A + I)\)?	\(\lambda_1 + \lambda_2 + \lambda_3 + \lambda_4 + 4\)	Adding \(I\) shifts every eigenvalue by 1

Problem 4: Tridiagonal determinants and period 6 (1984)

The family. \(A_n\) has 1s on the main diagonal and on the diagonals just above and below. Write \(D_n = \det A_n\).

(a) Recurrence by cofactors

Expand along row 1:

The \((1,1)\) entry contributes \(1 \cdot D_{n-1}\).
The \((1,2)\) entry sits in a minus position. Its minor has a single 1 at the top of its first column, so expanding down that column gives \(D_{n-2}\). The contribution is \(-D_{n-2}\).

D_n = D_{n-1} - D_{n-2}, \qquad D_1 = 1,\; D_2 = 0

This is the same result as in Lecture 19.

(b) Write it as a first-order system

Use the vector unknown \((D_n, D_{n-1})\), as with Fibonacci:

\begin{bmatrix} D_n \\ D_{n-1} \end{bmatrix} = \underbrace{\begin{bmatrix} 1 & -1 \\ 1 & 0 \end{bmatrix}}_{A}\begin{bmatrix} D_{n-1} \\ D_{n-2} \end{bmatrix}

(c) Eigenvalues

\det(A - \lambda I) = \lambda^2 - \lambda + 1 = 0 \;\Rightarrow\; \lambda = \frac{1 \pm i\sqrt3}{2}

This is not Fibonacci. The sign change in the recurrence makes the eigenvalues complex.

Magnitude.

|\lambda|^2 = \tfrac14 + \tfrac34 = 1

Both eigenvalues lie on the unit circle:

\lambda = \cos 60° \pm i\sin 60° = e^{\pm i\pi/3}

(d) What happens to \(D_n\)?

Sixth power. \(\lambda^6 = e^{\pm 2\pi i} = 1\).

So \(A^6 = I\). The two eigenvalues are distinct, so \(A\) is diagonalizable. Then

A^6 = S\Lambda^6S^{-1} = SIS^{-1} = I

Behavior. The sequence neither decays nor blows up. It cycles with period 6: \(1, 0, -1, -1, 0, 1, \dots\)

Problem 5: A family of matrices from the 1992 final

The family. \(A_n\) is symmetric and tridiagonal, with zeros on the diagonal and off-diagonal entries \(1, 2, 3, \dots\) For example:

A_3 = \begin{bmatrix} 0 & 1 & 0 \\ 1 & 0 & 2 \\ 0 & 2 & 0 \end{bmatrix}, \qquad A_4 = \begin{bmatrix} 0 & 1 & 0 & 0 \\ 1 & 0 & 2 & 0 \\ 0 & 2 & 0 & 3 \\ 0 & 0 & 3 & 0 \end{bmatrix}

(a) Is \(A_3\) singular? What is the projection onto \(C(A_3)\)?

Singular. Column 3 is \(2 \times\) column 1. So \(C(A_3)\) is a plane, and \(P\) is \(3\times3\).

The projection. Strang says to use \(A(A^\top A)^{-1}A^\top\), and hints that a quicker route exists.

The quicker route (my computation). The pivot columns \((0,1,0)\) and \((1,0,2)\) are already orthogonal. So \(P\) is just the sum of the two one-dimensional projections:

P = \begin{bmatrix} 0&0&0 \\ 0&1&0 \\ 0&0&0 \end{bmatrix} + \frac15\begin{bmatrix} 1&0&2 \\ 0&0&0 \\ 2&0&4 \end{bmatrix} = \begin{bmatrix} 1/5&0&2/5 \\ 0&1&0 \\ 2/5&0&4/5 \end{bmatrix}

(b) Eigenvalues of \(A_3\)

Expand with the big formula:

\det(A_3 - \lambda I) = -\lambda^3 + 5\lambda = -\lambda(\lambda^2 - 5)

The down-right products give \(-\lambda^3\).
The down-left products give \(-(-\lambda - 4\lambda) = +5\lambda\).

Eigenvalues: \(\lambda = 0,\; \sqrt5,\; -\sqrt5\).

Strang's habit: check the trace. The eigenvalues sum to 0, matching the trace of 0. He also notes the limitation of this check: it would not have caught an error that changed \(5\lambda\) to \(4\lambda\), since that still gives eigenvalues summing to 0.

(c) Projection onto \(C(A_4)\)

The exam said this is "not difficult". Strang wonders aloud whether that comment was meant to be helpful or insulting.

The trick. If \(A_4\) is invertible, then \(C(A_4) = \mathbb{R}^4\), and the projection is \(P = I\).

Check invertibility with cofactors.

Row 1 has a single nonzero entry, \(a_{12} = 1\), in a minus position.
Its minor has a single nonzero entry in its first column, a 1, and its own minor is \(\begin{vmatrix} 0 & 3 \\ 3 & 0 \end{vmatrix} = -9\).
Tracking the signs gives:

\det A_4 = 9 \neq 0

So \(A_4\) is invertible and \(P = I\).

Strang's guess. Odd-numbered members of the family are singular, and even-numbered ones are invertible.

Confirming the guess (my addition).

Odd \(n\) is always singular. The eigenvalues of a zero-diagonal symmetric tridiagonal matrix come in \(\pm\) pairs. With an odd count, one of them must be 0.
Even \(n\) is always invertible. The determinant is \(\pm(b_1b_3b_5\cdots)^2\), where \(b_1, b_3, \dots\) are the 1st, 3rd, 5th, … off-diagonal entries. For \(n = 4\) this is \((1 \cdot 3)^2 = 9\). Since every \(b_i \neq 0\), the determinant is never zero.

Quick review checklist

Projection onto a line: \(P = aa^\top/a^\top a\). It has rank 1, eigenvalues \(1, 0, \dots, 0\), and \(Pa = a\).
Projections are idempotent: \(P^k = P\), so \(u_{k+1} = Pu_k\) stops changing after one step.
Least squares: solve \(A^\top A\hat{x} = A^\top b\). The geometric picture is \(b\) projected onto \(C(A)\) in \(\mathbb{R}^m\), not points projected onto the fitted line.
Gram–Schmidt: \(B = a_2 - \dfrac{a_1^\top a_2}{a_1^\top a_1}a_1\).
Eigenvalue shortcuts:
- \(\det = \prod\lambda_i\) and \(\text{trace} = \sum\lambda_i\).
- \(A^{-1}\) has eigenvalues \(1/\lambda_i\).
- \(A + cI\) has eigenvalues \(\lambda_i + c\).
Recurrences: convert to \(u_{k+1} = Au_k\). Then \(|\lambda| > 1\) grows, \(|\lambda| < 1\) decays, and \(|\lambda| = 1\) oscillates. For example, \(e^{\pm i\pi/3}\) gives period 6.
Invertible means \(C(A)\) is the whole space, so the projection onto it is \(I\).