Lecture 30: Linear Transformations and Their Matrices

Strang's opening point. Many linear algebra courses begin here, with transformations described without coordinates. Physicists often prefer it that way. But to compute anything you must choose coordinates, meaning a basis. Once you do, every linear transformation becomes a matrix, and everything in the course (nullspace, rank, determinant, eigenvalues) comes from that matrix.

1. What makes a transformation linear

A transformation $T$ takes each input vector $v$ to an output $T(v)$, like a function. The words map and mapping mean the same thing.

$T$ is linear if it respects the two vector operations:

T(v + w) = T(v) + T(w), \qquad T(cv) = c\,T(v)

Combined into one rule:

T(cv + dw) = c\,T(v) + d\,T(w)

Consequence: $T(0) = 0$ always. Take $c = 0$ in the scaling rule.

2. Examples and non-examples

Transformation	Linear?	Reason
Projection of the plane onto a line	✓	Doubling $v$ doubles the projection; reversing $v$ reverses it
Rotation by 45°	✓	Rotating a sum is the same as summing the rotations
Multiplication by a matrix, $T(v) = Av$	✓	$A(v + w) = Av + Aw$ and $A(cv) = cAv$
Shift the plane: $T(v) = v + v_0$	✗	$T(0) = v_0 \neq 0$; doubling $v$ still adds only one $v_0$
Length: $T(v) = \Vert v\Vert $	✗	$T(-2v) = 2\Vert v\Vert $, not $-2\Vert v\Vert $

Note on the length example. $T(0) = 0$ holds and positive scaling works, but negative scalars fail. So $T(0) = 0$ is necessary but not sufficient for linearity.

Seeing a transformation act on a whole shape

The cover of Strang's textbook shows a house in the plane. Applying $T$ to every point of its outline shows where the entire shape goes.

A rotation turns the whole house.
$A = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix}$ keeps $x$ and flips the sign of $y$. The house turns upside down, door included.

3. A basis determines everything

Question. How much information do you need to know $T(v)$ for every $v$?

Answer. Just $T(v_1), \dots, T(v_n)$ for a basis $v_1, \dots, v_n$ of the input space.

Why.

Every input can be written as $v = c_1v_1 + \cdots + c_nv_n$.
Linearity then forces $T(v) = c_1T(v_1) + \cdots + c_nT(v_n)$
So there is no freedom left once the basis images are fixed.

Independence matters. If $v_2 = 6v_1$, then telling you $T(v_2)$ adds nothing new, since it must equal $6\,T(v_1)$.

Coordinates come from a basis

Definition. The coordinates of $v$ are the numbers $c_1, \dots, c_n$ in its expansion in the chosen basis.

The usual default. We almost always assume the standard basis. Then a vector like $(3, 2, 4)$ already is its coordinates.

Other bases change the coordinates. You could, for example, use the eigenvectors of some matrix as the basis. The same vector then has different coordinates.

4. Constructing the matrix of $T$

Setup. $T$ maps $\mathbb{R}^n \to \mathbb{R}^m$. Choose two bases:

an input basis $v_1, \dots, v_n$ for $\mathbb{R}^n$,
an output basis $w_1, \dots, w_m$ for $\mathbb{R}^m$.

Rule for column $j$:

Apply $T$ to the $j$-th input basis vector.
Write the result in the output basis: $T(v_j) = a_{1j}w_1 + a_{2j}w_2 + \cdots + a_{mj}w_m$
The coefficients $a_{1j}, \dots, a_{mj}$ form column $j$ of $A$.

What the matrix does:

A \times (\text{input coordinates}) = (\text{output coordinates})

Why the rule works.

The input $v_1$ has coordinates $(1, 0, \dots, 0)$.
Multiplying $A$ by that vector picks out column 1, which is exactly the coordinates of $T(v_1)$.
The same holds for every basis vector, and linearity extends it to all inputs.

5. Example: the same projection in two bases

$T$ is projection onto a line in the plane. Use the same basis for inputs and outputs in each case.

Eigenvector basis (the good choice)

Basis. $v_1$ is a unit vector on the line, and $v_2$ is a unit vector perpendicular to it.

What $T$ does to each basis vector.

$T(v_1) = v_1$, since it is already on the line.
$T(v_2) = 0$, since it projects to nothing.

So the input $c_1v_1 + c_2v_2$ maps to $c_1v_1$. In coordinates, $(c_1, c_2) \mapsto (c_1, 0)$.

A = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} = \Lambda

Why it came out diagonal. These basis vectors are the projection's eigenvectors, with eigenvalues 1 and 0. Strang calls this the basis a physicist would pick if forced to introduce coordinates.

Standard basis

Basis. $(1,0)$ and $(0,1)$, projecting onto the 45° line.

What $T$ does to each basis vector. Both map to $(\tfrac12, \tfrac12)$.

A = \begin{bmatrix} \tfrac12 & \tfrac12 \\ \tfrac12 & \tfrac12 \end{bmatrix}

This is the familiar $P = \dfrac{aa^\top}{a^\top a}$ from Chapter 4, with $a = (1, 1)$.

Comparison.

It is still symmetric with $P^2 = P$.
It is not diagonal.
Strang notes the course has mostly used "the handiest basis" rather than the best one. In the eigenvector basis, all the properties are obvious at a glance.

6. Example: the derivative as a linear transformation

Inputs: $c_1 + c_2x + c_3x^2$, with basis $1, x, x^2$. This space is 3-dimensional.

Output: the derivative $c_2 + 2c_3x$, with output basis $1, x$. This space is 2-dimensional.

Why the derivative is linear. The derivative of a combination is the combination of the derivatives. Strang points out that this is exactly why we can differentiate efficiently: learn the derivatives of a few basic functions, then combine them.

The matrix ($2\times3$):

A = \begin{bmatrix} 0 & 1 & 0 \\ 0 & 0 & 2 \end{bmatrix}, \qquad A\begin{bmatrix} c_1 \\ c_2 \\ c_3 \end{bmatrix} = \begin{bmatrix} c_2 \\ 2c_3 \end{bmatrix}

Check with the column rule.

$T(1) = 0$, so column 1 is $(0, 0)$.
$T(x) = 1$, so column 2 is $(1, 0)$.
$T(x^2) = 2x$, so column 3 is $(0, 2)$.

7. Closing point: where matrix rules come from

Matrix multiplication comes from composing transformations.

The product of two matrices is the matrix of the composed transformation.
The inverse matrix is the matrix of the inverse transformation.

Strang says he'll pick up this theme next lecture.

Big ideas from Lecture 30

Linear: $T(cv + dw) = cT(v) + dT(w)$. In particular $T(0) = 0$.
Non-examples: shifting the plane, and taking the length of a vector.
A basis determines $T$: knowing $T(v_1), \dots, T(v_n)$ fixes $T$ on all inputs.
Building the matrix: column $j$ holds the output-basis coordinates of $T(v_j)$.
The matrix depends on the bases. In the eigenvector basis, the matrix is diagonal, $\Lambda$. In the standard basis, you get the usual matrix.
Linear transformations come first. Matrix multiplication and inverses are defined to match composition and inversion of transformations.

Connection to Lecture 28. Using the same transformation with a different basis, where the input and output basis are the same, gives a similar matrix: $B = M^{-1}AM$. That is why similar matrices share eigenvalues. They are one transformation written in two coordinate systems.

Connection to ML.

A linear layer $x \mapsto Wx$ is a linear transformation in this sense.
Adding a bias, $Wx + b$, is exactly the "shift the plane" non-example. That map is affine, not linear.
Choosing the right basis, such as eigenvectors or singular vectors, to make a learned map look simple is the same idea as Strang's "good basis".