← All notes · ML revision (DA5400)

Lecture 27 · Part 2 of the course begins

Supervised Learning

So far the data was just $x$'s. Now each data point comes with a label $y$ telling us the right answer. The task is to learn a rule that predicts the label of new points. What kind of value the label takes decides what kind of problem it is.

Source: this lecture on YouTube (≈9 min). Next: L28 · Linear regression.

1 · The setup

We have $n$ data points $x_1,\dots,x_n\in\mathbb R^d$, the same as in unsupervised learning. Each entry of $x$ is called a feature (also an attribute or predictor). Now each point also comes with a label $y_1,\dots,y_n$. The labels are the "supervision": someone has supplied the correct answer for each training example.

Goal: learn from the pairs $(x_i, y_i)$ so that for a new $x$ you've never seen, you can predict its label well.

2 · Three kinds of supervised problems

binary classification

Labels: two values

$y\in\{+1,-1\}$ or $\{0,1\}$ (the two are interchangeable).
Example: is an email spam or not spam? Each email is turned into a vector of numbers (features).

multi-class classification

Labels: a finite set

$y\in\{0,1,\dots,k\}$.
Example: handwritten digit recognition. The image is the feature vector, and the label is which of the digits 0–9 was written.

regression

Labels: any real number

$y\in\mathbb R$.
Example: predict a day's rainfall (say 100.23 cm) from temperature, precipitation, air pressure and so on.

Each kind needs its own algorithms. The reasons come up as the course goes on.

3 · Quick check: which kind is it?

Pick an answer for each task.

4 · Roadmap for part 2

  1. Regression first, starting with linear regression, for a couple of weeks. It's the oldest supervised problem, with deep roots in statistics.
  2. Binary classification next, which gets the most time.
  3. Multi-class briefly, mainly by building it out of binary classifiers.

Key takeaways

  1. Supervised learning = features $x$ plus labels $y$. Learn to predict $y$ for new $x$.
  2. Two labels gives binary classification; a finite set gives multi-class; real numbers give regression.