Ashutosh Patidar · notes

Machine Learning Revision (Notes)

Lecture-by-lecture revision notes for the DA5400 machine learning course: clustering, estimation and EM, and supervised learning (regression). Each page summarises one lecture with interactive demos and links to its video.

Source: Arun Rajkumar · DA5400. These are my personal, unofficial notes; explanations and extensions are my own, so check the lectures for the definitive treatment. Maths and charts load from public CDNs.

Clustering

  1. L13Introduction to clustering: Partitioning data, the K-means objective, and why exhaustive search is NP-hard video
  2. L14K-means (Lloyd's algorithm): The two-step algorithm and the open questions it raises video
  3. L15Convergence of K-means: Why the objective strictly decreases and the algorithm must stop video
  4. L16Nature of K-means clusters: Perpendicular bisectors, Voronoi regions, and kernel K-means video
  5. L17Initialising K-means: Random restarts and K-means++ video
  6. L18Choosing K: Objective + penalty, AIC/BIC, elbow and silhouette video

Estimation

  1. L19Introduction to estimation: Probabilistic models, unknown parameters, and the i.i.d. assumption video
  2. L20Maximum likelihood: Fisher's principle, log-likelihood, and densities for continuous data video
  3. L21Bayesian estimation: Priors, posteriors, the Beta–Bernoulli update, MAP video
  4. L22Gaussian mixture models: The two-step generative story and latent variables video
  5. L23Likelihood of a GMM: Why a sum inside the log blocks a closed-form answer video
  6. L24Convexity and Jensen's inequality: The chord test, and why log being concave helps video
  7. L25Estimating GMM parameters: A Jensen lower bound that is easy to maximise in two halves video
  8. L26The EM algorithm: E and M steps, soft K-means, and local maxima video

Supervised learning: regression

  1. L27Supervised learning: Labels, and binary vs multi-class vs regression video
  2. L28Linear regression: Squared error, overfitting, and restricting to linear functions video
  3. L29Solving linear regression: Normal equations, w* = (XXᵀ)⁺Xy, and the pseudo-inverse video
  4. L30Regression as projection: Projecting the labels onto the feature subspace video
  5. L31Gradient descent and SGD: Avoiding the matrix inverse, and mini-batches for huge n video
  6. L32Kernel regression: α = K⁻¹y and non-linear fits with kernels video
  7. L33Probabilistic view of regression: Gaussian noise ⇔ squared error video
  8. L34Goodness of the ML estimator: E‖ŵ − w‖² = σ² tr((XXᵀ)⁻¹) video
  9. L35A better estimator and cross-validation: (XXᵀ + λI)⁻¹Xy and K-fold CV video

Extra experiments

  1. +K-means in 3-D: clusters, the elbow method, random starts vs K-means++
  2. +Chart: silhouette score vs K
  3. +Chart: elbow method (2-D data)
  4. +Chart: elbow method (3-D data)
  5. +Chart: random starts vs K-means++