Unsupervised Learning

1. Purpose & Motivation

1.1 Why unsupervised learning exists

  • Labeled data is expensive and scarce; unlabeled data is abundant
  • Many tasks require discovering structure rather than predicting labels
  • Understanding data distribution is often a prerequisite to downstream tasks

1.2 The core problem it solves

Extract meaningful patterns, groupings, or representations from data without any target labels to guide the learning process.

2. Definition & Core Idea

2.1 Formal definition

Unsupervised learning is a paradigm where a model learns patterns from a dataset {x_i} containing only inputs, with no corresponding labels. The objective is to model the underlying structure or distribution P(X).

2.2 Intuitive mental model

"Sort a pile of unlabeled photographs into groups that make sense, without anyone telling you the categories."

3. Core Tasks

3.1 Clustering

  • Goal: Group similar data points together
  • Algorithms: K-Means, DBSCAN, Hierarchical Clustering
  • Applications: Customer segmentation, document grouping, anomaly detection

3.2 Dimensionality reduction

  • Goal: Compress data into fewer meaningful dimensions
  • Algorithms: PCA, t-SNE, UMAP, Autoencoders
  • Applications: Visualization, noise reduction, feature compression

3.3 Density estimation

  • Goal: Model the probability distribution of the data
  • Algorithms: Gaussian Mixture Models, Kernel Density Estimation
  • Applications: Anomaly detection, generative modeling

4. How It Differs from Supervised Learning

Aspect Supervised Unsupervised
Labels required Yes No
Objective Minimize prediction error Discover structure
Evaluation Accuracy, F1, MSE Silhouette score, reconstruction error
Typical output Predictions Clusters, embeddings, distributions

5. Evaluation Challenges

5.1 No ground truth

  • Without labels, there is no single "correct" answer
  • Evaluation relies on internal metrics (cohesion, separation) or downstream task performance

5.2 Common evaluation approaches

  • Intrinsic: Silhouette score, Davies-Bouldin index, reconstruction error
  • Extrinsic: Use discovered structure in a supervised task and measure improvement

6. Strengths & Limitations

6.1 Strengths

  • No labeling cost
  • Reveals hidden structure humans may miss
  • Scales to massive unlabeled datasets

6.2 Limitations

  • Results can be ambiguous or hard to interpret
  • No clear optimization target
  • Sensitive to hyperparameters (e.g., number of clusters)

7. One-Sentence Summary

Unsupervised learning discovers hidden patterns and structure in unlabeled data without any target signal to guide the model.

8. Related Knowledge Pages