Unsupervised Learning
1. Purpose & Motivation
1.1 Why unsupervised learning exists
- Labeled data is expensive and scarce; unlabeled data is abundant
- Many tasks require discovering structure rather than predicting labels
- Understanding data distribution is often a prerequisite to downstream tasks
1.2 The core problem it solves
Extract meaningful patterns, groupings, or representations from data without any target labels to guide the learning process.
2. Definition & Core Idea
2.1 Formal definition
Unsupervised learning is a paradigm where a model learns patterns from a dataset {x_i} containing only inputs, with no corresponding labels. The objective is to model the underlying structure or distribution P(X).
2.2 Intuitive mental model
"Sort a pile of unlabeled photographs into groups that make sense, without anyone telling you the categories."
3. Core Tasks
3.1 Clustering
- Goal: Group similar data points together
- Algorithms: K-Means, DBSCAN, Hierarchical Clustering
- Applications: Customer segmentation, document grouping, anomaly detection
3.2 Dimensionality reduction
- Goal: Compress data into fewer meaningful dimensions
- Algorithms: PCA, t-SNE, UMAP, Autoencoders
- Applications: Visualization, noise reduction, feature compression
3.3 Density estimation
- Goal: Model the probability distribution of the data
- Algorithms: Gaussian Mixture Models, Kernel Density Estimation
- Applications: Anomaly detection, generative modeling
4. How It Differs from Supervised Learning
| Aspect | Supervised | Unsupervised |
|---|---|---|
| Labels required | Yes | No |
| Objective | Minimize prediction error | Discover structure |
| Evaluation | Accuracy, F1, MSE | Silhouette score, reconstruction error |
| Typical output | Predictions | Clusters, embeddings, distributions |
5. Evaluation Challenges
5.1 No ground truth
- Without labels, there is no single "correct" answer
- Evaluation relies on internal metrics (cohesion, separation) or downstream task performance
5.2 Common evaluation approaches
- Intrinsic: Silhouette score, Davies-Bouldin index, reconstruction error
- Extrinsic: Use discovered structure in a supervised task and measure improvement
6. Strengths & Limitations
6.1 Strengths
- No labeling cost
- Reveals hidden structure humans may miss
- Scales to massive unlabeled datasets
6.2 Limitations
- Results can be ambiguous or hard to interpret
- No clear optimization target
- Sensitive to hyperparameters (e.g., number of clusters)
7. One-Sentence Summary
Unsupervised learning discovers hidden patterns and structure in unlabeled data without any target signal to guide the model.