Weakly Supervised Learning
1. Purpose & Motivation
1.1 Why weakly supervised learning exists
- Perfect labels are often unavailable in practice
- Labels may be noisy, incomplete, coarse-grained, or provided by non-experts
- Discarding imperfect labels wastes valuable signal
1.2 The core problem it solves
Train effective models using imprecise, incomplete, or inexact supervision rather than requiring perfectly annotated datasets.
2. Definition & Core Idea
2.1 Formal definition
Weakly supervised learning encompasses methods that learn from training signals that are less informative than exact ground-truth labels, including noisy labels, partial labels, or labels at a coarser granularity than the task requires.
2.2 Intuitive mental model
"Learning to identify bird species from a guidebook that only says 'this is some kind of sparrow' rather than giving the exact species for every photograph."
3. Types of Weak Supervision
3.1 Taxonomy
| Type | Description | Example |
|---|---|---|
| Incomplete | Only a subset of data is labeled | Semi-supervised setting |
| Inexact | Labels are at a coarser granularity than needed | Image-level labels for pixel-level task |
| Inaccurate | Labels contain noise or errors | Crowdsourced annotations with mistakes |
3.2 Common sources of weak labels
- Crowdsourcing: Multiple non-expert annotators with disagreements
- Heuristic rules: Programmatic labeling functions (Snorkel-style)
- Distant supervision: Aligning data with external knowledge bases
- Data augmentation labels: Inherited labels from transformations
4. Key Techniques
4.1 Noise-robust training
- Loss correction methods that adjust for estimated label noise
- Sample weighting to downweight suspected noisy examples
- Co-teaching: two networks filter noisy samples for each other
4.2 Label aggregation
- Combine multiple weak labelers to produce better labels
- Majority voting, Dawid-Skene model, data programming (Snorkel)
4.3 Multi-instance learning (MIL)
- Labels are given to bags of instances, not individual instances
- The model learns to identify which instances within a bag carry the label
- Common in medical imaging (slide-level diagnosis, patch-level prediction)
5. Strengths & Limitations
5.1 Strengths
- Dramatically lowers annotation cost and time
- Enables learning in domains where perfect labels are infeasible
- Can combine multiple weak signals for strong supervision
5.2 Limitations
- Model performance is bounded by label quality
- Noise can accumulate and degrade training
- Requires careful validation with a clean held-out set
6. One-Sentence Summary
Weakly supervised learning trains models from noisy, incomplete, or coarse-grained labels, making machine learning feasible when perfect annotations are impractical.