Weakly Supervised Learning

1. Purpose & Motivation

1.1 Why weakly supervised learning exists

  • Perfect labels are often unavailable in practice
  • Labels may be noisy, incomplete, coarse-grained, or provided by non-experts
  • Discarding imperfect labels wastes valuable signal

1.2 The core problem it solves

Train effective models using imprecise, incomplete, or inexact supervision rather than requiring perfectly annotated datasets.

2. Definition & Core Idea

2.1 Formal definition

Weakly supervised learning encompasses methods that learn from training signals that are less informative than exact ground-truth labels, including noisy labels, partial labels, or labels at a coarser granularity than the task requires.

2.2 Intuitive mental model

"Learning to identify bird species from a guidebook that only says 'this is some kind of sparrow' rather than giving the exact species for every photograph."

3. Types of Weak Supervision

3.1 Taxonomy

Type Description Example
Incomplete Only a subset of data is labeled Semi-supervised setting
Inexact Labels are at a coarser granularity than needed Image-level labels for pixel-level task
Inaccurate Labels contain noise or errors Crowdsourced annotations with mistakes

3.2 Common sources of weak labels

  • Crowdsourcing: Multiple non-expert annotators with disagreements
  • Heuristic rules: Programmatic labeling functions (Snorkel-style)
  • Distant supervision: Aligning data with external knowledge bases
  • Data augmentation labels: Inherited labels from transformations

4. Key Techniques

4.1 Noise-robust training

  • Loss correction methods that adjust for estimated label noise
  • Sample weighting to downweight suspected noisy examples
  • Co-teaching: two networks filter noisy samples for each other

4.2 Label aggregation

  • Combine multiple weak labelers to produce better labels
  • Majority voting, Dawid-Skene model, data programming (Snorkel)

4.3 Multi-instance learning (MIL)

  • Labels are given to bags of instances, not individual instances
  • The model learns to identify which instances within a bag carry the label
  • Common in medical imaging (slide-level diagnosis, patch-level prediction)

5. Strengths & Limitations

5.1 Strengths

  • Dramatically lowers annotation cost and time
  • Enables learning in domains where perfect labels are infeasible
  • Can combine multiple weak signals for strong supervision

5.2 Limitations

  • Model performance is bounded by label quality
  • Noise can accumulate and degrade training
  • Requires careful validation with a clean held-out set

6. One-Sentence Summary

Weakly supervised learning trains models from noisy, incomplete, or coarse-grained labels, making machine learning feasible when perfect annotations are impractical.

7. Related Knowledge Pages