Classification
What the Task Does
Classification assigns one or more category labels to an input such as an image, document, or signal. It supports applications including diagnosis, content moderation, quality inspection, spam detection, and species identification.
Unlike detection or segmentation, ordinary image classification describes the input as a whole. It does not say where the evidence appears.
Core Idea
A model learns a function that maps an input to class scores called logits. An output layer turns those scores into probabilities or independent label scores, and a decision rule selects the prediction.
For mutually exclusive classes, softmax converts logit into
The class with the highest probability is the usual prediction. During training, cross-entropy increases the score of the correct class relative to the alternatives.
Main Variants
Binary Classification
Binary classification separates two outcomes, such as defective and non-defective. A single sigmoid output commonly represents and is trained with binary cross-entropy.
Multi-class Classification
Multi-class classification chooses one class from several mutually exclusive options, such as cat, dog, or bird. It commonly uses one logit per class, softmax, and categorical cross-entropy.
Multi-label Classification
Multi-label classification allows several labels to be true at once, as with tags describing one image. Each class receives an independent sigmoid output and usually its own binary cross-entropy term.
| Type | Valid output | Typical activation | Typical loss |
|---|---|---|---|
| Binary | One of two classes | Sigmoid | Binary cross-entropy |
| Multi-class | One of classes | Softmax | Categorical cross-entropy |
| Multi-label | Any subset of labels | Sigmoid per label | Binary cross-entropy per label |
Temperature scaling replaces with before softmax. sharpens the distribution and softens it; fitting on validation data can improve calibration without changing class order.
How It Works
A feature extractor such as a CNN or Vision Transformer converts the input into a representation. A classification head then pools spatial features, applies one or more learned layers, and produces class logits.
Image-classification architectures illustrate the evolution of this feature extractor:
| Era | Architecture | Main contribution |
|---|---|---|
| 2012 | AlexNet | Deep CNN training on GPUs |
| 2014 | VGGNet | Depth built from small 3 × 3 kernels |
| 2015 | ResNet | Residual connections |
| 2019 | EfficientNet | Compound depth, width, and resolution scaling |
| 2020 | Vision Transformer | Self-attention over image patches |
| 2022 | ConvNeXt | A modernized convolutional design |
Fine-tuning a pretrained feature extractor is often more effective than training from scratch, especially when the task dataset is small.
How It Is Evaluated
| Metric | What it measures | Useful when |
|---|---|---|
| Accuracy | Fraction of predictions that are correct | Classes and error costs are balanced |
| Precision | Fraction of positive predictions that are correct | False positives are costly |
| Recall | Fraction of actual positives that are found | False negatives are costly |
| F1 score | Harmonic mean of precision and recall | A single balance of both is needed |
| Top- accuracy | Whether the correct class is among the top scores | The label space is large |
A confusion matrix compares actual and predicted classes. It reveals systematic confusions that disappear inside a single aggregate score. For imbalanced data, inspect per-class and macro-averaged metrics rather than accuracy alone.
Calibration is a separate property from accuracy. Reliability diagrams, expected calibration error, or the Brier score test whether events assigned probability occur approximately a fraction of the time.
Practical Considerations
- Transfer learning: Start from a relevant pretrained backbone when labeled data is limited.
- Data augmentation: Use transformations that preserve the label while reflecting deployment variation.
- Class imbalance: Consider weighted losses, balanced sampling, focal loss, and class-specific thresholds.
- Thresholds: Choose them from operational error costs rather than assuming 0.5 is optimal.
- Dataset design: Prevent near-duplicates, subjects, or acquisition sessions from leaking across splits.
Limitations and Failure Modes
- A classifier may exploit background, watermark, or acquisition artifacts instead of the intended object.
- Closed-set models force unfamiliar inputs into known classes unless rejection or out-of-distribution handling is added.
- Softmax values can be overconfident and are not automatically calibrated probabilities.
- Aggregate accuracy can hide poor performance on rare classes or demographic and environmental subgroups.
- Distribution shifts in lighting, sensors, geography, or class prevalence can invalidate validation results.