Anomaly Detection
What the Task Does
Anomaly detection assigns high scores to inputs or regions that differ from expected patterns. Depending on the application, it can flag an unusual image, localize a defect, or both.
It is useful when failures are rare, diverse, or expensive to label, as in industrial inspection, network security, and medical screening. Many methods train only on normal examples rather than a complete catalogue of anomalies.
Core Idea
Normal training data defines a reference distribution or representation. At inference, the model measures reconstruction error, feature distance, disagreement, or likelihood and converts that signal into an anomaly score.
The score is continuous. A threshold turns it into an operational accept-or-reject decision, while spatial scores can be resized or decoded into an anomaly heatmap.
Main Variants
Reconstruction-based Methods
An autoencoder or related model learns to reconstruct normal inputs. A large difference between input and reconstruction is treated as evidence of an anomaly, for example
This approach is intuitive, but a sufficiently expressive model may also reconstruct anomalies well.
Embedding-based Methods
A pretrained backbone maps images or patches into feature space. The score measures distance from normal embeddings or from a fitted distribution. PaDiM, SPADE, and PatchCore use this general strategy.
PatchCore stores a representative coreset of normal patch features. A test patch receives the distance to its nearest normal neighbor, enabling both image-level scoring and localized heatmaps.
Student-teacher Methods
A student learns to match a fixed teacher's features on normal data. An unusual input can produce a larger teacher-student discrepancy. EfficientAD, STPM, and Reverse Distillation are examples.
Normalizing-flow Methods
An invertible model transforms normal features toward a tractable distribution such as a Gaussian. Low likelihood under the learned density becomes an anomaly signal. FastFlow and CS-Flow follow this approach.
| Approach | Primary signal | Strength | Common risk |
|---|---|---|---|
| Reconstruction | Input-reconstruction error | Direct visual interpretation | Anomalies may reconstruct well |
| Embedding | Distance from normal features | Strong pretrained representations | Memory and nearest-neighbor cost |
| Student-teacher | Feature disagreement | Fast learned comparison | Student generalization can reduce contrast |
| Normalizing flow | Low learned likelihood | Explicit density model | Likelihood may track nuisance statistics |
How It Works in Inspection
Industrial systems capture an aligned image, extract features, compare them with a normal reference, and produce a score plus heatmap. A threshold then maps the score to an operator action such as accept, review, or reject.
MVTec AD contains 15 industrial object and texture categories with pixel-level anomaly annotations. MVTec LOCO adds logical and structural anomalies, while VisA includes visual anomalies in more complex scenes.
PatchCore is a strong training-light baseline when memory and nearest-neighbor search are acceptable. EfficientAD targets high-throughput student-teacher inference and includes an autoencoder branch for broader structural discrepancies.
How It Is Evaluated
Image-level AUROC measures how well scores rank anomalous images above normal ones across thresholds. An AUROC of 1 indicates perfect ranking and 0.5 corresponds to random ranking under the usual interpretation.
Pixel-level AUROC evaluates anomaly-map ranking at the pixel level. AUPRO aggregates per-region overlap while controlling false positives, reducing the dominance of large anomalous regions.
Ranking metrics do not select a production threshold. Validate the threshold on representative normal and anomalous samples, and report false accepts, false rejects, precision, recall, and latency at that operating point.
Practical Considerations
- Verify that the normal training set is clean and covers acceptable variation in pose, lighting, material, and process state.
- Split data by part, batch, capture session, or time to avoid near-duplicate leakage.
- Align objects when position is stable and misalignment is not itself anomalous.
- Choose image-level and pixel-level thresholds from operational costs, then monitor score drift.
- Validate ImageNet-pretrained features on the target sensor and material rather than assuming transfer.
Limitations and Failure Modes
- Unseen but acceptable variation can be scored as anomalous, causing false rejects.
- Normal contamination in training can teach the reference model to accept defects.
- Some models reconstruct or embed genuine anomalies too similarly to normal data.
- Heatmaps derived from coarse feature maps may not recover pixel-accurate boundaries.
- A high benchmark AUROC can coexist with an unusable false-positive rate at the required threshold.
- Logical anomalies, such as a missing component, may require wider context than local texture methods capture.