PatchCore
What PatchCore Does
PatchCore finds unusual regions in product images. It builds its detector from normal examples, so defective examples are not required to create its memory bank.
Think of it as a visual inspector that has studied many acceptable products. When a new region looks unlike anything it remembers, PatchCore gives that region a higher anomaly score.
Key paper: Towards Total Recall in Industrial Anomaly Detection (2021)
Core Idea
- Extract and record visual patterns from normal product images
- Keep a representative collection of those patterns in a memory bank
- Compare each region of a new image with its closest match in memory
- Treat large differences as possible anomalies
How It Works
1. Describe Image Regions
The whole image passes through a pretrained neural network. Locations in its internal feature maps describe nearby image regions through patterns such as edges, textures, and shapes.
These descriptions are called patch features. PatchCore combines features from more than one network layer so it can capture both small details and broader structure.
2. Build a Smaller Memory Bank
Normal training images can produce millions of patch features. Storing all of them would use too much memory and make comparisons slow.
PatchCore keeps a representative subset, called a coreset. This removes many similar examples while preserving the variety found in the normal images.
3. Compare New Regions with Memory
For each patch feature in a test image, PatchCore finds the nearest feature in the memory bank. The distance between them becomes the patch's anomaly score.
A close match suggests that the region looks normal. A distant match suggests that the region may contain something unusual.
4. Produce Scores and a Heatmap
Patch scores are resized into a heatmap that highlights likely anomaly regions. The most anomalous patch provides the starting point for an image-level score, which PatchCore adjusts using nearby matches in memory.
The scores are continuous, not automatic yes-or-no decisions. A deployment can apply a chosen threshold when it needs to label an image as normal or anomalous.
Why It Works
- Useful visual features: The pretrained network already recognizes many common shapes and textures.
- Normal examples define the reference: The memory bank records the range of acceptable appearance.
- Local comparison: Patch-level scores can identify an unusual region even when most of the image looks normal.
- No task-specific backbone training: Building the detector requires feature extraction and memory-bank construction rather than gradient-based model training.
Results on MVTec AD
For its standard PatchCore configuration, the paper reports 99.1% image-level AUROC and 98.1% pixel-level AUROC on MVTec AD. AUROC measures separation across thresholds for images or pixels.
Limitations
- A large memory bank increases storage use and comparison time. A coreset and approximate nearest-neighbor search, such as FAISS, can reduce these costs.
- Normal changes in pose, lighting, or alignment may receive high anomaly scores if they are missing from the training examples.
- Features from a general-purpose pretrained network may transfer poorly to a very different visual domain.
- The heatmap is based on resized patch scores, so anomaly boundaries may be approximate rather than pixel-perfect.
Related
- Anomaly Detection
- PatchCore Anomaly Detection project - implementation on MVTec AD