Metrics
Precision / Recall
Two complementary metrics for evaluating classification performance, especially when classes are imbalanced.
- Precision = TP / (TP + FP) — "Of all positive predictions, how many were correct?"
- Recall = TP / (TP + FN) — "Of all actual positives, how many did we find?"
| High Precision | Low Precision | |
|---|---|---|
| High Recall | Ideal | Many false positives |
| Low Recall | Missing real positives | Poor model |
Trade-off: Increasing recall usually decreases precision and vice versa. The right balance depends on the application cost of false positives vs false negatives.
False Reject / False Accept
Quality inspection terms mapping directly to classification errors:
- False Reject (Type I error) — A good part is incorrectly rejected
- Wastes material and production time
- Maps to False Positive in defect detection
- False Accept (Type II error) — A defective part is incorrectly accepted
- Escapes to customer, potential safety/quality issue
- Maps to False Negative in defect detection
In manufacturing, false accepts are almost always worse than false rejects — a missed defect reaching the customer is far more costly than scrapping a good part.
| Metric | Detects | Reduces |
|---|---|---|
| High Precision | Few false rejects | Scrap waste |
| High Recall | Few false accepts | Customer escapes |
F1 Score
The harmonic mean of Precision and Recall. Useful when you need a single metric that balances both.
- F1 = 2 × (Precision × Recall) / (Precision + Recall)
- Ranges from 0 (worst) to 1 (best)
- Preferred over accuracy when classes are imbalanced
- Penalizes extreme imbalance between precision and recall
Confusion Matrix
A table showing the counts of TP, FP, FN, TN predictions, giving a complete picture of classifier behavior.
| Predicted Positive | Predicted Negative | |
|---|---|---|
| Actual Positive | TP | FN |
| Actual Negative | FP | TN |
- Foundation from which Precision, Recall, F1, and Accuracy are derived
- Reveals which specific error types the model makes
AUROC / AUC
Area Under the Receiver Operating Characteristic curve. Measures classifier performance across all decision thresholds.
- ROC curve: plots True Positive Rate vs False Positive Rate at each threshold
- AUC = 1.0: perfect classifier; AUC = 0.5: random
- Threshold-independent — useful when the operating point isn't fixed
- Used in PatchCore evaluation (image-level AUROC)
mAP (mean Average Precision)
The primary metric for object detection benchmarks. Averages the Average Precision (AP) across all object classes.
- AP: area under the Precision-Recall curve for one class
- mAP@0.5: IoU threshold of 0.5 to count a detection as correct
- mAP@0.5:0.95: averaged over IoU thresholds 0.50–0.95 (stricter, used in COCO)
- Standard for YOLO, Faster R-CNN, and similar detectors
IoU (Intersection over Union)
Measures the overlap between a predicted bounding box (or mask) and the ground truth.
- IoU = Area of Overlap / Area of Union
- Ranges from 0 (no overlap) to 1 (perfect)
- Threshold (e.g., 0.5) determines what counts as a correct detection
- Also used in segmentation as pixel-level overlap metric