Metrics

Precision / Recall

Two complementary metrics for evaluating classification performance, especially when classes are imbalanced.

  • Precision = TP / (TP + FP) — "Of all positive predictions, how many were correct?"
  • Recall = TP / (TP + FN) — "Of all actual positives, how many did we find?"
High Precision Low Precision
High Recall Ideal Many false positives
Low Recall Missing real positives Poor model

Trade-off: Increasing recall usually decreases precision and vice versa. The right balance depends on the application cost of false positives vs false negatives.

False Reject / False Accept

Quality inspection terms mapping directly to classification errors:

  • False Reject (Type I error) — A good part is incorrectly rejected
    • Wastes material and production time
    • Maps to False Positive in defect detection
  • False Accept (Type II error) — A defective part is incorrectly accepted
    • Escapes to customer, potential safety/quality issue
    • Maps to False Negative in defect detection

In manufacturing, false accepts are almost always worse than false rejects — a missed defect reaching the customer is far more costly than scrapping a good part.

Metric Detects Reduces
High Precision Few false rejects Scrap waste
High Recall Few false accepts Customer escapes

F1 Score

The harmonic mean of Precision and Recall. Useful when you need a single metric that balances both.

  • F1 = 2 × (Precision × Recall) / (Precision + Recall)
  • Ranges from 0 (worst) to 1 (best)
  • Preferred over accuracy when classes are imbalanced
  • Penalizes extreme imbalance between precision and recall

Confusion Matrix

A table showing the counts of TP, FP, FN, TN predictions, giving a complete picture of classifier behavior.

Predicted Positive Predicted Negative
Actual Positive TP FN
Actual Negative FP TN
  • Foundation from which Precision, Recall, F1, and Accuracy are derived
  • Reveals which specific error types the model makes

AUROC / AUC

Area Under the Receiver Operating Characteristic curve. Measures classifier performance across all decision thresholds.

  • ROC curve: plots True Positive Rate vs False Positive Rate at each threshold
  • AUC = 1.0: perfect classifier; AUC = 0.5: random
  • Threshold-independent — useful when the operating point isn't fixed
  • Used in PatchCore evaluation (image-level AUROC)

mAP (mean Average Precision)

The primary metric for object detection benchmarks. Averages the Average Precision (AP) across all object classes.

  • AP: area under the Precision-Recall curve for one class
  • mAP@0.5: IoU threshold of 0.5 to count a detection as correct
  • mAP@0.5:0.95: averaged over IoU thresholds 0.50–0.95 (stricter, used in COCO)
  • Standard for YOLO, Faster R-CNN, and similar detectors

IoU (Intersection over Union)

Measures the overlap between a predicted bounding box (or mask) and the ground truth.

  • IoU = Area of Overlap / Area of Union
  • Ranges from 0 (no overlap) to 1 (perfect)
  • Threshold (e.g., 0.5) determines what counts as a correct detection
  • Also used in segmentation as pixel-level overlap metric