YOLO (You Only Look Once)

What YOLO Does

YOLO models locate objects and assign class labels in one detector pass. The output is a set of boxes, confidence scores, and classes that can support real-time tasks such as robotics, visual inspection, and safety monitoring.

The original You Only Look Once: Unified, Real-Time Object Detection reframed detection as a single neural-network prediction problem instead of a proposal stage followed by per-region classification. “YOLO” now refers to several related lineages, not one continuously governed architecture, so version names and benchmark numbers should be interpreted with their implementation and evaluation setup.

Core Idea

An image is encoded once, and a dense detection head predicts candidate locations and classes across the image. Modern YOLO versions use feature maps at multiple resolutions so small and large objects can be detected in the same pass.

YOLO Detection Pipeline

YOLO: Single-pass detection pipeline

Source: Redmon et al., 2015 — You Only Look Once (CVPR)

YOLO Grid Prediction

YOLO grid cell predictions

Source: Redmon et al., 2015 — You Only Look Once (CVPR)

How It Works

1. Extract Features Once

A convolutional backbone transforms the image into feature maps. In the original YOLO, 24 convolutional layers fed two fully connected layers. For PASCAL VOC, its 7×77\times7 grid output had shape S×S×(B⋅5+C)=7×7×30S\times S\times(B\cdot5+C)=7\times7\times30 for two boxes and 20 classes.

2. Predict Boxes and Classes

YOLOv1 assigns responsibility based on the grid cell containing an object's center. Each candidate predicts box coordinates (x,y,w,h)(x,y,w,h), confidence, and class information. Later families changed this formulation:

  • YOLOv2 introduced anchor boxes and offset prediction.
  • YOLOv3 predicted at three feature-map scales.
  • YOLOv4 and several descendants used CSP-style backbones and feature-pyramid/path-aggregation necks.
  • Ultralytics YOLOv8 uses an anchor-free, decoupled head with separate classification and box branches.
  • YOLOv10 proposes consistent dual assignments for end-to-end, NMS-free inference.

3. Fuse Multiple Scales

A modern detector commonly has three parts:

  1. a backbone extracts visual features;
  2. a neck combines high-resolution location detail with low-resolution semantic context; and
  3. a head predicts boxes and classes at several scales.

For a 640×640640\times640 input, one implementation may predict from 80×8080\times80, 40×4040\times40, and 20×2020\times20 maps. Exact channels, blocks, and outputs vary by version and model size.

4. Train Jointly

The loss combines localization and classification terms, and—where the version predicts it explicitly—objectness. Modern box losses are usually IoU-based; CIoU additionally considers center distance and aspect ratio. Label assignment determines which predictions train against each ground-truth object and is a major source of differences between versions.

5. Remove Duplicate Predictions

Most versions produce overlapping candidates and use non-maximum suppression (NMS): sort by confidence, keep a strong box, and suppress highly overlapping alternatives. NMS thresholds affect precision, recall, and crowded-scene behavior. End-to-end variants such as YOLOv10 are designed to avoid NMS at inference.

Why It Works

  • Shared computation: the backbone processes the image once rather than running a classifier for every proposed region.
  • Global context: predictions are conditioned on features from the whole image.
  • Dense prediction: all image regions are considered in parallel on accelerator-friendly tensors.
  • Multi-scale features: high-resolution maps help small objects while coarse maps carry wider context.
  • Scalable families: nano through large variants trade accuracy for latency, but should be benchmarked on the target runtime.

Limitations

  • Small, crowded, or heavily overlapping objects remain difficult, especially at low input resolution.
  • NMS can discard valid overlapping instances; NMS-free approaches introduce their own assignment constraints.
  • Results across YOLO versions are not directly comparable when datasets, model scales, image sizes, augmentation, or latency measurement differ.
  • Anchor-based versions require suitable anchor settings; anchor-free versions still depend on label assignment and loss design.
  • Export support does not guarantee identical numerical behavior or speed across TensorRT, ONNX, Core ML, and TFLite.

Evolution and Applications

Common applications include defect localization, PCB component checks, package inspection, PPE monitoring, and robotic pick-and-place. Quantization and accelerator-specific compilation can reduce latency, but accuracy and throughput must be measured on representative data and hardware.

Related

Sources