YOLO Detection Pipeline: Data to Inference

Classification answers “what is in this image”. Detection answers “what, where, and how many”. The jump is bigger than it sounds: a single forward pass produces 8,400 candidate boxes, and turning those into a clean answer is most of the work.

Prerequisites: CNN architectures and IoU and mAP.

The problem: hard hats on a construction site

A site safety officer wants an alert when someone walks into the excavation zone without a hard hat. A fixed camera watches the entrance. Two classes: hardhat and no_hardhat.

Classification cannot do this. There are usually several people in frame, some wearing hats and some not, and the answer needs to identify which person, so someone can be spoken to.

How one forward pass produces 8,400 boxes

Modern YOLO feeds a 640×640 image into a backbone, then makes predictions at three different strides.

Each of those 8,400 cells outputs 4 box numbers, 1 objectness score, and one score per class. With 2 classes that is 7 numbers × 8,400 = 58,800 values from a single forward pass.

graph TD
  A["640×640 image"] --> B["Backbone (CSPDarknet)"]
  B --> C["80×80 features<br/>stride 8"]
  B --> D["40×40 features<br/>stride 16"]
  B --> E["20×20 features<br/>stride 32"]
  C --> F["Neck: PANet<br/>fuses the three scales"]
  D --> F
  E --> F
  F --> G["Head: 8,400 predictions<br/>box + objectness + classes"]
  G --> H["Filter by confidence"]
  H --> I["Non-maximum suppression"]
  I --> J["Final boxes"]

Decoding one cell’s prediction

Each cell predicts an offset from its own position rather than an absolute coordinate. That keeps the numbers small and easy to learn.

Take cell $(12, 7)$ on the 80×80 grid, which has stride 8. Suppose it predicts offsets $t_x = 0.35$, $t_y = 0.62$ and sizes $w = 68$, $h = 154$ px:

$c_x = (12 + 0.35) \times 8 = 12.35 \times 8 = 98.8$ $c_y = (7 + 0.62) \times 8 = 7.62 \times 8 = 60.96$

So the box centre is at pixel $(98.8, 61.0)$, and the corners are:

$x_1 = 98.8 - 34 = 64.8, \quad y_1 = 61.0 - 77 = -16.0 \rightarrow 0$ $x_2 = 98.8 + 34 = 132.8, \quad y_2 = 61.0 + 77 = 138.0$

The negative $y_1$ is clamped to 0 — a person whose head is cut off by the top of the frame.

Getting the labels right

This is where most projects fail silently. YOLO wants one .txt per image, one line per object:

class_id  cx  cy  w  h

All four numbers normalised to 0–1 by image width and height, and the centre, not the corner.

Convert a box drawn at pixel corners $(98, 61)$ to $(210, 305)$ in a 640×640 image:

$c_x = \frac{98 + 210}{2} = 154 \;\rightarrow\; \frac{154}{640} = 0.2406$ $c_y = \frac{61 + 305}{2} = 183 \;\rightarrow\; \frac{183}{640} = 0.2859$ $w = 210 - 98 = 112 \;\rightarrow\; \frac{112}{640} = 0.1750$ $h = 305 - 61 = 244 \;\rightarrow\; \frac{244}{640} = 0.3813$

The label line is:

1 0.240625 0.285938 0.175000 0.381250

def voc_to_yolo(x1, y1, x2, y2, W, H):
    return ((x1 + x2) / 2 / W, (y1 + y2) / 2 / H,
            (x2 - x1) / W,     (y2 - y1) / H)

# Always verify by drawing the boxes back onto the images
import cv2
def draw_labels(img_path, label_path):
    img = cv2.imread(img_path); H, W = img.shape[:2]
    for line in open(label_path):
        c, cx, cy, w, h = map(float, line.split())
        x1 = int((cx - w/2) * W); y1 = int((cy - h/2) * H)
        x2 = int((cx + w/2) * W); y2 = int((cy + h/2) * H)
        cv2.rectangle(img, (x1, y1), (x2, y2), (0, 255, 0), 2)
    return img

Non-maximum suppression, worked out

After confidence filtering you still have several boxes on the same person. non-maximum suppression Keep the highest-scoring box, remove every remaining box that overlaps it beyond an IoU threshold, and repeat with what is left. cleans that up.

Four surviving detections:

Sort by score: A (0.92), B (0.85), C (0.78), D (0.61). Keep A. Now compute IoU against each remaining box.

A versus B. The intersection rectangle is the overlap of the two:

$x_1 = \max(100, 110) = 110, \quad y_1 = \max(100, 105) = 105$ $x_2 = \min(200, 205) = 200, \quad y_2 = \min(200, 210) = 200$ $\text{intersection} = (200-110)(200-105) = 90 \times 95 = 8{,}550$ $\text{union} = 10{,}000 + 9{,}975 - 8{,}550 = 11{,}425$ $\text{IoU} = \frac{8{,}550}{11{,}425} = 0.748$

0.748 is above the 0.5 threshold, so B is suppressed.

A versus C. $\max(100,400) = 400$ but $\min(200,480) = 200$, so $x_2 < x_1$ and the intersection is empty. IoU = 0, C is kept.

A versus D.

$\text{intersection} = (198-105)(195-100) = 93 \times 95 = 8{,}835$ $\text{union} = 10{,}000 + 9{,}021 - 8{,}835 = 10{,}186$ $\text{IoU} = \frac{8{,}835}{10{,}186} = 0.867 \;\rightarrow\; \textbf{suppressed}$

Set the NMS threshold too low and two workers standing shoulder to shoulder get merged into one detection. Set it too high and each worker keeps three boxes. For a crowded entrance, 0.6 to 0.7 is often better than the 0.45 default.

Training

from ultralytics import YOLO

model = YOLO("yolov8n.pt")          # nano, pretrained on COCO

model.train(
    data="site.yaml",
    epochs=100,
    imgsz=640,
    batch=16,
    patience=20,
    hsv_h=0.015, hsv_s=0.7, hsv_v=0.4,
    degrees=0.0,           # people are upright — do not rotate
    translate=0.1,
    scale=0.5,
    fliplr=0.5,
    mosaic=1.0,
    close_mosaic=10,       # turn mosaic off for the last 10 epochs
)

site.yaml:

path: /data/site
train: images/train
val: images/val
names:
  0: hardhat
  1: no_hardhat

Two settings deserve explanation. degrees=0.0 because people and hard hats are always upright, so rotation would create training images that cannot occur. And close_mosaic=10 disables mosaic augmentation for the final ten epochs — mosaic stitches four images together, which is great for variety but produces layouts unlike anything at inference, so the model finishes on realistic images.

Reading the results

The class that triggers the alert is the weaker one, for a familiar reason: 223 instances against 981. It is also the class where a miss is expensive.

Use the threshold slider above to watch precision and recall trade against each other. That trade-off is the next decision, and it should be made by cost.

Choosing the confidence threshold by cost

Now assign costs. A missed unprotected worker risks a head injury; a false alarm costs a supervisor thirty seconds to glance at a screenshot. Suppose a miss is worth 100 units and a false alarm 1 unit:

At that cost ratio, 0.15 is correct even though its precision is only 0.52 — half the alerts will be wrong, and that is the right trade. Change the ratio to 5:1 and the answer moves to 0.40. The threshold is a business decision that happens to be implemented in code.

Small objects are the hard part

A hard hat 30 metres from the camera might be 12 pixels across. The finest grid has stride 8, so the object spans one and a half cells and its features have been pooled away several times over.

The last two rows carry the lesson. Repositioning the camera so the far zone is 15 metres instead of 30 doubles every object’s pixel size and beats every algorithmic fix combined. Check the physical setup before optimising the model.

RT-DETR is worth knowing about: it predicts a fixed set of boxes directly and needs no NMS, which removes a whole class of tuning problems in crowded scenes.

Practice task

Use a public hard-hat dataset, or label 300 of your own images.

1. Convert labels to YOLO format and draw them back onto 30 images. Fix whatever is wrong.
2. Train YOLOv8n for 50 epochs. Record per-class precision, recall, and mAP@50.
3. Sweep the confidence threshold over 0.1 to 0.8 and tabulate misses and false alarms per 100 objects.
4. Assign a cost to each error type and compute expected cost per threshold. Which wins?
5. Sweep the NMS IoU threshold over 0.3 to 0.8 on a crowded image. Where do people start merging?
6. Split validation results by object size using COCO’s small/medium/large bands.
7. Retrain at imgsz=1280 and compare small-object mAP against the compute cost.
8. Pick 30 false negatives and look at them. Group them by cause.

Step 8 is where the real information is. You will usually find one dominant cause — heavy occlusion, motion blur, or one particular hat colour — and that group is worth more than any hyperparameter change.

Summary

A YOLO forward pass predicts at three strides — 80×80, 40×40 and 20×20 — for 8,400 candidate boxes. Each cell predicts an offset from itself: cell $(12,7)$ at stride 8 with offsets $(0.35, 0.62)$ decodes to centre $(98.8, 61.0)$.

Labels are normalised class cx cy w h. A box from $(98,61)$ to $(210,305)$ in a 640×640 image becomes 0.2406 0.2859 0.1750 0.3813. Always draw them back and look.

NMS keeps the top box and removes overlaps. You computed IoU 0.748 for A–B and 0.867 for A–D, both suppressed, and 0.0 for A–C, kept — two people from four boxes.

The confidence threshold is a cost decision. At 100:1 cost for a missed unprotected worker versus a false alarm, 0.15 beat 0.40 by 1,235 units of expected cost despite 52% precision.

And objects under 16 pixels sit around 0.21 mAP no matter what you do in software. Moving the camera closer beats every algorithmic fix.

What comes next

A box says roughly where an object is. Sometimes you need its exact outline — to measure area, to separate touching objects precisely, or to composite. Semantic and instance segmentation covers U-Net and Mask R-CNN, where the output is a class per pixel rather than a rectangle, and where the metrics shift from IoU on boxes to Dice and IoU on masks.