Object Detection 2: The One-Stage YOLO Series, Real-Time Performance, and Licensing

Status: Draft (2026-10-13), partially measured. Grid and AP examples are measured directly on a laptop CPU(code/06_yolo_ap.py, under 1 second). Paper metrics are cited from the original texts along with their conditions, and hardware is noted by tier. License details were verified against each repository's LICENSE file and official documentation on 2026-10-13 and do not constitute legal advice. Writing rule: Measured data is the default; unmeasured items are marked as "Unmeasured."

Table of Contents

• 1. Looking Only Once
  • Try It Yourself: Which Cell Is Responsible
  • Loss Design
  • Results (Paper)
• 2. Lineage: Anchors, Multi-Scale, Anchor-Free, NMS-Free
• 3. Try It Yourself: Calculating AP
• 4. What Is Real-Time Performance
• 5. Licensing: What AGPL Means for Web Services
• 6. Limitations and Our Project
• References

1. Looking Only Once

Part 5's Faster R-CNN achieved about 5 fps per image. This is because it extracts candidate regions (Stage 1) and crops and classifies each candidate (Stage 2). Redmon et al.'s YOLO (You Only Look Once, 2016) changes the question: "Can we solve detection as a single regression problem?" The idea is to pass the image through a network once so that all bounding boxes and classes emerge simultaneously.

YOLO v1's rules are as follows:

• The 448x448 input is divided into an S×S = 7×7 grid.
• The cell where the object's center fallsis responsible for that object.
• Each cell predicts B=2 boxes(each with x, y, w, h, and confidence) and C=20 class probabilities(VOC) as a set.
• Thus, the output is 7 × 7 × (2·5 + 20) = 7×7×30, totaling 1,470 numbers.

Try It Yourself: Which Cell Is Responsible

In a 448px image, a single cell is 64px. We applied this to a table tennis scene (measured).

선수 몸 중심  (200,260) → 칸 (행 4, 열 3)
공 중심       (330,95)  → 칸 (행 1, 열 5)
라켓 중심     (215,270) → 칸 (행 4, 열 3)   ← 선수와 같은 칸

The centers of the player and the racket fell into the same cell. Because that cell outputs only one setof class probabilities, "person" and "racket" compete. The paper also notes that a single cell predicting a limited number of objects is a weakness, stating it struggles when small objects like flocks of birds are clustered together.

Loss Design

The loss is the sum of squared errors for box coordinates, confidence, and classes. Several tweaks reveal the "why."

• Width and height are compared using square roots. This is because a 10px error in a large box should not be penalized the same as a 10px error in a small box.
• Coordinate errors are multiplied by λ_coord = 5, and confidence errors for cells without objects are multiplied by λ_noobj = 0.5. Because most cells are empty, training would bias toward "everything is empty" unless the voice of empty cells is reduced. This effectively addresses the same issue as focal loss in Part 5 using constant weights.

Results (Paper)

In the error analysis of the same paper, YOLO's most common error type was localization error. On the other hand, it mistook background for objects less often than Fast R-CNN; combining the two raised VOC 2012 to 75.0 (compared to 71.8 for Fast R-CNN alone). Because it sees the entire image at once, contextual mistakes are rare, but because the grid is coarse, localization is less precise.

2. Lineage: Anchors, Multi-Scale, Anchor-Free, NMS-Free

A few design rationales can be pointed out as follows:

• k-means Anchors (v2). Just like the experiments in Part 5, if hand-crafted anchors do not match the data's object shapes, blind spots occur. v2 clustered training boxes by 1 − IoU distance to determine anchors, achieving an average IoU of 61.0 with 5 anchors, comparable to 9 hand-picked ones (60.9). Determining anchors from data makes a small number sufficient.
• Multi-Scale (SSD, v3). Small objects are found in high-resolution maps, and large objects in low-resolution maps. This is the same concept as FPN in Part 5.
• Anchor-Free (YOLOX). Anchors increase hand-tuned parameters (count, size, IoU threshold). By directly predicting a box per cell and dynamically assigning which cell handles which ground truth during training, anchors become unnecessary.
• NMS-Free (v10). NMS is post-processing outside the network, which takes time and fails to separate overlapping objects and duplicates with a single threshold (Part 5). Training the network to map one prediction per ground truth eliminates the need for NMS—a concept first demonstrated by DETR in Part 8. v10 compensated for the reduced training signal when using only one-to-one matching by keeping a one-to-many head active during training.

3. Try It Yourself: Calculating AP

Almost all numbers in detection papers are AP(Average Precision). Suppose there are 5 ground truth balls in a scene and the detector outputs 8 boxes. Ordered by descending score, if a box matches a ground truth with an IoU of 0.5 or higher, it is correct (1); otherwise, it is incorrect (0) (measured).

점수 순 맞음/틀림: [1, 1, 0, 1, 0, 1, 0, 0]
재현율: [0.2, 0.4, 0.4, 0.6, 0.6, 0.8, 0.8, 0.8]
정밀도: [1.0, 1.0, 0.67, 0.75, 0.6, 0.67, 0.57, 0.5]
AP = 0.683

AP is the area under the curve plotting recall (ratio of found ground truths) on the horizontal axis against precision (ratio of correct predictions among those output) on the vertical axis. Precision is smoothed using the "maximum value seen from the right" (interpolated AP across all points). Since only 4 out of 5 ground truths were found, recall stops at 0.8, and the area beyond is 0.

Even with the same 8 boxes, if the score rankingchanges, the AP changes.

AP evaluates not only whether it got it right, but also whether it gave higher scores to correct predictions. A detector with terrible confidence scores will have a low AP even if it outputs the exact same boxes. This is something to keep in mind when reading AP in papers.

4. What Is Real-Time Performance

"Real-time" is a relative term. If a table tennis video is 60 frames per second, the time available for a single frame is 16.7ms, which must encompass decoding, preprocessing, model inference, post-processing, and decision-making. When reading fps in papers, conditions must be examined together.

• Hardware Tier: There is a tenfold difference between datacenter GPUs and mobile phones.
• Precision and Runtime: FP32 PyTorch and FP16 TensorRT can differ by several times even for the same model (YOLOv10 and RT-DETR tables use TensorRT FP16).
• Batching: SSD's 59fps was with a batch size of 8; with batch 1, it was 46fps. Real-time video is typically batch size 1.
• Inclusion of NMS: Whether the measured time covers only the network or includes post-processing as well.

TT Score's ball detection (BlurBall) took 10.5ms for a single window (3 frames) on a server GPU (measured, TT Score Model Anatomy Part 2). Comparisons should be made using "ms per frame and what is included" rather than "fps."

5. Licensing: What AGPL Means for Web Services

The name YOLO is shared across multiple repositories. Licenses vary by repository (verified per LICENSE file on 2026-10-13).

AGPL-3.0 is GPL with the network clause(Section 13) added. If you modify the program and allow users to interact with it over a network, you must provide those users with a way to obtain the source code of the modified version. It differs from GPL in that obligations arise even if you only run it on a server without distributing it. Ultralytics explains on its guide page that using it without AGPL's disclosure obligations (for commercial products, SaaS/API, embedded devices, commercially used fine-tuned models, etc.) requires a enterprise license, and takes the stance that trained models also default to AGPL-3.0. This article cannot judge the extent to which this final claim (that AGPL also applies to weights) holds up legally. However, the mere fact that the company makes such a claim is already a risk factor.

TT Score is a web service. Therefore, for ball detection, it chose WASB and BlurBall, which have MIT licenses, and for table tennis tables, it chose SAM 2.1, which has an Apache-2.0 license, while excluding YOLO-family segmentation models due to AGPL conditions (TT Score Model Anatomy Part 1). This is why licenses are considered with the same weight as accuracy and speed when selecting a model.

6. Limitations and Our Project

• Small Objects. Table tennis balls, which are much smaller than a single grid cell (64px in v1), remain challenging even for one-stage detectors. While multi-scale features have improved this, TT Score finds balls using high-resolution heatmaps instead of bounding boxes (Part 3).
• NMS. Post-processing outside the network makes deployment tricky. In the TT Score posture experiment, OpenCV DNN failed to read the NMS operation inside the ONNX file of the human detector (RTMDet), so we bypassed it by using bounding boxes around the ball instead of human bounding boxes (TT Score Model Anatomy Part 4). This is why NMS-free detectors (v10, the DETR family in Part 8) are so welcome.
• Same name, different model. The name "YOLO" alone does not determine the architecture or the license. You must specify the version number and the repository together.

The next post is about Vision Transformers that dropped convolutions.

TT's Word. We had the same eight boxes, but just by changing the score order, the AP dropped from 0.8 to 0.4. Giving confidently high scores to the correct predictions is also a skill.

References

• Redmon, J., Divvala, S., Girshick, R., Farhadi, A. (2016). You Only Look Once: Unified, Real-Time Object Detection. CVPR 2016. https://arxiv.org/abs/1506.02640
• Liu, W. et al. (2016). SSD: Single Shot MultiBox Detector. ECCV 2016. https://arxiv.org/abs/1512.02325
• Redmon, J., Farhadi, A. (2017). YOLO9000: Better, Faster, Stronger. CVPR 2017. https://arxiv.org/abs/1612.08242
• Redmon, J., Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv. https://arxiv.org/abs/1804.02767
• Ge, Z., Liu, S., Wang, F., Li, Z., Sun, J. (2021). YOLOX: Exceeding YOLO Series in 2021. arXiv. https://arxiv.org/abs/2107.08430
• Wang, A. et al. (2024). YOLOv10: Real-Time End-to-End Object Detection. NeurIPS 2024. https://arxiv.org/abs/2405.14458
• Zhao, Y. et al. (2024). DETRs Beat YOLOs on Real-time Object Detection. CVPR 2024. https://arxiv.org/abs/2304.08069
• GNU Affero General Public License v3.0: https://www.gnu.org/licenses/agpl-3.0.html
• Ultralytics License Guide: https://www.ultralytics.com/license, repository https://github.com/ultralytics/ultralytics
• Sample code and outputs: in this folder, code/06_yolo_ap.py, code/06_yolo_ap.out.txt