Precise Event Spotting: E2E-Spot, T-DEED, AdaSpot — How to Train Models to Hit "Which Frame" and mAP@δ

Status: Draft (2026-10-13), partially measured. Paper numbers were verified with original sources along with conditions (the conference for AdaSpot was confirmed via official repository notations). Hands-on tests use numpy examples on a notebook CPU (code/10_map_delta.py). Event measurements for TT Score are values recorded as "Adopted" in TT Score Model Anatomy Part 5 and the 2026-10-13 research log (ongoing research, outside the repository). The PES model from this part has never been run on table tennis videos (unmeasured). Added on 2026-10-15: Speeds of the public weights for AdaSpot (Tennis small) and E2E-Spot (Tennis, RegNet-Y 200MF) were re-measured on an 8GB GPU and added to Section 6. Table tennis accuracy remains unmeasured as it requires retraining to be meaningful. Series "Reading Vision Models 2" Part 10. Previous part: Small and Fast Ball Tracking. Video model basics are in Part 6.

Table of Contents

• 1. Intuition: Not "What," But "When"
• 2. How to Train
  • E2E-Spot (ECCV 2022): End-to-End Per-Frame Classification
  • T-DEED (CVPRW 2024): Explicitly Predicting "How Many Frames Away"
  • AdaSpot (CVPR 2026): Resolution Where It Matters
• 3. Evaluation: mAP@δ
  • Trying It Out
• 4. Paper Metrics
• 5. Limitations
• 6. Connection to Our Project
• References

1. Intuition: Not "What," But "When"

Video models in Part 6 assigned a single label to a clip, such as "playing table tennis for these 2 seconds." However, scoring requires a different question: Exactly which frame did the ball touch the receiver's table?

The task of finding the "exact frame of an event happening at a single moment" is called Precise Event Spotting (PES) . Unlike action spotting, which tolerates 1–5 second errors like "goals" or "corner kicks" in soccer, PES requires precision within 1–2 frames . Representative tasks include bounces and strokes in tennis, jump landings in figure skating, and water entry in diving.

Two things make this difficult.

• Extreme Imbalance. Event frames make up less than 3% of the total (E2E-Spot paper). This is the ball pixel problem from Part 9 encountered again along the temporal axis.
• Neighboring Frames Look Nearly Identical. A bounce frame and its adjacent frames look almost identical on screen. Spotting "this exact frame" requires capturing minute differences.

2. How to Train

E2E-Spot (ECCV 2022): End-to-End Per-Frame Classification

The architecture of E2E-Spot is intentionally simple.

1. It inserts the Channel Shift Module (GSM) from Part 6 into a lightweight 2D image network (RegNet-Y 200MF, initialized with ImageNet) to extract features per frame while mixing information from neighboring frames.
2. Stacked on top is a single Bidirectional GRU layer to capture the context of the entire clip.
3. It performs a (K+1)-class classificationper frame, where K is the event types (e.g., near-side bounce, far-side stroke, etc.) and +1 is "no event." The loss is softmax cross-entropy.

The core of the training method is, as the name suggests, end-to-end . Instead of training only the head on pre-extracted fixed features, it feeds 100-frame clips (224×224) whole to train the image network concurrently. Imbalance is handled by applying a 5x weight to the event class loss. Augmentations like cropping, color jitter, and mixup are used, and during testing, clips are evaluated with 50% overlap and averaged (paper).

It outperformed methods that train only heads on fixed features by 4–11 mAP at δ=1 (paper). This suggests that pretrained features are tailored for "what" and fail to capture minute differences in a single frame.

T-DEED (CVPRW 2024): Explicitly Predicting "How Many Frames Away"

When relying solely on per-frame classification, frames immediately adjacent to an event also look "almost like an event," causing probabilities to spread out. T-DEED introduces two branches:

• Classification Branch: Per-frame (C+1)-class classification (positive-weighted cross-entropy) similar to E2E-Spot.
• Displacement Branch: Regresses "how many frames away until the real event" for each frame within a radius around the event (mean squared error).

During testing, classification finds approximate locations, and displacement refines the exact frame. The authors reported that this displacement branch is superior to label dilation, which simply broadens surrounding frames to "positive" (paper). The network uses an encoder-decoder structure that reduces and then restores temporal resolution, mixing contexts of various temporal scales.

AdaSpot (CVPR 2026): Resolution Where It Matters

In tennis and table tennis, the subject of interest is a small ball. On a screen reduced to 224px, the ball practically vanishes. However, viewing every frame at high resolution is computationally expensive. AdaSpot employs two branches:

1. Low-Resolution Branch: Views the entire screen at a low resolution.
2. Region of Interest Selection: Generates a "salient region" map using channel averages of low-resolution features, applies spatial-temporal smoothing, selects the smallest bounding box containing sufficient importance, and crops it to 112×112 from the original. This selection is not learned; no separate region of interest labels are required.
3. High-Resolution Branch: Processes only the cropped small region at its original resolution.

Loss is weighted cross-entropy (event weight of 5), with auxiliary GRU heads attached to each of the two branches adding losses (1/3 each). The paper notes that training becomes unstable without these auxiliary losses.

3. Evaluation: mAP@δ

PES scoring works as follows:

1. Predictions (frame, class, confidence) are sorted in descending order of confidence.
2. If the frame difference from a ground-truth of the same class is within δ, it is a match. Each ground truth is paired with only one prediction.
3. The area under the precision-recall curve is AP, and the per-class average is mAP.

δ=1 means "the ground truth frame or its immediate neighbor." At 60fps this is ±17ms, and at 25fps it is ±40ms, meaning the actual time tolerance varies by frame rate even for the same δ.

Trying It Out

code/10_map_delta.py implements this scoring and evaluates three "models" with differing temporal errors against 400 events (5% misses, 5% false positives in wrong locations) (code/10_map_delta.out.txt). Because misses and false positives were sampled randomly per row, compare within the same row .

The same model (2.0-frame error) scores anywhere from 0.04 to 0.93 depending on δ. mAP numbers cannot be interpreted without δ. Furthermore, δ=0 (exact frame) drops to the 0.3 range even with a temporal error of just 0.7 frames. This is why papers have started reporting δ=0 separately (see the AdaSpot table below).

Redundant predictions were also tested. Two less-confident predictions were added within ±1–2 frames for each event.

중복 그대로   δ=0: 0.294  δ=1: 0.654  δ=2: 0.739
NMS(±2) 뒤     δ=0: 0.208  δ=1: 0.690  δ=2: 0.875

Redundancies are counted as "predictions without ground truth," degrading precision. NMS, which removes redundancies within ±2 frames, restored δ=2 from 0.739 to 0.875. Conversely, δ=0 dropped. This happens because the "most confident" prediction left by NMS is not always the exact frame. Post-processing benefits also vary depending on δ.

4. Paper Metrics

A few points of caution.

• The comparison table in the AdaSpot paper re-evaluates other methods using the same post-processing (SNMS), which is why E2E-Spot tennis δ=1 is 97.01, differing from the original paper (96.1). Sources must be aligned when combining tables.
• Tennis δ=1 is already saturated near 97%. Differences show up at δ=0 (AdaSpot 74.02).
• Table tennis is significantly harder. In handheld table tennis (TTA), δ=1 is 69.50, more than 25 points lower than tennis figures for the same family of models. The ball is smaller, events are denser (7.73 per 100 frames), and the camera shakes. The table tennis portion of F3Set contains only 42 rallies, providing scarce training data (paper).

5. Limitations

• Clip-based = Latency. Bidirectional GRU and overlapping clip averaging look at future frames. Real-time use requires waiting until the end of the clip, introducing a 1–2 second latency (estimated). It is better suited for post-rally review.
• Gap Between Sports. Public weights are for tennis, figure skating, etc., with no table tennis weights available. Retraining on table tennis is required, and public table tennis data is mostly non-commercial.
• Video Language Models Cannot Substitute. The F3Set paper notes that while GPT-4V recognized sports types and scoreboards, it failed to capture temporal relationships between stroke directions and frames, and was thus excluded from comparisons (Part 12).

6. Connection to Our Project

• Our engine looks at trajectories, not pixels. TT Score derives bounces through rules from trajectories obtained via ball detection (Part 9) (TT Score Model Anatomy Part 5). A large share of remaining errors comes not from missed bounces, but from false bounces(such as reading strokes as bounces).
• A learned bounce classifier is a miniaturized version of this approach. Using 9 trajectory features looking at the past 16 frames with a 1D convolution (approx. 18k parameters), it outputs the probabilities for bounds, hits, and nets on a per-frame basis. Training adopts the same logic directly: smooth labels with 1 for the bound frame and 0.5 for adjacent frames, weighted BCE counted only within labeled segments, and training with mixed 30fps copies. When this model, used as a board to filter rule-based bounds, was placed into the actual deployment streaming pipeline, the bound F1 (±2 frames) on 7 clean videos from the OpenTTGames test increased from 0.801 to 0.830, and the rally winner accuracy on 627 rallies across 11 degradation conditions improved by +4.9%p (95% CI +2.9…+7.2). This setup was adopted and has been enabled for both live and recorded workers since 2026-10-14 (based on the research log, ongoing research — the log is still outside the repository).
• The trap of zero look-ahead. In initial experiments, when a model with zero look-ahead was tasked with directly selecting bound peaks, it collapsed. Selecting peaks requires the probabilities of future frames, but a model unable to see ahead spread its probabilities aheadby 1–2 frames. Therefore, the trained model is used solely to veto bounds generated by the rules. This is the exact same point as "δ=0 is extremely sensitive to visual error" in the mAP experiment above.
• Recorded video teacher candidates. Since AdaSpot has its code and tennis weights open-sourced under the MIT license, retraining it on table tennis makes it one of 4 "recorded video teacher" candidates. Speeds were measured on 2026-10-15 (8GB-class GPU, random input, median after warmup). The tennis small version (RegNet-Y 200MF dual-branch, 7.58M parameters for inference) processed a 100-frame clip (high-res branch 448×796) in 85.9ms in bf16, 0.86ms per frame , with a peak GPU memory usage of 1.4GB. On the same card, E2E-Spot (RegNet-Y 200MF + GRU, 4.45M parameters, 224×224) achieved 29.2ms in fp16, or 0.29ms per frame. Both are fast enough to be used as teachers for re-evaluating recorded videos. Accuracy after retraining on table tennis is unmeasured (training with table tennis ground truth data comes first).

TT note. When you hear someone say, "97 points in finding bounds!", first ask, "What is δ?" The exact same model might score 4 points at δ=0 and 93 points at δ=4. I graded it myself.

References

• Hong et al., Spotting Temporally Precise, Fine-Grained Events in Video(E2E-Spot), ECCV 2022. https://arxiv.org/abs/2207.10213
• Xarles et al., T-DEED: Temporal-Discriminability Enhancer Encoder-Decoder for Precise Event Spotting in Sports Videos, CVPR 2024 Workshops. https://arxiv.org/abs/2404.05392
• Xarles et al., AdaSpot: Spend Resolution Where It Matters for Precise Event Spotting, CVPR 2026 (repository notation). https://arxiv.org/abs/2602.22073
• Xu et al., Multi-Focus Temporal Shifting for Precise Event Spotting in Sports Videos(TTA table tennis data), arXiv 2025. https://arxiv.org/abs/2507.07381
• Liu et al., F³Set: Towards Analyzing Fast, Frequent, and Fine-grained Events from Videos, ICLR 2025. https://arxiv.org/abs/2504.08222
• Sudhakaran et al., Gate-Shift Networks for Video Action Recognition, CVPR 2020. https://arxiv.org/abs/1912.00381
• Our measurement: Adoption record in the 2026-10-13 research log (ongoing research, outside the repository). Example code and output: code/10_map_delta.py, code/10_map_delta.out.txt