Finding the Ball: From TrackNet to WASB and BlurBall — Heatmaps, Multiple Frames, and the Center of Blur

Status: Draft (2026-10-13), actual measurements reflected. Our figures were measured by running BlurBall pretrained weights without fine-tuning on a server GPU (24GB class) on 2026-10-11–12. Paper figures are marked separately as "[Paper]". The cited original repository codes are referenced by commit and shortened. Writing Rule: Measured data is the default, and unmeasured items are marked as "[Unmeasured]".

Table of Contents

• 1. Why Is It Difficult
• 2. Intuition: Draw a Map, Don't Guess Coordinates
• 3. Lineage: TrackNet → TTNet → TrackNetV2 → WASB → BlurBall
  • TrackNet (2019): Heatmaps and Multiple Frames
  • TTNet (2020): Table Tennis, 120fps, and OpenTTGames
  • TrackNetV2 (2020): Feed 3, Receive 3
  • WASB (2023): Small HRNet and Blob Post-Processing
  • BlurBall (2025–2026): The Center of Blur
• 4. HRNet: Preserving Resolution to the End
• 5. In Our Code
• 6. Measurements: How Well Does It Find?
  • Baseline (2026-10-11)
  • Comparison with Other Models (2026-10-11–12)
  • Under Poor Conditions (2026-10-11–12)
  • Outside of OpenTTGames (2026-10-11)
• 7. Why BlurBall, and What's Next
• References

1. Why Is It Difficult

A table tennis ball is 4 cm in diameter. In a 1080p laboratory video (TT3D) filmed from the side showing the entire table, the measured ball diameter was 11–12px in the center, and when reduced to 512×288 to be fed into the model, it becomes just about 3px. Moreover, it is fast. A ball flying at 70 km/h (approx. 20 m/s) travels 17 cm during a 1/120s exposure and 33 cm during a 1/60s exposure. This means a blurred trace4 to 8 times the ball's diameter is captured. On top of that, players' hands and rackets occlude the ball, and ceiling fluorescent lights and white shoes look just like the ball.

Therefore, general object detectors (bounding box detectors for people and cars) do not work well out of the box. This field has a separate lineage dedicated exclusively to finding balls, and BlurBall, used by TT Score, sits at the end of it.

2. Intuition: Draw a Map, Don't Guess Coordinates

Ball-finding models do not directly output two numbers saying "the ball is at (x, y)". Instead, they draw a black-and-white picture of the same size as the input, i.e., a heatmapIt is a map where each pixel is colored from 0 to 1 representing the "probability that this is the ball". The ground truth is also created as a map with a small Gaussian hill drawn at the ball's location, allowing pixel-by-pixel comparison.

There are three reasons for doing this.

1. Position error is learned per pixel. If you only try to match two coordinates, the signal for "how wrong it is" consists of just two numbers, whereas a map provides tens of thousands of pixels, each giving its own learning signal. The smaller the object, the greater this difference.
2. "No ball" is natural. If the entire map is low, there is no ball. Coordinate regression points somewhere even when there is no ball.
3. There may be multiple candidates. If both the ball and a fluorescent light are painted brightly, both can be kept as candidates, and the one whose trajectory connects smoothly later can be selected.

The second idea is feeding multiple frames togetherLooking at just a single frame, it is impossible to tell whether a white dot is a ball or a fluorescent light. Looking at 3 consecutive frames distinguishes moving objects from stationary ones. This is similar to how humans find a ball more easily in a video than in a still image.

3. Lineage: TrackNet → TTNet → TrackNetV2 → WASB → BlurBall

TrackNet (2019): Heatmaps and Multiple Frames

This is a model created to find balls in tennis broadcasts (arXiv 1907.03698). It extracts features using the first 13 layers of VGG-16 and upsamples them via deconvolution to produce a 640×360 heatmap. It compared a single-frame version (Model I) and a 3-consecutive-frame version (Model II), showing precision and recall jumping from 95.7% / 89.6% to 99.8% / 96.6% (Paper, evaluated on a 7:3 split of the same match). This was the first instance demonstrating the numerical effect of multiple frames.

Decoding is simple: threshold the heatmap at 128 (out of 0–255) to make it black and white, find circles using Hough circle detection, and output the center as the ball only when there is precisely one In the same paper, 10-fold cross-validation on non-mixed matches yielded a recall of 75.7%. The recurring lesson in this field—that a model working well within the same match drops significantly on unseen matches—already emerges here.

The labeling rule is also worth remembering. If the ball was blurred, TrackNet placed a dot at the **latest position (front end)** of the trace. BlurBall takes issue with precisely this rule.

TTNet (2020): Table Tennis, 120fps, and OpenTTGames

This model was created to analyze table tennis matches in real time (arXiv 2004.09927), and the OpenTTGames dataset we use for evaluation was released along with this paper. It stacks 9 consecutive frames as channels (27 input channels) from 120fps Full HD video. It uses a two-stage structure: reducing the full screen to 320×128 to find the approximate location, and then cropping that location from the original resolution for precise localization.

The output is interesting. Instead of a 2D heatmap, it outputs two 1D vectors of lengths 320 and 128(x-axis, y-axis). Splitting a 2D map into 1D distributions for the two axes reduces computation. RTMPose (SimCC) uses the same idea later. The paper reported 97.0% event recognition, 97.5% ball detection, and 6.5ms for the entire pipeline (on a desktop-class single GPU) (Paper). For our 60fps mobile video, this is unmeasured.

TrackNetV2 (2020): Feed 3, Receive 3

This is a version adapted for badminton. By switching to a U-Net architecture and using a Multi-Input Multi-Output (MIMO) approach of feeding 3 frames and receiving 3 heatmaps at once, it increased speed from 2.6 to 31.8 FPS (Paper abstract). The input size of 288×512 and "feeding 3 and receiving 3" were directly inherited by WASB and BlurBall.

WASB (2023): Small HRNet and Blob Post-Processing

This is a baseline model created to be used generally across five sports: soccer, tennis, badminton, volleyball, and basketball (BMVC 2023, arXiv 2311.05237, code MIT). Table tennis is not among those five. Its backbone is a small HRNet (Section 4 below) with 1.5 million parameters. The input is (B, 9, 288, 512), meaning RGB 3 channels × 3 frames, and the output is 3 heatmaps passing through a sigmoid.

Post-processing uses connected component blob findinginstead of TrackNet's circle detection. Here is the relevant part from the original repository:

# nttcom/WASB-SBDT src/detectors/postprocessor.py @923462ca, 49–59행(MIT, NTT Communications)
if np.max(hm) > self._score_threshold:
    th, hm_th        = cv2.threshold(hm, self._score_threshold, 1, cv2.THRESH_BINARY)
    n_labels, labels = cv2.connectedComponents(hm_th.astype(np.uint8))
    for m in range(1,n_labels):
        ys, xs = np.where( labels==m )
        ws     = hm[ys, xs]
        if self._use_hm_weight:
            score  = ws.sum()
            x      = np.sum( np.array(xs) * ws ) / np.sum(ws)
            y      = np.sum( np.array(ys) * ws ) / np.sum(ws)

It thresholds at 0.5, calculates the centroid for each blob weighted by heatmap values, and outputs the sumof values as the score. Using the sum favors large and dark blobs. Although the tracker connecting frames is described in the paper as predicting the next position using "position + velocity + acceleration/2", the public code does not use prediction and instead picks the candidate with the highest score within 300px (based on original resolution) from the previous position. When the paper and code differ, the code is what actually runs.

BlurBall (2025–2026): The Center of Blur

This is a model adapting WASB specifically for table tennis (arXiv 2509.18387, CVPRW 2026, code MIT). The changes are primarily in the labels.

• If the ball is blurred, a dot is placed at the centerof the trace. The endpoint (TrackNet method) fluctuates back and forth depending on the exact moment the shutter closes, whereas the center represents the midpoint of the exposure time and remains consistent. Since it points to the "ball at the same time point" in every frame, the trajectory becomes smooth.
• The blur's direction θ and half-length l are also labeled together. The ground truth heatmap is not a single dot but an elongated hill covering the entire trace.
• Channel attention (SE blocks) was added to every backbone block, and the loss was changed to quality focal loss.

Input and output shapes are identical to WASB ((B, 9, 288, 512) → 3 heatmaps). The blur length and angle are read via post-processing from the shape of the heatmap blob, rather than using separate output heads.

# cogsys-tuebingen/BlurBall src/detectors/blurball_postprocessor.py @2f0f5496, 48–50·60–71행에서 줄임(MIT)
angle = 0.5 * np.arctan2(2 * moments["mu11"], moments["mu20"] - moments["mu02"])
mean, eigenvectors = cv2.PCACompute(coords.astype(np.float32), mean=None)
principal_axis = eigenvectors[0]                 # 가장 긴 축
projected = np.dot(coords - mean, principal_axis)
l = projected.max() - projected.min()            # 79행에서 l / 2 를 반길이로 저장

There are also two discrepancies between the paper and the code here. While the paper describes blob reliability as the averageof the heatmap, the code uses the sum, just like WASB. The angle is also obtained using principal axis vectors in the paper, but via image moments in the code.

The paper's figures (custom table tennis test set, 288×512, desktop-class GPU) are frequently cited together as F1 96.52 and 79 FPS, but these two numbers are values from different settings. 96.52 is 1-step inference sliding the window by one frame at a time, and 79 FPS is 3-step inference sliding by three frames at a time (3-step F1 is 96.16). The FPS for 1-step is not in the paper's table.

4. HRNet: Preserving Resolution to the End

The backbones of WASB and BlurBall belong to the HRNet family (CVPR 2019, arXiv 1902.09212). Typical classification models compress images down by 1/2, 1/4, …, 1/32 to figure out "what it is". A 3px ball becomes smaller than a single pixel even after just a 1/8 reduction. Even if upsampled again, the location becomes blurred.

HRNet maintains the original resolution branch all the way to the end, adds lower-resolution branches one by one alongside it running in parallel, and repeatedly fuses information across branches. The low-resolution branch handles broad context like "there is a person and a table around here," while the high-resolution branch handles "precisely this pixel." Our configuration uses 4 branches with 16, 32, 64, and 128 channels, setting the initial stem stride to 1 to feed the input resolution directly into the branches (in our code, "STEM": {"STRIDES": [1, 1]}).

5. In Our Code

Model code (MIT) is not included in the repository but read from the server's storage space. This is because importing someone else's code requires us to maintain and patch our own copy of it. Only its wrapper exists in the repository (live/blurball.py, batch/model_blurball.py).

Weight Verification. Downloaded weights are read strictly torch.load(weights_only=True) after verifying their sha256. PyTorch checkpoints are pickles and can execute arbitrary code. If the verification fails or if weights_only If it cannot be read as , startup is stopped.

Preprocessing. RGB, 512×288, normalized using ImageNet mean and standard deviation. Since the browser already downscales to 512×288, the server does not resize it. I learned this the hard way once. When I stacked the decoded array by changing axes only to transpose , the tensor became a channels_last layout, causing cuDNN to take a different calculation path than during warmup. The first actual frame after warmup took 12.5 seconds . np.ascontiguousarray I fixed it in one line.

Postprocessing. To find blobs like the original repository, one additional value is produced for on-screen display.

# live/blurball.py blobs() 에서 줄임
_, th = cv2.threshold(hm, SCORE_T, 1, cv2.THRESH_BINARY)          # SCORE_T = 0.5
n, labels = cv2.connectedComponents(th.astype(np.uint8))
for m in range(1, n):
    ys, xs = np.where(labels == m); ws = hm[ys, xs]; sw = float(ws.sum())
    x, y = float((xs * ws).sum() / sw), float((ys * ws).sum() / sw)
    pts = np.stack([xs, ys], 1).astype(np.float64); pts -= pts.mean(0)
    vals, vecs = np.linalg.eigh(np.cov(pts.T))                     # 주축 = 번짐 방향
    pm = pts @ vecs[:, 1]; length = float(pm.max() - pm.min()) / 2.0
    out.append({"x": x, "y": y, "score": sw, "peak": float(ws.max()), "len": length, ...})

score(sum) is used when the tracker selects candidates, while peak(maximum, 0–1) is used for displaying confidence on the screen. The sum exceeds 1 depending on the blob size, making it unsuitable for values shown to humans. Unlike the original repository, the angle is obtained from the principal axis of the covariance.

Difference between real-time and recorded video. For real-time, a sliding window of the last 3 frames is processed for every new frame, and only the last heatmap is used (OUT_INDEX = 2). Recorded video uses non-overlapping 3-frame windows, using all three frames. Therefore, real-time requires three times the GPU workload per frame (Part 1, Section 3).

6. Measurement: How well it detects

Baseline (2026-10-11)

Conditions: 7 OpenTTGames test matches, even frames of original 120fps only (60fps), only 3,483 frames with annotations, correct if within 5px of ground truth at 512×288. Weights are as pretrained.

There were three weak points.

Stationary balls are mostly held in hand before a serve, so the impact on the score is small. Overlapping balls and fast balls are the actual pain points.

The precision of test_7 was uniquely low at 0.882, but it was not the model's fault. This match has 104 "no ball" markers, 41 of which actually show the ball (verified visually across 6 frames). The balls correctly found by the model were graded as "found a non-existent ball." Excluding these 41 places, it is 0.9937. Dataset discussions will continue in Part 7.

Comparison with other models (2026-10-11–12)

Measured under the same conditions using comparison model weights released by the BlurBall author.

There are three takeaways.

1. Even with WASB, the version trained with center labels performs better than the one with endpoint labels. 0.855 versus 0.513 for fast balls. This is direct evidence that blur-center labeling is effective (the fact that OpenTTGames markers are closer to the center also plays a part).
2. Most of the performance gains came from lowering the threshold rather than adding models. Even with BlurBall alone, lowering the threshold to 0.3 increases recall from 0.979 to 0.985. The pure gain from adding WASB is +0.35%p in recall.
3. In clean footage, finding more balls did not increase the score. Out of 57 rallies across 7 test matches, the correct winner counts were 53 for BlurBall, 52 for threshold 0.3, and 52 for the combined model. Lowering the threshold also increases false bounds.

Under poor conditions (2026-10-11–12)

Measured by applying critical degradations to the same test video (ball detection recall).

In dark environments, combining the two models helped significantly (test 57-rally winner 36 → 45), but in front of ball-like distractors, WASB produced many false positives, making it worse. There was no universally winning combination. Moreover, combining them doubles the GPU time (10.5 → 20.0ms per window). Therefore, we currently run only BlurBall by default and are experimenting with the approach of "turning on the second model only when the ball signal is weak" (in progress).

Outside OpenTTGames (2026-10-11)

The last line captures the main theme of this post. The ball coordinates in LATTE-MV are not human labels but the tracing results of its data pipeline, pointing to one end of the blur trace. BlurBall points to the center. 62% of the cases graded as "different" were actually the same ball. If the labeling convention differs, even the same model will be graded differently.

7. Why BlurBall, and What's Next

• Labels suited for 60fps mobile phone videos. The longer the shutter speed, the longer the blur, and that is when the advantage of labeling the center becomes significant.
• Small and fast. 1.49 million parameters, 10.5ms per window. There is room to handle multiple sessions on the same card.
• The code is MIT licensed. However, there are no license markings for the weights and training data. The paper's appendix states that the data was created using "public online videos under Creative Commons licenses," and since the list of training videos is not public, it is unclear whether it overlaps with OpenTTGames. Therefore, we do not interpret our empirical measurements as generalization performance.

For the next steps, we are investigating and experimenting with occlusion-robust models (conditional models activated only during occlusion frames) and fine-tuning with mobile phone videos. We will add the results to this post as they become available (in progress). The biggest challenge is that a ground truth dataset of human labels for mobile phone videos does not yet exist.

Titi's Word: It is the same WASB model, but when we trained it to predict the "center" of the blurred ball instead of the end of the trace, the recall for fast balls jumped from 0.51 to 0.86. That is the power of a single labeling convention.

Read more about the principles. The structure that preserves heatmaps and resolution to the end is covered in Vision Model Reading Parts 1 and 3: U-Net and HRNet, and the heatmap loss and training techniques are organized along with the original papers in Part 4: Loss Functions and Training Techniques.

Reference

• TrackNet: https://arxiv.org/abs/1907.03698
• TTNet: https://arxiv.org/abs/2004.09927
• WASB: https://arxiv.org/abs/2311.05237, https://github.com/nttcom/WASB-SBDT (MIT)
• BlurBall: https://arxiv.org/abs/2509.18387, https://github.com/cogsys-tuebingen/BlurBall (Code MIT; weights and data license unspecified)
• HRNet: https://arxiv.org/abs/1902.09212
• Our measurements: Repository docs/tt/PERFORMANCE.md 2.1, 2.8, 2.9, Real-time Demo Design Document Section 9