Counting Table Tennis Datasets Firsthand: OpenTTGames, Extended, T3Set, TT Sounds, RacketVision, LATTE-MV

Status: Draft (2026-10-13), incorporating actual measurements. The numbers in the table were directly measured from dataset copies downloaded on 2026-10-13, using ffprobe and Python standard libraries within a read-only temporary pod (no CPU, GPU). Archive files were not extracted; only the internal file lists and label files were read. Numbers from papers or official pages are noted separately as "Official." Scripts and raw outputs are available in the repository docs/experiments/2026-10-13-tt-dataset-stats/ . Authoring Rules: Measured data is the default; unmeasured items are marked as "Unmeasured."

Table of Contents

• 1. Why Count Firsthand
• 2. At a Glance
• 3. OpenTTGames: The Foundation of Evaluation
  • Video
  • Labels
  • Pitfalls
• 4. Extended OpenTTGames: How Rallies Ended
  • Pitfalls
• 5. T3Set: Training Videos with Audio and Sensors
  • Pitfalls
• 6. TT Sounds: Clean Impact Sounds
  • Pitfalls
• 7. RacketVision: Five Racket Points (Table Tennis Portion)
  • Pitfalls
• 8. LATTE-MV: International Broadcasts and Automatic Tracking
  • Pitfalls
• 9. Bonus: BlurBall Training Data (Public Release)
• 10. Licenses and Our Policy
• 11. How to Recount
• References

1. Why Count Firsthand

Dataset tables in papers are rarely re-verified once published. However, model metrics directly inherit the properties of the data. If frames marked as "no ball" actually contain visible balls, precision artificially drops; if frame rates are mixed, criteria like "displacement per frame" take on different meanings across clips. Therefore, we count firsthand before use. Doing so exposes the hidden traps within the labels.

2. At a Glance

3. OpenTTGames: The Foundation of Evaluation

Released alongside the TTNet paper (arXiv 2004.09927), this dataset forms the basis for nearly all accuracy metrics in TT Score.

Video

All videos are H.264, 1920×1080, 120fps, and lack an audio track(verified via ffprobe across all 12 videos). Each training video is 3.9–10.8GB. Since real-time mobile processing caps out at 60fps, we evaluate by taking only even-numbered frames to downsample to 60fps.

Labels

Each match zip file contains three components.

• ball_markup.json: {"프레임": {"x": …, "y": …}}. If x and y are −1, it means "no ball."
• events_markup.json: {"프레임": "bounce" | "net" | "empty_event"}.
• segmentation_masks/*.png: Masks coloring humans, tables, and scoreboards as channels. Contains the exact same countas the frames with ball markers.

The event sum of 4,271 matches the official page's "4,271 events" exactly, confirming our counting method is correct.

Pitfalls

1. Frames without markers indicate "Unknown," not necessarily "No Ball." Ball markers exist only from 4 frames before to 12 frames after an event (officially). Out of all 644,199 frames, 52,987 frames, or 8.2% have markers. If frames without markers are treated as "no ball," any genuine balls found by the model will be counted as false positives, drastically dropping precision. Our loader (ttscore/data/openttgames.py) separates the two.
2. The "No Ball" labels in test_7 are unreliable. While other test matches have 6–14 "no ball" instances, test_7 alone has 104. Among them, 41 actually show a visible ball (verified visually across 6 chapters). Because of this, BlurBall's precision on test_7 was reported as 0.882, but rises to 0.9937 when those 41 instances are excluded (Chapter 2).
3. Most "net" events do not actually hit the net. Contrary to the name, these markers are mostly attached the moment the ball passes over the net. The horizontal velocity ratio before and after is 0.75–1.01, showing almost no deceleration (Chapter 5). To find balls actually "caught" by the net, you must use Extended's net end label.
4. The distribution of empty_event varies wildly across matches. game_2 has 536 while game_5 has 19. Because these labels require meaning verification before use, we exclude them.

4. Extended OpenTTGames: How Rallies Ended

An extension of the same 12 videos annotating strokes and the causes of rally endings (arXiv 2512.19327, github moamal01/table_tennis_data). This serves as the ground truth for our referee engine.

Labels consist of one JSON file per match, {"120fps 프레임": "값"} . Three types of values are mixed within each file.

The paper reported 282 rally ends, but the sum by cause is 281. Our firsthand count also yielded 281. Among these, 275 rallies where "who served" can be identified are used for evaluation.

Pitfalls

1. Attaching two labels to the same frame causes one to disappear. Since JSON keys are frame numbers, assigning an ending label to a frame that already had a bounce overwrites the bounce (e.g., test_4 frame 4530's bounce becomes right_net). test_2 lacks original events entirely. Therefore, bounce and net ground truth are read from the original OpenTTGames events_markup.json rather than this file.
2. The prefix subject differs depending on the cause. The README states the prefix represents the "player who created the end." For winner and double_bounce, the prefixed player winsthe point, while for others, the prefixed player loses it. Cross-referencing with the preceding stroke side confirmed this (matching 30 out of 34 winners, and 129 out of 140 outs), and for not_hitting_ball, 8 out of 10 corresponded to the player who failed to return.
3. The README lists six techniques, but the data contains eight. smash and chop are additionally present.
4. Left-handed players. test_3 and test_7 each have one, and test_5 has two (per README). Inferring forehand and backhand based on screen direction will lead to errors.

5. T3Set: Training Videos with Audio and Sensors

A dataset recording amateur multi-ball training using two cameras, a racket sensor (9-axis IMU, 100Hz), and coach voice prompts (KDD 2025, Zenodo). The 28GB dataset is provided as split zips (.zip + .z01–.z38); we counted the contents by reading only the central directory of the final piece without merging or extracting them .

Video files (mkv) have audio (AAC 44.1kHz) multiplexed in the same container (verified 2026-10-11). Because it is a rare dataset combining video, audio, and sensors, we used it for the audio experiments in Chapter 6.

Pitfalls

1. There is a typo in a camera folder name. Most are remote(376) and right(354), but romote 1, rIGHT 2, rIght includes a 1. Selecting cameras by folder name will quietly omit 4 instances.
2. The round count differs from the official figure. 387 vs. 380. Empty rounds or duplicates may exist, requiring verification prior to use (cause unverified).
3. Sensor clock and video synchronization drift varies by round. The offset between video filenames (in seconds) and sensor timestamps ranged from −1.5 seconds to +3 seconds. Assumptions like "sensor stroke timestamp ±30ms" cannot be taken at face value.
4. There are no ball coordinate labels. Ball detection precision could only be assessed visually; due to multi-ball training, resting balls on the floor and in baskets accounted for 29% of detections (35 out of 120 frames). Counting only "balls in play" yielded a precision of 0.54.

6. TT Sounds: Clean Impact Sounds

A racket and ball impact sound dataset published by the University of Tübingen (arXiv 2409.11760). A single zip file (845MB) contains original recordings and 15ms segments.

Pitfalls

1. The number of clips and the number of labels do not match. Out of 6,037 wav files, 5,702 are in the label table, and out of 149 original recordings, 115 are pointed to by the label table. Using all files in the folder mixes in unlabeled sounds.
2. The provided split divides the same recording into both sides. Neighboring 15ms segments are included in both training and testing, inflating the score. Dividing by original recording units lowered it by 0.1–0.8%p (Part 6).
3. There are spin labels even for 311 non-racket samples. This is why the total sum of the spin column differs from the README table. You should not assume "has spin = racket sound".
4. It is close-mic recording in a quiet place. It differs from cell phone microphones and table tennis hall reverberation. As seen in Part 6, the effect was not confirmed in actual recordings.

7. RacketVision: Five-Point Racket (Table Tennis Part)

A dataset containing ball positions, racket bounding boxes, and five points in international table tennis, tennis, and badminton broadcasts (arXiv 2511.17045, AAAI 2026). The total for the three sports is 1,672 clips, 435,179 frames (dataset meta). We only downloaded the table tennis portion. Detailed shapes of racket annotations are in Part 4.

Pitfalls

1. Five different fps values are mixed. The same "displacement per frame" means different speeds for each clip. When we downloaded it, we wrote "60fps" in our notes, but counting it this time revealed it was wrong. To use time-based rules (like 0.12 sec bounce intervals), you must read the fps for each clip.
2. The annotation count field in metadata is empty. dataset_info.json The ball_annotated_frames·racket_annotated_frames of is 0 for all 780 table tennis clips. Actual annotations are in clip-specific annotation files.
3. Annotations are sparse. The median interval between annotated frames is 10 frames. Ground truth is not available for every frame.
4. Video rights. Annotations are marked as MIT, but the videos are clipped from professional match broadcasts on YouTube. We use them only for internal research and do not publish the frames.

8. LATTE-MV: International Broadcasts and Automatic Tracking

A dataset that automatically tracks 3D poses of balls, rackets, tables, and players by clipping rallies from international match broadcasts (arXiv 2503.20936, CVPR 2025). Officially, it features 73,222 rally exchanges filtered from about 800 hours of broadcasts, totaling about 26.7 hours.

Pitfalls

1. Ball coordinates are not human labels. They are tracking results from that data pipeline. Even when the ball is occluded, Visibility is 1 and coordinates are filled in (out of 20 frames "missed" by BlurBall, 14 had the ball invisible).
2. It points to the end of the blur mark. Out of 40 frames graded as different from BlurBall, 25 were the other end of the same mark (Part 2). If you evaluate a model with data having different labeling rules, that difference appears as an error.
3. Duplicated frames. Frames nearly identical to the previous frame made up 27% of clip centers (measured on 2026-10-11). Actual fps may be lower than 30.
4. Unknown video rights. The data card is Apache-2.0, but the rights to the original broadcast footage are not specified. Used only for internal testing.

9. Bonus: BlurBall Training Data (Public Portion)

We counted the public portion to see what BlurBall from Part 2 learned from.

Nearly half of the training videos are 25fps. This means it learned from videos with long shutter speeds and heavy ball motion blur, which connects to why the "blur center" label was needed. We were unable to confirm why the video and frame counts differ slightly from the paper. We also do not know if this video list matches the public split or overlaps with OpenTTGames. Therefore, we do not read BlurBall numbers on OpenTTGames as generalization performance.

10. Licenses and Our Policy

TT Lab is a non-commercial service. Therefore, we can also use models trained on non-commercial condition data. Instead, we record which non-commercial data each model was trained on, and retrain with our own data if policies change. When modified data under the same condition (SA) is released, we release it under the same license. Data with unknown original video rights is used only for internal testing, and those frames are not included on screens or in articles. This is also why there are no dataset frame images in this article.

11. How to Recount

# 저장소 docs/experiments/2026-10-13-tt-dataset-stats/
python3 dataset_stats.py <데이터셋 뿌리> > results/dataset_stats_YYYY-MM-DD.txt   # ffprobe, numpy 필요
python3 label_stats.py <라벨 뿌리> > results/label_stats_YYYY-MM-DD.json          # 표준 라이브러리만

Since we read only the lists and label files without extracting compressed files, it finished within 1GB of memory. For split zip (T3Set), standard zipfile cannot read zip files spanning multiple parts, so we use a small tool that directly parses only the central directory of the last part (experiments/sound/splitzip.py).

TT Note: If a ball is visible where it says "No ball", even if the model gets it right, it is graded as wrong. Counting data manually is tedious, but before trusting a model, you must check whether you can trust the ground truth.

References

• OpenTTGames: https://lab.osai.ai/, TTNet https://arxiv.org/abs/2004.09927
• Extended OpenTTGames: https://arxiv.org/abs/2512.19327, https://github.com/moamal01/table_tennis_data
• T3Set: https://zenodo.org/records/15516144, https://github.com/jima-cs/t3set
• TT Sounds: https://github.com/cogsys-tuebingen/tt_sounds
• RacketVision: https://arxiv.org/abs/2511.17045, https://huggingface.co/datasets/linfeng302/RacketVision
• LATTE-MV: https://arxiv.org/abs/2503.20936, https://huggingface.co/datasets/ember-lab-berkeley/LATTE-MV
• BlurBall: https://arxiv.org/abs/2509.18387