Status: Draft (2026-10-16), collected measurements. Server-side steps are measured in this series with a single 8GB laptop-class GPU at Home Lab, while the operator (server GPU, sharing cards with other tasks) and browser endpoint values cite measurements from the repository
docs/tt/PERFORMANCE.md(2026-10-11–15). On-device times and the network segment are unmeasured. Series 「Reading Vision Models 3」 Part 8 (Final). The model itself is in TT Score Model Anatomy.
Real Budget Table for the TT Score Real-Time Path
Table of Contents
- 1. Tracing the Path Once
- 2. Budget Table
- 3. End-to-End Measurements
- 4. What to Reduce First
- 5. Unmeasured Cells
- 6. Conclusion of the Series
- References
1. Tracing the Path Once
When 「Auto Score」 is turned on in the camera scoreboard (/tt/play/cam), the flow works as follows.
- Phone (Browser): Captures the camera screen at up to 30fps, scales and center-crops it to 512×288, creates a JPEG (quality 0.7) prefixed with an 8-byte 「sequence number + capture timestamp」, and sends it via WebSocket. If 3 frames accumulate without receiving a response, sending is paused.
- Web Server: Checks login, seats, and daily limits, and relays frames directly to the GPU worker. Size and speed limits (36 frames per second) are enforced once more here.
- GPU Worker: Decodes and normalizes the JPEG, building a window from the last 3 frames for each session. Puts the latest windows of waiting sessions into a single batch for the ball model, overwriting backlogged windows with new ones.
- Post-processing: Finds ball blobs in the heatmap of the last frame, chains them with a tracker, detects bounds, nets, and rally ends via rules, and a small trained CNN filters bounces once more. Umpire rules determine the score.
- Response: Ball positions are returned per frame, and bounces/decisions are returned as JSON when they occur for the phone to render.
2. Budget Table
| Step | Location | Measured Value | Source & Conditions | Reduction Method (Part) |
|---|---|---|---|---|
| Capture Camera | Phone | Unmeasured | — | — |
| Scale to 512×288 + JPEG | Phone | Unmeasured (simulated server CPU 0.27 + 0.30ms) | Part 1 of this series | Part 7 (on-device) |
| Upload (JPEG 12–18KB) | Internet | Unmeasured | Size is based on production design measurements | Parts 5 & 7 (sending smaller) |
| Web Server Relay | Server | Unmeasured (not measured separately) | — | — |
| Decode JPEG | Worker CPU | 0.31ms | Part 1 of this series | — |
| Normalize | Worker CPU | 1.35ms | Part 1 of this series | Execute on GPU (Part 1, unmeasured) |
| Build Window + Send to GPU | CPU → GPU | 1.76ms | Part 1 of this series | Move to uint8 (Part 1, unmeasured) |
| Ball Model Inference | GPU | 15.2ms(FP32) · 10.0ms (FP16) · 4.3ms (FP16 + Compile) | Experimental GPU, Parts 2 & 3 | Parts 2 & 3 |
| Same inference, production server GPU | GPU | 10.5ms per window (idle card) · 34ms (with 2–3 other tasks) | Performance docs 2.9.8 | Part 6 (splitting cards) |
| Heatmap to CPU + Blob Finding | GPU → CPU | 0.18 + 0.07ms | Part 1 of this series | — |
| Tracking Table (Optical Flow & Cut Judgment) | Worker CPU | p50 1.72 · p95 1.77ms | Performance docs 2.12 (quiet node) | Part 5 (re-detecting only occasionally) |
| Look-ahead for Bounce Rules | Engine | 3 frames (50ms at 60fps, 100ms at 30fps) | Engine configuration | — |
| Rally End Judgment | Engine | Ball must not be visible for a set duration to end | Engine configuration | Part 1 (late correct answer vs. wrong answer) |
| Receive Response + Render | Internet & Phone | Unmeasured | — | — |
Summing up the GPU worker side gives approximately 19ms, which fits within a single 30fps frame (33.3ms). This aligns with a single real-time worker session handling 30fps.
3. End-to-End Measurements
The sum of steps differs from the actual end-to-end measurements because of queue wait times, thread handoffs, and other tasks on the same card.
| Metric | Measured Value | Conditions (Operations Performance Docs) |
|---|---|---|
| Worker Round Trip, Session 1 | p50 24–28ms · p99 31–32ms, 29.4fps | Client on the same node, sharing card with TTS (4.2) |
| Worker Round Trip, 3 Concurrent Sessions | 16–20fps per session, p50 70–95ms, p99 approx. 127ms | Same as above, 21–30% of sent frames overwritten and dropped (4.2) |
| Browser End-to-End | p50 65 · p90 84 · p99 146ms, 24.6fps | Headless browser fake camera → actual relay → worker, card under contention (4.1) |
| Rally End → Decision Arrives at Browser | Median 0.67–0.90s | Replayer running operations engine code in time units, various public video conditions, camera shake, 60ms round trip, and 15ms worker time set close to production (2.14). Target 1.5s |
The final line is the latency users actually experience. It is over ten times longer than a single frame's round trip (tens of ms). Decisions can only be made after confirming that 「the ball hit the floor or went out」, and distinguishing between a briefly occluded ball and a finished rally requires waiting a certain time for the ball to disappear. In research logs, reducing this wait significantly cuts the p90 of decision latency but slightly lowers winner accuracy, so it has not been changed yet (ongoing research).
This is the same structure as Streaming Perception in Part 1: the trade-off between late correct answers and early incorrect answers, and making the model a few ms faster hardly moves this number.
4. What to Reduce First
| Desired Goal | Largest Leverage | Rationale |
|---|---|---|
| Faster Decisions | Rally End Judgment (Waiting vs. Accuracy) | Section 3 |
| More Sessions per Card | Inference: FP16 + Compile (15.2 → 4.3ms, experimental GPU) | Parts 2 & 3. Need to verify ball position consistency before enabling |
| Reduce Latency Tail | Do Not Share Cards with Other Tasks | Part 6 (10.5 → 34ms) |
| Support Slow Networks | Maintain 512×288 & JPEG Quality, Upload Bandwidth ~3Mbit/s | Part 7 |
| Without Servers | Table Detection On-Device | Parts 4 & 7 |
In Part 2, FP16 ball positions matched FP32 in both position and decisions, but that measurement was based on 388 frames of a single match on an experimental GPU. Enabling it in production requires re-verification under production GPUs and various match/degradation conditions.
5. Unmeasured Cells
- On-device time: Camera capture, scaling, JPEG encoding, rendering. Will vary by device model.
- Latency isolating the internet segment and web server relay. Current endpoint values measure them combined.
- FP16 and compilation effects and accuracy on production GPUs.
- Speed when running the ball model on-device (Part 7).
6. Conclusion of the Series
The most frequent sentence in this series was 「Measuring it showed it was different from expectations」. Multiplying calculations by 8x resulted in a 1.6x speedup (Part 4); batching, which was expected to speed things up when grouped, was actually slower on this GPU (Part 6); and JPEG compression perturbed ball positions more than halving precision did (Part 2). Furthermore, the latency experienced by users was determined by the judgment rule's waiting time rather than the model (this post). Papers and documentation show directions, but numbers must be measured on your own model, your own device, and your own usage patterns.
TT's Note. It only takes me 15ms to find the ball, but it takes about 0.8 seconds for the scoreboard to change because I watch for a moment to see if the ball really landed. Better safe than rushing and getting it wrong!
References
- Our Measurements: Empirical data from Parts 1–7 of this series (raw data in research logs outside the repository), repository
docs/tt/PERFORMANCE.md2.9.8·2.12·2.14·4.1·4.2 - TT Score Model Anatomy Series: README
- Li, M., Wang, Y.-X., Ramanan, D. (2020). Towards Streaming Perception. ECCV 2020. https://arxiv.org/abs/2005.10420