Improvement Directions for the DeepMind Table Tennis Robot, Analysed via Simulator and Original Paper

Status: Draft (2026-10-18), original reading and practical measurement. Paper figures are limited to those re-verified in the respective sections directly from the arXiv HTML version (2408.03906v3) on 2026-10-18, and section and table numbers are used instead of page numbers. Measurements were performed on the same temporary pod as parts 2 and 5 (mini PC node Intel Core Ultra 9 185H, capped CPU 3 cores, 3 GiB memory, no GPU, MuJoCo 3.16.0, time step 1 ms), sharing the CPU with other simulation experiments. All return rates are relative comparison values within the simulator. These values are not measured with a real robot or a real ball. Conditions such as observation noise σ of 3 mm, racket rubber, and decision horizons are assumed values. Iterations use 3 observation noise seeds, and the values in the table are the medians of the success rates per seed. Wall-clock times were 1,702 s for DM1 and 560 s for DM2. Raw data is located in the working directory measure/sim-tt-deepmind/ .

Table of Contents

• 1. Intuition: Where did a robot that reached intermediate amateur level get blocked?
• 2. What the paper built
• 3. Nine limitations identified by the authors (§V)
• 4. Simulator Experiment DM1: Decision Horizon and Spin Knowledge
• 5. Simulator Experiment DM2: How Much Recovery Is Achieved Just by Knowing Spin "Type"?
• 6. Bonus: Rebound for the Same Contact Value Varies Depending on the Integrator (DM3)
• 7. Improvement Directions: Five Things We Will Do First
• 8. What Was Disclosed and What Was Not Disclosed
• 9. Limitations
• 10. Moving on to the Next Part
• References

1. Intuition: Where did a robot that reached intermediate amateur level get blocked?

DeepMind's table tennis robot played matches against humans and reached an "intermediate amateur" level. However, experienced players quickly found its weaknesses: low and strong underspin, fast balls, and backhands. This article verifies the limitations self-reported by the paper using the original text, and directly re-measures spin and decision timing issues within our simulator. As someone building a table tennis robot from scratch, this aims to determine "where to expend effort first."

2. What the paper built

Achieving Human Level Competitive Robot Table Tennis(arXiv 2408.03906, ICRA 2025). Structure confirmed in:

A recall of 0.4 means six out of ten underspin serves were misread as topspin.

3. Nine limitations identified by the authors (§V)

1. Fast balls. Due to lack of data, system latency of about 100 ms, and the reset returning to the same ready posture for every ball.
2. High balls. When exceeding about 6 ft above the table, they leave the camera field of view. The paper notes this was rare in practice.
3. Low balls. Collision avoidance rules protecting the racket prevented it from properly receiving balls coming close to the table, and skilled players frequently targeted this with low underspins.
4. Short-pimpled racket. They chose a rubber that could be modeled with the simulator. Smooth rubber for advanced players is non-linear and has multiple modes, making it extremely difficult to model, as noted.
5. Spin reading. Failure to accurately read extreme spins limited the amount of spin it could handle. It is expected that performance will improve immediately once spin recognition is enhanced.
6. No multi-ball strategy. This is due to learning ball by ball.
7. Motion capture instability. Occasional tracking errors impaired performance.
8. Backhand weaker than forehand.
9. Predictable placement. The top three improvement areas cited by referees were placement, spin reading, and backhand.

Three out of the nine limitations (1 latency, 3 low underspins, 5 spin) bundle into "when and how well spin is known." Therefore, we re-measured this problem first.

4. Simulator Experiment DM1: Decision Horizon and Spin Knowledge

Setup. From DeepMind's publicly released initial ball states (balls coming toward the robot during a rally), 200 were sampled, and trajectories rolled out using MuJoCo with DeepMind MJCF values were treated as "ground truth." Ten balls where the ball center at the impact plane (y = −1.6 m) was 5 cm lower than the table height, and four balls involving the net or double bounce, were excluded, leaving 186 balls. Cameras (120 fps or 240 fps) observe ball positions with a σ of 3 mm noise added per axis, and use observations up to the time minus the decision horizon H(perceptual delay + motion preparation time) from the impact time. The estimator matches position and velocity, and the planner determines the racket face normal and velocity. Spin is provided differently depending on the method: "Ignore spin" is 0, "True spin" is the ground truth value, "Spin error x%" is ground truth mixed with x% error, and "Trajectory fitting" matches up to spin from trajectory curvature. When there is no perceptual error, all 186 balls were returned (100% ceiling).

• Ignoring spin yields 84% even with a short horizon. It hits the ball but miscalculates racket reflection, sending it off the table. When the horizon exceeds the remaining time after the bounce (median 284 ms), it drops to around 50%. Spin is required for both flight prediction and racket reflection.
• Trajectory fitting matched true spin up to a 150 ms horizon. As the horizon lengthened, it dropped to 85.5% at 300 ms and 66.7% at 400 ms. The 240 fps camera showed differences only at longer horizons (87.6% at 300 ms, 76.9% at 400 ms).
• Even knowing true spin, it is 96.2% at 400 ms. The remaining error stems from discrepancies between the estimator's simple bounce model and MuJoCo contact. This is the same story as the "model discrepancy" seen in the baseline of Part 5.

5. Simulator Experiment DM2: How Much Recovery Is Achieved Just by Knowing Spin "Type"?

We separately measured the case of knowing only the spin type, similar to DeepMind's classifier. Types are divided based on DeepMind's appendix criteria (top ωx > 50, back ωx < −25 rad/s, others no spin), and 80 balls were sampled for each type. Since the data spin is weaker than that of advanced players, we also created balls with spin multiplied by 2x and 3x. Horizons are 150 ms assuming a fast arm and 300 ms assuming a slow gantry (both assumptions). "Type + Trajectory" assumes that signals outside the ball (such as opponent posture or hit sound) correctly identified the type, and uses trajectory fitting centered on the mean spin prior for that type.

(This is the combined return rate (%) for topspin, backspin, and no-spin balls. Valid balls are 220 for ×1, 176 for ×2, and 122 for ×3. Increasing spin reduced counts as some balls failed to land on the robot's table side.)

• Adding type hints to trajectory fitting increased performance by 7–19 percentage points across all conditions. Except for true spin, it was the best method.
• Using only the type mean was equal to or worse than trajectory fitting. Type information complements rather than replaces trajectory fitting. If the type was only 80% accurate, the type mean method dropped by 5–13 percentage points.
• The most difficult balls were topspins. At 3x topspin and 300 ms horizon, it was 0% for ignore spin, 32% for trajectory fitting, 53% for type + trajectory, and 84% for true spin. Backspin achieved 91–100% with trajectory fitting alone. This is because under DeepMind's table contact values, the bounce nearly wiped out backspin (average ωx of 1x backspin was −41 before bounce and −5 rad/s after).
• Therefore, DeepMind's backspin weakness experienced in hardware (limitations 3 and 5) might have come from the racket side(racket angle lifting the ball, rubber model, low ball collision avoidance) rather than flight prediction. This is a hypothesis and has not been verified. Since this experiment uses the same racket model for both planner and ground truth, the racket-side effect may have been underestimated. Re-running experiments with table friction changed to literature values and a racket model different from ground truth is the next step.

6. Bonus: Rebound for the Same Contact Value Varies Depending on the Integrator (DM3)

In Section 5 of Part 2, there was a cell (1.081) where the normal restitution of an oblique bounce exceeded 1. Re-measuring 72 cells with 3 time steps, 2 integrators, 2 friction cones, and 6 incidences revealed that the only case exceeding 1 was implicitfast · 1 ms · elliptic cone · 300 rad/s backspin (1.001 at 0.5 ms, 0.984 at 0.25 ms). At the same 1 ms, RK4 was 0.838–0.951 and implicitfast was 0.961–1.081. Since the post/pre-contact ratio of total kinetic energy (translational + rotational) was less than 1 across all 72 cells (0.851–0.996), energy was not created; rather, tangential energy transferred toward the normal direction. The contact values tuned via system identification must be handled together as a single set with the integrator and time interval used at that time.

7. Direction for Improvement: Five Things We Should Do First

Number 1 connects directly to the multi-signal recognition of ball, sound, and pose pursued by TT Lab in scoreboard recognition. Because "knowing the spin type before the ball flies" is the link that leads to robot performance, it is the direction to reproduce first.

8. What Was Open-Sourced and What Was Not

• The public repository is google-deepmind/competitive_robot_table_tennis and has been in an archived state since 2025-04-26. The code is licensed under Apache-2.0, and the data is under CC BY 4.0.
• It contains only the initial ball states (13,088 rallies, 2,704 serves) and MuJoCo ball and table notebooks. There are no policy weights or training code. There are also no robot, gantry, or racket models.
• Therefore, reproducing it by "running the DeepMind robot as-is" is not possible, making it realistic to borrow the ball distribution and physics settings to measure parts, as done in this article.

9. Limitations

• Return rates are all comparisons within a simulator treating MuJoCo as ground truth. They do not predict the return rate of a physical robot.
• Observation noise σ 3 mm, racket rubber (e 0.85, μ 0.6), hitting tolerance radius 6 cm, the assumption of knowing the bounce segment, and horizons of 150 and 300 ms are all assumed values.
• The impact of misidentifying the spin type in the "type + trajectory" method was not measured.
• Estimation computation time (median 77–186 ms when fitting up to spin) was measured using pure Python shared across CPUs, so it should not be used as the basis for a real-time budget.

10. Moving on to the Next Part

Next is placing reinforcement learning on top of Part 5's baseline to see if this article's direction #1 (spin type hint) yields an effect of the same magnitude in a trained policy(Part 7). To spoil the result, at a 100 ms latency, the hint added +10.4%p, but that gain came from the prior values of trajectory fitting rather than the training.

TT's Word. Simply knowing in advance what kind of spin the ball has allowed us to return over 10% more shots. This means that eyes and ears perceiving the opponent's pose and hitting sound can help the robot's hand!

References

All links were verified as accessible on 2026-10-18.

• DeepMind, Achieving Human Level Competitive Robot Table Tennis(ICRA 2025), arXiv HTML version v3: https://arxiv.org/html/2408.03906v3
• Public Repository (Archived State): https://github.com/google-deepmind/competitive_robot_table_tennis(Code Apache-2.0, Initial Ball State © 2025 Google LLC, CC BY 4.0. Data is not redistributed, and only simulation results are presented.)
• DeepMind, Robotic Table Tennis: A Case Study into a High Speed Learning System(RSS 2023): https://arxiv.org/abs/2309.03315
• MuJoCo: https://github.com/google-deepmind/mujoco