From Simulation to Reality: Domain Randomization, Delay and Noise Modeling, System Identification, and Massively Parallel Simulation

Status: Draft (2026-10-18), Measured. The numbers are simulation values directly calculated using Python 3.9.6 and numpy 2.0.2 on a laptop (Apple M1 Max, macOS 26.5). The "reality" in this article is another simulator with modified physical constants, not an actual robot (hardware not measured). Raw data and code: working directory measure/robot-arm-basics/rl/of ttenv.py, ppo.py, exp_rl.py(group sim2real), result out/rl/s2r_*_s{0,1,2}.json. References were accessed on 2026-10-18. Writing rules: Measured data is the default; unmeasured items are marked as "Unmeasured," and unverified items are marked as "Unverified."

Table of Contents

• 1. Intuition: Learning the Simulator's Flaws
• 2. What Actual Table Tennis Robots Did
• 3. Measurement: The "Reality" of the Pendulum Racket
  • 3.1 Setup
  • 3.2 Results
  • 3.3 What We Read
  • 3.4 Ways to Change Observation
• 4. Limitations
• 5. Conclusion
• References

1. Intuition: Learning the Simulator's Flaws

Reinforcement learning policies find every loophole to increase reward (Episode 13). Inside the simulator, those loopholes include parts where the simulator is wrong. If the racket restitution is slightly stronger than in reality, the policy learns to hit gently to match that strong restitution. In reality, the ball hits the net.

As parts 1–10 showed, the physical constants of table tennis are uncertain. The literature's Magnus coefficients vary by over 7 times (Part 6), a difference of 0.05 in the restitution coefficient resulted in a 13 cm landing point difference (Part 7), and camera and controller delays ranged from tens to 100 ms (Part 1). There are roughly four ways to handle the problem of going from simulation to reality (sim-to-real, sim2real).

2. What Actual Table Tennis Robots Did

• Delays must be measured and injected, and if they are wrong, things fall apart. D'Ambrosio et al. 2023 reported that when simulation delay was set to the value measured on hardware, the real-world transfer score averaged 1.83 (out of 2.0), dropping to 1.33 when reduced to 50%, and performing very poorly at 0%, 20%, and 150% (Section III-A2, Figure 4). Deliberately increasing observation delay in reality caused performance to plummet at 150 ms (Section III-B).
• Failing to randomize the restitution coefficient caused a 14% drop. The same paper noted that removing randomization of the table restitution coefficient decreased performance by 14%, increasing the ball restitution coefficient by 2% decreased it by 25%, and increasing table restitution by 8% decreased it by 36% (Section III-A5). Interestingly, the measured restitution coefficient performed worse than the tuned value, and the authors believe the simulator failed to properly model spin, so the tuned value compensated for it.
• D'Ambrosio et al. 2024 randomized observation noise and delay, table and racket damping and friction, and performed system identification for each actuator and joint. The problem of vulnerability to topspin balls in reality was fixed by attaching a thin FiLM adapter with 2.8k parameters, and they noted that breaking rallies into short segments and resetting the robot state at each step was crucial for sim-to-real transfer (Sections II-E1, II-E2, and II-E4). The base policy learned about 2.4 billion steps using 6,000 parallel simulation workers (Section C).
• The origin of randomization. Tobin et al. 2017 reported that an object position detector trained solely in simulation by randomly changing rendering textures achieved 1.5 cm accuracy in reality, and Peng et al. 2017 randomized dynamics constants to transfer a policy for pushing objects with a robot arm without real-world training (abstracts for both papers).

3. Measurement: The "Reality" of the Pendulum Racket

3.1 Setup

The "reality" is a simulator with the following five changes. Each number was chosen to match the scale of uncertainty seen in parts 1–10.

There are four training distributions. All use direct control, shape reward, default PPO settings, 19,200 balls, and 3 seeds, and evaluations are performed on both nominal simulation and "reality" using 1,000 balls not used in training.

Narrow randomization does not include"reality" (restitution 0.72 is out of range, delay of 2 steps is also out of range). Wide randomization includes it. The last row is the case where system identification succeeded perfectly.

3.2 Results

3.3 What We Read

• The policy learned without randomization lost 10%p in "reality." This difference was caused solely by slight variations in restitution, delay, noise, and aerodynamic effects. The ratio of hits on the racket was also 76–83%, lower than nominal simulation (88–94%). During the 20 ms delay, the ball moves about 6 cm, causing timing mismatches.
• Narrow randomization performed best in "reality" (79.5%). This is true even though "reality" was not included in its range. We believe that simply learning in a world where constants fluctuate slightly prevented the policy from overfitting to specific values. Nominal simulation performance (91%) was also higher than no randomization (83%), though with only 3 seeds this difference is not definitive.
• Wide randomization reduced the difference to 3%p, but learned less with the same amount of training. Both simulation and "reality" hovered around 55%, and learning curves were slowly rising until the end of 300 iterations (one seed reached 82%–75%). Wider randomization makes the problem harder, requiring more samples. This is why massively parallel simulation pairs well with sim2real.
• Perfect system identification (74.4%) scored lower than narrow randomization (79.5%). Training directly with "reality" values meant that 20 ms of delay and 5 mm of noise were present from the start, making learning harder and yielding less under the same budget. You can think of it as identification setting the starting point, while randomization helps endure remaining uncertainties. In practice, the two are used together (placing the identified values at the center and randomizing narrowly around them).

3.4 Ways to Change Observation

Randomization is a way of "withstanding what you do not know." There is also a way of "figuring out what you do not know." A separate MuJoCo simulator experiment in the same project reported that when a classical planner used prior knowledge of spin categories ("topspin, backspin, no-spin") and their average spin instead of estimating spin solely from the ball trajectory, return rates were 7–19%p higher depending on conditions (e.g., 77% → 94% when the decision point was 300 ms before impact and spin matched the dataset). Spin categories can be identified from pre-flight signals like the opponent's posture or hitting sound. D'Ambrosio et al. 2024 also noted the inability to accurately read heavy spin as a limitation, writing that opponents exploited that weakness (Section V, item 5). Expanding what the policy can see is sometimes cheaper than widening randomization.

4. Limitations

• "Reality" is also a simulator governed by the same equations. Differences from real robots lie not only in constants but also in model structures (rubber spin transfer, reducer backlash, dropped camera frames) (unmeasured).
• Training volume was kept identical across all settings (19,200 balls). Training wide randomization longer might yield different results (unmeasured).
• With only 3 seeds, most differences between settings overlap within ranges (Part 15).
• The spin category experiment in Section 3.4 is adapted from results of a different task and was not measured for the pendulum racket problem in this article.

5. Conclusion

• Policies fail to the extent that simulators are wrong. In "reality" where five constants were slightly altered, the policy without randomization lost 10%p.
• Narrow randomization performed best for this budget. Wide randomization narrows the gap but requires more training.
• System identification sets the starting point, and randomization helps withstand remaining uncertainties. The two are used together.
• Delays must be measured and included. In Google's experiments, even cutting the delay estimate incorrectly in half dropped scores from 1.83 to 1.33.
• The next step is MuJoCo table tennis arm reinforcement learning currently underway in the same project. Once those results are out, I will continue this series with a "Real Training Results" post.

A Word from TT. Relying solely on the simulator causes balls to hit the net on a real table tennis table. Practicing while slightly perturbing physics helps you panic less on unfamiliar tables!

References

All materials were accessed on 2026-10-18.

• D'Ambrosio et al., Robotic Table Tennis: A Case Study into a High Speed Learning System, arXiv:2309.03315, Sections III-A2, III-A5, III-B, Figure 4.
• D'Ambrosio et al., Achieving Human Level Competitive Robot Table Tennis, arXiv:2408.03906, Sections II-C, II-E, Section V (Limitations).
• Tobin et al., Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World, arXiv:1703.06907, Abstract.
• Peng, Andrychowicz, Zaremba, Abbeel, Sim-to-Real Transfer of Robotic Control with Dynamics Randomization, arXiv:1710.06537, Abstract.
• Rudin et al., arXiv:2109.11978, Abstract.