Until a Table Tennis Robot Returns a Ball: Calculations Seen Through a Step-by-Step Simulation Video

Status: Draft (2026-10-18), measured. All numbers in the screen and text are values from within the simulator, and camera period, delay, and noise are assumed values. Recording and rendering were performed on 2026-10-18 as a temporary pod on a single mini PC node of the home lab Kubernetes cluster. The pod limits were 6 CPU cores and 3 GiB of memory (peak usage 1.5 GiB), and no GPU was used. The environment consists of MuJoCo 3.16.0, Python 3.11, numpy 2.3.4, Mesa 23.2.1 (llvmpipe, OSMesa), and ffmpeg 4.4.2 (libx264). Physics and control are the same as in Part 7 (time step 0.5 ms, control 1 kHz, learned hybrid policy seed 0). Evaluation ball #201 was run in four variations (learned hybrid delay 30 ms and 100 ms, classical baseline 30 ms, learned-free nominal planner 30 ms). Because it is a deterministic run, each was executed once, and all four results matched the results (result type and landing point) of the same ball and conditions in the Part 7 evaluation file. Raw data (recording out/rec/ball201.json, numbers in the text out/facts.json, and recording, rendering, and plotting scripts) are located in the working directory measure/sim-tt-video/ .

Table of Contents

• 1. Intuition: Seven Calculations Happening Over 1.3 Seconds
• 2. How the Video and Figures Were Created
  • 2.1 Overall Process
  • 2.2 Recording: Extracting Calculations Without Changing Them
  • 2.3 Rendering: Numbers from the Recording, Figures Using a Visualization-Only Model
• 3. Stage 1 Observation: Captured Points Arrive Late
• 4. Stage 2 Prediction: Quadratic Fitting and RK4/Bounce Models
• 5. Stage 3 Strike Decision: Working Backwards from the Landing Point
• 6. Stage 4 Inverse Kinematics: Four Lines of Closed-Form Equations
• 7. Stage 5 Trajectory and Swing: Redrawing the Path in 39 ms
• 8. Stage 6 Contact: Events Within 1 ms
• 9. Stage 7 Landing: 3.7 cm from the Target
• 10. Comparison: Same Ball, Different Methods, and Different Delays
• 11. Experiences While Building
• 12. Limitations and Unmeasured Aspects
• 13. To the Next Part
• References

1. Intuition: Seven Calculations Happening Over 1.3 Seconds

Parts 5 and 7 discussed success rates across 500 balls. While the numbers are accurate, what the robot calculates and when it calculates it while receiving a single ball is not clearly visible. This time, we follow a single ball from beginning to end and expose the values inside the robot's mind at every moment onto the screen. what and when it calculatesis not readily apparent. This time, we follow a single ball from start to finish, bringing the values inside the robot's mind at every moment onto the screen.

It takes 1.307 s from the initial state in the public data (a ball above the opponent's baseline, flying toward our side) until it passes through our robot and lands on the opponent's court. There are seven stages in between.

1. Observation. The camera captures the ball's position, and those points arrive late.
2. Prediction. The arrived points are used to fit the ball state and project the future path.
3. Strike Decision. Working backward from the target landing point, it determines when, where, with what face orientation, and how hard to hit.
4. Inverse Kinematics. That racket posture is translated into four joint values.
5. Trajectory and Swing. Fifth-order polynomials are connected for each joint from the current posture to the target posture, and followed.
6. Contact. The ball hits the racket, and its velocity and spin change.
7. Landing. The ball lands on the opponent's court.

The robot uses the same real-scale 2-axis gantry (x for side, z for height) + 2-axis wrist (yaw, pitch) as in Part 5, and control is the learned hybrid from Part 7. The policy corrects only 6 strike parameters, while all others are handled by the classical calculations shown below.

2. How the Video and Figures Were Created

2.1 Overall Process

The video is 1280×720, 30 fps, 65.6 s (1,967 frames), H.264 2.6 MB. Since videos embedded in a single article are limited to 30 s, stage-specific clips (4.7–8.1 s) were placed in each section, with a still image of the exact moment placed right below each.

2.2 Recording: Extracting Calculations Without Changing Them

The recorded trajectory from Part 7 (rec_*.json) contains only ball and racket positions at 5 ms intervals. To show the step-by-step calculations, we needed observation points, fitted quadratics, prediction paths, plans per decision, and joint reference values, so the same environment was re-run with the same seed (777000 + ball number). The environment code was left unchanged and read from a subclass. States are transcribed at every 1 ms control step, and estimated values and plans are transcribed at every decision.

class RecEnv(ttenv.TTEnv):
    def _tick(self, qr, vr, ar):
        done = super()._tick(qr, vr, ar)          # 물리·제어는 원래 코드 그대로
        d = self.d
        v, w = ball_world(self)                   # 세계 좌표 속도·각속도
        self.ticks.append([round(self.t, 6), *d.qpos[:11].tolist(), *d.qvel[6:10].tolist(),
                           *np.asarray(qr).tolist(), *v.tolist(), *w.tolist(), ...])
        return done

Source: TT Lab experiment code sim-tt-video/src/vid_record.py(contact indicator column omitted).

Recording results (result type, landing point) were checked against the values for the same ball and conditions in the Part 7 evaluation file, matching across all four cases. This confirms that the exact same calculations were re-executed.

2.3 Rendering: Numbers from the Recording, Figures Using a Visualization-Only Model

The robot in the physics model has no geometry other than the racket, so rendering it directly leaves only the racket floating in mid-air. Therefore, a separate visualization-only modelwith identical joint layouts (qpos order, positions, axes, racket size) was created by attaching rails, carriages, wrists, and handles, and the recorded qpos values were fed directly into it to render using MuJoCo's off-screen renderer. This model does not calculate contacts. Indicators such as observation points, prediction paths, and strike points were added to the MuJoCo scene as geometries, and subtitles and numbers were rendered on top of the frames using NanumGothic. All numbers on the screen were read from the recording file, with only direct transformations applied, such as speed (vector magnitude), angles, and cm conversions. Frames between 1 ms records (1.67 ms intervals at 1/20 playback speed) were rendered by linearly interpolating ball and joint positions.

3. Stage 1 Observation: Captured Points Arrive Late

• The camera captures at 120 Hz (approx. 8.3 ms intervals) with position noise of σ 3 mm. In the simulator, capturing is synchronized with the 1 ms control steps, so actual intervals are 8–9 ms.
• Captured points arrive at the robot only after a 30 ms delay. When making the first decision at 0.2 s, the number of captured points was 24, and the number of arrived points was 21. The ball's movement during the last 30 ms is not yet visible to the robot.
• The initial speed of the opponent's hit ball is 7.78 m/s, with a topspin of 53.8 rad/s relative to the direction of travel (initial state from DeepMind open data).

General principles for estimating ball states from observations (such as Kalman filters) can be found in "Basics of Table Tennis Robot Arms", Chapter 6, Ball Flight Equations.

4. Stage 2 Prediction: Quadratic Fitting and RK4/Bounce Models

Calculation. A quadratic equation of position with respect to time is fitted via least squares to arrived points (up to the latest 36), yielding the position, velocity, and acceleration at the last observation time. The fit at 0.2 s used 21 observations, with an acceleration of (1.56, 6.90, −13.49) m/s². Subtracting gravity and drag from the acceleration, the remaining share is treated as the Magnus force (k_M/m) ω×v to solve for ω. Since spin only appears as slight differences in acceleration, a prior term pulling toward 0 (standard deviation 300 rad/s) is applied. With this state, the planner's physics (drag coefficient 0.471, Magnus coefficient, table restitution 0.975, friction 0.10) are simulated using RK4 at 4 ms intervals, bouncing using an impulse model when contacting the table.

The predicted bounce was 14 ms faster and 10.3 cm ahead of the actual. Spin estimation shows the largest discrepancy (as seen in Chapter 6, position alone provides only weak visibility into spin).

Re-fitting After Bounce. The robot detects a bounce when the height of arrived points reaches a minimum and rises again by more than 1 cm. For this ball, 0.425 s was identified as the bounce, and the second decision was made at 0.497 s , after 4 subsequent observations arrived. If fewer than 6 observations are available, acceleration is set to gravity, only a straight line is fitted, and spin directly uses the value passing the first estimate through the bounce model, (131, −25, 4) rad/s (true value: (98, −7, 15)). The equations for this calculation are found in the Gravity, Drag, Magnus, and Bounce Model section of "Basics of Table Tennis Robot Arms", Chapter 6.

5. Stage 3 Strike Decision: Working Backwards from the Landing Point

Calculation. Three items are solved sequentially.

1. Outgoing Velocity. The target landing point is the center of the opponent's court at (0, 0.75) m. Setting the horizontal velocity to 4.5 m/s and determining the flight time T = distance / 4.5, v_z is chosen such that a parabola with only gravity reaches table height.
2. Face Normal and Face Velocity. The direction of the difference between outgoing and incoming velocities is taken as the normal vector n, and the normal component of face velocity v_pn = v_in·n + (Δv·n)/(1+e) is determined using the restitution coefficient e. Expected outcomes are recalculated three times using an impulse model incorporating friction.
3. Hit Time. Since the gantry lacks a depth (y) axis, the y coordinate reachable by the racket is determined solely by the wrist angles. The time when the predicted path intersects that y coordinate is the hit time.

def _v_des(self, b):
    dx, dy = TARGET[0] - b[0], TARGET[1] - b[1]
    dist = math.hypot(dx, dy)
    vh = 4.5
    Tf = dist / vh
    vz = (T.BALL_R - b[2] + 0.5 * G * Tf * Tf) / Tf
    return np.array([dx / dist * vh, dy / dist * vh, vz])

Source: TT Lab experiment code rl/src/ttenv.py(same file as Part 7).

The learned policy adds six corrections to these nominal values (within the ranges of ±20° for face yaw and pitch, ±1.5 m/s for face velocity, ±20 ms for hit time, and ±6 cm for racket center x and z).

Converting the policy output into units gives yaw +4.7°, pitch −4.0°, face velocity +0.63 m/s, hit time +7.0 ms, center x −0.6 cm, and z −2.7 cm. The equations for inversely calculating the racket velocity and normal at the desired landing point are found in Part 7, Hitting Equations, of "Fundamentals of Table Tennis Robot Arms".

6. Step 4 Inverse Kinematics: Four Lines of Closed-Form Equations

Because the gantry uses Cartesian coordinates, inverse kinematics finishes without iterative calculations. The two wrist angles come directly from the desired normal, and the two gantry axes are obtained by subtracting the wrist offset from the racket center.

def ik(c, n, rb: Robot):
    n = np.asarray(n, float) / np.linalg.norm(n)
    th = math.asin(max(-1.0, min(1.0, n[2])))
    psi = math.atan2(-n[0], n[1])
    x = c[0] - rb.L * math.sin(th) * math.sin(psi)
    z = c[2] - rb.L * math.cos(th)
    y_res = c[1] - (rb.y_g - rb.L * math.sin(th) * math.cos(psi))
    q = np.array([x, z, psi, th])
    ok = bool(np.all(q >= np.array(rb.lo)) and np.all(q <= np.array(rb.hi)))
    return q, y_res, ok

Source: TT Lab Experiment Code sim/ttsim.py(explanatory comments omitted).

The racket center is c = b − n·(ball radius + racket thickness/2) plus the policy's center correction. All four values were within limits, so no clipping occurred. y_res is the depth-direction difference that the gantry cannot match; it was 0 in the nominal plan and −2.9 cm in the first target incorporating policy correction. General arm topics such as dual solutions, singularities, and no-solution cases are covered in Part 4, Inverse Kinematics, of "Fundamentals of Table Tennis Robot Arms".

7. Step 5 Trajectory and Swing: Redrawing the Path in 39 ms

Calculation. Every time a decision is made, a quintic polynomial is connected for each joint from the current joint position, velocity, and acceleration to the target at the moment of impact (position q, velocity (0, 0, 0, −v_pn/L), acceleration 0). After the swing, pitch stops over 0.12 s. 1 kHz computed torque control (PD + feedforward, natural frequency 50 rad/s) follows this reference, and accelerations and velocities are clipped at axis limits.

• First plan (0.200 s, 356 ms). The required peak accelerations were 27 m/s² for the horizontal axis (limit 40) and 82 rad/s² for pitch (limit 600), leaving headroom. The horizontal axis peak velocity was 3.13 m/s, slightly exceeding the 3 m/s limit. Up to 0.497 s, the difference between actual and reference was under 0.27 cm for x and 0.20° for pitch.
• Second plan (0.497 s, remaining 39 ms). The hit position recalculated after the bounce shifted x by +4.2 cm and z by −5.2 cm, and the policy corrected pitch by +6.1° and center x by +3.0 cm. As a result, face pitch changed from 0.4° → 8.8°, and face velocity from 1.10 → 0.33 m/s. Connecting a new quintic polynomial over the remaining 39 ms required the horizontal axis to reach 323 m/s²(limit 40) and the vertical axis to reach 98 m/s² (limit 30). The actual motion was clipped at the limits, causing the horizontal axis to deviate from the reference by 6.6 cm at the moment of contact.
• It still hit. The closest distance between the ball and the racket center was 5.0 cm, which fell well within the 15×16 cm racket face. Contact occurred at 0.543 s, which was 7 ms later than the planned 0.536 s.

The principle of quintic polynomials matching positions, velocities, and accelerations at both ends and minimum-time trajectories within limits are covered in Part 8, Trajectory Generation, and computed torque control in Part 9, Control, of "Fundamentals of Table Tennis Robot Arms".

8. Step 6 Contact: Events Within 1 ms

The video shows this scene at 1/10 speed. Contact between the ball and the racket was recorded for 1 ms, which is one log slot (two physics steps).

• The outgoing speed desired by the plan was 5.06 m/s (calculated with gravity only); the actual was 4.89 m/s.
• Just before contact, the pitch joint velocity was −2.11 rad/s, and the velocity pushing the racket face was L·|θ̇| = 0.32 m/s. This is almost identical to the face velocity of 0.33 m/s from the second plan. The speed at the center of the racket face, including the gantry movement, was 0.69 m/s.
• The spin vector changed by only 1.5 rad/s. Because the ball's direction inverted, the incoming topspin turned into backspin upon leaving. Since the restitution and friction of the racket rubber are assumed values (friction 0.5), it remains unknown whether this exact amount of spin will be preserved on a real racket.

9. Step 7 Landing: 3.7 cm from Target

• The ball from the learned hybrid crossed 36.7 cm above the net and fell 0.764 s after contact.
• The plan estimated the peak height to be 0.54 m using a gravity-only parabola, but the actual ball rose to 0.68 m. The Magnus force from backspin is what lifted the ball higher. We can view the policy's correction as compensating for the error in the planning formula. For the same ball, the non-learning nominal planner fell short by 22.1 cm.
• However, this is just a single ball. Across 500 evaluation balls (120 Hz, 30 ms), the net gain added by learning improved success from 80.6% nominally to 83.8% for this policy (Part 7).

10. Comparison: Same Ball, Different Methods and Latencies

Classical baseline (30 ms latency): Missed, closest approach 28.2 cm. The baseline fitted position, velocity, and spin all at once using least squares from 21 observations at 0.2 s, reading the spin as (111, 279, −27) rad/s (true value: (55, 3, 15)). Based on that prediction, it set the hit position to x −0.83 m, while the actual ball was at x −0.54 m at that time. The baseline only replans from 80 ms after the predicted bounce until 80 ms before the hit time. Because the predicted bounce was at 0.440 s and the hit time at 0.545 s, that window was empty, leaving only the single plan from 0.2 s. The hybrid detected the bounce from observations and immediately replanned and corrected as soon as 4 observations were gathered.

Learned hybrid (100 ms latency): Missed, closest approach 32.4 cm. This is the same policy and the same ball. Because there were only 13 observations arrived at 0.2 s (compared to 21 for 30 ms latency), it read the spin as (67, −5, 87) rad/s. Due to a large sidespin component around the vertical axis (z), it set the hit position to x −0.19 m (actual ball at x −0.55 m at that time). Four observations after the bounce only arrived at 0.567 s, and by then the hit time of 0.545 s from that plan had already passed, rendering it unusable. At 100 ms latency, the return success rate of this policy is 58.2% based on 500 balls, whereas the baseline is 26.6% (Part 7).

11. Things Encountered During Development

• The GPU was not empty. One card on the GPU node was being used by another training job. To avoid interfering with it, rendering was done on a CPU node using Mesa's software renderer (OSMesa, llvmpipe). A simple 1280×720 scene took 32 ms, and the final frame including rendering and subtitles took an average of 79 ms (2 minutes and 36 seconds for 1,967 frames).
• Screen projection divided by zero. To attach subtitle leader lines to 3D points, we read the camera's field of view (frustum), but the frustum width right after a scene update was 0. Because this value is filled during the render phase based on the aspect ratio, we calculated it using the same formula (height × width/height ratio).
• Cylinder size argument order was different. We inserted a landing target indicator as a flat cylinder, but it came out as a 4 cm tall can. The size arguments for cylinders placed directly into the scene were in the order of (radius, half-height) instead of (radius, radius, height).
• Observation count was off by one. The 0.2 s screen showed "25 observations captured," but the recording's decision log had 24. The environment captures at the start of each 1 ms period, and decisions are made aheadof that period, so the point captured at 0.200 s does not yet exist at the decision moment. We fixed the screen to follow the same rule and added an assert to verify it matches the decision log.
• The font lacked a minus sign. The mathematical minus (U+2212) appeared as a square box in NanumGothic. We changed the screen characters to hyphens.
• The gantry frame blocked the scene. Initially, we rendered a frame featuring upper and lower rails and columns, but in every view from behind the robot, the top rail crossed the ball's trajectory. We removed the top rail from the presentation-only model, leaving only the bottom rail and moving columns. This has no effect on physics (the robot in the physics model has no shapes other than the original racket).
• Video metadata. FFmpeg embeds the encoder name in a box by default, and x264 writes encoder setting strings inside the video. Because the blog's video checker rejects both, we cleared them using -map_metadata -1 -map_chapters -1 -fflags +bitexact -flags:v +bitexact and -bsf:v filter_units=remove_types=6 , allowing all 9 clips to pass the checker. The full 65.6 s video exceeded the 30 s limit per post, so it is split into step-by-step clips in the article.

12. Limitations and Unmeasured Factors

• It is a single ball. The numbers in this article are values for a single evaluation ball, #201. Other balls differ in replanning timing and degree of hitting axis limits. Please refer to Parts 5 and 7 for distributions over 500 evaluation balls.
• Sim-to-real transfer, actual camera latency, spin estimation accuracy, racket rubber restitution and friction, and axis limitsare all assumed values and unmeasured, just like in Part 7.
• Interpolation. The slow-motion scenes in the video were rendered by linearly interpolating between 1 ms records. Changes within contacts shorter than 1 ms (with a 0.5 ms physics step size) are not visible in the video.
• Display-only shape. The shapes of the rail, carriage, wrist, and handle are for illustration purposes and do not represent the dimensions in the hardware design (Part 3). Only the racket board (15×16 cm, 14 mm thickness) and joint arrangement are identical to the physical model.

13. To the Next Part

Next, we will add low balls and a depth axis that expands the workspace to the robot model and run the same evaluation again (as previewed in Part 7). Using the same recording and rendering tools, we plan to show in the same seven steps how the definition of the "hitting time" changes when a depth axis is introduced.

A word from TT. Following the robot that we only saw as success rate tables with a single ball, there was a scene where it replanned in the final 39 ms and managed to hit the ball into the racket even when demanded 8 times its limit. Such a thrilling moment was hidden behind a single number!

Notes

External links share the same URL as Part 7 (confirmed to open on 2026-10-18 in Part 7).

• MuJoCo: https://github.com/google-deepmind/mujoco
• Initial public ball state: https://github.com/google-deepmind/competitive_robot_table_tennis(© 2025 Google LLC, CC BY 4.0. Data is not redistributed, only simulation results are included.)
• Subtitle font: NanumGothic (SIL Open Font License 1.1, Ubuntu fonts-nanum package)
• Further Reading: Part 5 Classical Control Baseline, Part 6 DeepMind Table Tennis Robot Improvement Directions, Part 7 Table Tennis Arm Reinforcement Learning
• "Basics of Table Tennis Robot Arms": Part 4 Inverse Kinematics, Part 6 Ball Flight Equations, Part 7 Hitting Equations, Part 8 Trajectory Generation, Part 9 Control
• The cited code, figures, and video trajectories are outputs of the TT Lab experimental code.