Running inside Phones and Browsers: WebGPU, WASM, WebNN, Core ML, and What to Do Where

Status: Draft (2026-10-16), mostly based on documentation. There are no measurementsof running our models inside phones and browsers. Standards and browser support were verified on 2026-10-16 using official W3C and browser vendor publications. JPEG sizes and model parameter counts were taken from server measurements in this series. Part 7 of the series "Reading Vision Models 3". The format for carrying models (ONNX) is covered in Part 3.

Table of Contents

• 1. Intuition: Where to Compute
• 2. The Path to Browser Computation
  • WebGL → WebGPU
  • WebAssembly (WASM): Utilizing the CPU Properly
  • WebNN: Device Neural Network Accelerators
  • ONNX Runtime Web: One File, Multiple Backends
• 3. In Apps: Core ML
• 4. Models Already Running on Phones
• 5. Porting Our Model to the Phone: Size and Bandwidth
• 6. Limitations
• 7. Connection to Our Project
• References

1. Intuition: Where to Compute

The camera is on the phone, and the results (scoreboard) appear on the phone screen. If only the model in between sits on the server, every frame requires a phone → server → phone round trip. Moving the model to the phone eliminates this round trip. However, phones have smaller GPUs, heat up under sustained workloads, and vary in performance by up to ten times across models.

2. The Path to Browser Computation

WebGL → WebGPU

TensorFlow.js (Smilkov et al., 2019) ran neural networks using the browser's graphics API (WebGL). It was a workaround that packed tensors into textures and ran a graphics shader once per output element. Even so, the performance difference was substantial.

WebGPU is a new standard that invokes GPU computations directly without borrowing a graphics API. Its shader language is WGSL. As of 2026-10-16, both WebGPU and WGSL are at the "Candidate Recommendation Draft" stage at W3C (versions dated 2026-09-15 and 2026-09-21, respectively). Releases confirmed via official browser vendor posts are as follows.

MDN still marks it as "Limited availability" and restricts its use to secure contexts like HTTPS. 16-bit floating point (f16) has been used on devices with shader support since Chrome 120. Essentially, the FP16 from Part 2 has become viable in browsers as well.

WebAssembly (WASM): Utilizing the CPU Properly

Because offloading to the GPU incurs a fixed overhead, the CPU can win for smaller models. WebAssembly is the path to fast CPU execution in browsers, and enabling SIMD (single instruction, multiple data) and threads makes a dramatic difference. Below is a table from the official TensorFlow.js repository's WASM backend (Chrome 79, older figures from around 2020 hardware).

The same document notes that for lightweight models requiring 20 million to 60 million multiply-accumulate operations, WASM performs similarly to WebGL, whereas for 100 million to 500 million operations, WASM is several times slower. The tradeoff between "invocation overhead versus computation" seen in Part 3 applies equally in the browser.

WebNN: Device Neural Network Accelerators

Modern phones and laptops feature dedicated neural network accelerators (NPUs). These cannot be utilized via WebGPU. WebNN is a W3C standard that passes computational graphs using neural network operation units like "convolution" and "matrix multiplication" so that the browser can execute them on the appropriate hardware among CPU, GPU, or NPU. The 2026-10-10 version is a Candidate Recommendation Draft, and in the current version, developers specify preferences such as "low power" or "high performance" rather than choosing hardware directly. Default release status per browser could not be verified through primary sources.

ONNX Runtime Web: One File, Multiple Backends

This is a runtime for running the ONNX files from Part 3 in the browser. Execution providers include wasm(CPU) andwebgpu·webgl·webnn , and since only WASM supports all operators, operations unsupported by other providers fall back to WASM. The WebGL provider is maintained only for legacy purposes, with WebGPU recommended for performance (according to official documentation). The browser support table in this document appeared older than the vendor posts above—stating that "the runtime has been tested in that browser" is distinct from "the browser has shipped WebGPU."

3. In Apps: Core ML

Apple's Core ML utilizes the CPU, GPU, and Neural Engine concurrently. Setting computeUnits to "All" allows the operating system to select the hardware, while developers can narrow it down to CPU only, CPU+GPU, or CPU+Neural Engine. The model compression tool (coremltools) provides weight palettization (1-, 2-, 3-, 4-, 6-, 8-bit lookup tables), INT4 and INT8 weight quantization, INT8 activation linear quantization, and pruning. The documentation notes that while compression reliably reduces size, it "does not guarantee latency gains" (as some backends dequantize weights for computation) and recommends re-evaluating on your own model and device—reaching the same conclusion as Parts 2 and 4. latency gains are not guaranteed(some backends dequantize weights before computing)

4. Models Already Running on Phones

• BlazePose (Bazarevsky et al., 2020): Achieves approximately 30 FPS with the Lite model for 33 human joints on a single 2017 mobile phone CPU core. The detector runs only on the first frame and when tracking is lost, while tracking regions cropped around previous joints handle the rest (Part 5).
• EfficientTAM (Xiong et al., 2024): 261–1,011ms per input frame on high-end 2023 smartphones for 1024×1024 input, and 80.6ms for 512×512 input (Paper Tables 1 and 5, precision not specified).
• MobileNetV2 (2018): 75ms on a single performance CPU core of a mobile phone (Part 4).

Their common trait is small models + less frequent execution . Achieving real-time performance on phones is more a matter of pipeline design than simply speeding up a single model.

5. Porting Our Model to the Phone: Size and Bandwidth

(Parameter count × 4, 2, 1 bytes. Actual files include additional structural metadata.)

The ball model is small enough to be downloaded alongside a webpage without issue. The challenge is computation. Because this model maintains a 512×288 resolution throughout, it took approximately 10ms per FP16 inference even on a server-grade laptop GPU (Part 1). Whether it can achieve 30fps on a phone GPU was not measured (unmeasured). Since table detection only needs to run once at the start of a match, hundreds of milliseconds on the phone is acceptable, but downloading 34–136MB is required.

Conversely, running on the server as we currently do requires uploading frames. Server measurements from this series showed the following JPEG sizes for the same match footage:

(Based on the server CPU's OpenCV encoder. Browser JPEG encoders may yield different sizes at the same quality setting—operational design documentation measured 12–18KB for match video.) Simply scaling down to 512×288 reduces bandwidth to one-fourth. In table tennis venues with slow upload Wi-Fi, this difference determines whether the system works or fails.

6. Limitations

• There are no measurements of running our models on phones or browsers. The evaluations for phones in this article rely on documentation and published figures for other models.
• Browser support changes rapidly. The table above is based on vendor posts accessed on 2026-10-16, and we could not verify Firefox's release status on other operating systems or WebNN's default release status.
• Public benchmark figures (such as TensorFlow.js tables) reflect devices and browsers from several years ago. Only the general trends (varying by dozens of times depending on the backend) should be taken, while absolute values should not be carried over.

7. Connection to Our Project

• Current division: The phone (browser) captures the camera feed, center-crops and resizes it to 512×288, streams JPEG (quality 0.7) at up to 30fps, and renders the returned ball coordinates and judgments. The server handles JPEG decoding, the ball model, bounds and referee rules, and table detection. This ensures models and rules can be updated in a single place while delivering consistent judgments to everyone.
• The first candidate to move from the phone is finding the table.It happens once at the start of a match so latency is less critical, and succeeding eliminates the "server round-trips while finding the table." The ball model runs every frame, so benchmarking the phone's GPU comes first.
• To migrate it, we need to go through ONNX export from Part 3 and precision verification from Part 2 (checking if ball positions match in FP16). That benchmarking is left as the next task.

The next post is an actual budget table putting all the pieces of this series together into a single real-time TT Score pipeline.

TT's Word. My ball model is 6MB so it fits right into the phone. But nobody has measured yet whether it can actually think 30 times a second inside the phone. I won't say it "works" until I measure it!

References

• Smilkov, D. et al. (2019). TensorFlow.js: Machine Learning for the Web and Beyond. SysML 2019. https://arxiv.org/abs/1901.05350
• TensorFlow.js WASM Backend README. https://github.com/tensorflow/tfjs/blob/master/tfjs-backend-wasm/README.md
• W3C WebGPU. https://www.w3.org/TR/webgpu/ · WGSL. https://www.w3.org/TR/WGSL/ · WebNN. https://www.w3.org/TR/webnn/
• Chrome Developers Blog — WebGPU Launch (113), Android (121), f16 (120). https://developer.chrome.com/blog/webgpu-release
• WebKit Blog — Safari 26.0. https://webkit.org/blog/17333/webkit-features-in-safari-26-0/
• Mozilla Gfx Blog — Firefox 141 WebGPU. https://mozillagfx.wordpress.com/2025/07/15/shipping-webgpu-on-windows-in-firefox-141/
• MDN WebGPU API. https://developer.mozilla.org/en-US/docs/Web/API/WebGPU_API
• ONNX Runtime Web. https://onnxruntime.ai/docs/tutorials/web/
• Apple Core ML (MLComputeUnits), coremltools Compression Guide. https://developer.apple.com/documentation/coreml · https://apple.github.io/coremltools/docs-guides/source/opt-overview.html
• Bazarevsky, V. et al. (2020). BlazePose. CVPR Workshop. https://arxiv.org/abs/2006.10204
• Xiong, Y. et al. (2024). Efficient Track Anything. arXiv. https://arxiv.org/abs/2411.18933