Hugging Face Releases 207 WebGPU Kernels and the Fleet Tool

Hugging Face releases a collection of 207 versioned WebGPU kernels and a JavaScript loader to run them from the Hub. A new tool, Fleet, measures correctness and performance in the browser on real GPUs. The company reports gains against ORT WebGPU on an Apple M4, with methodological caveats and expected variations depending on devices and browsers.

Measured Gains Against ORT WebGPU and Their Limits

Hugging Face compared its WebGPU kernels to ORT WebGPU on an Apple M4 GPU, using ONNX Runtime Web version 1.30.0-dev.20260826-b1f76d586a. The tests began with 1,756 cases covering 207 operations and retained 809 cases with matching outputs and deemed reliable times. The company indicates accelerations of 2.57 times on average geometric and 1.90 times at the median, with 629 gains, 176 losses, and 4 ties. In a bilinear Einsum case i,ij,j of size 4096, the Hugging Face kernel executed in 0.136 ms compared to 1,396 ms with ORT WebGPU, presenting an order of magnitude claimed to be over 10,000 times. A line-by-line CumSum on [256, 4096] showed a 301 times difference, with 0.016 ms against 4.784 ms. The measurements focus solely on GPU work, excluding loading, session creation, input transfer, shader compilation, and output reading. The publisher warns that small loads are harder to measure and may benefit from GPU caching, and that these figures compare isolated operations, not entire models. It anticipates variations depending on GPUs and browsers, hence the importance of Fleet to broaden the landscape.

Fleet Organizes Evidence on Real Hardware with Consent

Fleet executes and evaluates kernels directly in the user's browser. With consent, each run adds private evidence that helps identify incorrect results or pathological slowdowns, compare variants, and improve selection rules. The tool aims to aggregate data from a variety of GPUs and browsers to guide optimizations. The stated goal is to rely on broad and real coverage to make kernels faster and more reliable for everyone.

Kernels Published as Complete and Versioned Artifacts

The 207 kernels are published in separate repositories within the webgpu-kernels organization, under the Apache-2.0 license. Each kernel includes a documented interface, WGSL models, correctness tests, benchmark cases, and usage instructions. The repositories embed manifest.json for the operation contract, metadata.json for the identifier and provenance, test.json for functional validation, bench.json for performance evaluation, and .wgsl.jinja to produce shaders corresponding to the request and the device used. This method makes each shader a reusable software component, whose interface can be examined without consulting the WGSL code, and which loads via published versions rather than non-permanent URLs. These kernels can also be used as reference resources for developers creating their own kernels or integrating these operations into runtimes.

The JavaScript Module Allows Direct Execution of Kernels from the Hub

The JavaScript module @huggingface/kernels retrieves, prepares, and launches kernels directly from the Hub. The package can be obtained via npm in preview version. Its usage requires a WebGPU-compatible browser, the availability of which depends on the browser, operating system, GPU, and driver; the presence of the API can be checked by the gpu property of navigator. The getKernel API loads a kernel by repository identifier and contract version, and the returned function is called with typed data and tensor shapes. In the example of bias addition, the output has the shape [2, 3]; the loader infers shape and type from the contract and automatically allocates the output.

Execution Variants and Contract Stability

The kernel ai.onnx.Add serves as an example for element-wise addition with broadcasting. Its documentation describes two inputs, the output shape after broadcasting, compatible types, and several alternatives. When the scale is very small, the cost of GPU transfer can exceed that of computation, but the way to call the operation remains the same for heavier tasks like ai.onnx.MatMul. The available Add kernel includes versions for identical shapes, for vectorized broadcasting, for scalar processing, and for general broadcasting; execution automatically selects the appropriate version without modifying the API. The option version: 1 allows for the selection of a specific kernel contract, which does not correspond to ONNX opset concepts, operator since_version, or model revision, to ensure stability on the JavaScript side.

Initial Scope, ONNX Runtime Collaboration, and Shared Ambition

207 WebGPU kernels and a benchmarking tool in the browser, relying on WebGPU and the WGSL language for executing operations, have been presented. Hugging Face mentions collaborating with the ONNX Runtime team to ensure that its advancements benefit the entire ONNX Runtime Web ecosystem. The publisher considers these 207 kernels as a first step, aiming to provide a shared space on the Hub to examine contracts, compare implementations, verify results, and optimize performance without multiplying shaders in each application. This initiative aligns with the stated desire to make inference fast and accessible in the browser.