skip to content

In Transformers.js, which ONNX Runtime package executes the model in a browser versus in Node, and which device does each use by default?

level: juniorimportance: should knowfreq 9%

answer

  1. two runtime packages, one API
  2. chosen by the environment it loads in
  3. WebAssembly in the page, native on servers
  4. default device differs per side
  5. wasm versus cpu, unless device is set

basics

~20 s

Transformers.js hands the ONNX graph to onnxruntime-web in a browser or worker and to onnxruntime-node under Node. With no device option the browser runs on 'wasm' (WebAssembly on the CPU) and Node on 'cpu' (native code).

solid answer

~40 s

`@huggingface/transformers` does tokenization, pre- and post-processing and the generation loop in JavaScript, but the forward pass runs in ONNX Runtime. Under Node it loads `onnxruntime-node`, a native binding; in a browser, a Web Worker or Deno's web runtime it loads `onnxruntime-web`, ONNX Runtime compiled to WebAssembly with WebGPU and WebNN providers added. Leave `device` unset and the browser uses `'wasm'` while Node uses `'cpu'`. The other devices depend on the environment: the browser offers `'webgpu'` only when `navigator.gpu` exists and the WebNN devices only when `navigator.ml` exists; Node offers `'webgpu'` and `'cpu'`, plus `'cuda'` on Linux x64, `'dml'` on Windows and `'coreml'` on macOS. A device that is not on the list throws instead of falling back.

go deeper

for a junior

Remember the split: onnxruntime-web in browsers and workers, onnxruntime-node on servers. The defaults are 'wasm' in the page and 'cpu' in Node.

for a middle

Explain that a device string names an ONNX Runtime execution provider, which providers each environment lists, and that an unlisted device throws rather than falling back.

for a senior

Show you choose the device deliberately: GPU providers per server OS, the visitor's browser deciding WebGPU, and thread settings living in different places on each side.

for a principal

Frame where inference should run at all: a visitor's browser with unknown hardware against a server whose providers you install and pin, and what each costs to support.

## Two runtimes behind one package **Transformers.js** (the npm package `@huggingface/transformers`) is the JavaScript port of the Hugging Face pipeline idea. It does the model-agnostic work in JavaScript: reading `config.json`, tokenizing text, preprocessing images and audio, running the generation loop and post-processing the outputs. The neural network itself arrives as an **ONNX graph** (a portable, framework-neutral file describing the model's operations and weights), and that graph is executed by **ONNX Runtime**, the inference engine for ONNX models. ONNX Runtime ships for JavaScript as two different packages, and the library's backend module chooses between them when it is loaded: - **Node** (detected when `process.release.name` is `'node'`): **`onnxruntime-node`**, a native binding with prebuilt binaries for each operating system and CPU architecture. The browser package is not bundled into this path. - **Browser, Web Worker or Deno's web runtime**: **`onnxruntime-web`**, ONNX Runtime compiled to **WebAssembly** and imported through its `onnxruntime-web/webgpu` entry, so it also carries the **WebGPU** and **WebNN** execution providers. The native package is not bundled into this path. - **A host that brings its own engine**: if the JavaScript runtime publishes an ONNX Runtime on `globalThis[Symbol.for('onnxruntime')]`, Transformers.js uses that one instead of either package. The two sides are not even the same ONNX Runtime release: Transformers.js 4.3.0's `package.json` pins `onnxruntime-node` 1.30.0 and an `onnxruntime-web` 1.31 dev build. ## Devices are execution providers A Transformers.js `device` string is a thin name for an ONNX Runtime **execution provider**, the component that actually runs operators on some hardware. Most map one to one: `'wasm'`, `'webgpu'`, `'cpu'`, `'cuda'`, `'dml'` (DirectML on Windows) and `'coreml'` (Core ML on macOS). The WebNN family (`'webnn'`, `'webnn-npu'`, `'webnn-gpu'`, `'webnn-cpu'`) maps to the `webnn` provider with a device type attached. Two strings are not providers at all: `'auto'` and `'gpu'` are shorthands the backend expands into a list. ## The default and the supported list | Environment | Runtime | Default `device` | Also on the supported list | |---|---|---|---| | Browser, worker | `onnxruntime-web` | `'wasm'` | `'webgpu'` when `'gpu' in navigator`; the four WebNN devices when `'ml' in navigator` | | Node on Linux | `onnxruntime-node` | `'cpu'` | `'cuda'` (x64 only), `'webgpu'` | | Node on Windows | `onnxruntime-node` | `'cpu'` | `'dml'`, `'webgpu'` | | Node on macOS | `onnxruntime-node` | `'cpu'` | `'coreml'`, `'webgpu'` | How a `device` value is turned into providers: 1. **No `device`** gives the environment's default: `'wasm'` in web environments, `'cpu'` under Node. 2. **A device on the supported list** is passed to ONNX Runtime alone. 3. **`'auto'`** passes the whole supported list, in the order shown; **`'gpu'`** passes only the GPU-class entries (`'webgpu'`, `'cuda'`, `'dml'`, `'webnn-gpu'`). 4. **Anything else** throws `Unsupported device: "<name>". Should be one of: ...`, naming the supported list. Nothing falls back by itself. The source's own comment table marks the Node WebGPU provider as experimental and lists CUDA 12 for the Linux x64 `'cuda'` build, so a server GPU is a deliberate choice, not something the library reaches for. ## Why the split matters in practice - **The CPU path differs in kind.** In a browser, "CPU" means a WebAssembly build inside the page's sandbox; under Node it means native code. The same model and input are not expected to cost the same on both. - **Configuration lives in different places.** The `env.backends.onnx.wasm` flags (threads, proxying, file locations) belong to `onnxruntime-web`. Under Node, thread counts go through the `session_options` object that `pipeline()` passes to ONNX Runtime, for example `intraOpNumThreads`. - **GPU access is a server decision in Node and a visitor's browser decision on the web.** You can install a CUDA build on your own machine; you cannot make a visitor's browser expose WebGPU. - **The device also changes the default precision.** In a clause: `'wasm'` defaults to the 8-bit `q8` file and every other device to `fp32`, so switching device can switch the file you download. - **Web sessions are serialized.** In web environments the backend chains session creation and inference calls one after another, so two pipelines in the same page (or the same worker) do not run in parallel on the WASM or WebGPU backend. ## Checking what you got After `import { env } from '@huggingface/transformers'`, `env.backends.onnx` exposes the ONNX Runtime environment that was loaded, and `env.version` reports the Transformers.js release. Pass `device` explicitly in production code so the choice is visible in review rather than inherited from whichever environment the code happens to run in.

  • Can a Node service get GPU inference from Transformers.js, and how?
    Yes, through `onnxruntime-node`'s native providers: `device: 'cuda'` on Linux x64, `'dml'` on Windows, `'coreml'` on macOS, or the experimental `'webgpu'`. `device: 'gpu'` passes every GPU-class device on the platform's list. None of this is chosen by default; without a `device` option Node stays on `'cpu'`.
  • Two pipelines on the same page both run on WASM. Do their inferences overlap?
    No. In web environments the backend chains session creation and every `run` call onto a single promise chain, because simultaneous sessions are not supported on WASM or WebGPU there. The calls queue one after another; parallelism has to come from elsewhere, such as separate workers each holding its own pipeline.

  • Transformers.js implements the network layers itself in plain JavaScript.
  • The browser picks WebGPU by default whenever the machine has a GPU.
  • Node runs the same WebAssembly build of ONNX Runtime as the browser.
  • An unsupported device quietly falls back to the CPU.
  • Transformers.js in Node can only ever run on the CPU.

Assumes Transformers.js 4.3.0, whose package.json pins onnxruntime-node 1.30.0 and an onnxruntime-web 1.31 dev build; the device lists are read from that release's backend code.