skip to content

Devices & Execution Backends

Transformers.js hands the graph to onnxruntime-web on WASM or WebGPU in the browser, or to onnxruntime-node on a server. Interviewers ask because WebGPU speedups come with real availability caveats.

on this pageshow

explore

questions

5

A Transformers.js demo created with device 'webgpu' throws on some teammates' browsers — what fails, and how should the page choose its device instead?

level: seniorimportance: must knowfreq 11%

answer

  1. explicit device, no safety net
  2. supported list built from navigator checks
  3. Unsupported device error names the list
  4. adapter can still come back null
  5. detect, pass, catch, retry on wasm

basics

~20 s

On a browser without WebGPU, Transformers.js throws Unsupported device: "webgpu", listing the devices it does support; an explicit 'webgpu' never falls back by itself. Detect WebGPU yourself (navigator.gpu plus a non-null adapter), pass 'webgpu' or 'wasm', and retry on 'wasm' if loading fails.

solid answer

~40 s

In a browser the backend lists `'webgpu'` only when `'gpu' in navigator`, after any WebNN devices and before `'wasm'`. An explicit `device: 'webgpu'` that is not on that list throws `Unsupported device: "webgpu". Should be one of: ...` while the session is being built, and nothing retries. Even when `navigator.gpu` exists, `requestAdapter()` can resolve to `null` and ONNX Runtime then fails to initialise the WebGPU provider, which is fatal when it is the only provider. So decide in code: request an adapter, pass `'webgpu'` if one comes back and `'wasm'` otherwise, and catch a failed WebGPU load and rebuild the pipeline on `'wasm'`. `device: 'auto'` avoids the hard error by handing ONNX Runtime the whole list, but it puts WebNN first and makes the default dtype `fp32`, so set `dtype` yourself.

code

javascript · 23 lines
javascript
import { pipeline } from '@huggingface/transformers';

async function pickDevice() {
  if (!('gpu' in navigator)) return 'wasm';
  try {
    const adapter = await navigator.gpu.requestAdapter();
    return adapter ? 'webgpu' : 'wasm';
  } catch {
    return 'wasm';
  }
}

const model = 'mixedbread-ai/mxbai-embed-xsmall-v1';
let device = await pickDevice();
let extractor;
try {
  extractor = await pipeline('feature-extraction', model, { device });
} catch (err) {
  if (device !== 'webgpu') throw err;
  device = 'wasm';
  extractor = await pipeline('feature-extraction', model, { device });
}
console.info('feature-extraction running on', device); // report with your metrics

go deeper

for a junior

Recall that 'webgpu' is an explicit choice that throws when the browser lacks WebGPU, and that 'wasm' is the browser default that always works.

for a middle

Explain how the supported list is built from navigator checks, what the Unsupported device error lists, and why requesting an adapter is a better test than checking the property.

for a senior

Show a load path that detects, chooses, catches and retries, sets dtype per branch, and records the device that actually ran so performance reports can be split.

for a principal

Weigh shipping one code path on WASM everywhere against a WebGPU fast path whose availability you do not control, including the support cost of two behaviours.

## How the browser backend builds its device list When Transformers.js loads in a web environment it builds a **supported device list** once, from feature checks on `navigator`: 1. If `'ml' in navigator`, it adds the WebNN devices `'webnn-npu'`, `'webnn-gpu'`, `'webnn-cpu'` and `'webnn'`. 2. If `'gpu' in navigator`, it adds `'webgpu'`. 3. It always adds `'wasm'`, which is also the default device. A `device` option is checked against that list when a model's inference session is being built, before the ONNX weights file is fetched. ## What an explicit device does when it is missing `device: 'webgpu'` on a browser without the WebGPU API is not on the list, so the backend throws; on a browser that also lacks WebNN the message reads: `Unsupported device: "webgpu". Should be one of: wasm.` There is **no automatic fallback**. The library does not retry on `'wasm'`, does not warn and continue, and does not wait for the API to appear. Because the check runs when the session is built, small files such as `config.json` may already have been fetched by then, but the large weights download has not started, so the failure is cheap to recover from in your own code. This is the usual shape of "the demo works on my machine and throws on half the team's": WebGPU availability varies by browser, version, operating system and flags, and the library's own WebGPU guide notes that global support is not universal. ## When navigator.gpu exists but WebGPU still fails The presence check is shallow. `navigator.gpu` can exist while the browser grants no usable GPU: in the WebGPU API, `navigator.gpu.requestAdapter()` can resolve to `null`. Then ONNX Runtime's WebGPU provider fails to initialise when the session is created. What happens next depends on how many providers were passed: - **One provider** (`device: 'webgpu'`): ONNX Runtime has nothing left and throws `no available backend found`, listing each provider's error. - **Several providers** (`device: 'auto'` or `'gpu'`): ONNX Runtime drops the provider that failed, prints a console warning that it is removing it, and builds the session on the next one that initialises. ## Choosing the device yourself The robust pattern is to decide in your own code and log what you chose: 1. **Feature-detect properly**: check `'gpu' in navigator`, then call `navigator.gpu.requestAdapter()` inside a `try` and treat `null` or an exception as "no WebGPU". 2. **Pass the decision explicitly**: `'webgpu'` when an adapter came back, `'wasm'` otherwise. 3. **Catch the load**: wrap `pipeline()` in `try/catch` and, if the WebGPU attempt fails, create the pipeline again with `device: 'wasm'`. 4. **Set the dtype per device**: with no `dtype`, `'wasm'` loads the 8-bit `q8` file and `'webgpu'` the full `fp32` file, so the two branches download different files unless you choose. 5. **Record the outcome** in your telemetry, so "slow for some users" can be split by the device that actually ran. ## What 'auto' and 'gpu' do, and what they cost | `device` value | Providers handed to ONNX Runtime | Default dtype with no `dtype` | When a provider fails | |---|---|---|---| | unset | `['wasm']` | `q8` | error | | `'wasm'` | `['wasm']` | `q8` | error | | `'webgpu'` | `['webgpu']` if listed, else thrown | `fp32` | error | | `'gpu'` | GPU-class entries of the list | `fp32` | dropped with a warning | | `'auto'` | the whole supported list, in order | `fp32` | dropped with a warning | `'auto'` looks like the answer, and it does avoid the hard failure, but three costs come with it: - **Order is not yours.** WebNN devices are listed before WebGPU, so a browser exposing `navigator.ml` tries WebNN first. - **The default dtype becomes `fp32`** even when the session ends up on WASM, because the per-device default is looked up for `'auto'` itself, not for the provider that wins. - **The fp16 guard is skipped**: the library checks the adapter's `shader-f16` feature only when the device is exactly `'webgpu'`. `'gpu'` has the same costs and no CPU entry: in a browser without any GPU-class device it passes an empty provider list, which ONNX Runtime treats as no preference and resolves by its own registered-backend priority. Explicit detection plus an explicit fallback keeps both the device and the file under your control.

  • Why not simply pass device: 'auto' and let ONNX Runtime sort it out?
    `'auto'` hands ONNX Runtime the whole supported list, so a failing provider is dropped with a warning instead of throwing. But the list puts WebNN devices before WebGPU, the default dtype for `'auto'` is `fp32` even when WASM wins, and the library's fp16 check only runs for an explicit `'webgpu'`. You trade a visible error for a choice you no longer control.
  • The model now loads on 'wasm' for some users. How do you make that visible?
    Keep the device your code finally passed, after any retry, and send it with your timing metrics so slow reports can be split by WebGPU against WASM rather than guessed at. This is one more reason to choose explicitly: with `'auto'`, your own code only knows it asked for `'auto'`, and which provider won appears only in ONNX Runtime's console warning.
  • Is the WebGPU check in Transformers.js done again on every pipeline() call?
    No. The supported list is computed once when the backend module loads, from `'gpu' in navigator` and `'ml' in navigator`. Each call only checks its `device` against that list; an actual adapter is requested later, when ONNX Runtime initialises its WebGPU provider and, for `dtype: 'fp16'`, by the library's `shader-f16` check.

An explicit 'webgpu' is a ticket for one named carriage: if the train has none, you are refused at the gate, not seated elsewhere. 'auto' is an open ticket: you board the first carriage with room, in the train's own order.

  • Transformers.js falls back to WASM automatically when WebGPU is missing.
  • If navigator.gpu exists, WebGPU is guaranteed to work on that machine.
  • device 'auto' always picks WebGPU when the browser supports it.
  • Switching to 'auto' keeps the same model file as 'wasm' would load.
  • Sniff the user agent string to decide whether WebGPU will work.
open as a page

In Transformers.js, which ONNX Runtime package executes the model in a browser versus in Node, and which device does each use by default?

level: juniorimportance: should knowfreq 9%

basics

~20 s

Transformers.js hands the ONNX graph to onnxruntime-web in a browser or worker and to onnxruntime-node under Node. With no device option the browser runs on 'wasm' (WebAssembly on the CPU) and Node on 'cpu' (native code).

open as a page

In Transformers.js, why does the same pipeline download a much larger ONNX file after you switch its device from 'wasm' to 'webgpu'?

level: middleimportance: should knowfreq 7%

basics

~10 s

With no dtype option (and none in the model's config), Transformers.js picks it from the device: q8 on 'wasm', fp32 on every other device including 'webgpu'. Switching device silently swaps onnx/model_quantized.onnx for onnx/model.onnx.

open as a page

How do you make Transformers.js load models only from your own files, in a Node service and in a browser app, instead of from the Hugging Face Hub?

level: seniorimportance: should knowfreq 5%

basics

~20 s

Turn off remote loading with env.allowRemoteModels = false and point env.localModelPath at your files. In Node local loading is already on; in a browser you must also set env.allowLocalModels = true, or every load fails. env.remoteHost can target a mirror instead.

open as a page

When Transformers.js runs on its WASM backend, what do env.backends.onnx.wasm numThreads, proxy and wasmPaths control, and when do you change each?

level: seniorimportance: nice to knowfreq 3%

basics

~20 s

ONNX Runtime Web's WebAssembly flags: numThreads sizes the thread pool (0 lets the runtime decide), proxy runs inference in a runtime-managed worker (off by default), and wasmPaths sets where the runtime's .wasm and .mjs files are fetched.

open as a page