skip to content

Transformers.js

Transformers.js runs ONNX models from the Hub in the browser or Node with the same pipeline() API, on WASM or WebGPU. Interviewers probe download size, first-load cost and when on-device pays.

on this pageshow

explore

questions

10

In Transformers.js, what happens when a web page runs `await pipeline('sentiment-analysis')`, and why is that first call slow?

level: juniorimportance: must knowfreq 12%

answer

  1. an async factory, not a constructor
  2. the alias resolves to a task
  3. config, tokenizer, ONNX weights from the Hub
  4. first visit downloads, later visits hit the cache
  5. onnx subfolder, revision main by default

basics

~20 s

pipeline() is an async factory: it resolves the task, picks a model (the task default if none is given), downloads config, tokenizer and ONNX weights from the Hub, and builds a callable pipeline. That download makes the first call slow.

solid answer

~40 s

`pipeline(task, model, options)` from `@huggingface/transformers` returns a Promise. It maps an alias such as `'sentiment-analysis'` to `'text-classification'`, uses the task's default model when `model` is omitted, works out which files that model needs (`config.json`, `tokenizer.json`, `tokenizer_config.json` and a weight file such as `onnx/model_quantized.onnx`), fetches them from the Hub's `resolve/<revision>/` URLs with `revision` defaulting to `'main'`, and then loads the tokenizer and creates the ONNX Runtime session in parallel. The resolved object is itself callable: `await classifier('I love transformers!')` returns `[{ label: 'POSITIVE', score: 0.99... }]`. The first call is slow because it is mostly a download of the weight file; on later visits the files come from the browser's Cache API, so only session creation remains. In real code pin `model` and `revision` instead of relying on the task default.

go deeper

for a junior

Recall that pipeline() is awaited, takes a task string and an optional model id, and returns something you call with your input. Say that the first call downloads files from the Hub.

for a middle

Walk through the load: alias resolution, default model, the file list, cache lookup then download from the resolve URL, and the session build. Explain why a second visit is faster but not free.

for a senior

Show production habits: pin model and revision, choose the weight variant deliberately, surface progress, and keep session creation and inference off the page thread so the first load does not freeze the UI.

for a principal

Weigh shipping a model in the page against calling a hosted model: first-load size, cache behaviour and device variance against latency, cost and privacy, and decide which features deserve a client-side model.

## What `pipeline()` is **Transformers.js** is the JavaScript port of Hugging Face Transformers, published on npm as `@huggingface/transformers`. It runs **ONNX** models (a portable graph format) in a browser, a Web Worker or Node, through ONNX Runtime. Its `pipeline()` function mirrors the Python one in spirit — one call gives you pre-processing, the model and post-processing — but with one big difference: nothing is installed with the package except code. The weights live in a model repository on the Hugging Face Hub and are fetched at run time. That is why `pipeline()` is **asynchronous**: it returns a Promise that settles only once every file has arrived and the model is ready to run. ## What the first call does, step by step 1. **Resolve the task.** Aliases are applied first, then the task is looked up in the library's task table. An unknown string throws immediately with `Unsupported pipeline: <task>. Must be one of [...]`. 2. **Pick the model.** If you pass no model id, the task's default is used — for `text-classification` in Transformers.js 4.3 that is `Xenova/distilbert-base-uncased-finetuned-sst-2-english`. 3. **Work out the file list.** The library decides which files the model needs: `config.json`, the tokenizer files, and one weight file inside the repo's `onnx/` folder whose name depends on the chosen `dtype` (in the browser the default is a quantized variant, `onnx/model_quantized.onnx`). 4. **Fetch each file.** Every file is first looked up in the cache; on a miss it is downloaded from `https://huggingface.co/<model>/resolve/<revision>/<file>`, with `revision` defaulting to `'main'`, and then stored. 5. **Build the pieces in parallel.** The tokenizer (or processor, for image and audio tasks) is constructed while the ONNX Runtime session is created from the weight bytes. 6. **Return a callable.** You get a pipeline object; calling it runs tokenization, the model and post-processing, and is itself async. ```javascript import { pipeline } from '@huggingface/transformers'; const classifier = await pipeline('sentiment-analysis'); const result = await classifier('I love transformers!'); // [{ label: 'POSITIVE', score: 0.9998 }] ``` ## Task strings and aliases | You write | It builds | |---|---| | `'sentiment-analysis'` | `'text-classification'` | | `'ner'` | `'token-classification'` | | `'asr'` | `'automatic-speech-recognition'` | | `'text-to-speech'` | `'text-to-audio'` | | `'embeddings'` | `'feature-extraction'` | The JavaScript task list is **its own release line**. Python Transformers v5 removed the summarization, translation, text-to-text and question-answering pipelines in favour of chat models, but Transformers.js 4.3 still registers `'summarization'`, `'translation'`, `'text2text-generation'`, `'question-answering'`, `'image-to-text'` and `'image-to-image'`. Porting advice from the Python migration guide does not apply here. ## Why the first call is slow and the second is not - **Most of the wait is the download.** The tokenizer and config are small; the ONNX weight file is tens to hundreds of megabytes depending on the model and the `dtype` variant. - **The browser caches the files.** By default every downloaded file is written to the browser's Cache API, so a returning visitor reads them locally and skips the network. - **Session creation still happens every time.** Even from cache, the runtime has to build an inference session from the bytes, so a second visit is faster, not instant. - **Only ONNX repos load.** A Hub model works only if it ships ONNX weights, by default in an `onnx` subfolder; converting a PyTorch checkpoint is a separate, ahead-of-time step. ## Habits that make it predictable - **Pin the model id.** The task default is a convenience for demos and is a property of the library release, not a promise. - **Pin the revision.** Passing `revision` (a commit or tag) fixes which bytes users get and which cache entry they hit. - **Choose the weight variant on purpose** with `dtype`, because it decides the size of that first download. - **Show progress.** A `progress_callback` turns the silent wait into a loading bar. - **Keep it off the page thread.** Session creation and inference are CPU work; running the pipeline in a Web Worker keeps the page responsive. ## Calling the pipeline you get back The returned object is reused for every input, so create it once and keep it. Calling it is also asynchronous. It accepts a single input or an array: `await classifier(['I love transformers!', 'I hate transformers!'])` returns one result per text. Task-specific options go in the second argument; for text classification `top_k` defaults to `1`, and `{ top_k: null }` returns a score for every label. A feature-extraction pipeline returns a `Tensor` rather than plain objects, and by default it holds one vector per token until you ask for pooling.

  • What does `pipeline('feature-extraction', 'onnx-community/all-MiniLM-L6-v2-ONNX')` return for one sentence, and how do you get a single sentence vector?
    A `Tensor` of per-token hidden states, for example dims `[1, 8, 384]` for an eight-token input, because `pooling` defaults to `'none'` and `normalize` to `false`. Pass `{ pooling: 'mean', normalize: true }` to get dims `[1, 384]`, then `.tolist()` or `.data` for plain numbers. Which pooling a checkpoint expects is a question about the embedding model, not this call.
  • Python Transformers v5 dropped the summarization pipeline. Does `pipeline('summarization')` still work in Transformers.js?
    Yes. Transformers.js is a separate release line, and 4.3 still registers `'summarization'` (default model `Xenova/distilbart-cnn-6-6`), `'translation'`, `'text2text-generation'` and `'question-answering'`. The Python v5 removals and their advice to switch to `text-generation` with a chat model describe the Python library only.
  • What happens if you pass a misspelled task string to Transformers.js `pipeline()`?
    It throws before any download: `Unsupported pipeline: <task>. Must be one of [...]`, listing the registered task names. Aliases are resolved first, so `'sentiment-analysis'` passes while a typo fails fast, which is cheaper than discovering it after a large download.

  • pipeline() returns a ready pipeline synchronously, so no await is needed
  • The model weights ship inside the npm package
  • Any Hub model loads, even one that ships no ONNX weights
  • Python v5 removed summarization, so Transformers.js has no summarization pipeline
  • Every page load downloads the whole model again
  • Relying on the task's default model is fine in production

Assumes Transformers.js 4.3; its task list still includes summarization, translation and question-answering, which Python Transformers v5 removed.

open as a page

An in-browser Transformers.js text-classification widget freezes the page for 40 seconds on first load; how do you move the pipeline into a Web Worker?

level: middleimportance: must knowfreq 10%

basics

~20 s

Create the pipeline inside a module Web Worker, keep the pipeline() promise in a lazy singleton, and forward progress events and results to the page with postMessage. Session creation and inference then run off the main thread.

open as a page

A Transformers.js demo created with device 'webgpu' throws on some teammates' browsers — what fails, and how should the page choose its device instead?

level: seniorimportance: must knowfreq 11%

basics

~20 s

On a browser without WebGPU, Transformers.js throws Unsupported device: "webgpu", listing the devices it does support; an explicit 'webgpu' never falls back by itself. Detect WebGPU yourself (navigator.gpu plus a non-null adapter), pass 'webgpu' or 'wasm', and retry on 'wasm' if loading fails.

open as a page

In Transformers.js, which ONNX Runtime package executes the model in a browser versus in Node, and which device does each use by default?

level: juniorimportance: should knowfreq 9%

basics

~20 s

Transformers.js hands the ONNX graph to onnxruntime-web in a browser or worker and to onnxruntime-node under Node. With no device option the browser runs on 'wasm' (WebAssembly on the CPU) and Node on 'cpu' (native code).

open as a page

In Transformers.js, why does the same pipeline download a much larger ONNX file after you switch its device from 'wasm' to 'webgpu'?

level: middleimportance: should knowfreq 7%

basics

~10 s

With no dtype option (and none in the model's config), Transformers.js picks it from the device: q8 on 'wasm', fp32 on every other device including 'webgpu'. Switching device silently swaps onnx/model_quantized.onnx for onnx/model.onnx.

open as a page

In Transformers.js v3 and later, how does the `dtype` option of `pipeline()` decide which ONNX weight file the browser downloads?

level: middleimportance: should knowfreq 7%

basics

~20 s

dtype picks which prebuilt ONNX file is fetched from the repo's onnx folder: fp32 loads model.onnx, fp16 model_fp16.onnx, q8 model_quantized.onnx, q4 model_q4.onnx. Nothing is quantized in the browser, so a variant the repo lacks fails to load.

open as a page

How do you make Transformers.js load models only from your own files, in a Node service and in a browser app, instead of from the Hugging Face Hub?

level: seniorimportance: should knowfreq 5%

basics

~20 s

Turn off remote loading with env.allowRemoteModels = false and point env.localModelPath at your files. In Node local loading is already on; in a browser you must also set env.allowLocalModels = true, or every load fails. env.remoteHost can target a mirror instead.

open as a page

In Transformers.js, how do you drive a model-loading progress bar from the `progress_callback` option of `pipeline()`, and which events does it send?

level: middleimportance: nice to knowfreq 4%

basics

~20 s

Pass progress_callback to pipeline() and switch on each event's status: initiate, download, progress and done arrive per file, progress_total sums bytes across all files, and ready fires once when the pipeline can run. Drive a single bar from progress_total.

open as a page

When Transformers.js runs on its WASM backend, what do env.backends.onnx.wasm numThreads, proxy and wasmPaths control, and when do you change each?

level: seniorimportance: nice to knowfreq 3%

basics

~20 s

ONNX Runtime Web's WebAssembly flags: numThreads sizes the thread pool (0 lets the runtime decide), proxy runs inference in a runtime-managed worker (off by default), and wasmPaths sets where the runtime's .wasm and .mjs files are fetched.

open as a page

Why do returning users of a Transformers.js page keep running old weights after the model repo's main branch changed, and how do you control that cache?

level: seniorimportance: nice to knowfreq 3%

basics

~20 s

Transformers.js checks its Cache API store before any request, keyed by the resolve URL including the revision, and never revalidates a hit, so files cached from main stay. Pin revision to a commit, bump it for new weights, clear old entries.

open as a page