skip to content

In Workers AI, how do you embed support tickets with a @cf/baai/bge model, and what about its output must stay fixed?

level: middleimportance: should knowfreq 30%

answer

  1. one vector per text
  2. shape tells you the dimensions
  3. 768 for the base model
  4. mean versus cls pooling

basics

~20 s

Call env.AI.run("@cf/baai/bge-base-en-v1.5", { text: [...] }) with up to 100 texts; it returns { shape, data } with one 768-number vector per text. The model and its pooling must stay fixed, because vectors from another model or pooling cannot be compared.

solid answer

~40 s

An embeddings call passes `text`, a string or an array of up to 100 strings, to a text-embeddings model such as `@cf/baai/bge-base-en-v1.5`. It resolves to `{ shape, data, pooling }`: `data` holds one vector per input, and `shape` gives the count and the dimensions. The dimensions are fixed by the model: 384 for `bge-small-en-v1.5`, 768 for `bge-base-en-v1.5`, 1024 for `bge-large-en-v1.5`. These models also take a `pooling` input, `mean` by default or `cls`, and the catalogue warns that `cls` vectors are not compatible with `mean` ones. So every ticket, and every search query later, must be embedded with the same model and the same pooling; changing either means re-embedding the whole corpus. The bge models list a maximum input of 512 tokens per text, so long tickets are split or summarised first.

go deeper

for a junior

Recall the call shape: a text array in, { shape, data } out, with one vector per text and 768 numbers each for bge-base.

for a middle

Explain why the dimensions and pooling are fixed by the choice of model, and how batching up to 100 texts per call cuts requests.

for a senior

Show that a model or pooling change is a full re-embedding migration, and that the ingest and query paths must read one pinned model id.

for a principal

Weigh small, base and large models on cost per million tokens, vector size and quality, knowing the choice is expensive to reverse once the corpus is embedded.

## What an embeddings call does An **embedding** is a fixed-length list of numbers that represents a text's meaning, so that similar texts produce nearby vectors. In the support-ticket Worker, each ticket is embedded once when it arrives, and later searches embed the agent's query the same way and look for the closest ticket vectors. Workers AI exposes embedding models in the **Text Embeddings** task. The call is the ordinary `env.AI.run()` with a `text` input: ```javascript const { shape, data } = await env.AI.run("@cf/baai/bge-base-en-v1.5", { text: tickets.map((t) => t.subject + "\n" + t.body), }); // shape is [tickets.length, 768]; data[i] is ticket i's vector ``` ## The input side - `text` is a single string or an array of strings; the bge schema caps the array at **100** items, so a larger backfill is sent in chunks. - Each bge English model lists a **maximum of 512 input tokens** per text. A long ticket thread is better split into chunks, or summarised first, than sent whole. - `pooling` chooses how token vectors are combined into one: `mean` (the default) or `cls`. The model schema recommends `cls` for accuracy on longer inputs and warns that **`cls` embeddings are not compatible with `mean` ones**. ## The output side The call resolves to `{ shape, data, pooling }`: - `data` is an array of vectors, one per input text, in input order; - `shape` is `[number of texts, dimensions]`; - `pooling` echoes the method used. The dimensions belong to the model, not to the call: | Model | Output dimensions | Price in neurons | |---|---|---| | `@cf/baai/bge-small-en-v1.5` | 384 | 1,841 per million input tokens | | `@cf/baai/bge-base-en-v1.5` | 768 | 6,058 per million input tokens | | `@cf/baai/bge-large-en-v1.5` | 1024 | 18,582 per million input tokens | A 500-token ticket embedded with the base model costs about 3 neurons. ## What must stay fixed Vectors can only be compared with vectors from the **same model and the same pooling**. That gives the team three rules: 1. **Pin the model id** in configuration that both the ingest path and the search path read, so a query is never embedded with a different model from the tickets. 2. **Pin the pooling method.** Switching from `mean` to `cls` for new tickets leaves the old ones in an incompatible space, and searches silently get worse rather than failing. 3. **Treat a model change as a migration**: embed every stored ticket again with the new model, write the vectors to a store created for the new dimension count, and switch reads only when the backfill is done. Where the vectors are stored, and how an index is created to match a dimension count, is covered with Vectorize; the point here is that the dimension count is decided the moment you pick the model. ## Tickets longer than the input limit A ticket thread with weeks of replies easily passes 512 tokens. Three approaches fit Workers AI: - **Chunk it**: split the thread into passages of a few hundred tokens, embed each, and store each vector with the ticket id, so a search hit on any passage leads back to the ticket. - **Summarise first**: embed the summary the Worker already writes for the agent, which keeps one vector per ticket at the cost of losing detail. - **Embed the opening message only**: cheap and often good enough for routing, weak for finding a detail mentioned late in the thread. Whichever is chosen, the search query is embedded with the same model and pooling, and the choice is recorded with the vectors so a later change is visible. ## Volume and limits - Text Embeddings has a default rate limit of **3,000 requests per minute**, with `@cf/baai/bge-large-en-v1.5` at 1,500. Sending 100 texts per call stretches that a long way. - For a large backfill nobody is waiting for, the bge models support the asynchronous **Batch API**: the third argument `{ queueRequest: true }` and a `requests` array return a `request_id` to poll, and the total payload must stay under 10 MB. - Embedding cost is small next to text generation, but a backfill of millions of tickets is still worth estimating from the per-model neuron price first.

  • The team wants to move ticket search from bge-base to bge-large in Workers AI; what does that involve?
    A migration, not a config flip. bge-large returns 1024-dimension vectors where bge-base returned 768, so every stored ticket is embedded again, written to a store sized for 1024 dimensions, and searches switch only once the backfill completes. Until then, queries keep using bge-base so they stay comparable with the stored vectors.
  • How would you embed a backlog of two million old tickets with Workers AI without hitting rate limits?
    Send up to 100 texts per `env.AI.run()` call to cut the request count, and for work no one is waiting on use the Batch API: `{ queueRequest: true }` in the third argument with a `requests` array, keeping each payload under 10 MB, then poll with the returned `request_id`. Track neurons as you go; the base model costs about 6,058 per million input tokens.

  • You can choose the vector length per call to env.AI.run().
  • Switching pooling from mean to cls only changes accuracy, so old vectors still match.
  • Search queries can use a cheaper embedding model than the stored tickets.
  • An embeddings call accepts one text at a time, so a backfill needs one call per ticket.
  • A whole multi-page ticket thread fits in one bge embedding input.