skip to content

In Workers KV, how do you page through every key under a prefix with list(), and when should small values live in metadata instead?

level: middleimportance: should knowfreq 40%

answer

  1. keys come back, values do not
  2. loop on a completion flag
  3. an empty page is not the end
  4. 1024 bytes travel with the key

basics

~20 s

Call env.FLAGS.list({ prefix, cursor }) in a loop, passing the returned cursor and the same prefix each time, until list_complete is true. list() returns key names, expirations and metadata but not values, so small values kept in metadata avoid one get() per key.

solid answer

~40 s

`list({ prefix, limit, cursor })` returns `{ keys, list_complete, cursor }`, at most 1,000 keys per call, sorted by their UTF-8 bytes. Each entry has `name`, plus `expiration` (always absolute) and `metadata` when set, but never the value. Keep calling with the returned `cursor` and the same `prefix` until `list_complete` is `true`; an empty `keys` array does not mean the end, because recently deleted or expired keys are skipped but still walked. Listing is eventually consistent like reads, and each call is a billed list request. When a value fits in the 1024-byte metadata limit, storing it as metadata with an empty value lets one listing return everything, instead of a `list()` followed by a `get()` per key.

code

javascript · 17 lines
javascript
async function listAll(ns, prefix) {
  const keys = [];
  let cursor;
  do {
    const page = await ns.list({ prefix, cursor });
    keys.push(...page.keys);
    cursor = page.list_complete ? undefined : page.cursor;
  } while (cursor);
  return keys;
}

export default {
  async fetch(request, env) {
    const keys = await listAll(env.FLAGS, "site:eu:");
    return Response.json(keys.map((k) => ({ name: k.name, on: k.metadata?.on ?? false })));
  },
};

go deeper

for a junior

Recall that list() takes a prefix and returns key names in pages of up to 1,000, and that values need a separate read.

for a middle

Explain the cursor loop, why list_complete and not an empty page ends it, and how metadata lets one listing carry small values.

for a senior

Spot listing on the hot path, missing prefixes on later pages and stale listings after writes, and replace them with metadata, index keys or a cached result.

for a principal

Decide when a dataset's access pattern has become query-shaped enough that it should move from KV to a SQL store.

## What list() is for Workers KV is addressed by key, so most reads name the key they want. **`list()`** is the exception: it enumerates the keys in a namespace, optionally only those that start with a **prefix**. It is how a Worker finds every configuration entry for one site region, every key for one tenant, or every key it is about to clean up. It is not a query engine: it filters only by prefix and returns keys in a fixed order. ## The call and its result ```javascript const page = await env.FLAGS.list({ prefix: "site:eu:", limit: 1000, cursor }); ``` - **`prefix`** keeps only keys that begin with that string. - **`limit`** is the page size; the default is 1,000 keys, which is also the maximum. - **`cursor`** continues a previous listing. The promise resolves to: | Field | Meaning | |---|---| | `keys` | an array of `{ name, expiration?, metadata? }` | | `list_complete` | `false` while more keys remain | | `cursor` | the token for the next call, present while `list_complete` is `false` | Each key object carries `expiration` only if the key expires, and always as an **absolute UNIX time in seconds**, even if it was written with `expirationTtl`. It carries `metadata` only when the key has non-null metadata. **Values are never included.** Keys come back in **lexicographic order of their UTF-8 bytes**, which is why a key scheme such as `site:eu:000`, `site:eu:001` lists predictably. ## Paging correctly A correct loop has three properties: 1. It stops on **`list_complete === true`**, not on an empty `keys` array. A page can be empty while more keys remain, because recently deleted or expired keys (tombstones) are still walked but not returned. 2. It passes the **same `prefix`** on every call together with the cursor; the documentation requires the initial arguments to be repeated when de-paginating a prefixed listing. 3. It treats the result as **eventually consistent**: a key written in the last minute or so may not appear yet, and one just deleted may still be listed. ```javascript let cursor; do { const page = await env.FLAGS.list({ prefix: "site:eu:", cursor }); handle(page.keys); cursor = page.list_complete ? undefined : page.cursor; } while (cursor); ``` ## Values in metadata A listing followed by a `get()` per key costs one list request plus one read per key, and adds a round trip for each cold key. When each value is small, the documented alternative is to store it **in the metadata**, writing an empty value: ```javascript await env.FLAGS.put("site:eu:banner", "", { metadata: { on: true, text: "Spring sale" } }); ``` One `list({ prefix: "site:eu:" })` then returns names and values together. The limits are the deciding factor: - Metadata is at most **1024 bytes** of serialised JSON per key. - Metadata is returned by `list()` and by `getWithMetadata()`, so the same data can still be read by key. - A value that may grow past the limit belongs in the value, read with `get()`, or in a bulk `get()` of up to 100 keys. ## Cost and limits to keep in mind - Each `list()` call is a **list request**: the Free plan allows 1,000 a day, and the Paid plan includes 1 million a month, then charges $5.00 per million. - A listing of 50,000 keys is at least 50 calls, so listing on every request is a design smell; cache the result, or keep an index key that names what a request needs. - Within one invocation, a Worker may make up to 1,000 operations to external services such as KV, and each `list()` call counts as one. ## When list() is the wrong tool `list()` answers "which keys start with this string", nothing more. Filtering by a field inside a value, sorting by anything other than the key, or counting matching entries all mean fetching keys and doing the work in the Worker. A dataset that is mostly queried that way is better kept in a SQL store such as D1, with KV left for lookups by known key. An **index key** is the usual middle ground for a small set: one key whose value, or metadata, names the keys a request needs. The single writer rewrites it whenever the set changes, and readers make one `get()` instead of a listing.

  • In Workers KV, a key was written with expirationTtl: 600; what does list() report for it?
    An `expiration` field holding the absolute UNIX time in seconds when the key will be deleted, not the 600 seconds it was written with. list() always reports expiration in absolute form, and omits the field for keys that never expire.
  • In Workers KV, can list() return keys in reverse order or start from a given key name?
    No. The binding's `list()` takes only `prefix`, `limit` and `cursor`, and returns keys in ascending lexicographic order of their UTF-8 bytes. A key scheme has to put the useful ordering into the key itself, for example zero-padded numbers, or the application keeps its own index key.

  • An empty keys array means the listing is finished
  • list() returns each key's value, so no get() is needed
  • The cursor alone is enough; the prefix only matters on the first call
  • list() can filter on fields inside the stored JSON value
  • A single list() call can return 10,000 keys
  • A key written a second ago always appears in the next listing