In Cloudflare Workers, how does HTMLRewriter rewrite links in a proxied HTML page while it streams, and what catches people in its text handlers?
answer
- a streaming parser, not a DOM
- selectors map to handler objects
- element, comments and text callbacks
- text arrives in pieces
- escaped unless told otherwise
basics
~20 sHTMLRewriter parses the response as it streams, calling handlers registered with on(selector, handler); an element handler rewrites href or src with setAttribute(). Text handlers get chunks, not whole text nodes, so concatenate until lastInTextNode is true.
solid answer
~40 sYou build `new HTMLRewriter()`, register handlers with `.on(selector, handler)`, and call `.transform(response)`, which returns a new streaming `Response`. The parser never builds a DOM: it calls each matching handler as tags go past, so an `element(el)` handler on `a[href]` can read `el.getAttribute("href")` and write the public URL back with `setAttribute()`, while untouched bytes flow straight through. Check the origin's `Content-Type` first and return non-HTML responses unchanged. The common trap is the `text(chunk)` callback: one text node can arrive as several chunks, split wherever the network split it, so you must buffer until `chunk.lastInTextNode` is `true`. Content you insert with `append()`, `before()` or `setInnerContent()` is escaped as text unless you pass `{ html: true }`, and a handler that throws truncates the page the client is already receiving.
code
javascript · 29 linesconst LEGACY = "http://legacy.example.com";
const PUBLIC = "https://www.example.com";
class LinkRewriter {
constructor(attribute) {
this.attribute = attribute;
}
element(element) {
const value = element.getAttribute(this.attribute);
if (value !== null && value.startsWith(LEGACY)) {
element.setAttribute(this.attribute, PUBLIC + value.slice(LEGACY.length));
}
}
}
export default {
async fetch(request) {
const url = new URL(request.url);
const upstream = await fetch(`${LEGACY}${url.pathname}${url.search}`);
const type = upstream.headers.get("Content-Type") ?? "";
if (!type.startsWith("text/html")) return upstream;
return new HTMLRewriter()
.on("a[href]", new LinkRewriter("href"))
.on("img[src]", new LinkRewriter("src"))
.on("form[action]", new LinkRewriter("action"))
.transform(upstream);
},
};go deeper
Know the on(selector, handler).transform(response) shape and that an element handler uses getAttribute() and setAttribute() to rewrite links.
Explain the streaming model: no DOM and no look-ahead, text chunks that need buffering until lastInTextNode, escaping unless html is true, and checking Content-Type.
Anticipate production failures: a throwing handler truncating pages mid-stream, async handlers adding latency per element, and chunk-boundary bugs in text replacement.
Weigh rewriting a legacy site at the edge against fixing its templates: a cheap migration path with a lasting dependency that every page change must respect.
## What HTMLRewriter is `HTMLRewriter` is a Workers-specific class for changing HTML **while it streams** from an origin to a client. It is not a DOM: there is no tree to query, no `document`, and no way to look ahead. Cloudflare describes it as zero-copy streaming parsing, so memory stays flat no matter how large the page is, and the client starts receiving bytes before the origin has finished sending them. The shape of every use is the same: 1. Create `new HTMLRewriter()`. 2. Register handlers with `.on(selector, elementHandler)` and, for whole-document callbacks, `.onDocument(documentHandler)`. 3. Call `.transform(response)`; it returns a new `Response` whose body is the rewritten stream and whose headers you may edit. ## Handlers and selectors An **element handler** is any object with some of `element(el)`, `comments(c)` and `text(t)`. The `element` callback receives an `Element` with `tagName`, `getAttribute()`, `setAttribute()`, `removeAttribute()`, `before()`, `after()`, `prepend()`, `append()`, `replace()`, `setInnerContent()`, `remove()`, `removeAndKeepContent()` and `onEndTag()`. A **document handler** adds `doctype()` and `end()` and sees content outside the top-level element too. Selectors cover what a link rewriter needs: | Selector | Matches | |---|---| | `a[href]` | anchors that have an `href` | | `a[href^="http://legacy.example.com"]` | `href` starting with that prefix | | `img[src]`, `form[action]` | other URL-bearing attributes | | `div#user_info`, `p.warning` | id and class | | `nav a`, `ul > li` | descendant and child | For a legacy site moving domains, one handler class per attribute (`href`, `src`, `action`) that replaces the old origin prefix covers most pages. Relative links need no change because the browser resolves them against the public URL. ## The text-chunk trap The `text(t)` callback is where people get caught. Because parsing is streaming, a **text chunk is not a text node**. `<p>Hey. How are you?</p>` may arrive as `"Hey. How "` and then `"are you?"`, followed by a final empty chunk whose `lastInTextNode` is `true`. Code that runs a regular expression on each chunk misses matches that straddle a boundary, and code that replaces each chunk independently can corrupt text. The fix is to accumulate `t.text` in the handler, remove or replace each partial chunk, and emit the edited text when `lastInTextNode` is `true`. ## Other behaviour worth knowing - **Escaping.** Content passed to `append()`, `before()`, `replace()` and similar methods is treated as text and HTML-escaped. Pass `{ html: true }` to insert markup, and only with content you control. - **Async handlers.** A handler may be `async` and await a `fetch()` or a KV read. The output stream waits for it, so slow calls delay the page. - **Errors.** If a handler throws, parsing stops, the transformed body errors and the original body is cancelled. A client that has already received part of the page sees a truncated response, so guard handler code carefully. - **Content type.** `transform()` parses whatever it is given, even a `text/plain` body. Check `Content-Type` and return images, JSON and downloads untouched. - **End tags.** `element.onEndTag(handler)` runs when the element's closing tag passes, which is where to insert content just before `</body>` or `</head>`. ## Why not a regular expression on text()? Reading the body with `text()` and running a replace buffers the whole page, delays the first byte until the origin is done, and breaks on attributes split over lines or quoted differently. HTMLRewriter parses real HTML, touches only the matching tags, and keeps the response streaming. ## What attribute handlers do not reach A link rewriter built from `a[href]`, `img[src]` and `form[action]` handlers covers most of a legacy site, but a review should look for URLs in other places: - **`srcset`** holds a comma-separated list of URLs with sizes, so it needs its own handler that rewrites every entry, not a single prefix replace. - **Inline scripts and styles** contain URLs inside text; attribute handlers never see them, and editing them means buffering text chunks as described above. - **`<base href>`** changes how every relative link resolves, so rewrite or remove it deliberately. - **Headers** such as `Location` on redirects and `Link` are not HTML; edit them on the response copy. Testing against a saved copy of several real pages catches most of these before users do.
- How would you replace a phrase inside paragraph text safely?In the `text(t)` handler, append `t.text` to a buffer and call `t.remove()` on each chunk. When `t.lastInTextNode` is `true`, run the replacement on the buffered string, insert it with `t.after(result)` (escaped as text by default), and reset the buffer. Working per chunk would miss phrases split across chunk boundaries.
- An element handler awaits a KV read for every <a> on a large page. What happens?Handlers may be `async`, and the rewriter waits for each returned promise before continuing the stream. One read per link adds its latency to the page in sequence, so the first byte after each link waits on storage. Fetch the data once before `transform()`, or once in the first handler call, and reuse it.
- HTMLRewriter loads the whole page into a DOM before handlers run.
- A text handler always receives a complete text node in one call.
- Content passed to append() is inserted as raw HTML by default.
- If a handler throws, the client receives the original untransformed page.
- transform() skips responses that are not text/html on its own.