skip to content

Post-Connect Rerouting

The reply to a new session can contain addresses of its own, on a different protocol and path from the one you dialled, so what you keep talking to is not always what you connected to.

on this pageshow

explore

questions

4

Why does a remote WebDriver session reply carry addresses you never dialled?

level: middleimportance: should knowfreq 64%

answer

  1. the reply is not an echo
  2. an internal address would be useless
  3. black-box indistinguishable from a remote end
  4. the rewrite copies the user info across
  5. getCapabilities answers with the reply

basics

~20 s

A grid or router in front of the browser has to be indistinguishable from the real remote end, so it substitutes addresses its client can actually reach for the fleet-internal ones, then proxies. The reply, not your request, holds them.

solid answer

~50 s

The W3C WebDriver specification requires every remote end to be **black-box indistinguishable** from an endpoint node, so an intermediary that handed back the browser's own container-internal address would be handing you something you cannot open. It rewrites instead. Selenium Grid's `LocalNode` rebuilds the returned live-view and bidirectional addresses from one configured value, the public grid URI: it copies that URI's user info, host and port, derives a `ws` or `wss` scheme, and **prepends** its sub-path to a `/session/<id>/se/...` suffix. Because the user info is copied, a returned address can carry a credential — mask it before you log it. The node's real address stays behind in `se:vncLocalAddress` for the Grid to dial. Selenoid, unmaintained by its own README, does the same trick through its own front door, adding `se:cdp` as `ws://<the Host header you sent>/devtools/<sessionId>/`. `RemoteWebDriver` replaces its capabilities with the reply's, so `getCapabilities()` shows what you got.

code

java · 17 lines
java
// Selenium: RemoteWebDriver replaces its own capabilities with the ones the
// remote end returned, so this reads the reply and not your request.
RemoteWebDriver driver = new RemoteWebDriver(gridEndpoint, options);
Capabilities returned = driver.getCapabilities();

// Selenium Grid's LocalNode#rewrite copies the public grid URI's USER INFO
// into every address it builds, so se:vnc can be
//   ws://user:[email protected]/session/<id>/se/vnc
// Selenium's own masker swaps that user info for *** before anything prints.
Object vnc = returned.getCapability("se:vnc");
if (vnc != null) {
  log.info("live view: {}", JdkHttpClient.maskUrlCredentials(String.valueOf(vnc)));
}

// Bidding-screen suite: record WHICH channels came back, at session start.
// The key names are the diagnosis; the values are secret-bearing.
log.info("session capability keys: {}", returned.getCapabilityNames());

go deeper

for a junior

Know that the session reply describes the session that was actually created. Read secondary addresses out of it rather than building your own, and when you record it, record the key names — an address from the reply can carry a user name and password.

for a middle

Be ready to explain why an intermediary must substitute a reachable address for an internal one, and to name where the original is kept so the intermediary can still proxy to it.

for a senior

Be ready to debug a returned address that does not resolve from the runner: check which public URL the intermediary was configured with, whether a reverse-proxy sub-path is part of it — that prefix is prepended to every rewritten path — and whether a proxy or container network made its auto-detected address wrong.

for a principal

Decide how much of the reply your harness records, in what masked form, and for how long. It is the only evidence of what a far side agreed to, and it can carry a credential into every log sink that sees it.

## The reply is not an echo of the request When a new session succeeds, the remote end sends back a capability map describing the session it actually created. That map is **not** a copy of what you sent. Selenium's `RemoteWebDriver.startSession` reads the response payload into a fresh `MutableCapabilities` and assigns it over the driver's own field, so `getCapabilities()` answers with the reply. Anything an intermediary added, changed or deleted on the way back is visible there and nowhere else. For a suite driving a live-auction bidding screen, the consequence is that the address you keep talking to beyond plain commands is the far side's choice, not your configuration. ## Why an intermediary cannot return the endpoint's own address The specification defines an **endpoint node**, the last hop implemented by the user agent, and an **intermediary node**, a proxy implementing both halves of the protocol. It then binds them: all remote end node types must be *black-box indistinguishable* from a remote end, from the local end's point of view. That rule is what forces the rewrite. The browser's sockets listen inside the provider's or cluster's own network — a container address, a private host, a port nobody published — so returning them verbatim would be honest and useless. The intermediary returns an address on **itself**, on a path it owns, and proxies whatever you open there through to the real one. ## What Selenium Grid actually rebuilds In `LocalNode`, `createExternalSession` assembles the capabilities the client will see, and a helper named `rewrite` builds each address. It keeps the public grid URI's user info, host and port; swaps `http` for `ws` and `https` for `wss`; and **prepends that URI's own normalised sub-path** to a `/session/<sessionId>/se/...` suffix, rather than discarding a prefix a reverse proxy put there. So: | part of the address | what happens to it | |---|---| | scheme | becomes a WebSocket scheme | | host and port | taken from the public grid URI | | user info | copied across unchanged, credential and all | | path | the grid URI's sub-path, then a Grid-owned `/session/<id>/se/...` suffix | The user-info row is the one with teeth. Point a Grid at `https://user:[email protected]/` and the `se:vnc` it returns is `wss://user:[email protected]/session/<id>/se/vnc` — a Grid test asserts that exact string, credential included, because the address has to stay dialable. The returned map is therefore a **secret-bearing object**, not a diagnostic blob. Selenium says so four lines above the rewrite, where it deletes the client-advertised `se:remoteUrl` rather than echo it, and any embedded credentials, into the session-created log line. It ships `maskUrlCredentials` for the rest. The node's real addresses are meanwhile preserved for the Grid to dial: - `se:vncLocalAddress` keeps the node-internal live-view address and is not rewritten. - `se:gridWebSocketUrl` keeps the browser's own bidirectional socket. - `ProxyNodeWebsockets` reads those two when a client connects to the live-view or bidirectional path, and opens the upstream socket for it. It matches four public paths in all — those two plus the DevTools bridge and a forwarding path — and resolves upstream addresses from more sources than these two, the browser's own debugger address inside the vendor options among them. A second Grid test pins the path: a grid URL served under a reverse-proxy prefix returns `se:vnc` and `se:cdp` beginning with that prefix, not with `/session`. Build one by hand and you are right only where there is no prefix. ## The same idea, a different project Selenoid — which, like every Aerokube repository here, declares itself unmaintained in its own README — implements the identical mechanism with its own vocabulary. Its `processBody` function injects into `value.capabilities` a `se:cdp` entry built as `ws://<host>/devtools/<sessionId>/`, where `<host>` is the `Host` header from **your** request, and its own test dials the result. Two things follow: - The key name is shared with Selenium Grid, but the path, the port behaviour and the proxying code are Selenoid's own. A key name tells you nothing about who wrote it. - Built from the header you sent, the address is reachable by whatever name you used. ## Reading it in practice On a hosted provider the mechanism is the same though nothing about it is inspectable: it accepts a session on a public address, runs the browser where you cannot route, and hands back addresses on itself. The discipline is small and cheap: 1. Record what came back once, at session start — the **key names**. They are the evidence of what the far side agreed to and carry no secret. Log an address only after masking its user info. 2. Take every secondary address from the reply, never composing one from the endpoint you configured. 3. Read the returned authority as the **grid's**, not yours. User info, host and port all come from that one configured URI and move as a unit, so they equal what you dialled only when the grid advertises the address you dial — configure another port on the same host and the reply names it. Scheme and path differ by construction. 4. Treat a missing entry as an answer in its own right: no such channel, and still a session. Whether that is a refusal or the consequence of never asking is settled by your request, not by the reply. The rule is one sentence. **What you connected to decides what you keep talking to, and it tells you in the reply.**

  • If the returned address points at the Grid, what dials the browser's own socket?
    The Grid does. Selenium Grid's `ProxyNodeWebsockets` accepts the client on the public `/session/<id>/se/...` path and then opens the upstream connection itself, reading the address out of the capabilities that were kept behind — `se:vncLocalAddress` for live view, `se:gridWebSocketUrl` for the bidirectional socket. The client never learns the internal address, and never needs to.
  • What decides which public address Selenium Grid builds those rewritten URLs from?
    A configured grid URL always wins. Only when a Node falls back to its auto-detected address — which can be unreachable behind a proxy or a container network — does it use the client-advertised `se:remoteUrl` instead, and then only that value's scheme and authority: user info, host and port. Its path is deliberately dropped, because a client path such as `/wd/hub` is not a Grid sub-path, and folding it in would build websocket URLs the Node's own routes do not match.
  • Which parts of a returned address come from the grid, and which can differ from the endpoint you dialled?
    The returned address is assembled from one value: the configured public grid URI. Its user info, host and port are copied across as a unit, its scheme decides `ws` versus `wss`, and its sub-path is prepended to the Grid-owned `/session/<id>/se/...` suffix. Nothing is derived from the connection you actually opened, so the authority matches the endpoint you dialled only when the grid was configured to advertise that same address — configure it with a different port on the same host and the reply names that other port while the host is unchanged. The scheme and the path differ by construction, every time.
  • What is safe to log from a returned capability map?
    The key names, always — they are the whole diagnosis of which channels you were given, and they carry nothing secret. The values need care: the Grid's `rewrite` copies the public grid URI's user info into every address it builds, so the `se:vnc`, `se:cdp` and `webSocketUrl` values a Grid returns can each contain `user:password`. Selenium ships `maskUrlCredentials`, which rebuilds a URI with the user info replaced by `***`, and its own `LocalNode` deletes the client-advertised `se:remoteUrl` from the reply for exactly this reason. Mask, or log the keys.

saying these in an interview costs you the question

  • Assuming the returned capabilities are an echo of the request
  • Believing a returned socket address points at the browser host
  • Composing secondary addresses from the endpoint instead of the reply
  • Expecting the returned scheme and path to match what you dialled
  • Printing the returned capability map raw, user info and all
  • Assuming a shared key name means a shared implementation
open as a page

Your suite gets a Selenium Grid session, then fails at first BiDi use. Why?

level: seniorimportance: should knowfreq 52%

basics

~20 s

Selenium Grid recorded the outcome in the reply rather than in an error. It deleted the socket address from the returned capabilities and set a boolean flag false, so creation still succeeded and the client built no channel at all.

open as a page

Your grid sessions fail mid-run with a router-level 404, not a WebDriver error. Why?

level: seniorimportance: should knowfreq 46%

basics

~20 s

The rewritten reply bound the session to a route, and the router can no longer resolve it. That is a routing failure, answered by the router itself, so you get its own message rather than a WebDriver session error.

open as a page

Why must a WebDriver session id from a routed grid be treated as opaque?

level: juniorimportance: nice to knowfreq 50%

basics

~20 s

A router in front of the browsers can rewrite the identifier before returning it, packing its routing state into the string. What you hold is longer than the upstream id, and only a router with the same table reads it.

open as a page