skip to content

A Playwright locator using css= finds an issue tracker's submit button, but the equivalent xpath= matches nothing -- why?

level: seniorimportance: should knowfreq 36%

answer

  1. Two engines, two traversal rules
  2. One boundary only one of them crosses
  3. The host element is where it stops
  4. Open versus closed changes the answer
  5. Custom elements are the usual suspects

basics

~20 s

The button sits inside an open shadow root. Playwright's CSS engine pierces open shadow roots, so css= reaches it; the XPath engine does not cross a shadow boundary, so xpath= only ever finds the host element.

solid answer

~40 s

Almost always because the target is inside an **open shadow root**. Playwright's CSS engine pierces open shadow roots, so a descendant combinator crosses the boundary as if it were not there. The `xpath=` engine is evaluated the way `Document.evaluate` is, walks a single document tree, and stops at the shadow host -- so `//issue-composer//button` matches the host and nothing below it. Confirm it by narrowing the expression until the match disappears: the last matching step is the host. If the CSS selector *also* finds nothing, suspect a closed shadow root, which Playwright does not support at all. The practical rule is to avoid XPath for anything inside a componentised page; CSS, text and attribute engines all cross an open boundary.

code

typescript · 8 lines
typescript
// <issue-composer> renders its textarea and submit button in an open shadow root.

// Works: the CSS engine pierces open shadow roots.
await page.locator('css=issue-composer button[type="submit"]').click();

// Matches the host only, then nothing: the xpath engine stops at the boundary.
await page.locator('xpath=//issue-composer').click();          // resolves
await page.locator('xpath=//issue-composer//button').click();  // times out

go deeper

for a junior

Remember that Playwright's CSS selectors reach inside open shadow roots and its XPath selectors do not. If a component-rendered element is invisible to XPath, rewrite the selector in CSS rather than adding waits.

for a middle

Explain the mechanism: XPath is evaluated over a single document tree and stops at the shadow host, while the CSS engine crosses the boundary. Know that closed shadow roots defeat every engine.

for a senior

Diagnose by bisecting the expression to find the host, and treat a suite-wide XPath convention on a componentised app as a structural problem rather than a run of flaky tests to retry.

for a principal

Weigh whether test reach should constrain component design. Asking teams to keep shadow roots open, or to expose a stable host-level surface, is a platform decision with consequences well beyond one failing selector.

## The asymmetry between the two engines Playwright's `css=` and `xpath=` engines are not two spellings of the same traversal. They differ in one structural way that explains most "works with CSS, silent with XPath" reports: - The **CSS engine pierces open shadow roots**. A descendant combinator will cross a shadow boundary, so `css=issue-composer button[type="submit"]` reaches a button rendered inside the `<issue-composer>` component's shadow root as though the boundary were not there. - The **XPath engine does not**. `xpath=` is evaluated the way `Document.evaluate` evaluates XPath, so it walks one document tree and stops at the shadow host. `//issue-composer//button` finds the host and then finds nothing beneath it. That is the whole diagnosis. The button exists, the page is loaded, both selectors are well formed -- one engine can see through the boundary and the other cannot. ## Confirming it in a minute 1. Ask whether the target is inside a custom element. In an issue tracker, composers, chip pickers and rich-text editors are the usual suspects, because they are the parts most often shipped as components. 2. Swap the failing `xpath=` for the CSS equivalent. If CSS finds it and XPath does not, the shadow boundary is the difference. 3. Narrow the XPath: `xpath=//issue-composer` will match, `xpath=//issue-composer//button` will not. The point where the match disappears is the host element. ## Closed shadow roots stop both engines An open shadow root is one whose contents remain reachable from the host; a **closed** one is not. Playwright does not support closed-mode shadow roots at all -- neither engine will reach inside one. If your CSS selector *also* finds nothing, the shadow root being closed is the likely reason, and no selector syntax will rescue you. The fix is to ask the application to expose the root as open, or to drive the component through whatever public surface it does expose. ## What this means for the selector you should write | Situation | Reaches inside an open shadow root | |---|---| | `css=` selector | yes | | `xpath=` selector | no | | `text=` and attribute engines such as `id=` | yes | | Any selector, closed shadow root | no | Because everything except XPath crosses an open boundary, the practical rule is simple: **do not use XPath to reach into components**. If a suite has settled on XPath as a convention, componentised markup is the point where that convention starts producing selectors that are correct on paper and empty at runtime. ## Other reasons an XPath can come back empty Shadow DOM is the common cause but not the only one. Before concluding, check that: - The expression really starts from where you think. A leading `//` searches the whole document, so a locator chained onto a parent still evaluates document-wide unless you write it relative. - You are not matching on a text node whose whitespace differs from what you assumed. - The expression is valid at all -- Playwright evaluates `xpath=` exactly like the platform does, so every quirk of XPath in the browser belongs to the expression rather than to Playwright. ## In an issue tracker The comment composer ships as `<issue-composer>` with an open shadow root holding the textarea and the submit button. `css=issue-composer button[type="submit"]` clicks it. The visually identical `xpath=//issue-composer//button[@type="submit"]` times out with a message saying the locator resolved to no elements. Rewriting the selector in CSS -- or better, locating the button by its accessible role and name, which also crosses the boundary -- fixes it without anyone having to touch the component.

  • What if the css= selector also fails to match?
    Then the shadow root is probably closed. Playwright does not support closed-mode shadow roots, so no selector engine reaches inside one and no syntax will work around it. The options are to have the application open the root, or to drive the component through whatever attributes, events or public API it exposes on the host.
  • Do Playwright's text and attribute engines cross an open shadow boundary?
    Yes. XPath is the exception, not the rule -- the text and attribute engines, and the role and label lookups built on top of them, all reach into open shadow roots. That is why locating the same button by its accessible role and name usually works where the XPath expression does not.
  • How would you narrow down which element the XPath stops at?
    Shorten the expression one step at a time and see where matches stop. `//issue-composer` will resolve while `//issue-composer//button` will not, which identifies the host as the boundary. That takes seconds and distinguishes a shadow-root problem from a typo or a genuinely absent element.

saying these in an interview costs you the question

  • Assuming CSS and XPath traverse identically in Playwright
  • Blaming a timing problem when the cause is a shadow boundary
  • Thinking closed shadow roots can be pierced with the right selector
  • Believing only CSS crosses an open shadow root, not text lookups
  • Adding waits to a locator that will never resolve