skip to content

In Claude Code, how do WebFetch and WebSearch differ, and why can WebFetch report that a page lacks something it actually contains?

level: middleimportance: nice to knowfreq 5%

answer

  1. find pages versus read a page
  2. titles and URLs only
  3. extraction prompt in a separate model call
  4. lossy by design
  5. curl through Bash for raw content

basics

~20 s

WebSearch returns result titles and URLs; WebFetch fetches one URL, converts HTML to Markdown and usually runs an extraction prompt over it in a separate model call. That extraction is lossy, so a missing detail may only mean the prompt never asked.

solid answer

~40 s

`WebSearch` runs a query against Anthropic's web search backend and returns result titles and URLs; it never fetches the pages, so Claude follows up with `WebFetch` to read one. `WebFetch` takes a URL and a prompt describing what to extract, converts an HTML response to Markdown, and for most fetches runs that prompt against the content in a separate model call; Claude gets the extraction, not the raw page. That makes it **lossy by design**: "the page doesn't mention X" may mean the prompt didn't ask about X, or the page was truncated. Fix it by fetching again with a more specific prompt, or by using `curl` through `Bash` for the unprocessed page. `WebFetch` also refuses `localhost` and dotless hostnames, upgrades HTTP to HTTPS and caches responses for 15 minutes by default.

go deeper

for a junior

Know that WebSearch finds pages and returns titles and URLs, while WebFetch reads one page, and that both ask before running in Manual mode.

for a middle

Explain WebFetch's pipeline: Markdown conversion, truncation and a separate extraction call, and why that makes a 'not mentioned' result unreliable until re-fetched with a sharper prompt or curl.

for a senior

Diagnose web-tool surprises: refused local hosts, cross-host redirects, 15-minute caching, and the gap between WebFetch domain rules and Bash network access.

for a principal

Decide how a team lets the agent reach documentation: preapproved domains and WebFetch rules for convenience, Bash and sandbox network controls for anything that must actually be enforced.

## Two tools, two jobs Claude Code has two built-in web tools, and they answer different questions. | Aspect | `WebSearch` | `WebFetch` | | :- | :- | :- | | Input | A query, optionally `allowed_domains` or `blocked_domains` (not both) | A URL and a prompt describing what to extract | | What Claude receives | Result titles and URLs | An extraction from the page, usually written by a separate model call | | Fetches pages | No | Yes, one URL per call | | Permission rule form | Bare `WebSearch`, no specifier | `WebFetch(domain:example.com)` | | Manual-mode prompt | Yes | Yes, except preapproved documentation domains and domains your rules already decide | `WebSearch` may issue up to eight backend searches per call, refining internally before it returns. Its backend is not configurable; to search with another provider you connect an MCP server that exposes a search tool. Amazon Bedrock doesn't expose the server-side search tool at all. ## How WebFetch turns a page into an answer 1. Claude passes a URL and an extraction prompt. 2. `WebFetch` fetches the page, upgrading `http` to `https`. 3. When the server returns HTML, the response is converted to Markdown; this step is not configurable. 4. Large pages are truncated to a fixed character limit. 5. For most fetches, the prompt is run against the content in a **separate model call**. 6. Claude receives the result of that call, not the raw page. Steps 4 and 5 are why `WebFetch` is **lossy by design**. The extraction prompt decides what reaches Claude, so a result saying a page doesn't mention something may only mean the prompt didn't ask about it, or that the relevant section fell past the truncation point. ## Behaviours that shape the result - **No local hosts.** `WebFetch` refuses `localhost` and any hostname without a dot, such as a bare intranet name, before making a request; the error points Claude to `curl` through `Bash`. - **Cross-host redirects aren't followed.** The result names the original URL and the redirect target, and Claude makes a second `WebFetch` call to the new URL. - **Caching.** Each response is cached for 15 minutes by default, so an immediate re-fetch of a page that just changed can return the earlier copy. - **Deadline.** A page that hasn't finished downloading within five minutes fails with a deadline error. - **Headers.** The `User-Agent` begins with `Claude-User`, and the `Accept` header prefers Markdown, so servers that negotiate content can return Markdown directly. ## Getting the full content when it matters - Ask Claude to **fetch again with a narrower prompt** that names the exact flag, field or section. - Ask for the **raw page through `curl`** in `Bash` when you need every line, which then follows `Bash` permission rules instead of `WebFetch` ones. - For a local service or a dotless intranet host, `curl` through `Bash` is the only built-in path. ## A sensible sequence in a session 1. Use `WebSearch` to find candidate pages; scope it with `allowed_domains` when you trust only certain hosts, or `blocked_domains` to exclude some. 2. Use `WebFetch` on the one page that matters, with a prompt that names the exact detail you need. 3. If the answer is "not mentioned", re-fetch with a sharper prompt before believing it. 4. If you need the page verbatim, such as a full changelog or a configuration reference, fetch it raw with `curl` through `Bash`. ## Permissions in brief In Manual and `acceptEdits` modes `WebFetch` prompts before fetching, except for domains your rules already allow or deny and a built-in set of preapproved documentation domains. An explicit `WebFetch(domain:...)` rule overrides the preapproved set, so you can block or require a prompt for one of those domains. `auto` and `bypassPermissions` skip the prompt unless an explicit `ask` rule matches. Rule grammar beyond that belongs to the permission-rules material; the point here is that **`WebFetch` rules govern only `WebFetch`**: if `Bash` is allowed, `curl` and `wget` can still reach any URL.

  • Why does Claude Code's WebFetch refuse http://localhost:8080, and what should Claude do instead?
    `WebFetch` refuses `localhost` and any hostname without a dot before making a request. Its error tells Claude to reach local servers with `curl` through `Bash`, which then goes through the session's `Bash` permission rules rather than `WebFetch` domain rules.
  • Does denying WebFetch for a domain in Claude Code stop Claude from reaching that domain?
    No. A `WebFetch(domain:...)` deny rule governs only the `WebFetch` tool. If `Bash` is allowed, `curl`, `wget` or a script can still reach the URL, so a network restriction that must hold needs Bash rules plus the sandbox's network allowlist.

  • WebSearch reads the top result pages and summarises them for Claude.
  • WebFetch hands Claude the raw HTML of the page it fetched.
  • If WebFetch says a page lacks a detail, the page really lacks it.
  • WebFetch can read a dev server at localhost if you approve the prompt.
  • A WebFetch domain deny rule blocks all network access to that domain.