Consul compiles a destination service's `service-router`, `service-splitter` and `service-resolver` config entries into a discovery chain. In what order do those three stages apply, and which of them require the destination's protocol to be declared as HTTP?
answer
- three entries compile into one chain
- route first, then weight, then endpoints
- router and splitter read requests
- service-defaults Protocol gates the L7 stages
- ask the discovery-chain endpoint what compiled
basics
~20 sConsul applies them router first, then splitter, then resolver. The router picks a route from L7 request attributes and the splitter divides traffic by weight, so both need an HTTP-family protocol declared in service-defaults; the resolver, which selects subsets and failover targets, also works for plain TCP.
solid answer
~50 sThe chain runs **router → splitter → resolver**. `service-router` matches L7 request attributes — path, method, header, query — and sends the request to a different service or subset. `service-splitter` then divides matched traffic by percentage weights across subsets or services. `service-resolver` finally decides *which instances* satisfy the chosen target: it defines named `Subsets` with catalog `Filter` expressions, plus `Redirect`, `Failover`, `ConnectTimeout` and load-balancing policy. Because the first two stages reason about requests, they require the destination's `Protocol` to be `http`, `http2` or `grpc`, set on a `service-defaults` (or `proxy-defaults`) config entry; write a router or splitter for a service still on the default `tcp` and Consul rejects or ignores it. The resolver operates at connection level and works for any protocol. Consul compiles the whole chain per destination and turns it into the proxy's routes and clusters; `/v1/discovery-chain/<service>` shows you the compiled result.
code
hcl · 3 linesKind = "service-defaults"
Name = "api"
Protocol = "http"go deeper
Know that Consul can route and split traffic for a destination using config entries, and that a service must be declared as an HTTP protocol before request-level routing is possible.
Name the three entries, put them in order — router, splitter, resolver — and say which stage owns matching, which owns weighting, and which owns subsets and failover.
Debug from the compiled chain rather than the source files: spot the missing Protocol, the subset whose filter matches nothing, and the long-lived connections a weight change will not move.
Decide how much routing logic belongs in the mesh at all: what the platform owns centrally, what service teams may write, and how you keep version metadata trustworthy enough for subset-based routing to be safe.
## What the discovery chain is When a mesh proxy needs to reach an upstream, Consul does not just hand it a list of healthy instances. It compiles a **discovery chain** for that destination — a graph assembled from up to three config entries — and turns the result into the proxy's routing and cluster configuration. Understanding the order is what lets you predict which entry actually shapes a given request. ## Stage one: `service-router` The router is the L7 entry point. It matches on request attributes and picks a destination: ```hcl Kind = "service-router" Name = "api" Routes = [ { Match { HTTP { PathPrefix = "/admin" } } Destination { Service = "api-admin" } }, { Match { HTTP { Header = [{ Name = "x-debug", Exact = "1" }] } } Destination { ServiceSubset = "canary" } } ] ``` Routes are evaluated in order and the first match wins; anything unmatched falls through to the default destination, which is the service itself. This is the stage that gives you deterministic header-based access to a specific version. ## Stage two: `service-splitter` Whatever the router selected is then split by weight: ```hcl Kind = "service-splitter" Name = "api" Splits = [ { Weight = 90, ServiceSubset = "v1" }, { Weight = 10, ServiceSubset = "v2" } ] ``` Weights are percentages and must sum to 100. This is the mechanism behind weighted traffic shifting — note that it is only the mechanism; deciding *when* to shift and on what signal is a release-process question, not a Consul one. ## Stage three: `service-resolver` The resolver answers "which instances count as this target?": ```hcl Kind = "service-resolver" Name = "api" DefaultSubset = "v1" Subsets = { v1 = { Filter = "Service.Meta.version == v1" } v2 = { Filter = "Service.Meta.version == v2" } } Failover = { "*" = { Datacenters = ["dc2"] } } ConnectTimeout = "5s" ``` `Subsets` name a slice of the catalog using a filter expression over instance metadata — which is why version-based routing depends on instances actually registering the metadata the filter reads. The resolver also owns `Redirect` (send this name somewhere else entirely), `Failover` (where to go when no healthy instance remains), `ConnectTimeout`, and load-balancing policy. Because it selects endpoints rather than inspecting requests, it applies to any protocol. ## The protocol prerequisite A Consul mesh proxy treats an upstream as an opaque TCP stream unless told otherwise. Declaring the protocol is a separate, easily-forgotten step: ```hcl Kind = "service-defaults" Name = "api" Protocol = "http" ``` Without it, the proxy has no HTTP parsing on that upstream, so nothing can match a path or count requests for a percentage split — the router and splitter have no meaning and Consul will not accept them as part of a valid chain. The resolver still works. This is the single most common reason a newcomer's canary split "does nothing": the entry was written, and the service was still on the default `tcp` protocol. The same prerequisite governs L7 intention `Permissions` and per-request metrics, so declaring the protocol is really the switch that turns a connection-level mesh into a request-level one for that service. ## Reading the compiled result Rather than reasoning about three entries in your head, ask Consul what it compiled: `GET /v1/discovery-chain/api` returns the resolved graph including the router nodes, splitter weights, resolver targets and the effective protocol. If the chain shows a single resolver node and no router, your protocol is almost certainly still `tcp`. This is also the fastest way to see that a subset filter matched nothing — a subset with no instances behind it produces a target that will simply fail to connect. ## Practical cautions Subsets are only as good as the metadata instances register, so version labels must be applied consistently by whatever registers the service. Splits apply to new requests, so a long-lived streaming or gRPC connection stays where it landed. And a resolver `Redirect` is evaluated before failover, so a chain can hop between services in ways that are much easier to see in the compiled output than in the source entries.
- You write a `service-splitter` for a service and traffic still all goes to one subset. What do you check first?Whether the destination has `Protocol` set to an HTTP family value in `service-defaults`; a `tcp` service cannot be split by request. After that, check the compiled chain at `/v1/discovery-chain/<service>` to see whether the splitter node exists at all, and whether the referenced subsets resolve to any instances — a subset whose `Filter` matches no registered metadata produces an empty target.
- Where do subsets get their definition, and what makes a subset empty?Subsets are declared in the `service-resolver` entry, each with a `Filter` expression evaluated against catalog data such as `Service.Meta.version`. A subset is empty when no registered, passing instance carries the metadata the filter tests — usually because the deployment that registers the service never set that metadata, or set a different key.
- Does a splitter re-balance traffic on an already-open connection?No. The split is applied when a request is routed, so long-lived connections — a streaming gRPC call, or plain TCP that was never L7-aware to begin with — stay with the target they first reached. Weight changes therefore take effect as new requests or connections arrive, which matters when you are shifting traffic for a service with very long-lived sessions.
saying these in an interview costs you the question
- Thinks the resolver runs before the router
- Writes a splitter without setting the service protocol
- Assumes subsets exist without instance metadata to filter on
- Believes weights re-balance existing open connections
- Treats the three entries as alternatives rather than stages