skip to content

Where does DiscoveryClient sit relative to Spring Cloud LoadBalancer and @LoadBalanced clients, and what caching/staleness concerns follow?

level: principalimportance: should knowfreq 40%

answer

  1. Discovery=who exists, LB=which one, client=call it
  2. ServiceInstanceListSupplier wraps DiscoveryClient
  3. Default SCLB = round-robin (Ribbon successor)
  4. Staleness chain: lease + discovery cache + LB cache
  5. Mitigate: retry-next, health-check supplier, graceful deregister

basics

~20 s

DiscoveryClient only finds instances of a service. Spring Cloud LoadBalancer then picks one instance per call, and a @LoadBalanced RestClient/WebClient/Feign actually makes the HTTP request to it. Because discovery data is cached, picks can briefly target stale instances.

solid answer

~50 s

DiscoveryClient is the lookup layer: given a serviceId it returns known instances from a locally-cached, eventually-consistent view of the registry. Spring Cloud LoadBalancer sits on top: it consumes those instances (via a ServiceInstanceListSupplier that wraps discovery) and applies a selection algorithm — round-robin by default — to choose one per request, optionally layering caching, health checks, zone preference, and hints. The outermost layer is the caller: a @LoadBalanced RestClient/WebClient/RestTemplate or an OpenFeign client that takes http://service-id/path, asks the load balancer to resolve service-id to a concrete instance, and executes the HTTP call. The staleness chain matters at scale: registry heartbeats/leases lag, discovery caches lag the registry, and LoadBalancer may cache the instance list again, so a freshly-killed instance can still be picked. Mitigations: retries with next-instance, health-check-based suppliers, shorter cache TTLs, and readiness/graceful-shutdown so instances deregister before dying.

code

java · 26 lines
java
@Configuration
class HttpClients {
    // @LoadBalanced makes the client resolve service ids via LoadBalancer,
    // which pulls instances from DiscoveryClient under the hood.
    @Bean
    @LoadBalanced
    RestClient.Builder loadBalancedRestClientBuilder() {
        return RestClient.builder();
    }
}

@Service
class OrderClient {
    private final RestClient restClient;
    OrderClient(@LoadBalanced RestClient.Builder builder) {
        this.restClient = builder.build();
    }

    Order fetch(long id) {
        // 'order-service' is resolved to a real host:port per request.
        return restClient.get()
            .uri("http://order-service/orders/{id}", id)
            .retrieve()
            .body(Order.class);
    }
}

go deeper

for a junior

Should separate 'find instances' from 'call a service' at a high level.

for a middle

Should know @LoadBalanced clients resolve service-id via LoadBalancer using discovery data.

for a senior

Should explain ServiceInstanceListSupplier, default round-robin, and basic staleness.

for a principal

Should map the full staleness chain (lease + discovery cache + LB cache), choose mitigations (retry-next, health-check supplier, graceful deregister, timeouts), and treat discovery data as eventually consistent by design.

## Three layers, one request A call to another service passes through three distinct responsibilities that beginners often conflate: 1. **Discovery — 'who exists?'** `DiscoveryClient.getInstances(serviceId)` returns the known instances. It does **not** choose or call anything. 2. **Load balancing — 'which one this time?'** `Spring Cloud LoadBalancer` (SCLB, the successor to Netflix Ribbon) picks a single instance per request. Its default `ReactorServiceInstanceLoadBalancer` implementation is **RoundRobinLoadBalancer**; a `RandomLoadBalancer` is also provided. Instances are fed to it by a **`ServiceInstanceListSupplier`**, which under the hood wraps a `DiscoveryClient`/`ReactiveDiscoveryClient` — this is the seam where discovery plugs into balancing. 3. **Invocation — 'make the call'** A **`@LoadBalanced`** `RestClient`/`WebClient`/`RestTemplate`, or an **OpenFeign** client, accepts a URL like `http://order-service/orders/42`, calls the load balancer to swap `order-service` for a real `host:port`, and performs the HTTP request. In Spring Cloud Gateway the equivalent is the `lb://order-service` URI scheme. ## ServiceInstanceListSupplier decorators SCLB composes behavior via layered suppliers, configurable through `spring.cloud.loadbalancer.configurations` or a custom config: - **caching** supplier (default when a cache like Caffeine is present) — caches the instance list to avoid hammering discovery per request. - **health-check** supplier — actively pings instances and filters unhealthy ones. - **zone-preference**, **same-instance-preference**, **hint-based**, **weighted** suppliers — routing refinements. ## The staleness chain (the principal-level concern) Correctness under churn depends on several caches lining up, each adding lag: 1. Instance sends heartbeats; registry expires it only after a **lease timeout** (Eureka), so a crashed instance lingers in the registry. 2. `DiscoveryClient` refreshes from the registry on an interval → **discovery cache lag**. 3. SCLB's caching supplier caches the list again → **load-balancer cache lag**. Net effect: the load balancer can select an instance that is already dead, producing connection errors. Eureka's AP/self-preservation bias widens this window; Consul/Zookeeper (more CP) narrow it but can drop instances during partitions. ## Mitigations - **Retry with next instance:** `spring-retry` + LoadBalancer retry (`spring.cloud.loadbalancer.retry.enabled`) to fail over on connection errors. - **Health-check supplier:** filter out instances failing an actuator health ping. - **Graceful shutdown / readiness:** deregister and drain before the process exits (Spring Boot graceful shutdown + registry deregistration) so instances leave the pool cleanly. - **Tune TTLs/heartbeats:** shorter lease + cache intervals reduce the stale window at the cost of more registry traffic. - **Timeouts + circuit breakers:** cap the blast radius of hitting a bad instance. ## Design guidance Treat DiscoveryClient as an infrastructure detail behind the load balancer; consumer code should target `http://service-id/...` and let SCLB resolve it. Reach for raw `getInstances` only for diagnostics or bespoke routing. Always design for the reality that the instance list is **eventually consistent**, not authoritative.

  • Concretely, how does the load balancer get its candidate instances from discovery?
    Through a ServiceInstanceListSupplier bean. The base supplier wraps a DiscoveryClient/ReactiveDiscoveryClient and calls getInstances(serviceId); decorator suppliers (caching, health-check, zone-preference) then filter/cache that list before the balancing algorithm picks one.
  • A node crashes hard. Why might requests still route to it for a while, and how do you limit the damage?
    Registry lease timeout, discovery cache refresh interval, and LoadBalancer list cache each lag, so the dead node stays selectable briefly. Limit damage with LoadBalancer retry-to-next-instance, a health-check supplier, tight timeouts/circuit breakers, and graceful deregistration on shutdown.
  • If you only inject DiscoveryClient and loop instances yourself, what do you lose?
    Consistent load-balancing strategy, health filtering, zone/hint preferences, caching, and retry integration that SCLB provides — you'd have to reimplement them, error-prone. Prefer @LoadBalanced clients unless you truly need custom routing.

saying these in an interview costs you the question

  • Saying DiscoveryClient load-balances or makes the HTTP call itself
  • Believing the selected instance is guaranteed alive because it came from the registry
  • Not knowing ServiceInstanceListSupplier is the bridge between discovery and LoadBalancer
  • Thinking Ribbon is still the current default (it's Spring Cloud LoadBalancer now)
  • Assuming no caching between registry and the actual request

context

resolving links…