What is ServiceInstanceListSupplier and how does caching work in Spring Cloud LoadBalancer?

supplier feeds the list; balancer picks one
DiscoveryClientServiceInstanceListSupplier base
CachingServiceInstanceListSupplier default, TTL 35s / cap 256
Caffeine if present else default map cache
builder decorators; withCaching outermost; staleness trade-off

basics

~20 s

ServiceInstanceListSupplier provides the current list of instances for a service to the load balancer. By default SCLB wraps a discovery-backed supplier with CachingServiceInstanceListSupplier so it doesn't hit the discovery client on every request; cache TTL defaults to 35s.

solid answer

~40 s

The load balancer never talks to discovery directly; it asks a ServiceInstanceListSupplier for the instances of a service. The base is usually DiscoveryClientServiceInstanceListSupplier, which queries Eureka/Consul/K8s. Because that's expensive per request, SCLB by default wraps it in CachingServiceInstanceListSupplier, backed by a LoadBalancerCacheManager — Caffeine-based if Caffeine is on the classpath, otherwise a default map cache. Defaults: TTL 35s, capacity 256, tunable via spring.cloud.loadbalancer.cache.ttl / .capacity, and disableable with spring.cloud.loadbalancer.cache.enabled=false. Suppliers are composable via ServiceInstanceListSupplier.builder(): withDiscoveryClient(), withHealthChecks(), withZonePreference(), withCaching(), etc. — a decorator chain where withCaching() should wrap the outer layer. The trade-off: caching cuts discovery load and latency but delays visibility of scale-up/down and instance failures by up to the TTL.

code

java · 22 lines

java

// Per-client supplier config: discovery + health checks + caching.
public class CustomSupplierConfig {

    @Bean
    ServiceInstanceListSupplier serviceInstanceListSupplier(
            ConfigurableApplicationContext context) {
        return ServiceInstanceListSupplier.builder()
                .withDiscoveryClient()
                .withHealthChecks()
                .withCaching()   // outermost so it caches the composed, filtered list
                .build(context);
    }
}

// application.yml
// spring:
//   cloud:
//     loadbalancer:
//       cache:
//         ttl: 10s        # default 35s
//         capacity: 512   # default 256
//         # enabled: false  # disable caching entirely

for a junior

Know the supplier provides the instance list and results are cached by default.

for a middle

Name DiscoveryClientServiceInstanceListSupplier and CachingServiceInstanceListSupplier and the TTL property.

for a senior

Explain the builder decorator chain, ordering, Caffeine fallback, and the freshness-vs-load trade-off.

for a principal

Design TTL/health-check/retry policy for ephemeral fleets and reason about staleness-induced failure modes.

Role in the pipeline. A ReactorLoadBalancer decides which instance to use, but it needs the candidate set. That set comes from a ServiceInstanceListSupplier — a reactive supplier of List<ServiceInstance> (Flux<List<ServiceInstance>>). Separating supply from selection is the key design: you layer behaviors on the supplier without changing the algorithm.

The base supplier. With a discovery client on the classpath, the default base is DiscoveryClientServiceInstanceListSupplier, which pulls live instances from the underlying registry (Eureka, Consul, Zookeeper, Kubernetes, etc.). Calling discovery on every outbound request would be wasteful and slow.

Default caching. So SCLB's auto-config wraps the base in CachingServiceInstanceListSupplier. It's backed by a LoadBalancerCacheManager:

If Caffeine is on the classpath, SCLB uses a Caffeine-backed cache manager (recommended; SCLB logs a warning suggesting you add Caffeine if it's absent).
Otherwise it falls back to DefaultLoadBalancerCache (a simple map-based cache).

Cache tuning (properties).

spring.cloud.loadbalancer.cache.ttl — time-to-live, default 35s.
spring.cloud.loadbalancer.cache.capacity — max entries, default 256.
spring.cloud.loadbalancer.cache.enabled=false — turn caching off entirely (every choose hits the underlying supplier).

Composing suppliers with the builder. You customize the supplier chain in a per-client config:

java

ServiceInstanceListSupplier.builder()
    .withDiscoveryClient()
    .withHealthChecks()   // filters out instances failing a health probe
    .withCaching()        // outermost: caches the composed result
    .build(context);

Each with* decorates the previous. Ordering matters: withCaching() should be last/outermost so it caches the fully-composed list; putting it before health checks would cache pre-filtered data incorrectly. Other decorators include withZonePreference() (prefer same-zone instances), withSameInstancePreference() (sticky to the last-used instance when present), withHints() / hint-based routing, withRequestBasedStickySession(), and withWeighted() (weighted selection). If you declare a custom supplier bean, it replaces the default caching one — so remember to add .withCaching() yourself or you lose caching.

Health checks caveat. withHealthChecks() starts a background health-check flux that periodically probes instances (spring.cloud.loadbalancer.health-check.* — interval, initial-delay, path). It's most appropriate when your discovery source doesn't already remove unhealthy instances.

The core trade-off. Caching reduces registry load and per-request latency, but it introduces staleness: a scaled-down or crashed instance can remain in the cached list for up to the TTL (default 35s), causing transient failures until eviction/refresh. Mitigations: shorter TTL, enable health checks, enable retries (spring.cloud.loadbalancer.retry.enabled) so a request to a dead instance retries elsewhere. Conversely, disabling the cache maximizes freshness but can overload discovery and add latency to every call.

Gotchas.

Declaring your own ServiceInstanceListSupplier bean drops the default caching wrapper unless you re-add .withCaching().
No Caffeine on the classpath → you still get caching, just the simpler default cache, plus a startup warning.
The cache is keyed per service id inside that service's child context.

When to tune. Lower the TTL or add health checks for fast-scaling/ephemeral environments (autoscaling, spot instances, Kubernetes); keep defaults for stable fleets. Pair caching with retries so staleness degrades gracefully rather than failing requests.

**Role in the pipeline.** A `ReactorLoadBalancer` decides *which* instance to use, but it needs the candidate set. That set comes from a `ServiceInstanceListSupplier` — a reactive supplier of `List<ServiceInstance>` (`Flux<List<ServiceInstance>>`). Separating supply from selection is the key design: you layer behaviors on the supplier without changing the algorithm. **The base supplier.** With a discovery client on the classpath, the default base is `DiscoveryClientServiceInstanceListSupplier`, which pulls live instances from the underlying registry (Eureka, Consul, Zookeeper, Kubernetes, etc.). Calling discovery on every outbound request would be wasteful and slow. **Default caching.** So SCLB's auto-config wraps the base in `CachingServiceInstanceListSupplier`. It's backed by a `LoadBalancerCacheManager`: - If **Caffeine** is on the classpath, SCLB uses a Caffeine-backed cache manager (recommended; SCLB logs a warning suggesting you add Caffeine if it's absent). - Otherwise it falls back to `DefaultLoadBalancerCache` (a simple map-based cache). **Cache tuning (properties).** - `spring.cloud.loadbalancer.cache.ttl` — time-to-live, **default 35s**. - `spring.cloud.loadbalancer.cache.capacity` — max entries, **default 256**. - `spring.cloud.loadbalancer.cache.enabled=false` — turn caching off entirely (every choose hits the underlying supplier). **Composing suppliers with the builder.** You customize the supplier chain in a per-client config: ```java ServiceInstanceListSupplier.builder() .withDiscoveryClient() .withHealthChecks() // filters out instances failing a health probe .withCaching() // outermost: caches the composed result .build(context); ``` Each `with*` decorates the previous. **Ordering matters**: `withCaching()` should be last/outermost so it caches the fully-composed list; putting it before health checks would cache pre-filtered data incorrectly. Other decorators include `withZonePreference()` (prefer same-zone instances), `withSameInstancePreference()` (sticky to the last-used instance when present), `withHints()` / hint-based routing, `withRequestBasedStickySession()`, and `withWeighted()` (weighted selection). If you declare a custom supplier bean, it **replaces** the default caching one — so remember to add `.withCaching()` yourself or you lose caching. **Health checks caveat.** `withHealthChecks()` starts a background health-check flux that periodically probes instances (`spring.cloud.loadbalancer.health-check.*` — interval, initial-delay, path). It's most appropriate when your discovery source doesn't already remove unhealthy instances. **The core trade-off.** Caching reduces registry load and per-request latency, but it introduces **staleness**: a scaled-down or crashed instance can remain in the cached list for up to the TTL (default 35s), causing transient failures until eviction/refresh. Mitigations: shorter TTL, enable health checks, enable retries (`spring.cloud.loadbalancer.retry.enabled`) so a request to a dead instance retries elsewhere. Conversely, disabling the cache maximizes freshness but can overload discovery and add latency to every call. **Gotchas.** - Declaring your own `ServiceInstanceListSupplier` bean drops the default caching wrapper unless you re-add `.withCaching()`. - No Caffeine on the classpath → you still get caching, just the simpler default cache, plus a startup warning. - The cache is keyed per service id inside that service's child context. **When to tune.** Lower the TTL or add health checks for fast-scaling/ephemeral environments (autoscaling, spot instances, Kubernetes); keep defaults for stable fleets. Pair caching with retries so staleness degrades gracefully rather than failing requests.

What's the risk of the default 35s cache TTL in an autoscaling environment?
A terminated instance can linger in the cached list for up to 35s, so the balancer may still route to it and requests fail until eviction. Mitigate with shorter TTL, health checks, and retry enabled.
If you define your own ServiceInstanceListSupplier bean, what caching behavior do you get?
None automatically — your bean replaces the default CachingServiceInstanceListSupplier. You must chain .withCaching() yourself in the builder, or the balancer hits the underlying supplier on every request.
Why does Spring Cloud log a warning about Caffeine?
Without Caffeine on the classpath, SCLB falls back to a simpler default map cache. It still works but Caffeine is recommended for better eviction/TTL behavior, so it warns you to add it.

saying these in an interview costs you the question

Saying the load balancer queries discovery directly every request
Thinking caching is off by default
Not knowing a custom supplier bean drops default caching
Ignoring cache staleness as a source of transient routing failures
Putting withCaching() before health checks in the chain

What is ServiceInstanceListSupplier and how does caching work in Spring Cloud LoadBalancer?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~2 min

▾common follow-ups3 questions

▾red flags5 flags

§ context

▾related questions7 questions

▾related topics3 topics

answer

go deeper

context