Can an external service query a Flink job's keyed state directly, and should it?

## The short answer There is a mechanism, and you should not build on it. Flink's **Queryable State** feature lets an external client look up a key's value directly in a running job's keyed state. It was formally deprecated in Flink 1.18 with notice that it would be removed in a future major version. Flink 2.0 did not remove it, so it is still in Flink 2.3, but in a state that rules it out for new work: - Every public entry point — `KeyedStream.asQueryableState(...)`, `StateDescriptor.setQueryable(...)`, the `QueryableStateClient` and `QueryableStateOptions` — is marked `@Deprecated`, with removal still announced for a future major version. - It is off by default: the TaskManager-side proxy and server start only when `queryable-state.enable` is `true`, and the runtime ships as an optional `flink-queryable-state-runtime` jar in the distribution's `opt/` directory. - It no longer has a page in the Flink documentation. - It cannot be combined with state TTL: `setQueryable` rejects a descriptor that has TTL enabled. Interviewers ask this because the instinct is natural: the job already holds a per-key value in an embedded key/value store, so why not read it? The expected answer is that you still can, barely, and should not. ## Why it was a bad idea even when it was supported Four reasons, all worth being able to state: **Consistency.** A running job's state is mid-flight. It reflects everything processed so far, including work that has not been committed by a checkpoint and would be rolled back on failure. A reader gets a value that may never have existed in any consistent snapshot of the job. **Isolation.** Serving lookups from the TaskManagers puts external query traffic on the same processes — and, for the state access itself, competing with the same RocksDB instances and memory budget — that are doing record processing. A burst of queries becomes a throughput problem for the pipeline, and vice versa. **Availability coupling.** The state lives inside job execution. Redeploy the job, rescale it, or let it fail over, and the endpoints move, keys land on different subtasks, and the reader sees gaps. External consumers expect an availability model a stream job does not offer. **Topology coupling.** The client connects to a proxy on some TaskManager by host and port and names the job by its `JobID`; the proxy asks the JobManager which TaskManager holds the key's key group and caches that location. The client never picks the subtask itself, but it is bound to one job run's ID and to TaskManager endpoints — exactly the sort of coupling you do not want in a service boundary. ## Pattern one: push what should be readable The idiomatic answer is that if a value should be readable from outside, it is an output, not internal state. Emit it. A `KeyedProcessFunction` that maintains a `ValueState` can also `collect` the updated value whenever it changes — or on a timer, if you want to rate-limit the output — into a sink that writes a key/value store, a database, or a compacted topic. Readers query that store. This inverts every problem above. The store's consistency and availability are its own concern; serving load never touches the TaskManagers; the shape of the served record is a deliberate contract rather than an accident of your state layout; and you can change the internal state representation — swap `ValueState<Map<K,V>>` for `MapState<K,V>`, add a field, change the backend — without breaking a single consumer. The cost is a second system to run and an extra write path, and the output is eventually consistent with the stream by design. Bound that lag explicitly rather than pretending it is zero. ## Pattern two: read a savepoint offline When the requirement is inspection rather than serving — "what does the job think this user's state is?", "how many keys are we holding?", "why did this session never close?" — the State Processor API is the right tool. Take a savepoint, then load it in a batch job with `SavepointReader` and read the keyed state out as ordinary records that you can filter, count or export. The same API works in the other direction: `SavepointWriter` produces a savepoint from batch-computed data, which is how you bootstrap a new job with state derived from history rather than waiting for it to warm up from the live stream. It is also the escape hatch for state migrations Flink will not perform automatically — changing a field's type, re-keying, or changing maximum parallelism. What it is not: a live lookup. It operates on a savepoint file, so it is always a point-in-time view and always costs a batch run. ## How to answer it Say that Queryable State survives in Flink 2.3 only as a deprecated leftover — off by default, undocumented, incompatible with TTL and slated for removal — and that moving away from it is a design correction rather than a regression. Then give the two patterns and be explicit about which requirement each serves: push to an external store for low-latency serving, State Processor API for offline inspection and bootstrapping. If pressed on the trade-off, the honest one is that the push pattern trades an extra system and a bounded staleness window for isolation, a stable contract, and freedom to evolve the job's internals — which is nearly always the right trade. ## A neighbouring point of confusion Other stream processors made different choices here, and candidates sometimes import the vocabulary. Serving reads directly out of a stream processor's local state store is a real design in other systems; it is not the model current Flink supports, and answering the Flink question with another engine's mechanism is a common and visible slip.