An RxJava 3 Flowable chain fails with MissingBackpressureException: what does that mean, and how do you choose a backpressure strategy?
answer
- a producer that ignored the demand
- Observable has no request channel
- 128 items by default
- BUFFER, DROP, LATEST, ERROR, MISSING
- decide before the slow step
basics
~20 sA Flowable source emitted an item downstream had not requested, typical of sources that cannot slow down, such as interval, PublishProcessor or Flowable.create. Pick a strategy: buffer when every item matters, drop or keep the latest when only fresh data counts.
solid answer
~40 sA `Flowable` follows the Reactive Streams protocol: the consumer calls `request(n)`, and operators such as `observeOn` prefetch a bounded batch, 128 items by default (`Flowable.bufferSize()`). A source that cannot slow down, such as `Flowable.interval`, a `PublishProcessor`, or `Flowable.create` with `BackpressureStrategy.ERROR`, signals `MissingBackpressureException` when it has an item and no outstanding request. The fix is a deliberate choice: `onBackpressureBuffer()` keeps everything (without limit unless you give a capacity), `onBackpressureDrop()` discards items while downstream is busy, `onBackpressureLatest()` keeps only the newest, and `Flowable.create(source, BackpressureStrategy.X)` makes the same choice at the source. An `Observable` never signals this error because it has no request channel; its `observeOn` queues without bound, so a fast producer shows up as memory growth instead. Since RxJava 3.1.6, an internal operator queue that overflows reports `QueueOverflowException`.
code
kotlin · 28 linesimport io.reactivex.rxjava3.core.BackpressureStrategy
import io.reactivex.rxjava3.core.Flowable
import io.reactivex.rxjava3.schedulers.Schedulers
import java.util.concurrent.TimeUnit
fun interface ReadingListener {
fun onReading(value: Float)
}
interface Sensor {
fun register(listener: ReadingListener)
fun unregister(listener: ReadingListener)
}
// A callback that cannot slow down: keep only the newest reading.
fun readings(sensor: Sensor): Flowable<Float> =
Flowable.create<Float>({ emitter ->
val listener = ReadingListener { value -> emitter.onNext(value) }
sensor.register(listener)
emitter.setCancellable { sensor.unregister(listener) }
}, BackpressureStrategy.LATEST)
// A tick whose work can outlast the period: drop ticks that arrive while busy.
fun periodicSync(sync: () -> Unit): Flowable<Long> =
Flowable.interval(1, TimeUnit.MINUTES)
.onBackpressureDrop()
.observeOn(Schedulers.io(), false, 1) // request one tick at a time
.doOnNext { sync() }go deeper
Recall that Flowable supports backpressure and Observable does not, and that MissingBackpressureException means an item arrived that nobody had requested.
Explain request(n) and the default 128-item prefetch, and name the sources that cannot honour requests: interval, PublishProcessor and Flowable.create.
Choose and place a strategy for a real stream, size buffers on purpose, and trace a MissingBackpressureException or QueueOverflowException back to its source.
Decide where the team draws the Observable-versus-Flowable line and which data a product may lose under load, and make that loss explicit in the code.
## What backpressure means in RxJava 3 **Backpressure** is flow control: a consumer tells its producer how many items it is ready for, so a fast producer cannot flood a slow consumer. RxJava 3 splits the 0..N types on exactly this point. | | `Observable` | `Flowable` | |---|---|---| | Backpressure | none | yes, Reactive Streams `request(n)` | | Consumer type | `Observer` | `Subscriber` | | Overflow shows up as | growing memory in unbounded queues | an error, or the strategy you chose | | Intended use | short sequences, GUI interactions | producers that can outrun their consumers | `Single`, `Maybe` and `Completable` need no backpressure: at most one item can always be held until the consumer is ready. In a `Flowable` chain, each asynchronous operator has a bounded queue and requests in batches. The default batch, `Flowable.bufferSize()`, is **128**, and it can be changed globally with the `rx3.buffer-size` system property. `observeOn` requests 128, then requests more as it drains. ## Where MissingBackpressureException comes from Cold sources such as `Flowable.range`, `fromIterable` or `just` simply wait for requests and never raise it. The error comes from producers that **cannot** wait: - `Flowable.interval` and `timer`: slowing a clock down would shift its period, so the operator signals `MissingBackpressureException` ("Could not emit value ... due to lack of requests") when a tick finds no outstanding request. - `PublishProcessor`: it terminates a subscriber that is not ready for an item with `MissingBackpressureException`. - `Flowable.create(source, BackpressureStrategy.ERROR)`: callback code calls `onNext` regardless of demand, and the ERROR strategy fails as soon as demand runs out. - `onBackpressureBuffer(capacity)` with a bounded capacity: when the buffer is full it signals `MissingBackpressureException` ("Buffer is full"). Since **RxJava 3.1.6**, a different exception, `QueueOverflowException`, reports an overflowing internal queue inside operators such as `observeOn`, `flatMap` and `concatMap`: the upstream ignored backpressure entirely, or `onNext` was called concurrently without serialization. Older guides attribute that case to `MissingBackpressureException`, so read the stack trace, not just the exception name. ## The strategies | Choice | Source enum (`BackpressureStrategy`) | Operator | Keeps | Loses | |---|---|---|---|---| | buffer | `BUFFER` | `onBackpressureBuffer()` | every item | memory, without limit | | bounded buffer | (none) | `onBackpressureBuffer(capacity, ...)` | up to capacity | then errors, or drops per `BackpressureOverflowStrategy` | | drop | `DROP` | `onBackpressureDrop()` | items requested in time | everything that arrives while busy | | latest | `LATEST` | `onBackpressureLatest()` | the newest pending item | intermediate items | | error | `ERROR` | (none) | nothing extra | the stream, on first overflow | | missing | `MISSING` | (none) | nothing decided here | defers to a later `onBackpressureXxx` | `BackpressureOverflowStrategy` has three values for a full bounded buffer: `ERROR`, `DROP_OLDEST` and `DROP_LATEST`. ## Choosing for a real stream 1. **Ask whether every item matters.** Payment events and log lines must not be dropped: buffer, with a capacity sized to the largest real burst, or slow the source down. 2. **Ask whether only the newest value matters.** Sensor readings, location fixes and download progress: `LATEST` keeps the screen current without a backlog. Unlike `DROP`, it never loses the final value of a burst. 3. **Ask whether stale work is harmful.** A periodic sync that may outlast its period: `onBackpressureDrop()` skips ticks that arrive while the previous run is busy. 4. **Place the operator directly above the slow step**, and keep that step's own prefetch small (for example `observeOn(scheduler, false, 1)`) so demand reflects the real consumer instead of a 128-item queue. 5. **Treat ERROR as a tripwire** for streams that should never overflow; an error in production then points at a real design problem. ## Observable, toFlowable and the Android reality - Most Android UI streams (clicks, text changes, one-shot calls) are slow and short, so `Observable` is the right type and backpressure never comes up. - When an `Observable` feeds something slow or bounded, convert it with `toFlowable(BackpressureStrategy.X)`; the parameter forces the overflow decision. - `Observable.observeOn` uses an unbounded queue. A producer that stays faster than its consumer does not fail fast; it grows the heap until the process runs out of memory. ## Diagnosing - The stack trace usually ends in `observeOn` or a flattening operator, but the cause is the non-backpressured source above it. - `MissingBackpressureException` and `QueueOverflowException` both pass through `RxJavaPlugins.onError` unwrapped when they cannot be delivered, because RxJava treats them as likely bugs rather than lifecycle noise.
- Why did onBackpressureDrop() in front of observeOn drop nothing for a long time?`observeOn` prefetches 128 items by default and requests more as it drains, so the drop operator saw plenty of outstanding demand and passed items into `observeOn`'s queue. Dropping only began once that queue was full. Pass a small buffer size, as in `observeOn(scheduler, false, 1)`, so demand reflects the actual consumer.
- When would you convert an Observable to a Flowable, and what must you decide?When an `Observable` feeds a slow or bounded consumer, such as a database writer or a Reactive Streams library. `toFlowable(BackpressureStrategy)` makes you state the overflow behaviour: `BUFFER`, `DROP`, `LATEST`, `ERROR`, or `MISSING` when a later `onBackpressureXxx` operator will decide.
- Is onBackpressureBuffer() a safe default?Only for bounded bursts. Without a capacity it buffers without limit, so a producer that stays faster than its consumer turns into an out-of-memory crash later instead of an error now. Give it a capacity, and optionally a `BackpressureOverflowStrategy` such as `DROP_OLDEST`, so overflow is either a visible `MissingBackpressureException` or a deliberate loss.
- An Observable throws MissingBackpressureException when its consumer is too slow.
- Adding observeOn in front of the slow consumer fixes MissingBackpressureException.
- onBackpressureBuffer() with no arguments is bounded at 128 items.
- onBackpressureDrop and onBackpressureLatest lose exactly the same items.
- Flowable is always better than Observable, so every UI event should use it.
Assumes RxJava 3.1.x; since 3.1.6 an overflowing internal operator queue reports QueueOverflowException, where older guides describe MissingBackpressureException.