skip to content

An RxJava 3 Flowable chain fails with MissingBackpressureException: what does that mean, and how do you choose a backpressure strategy?

level: seniorimportance: should knowfreq 28%

answer

  1. a producer that ignored the demand
  2. Observable has no request channel
  3. 128 items by default
  4. BUFFER, DROP, LATEST, ERROR, MISSING
  5. decide before the slow step

basics

~20 s

A Flowable source emitted an item downstream had not requested, typical of sources that cannot slow down, such as interval, PublishProcessor or Flowable.create. Pick a strategy: buffer when every item matters, drop or keep the latest when only fresh data counts.

solid answer

~40 s

A `Flowable` follows the Reactive Streams protocol: the consumer calls `request(n)`, and operators such as `observeOn` prefetch a bounded batch, 128 items by default (`Flowable.bufferSize()`). A source that cannot slow down, such as `Flowable.interval`, a `PublishProcessor`, or `Flowable.create` with `BackpressureStrategy.ERROR`, signals `MissingBackpressureException` when it has an item and no outstanding request. The fix is a deliberate choice: `onBackpressureBuffer()` keeps everything (without limit unless you give a capacity), `onBackpressureDrop()` discards items while downstream is busy, `onBackpressureLatest()` keeps only the newest, and `Flowable.create(source, BackpressureStrategy.X)` makes the same choice at the source. An `Observable` never signals this error because it has no request channel; its `observeOn` queues without bound, so a fast producer shows up as memory growth instead. Since RxJava 3.1.6, an internal operator queue that overflows reports `QueueOverflowException`.

code

kotlin · 28 lines
kotlin
import io.reactivex.rxjava3.core.BackpressureStrategy
import io.reactivex.rxjava3.core.Flowable
import io.reactivex.rxjava3.schedulers.Schedulers
import java.util.concurrent.TimeUnit

fun interface ReadingListener {
    fun onReading(value: Float)
}

interface Sensor {
    fun register(listener: ReadingListener)
    fun unregister(listener: ReadingListener)
}

// A callback that cannot slow down: keep only the newest reading.
fun readings(sensor: Sensor): Flowable<Float> =
    Flowable.create<Float>({ emitter ->
        val listener = ReadingListener { value -> emitter.onNext(value) }
        sensor.register(listener)
        emitter.setCancellable { sensor.unregister(listener) }
    }, BackpressureStrategy.LATEST)

// A tick whose work can outlast the period: drop ticks that arrive while busy.
fun periodicSync(sync: () -> Unit): Flowable<Long> =
    Flowable.interval(1, TimeUnit.MINUTES)
        .onBackpressureDrop()
        .observeOn(Schedulers.io(), false, 1)  // request one tick at a time
        .doOnNext { sync() }

go deeper

for a junior

Recall that Flowable supports backpressure and Observable does not, and that MissingBackpressureException means an item arrived that nobody had requested.

for a middle

Explain request(n) and the default 128-item prefetch, and name the sources that cannot honour requests: interval, PublishProcessor and Flowable.create.

for a senior

Choose and place a strategy for a real stream, size buffers on purpose, and trace a MissingBackpressureException or QueueOverflowException back to its source.

for a principal

Decide where the team draws the Observable-versus-Flowable line and which data a product may lose under load, and make that loss explicit in the code.

## What backpressure means in RxJava 3 **Backpressure** is flow control: a consumer tells its producer how many items it is ready for, so a fast producer cannot flood a slow consumer. RxJava 3 splits the 0..N types on exactly this point. | | `Observable` | `Flowable` | |---|---|---| | Backpressure | none | yes, Reactive Streams `request(n)` | | Consumer type | `Observer` | `Subscriber` | | Overflow shows up as | growing memory in unbounded queues | an error, or the strategy you chose | | Intended use | short sequences, GUI interactions | producers that can outrun their consumers | `Single`, `Maybe` and `Completable` need no backpressure: at most one item can always be held until the consumer is ready. In a `Flowable` chain, each asynchronous operator has a bounded queue and requests in batches. The default batch, `Flowable.bufferSize()`, is **128**, and it can be changed globally with the `rx3.buffer-size` system property. `observeOn` requests 128, then requests more as it drains. ## Where MissingBackpressureException comes from Cold sources such as `Flowable.range`, `fromIterable` or `just` simply wait for requests and never raise it. The error comes from producers that **cannot** wait: - `Flowable.interval` and `timer`: slowing a clock down would shift its period, so the operator signals `MissingBackpressureException` ("Could not emit value ... due to lack of requests") when a tick finds no outstanding request. - `PublishProcessor`: it terminates a subscriber that is not ready for an item with `MissingBackpressureException`. - `Flowable.create(source, BackpressureStrategy.ERROR)`: callback code calls `onNext` regardless of demand, and the ERROR strategy fails as soon as demand runs out. - `onBackpressureBuffer(capacity)` with a bounded capacity: when the buffer is full it signals `MissingBackpressureException` ("Buffer is full"). Since **RxJava 3.1.6**, a different exception, `QueueOverflowException`, reports an overflowing internal queue inside operators such as `observeOn`, `flatMap` and `concatMap`: the upstream ignored backpressure entirely, or `onNext` was called concurrently without serialization. Older guides attribute that case to `MissingBackpressureException`, so read the stack trace, not just the exception name. ## The strategies | Choice | Source enum (`BackpressureStrategy`) | Operator | Keeps | Loses | |---|---|---|---|---| | buffer | `BUFFER` | `onBackpressureBuffer()` | every item | memory, without limit | | bounded buffer | (none) | `onBackpressureBuffer(capacity, ...)` | up to capacity | then errors, or drops per `BackpressureOverflowStrategy` | | drop | `DROP` | `onBackpressureDrop()` | items requested in time | everything that arrives while busy | | latest | `LATEST` | `onBackpressureLatest()` | the newest pending item | intermediate items | | error | `ERROR` | (none) | nothing extra | the stream, on first overflow | | missing | `MISSING` | (none) | nothing decided here | defers to a later `onBackpressureXxx` | `BackpressureOverflowStrategy` has three values for a full bounded buffer: `ERROR`, `DROP_OLDEST` and `DROP_LATEST`. ## Choosing for a real stream 1. **Ask whether every item matters.** Payment events and log lines must not be dropped: buffer, with a capacity sized to the largest real burst, or slow the source down. 2. **Ask whether only the newest value matters.** Sensor readings, location fixes and download progress: `LATEST` keeps the screen current without a backlog. Unlike `DROP`, it never loses the final value of a burst. 3. **Ask whether stale work is harmful.** A periodic sync that may outlast its period: `onBackpressureDrop()` skips ticks that arrive while the previous run is busy. 4. **Place the operator directly above the slow step**, and keep that step's own prefetch small (for example `observeOn(scheduler, false, 1)`) so demand reflects the real consumer instead of a 128-item queue. 5. **Treat ERROR as a tripwire** for streams that should never overflow; an error in production then points at a real design problem. ## Observable, toFlowable and the Android reality - Most Android UI streams (clicks, text changes, one-shot calls) are slow and short, so `Observable` is the right type and backpressure never comes up. - When an `Observable` feeds something slow or bounded, convert it with `toFlowable(BackpressureStrategy.X)`; the parameter forces the overflow decision. - `Observable.observeOn` uses an unbounded queue. A producer that stays faster than its consumer does not fail fast; it grows the heap until the process runs out of memory. ## Diagnosing - The stack trace usually ends in `observeOn` or a flattening operator, but the cause is the non-backpressured source above it. - `MissingBackpressureException` and `QueueOverflowException` both pass through `RxJavaPlugins.onError` unwrapped when they cannot be delivered, because RxJava treats them as likely bugs rather than lifecycle noise.

  • Why did onBackpressureDrop() in front of observeOn drop nothing for a long time?
    `observeOn` prefetches 128 items by default and requests more as it drains, so the drop operator saw plenty of outstanding demand and passed items into `observeOn`'s queue. Dropping only began once that queue was full. Pass a small buffer size, as in `observeOn(scheduler, false, 1)`, so demand reflects the actual consumer.
  • When would you convert an Observable to a Flowable, and what must you decide?
    When an `Observable` feeds a slow or bounded consumer, such as a database writer or a Reactive Streams library. `toFlowable(BackpressureStrategy)` makes you state the overflow behaviour: `BUFFER`, `DROP`, `LATEST`, `ERROR`, or `MISSING` when a later `onBackpressureXxx` operator will decide.
  • Is onBackpressureBuffer() a safe default?
    Only for bounded bursts. Without a capacity it buffers without limit, so a producer that stays faster than its consumer turns into an out-of-memory crash later instead of an error now. Give it a capacity, and optionally a `BackpressureOverflowStrategy` such as `DROP_OLDEST`, so overflow is either a visible `MissingBackpressureException` or a deliberate loss.

  • An Observable throws MissingBackpressureException when its consumer is too slow.
  • Adding observeOn in front of the slow consumer fixes MissingBackpressureException.
  • onBackpressureBuffer() with no arguments is bounded at 128 items.
  • onBackpressureDrop and onBackpressureLatest lose exactly the same items.
  • Flowable is always better than Observable, so every UI event should use it.

Assumes RxJava 3.1.x; since 3.1.6 an overflowing internal operator queue reports QueueOverflowException, where older guides describe MissingBackpressureException.