skip to content

How does a chunk-oriented step resume at the last good chunk on restart? What role does ExecutionContext play?

level: middleimportance: must knowfreq 55%

answer

  1. ExecutionContext saved in same txn as chunk commit
  2. ItemStream open/update/close = restart markers
  3. reader.open() seeds position on restart
  4. failing chunk is re-read → idempotency
  5. saveState=false kills resume

basics

~20 s

Each committed chunk saves progress (like the current read position) into the step's persisted ExecutionContext. On restart, the reader is re-opened and reads that saved state, so processing continues from just after the last successfully committed chunk instead of the beginning.

solid answer

~40 s

A chunk-oriented step reads N items, processes them, writes them, and commits in one transaction. In the SAME transaction that commits the chunk, Spring Batch persists the StepExecution's ExecutionContext to the JobRepository. Stateful readers (e.g. FlatFileItemReader, JdbcCursorItemReader, JpaPagingItemReader) implement ItemStream: their update() writes a restart marker (line count, page, or saved key) into that ExecutionContext. On restart, the new StepExecution is seeded with the last persisted ExecutionContext, and the reader's open() uses the marker to skip already-processed items. Because the context is committed atomically with the chunk, the recorded position and the written data can never diverge — you resume exactly after the last committed chunk. Items in the failing (uncommitted) chunk are rolled back and re-read, so processing must tolerate re-processing that partial chunk.

code

java · 26 lines
java
// A stateful reader persists its position via ItemStream so restart resumes.
@Bean
@StepScope
public FlatFileItemReader<Order> orderReader(
        @Value("#{jobParameters['inputFile']}") Resource file) {
    return new FlatFileItemReaderBuilder<Order>()
            .name("orderReader")          // key prefix for its ExecutionContext markers
            .resource(file)
            .delimited()
            .names("id", "amount")
            .targetType(Order.class)
            .saveState(true)              // default; MUST stay true to be restartable
            .build();
}

@Bean
public Step importStep(JobRepository repo, PlatformTransactionManager tx,
                       FlatFileItemReader<Order> reader, ItemWriter<Order> writer) {
    return new StepBuilder("importStep", repo)
            .<Order, Order>chunk(500, tx)   // commit every 500 items = checkpoint
            .reader(reader)
            .writer(writer)
            .build();
}
// On restart the reader's ExecutionContext holds ~"orderReader.read.count",
// so open() skips the already-committed lines.

go deeper

for a junior

Know that progress is saved per chunk and restart continues from there, not the start.

for a middle

Explain ItemStream open/update, the reader markers, and that the failing chunk is re-read.

for a senior

Emphasize the same-transaction atomicity of chunk write + ExecutionContext persistence, and idempotency requirements.

for a principal

Reason about multi-threaded/partitioned restart limits, saveState trade-offs, and resuming safely against mutable sources.

## Chunk-oriented processing recap A chunk step loops: read one item at a time until `chunk-size` items are gathered, hand the list to the processor and then the writer, and **commit the transaction**. The commit interval = chunk size. Each committed chunk is a durable checkpoint. ## The two ExecutionContexts Spring Batch keeps two `ExecutionContext` maps (both persisted in the JobRepository): - **Step ExecutionContext** — scoped to one StepExecution; holds per-step restart state (reader position, writer state). - **Job ExecutionContext** — scoped to the JobExecution; for cross-step data. For restart-within-a-step, the **Step ExecutionContext** is what matters. ## ItemStream: how readers persist position Stateful readers/writers implement the `ItemStream` interface with three methods: - `open(ExecutionContext)` — called at step start (and restart). On restart the passed context contains the previously saved keys, so the reader can *seek* to the right spot. - `update(ExecutionContext)` — called by the framework as part of each chunk; the reader writes its current marker (e.g. `FlatFileItemReader` stores the read line count under a key like `"<name>.read.count"`). - `close()` — releases resources. Examples of markers: `FlatFileItemReader` → lines read; `JpaPagingItemReader`/`JdbcPagingItemReader` → page number; `JdbcCursorItemReader` → row count. Custom readers should implement `ItemStream` (or extend `AbstractItemCountingItemStreamItemReader`) to be restartable. ## Atomicity is the key guarantee The crucial detail: the framework persists the updated ExecutionContext **in the same transaction that commits the chunk's writes**. So either both the data and the saved position commit, or neither does. This is why you can trust that on restart the marker points exactly to the boundary of durable work — there is no window where data is written but the position is not recorded (or vice-versa). ## What actually happens on restart 1. New JobExecution + new StepExecution created for the failing step. 2. The last persisted Step ExecutionContext is loaded into the new StepExecution. 3. `open()` seeds the reader from the markers → reader skips the already-committed items. 4. Reading/processing/writing continues from the next un-processed item. 5. Steps that already `COMPLETED` in the prior run are skipped entirely (unless `allowStartIfComplete`). ## Gotchas - **The failing chunk is re-read.** Items in the in-flight, uncommitted chunk roll back and are re-processed on restart. Your processor/writer should be **idempotent** or tolerate re-processing (this is at-least-once semantics per chunk). - **`saveState`**: readers have a `saveState` flag (default true). Setting it false stops the reader from persisting its position → that reader won't resume correctly. People sometimes disable it for multi-threaded steps and then lose restartability. - **Multi-threaded / partitioned steps**: a plain multi-threaded step reads items out of a single reader concurrently, so a simple line-count marker is not reliable — restart guarantees weaken; you typically disable saveState and accept non-restartability, or use partitioning where each partition has its own StepExecution/context. - **Non-restartable resources**: if the underlying source changed between runs (file replaced, rows shifted), a positional marker can resume at the wrong logical place. Prefer stable ordering / keyed resumption. - **Custom readers without ItemStream** don't save anything, so on restart they start from the beginning — silently reprocessing everything. ## When to rely on it Use chunk-based restart for large sequential imports/exports where reprocessing everything is expensive. Combine with a persistent JobRepository and idempotent writes for correct at-least-once behavior.

  • Why must the writer/processor be idempotent for safe restart?
    The chunk that was in-flight when the job failed was rolled back and its items are re-read and re-processed on restart. That yields at-least-once processing for those items, so re-applying them must not double-count or corrupt data.
  • What does saveState=false do and when might you set it?
    It stops the reader from writing its position into the ExecutionContext, so the step can't resume mid-way. It's sometimes set for multi-threaded steps where a single positional marker is meaningless, at the cost of restartability.

saying these in an interview costs you the question

  • Thinking the ExecutionContext is saved on a timer/periodically rather than atomically with each chunk commit
  • Believing restart resumes at the exact failing item rather than at the last committed chunk boundary (the failing chunk is re-read)
  • Claiming any reader is automatically restartable even without implementing ItemStream
  • Expecting exactly-once semantics for the in-flight chunk

context

resolving links…