skip to content

What thread-safety and executor concerns arise from running steps in a split, and how do you mitigate them?

level: middleimportance: should knowfreq 30%

answer

  1. branches = different threads, same context
  2. no shared stateful reader/writer; @StepScope
  3. bounded ThreadPoolTaskExecutor, named threads
  4. pool size <= DB connection pool
  5. blocking branch hangs the join -> timeouts

basics

~10 s

Branches run on different threads, so any shared mutable state can race. Use a bounded ThreadPoolTaskExecutor (not unbounded SimpleAsyncTaskExecutor), keep readers/writers per-branch or thread-safe, and size the pool against your DB connection pool.

solid answer

~40 s

Because a split submits each branch to a TaskExecutor, branches execute concurrently on separate threads sharing the same ApplicationContext. Two classes of concern follow. First, thread-safety: singleton beans with mutable fields, shared ItemReaders/ItemWriters, or shared collections can be raced. Give each branch its own reader/writer (or step-scoped beans) and avoid shared mutable state. Second, the executor: SimpleAsyncTaskExecutor spawns a new, unbounded thread per branch by default — fine for a couple of branches but dangerous at scale; a bounded ThreadPoolTaskExecutor caps concurrency. Also size the pool relative to the JDBC connection pool, since each branch step opens transactions and hits the JobRepository — more concurrent branches than available connections just causes waiting or exhaustion. Finally, a branch that blocks forever stalls the join, hanging the whole job, so bound external calls with timeouts.

code

java · 24 lines
java
@Bean
TaskExecutor splitExecutor() {
    ThreadPoolTaskExecutor exec = new ThreadPoolTaskExecutor();
    exec.setCorePoolSize(4);
    exec.setMaxPoolSize(4);              // bounded: never more than 4 branch threads
    exec.setQueueCapacity(0);           // fail fast rather than queue silently
    exec.setThreadNamePrefix("split-"); // shows up in logs/thread dumps
    exec.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
    exec.initialize();
    return exec;
}

// Per-branch reader: @StepScope gives each branch its own instance,
// so no shared cursor state is raced across threads.
@Bean
@StepScope
FlatFileItemReader<Row> branchReader(@Value("#{jobParameters['file']}") String file) {
    return new FlatFileItemReaderBuilder<Row>()
            .name("branchReader")
            .resource(new FileSystemResource(file))
            .delimited().names("a", "b")
            .targetType(Row.class)
            .build();
}

go deeper

for a junior

Aware that parallel branches run on different threads.

for a middle

Chooses a bounded executor, avoids shared stateful readers, and knows @StepScope isolates per-branch state.

for a senior

Sizes pools against DB connections, guards against hung joins with timeouts, and reasons about metadata contention.

for a principal

Sets executor/connection-pool conventions and idempotency/observability standards for all parallel batch work.

## Why concurrency concerns appear A split runs each branch by submitting it to a **`TaskExecutor`**, so branches run on **different threads** at the same time, all inside the **same Spring `ApplicationContext`**. Anything they share is shared across threads. ## Thread-safety concerns 1. **Singleton beans with mutable fields.** By default Spring beans are singletons. If two branches invoke a bean that mutates an instance field, you get a data race. Keep such beans stateless, or make branch-specific state **step-scoped** (`@StepScope`) so each `StepExecution` gets its own instance. 2. **Shared `ItemReader`/`ItemWriter`.** Most readers are **stateful** (they track a position). Sharing one reader across parallel branches corrupts its state. Give each branch its **own** reader/writer instance. 3. **Shared mutable collections / caches.** Use thread-safe structures or, better, per-branch data. 4. **`ExecutionContext`.** Each branch step has its **own** `StepExecution`/`ExecutionContext`, which is good — but the shared **`JobExecutionContext`** should not be written concurrently by branches expecting isolation. > Note: this is *less* fraught than a multi-threaded step, where a *single* reader is hit by many threads. In a split, each branch is a separate step, so you mostly just avoid *sharing* stateful beans between branches. ## Executor concerns 1. **`SimpleAsyncTaskExecutor`** creates a **new thread per task** and, unless you call `setConcurrencyLimit(...)`, is **unbounded**. Acceptable for a fixed handful of branches; risky if branch count or reuse grows. 2. **`ThreadPoolTaskExecutor`** is the production choice — set `corePoolSize`/`maxPoolSize` to cap concurrency and a `threadNamePrefix` for diagnosability. 3. **`SyncTaskExecutor`** runs branches sequentially on the caller thread — no parallelism; only useful for tests. ## Sizing against the database Every branch step runs transactions and writes `StepExecution` rows to the **`JobRepository`**. If you allow more concurrent branches than your **JDBC connection pool** (e.g. HikariCP `maximumPoolSize`) can serve, branches block waiting for connections — or you exhaust the pool and stall unrelated work. Rule of thumb: split/partition concurrency should fit within the connection pool with headroom. ## The hang risk The split **joins** — it waits for every branch. A branch that blocks on a slow external call **without a timeout** stalls the join indefinitely, hanging the whole job. Bound external I/O with timeouts and consider circuit breakers. ## Mitigation checklist - Bounded `ThreadPoolTaskExecutor`, named threads. - No shared stateful readers/writers; use `@StepScope` for per-branch state. - Stateless (or synchronized) shared beans. - Pool size <= DB connection capacity, with headroom. - Timeouts on external calls in branches. - Make each branch idempotent (no cross-branch rollback).

  • Why can sharing a single ItemReader across split branches corrupt data?
    Most readers are stateful — they hold a cursor/position. Concurrent branches on different threads mutate that position simultaneously, so items get skipped, duplicated, or read out of order. Each branch needs its own reader instance, e.g. via @StepScope.
  • How does the split's thread-pool size relate to the JDBC connection pool?
    Each concurrent branch runs transactions and writes JobRepository rows, consuming a DB connection. If branch concurrency exceeds the connection pool size, branches block on connection acquisition or exhaust the pool. Size the executor to fit within the connection pool with headroom.

  • Sharing one stateful ItemReader across parallel branches
  • Using unbounded SimpleAsyncTaskExecutor for many branches
  • Ignoring DB connection pool limits when sizing the executor
  • No timeout on branch external calls, risking a hung join

context