What thread-safety and executor concerns arise from running steps in a split, and how do you mitigate them?
answer
- branches = different threads, same context
- no shared stateful reader/writer; @StepScope
- bounded ThreadPoolTaskExecutor, named threads
- pool size <= DB connection pool
- blocking branch hangs the join -> timeouts
basics
~10 sBranches run on different threads, so any shared mutable state can race. Use a bounded ThreadPoolTaskExecutor (not unbounded SimpleAsyncTaskExecutor), keep readers/writers per-branch or thread-safe, and size the pool against your DB connection pool.
solid answer
~40 sBecause a split submits each branch to a TaskExecutor, branches execute concurrently on separate threads sharing the same ApplicationContext. Two classes of concern follow. First, thread-safety: singleton beans with mutable fields, shared ItemReaders/ItemWriters, or shared collections can be raced. Give each branch its own reader/writer (or step-scoped beans) and avoid shared mutable state. Second, the executor: SimpleAsyncTaskExecutor spawns a new, unbounded thread per branch by default — fine for a couple of branches but dangerous at scale; a bounded ThreadPoolTaskExecutor caps concurrency. Also size the pool relative to the JDBC connection pool, since each branch step opens transactions and hits the JobRepository — more concurrent branches than available connections just causes waiting or exhaustion. Finally, a branch that blocks forever stalls the join, hanging the whole job, so bound external calls with timeouts.
code
java · 24 lines@Bean
TaskExecutor splitExecutor() {
ThreadPoolTaskExecutor exec = new ThreadPoolTaskExecutor();
exec.setCorePoolSize(4);
exec.setMaxPoolSize(4); // bounded: never more than 4 branch threads
exec.setQueueCapacity(0); // fail fast rather than queue silently
exec.setThreadNamePrefix("split-"); // shows up in logs/thread dumps
exec.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
exec.initialize();
return exec;
}
// Per-branch reader: @StepScope gives each branch its own instance,
// so no shared cursor state is raced across threads.
@Bean
@StepScope
FlatFileItemReader<Row> branchReader(@Value("#{jobParameters['file']}") String file) {
return new FlatFileItemReaderBuilder<Row>()
.name("branchReader")
.resource(new FileSystemResource(file))
.delimited().names("a", "b")
.targetType(Row.class)
.build();
}go deeper
Aware that parallel branches run on different threads.
Chooses a bounded executor, avoids shared stateful readers, and knows @StepScope isolates per-branch state.
Sizes pools against DB connections, guards against hung joins with timeouts, and reasons about metadata contention.
Sets executor/connection-pool conventions and idempotency/observability standards for all parallel batch work.
## Why concurrency concerns appear A split runs each branch by submitting it to a **`TaskExecutor`**, so branches run on **different threads** at the same time, all inside the **same Spring `ApplicationContext`**. Anything they share is shared across threads. ## Thread-safety concerns 1. **Singleton beans with mutable fields.** By default Spring beans are singletons. If two branches invoke a bean that mutates an instance field, you get a data race. Keep such beans stateless, or make branch-specific state **step-scoped** (`@StepScope`) so each `StepExecution` gets its own instance. 2. **Shared `ItemReader`/`ItemWriter`.** Most readers are **stateful** (they track a position). Sharing one reader across parallel branches corrupts its state. Give each branch its **own** reader/writer instance. 3. **Shared mutable collections / caches.** Use thread-safe structures or, better, per-branch data. 4. **`ExecutionContext`.** Each branch step has its **own** `StepExecution`/`ExecutionContext`, which is good — but the shared **`JobExecutionContext`** should not be written concurrently by branches expecting isolation. > Note: this is *less* fraught than a multi-threaded step, where a *single* reader is hit by many threads. In a split, each branch is a separate step, so you mostly just avoid *sharing* stateful beans between branches. ## Executor concerns 1. **`SimpleAsyncTaskExecutor`** creates a **new thread per task** and, unless you call `setConcurrencyLimit(...)`, is **unbounded**. Acceptable for a fixed handful of branches; risky if branch count or reuse grows. 2. **`ThreadPoolTaskExecutor`** is the production choice — set `corePoolSize`/`maxPoolSize` to cap concurrency and a `threadNamePrefix` for diagnosability. 3. **`SyncTaskExecutor`** runs branches sequentially on the caller thread — no parallelism; only useful for tests. ## Sizing against the database Every branch step runs transactions and writes `StepExecution` rows to the **`JobRepository`**. If you allow more concurrent branches than your **JDBC connection pool** (e.g. HikariCP `maximumPoolSize`) can serve, branches block waiting for connections — or you exhaust the pool and stall unrelated work. Rule of thumb: split/partition concurrency should fit within the connection pool with headroom. ## The hang risk The split **joins** — it waits for every branch. A branch that blocks on a slow external call **without a timeout** stalls the join indefinitely, hanging the whole job. Bound external I/O with timeouts and consider circuit breakers. ## Mitigation checklist - Bounded `ThreadPoolTaskExecutor`, named threads. - No shared stateful readers/writers; use `@StepScope` for per-branch state. - Stateless (or synchronized) shared beans. - Pool size <= DB connection capacity, with headroom. - Timeouts on external calls in branches. - Make each branch idempotent (no cross-branch rollback).
- Why can sharing a single ItemReader across split branches corrupt data?Most readers are stateful — they hold a cursor/position. Concurrent branches on different threads mutate that position simultaneously, so items get skipped, duplicated, or read out of order. Each branch needs its own reader instance, e.g. via @StepScope.
- How does the split's thread-pool size relate to the JDBC connection pool?Each concurrent branch runs transactions and writes JobRepository rows, consuming a DB connection. If branch concurrency exceeds the connection pool size, branches block on connection acquisition or exhaust the pool. Size the executor to fit within the connection pool with headroom.
- Sharing one stateful ItemReader across parallel branches
- Using unbounded SimpleAsyncTaskExecutor for many branches
- Ignoring DB connection pool limits when sizing the executor
- No timeout on branch external calls, risking a hung join