skip to content

Idempotency & Duplicate Runs

Spring Batch refuses to run a completed instance again, but real safety comes from idempotent writers and upserts so a partial re-run does not double-apply. Interviewers press on this because a re-run that duplicates payments is the nightmare scenario.

part ofSpring Frameworkoverview, primer and where to startread it →
on this pageshow

explore

questions

4

What happens when you launch a Spring Batch job a second time with the exact same identifying job parameters, and why?

level: juniorimportance: must knowfreq 70%

answer

  1. job name + identifying params = JobInstance
  2. COMPLETED instance -> JobInstanceAlreadyCompleteException
  3. FAILED/STOPPED -> restart, same instance
  4. vary a param (incrementer) for fresh runs
  5. metadata in JobRepository tables

basics

~10 s

Spring Batch treats the same identifying parameters as the same JobInstance. If that instance already completed successfully, re-launching throws JobInstanceAlreadyCompleteException, so the job does not run again and duplicate work is avoided.

solid answer

~40 s

A JobInstance is identified by the job name plus its identifying job parameters. Spring Batch records every instance and its executions in the JobRepository (metadata tables). Re-launching with identical identifying parameters resolves to the same JobInstance. If that instance previously finished with COMPLETED status, the JobLauncher throws JobInstanceAlreadyCompleteException and refuses to run it — a built-in guard against accidental duplicate runs. If the previous execution FAILED or STOPPED, the same instance can be restarted, creating a new JobExecution that resumes from where it left off. To run genuinely fresh work each time (e.g. a daily job), you must vary an identifying parameter, typically via a JobParametersIncrementer.

code

java · 16 lines
java
// Launch #1: creates JobInstance for date=2026-07-22, runs to COMPLETED
JobParameters params = new JobParametersBuilder()
        .addString("date", "2026-07-22") // identifying by default
        .toJobParameters();
jobLauncher.run(billingJob, params); // OK

// Launch #2 with the SAME identifying params:
jobLauncher.run(billingJob, params);
// -> throws JobInstanceAlreadyCompleteException:
//    "A job instance already exists and is complete for parameters={date=2026-07-22}"

// To run again for genuinely new work, change an identifying parameter:
JobParameters tuesday = new JobParametersBuilder()
        .addString("date", "2026-07-23")
        .toJobParameters();
jobLauncher.run(billingJob, tuesday); // new JobInstance, runs fine

go deeper

for a junior

Must know that identical identifying params = same instance and that a completed instance can't be re-run.

for a middle

Should distinguish JobInstance vs JobExecution and know restart vs new-instance behavior.

for a senior

Explains the identifying flag, restart semantics, and how to deliberately produce fresh instances.

for a principal

Frames the guard as a correctness/idempotency contract and discusses parameter design (business key vs timestamp).

**Core model.** Spring Batch separates a *JobInstance* (a logical run of a job for a given set of inputs) from a *JobExecution* (a single physical attempt to run that instance). A `JobInstance` is uniquely identified by the **job name + the identifying job parameters**. All of this is persisted by the `JobRepository` into metadata tables (`BATCH_JOB_INSTANCE`, `BATCH_JOB_EXECUTION`, `BATCH_JOB_EXECUTION_PARAMS`, etc.). **Identifying vs non-identifying parameters.** Each `JobParameter` has an `identifying` boolean flag (default `true`). Only identifying parameters contribute to a JobInstance's identity. So launching with `date=2026-07-22` (identifying) creates one instance; launching again with the same `date` refers to the *same* instance. **The idempotency guard.** When you launch a job whose identifying parameters map to an existing JobInstance, Spring Batch inspects that instance's last execution: - If it **COMPLETED** successfully → `JobInstanceAlreadyCompleteException` is thrown (wrapped/surfaced by the launcher). The job body never executes. This is the framework preventing a duplicate run. - If it **FAILED** or was **STOPPED** → this is treated as a **restart**: a new JobExecution is created for the *same* JobInstance, and restartable steps resume from their saved state. - If an execution is still **running** for that instance → you get a `JobExecutionAlreadyRunningException`. **Why this exists.** It makes 'launch the monthly billing job for July' safe to invoke twice — the second invocation is rejected rather than double-charging customers. **How to run fresh work anyway.** For a job meant to run repeatedly (daily ETL), you deliberately change an identifying parameter each time. The idiomatic way is a `JobParametersIncrementer` (e.g. `RunIdIncrementer`, which adds/bumps a `run.id` parameter), or supplying a timestamp/business date. Then each launch is a new JobInstance and no exception occurs. **Gotchas.** - Passing a wall-clock `System.currentTimeMillis()` as identifying makes every run unique but pollutes the metadata and defeats the safety net — prefer a meaningful business key. - If you *want* restart behavior, do **not** change identifying parameters; changing them creates a brand-new instance instead of resuming the failed one. - The check is per JobInstance, not global — two different jobs with the same parameters are independent.

  • What is the difference between an identifying and a non-identifying job parameter?
    Only identifying parameters (the default, `identifying=true`) contribute to a JobInstance's identity. Non-identifying parameters (e.g. metadata like a `verbose` flag) are stored and passed to the job but do not create a new instance, so they can change between restarts without breaking resume.
  • If the first execution failed, does re-launching with the same parameters throw the exception?
    No. A FAILED or STOPPED instance is restartable, so re-launching creates a new JobExecution on the same JobInstance and resumes. The exception is only thrown when the instance already COMPLETED successfully.

saying these in an interview costs you the question

  • Thinking every job launch always creates a new run regardless of parameters
  • Believing the exception is thrown for failed jobs too (it's only for COMPLETED instances)
  • Confusing JobInstance with JobExecution

context

open as a page

How do you make a Spring Batch job re-runnable on demand (e.g. every night) without hitting JobInstanceAlreadyCompleteException?

level: middleimportance: must knowfreq 60%

basics

~10 s

Add a JobParametersIncrementer, such as RunIdIncrementer, so each launch gets a new identifying parameter (like run.id). That makes every launch a new JobInstance, so the completed-instance guard never triggers.

open as a page

Even with restart support, why must a Spring Batch ItemWriter be idempotent, and how do you make one idempotent?

level: seniorimportance: should knowfreq 50%

basics

~20 s

On restart after a failure, Spring Batch re-reads and re-writes the chunk that was in flight when the job crashed, so a writer can see the same items twice. Making writes upserts (insert-or-update on a key) keeps re-runs safe.

open as a page

Design the launch/idempotency strategy for a nightly financial reconciliation job that must be safe to re-trigger by an operator, resumable after a crash, and must never double-post entries. What are the trade-offs?

level: principalimportance: should knowfreq 35%

basics

~20 s

Key the job by business date (identifying param). Same date + completed = rejected (no double-run); same date + failed = restart and resume. Make writers idempotent via upserts so replayed chunks never double-post. Reserve an incrementer only for deliberately advancing to a new date.

open as a page