skip to content

What is the difference between filterCount and skipCount on a StepExecution?

level: middleimportance: should knowfreq 45%

answer

  1. filter = processor returns null, no error
  2. skip = exception + SkipPolicy tolerated it
  3. skipCount = read+process+write skips
  4. skips cause rollback + re-read; filters don't
  5. SkipListener fires only for skips

basics

~10 s

filterCount counts items an ItemProcessor intentionally dropped by returning null — normal, no error. skipCount counts items that threw an exception and were skipped by a skip policy instead of failing the step.

solid answer

~40 s

Both reduce how many items reach the writer, but for different reasons. filterCount increments when the ItemProcessor returns null for an item: a deliberate business decision to exclude it. No exception occurs, no rollback, the item simply isn't written. skipCount is the sum of readSkipCount, processSkipCount and writeSkipCount — items where an exception was thrown during read, process, or write and a configured SkipPolicy (via faultTolerant().skip(...).skipLimit(...)) chose to skip that item rather than fail the whole step. Skips are error-handling; filters are normal flow control. A skip on process/write also typically causes a chunk rollback and re-processing (raising rollbackCount), whereas a filter never does. In auditing, filterCount tells you 'valid input we chose to ignore'; skipCount tells you 'bad input we tolerated'.

code

java · 16 lines
java
// Filter: return null -> item is filtered (filterCount++)
public class ActiveOnlyProcessor implements ItemProcessor<User, User> {
    @Override
    public User process(User u) {
        return u.isActive() ? u : null;   // null => filtered, NOT written
    }
}

// Skip: exception tolerated by a fault-tolerant step (skipCount++)
Step step = new StepBuilder("import", jobRepository)
    .<User, User>chunk(100, txManager)
    .reader(reader).processor(new ActiveOnlyProcessor()).writer(writer)
    .faultTolerant()
    .skip(FlatFileParseException.class)   // bad line -> readSkipCount++
    .skipLimit(10)
    .build();

go deeper

for a junior

Know filter = processor returns null; skip = an error that was tolerated.

for a middle

Explain skipCount is the sum of read/process/write skips and needs faultTolerant().skip().skipLimit().

for a senior

Explain that process/write skips trigger rollback + single-item re-read, inflating rollbackCount, while filters don't.

for a principal

Use the counts to reconcile input/output for audit and to set alerting thresholds distinguishing intentional exclusion from data quality problems.

### Two very different mechanisms Both `filterCount` and `skipCount` describe items that were **read but never written**, but the cause and the machinery differ completely. #### filterCount — deliberate exclusion An `ItemProcessor<I,O>` transforms each item. If `process(item)` returns **`null`**, Spring Batch treats the item as **filtered**: it is excluded from the chunk sent to the `ItemWriter`, and `filterCount` is incremented. This is **normal control flow** — no exception, no rollback, no retry. It's how you say 'this record is valid input but I don't want it written' (e.g. skip records that don't match a business rule). #### skipCount — tolerated failure `skipCount()` on StepExecution returns `readSkipCount + processSkipCount + writeSkipCount`. A skip happens only in a **fault-tolerant step** (`stepBuilder.faultTolerant().skip(SomeException.class).skipLimit(n)`). When an item throws a configured exception during read, process, or write, the `SkipPolicy` decides whether to skip it. If skipped, the item is dropped and the appropriate `*SkipCount` increments; if the skip limit is exceeded, the step fails instead. ### Why the distinction matters - **Filter = intent**, no error. **Skip = error you chose to tolerate.** - A process/write **skip triggers a rollback and chunk re-processing** (Spring Batch re-reads the chunk one item at a time to isolate the bad item), so skips inflate `rollbackCount`. Filters never cause rollbacks. - `SkipListener` fires for skips (`onSkipInRead/Process/Write`); no such callback exists for filters. ### Common gotcha Candidates conflate the two because both 'remove' items. If your writer sees fewer items than were read and there were **no exceptions**, the gap is **filterCount**. If exceptions were caught by a skip policy, it's **skipCount**. Both can happen in the same step. ### Practical accounting With skips present: `readCount = writeCount + filterCount + processSkipCount + writeSkipCount` (read skips reduce readCount itself). Use these to reconcile input vs. output during audits.

  • Does filtering an item cause a chunk rollback?
    No. Returning null from the processor is normal flow — the item is just excluded from the write and filterCount increments. Only exceptions (leading to skip/retry) cause rollbacks and re-processing.

saying these in an interview costs you the question

  • Saying returning null from a processor causes a skip or an error
  • Claiming skipCount and filterCount are interchangeable
  • Not knowing skips require a fault-tolerant step with a skip limit

context

resolving links…