A job must load 20 million rows from a file into one table. How do you choose between Hibernate's StatelessSession, a regular Session with periodic flush and clear, and plain JDBC batch inserts?
answer
- What does the job need from the ORM?
- Native COPY/LOAD DATA or staging + MERGE first
- Stateless = mappings without machinery
- Batched Session when cascades/events matter
- Identity ids defeat JDBC batching
basics
~20 sDecide by how much ORM machinery the rows actually need. Flat rows with no cascades or callbacks: StatelessSession, or plain JDBC if mappings add nothing. Graphs, events or auditing: batched Session with flush/clear. Pure single-table dumps: the database's native bulk loader beats all three.
solid answer
~60 sPick by what the job genuinely needs from the ORM. - **Native bulk load** (`COPY`, `LOAD DATA`, a bulk-insert utility) is the throughput winner for a plain single-table dump with no per-row logic. If nothing in the row needs Java, this is usually the right answer and often an order of magnitude faster. - **Plain JDBC batch** suits flat rows needing a little transformation but no mappings — no type converters, no inheritance, no embeddables. - **StatelessSession** is the middle ground: you keep the mappings, converters and dialect handling, but drop the persistence context, dirty checking, cascades, events and second-level cache. Memory is constant with no `clear()` discipline. You own insert ordering and any callback behaviour. - **Batched Session with `flush()`+`clear()` every N rows** is right when the job needs cascades, lifecycle callbacks, auditing, lazy navigation or the identity map. It costs the most per row but preserves domain behaviour. Whichever you pick, enable JDBC batching, use non-identity id generation, drop and rebuild indexes if you can, and measure.
code
java · 11 linesint batchSize = 500; // == hibernate.jdbc.batch_size
int i = 0;
for (SourceRow r : rows) {
session.persist(toEntity(r));
if (++i % batchSize == 0) {
session.flush();
session.clear();
}
}
session.flush();
session.clear();go deeper
Know the options exist and that a bulk load should not use a plain session without periodic flush and clear.
Contrast stateless versus batched stateful concretely, and name the mechanics that matter: JDBC batch size, flush-then-clear cadence, transaction chunking.
Diagnose before choosing — id generation strategy, index maintenance, statement counts — and be explicit about what stateless costs you in callbacks, cascades and cache coherence.
Lead with the decision axes and be willing to conclude the ORM should not be in the path at all: native bulk load into staging plus a set-based merge is frequently the right architecture, with restartability, idempotence and measurement designed in from the start.
## Start by asking what the ORM is for All four options can load 20 million rows. They differ in how much of Hibernate's machinery you pay for and how much behaviour you keep. The decision is therefore: **what does this job actually need from the object model?** ### Option 1 — the database's native bulk loader PostgreSQL `COPY`, MySQL `LOAD DATA INFILE`, and equivalents exist precisely for this. They bypass the JDBC statement path, often the SQL parser, and sometimes parts of the write path. For a plain file-into-one-table load with no per-row logic, nothing in Java will match them. Choose this when the rows arrive in (or can cheaply be turned into) a flat delimited form and the transformation is nil or expressible in SQL. A very common and very effective pattern is: bulk-load into a staging table, then do the real work with set-based SQL — one `INSERT ... SELECT` or `MERGE` from staging into the target. That gets bulk speed and keeps the logic in one place. Ruling this out too quickly is the single most common mistake in answers to this question. ### Option 2 — plain JDBC batch `PreparedStatement` with `addBatch()`/`executeBatch()`, batches of a few hundred to a few thousand, autocommit off. Fast, predictable, no ORM overhead at all. The cost is that you hand-write the SQL and the parameter binding, and you lose everything the mappings gave you: attribute converters, embeddables, inheritance strategies, enum handling, type mapping. That is fine for three columns and painful for thirty. ### Option 3 — StatelessSession The ORM-flavoured version of option 2. You keep the mappings, so an entity's converters, embedded types and column mapping all still apply, and the dialect writes correct SQL for you. You give up the persistence context, dirty checking, write-behind, cascades, collection handling, lifecycle events and interceptors, and second-level cache interaction. The operational advantage over a stateful session is that memory is **constant by construction** — nothing is retained, so there is no `clear()` cadence to get wrong and no risk of a forgotten reference growing the heap. The disadvantages are the responsibilities you inherit: insert ordering for dependent rows, foreign keys set by hand, any audit or timestamp behaviour previously done by callbacks re-implemented explicitly, and stale second-level cache entries for anything you write. This is the sweet spot when the data is flat, the mappings are non-trivial, and no cross-cutting behaviour must fire. ### Option 4 — regular Session with batched flush/clear Enable JDBC batching, then every N rows call `flush()` and `clear()`. This bounds memory while keeping cascades, lifecycle callbacks, the identity map, lazy navigation and second-level cache coherence. It is the slowest per row: every entity gets a persistence-context entry and a snapshot, every flush walks the dirty-checking loop, and every action goes through the event pipeline. Choose it when the job genuinely needs those things — writing an object graph with cascades, relying on auditing or generated timestamps implemented as callbacks, or interleaving reads that benefit from the identity map. Get the details right: match the flush cadence to `hibernate.jdbc.batch_size`, flush **before** clear, and remember that `clear()` detaches anything you still hold. ## Constraints that decide it for you - **Identity id generation kills batching.** With a database identity column, the driver must round-trip per insert to learn the key, so JDBC batching cannot group statements. Use a sequence with a pooled/hi-lo optimizer, or assign ids in the application, if you want batches to be real. This applies to stateless and stateful sessions alike and is the number-one reason a "batched" load turns out not to be. - **Indexes and constraints.** Writing 20 million rows into a heavily indexed table pays index maintenance per row. Where the table is offline for the load, dropping secondary indexes and rebuilding after is often the biggest single win, and dwarfs the ORM choice. - **Transaction sizing.** One 20-million-row transaction is a restartability and resource problem. Commit per chunk with a checkpoint so a failure resumes. - **Idempotence.** If reruns are possible, design for them — an upsert-style write or a staging-table merge — rather than hoping the job never fails halfway. ## How to present the decision A strong answer refuses to pick blind and instead names the axes: 1. Does any per-row **application logic** exist that must run in Java? If no → native bulk load or staging-table merge. 2. Do the rows need **mapping fidelity** (converters, embeddables, inheritance)? If yes → prefer a Hibernate option over hand-written JDBC. 3. Does anything need **cascades, callbacks, auditing, or cache coherence**? If yes → batched stateful `Session`. If no → `StatelessSession`. 4. Then fix the mechanics regardless of choice: batching enabled, non-identity ids, chunked transactions with checkpoints, indexes considered, and measurement in place. And close on measurement: state that you would run a representative subset through two candidates and compare rows-per-second and heap, because the ORM choice is frequently not the bottleneck — id generation, index maintenance, and network round trips usually are.
- Why can a database identity column make a "batched" insert job no faster at all?Because the driver has to retrieve the generated key for each row, which forces a round trip per insert and prevents JDBC from grouping statements into a batch. Hibernate disables insert batching for identity-generated ids for exactly this reason. Switching to a sequence with a pooled or hi-lo optimizer, or assigning ids in the application, restores real batching and is often the largest single improvement in a bulk-load job.
- When would a batched regular Session beat a StatelessSession despite being slower per row?When the job needs the machinery: writing an object graph where cascades save you from hand-ordering inserts and wiring foreign keys, relying on lifecycle callbacks or auditing that only fire through the event pipeline, needing lazy navigation while processing, or needing second-level cache coherence after the writes. Re-implementing all of that by hand around a stateless session is usually slower to write, easier to get wrong, and not much faster to run.
- What would you measure before committing to one approach?Rows per second and peak heap on a representative subset for two candidate approaches, with SQL statement counts captured so you can see whether batching is genuinely happening. Also check where the time actually goes: id generation round trips, index maintenance, network latency and commit frequency commonly dominate, in which case the choice between stateless and batched stateful barely moves the number.
Choosing here is like choosing a vehicle for moving house: a freight container (native bulk load) if it is just boxes, a van (stateless) if you want some handling, and a full moving crew (stateful session) only when the furniture must be disassembled, labelled and insured on the way.
saying these in an interview costs you the question
- Never considering the database's native bulk-load path or a staging-table merge.
- Assuming StatelessSession is always fastest, regardless of what the rows need.
- Enabling batching while using identity id generation and expecting batches.
- Calling clear() without flush() in the stateful variant, silently dropping writes.
- Loading everything in one enormous transaction with no checkpoint or resume path.