In Django, why does catching IntegrityError inside a transaction.atomic block and carrying on lead to TransactionManagementError, and where should the try/except go?
answer
- the block cannot see what you swallowed
- needs_rollback flag on the connection
- next query is refused
- wrap the risky write in its own block
basics
~20 sWhen an ORM write fails inside atomic, Django marks the transaction as needing rollback; the next query raises TransactionManagementError and the block rolls back on exit. Put the try/except around an inner atomic block that wraps only the risky write.
solid answer
~40 s`save()`, `update()` and `delete()` run under a guard that, on a database error inside an `atomic` block, sets a "needs rollback" flag on the connection. Swallowing the `IntegrityError` does not clear it. Any further query in that block is refused before it reaches the database with `TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.` — and if you run no query, the block still sees the flag on exit and rolls everything back without raising. On PostgreSQL the server has aborted the transaction anyway. The fix is to give the risky statement its own inner `atomic` block and put the `try` *around* it: the failure rolls back to that block's savepoint, the flag is cleared, and the outer work can continue and commit.
code
python · 12 linesfrom django.db import IntegrityError, transaction
@transaction.atomic
def apply_transfer(ref, source_id, target_id, amount):
try:
with transaction.atomic():
TransferLog.objects.create(reference=ref, amount=amount)
except IntegrityError:
return "duplicate"
move_funds(source_id, target_id, amount)
return "applied"go deeper
Remember the rule from the docs: do not catch database errors inside an atomic block; catch them around one.
Explain the needs-rollback flag, why the next query is refused, and why exiting without a query still rolls back.
Diagnose the error from a chained traceback, fix it with an inner savepoint block, and review code under ATOMIC_REQUESTS for the same pattern.
Decide how the codebase signals expected constraint violations, a savepoint per risky write or checks before writing, given each savepoint's cost.
## The symptom A transfer inserts a `TransferLog` row with a unique `reference` so a retried request is not applied twice. The first version looks reasonable: ```python @transaction.atomic def apply_transfer(ref, source, target, amount): try: TransferLog.objects.create(reference=ref, amount=amount) except IntegrityError: return "duplicate" # looks handled... move_funds(source, target, amount) ``` On the duplicate path the function returns and — surprisingly — nothing it did survives; on other variants where code keeps querying after the `except`, the next query raises **`TransactionManagementError`**. ## What Django is doing 1. `Model.save()` (and so `create()`), `QuerySet.update()` and deletions run their SQL under an internal guard. If an exception is raised while the connection is **inside an atomic block**, the guard sets `connection.needs_rollback = True` and re-raises. 2. Your `except` swallows the exception, but the flag stays set: the `atomic` block decides commit or rollback by looking at *both* the exception and that flag. 3. Before executing any later SQL, Django's cursor wrapper checks the flag and raises `TransactionManagementError` with the message "An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block." 4. On exit, a block whose flag is set rolls back even though no exception left it — the duplicate-path `return` above is silently discarded. The database agrees: after an error PostgreSQL refuses further statements in that transaction until it is rolled back. Django refuses first, with a clearer message, and the behaviour is the same on every backend. ## The fix: a savepoint around the risky statement ```python @transaction.atomic def apply_transfer(ref, source, target, amount): try: with transaction.atomic(): # savepoint TransferLog.objects.create(reference=ref, amount=amount) except IntegrityError: return "duplicate" # outer transaction is healthy move_funds(source, target, amount) ``` Now the inner block rolls back to its savepoint and clears the flag at its own level before the exception reaches your `except`. The outer transaction is still usable, and on the duplicate path it commits (empty) cleanly. | Placement of `try/except` | After the error | |---|---| | Inside the only `atomic` block, around `create()` | Flag set; next query raises; exit rolls back silently | | Around an inner `atomic` block | Savepoint rolled back; queries allowed; outer can commit | | Around the outermost block | Whole transaction rolled back; handle it as a failure | ## Related rules - **Raw SQL**: if you catch an exception from `cursor.execute()` inside a block, Django's behaviour is documented as unspecified and database-dependent; use the same inner-block pattern. - **Signal handlers**: an exception raised by an ORM signal receiver during a save can produce the same broken-transaction state. - **In-memory state**: the rollback does not reset attributes of the instance you tried to save; reset them in the handler if the object is used later. - **`set_rollback(False)`** clears the flag by hand, but it is only safe after you have rolled back to a known-good savepoint yourself; using it to "unbreak" a block without that corrupts atomicity. ## How to spot it in review - An `except DatabaseError`/`IntegrityError` whose `try` body is an ORM write and whose enclosing scope is an `atomic` block or an `ATOMIC_REQUESTS` view. - Logs showing `TransactionManagementError` a line after a swallowed integrity error — the real cause is the earlier error, chained as the `__cause__`. - A handler that "handles" the error and returns success while the data never appears. ## Why Django chose this design Django could have tried to recover automatically, but it cannot know which statements your handler expects to keep. Rolling back to an implicit savepoint before every write would cost a round trip per statement and would hide bugs. Instead, the framework makes the rule explicit: **the unit that may fail is the unit you wrap in `atomic`**. The inner block both limits what is undone and documents, in the code, which write is allowed to fail. That is why the documentation's advice is phrased as a placement rule — catch around a block, never inside one.
- The TransactionManagementError points at an innocent query. How do you find the real cause?Django raises it `from` the stored original exception, so the traceback's chained cause shows the earlier database error — usually an `IntegrityError` that some `except` swallowed a few lines up. Search the enclosing atomic scope for a broad `except` around an ORM write.
- When is transaction.set_rollback(False) legitimate?Only after you have manually rolled back to a savepoint with `savepoint_rollback(sid)` inside the block, so the transaction really is back in a known-good state. Clearing the flag without that tells Django to commit a transaction whose failed part was never undone.
- Does this apply if the view runs under ATOMIC_REQUESTS rather than an explicit block?Yes. `ATOMIC_REQUESTS` wraps the view in `atomic`, so a swallowed `IntegrityError` in the view breaks the request's transaction the same way. The fix is the same inner `atomic` block around the risky write.
saying these in an interview costs you the question
- Catching the IntegrityError is enough to make the transaction usable again
- Only PostgreSQL has this problem, SQLite and MySQL keep going
- Call transaction.rollback() in the except block and continue
- The atomic block will commit because no exception left it
- Use set_rollback(False) whenever the error has been handled