In SQLAlchemy, why do the docs steer audit stamping from mapper before_update listeners to Session before_flush, and what may each one change?
answer
- is the flush plan already decided?
- new, dirty and deleted collections
- row-local attributes plus the Connection
- Session.add() inside a flush warns
basics
~20 sSession before_flush runs before SQLAlchemy plans the flush, so it may add, delete or change any object. Mapper before_update runs mid-flush with the plan fixed: only the row's own column attributes and SQL on the given Connection are safe.
solid answer
~40 s`before_flush(session, flush_context, instances)` fires before the unit of work computes its INSERT, UPDATE and DELETE plan. It walks `session.new`, `session.dirty` and `session.deleted` and may add objects, delete them or change any attribute; the changes join the flush. `session.dirty` is optimistic, so check `session.is_modified(obj)` before stamping. Mapper events such as `before_update(mapper, connection, target)` fire per object after the plan is fixed: setting a column on `target` (`target.updated_by`) is fine, and so is SQL on the passed `Connection`, but `Session.add()`, `Session.delete()`, collection changes and relationship assignments are unsupported; `Session.add()` there emits an `SAWarning` that results may not be consistent. Neither hook sees ORM bulk DML or Core statements. In 2.1, autoflush also runs before `text()` executions, so `before_flush` can fire there too.
code
python · 61 linesfrom contextvars import ContextVar
from sqlalchemy import String, create_engine, event, select, text
from sqlalchemy.orm import DeclarativeBase, Mapped, Session, mapped_column, sessionmaker
current_actor: ContextVar[str] = ContextVar("current_actor", default="system")
class Base(DeclarativeBase):
pass
class HasUpdatedBy:
updated_by: Mapped[str | None] = mapped_column(String(50))
class Invoice(HasUpdatedBy, Base):
__tablename__ = "invoice"
id: Mapped[int] = mapped_column(primary_key=True)
total: Mapped[int]
class AuditEntry(Base):
__tablename__ = "audit_entry"
id: Mapped[int] = mapped_column(primary_key=True)
table_name: Mapped[str] = mapped_column(String(50))
row_id: Mapped[int | None]
actor: Mapped[str] = mapped_column(String(50))
engine = create_engine("sqlite://")
Base.metadata.create_all(engine)
SessionLocal = sessionmaker(engine)
@event.listens_for(SessionLocal, "before_flush")
def stamp_and_audit(session: Session, flush_context, instances) -> None:
actor = current_actor.get()
for obj in list(session.new) + list(session.dirty):
if isinstance(obj, HasUpdatedBy) and (
obj in session.new or session.is_modified(obj)
):
obj.updated_by = actor
if obj not in session.new:
session.add(
AuditEntry(table_name=obj.__tablename__, row_id=obj.id, actor=actor)
)
current_actor.set("alice")
with SessionLocal() as session, session.begin():
session.add(Invoice(id=1, total=100))
with SessionLocal() as session, session.begin():
invoice = session.get(Invoice, 1)
invoice.total = 120
session.execute(text("SELECT 1")) # 2.1: autoflush runs here too
with SessionLocal() as session:
print(session.scalars(select(Invoice.updated_by)).all())
print(session.execute(select(AuditEntry.table_name, AuditEntry.row_id, AuditEntry.actor)).all())go deeper
Recall that before_flush is a Session event, before_insert and before_update are mapper events, and that before_flush is the documented general-purpose hook.
Explain the flush plan timing, what is unsupported inside mapper events, why session.dirty needs is_modified(), and the after_commit restriction.
Diagnose the Session.add() SAWarning, move object creation into before_flush or connection-level SQL, and spot stamps that create needless UPDATEs.
Decide which hook family a codebase standardises on for persistence side work, and how its invisible behaviour is documented and tested.
## Two layers of persistence events SQLAlchemy's **flush** is the step where the Session turns pending changes on objects into INSERT, UPDATE and DELETE statements. Two families of events hook into it at different moments: | | Session `before_flush` | Mapper `before_insert` / `before_update` / `before_delete` | |---|---|---| | Registered on | a `Session`, `sessionmaker` or the `Session` class | a mapped class (or a base with `propagate=True`) | | Signature | `(session, flush_context, instances)` | `(mapper, connection, target)` | | Fires | once per flush, before the plan exists | per object, after the plan is decided | | May add or delete objects | yes | no | | May change any attribute | yes, including relationships | only column attributes local to the row | | May emit SQL | through the Session | directly on the given `Connection` | ## What before_flush can do The docs call `before_flush` by far the most generally useful flush hook. Inside it, it is safe to manipulate the Session's state: attach new objects, delete objects, change attributes freely. The changes are pulled into the flush when the hook returns. A typical listener scans three collections: - `session.new`: pending objects about to be inserted. - `session.dirty`: persistent objects with attribute changes. - `session.deleted`: objects marked with `Session.delete()`. `session.dirty` is **optimistic**: any attribute set operation places an object there, even when the value did not actually change. `session.is_modified(obj)` compares current values with the loaded ones and tells you whether an UPDATE will really be emitted. Stamping `updated_by` on every object in `session.dirty` without that check turns no-op assignments into real UPDATEs. ## What mapper events can and cannot do Mapper-level flush events fire while the flush is already executing. The docs say the plan of every INSERT, UPDATE and DELETE has *already been decided* when they are called, so the only changes possible are to attributes **local to the object's row**. Unsupported operations: 1. `Session.add()` and `Session.delete()`. 2. Mapped collection operations: append, add, remove, discard. 3. Relationship attribute set or delete, such as `invoice.customer = other`. Calling `Session.add()` there produces this warning: ```plaintext SAWarning: Usage of the 'Session.add()' operation is not currently supported within the execution stage of the flush process. Results may not be consistent. Consider using alternative event listeners or connection-level operations instead. ``` The object may even get inserted in a simple case; the warning says the result is not guaranteed, not that it always fails. The supported alternative inside a mapper event is plain SQL on the `Connection` argument, for example `connection.execute(insert(AuditEntry.__table__).values(...))`, which the docs encourage for incrementing counters or writing log rows. `before_update` also has a subtle trigger: it fires for **every object marked dirty, even those with no net changes to column attributes**. If the listener then sets `target.updated_by`, it creates a net change and therefore an UPDATE that would otherwise not have happened. ## Choosing between them - Use **`before_flush`** to create related objects (an `AuditEntry` per changed row), to stamp several kinds of objects from one place, or to validate the state of the whole unit of work. - Use a **mapper event** for a narrow, per-class rule that touches only the row's own columns or writes with raw SQL on the connection; it saves you from scanning collections. - Consider `__init__` or a validator when the extra state can be set before the object ever reaches a flush. ## What neither hook sees Both families belong to the unit-of-work flush. The docs state that mapper events do not apply to ORM-enabled `update()` and `delete()` statements; ORM bulk INSERT from a list of dictionaries and Core statements bypass objects too, so no flush hook stamps them. `do_orm_execute` is the hook that sees ORM-enabled DML. ## Neighbouring Session events - `after_flush` runs after the SQL is emitted but before object state is updated, so `session.new` and friends still show what was flushed. - `after_commit` runs when the Session is no longer in a transaction: it **cannot emit SQL**, and trying raises `InvalidRequestError` saying the session is in 'committed' state. Use `before_commit` for SQL that must run in every transaction. - `after_rollback` fires only when a real DBAPI rollback happens, not again for a `Session.rollback()` call whose DBAPI transaction was already rolled back. ## The 2.1 autoflush change SQLAlchemy 2.1 made autoflush unconditional: the Session flushes before every execution, Core `text()` included, not only before ORM statements. With pending changes, `session.execute(text("SELECT 1"))` now fires `before_flush` too, so a stamping listener can run earlier than code written for 2.0 expected.
- Can a SQLAlchemy after_commit listener write the audit row instead?No. When `after_commit` fires, the Session is not in an active transaction, so `session.execute()` raises `InvalidRequestError` because the session is in 'committed' state. Any row that must commit with the change belongs in `before_flush` or `before_commit`; `after_commit` is for work outside the database that must not happen unless the commit succeeded.
- In SQLAlchemy 2.1, why might a before_flush listener run during session.execute(text(...))?2.1 made autoflush unconditional: the Session flushes pending changes before every execution, Core and `text()` included, whereas 2.0 autoflushed only before ORM statements. If the Session holds new or dirty objects and `autoflush` is on, the raw SQL call triggers a flush first, and `before_flush` with it.
before_flush is amending the packing list before the truck is loaded: add boxes, remove boxes, relabel anything. A mapper event is standing at the conveyor while boxes go in, in a fixed order: you may relabel the box in your hands, or note it in a separate log book, but a new box handed over now may or may not make it onto this truck.
- Session.add() inside before_update is fine; the new object just joins the flush.
- before_update fires only for objects whose columns have net changes.
- Every object in session.dirty is guaranteed to produce an UPDATE.
- after_commit is a safe place to insert the audit row for the transaction.
- before_flush lets you stamp the rows a session.execute(update(Invoice)) statement changes.
Assumes SQLAlchemy 2.1, where autoflush, and with it before_flush, also runs before Core and text() executions.