skip to content

Engine & Mapper Listeners

event.listens_for attaches hooks to engines, sessions and mappers, from before_cursor_execute to before_flush. Interviewers ask when a hook is the wrong place for business logic.

on this pageshow

explore

questions

6

In SQLAlchemy 2.1, what does the @validates decorator do on a mapped class, and when does it not run?

level: juniorimportance: should knowfreq 30%

answer

  1. a hook beside the column it guards
  2. assignment in Python, not loading
  3. the return value is what gets stored
  4. collections: appends yes, removals opt-in

basics

~20 s

SQLAlchemy's @validates marks a method that runs whenever the named attribute is set or its collection appended to; it raises to reject a value or returns the value to store. Loading rows, bulk and Core statements skip it.

solid answer

~40 s

`@validates("currency")` from `sqlalchemy.orm` turns a method with the signature `(self, key, value)` into a validator for that attribute. SQLAlchemy calls it on every assignment made by Python code, including the keyword arguments the default declarative constructor sets, and stores whatever it returns, so it can normalise (`value.upper()`) or raise `ValueError` to reject. On a `relationship()` collection it receives each appended object; removals arrive only with `include_removes=True`, and `include_backrefs=False` skips events caused by a backref. It is built on attribute events, so it does not run when the ORM populates an object from a row, nor for `session.execute(update(...))`, ORM bulk inserts from dicts or Core statements, which write SQL without touching an object. Since 2.1, a subclass validator with the same method name replaces the parent's; call `super()` to run both.

code

python · 37 lines
python
from sqlalchemy import String, create_engine, select, text
from sqlalchemy.orm import DeclarativeBase, Mapped, Session, mapped_column, validates


class Base(DeclarativeBase):
    pass


class Invoice(Base):
    __tablename__ = "invoice"
    id: Mapped[int] = mapped_column(primary_key=True)
    currency: Mapped[str] = mapped_column(String(3))

    @validates("currency")
    def normalise_currency(self, key: str, value: str) -> str:
        if len(value) != 3:
            raise ValueError(f"{key} must be a three-letter code")
        return value.upper()


invoice = Invoice(id=1, currency="eur")  # constructor kwargs pass through the validator
print(invoice.currency)  # EUR

try:
    invoice.currency = "euro"
except ValueError as exc:
    print(exc)  # currency must be a three-letter code

engine = create_engine("sqlite://")
Base.metadata.create_all(engine)
with Session(engine) as session, session.begin():
    session.add(invoice)
    session.execute(text("INSERT INTO invoice (id, currency) VALUES (2, 'usd')"))

with Session(engine) as session:
    print(session.scalars(select(Invoice.currency).order_by(Invoice.id)).all())
    # ['EUR', 'usd'] - loading row 2 does not run the validator

go deeper

for a junior

Recall the signature (self, key, value), that the returned value is stored, and that raising rejects the assignment, including keyword arguments passed to the constructor.

for a middle

Explain that validates() sits on attribute events, so it fires on set and append but never on load, ORM bulk DML or Core statements, and what include_removes changes.

for a senior

Treat validators as guards for objects built in Python only, back them with database constraints, and audit which code paths write through statements and skip them.

for a principal

Decide where a team's field rules live: validators for fast feedback, constraints for integrity, and a review rule for bulk paths that bypass both layers of the ORM.

## What a validator is A **validator** is a method on a mapped class that SQLAlchemy calls whenever a named attribute is assigned. You declare it with the `validates()` decorator from `sqlalchemy.orm`: ```python @validates("currency") def normalise_currency(self, key, value): if len(value) != 3: raise ValueError(f"{key} must be a three-letter code") return value.upper() ``` The method receives the object, the attribute name (`key`) and the incoming value. It decides the fate of that value in one of two ways: - **Raise** an exception, which aborts the assignment; the attribute keeps its old value and the exception reaches the caller. - **Return** a value, which is what SQLAlchemy actually stores. Returning a normalised value (`"EUR"` for `"eur"`) is the common use; forgetting the `return` stores `None`. One decorator can name several attributes (`@validates("email", "backup_email")`), which is why `key` is passed in. ## When it runs `validates()` is a convenience built on SQLAlchemy's **attribute events** (the `set`, `append` and `remove` events of `AttributeEvents`). It therefore fires on the operations those events cover: 1. Assigning a scalar attribute: `invoice.currency = "eur"`. 2. Constructing an object with keyword arguments: the default constructor that `DeclarativeBase` provides sets each attribute, so `Invoice(currency="eur")` is validated too. 3. Appending to a `relationship()` collection: the validator receives the related object being added. 4. Removing from a collection, but only when the decorator says `include_removes=True`; the method then takes a fourth argument, `is_remove`. With `include_backrefs=False`, the validator ignores events that arrive through a backref, for example when `address.user = user` appends to `user.addresses` behind the scenes. ## When it does not run The documentation states that validators, like all attribute extensions, are only called by normal userland code and are not issued when the ORM is populating the object. The gaps follow from that: | Write or read path | Validator runs? | Why | |---|---|---| | `obj.attr = value` | yes | an attribute set event | | `Model(attr=value)` | yes | the constructor assigns attributes | | loading a row with `select()` or `session.get()` | no | the ORM populates the object directly | | `session.execute(update(Model).values(...))` | no | ORM-enabled DML sends SQL without touching objects | | `session.execute(insert(Model), [dicts])` | no | bulk INSERT works from dictionaries | | `connection.execute(...)` or `text()` | no | Core never sees mapped objects | Two consequences follow. Rows that are already in the table, written by an older release, a migration or another client, are never re-checked; and any code path that writes through statements instead of objects bypasses the rule. A validator is a guard for objects built in Python, not a data-integrity constraint. Rules that must hold for every row belong in the database as well, as a `CHECK` or `NOT NULL` constraint. ## Validators and inheritance in 2.1 SQLAlchemy 2.1 changed how validators behave on subclasses. When a subclass defines a `@validates` method **with the same method name** as the parent's validator, only the subclass method now runs for instances of the subclass. To keep the parent rule, call it explicitly: ```python class VerifiedUser(User): @validates("email") def validate_email(self, key, address): address = super().validate_email(key, address) if not address.endswith("@example.com"): raise ValueError("only example.com addresses allowed") return address ``` Under 2.0 the parent validator was used regardless of the override, so code written for 2.0 that relied on that should be checked after the upgrade. ## Choosing between @validates and attribute events - **`@validates`** keeps the rule next to the column, always uses the return value and needs no registration call. It is the right default for per-field checks and normalisation. - **`event.listen(Invoice.total, "set", fn, retval=True)`** receives `(target, value, oldvalue, initiator)`, so it sees the previous value, can be registered from outside the class (a shared module of rules), and with `propagate=True` reaches subclasses. Without `retval=True`, the listener's return value is ignored and it can only observe or raise. Neither replaces flush-time logic: a validator sees one attribute at the moment of assignment, before the object is necessarily complete, so a rule that compares several fields is often clearer in a `before_flush` listener or in the service code that builds the object.

  • How is @validates different from listening to the attribute's "set" event directly?
    `@validates` is a convenience over `AttributeEvents`. A direct listener, `event.listen(Invoice.total, "set", fn, retval=True)`, receives `(target, value, oldvalue, initiator)`, so it sees the old value, can be registered outside the class, and covers subclasses with `propagate=True`. Without `retval=True` its return value is ignored. `@validates` keeps the rule beside the column and always stores its return value.
  • Does a SQLAlchemy validator run when a row is loaded or refreshed from the database?
    No. Validators are only called by userland code, not when the ORM populates an object, so loads, `refresh()` and reloads after expiry bypass them. Rows already in the table are never re-checked; existing bad data needs a data migration, and a rule that must always hold needs a database constraint such as `CHECK`.

  • @validates runs again on every row the ORM loads from the database.
  • A validator's return value is ignored; it can only raise to reject.
  • session.execute(update(Invoice).values(...)) passes each value through @validates.
  • A collection validator sees removals by default, without include_removes=True.
  • A @validates method makes a database CHECK constraint unnecessary.

Assumes SQLAlchemy 2.1; since 2.1 a same-named subclass validator replaces the parent's instead of being ignored.

open as a page

In SQLAlchemy, why do the docs steer audit stamping from mapper before_update listeners to Session before_flush, and what may each one change?

level: middleimportance: should knowfreq 35%

basics

~20 s

Session before_flush runs before SQLAlchemy plans the flush, so it may add, delete or change any object. Mapper before_update runs mid-flush with the plan fixed: only the row's own column attributes and SQL on the given Connection are safe.

open as a page

With SQLAlchemy's before_cursor_execute and after_cursor_execute events, how do you log every statement slower than a set threshold?

level: middleimportance: should knowfreq 32%

basics

~20 s

Register both events on the engine: before_cursor_execute pushes a start time onto conn.info, after_cursor_execute pops it and logs the SQL string when the elapsed time passes the threshold, leaving the bound parameters out of the log.

open as a page

In SQLAlchemy, how do event.listen() and @event.listens_for decide what a listener applies to, and when is propagate=True needed?

level: middleimportance: should knowfreq 30%

basics

~20 s

SQLAlchemy scopes a listener by the target passed in: an Engine instance or the Engine class, a Session or sessionmaker, a mapped class or attribute. Mapper, instance and attribute events reach subclasses only with propagate=True, which an unmapped base class always needs.

open as a page

A SQLAlchemy app adds a tenant filter to every query with do_orm_execute and with_loader_criteria; which statements does it still miss, and why?

level: seniorimportance: should knowfreq 25%

basics

~20 s

The hook rewrites only ORM statements run through the Session's execute methods, so it misses text() SQL, Core on a Connection, flush writes, aliased() entities without include_aliases=True, ORM UPDATE and DELETE behind an is_select guard, and identity-map hits.

open as a page

Should a team stamp updated_by through SQLAlchemy flush events, a mapped_column onupdate callable, or explicit service code, given the write paths each one misses?

level: principalimportance: nice to knowfreq 18%

basics

~20 s

It depends on the write paths: flush events see only the unit of work, an onupdate callable also covers ORM bulk and Core UPDATEs but not text(), and service code covers what developers remember. Compliance-grade audit belongs in the database.

open as a page