skip to content

When Django's squashmigrations meets RunPython operations, why does it report manual porting, and how does elidable=True change the squashed result?

level: seniorimportance: nice to knowfreq 20%

answer

  1. functions live in old files
  2. numbered modules
  3. an optimizer barrier
  4. disposable once applied

basics

~20 s

The squashed file references RunPython functions defined in the old numbered migration modules, which you will delete, so you copy them in by hand. Non-elidable RunPython and RunSQL also block the optimizer; elidable ones may be dropped.

solid answer

~50 s

`squashmigrations` copies operations, not code. A `RunPython` in `0017_backfill_totals` points at a function inside that migration module, and a numbered module name like that cannot be imported cleanly and will be deleted after the transition. So the writer strips the import, adds a comment listing the migrations whose functions need manual copying, and the command prints **Manual porting required**; you paste the functions into the squashed file and point `RunPython` at the local copies. Separately, `RunPython` and `RunSQL` act as **optimizer barriers**: an `AddField` before one and a `RemoveField` after it cannot cancel. Marking an operation `elidable=True` says its effect only mattered when it ran, so the optimizer may drop it and optimize across it, which suits one-off backfills. Passing `start_migration_name` to squash only the range after the last such operation is the other workaround.

go deeper

for a junior

Know that a squash with RunPython needs the functions copied into the new file by hand.

for a middle

Explain why the functions cannot be imported from old numbered modules and what the manual-porting comment asks you to do.

for a senior

Decide which scripted operations to mark elidable before squashing, use start_migration_name to limit the range, and verify the squash on a fresh database.

for a principal

Treat elidable as a data-lifecycle decision: agree which one-off data changes may vanish from history and which must stay for new installs.

## What squashing does with scripted operations `squashmigrations` gathers the operations of a range of migrations, optimizes them, and writes a new migration with `replaces` naming the originals. Schema operations are plain data (a field name, a field definition) and serialise easily. **Scripted operations** are different: - `RunPython(code, reverse_code)` holds references to Python functions. - `RunSQL(sql, reverse_sql)` holds opaque SQL strings the optimizer cannot reason about. ## Why functions need manual porting Data-migration functions are normally defined inside the migration file itself, for example `orders/migrations/0017_backfill_totals.py`. When the squashed migration is written, Django serialises each function by its import path, which points into that numbered module. Module names starting with a digit cannot be written as a normal `import` statement, and the old files are going to be deleted anyway. So the writer: 1. Removes the import of the old migration module. 2. Adds a comment at the top: functions from the following migrations need manual copying; move them and any dependencies into this file, then update the `RunPython` operations to refer to the local versions. 3. Makes the command print **Manual porting required** after it writes the file. Until you copy the functions in, the squashed file cannot run, and it will break outright once the old files are deleted. ## Scripted operations block the optimizer The optimizer shrinks a squash by combining or cancelling operations, but only across operations it can "optimize through". A non-elidable `RunPython` or `RunSQL` could depend on anything, so it is a barrier: | Sequence in the range | Without elidable | With `elidable=True` on the RunPython | |---|---|---| | `AddField(tmp)`, `RunPython(fill)`, `RemoveField(tmp)` | all three kept | the RunPython may be dropped and the field ops cancel | | `CreateModel`, `RunPython(backfill)`, `AddField` | AddField stays separate | AddField may fold into CreateModel | | a long run of `AlterField` around a `RunSQL` | only runs on each side merge | the whole run can merge | ## What elidable means `elidable=True` on `RunPython` or `RunSQL` declares that the operation's effect is only needed at the moment it runs on an existing database, typically a one-off backfill or clean-up. During squashing the optimizer may **remove** such an operation, and in doing so optimize across where it stood. Choose it carefully: - Good fit: backfilling a column that a later migration drops, or normalising values that every existing database already has. - Bad fit: seeding rows that a fresh database needs, such as default groups or lookup rows, because a squashed history would create an empty table on new installs. `elidable` changes nothing on a normal `migrate`; it only matters when migrations are squashed. `--no-optimize` disables the optimizer, so nothing is elided or combined. ## A practical checklist 1. Before squashing, scan the range for `RunPython` and `RunSQL`, and mark truly one-off ones `elidable=True` in their original files. 2. Consider `start_migration_name` to squash only the range after the last non-elidable scripted operation. 3. After squashing, copy every function named in the manual-porting comment into the new file and update the `RunPython` references. 4. Run the squashed migration on a fresh database, and compare the resulting schema with one built from the original files.

  • Why is elidable=True wrong for a RunPython that creates default permission groups?
    Fresh databases built from the squashed history would never get those groups, because the optimizer is allowed to drop the operation. Only operations whose effect every future database can do without, such as a one-off backfill, should be elidable.
  • What does --no-optimize change for a squash containing RunPython?
    The operations are concatenated as they are, with no folding and no eliding, so elidable operations stay in the file. The functions still have to be copied in by hand; --no-optimize only avoids optimizer mistakes, not the porting step.

saying these in an interview costs you the question

  • squashmigrations copies RunPython functions into the new file automatically
  • elidable=True skips the operation on a normal migrate
  • RunSQL can always be optimized through like a schema operation
  • Django cannot serialise any function into a migration
  • Marking seed-data RunPython elidable is harmless