skip to content

In a coffee-chain Rails 8.1 app, should a one-off backfill that recomputes every customer's loyalty points live in a migration or a Rake task, and why?

level: seniorimportance: should knowfreq 45%

answer

  1. migrations change schema, tasks change data
  2. fresh databases load schema, skip old migrations
  3. deploy-time db:prepare runs migrations
  4. batch with find_each, make it idempotent
  5. dry run, logging, then delete the task

basics

~20 s

Put it in a Rake task. Migrations should describe schema and run at deploy time, inside a transaction on PostgreSQL, and since Rails 8.0 a fresh database loads schema.rb instead of replaying them. A task can be batched, idempotent, rerun, dry-run first and deleted once done.

solid answer

~50 s

Recomputing points is a data change, not a schema change, and a migration is the wrong home for it. Migrations run inside the deploy: the generated Docker entrypoint runs `bin/rails db:prepare` before the server starts, so a slow backfill delays boot. On PostgreSQL each migration runs in one transaction by default, holding locks until the whole backfill ends. Migrations also reference app models that will later change, and since Rails 8.0 `db:migrate` on a fresh database loads the schema first, so the data step never runs on a new setup anyway. A `lib/tasks/loyalty.rake` task can be run deliberately in each environment, after the code that uses the new numbers is deployed. Make it **idempotent** (recompute from purchases, never add a delta), **batched** (`find_each`, 1000 rows a batch by default) and observable (log counts, support `DRY_RUN=1`), then delete it once it has run everywhere.

code

ruby · 20 lines
ruby
# lib/tasks/loyalty.rake
namespace :loyalty do
  desc "Recompute loyalty points from purchases (DRY_RUN=1, START_ID=n)"
  task recompute: :environment do
    dry_run = ENV["DRY_RUN"] == "1"
    start   = Integer(ENV.fetch("START_ID", "1"))
    changed = 0

    Customer.find_each(start:) do |customer|
      points = Loyalty::Recalculator.points_for(customer)
      next if customer.loyalty_points == points

      changed += 1
      customer.update_columns(loyalty_points: points) unless dry_run
      Rails.logger.info("loyalty:recompute id=#{customer.id} points=#{points}")
    end

    puts "#{dry_run ? 'Would change' : 'Changed'} #{changed} customers"
  end
end

go deeper

for a junior

Know that schema changes go in migrations and that one-off data fixes are usually Rake tasks run by hand.

for a middle

Explain the practical reasons: deploy-time execution, transactions on PostgreSQL, fresh databases loading the schema, and batching with find_each.

for a senior

Design the task to be idempotent, resumable, observable and dry-runnable, sequence it with the deploy, and delete it once it has run everywhere.

for a principal

Set the team's policy for data changes: who runs them in production, how they are reviewed and audited, and when they become jobs instead of tasks.

## The situation A coffee chain's app stores `customers.loyalty_points`. A bug double-counted refunds for months, and every customer's balance must be **recomputed** from their `purchases` and `refunds`. There are a few million customers. The question interviewers ask: migration or Rake task? ## Why not a migration A migration feels natural because it runs once per environment and is tracked in `schema_migrations`. The trouble is everything else a migration implies: | Property of migrations | Effect on a data backfill | |---|---| | They run **at deploy time** | The generated `bin/docker-entrypoint` runs `bin/rails db:prepare` before `rails server`; a 40-minute backfill holds the release and its health checks | | On PostgreSQL and SQLite each runs **in one transaction** | Every updated row stays locked until the whole backfill commits; one failure rolls all of it back | | They live **forever** in `db/migrate` | A migration that calls `Customer` or `Loyalty::Recalculator` breaks when those classes change or disappear | | Since **Rails 8.0** a fresh database **loads the schema first** | On a new developer machine or CI database, the backfill never runs, so its effect is not reproducible from migrations anyway | | They are **ordered with schema changes** | A slow data step blocks every schema migration queued behind it | None of that is wrong for adding a column; all of it is wrong for rewriting millions of rows. ## Why a Rake task fits A task in `lib/tasks/loyalty.rake` separates **when** the work runs from **when code ships**: - the deploy adds the code (and, through a normal migration, any new column); - an operator runs `RAILS_ENV=production bin/rails loyalty:recompute` when ready, can watch it, and can stop and rerun it; - staging and production each get it once, by hand or from a one-off job; - once it has run everywhere, the file is deleted. ## Making the task safe 1. **Idempotent.** Recompute the balance from source data (`purchases` minus `refunds`) and write the result; never `loyalty_points += delta`. Running it twice must give the same numbers. 2. **Batched.** Iterate with `Customer.find_each` (batches of 1000 by default) or `in_batches`, so memory stays flat and each update is its own short transaction. 3. **Resumable.** Accept a starting id (`START_ID=...`, passed to `find_each(start:)`) or record progress, so a failure halfway does not mean starting over. 4. **Dry run.** A `DRY_RUN=1` mode logs what would change without writing. 5. **Observable.** Log counts and the last processed id to `Rails.logger` and stdout. 6. **Thin.** The task parses input and reports; the calculation lives in `Loyalty::Recalculator`, which has unit tests. For work that must survive a deploy restart, the same class can run inside a background job instead; 8.1's job continuations exist for exactly that kind of resumable step, but that is a jobs design question. ## Where the line falls - **Migration**: adding `loyalty_points`, changing its type, adding an index or constraint, and tiny data tweaks that the schema change depends on, written without app models. - **Rake task**: recomputing, cleaning or reconciling rows; anything slow, anything that needs app logic, anything you may need to rerun. - **Seeds**: reference data every environment needs from scratch. The interview signal is not "never touch data in a migration" but knowing the deploy, locking and reproducibility costs, and choosing the tool whose life cycle matches the job. ## Sequencing with the deploy A typical order for the loyalty fix: 1. Deploy the code that computes points correctly, so new purchases are right from now on. 2. Run `DRY_RUN=1` in production and compare the reported count with expectations. 3. Run the real backfill, watching the log. 4. Spot-check a handful of customers in the console. 5. Remove the task in the next release. If the fix needs a new column, that column arrives through an ordinary migration in step 1, and the task fills it in step 3.

  • Why does a data backfill written as a migration fail to reproduce on a new developer's database in Rails 8.1?
    Since Rails 8.0, `db:migrate` on a fresh database loads `schema.rb` (or `structure.sql`) first and marks the existing migrations as run, then applies only pending ones. The backfill migration is never executed there, so its data effect exists only in databases that ran it historically. Data that every environment needs belongs in seeds; one-off corrections belong in a task.
  • Why does the task above use update_columns rather than update!?
    `update_columns` writes the one column directly, skipping validations, callbacks and `updated_at`, which is what a corrective backfill usually wants: no emails, no broadcasts, no unrelated validation failures halting the run. The trade-off is that nothing else reacts to the change, so use `update!` when callbacks must run.
  • How do you make the backfill restartable after it crashes at customer 1,800,000?
    Log the last processed id and accept a starting point, then pass it to `find_each(start:)`, which walks the primary key upward from there. Because the recomputation is idempotent, rerunning a few rows before the crash point is harmless.

  • Data migrations are the Rails-standard place for every backfill
  • A migration's data change runs on fresh databases in Rails 8.1
  • Adding loyalty_points += delta is fine if the task runs once
  • Customer.all.each is fine for a few million rows
  • On PostgreSQL a long backfill migration commits row by row
  • A backfill task should stay in lib/tasks forever as documentation

Assumes Rails 8.1; since Rails 8.0, db:migrate on a fresh database loads the schema file before running pending migrations.