skip to content

Background Jobs & Scheduling

Getting work off the request path: task queues, background workers and scheduled jobs in .NET and Python. Asked because retries and at-least-once delivery are where it silently fails.

on this pageshow

explore

questions

61 · 2 sections

With Hangfire, what is the difference between BackgroundJob.Enqueue, BackgroundJob.Schedule and RecurringJob.AddOrUpdate, and when do you use each?

level: juniorimportance: must knowfreq 72%
basics
~20 s

BackgroundJob.Enqueue creates a fire-and-forget job that runs as soon as a worker is free; BackgroundJob.Schedule creates a job that runs after a delay or at a set time; RecurringJob.AddOrUpdate registers a cron schedule that creates a new job at every occurrence.

open as a page

In an ASP.NET Core 10 app, how do you wire Hangfire with AddHangfire, a PostgreSQL job storage and AddHangfireServer, and what does each call do?

level: juniorimportance: must knowfreq 62%
basics
~20 s

AddHangfire configures Hangfire globally, sets the job storage (UsePostgreSqlStorage for PostgreSQL) and registers the job clients. AddHangfireServer adds a hosted service that runs a processing server: workers that execute jobs plus the schedulers, all reading the same storage.

open as a page

With Hangfire, why should a job method take an order id rather than the Order entity, and what happens to its arguments at enqueue time?

level: middleimportance: must knowfreq 58%
basics
~20 s

Hangfire evaluates a job's arguments and serializes them to JSON in storage when the job is created, so an entity becomes a stale, bulky snapshot. Passing the id lets the job load current data when it actually runs.

open as a page

With Hangfire, how does [AutomaticRetry] behave by default, and how do you stop a flaky payout job from retrying for days on errors that will never succeed?

level: middleimportance: must knowfreq 50%
basics
~20 s

AutomaticRetryAttribute is global: a throwing job is retried 10 times with growing delays, about 4.5 hours in all, then left Failed. A method-level [AutomaticRetry] with fewer Attempts or ExceptOn for permanent errors stops pointless retries.

open as a page

With Hangfire on ASP.NET Core, why does the dashboard work on a developer machine but return 401 once deployed, and how do you secure it properly?

level: middleimportance: must knowfreq 52%
basics
~10 s

DashboardOptions.Authorization defaults to LocalRequestsOnlyAuthorizationFilter, which admits only loopback or same-IP requests, so a deployed dashboard answers 401. Replace it with your own IDashboardAuthorizationFilter, or map it with MapHangfireDashboardWithAuthorizationPolicy and an ASP.NET Core policy.

open as a page

In Celery, what does the celery beat process do, and how do you declare a periodic task in the beat_schedule setting?

level: juniorimportance: must knowfreq 55%
basics
~20 s

celery beat is a scheduler process that sends a task message whenever a beat_schedule entry comes due; workers run it. Each entry names a registered task, a schedule (seconds, timedelta, crontab or solar) and optional args, kwargs and options.

open as a page

In Celery, what is the difference between the broker and the result backend, and does every app need a result backend?

level: juniorimportance: must knowfreq 62%
basics
~20 s

Celery's broker carries task messages from the caller to the workers; the result backend stores each task's state and return value. The broker is required; the result backend is optional and unset by default, so fire-and-forget tasks need none.

open as a page

In Celery, how does `self.retry()` re-run a failing task, and what do its `countdown`, `max_retries` and `exc` arguments control?

level: juniorimportance: must knowfreq 62%
basics
~20 s

A bound Celery task calls raise self.retry(exc=exc, countdown=N) in its except block: Celery publishes a new message for the same task id, delayed N seconds (180 by default), up to max_retries (3), then re-raises exc as the failure.

open as a page

In Celery, how does calling a task with delay() differ from apply_async(), and what does either call return to the caller?

level: juniorimportance: must knowfreq 65%
basics
~20 s

delay(*args, **kwargs) is a shortcut for apply_async(args, kwargs) with no execution options. Both serialize the arguments, publish one message to the broker and return an AsyncResult with the task id at once; neither waits for a worker.

open as a page

In Celery, how do chain, group and chord differ, and which fits resizing thumbnails in parallel and then running one step after all finish?

level: middleimportance: must knowfreq 45%
basics
~20 s

In Celery, a chain runs tasks in sequence and passes each result on; a group runs tasks in parallel; a chord is a group plus one body task that receives the list of all header results. Parallel thumbnails, then publish, is a chord.

open as a page