skip to content

A Django project's Celery worker ignores BROKER_URL in settings.py; how does namespace='CELERY' in config_from_object explain it?

level: middleimportance: should knowfreq 35%

answer

  1. a prefix on every option
  2. uppercase in Django settings
  3. old names versus new names
  4. the setting Celery actually reads

basics

~10 s

With namespace='CELERY', Celery reads each option as an uppercase CELERY_-prefixed Django setting: broker_url becomes CELERY_BROKER_URL. A plain BROKER_URL or a lowercase broker_url in settings.py is never read, so the default broker is used.

solid answer

~30 s

`app.config_from_object("django.conf:settings", namespace="CELERY")` tells Celery to look up each lowercase option through the prefix: `broker_url` is read from `CELERY_BROKER_URL`, `task_always_eager` from `CELERY_TASK_ALWAYS_EAGER`, `worker_concurrency` from `CELERY_WORKER_CONCURRENCY`. Django's settings object only exposes uppercase names from `settings.py`, so a lowercase `broker_url` there never reaches Celery, and an unprefixed `BROKER_URL` is an old-style name Celery does not look up when a namespace is set. The worker falls back to its built-in default and the symptom looks like a broker problem. The fix is renaming to `CELERY_BROKER_URL`. The namespace is optional but recommended, because it keeps Celery options from colliding with Django's own settings.

code

python · 8 lines
python
# settings.py, read by config_from_object(..., namespace="CELERY")
CELERY_BROKER_URL = "redis://broker:6379/0"
CELERY_TASK_TIME_LIMIT = 300
CELERY_TIMEZONE = "UTC"

# Ignored with the namespace: old-style and lowercase names
BROKER_URL = "redis://broker:6379/0"
broker_url = "redis://broker:6379/0"

go deeper

for a junior

Remember that with the CELERY namespace every Celery option in settings.py is uppercase and starts with CELERY_.

for a middle

Explain the mapping from lowercase Celery options to prefixed Django settings, and why old-style or lowercase names are silently ignored.

for a senior

Diagnose a misconfigured worker by printing app.conf, checking the environment override, and adding a startup assertion for critical options.

for a principal

Set a convention that keeps all queue configuration in one namespaced settings block per environment, with checks that fail deployment on missing values.

## Two naming systems meet Celery's configuration options are **lowercase** names such as `broker_url`, `result_backend`, `task_time_limit` and `worker_concurrency`. Django's settings are **uppercase** module-level names in `settings.py`; Django's settings object only loads names that are all uppercase, so a lowercase variable in `settings.py` is invisible to anything reading `django.conf.settings`. `config_from_object()` bridges the two, and `namespace` controls how. ## What namespace="CELERY" does ```python app.config_from_object("django.conf:settings", namespace="CELERY") ``` When Celery looks up an option, it builds the prefixed, uppercased key and reads that from Django settings: | Celery option | Django setting read | |---|---| | `broker_url` | `CELERY_BROKER_URL` | | `result_backend` | `CELERY_RESULT_BACKEND` | | `task_always_eager` | `CELERY_TASK_ALWAYS_EAGER` | | `task_time_limit` | `CELERY_TASK_TIME_LIMIT` | | `worker_concurrency` | `CELERY_WORKER_CONCURRENCY` | | `timezone` | `CELERY_TIMEZONE` | The Celery-for-Django guide spells this out: with the uppercase namespace, all Celery options must be uppercase and start with `CELERY_`. ## Why BROKER_URL is ignored `BROKER_URL` is an **old-style** setting name from before Celery 4 introduced lowercase names. Celery can still translate old names in some setups, but when a namespace prefix is set it always uses the new naming scheme. So: 1. Celery wants `broker_url`. 2. It looks for `CELERY_BROKER_URL` in Django settings — not there. 3. It falls back to the unprefixed lowercase key and then to its own defaults — `settings.py` cannot provide a lowercase key, because Django never loads it. 4. The worker starts with Celery's built-in default broker, and the error looks like a connection problem rather than a configuration one. ## Diagnosing it - Print what Celery actually sees: `celery -A shop inspect conf` against a running worker, or in a shell `from shop.celery import app; app.conf.broker_url`. - Grep `settings.py` for Celery options without the `CELERY_` prefix. - Check for an environment variable: Celery also reads `CELERY_BROKER_URL` from the process environment, which overrides the setting — a surprise when a stale value is exported in the worker's environment. ## Choosing the namespace | Choice | Effect | |---|---| | `namespace="CELERY"` (recommended) | All options as `CELERY_*` uppercase settings; no clash with Django's own names | | No namespace | Celery looks options up by their own names; from Django settings only old-style uppercase names can match, which is why the guide recommends the namespace | | Separate module, `config_from_object("shop.celeryconfig")` | Celery options live outside `settings.py`; two places to keep in sync per environment | The namespace is optional, but it keeps Celery options visibly separate from Django's own settings and makes every one of them easy to find with one search. ## A corrected settings block ```python # settings.py CELERY_BROKER_URL = env("CELERY_BROKER_URL") CELERY_RESULT_BACKEND = env("CELERY_RESULT_BACKEND", default=None) CELERY_TASK_TIME_LIMIT = 5 * 60 CELERY_TIMEZONE = TIME_ZONE ``` Two habits help: keep every Celery option in one clearly labelled block, and add a startup check (or a test) that asserts `app.conf.broker_url` is what the environment intended.

  • Why can a lowercase broker_url in settings.py never configure Celery through django.conf:settings?
    Django's settings object only picks up module-level names that are entirely uppercase. A lowercase `broker_url` stays a plain module variable, so `django.conf.settings` never exposes it and Celery, which reads through that object, cannot see it, whatever namespace is used.
  • Where else can the broker URL come from besides settings.py?
    Celery also checks a `CELERY_BROKER_URL` environment variable and gives it precedence over the configured value. That is convenient for containers, but a stale exported value in a worker's environment silently overrides what `settings.py` says, so check the process environment when the two disagree.

saying these in an interview costs you the question

  • Writes BROKER_URL in settings.py while using namespace='CELERY'
  • Puts lowercase Celery option names in settings.py
  • Thinks the namespace only affects worker options, not broker settings
  • Assumes a wrong setting name raises an error at startup
  • Believes the namespace argument is required for Celery to read Django settings