In Ansible, a variable resolves to an unexpected value at runtime. How does Ansible's variable precedence decide which definition wins, and how do you trace where the value actually came from?

## Why there is a ladder at all Ansible deliberately lets the same variable name be defined in many places: a role ships a sane default, an inventory group narrows it for staging, a single host overrides it again, and an operator forces a one-off value on the command line. For that to be usable there has to be a single, documented, total order — and there is. Ansible resolves a name by consulting sources from weakest to strongest and taking the strongest definition. It is not "last file loaded wins" and it is not alphabetical. ## The order, grouped so you can actually remember it The documented ladder for ansible-core 2.x, lowest first, is: 1. command line values that are not variables (`-u`, `-b`) 2. role defaults (`defaults/main.yml`) 3.–7. inventory group vars, then `group_vars/all`, then `group_vars/<group>` (inventory-adjacent before playbook-adjacent) 8.–10. inventory host vars, then `host_vars/<host>` 11. host facts and cached `set_fact` values 12.–14. play `vars`, `vars_prompt`, `vars_files` 15. role `vars/main.yml` 16.–17. block vars, then task vars 18. `include_vars` 19. `set_fact` and registered variables from this run 20.–21. role / `include_role` params, then include params 22. extra vars (`-e` / `--extra-vars`) — always win Four groupings carry almost all the interview value: - **Role `defaults/` is the floor, role `vars/` is high.** Same directory tree, opposite intent: `defaults/main.yml` is "override me", `vars/main.yml` is "internal, do not override". Putting a tunable in `vars/` and then wondering why `group_vars` is ignored is the single most common real-world confusion. - **Host beats group.** `host_vars/web1.yml` overrides `group_vars/web.yml`. Within groups, a child group beats its parent, and `ansible_group_priority` breaks ties between groups at the same depth. - **Facts sit *below* play vars.** A play-level `vars:` entry named `ansible_distribution` shadows the gathered fact. This is why reading facts through the `ansible_facts` dictionary (`ansible_facts['distribution']`) is safer than the injected top-level `ansible_distribution` name, which the `inject_facts_as_vars` setting can disable entirely. - **`-e` always wins.** There is no way to protect a variable from extra vars. That is a feature for break-glass operations and a hazard for anything you wanted to be constant. ## Merging, not just winning By default Ansible *replaces* a whole dictionary rather than merging it: if `group_vars/all.yml` sets `app: {port: 80, user: app}` and `host_vars/web1.yml` sets `app: {port: 8080}`, web1 loses `user`. The global `hash_behaviour = merge` setting in `ansible.cfg` changes this, but it is discouraged because it changes semantics for every variable in the project and makes roles behave differently between repositories. The maintainable answer is to keep the dictionary flat, or to merge explicitly with the `combine` filter: ```yaml app_config: "{{ app_defaults | combine(app_overrides, recursive=True) }}" ``` ## How you actually debug it Reciting 22 levels is not the answer an interviewer wants. Demonstrate a procedure: ```yaml - name: What is the value here, right now ansible.builtin.debug: var: app_port - name: Everything this host can see ansible.builtin.debug: var: hostvars[inventory_hostname] verbosity: 2 ``` Outside a playbook, the ad-hoc form is fastest: `ansible web1 -m ansible.builtin.debug -a "var=app_port"`. To separate *inventory* contributions from playbook ones, `ansible-inventory --list` (or `--host web1`, or `--graph --vars`) prints exactly what the inventory sources resolve to, with no play involved — if the surprising value shows up there, the culprit is a `group_vars`/`host_vars` file; if it does not, the culprit is inside the play, a role, or `-e`. After that: run with `-vvv` to see the arguments the module was actually invoked with (the rendered value, not the expression), and grep the repository for the name across `group_vars/`, `host_vars/` and every role's `defaults/` and `vars/`. Check the wrapper script or CI job for a stray `-e` — a value that ignores every file you edit is almost always extra vars. ## The trap behind the trap Jinja expressions are evaluated lazily, at the moment the variable is used, not where it is written. A variable defined as `"{{ base_url }}/health"` picks up whichever `base_url` wins *at task time*, so the same definition can render differently in two plays of the same run. When you are chasing a value, note *where* it was consumed, not only where it was defined.