skip to content

In Zabbix, how do templates and low-level discovery keep hundreds of hosts configured?

level: middleimportance: should knowfreq 55%

answer

  1. written once, linked to many hosts
  2. editing it changes every linked host
  3. a JSON list of found entities
  4. prototypes expand per discovered entity
  5. macro substituted into the item key

basics

~20 s

A Zabbix template is a reusable container of items, triggers and discovery rules linked to hosts; editing it changes every linked host at once. Low-level discovery expands prototypes into one item or trigger per entity found on each host.

solid answer

~50 s

A **template** holds items, triggers, graphs, dashboards, value maps, user macros and discovery rules, and is **linked** to hosts rather than copied into them — so adding a trigger to a template linked to 1,847 hosts creates it on all 1,847 immediately. Unlinking leaves those objects behind as host-level ones; unlinking *and clearing* deletes them together with their history. **Low-level discovery** covers what a template cannot: entities that differ per host. An LLD rule such as the agent key `vfs.fs.discovery` returns a JSON list of what it found, each entry a set of LLD macros like `{#FSNAME}` or `{#IFNAME}`. Item, trigger, graph and host **prototypes** written with those macros are expanded once per entry, so a host with eleven mounts gets eleven free-space items automatically. Filters keep ephemeral entities out, and a lost-resources delay stops one failed discovery run destroying history.

code

json · 4 lines
json
[
  { "{#FSNAME}": "/", "{#FSTYPE}": "ext4" },
  { "{#FSNAME}": "/var/lib/ferry-timetable", "{#FSTYPE}": "xfs" }
]

go deeper

for a junior

Know that a Zabbix template holds items and triggers, that it is linked to a host rather than copied into it, and that low-level discovery exists so per-filesystem and per-interface items do not have to be created by hand.

for a middle

Explain what a discovery rule returns, how an item prototype expands once per discovered entity with the LLD macro substituted into the key, and the difference between unlinking a template and unlinking and clearing it.

for a senior

Demonstrate control: filters that stop ephemeral mounts multiplying item count across the fleet, the lost-resources delay that protects history from one failed discovery run, and treating template exports as reviewed code.

for a principal

Own the template estate. Decide how templates are composed and versioned, where per-host variation is allowed to live through user macros, and what change process a template linked to thousands of hosts deserves.

## What a Zabbix template is A **template** is a reusable configuration object holding everything you would otherwise create on a host by hand: items, triggers, graphs, dashboards, web scenarios, value maps, user macros and **discovery rules**. On its own it collects nothing. It becomes live when it is **linked** to a host, at which point Zabbix creates each templated object on that host, marked as belonging to the template. Templates may be linked to other templates, so a broad operating-system template can be composed from narrower ones, and one host may be linked to several templates at once. **User macros** give the lever you need for per-host variation: the template defines a macro such as `{$CPU.UTIL.CRIT}` with a default, a trigger expression uses it, and one unusually noisy host overrides the value at host level without forking the template. ## What happens to linked hosts when the template changes This is the whole reason templates exist, and it is the part interviews probe. Editing a template is not editing a copy. Add a trigger to a template linked to 1,847 hosts and all 1,847 have it immediately, with no re-linking step. Change an interval, a macro default, a trigger expression — the same. That is enormous leverage and it cuts both ways: a careless edit to a widely linked template is a fleet-wide change, which is why templates are exported (Zabbix can export and import them as YAML, JSON or XML) and reviewed like code. Removing a link has two forms, and confusing them is a classic mistake: - **Unlink** leaves the objects on the host but detaches them from the template. They stop tracking template changes and become ordinary host-level items and triggers, keeping their collected history. - **Unlink and clear** deletes them from the host along with that history. ## Low-level discovery: prototypes instead of objects Templates solve "every host needs the same twelve items". They do not solve "every host needs one item per mounted filesystem, and no two hosts have the same filesystems". **Low-level discovery (LLD)** solves that. An LLD rule is a special item whose value is not a number but a JSON list of entities found on the host. The agent keys `vfs.fs.discovery` and `net.if.discovery` do exactly this for mounted filesystems and network interfaces; SNMP, database and script-based discovery rules do it for anything else. Each entry is a map of **LLD macros** to values — `{#FSNAME}` and `{#FSTYPE}` for a filesystem, `{#IFNAME}` for an interface, `{#SNMPINDEX}` for a row of an SNMP table. Hanging off the rule are **prototypes**: objects written with those macros standing in for the concrete name. | Prototype | What discovery creates | Example | |---|---|---| | Item prototype | one item per discovered entity | `vfs.fs.size[{#FSNAME},pfree]` | | Trigger prototype | one trigger per entity | free space on `{#FSNAME}` below a macro-defined limit | | Graph prototype | one graph per entity | throughput for `{#IFNAME}` | | Host prototype | an entire new monitored host | one host per discovered virtual machine | When the rule runs, Zabbix expands every prototype once per entry. A booking host with eleven mounted filesystems ends up with eleven free-space items and eleven triggers, created and named automatically. Nobody enumerated them and nobody has to revisit them when a disk is added. Two controls stop that being indiscriminate: 1. **Filters** — regular expressions matched against the LLD macros, so you discover interfaces but exclude the loopback and the virtual bridges, or discover filesystems but exclude temporary ones. This matters more than it looks: a container-heavy host can present hundreds of mounts, and each one multiplies through every prototype on every linked host. 2. **Overrides** — rules that change what a prototype produces for entries matching a condition, such as creating the item but suppressing the trigger for a filesystem that is expected to run full. ## The lifecycle of a discovered entity Discovered objects are not permanent. Each time the rule runs, Zabbix compares the returned list against what already exists: - entries that are new create objects from the prototypes; - entries still present leave existing objects alone and keep their history; - entries that have disappeared are marked as no longer discovered, and the rule's lost-resources setting decides how long the objects linger before deletion. That delay is deliberate. A filesystem missing from one run because the check timed out should not take its history with it. Set the retention too short and a transient blip destroys data; keep everything forever and a fleet that churns accumulates dead items that still cost storage and configuration-cache space. ## Where it goes wrong - Items created manually on a host alongside the template, which the next template edit does not touch and which nobody remembers exist. - Discovery with no filter on a host carrying many ephemeral mounts or interfaces, multiplying item count across the entire fleet. - Confusing `{#FSNAME}`, an LLD macro produced by discovery, with `{$THRESHOLD}`, a user macro you set — they look alike and behave nothing alike. - Editing a discovered item on the host rather than its prototype; the prototype is the only place a change survives and propagates.

  • What is the difference between unlinking a Zabbix template from a host and unlinking and clearing?
    Unlinking detaches the objects from the template but leaves them on the host as ordinary items and triggers, keeping their history and no longer tracking template changes. Unlink and clear deletes them and their collected history outright. The first is a safe way to fork one host's configuration; the second is destructive and irreversible.
  • A filesystem is unmounted permanently. What happens to the Zabbix items discovered for it?
    The next discovery run stops returning it, so Zabbix marks the item and its trigger as no longer discovered rather than deleting them at once. The discovery rule's lost-resources setting decides how long they survive before removal. That delay exists so a single failed or timed-out run does not destroy history for entities that are actually still present.
  • How do you vary one threshold on a single host without forking the Zabbix template?
    Define a user macro in the template, reference it from the trigger expression, and override its value on the individual host. The host-level definition wins over the template's, so one noisy host gets a looser limit while every other linked host keeps the shared default and all of them keep tracking future template edits.

A template is a class and a linked host is an instance of it, so editing the class changes every instance at once; low-level discovery is the loop that instantiates one set of members per thing it finds on the host.

saying these in an interview costs you the question

  • Thinks a template is copied onto the host so later template edits do not propagate
  • Creates one item per filesystem by hand instead of using a discovery prototype
  • Believes vanished entities have their items and history deleted immediately
  • Confuses an LLD macro produced by discovery with a user macro you set
  • Assumes unlinking a template always removes the items it created