skip to content

Why does an IPFIX collector that just restarted fail to decode UDP exports for minutes, and what in the protocol bounds that gap?

level: seniorimportance: should knowfreq 12%

answer

  1. data records are not self-describing
  2. Set ID points at a Template ID
  3. UDP has no session to reset
  4. periodic template retransmission
  5. SCTP or TCP restarts the session

basics

~20 s

IPFIX and NetFlow v9 data records can only be parsed with the template they reference, and a restarted collector has lost its templates. Over UDP the exporter cannot tell, so decoding resumes at its next periodic template retransmission.

solid answer

~50 s

In IPFIX (RFC 7011) a Data Set's Set ID is the Template ID, 256 or above, of a Template Record sent earlier in a Set with ID 2, or ID 3 for an options template. Without that template, the data is opaque bytes. Over UDP there is no session for the restart to break, and RFC 7011 §10.3.5 says the exporter cannot detect that the collector has gone. RFC 7011 §8.4 makes the exporter retransmit every active template periodically at a configurable interval, so the gap lasts up to one refresh interval. The collector MAY buffer undecodable records briefly and decode them once the template arrives. Over SCTP or TCP, the restart ends the transport session, and the new session carries templates before data. RFC 3954 gives NetFlow v9 the same rule, refreshing every N packets or N minutes.

go deeper

for a junior

Remember that NetFlow v9 and IPFIX records need a template to be read, while NetFlow v5's fixed layout does not.

for a middle

Explain Set IDs 2 and 3 versus data Sets numbered 256 and up, and why UDP export repeats templates on a timer.

for a senior

Diagnose a post-restart decoding gap, size the template refresh interval against it, and weigh SCTP or TCP sessions, collector buffering and duplicate export.

for a principal

Treat collector restarts as routine: choose transport, refresh policy and collector redundancy so maintenance never opens a blind window at the edge.

## Why a data record needs a template **NetFlow v5**, a fixed-format vendor export with no RFC, has one record layout: a collector can decode the first packet it ever sees, but the layout is IPv4-only and cannot grow. **NetFlow v9** (RFC 3954, Informational) and **IPFIX** (RFC 7011, Standards Track) replaced the fixed layout with **templates**: the exporter first describes a record's fields, then sends records that are bare values in that order. In IPFIX the Set header's **Set ID** says what a Set contains: | Set ID | Contents | |---|---| | 2 | Template Set: Template Records | | 3 | Options Template Set: Options Template Records | | 4-255 | reserved | | 256 and above | Data Set, using the Template ID equal to the Set ID | NetFlow v9 uses FlowSet ID 0 for templates and 1 for options templates; Set IDs 0 and 1 are not used in IPFIX "for historical reasons". **Options templates** matter here too: they carry metadata such as the sampling rate, which RFC 3954 exports as `SAMPLING_INTERVAL` (field type 34). A collector that decodes data records but lacks that options data cannot scale sampled counts. ## What the restart destroys The collector keeps template state keyed per exporter. For UDP, RFC 7011 §8.4 lists the key: exporter, exporter source port, collector address and port, **Observation Domain ID**, and **Template ID**. A restart empties that table. From then on: 1. Data Sets keep arriving, each naming a Template ID the collector no longer knows. 2. The exporter cannot notice. RFC 7011 §10.3.5: over UDP it "is unable to determine from the transport protocol that the Collecting Process is no longer able to receive", so nothing triggers an early resend. 3. Decoding resumes when the next periodic template retransmission arrives. ## What the protocol says bounds the gap - **Periodic retransmission (UDP).** RFC 7011 §8.4: exporters using UDP MUST periodically retransmit each active template, at an interval that MUST be configurable. It names `templateRefreshTimeout` and `optionsTemplateRefreshTimeout` as example parameters. The RFC says defaults are "deployment- and application-specific". The worst-case gap is one refresh interval. - **NetFlow v9's two triggers.** RFC 3954 §7 requires refresh both every N export packets and every N minutes, both configurable. At high export rates the packet-count trigger can close the gap sooner. - **Collector buffering.** RFC 7011 §9.3: the collector SHOULD accept data records without a template, and MAY store them "for a short period of time" and decode them once the template arrives. The RFC warns this can misinterpret records if a Template ID has been reused. - **Template lifetime.** A collector MAY expire templates that stop being refreshed. If it derives the lifetime from observed retransmissions, it SHOULD default to at least **3 times** that interval. This rule governs expiry and has nothing to do with how fast a restart recovers. ## Why SCTP or TCP behave differently IPFIX says SCTP with the PR-SCTP extension MUST be implemented, while UDP and TCP MAY be. Over SCTP or TCP, a collector restart ends the **transport session**. The exporter reconnects, and in the new session it sends templates before the data that uses them. RFC 7011 also forbids a collector from using templates from one transport session to decode another. The gap shrinks to the reconnect time. UDP, by contrast, is allowed only where the export path is provisioned against congestion, and RFC 7011 §10.3.2 says it MUST NOT be used unless the application tolerates losing some messages. ## Spotting the loss - IPFIX's header **Sequence Number** counts Data Records sent, modulo 2^32, so a jump shows how many records went missing. NetFlow v9's sequence number counts **export packets** instead. - Template Records do not advance the IPFIX sequence number. - Template Withdrawals MUST NOT be sent over UDP and MUST be ignored there. Template ID reuse over UDP instead waits at least 3 times the retransmission delay. ## Operating choices - Set the template refresh interval as the longest decoding gap you can tolerate after a collector restart. Shorter intervals cost a little export bandwidth. - Prefer SCTP or TCP where the collector restarts often, or send the same export to two collectors. RFC 7011 §10.3.5 allows an exporter to duplicate messages to several collectors. - Keep messages within the path MTU. RFC 7011 §10.3.3 says to use 512 octets when the PMTU is unknown, since a lost fragment loses the whole message, templates included.

  • An IPFIX collector restarted and now decodes data records, but sampled byte counts look 1000 times too small. What is missing?
    The sampling metadata travels in options data records, described by an Options Template in a Set with ID 3. Until that options template and its data arrive, the collector decodes the flow records but does not know the sampling interval, so it cannot multiply the counts up. The fix is the same periodic refresh, applied to options templates.
  • Why does IPFIX forbid Template Withdrawals over UDP, and how is a Template ID reused instead?
    A withdrawal sent over UDP can be lost or reordered, so the collector might keep decoding with a dead template or drop a live one. RFC 7011 §8.4 says exporters MUST NOT send withdrawals over UDP and collectors MUST ignore them. To reuse an ID, the exporter waits at least 3 times the retransmission delay and then sends the new template.

saying these in an interview costs you the question

  • An IPFIX collector can ask the exporter to resend its templates.
  • Each IPFIX data record carries its field names, so no template is needed.
  • Over UDP, the exporter sends a Template Withdrawal before reusing an ID.
  • Switching IPFIX to TCP changes nothing, since templates are sent once at startup.
  • A collector restart is recovered after three times the template refresh interval.