skip to content

When syslog over UDP 514 bursts from a firewall pair to one collector during an incident, what is lost and how can you tell?

level: seniorimportance: must knowfreq 30%

answer

  1. simplex, no acknowledgement
  2. one message per datagram
  3. one lost fragment, whole message
  4. number the messages yourself

basics

~20 s

Whole messages vanish and nothing tells the collector: RFC 5426 gives UDP syslog no acknowledgement, retransmission or congestion control. You can only see the loss if messages carry a sequence number, such as RFC 5424's meta sequenceId, or syslog-sign signature blocks.

solid answer

~40 s

RFC 5426 puts exactly one syslog message in each UDP datagram and provides no mechanism to detect or correct loss, so any message dropped on a congested link or by an overrun receiver is simply gone, and neither end is told. Loss of one IP fragment discards the whole message, which is why senders should stay within the path MTU, or 480 octets on IPv4 and 1180 on IPv6 when it is unknown. UDP has no congestion control, so the burst competes blindly, low-severity floods can crowd out high-severity messages, and arrival order is not event order. To make loss visible, have originators send `[meta sequenceId="..."]` and look for gaps, or use RFC 5848 signed syslog. RFC 5424 recommends the TLS transport instead, and allows UDP only on networks provisioned for it.

go deeper

for a junior

Say clearly that syslog over UDP can lose messages without anyone being told, because nothing acknowledges them.

for a middle

Name the RFC 5426 rules: one message per datagram, no loss detection, fragmentation risk with the 480 and 1180 octet guidance, and arrival order not being event order.

for a senior

Diagnose a burst: where datagrams drop, how sequenceId gaps reveal it, why a reset to 1 is not loss, and when to rate-limit, provision the path or move to TLS.

for a principal

Weigh lossy UDP against back-pressure from a reliable transport for each device class, and decide which event streams justify signed or numbered messages as evidence.

## What UDP 514 promises, and what it does not RFC 5424 describes syslog as a pure **simplex** protocol: the originator sends and nothing comes back. RFC 5426 maps it onto UDP port 514, and each property of that mapping matters during a burst: | Property | RFC 5426 says | Consequence in a burst | |---|---|---| | Framing | exactly one message per datagram, complete or truncated | each loss removes whole messages | | Loss | no mechanism to detect or correct lost datagrams | nobody is told | | Fragments | losing one fragment discards the entire message | large messages are lost more often | | Congestion | UDP has no congestion control | the burst does not slow down | | Order | arrival order should not be used as the authoritative event order | the timeline must be rebuilt | | Priority | severity-based prioritisation is not mandated | Debug floods compete equally with Emergency | | Checksums | senders must not disable them; they detect some corruption | they do not make delivery reliable | ## Where a burst drops messages Picture two firewalls, `fw-a` and `fw-b`, both logging every denied connection to one collector at `198.51.100.20` while a scan hits them: 1. **On the originator**, if its sending queue fills. RFC 5424 section 8.6 recommends storing messages rather than dropping them and, if dropping is unavoidable, dropping lower severity first. 2. **On any congested link** between firewall and collector; UDP gives the sender no signal to back off. 3. **In fragment reassembly**, if a message exceeded the path MTU and one fragment did not arrive. 4. **At the collector**, if datagrams arrive faster than it reads them and its receive queue overflows. None of these losses is reported to the collector: the originator may know about its own queue drops, but the UDP transport carries no notice of them. RFC 5424 section 8.5 adds that an attacker on the path can intercept and discard messages to hide activity, and RFC 5426 section 5.1 notes that UDP syslog cannot authenticate the sender, so forged messages can be mixed in. ## Making loss visible The protocol cannot report loss, but the message can carry what is needed to detect it: - **`meta sequenceId`** (RFC 5424 section 7.3.1). The originator sets it to 1 when its syslog function starts and increases it with every message up to 2147483647, then wraps to 1. If one originator's messages arrive with 48120, 48121, then 48127, five messages (48122 to 48126) are missing. - **A reset is not loss.** A jump back to 1 means a restart or a wrap. RFC 5424 notes that a change in **PROCID** signals a discontinuity in reporting, though a restarted process can get the same identifier. - **Signed syslog** (RFC 5848, "syslog-sign") sends Signature Blocks in the `ssign` structured-data element that carry hashes of earlier messages, adding origin authentication, integrity, replay resistance and detection of missing messages. - **Originator notification.** RFC 5424 section 8.5 points out that a reliable transport lets an originator that must discard messages knowingly tell the collector; over UDP the message is simply lost without any indication. ## Keeping datagrams small RFC 5426 requires IPv4 receivers to accept messages up to 480 octets and IPv6 receivers up to 1180, and says all receivers should accept 2048. Those minimums come from the smallest MTUs hosts must support, 576 octets for IPv4 and 1280 for IPv6. Because syslog has no acknowledgement it cannot use packetization-layer path MTU discovery, so when the MTU is unknown the safest choice is 480 octets for IPv4 and 1180 for IPv6. RFC 5424 also advises putting the important information early, where truncation is least likely to cut it. ## Choosing the transport RFC 5424 makes the TLS transport of RFC 5425 mandatory to implement and recommended, and both RFC 5424 and RFC 5426 say UDP may be used instead only on managed networks where the path has been explicitly provisioned for syslog, by rate limiting or capacity reservation. In practice that leaves an operator three levers: - **Reduce the burst**: rate-limit at the originator, keeping the higher severities. - **Provision the path** if UDP must stay. - **Move to TLS on TCP 6514**, which retransmits and backs off, while knowing it still has no application-level acknowledgement. How long a log shipper should buffer while its destination is unreachable is a pipeline design decision. The protocol's part ends at whether the message reached the wire and whether its loss can be seen.

  • One firewall's syslog meta sequenceId jumps from 48121 back to 1. Is that message loss?
    Not by itself. RFC 5424 requires sequenceId to be set to 1 when the syslog function starts and to wrap to 1 after 2147483647, so a jump to 1 indicates a restart or wrap. Check for a PROCID change, which signals a discontinuity in reporting; messages sent just before the restart may still have been lost, but the gap cannot be counted.
  • Why keep UDP syslog messages within 480 octets on IPv4 when the path MTU is unknown?
    IPv4 hosts must support a 576-octet MTU, so a 480-octet message plus headers avoids fragmentation. Losing one fragment discards the whole message, and syslog cannot run packetization-layer path MTU discovery because nothing is acknowledged. RFC 5426 gives 1180 octets as the equivalent safe size on IPv6.
  • Can an attacker exploit UDP syslog's lack of delivery guarantees?
    Yes. RFC 5424 and RFC 5426 note that messages can be intercepted and discarded to hide activity, and that UDP syslog offers no sender authentication, so forged messages can be injected. Defences are the TLS transport with mutual authentication, RFC 5848 signing, IPsec on the path, and accepting messages only from known sources.

saying these in an interview costs you the question

  • The collector logs an error for every UDP syslog message it missed.
  • UDP checksums make syslog delivery reliable.
  • UDP syslog messages arrive in the order the firewall sent them.
  • Packing several syslog messages into one UDP datagram reduces loss.
  • Bigger UDP syslog messages are safe because IP fragmentation is reliable.
  • A restart that resets sequenceId to 1 proves messages were lost.