skip to content

In Ruby, how do you use Signal.trap so a video-transcoding worker shuts down gracefully on TERM or INT, and what must the handler avoid?

level: seniorimportance: should knowfreq 35%

answer

  1. default: Interrupt or SignalException
  2. handler runs later, on the main thread
  3. set a flag, finish the job
  4. no Mutex in trap context
  5. KILL and STOP cannot be trapped

basics

~20 s

Without a trap, INT raises Interrupt and TERM raises SignalException in the main thread. Signal.trap("TERM") { @stopping = true } lets the worker finish the current job and exit; the handler must avoid Mutex, Monitor and buffered I/O.

solid answer

~40 s

By default Ruby turns INT into an `Interrupt` and TERM into a `SignalException` raised in the **main thread**; neither is a `StandardError`, so `rescue => e` does not catch them, but `ensure` blocks run. For a graceful stop, install `Signal.trap("TERM") { @stopping = true }` (and the same for `"INT"`), and have the worker loop check the flag between jobs so the current ffmpeg run finishes. `Signal.trap` returns the previous handler; `"DEFAULT"`, `"IGNORE"` and `"SYSTEM_DEFAULT"` are accepted as commands. Ruby runs the handler later, on the main thread, so it must be reentrant: `Mutex#lock`, `Mutex#synchronize` and anything built on `Monitor` raise `ThreadError` there. Setting variables and `Thread::Queue#push` are safe. Trapping `SEGV` or `VTALRM` raises `ArgumentError`, and `KILL` or `STOP` cannot be caught at all.

code

ruby · 14 lines
ruby
stopping = false
%w[TERM INT].each do |sig|
  Signal.trap(sig) { stopping = true }   # only set a flag
end

until stopping
  job = jobs.pop(timeout: 5) or next      # a Thread::Queue
  transcode(job)                          # runs ffmpeg to completion
end
puts "worker stopped cleanly"

Signal.trap("TERM") { logger.info("bye") }
# Logger locks a Monitor, so the line is dropped with a warning:
# log writing failed. can't be called from trap context

go deeper

for a junior

Recall that TERM and INT ask a process to stop, and that Signal.trap("TERM") { ... } replaces Ruby's default of raising an exception.

for a middle

Explain the defaults, Interrupt and SignalException outside StandardError, the previous handler Signal.trap returns, and the IGNORE and DEFAULT commands.

for a senior

Design shutdown: flag or queue in the handler, work finishing between jobs, no Mutex or Monitor in trap context, forwarding to children, and retry-safe jobs for KILL.

for a principal

Set the shutdown contract with the platform: grace periods, which signals mean what, and how every worker type proves it drains within the window.

## What happens without a trap A **signal** is an asynchronous notification from the operating system. Orchestrators send **TERM** to ask a process to stop; pressing Ctrl-C in a terminal sends **INT**. Ruby installs its own defaults: - **INT** raises `Interrupt` in the main thread. - **TERM**, **HUP**, **QUIT**, **ALRM**, **USR1** and **USR2** raise a `SignalException` in the main thread. Both exceptions sit under `Exception`, not `StandardError`, so a bare `rescue => e` does not swallow them, while `ensure` blocks still run as the stack unwinds. That default already cleans up, but it interrupts whatever the main thread is doing, possibly halfway through writing an output file. ## A graceful stop For a worker that transcodes uploads one job at a time, the aim is: stop taking new jobs, let the current one finish, then exit. 1. Install handlers early: `Signal.trap("TERM") { @stopping = true }` and the same for `"INT"`. 2. Check the flag at a safe point, between jobs. 3. Exit normally after the current job, so the job queue sees it completed. `Signal.trap(signal, command)` or `Signal.trap(signal) { ... }` takes a name with or without the `SIG` prefix, or a number, and **returns the previous handler**, which lets you restore or chain it. Besides a block, the command can be `"IGNORE"`, `"DEFAULT"` (Ruby's default), `"SYSTEM_DEFAULT"` (the operating system's) or `"EXIT"`. ## When and where the handler runs Ruby does not run your block inside the low-level signal handler. A tiny C handler records the signal, and the VM runs the Ruby block later, at a safe point, **on the main thread**. So: - a long blocking call in the main thread delays the handler until the call yields to the VM; - the handler interrupts whatever Ruby code the main thread was running, and must not corrupt that code's data. ## What the handler must avoid Ruby's signal documentation lists the rules. **Unsafe** inside `Signal.trap`: | Operation | Why | |---|---| | `Mutex#lock`, `Mutex#synchronize` | raise `ThreadError`: can't be called from trap context | | anything built on `Monitor` | uses a Mutex underneath, so it fails the same way | | writes to an IO with `sync` false | buffered writes are not reentrant | | `Dir.chdir` with a block | changes process-wide state mid-flight | **Safe**: assigning local, instance and class variables; creating common objects; `Thread::Queue#push`; starting a new `Thread` to do heavier work; writing to pipes and sockets, which default to `sync = true`. The pattern is always the same: record the request in the handler, act on it in normal code. ## Signals you cannot handle - `SEGV`, `BUS`, `ILL`, `FPE` and `VTALRM` are reserved by Ruby: trapping them raises `ArgumentError` (`can't trap reserved signal`). - `KILL` and `STOP` cannot be caught by any process; `Signal.trap("KILL")` raises `Errno::EINVAL`. An orchestrator that sends KILL after a grace period ends the worker whatever you do, so a job must be safe to retry. ## Chaining and testing handlers Because `Signal.trap` returns the previous handler, code that installs its own can keep the old one and call it afterwards. The returned value may be a `Proc`, or a String such as `"DEFAULT"` or `"IGNORE"`, so check that it responds to `call` before calling it. In a test, `Process.kill("TERM", Process.pid)` sends the signal to the current process, which lets you assert that the flag is set and the loop exits after the current job. ## Children such as ffmpeg A TERM sent to the worker's pid reaches only the worker. If you want the running ffmpeg to stop early, forward the signal yourself with `Process.kill("TERM", pid)` using the pid from `spawn` or `wait_thr.pid`. Ctrl-C in a terminal, by contrast, sends INT to the whole foreground process group, so the child receives it too. ## Checklist - Trap TERM and INT; set a flag or push onto a queue. - Check the flag between units of work. - No locks, loggers built on Monitor, or buffered writes inside the handler. - Design jobs to survive a KILL after the grace period.

  • In Ruby, why does rescue => e in the worker loop not stop a TERM from ending the process?
    Without a trap, TERM raises `SignalException` in the main thread, and INT raises its subclass `Interrupt`. Both descend from `Exception`, not `StandardError`, and a bare `rescue` catches only `StandardError`. The exception propagates, `ensure` blocks run, and the process ends. That is intended: use `Signal.trap` for graceful handling rather than rescuing `Exception`.
  • In Ruby, how can a trap handler hand real work to the rest of the program safely?
    Push a message onto a `Thread::Queue`, which Ruby's signal documentation lists as safe in trap context, or set a flag. A thread blocked on `queue.pop` then wakes up and does the work, such as logging or closing connections, outside the handler, where Mutex and Monitor are allowed.

  • rescue => e in the main loop catches a TERM signal.
  • Calling logger.info inside Signal.trap is always safe.
  • Signal.trap("KILL") lets a Ruby process clean up before being killed.
  • Ruby runs the trap block immediately inside the OS signal handler.
  • Ruby automatically forwards a TERM sent to its pid to the ffmpeg children.