skip to content

You are deciding whether to enable YJIT for a Ruby 4.0 JSON API served by forked workers; how do you roll it out and keep its memory within budget?

level: seniorimportance: should knowfreq 35%

answer

  1. profile before deciding
  2. enable after boot, per worker
  3. --yjit-mem-size, default 128 MiB
  4. code_region_size plus yjit_alloc_size
  5. ratio_in_yjit needs a stats build

basics

~20 s

Confirm from a profile that Ruby-level CPU dominates, then enable YJIT after boot on a canary, cap it with --yjit-mem-size or RubyVM::YJIT.enable(mem_size:), and compare latency, CPU and per-worker memory using runtime_stats before rolling it wider.

solid answer

~40 s

First check it can pay off: a JSON API often spends its time in queries and in the JSON C extension, and YJIT only shortens Ruby-level execution. If Ruby code dominates, enable it lazily - `RubyVM::YJIT.enable` once the app has loaded, or `--yjit-disable` with tuning flags - so boot-only code is not compiled. Budget memory per worker: `--yjit-mem-size` (128 MiB by default) is a soft limit on generated code plus metadata, and `RubyVM::YJIT.runtime_stats[:code_region_size]` and `[:yjit_alloc_size]` show what each worker actually uses, even without `--yjit-stats`. When the limit is hit, YJIT stops compiling new code (code GC is off unless `--yjit-code-gc`). Canary it, compare latency, CPU per request and worker RSS, avoid frequent worker restarts, and keep the environment toggle as the rollback.

code

ruby · 10 lines
ruby
# in each worker, after the application has loaded
if defined?(RubyVM::YJIT) && ENV["APP_YJIT"] == "1"
  RubyVM::YJIT.enable(mem_size: 96)
end

# periodically, e.g. from a health endpoint
if defined?(RubyVM::YJIT) && (stats = RubyVM::YJIT.runtime_stats)
  used = stats[:code_region_size] + stats[:yjit_alloc_size]
  warn "yjit_mem_mib=#{used / 1024 / 1024}"
end

go deeper

for a junior

Recall that YJIT costs memory per process and that its gain should be measured on real traffic before it is rolled out.

for a middle

Explain --yjit-mem-size, lazy enabling with RubyVM::YJIT.enable, and the runtime_stats memory counters.

for a senior

Demonstrate the rollout: profile first, canary with a control, size memory against worker count, keep workers alive, and keep a restart-only rollback.

for a principal

Weigh YJIT memory against worker count per host and decide per service whether CPU saved justifies the capacity it consumes.

## Step 1: will it pay off at all? YJIT compiles Ruby bytecode, so it only shortens time spent **executing Ruby code**. A typical JSON API request spends time in three places: - waiting on the database and other services - untouched by a JIT; - **encoding JSON** - Ruby 4.0 ships json 2.18, whose generator is a C extension, so the encoding loop itself is already machine code; - **Ruby code around it** - routing, authorisation, building the hashes and arrays to serialise, formatting values - which is what YJIT speeds up. Profile a representative slice of traffic first. If most time is in waits or C functions, the gain is small and the memory cost remains; if Ruby frames dominate, continue. ## Step 2: enable it where it does the most good YJIT's documentation recommends **enabling it lazily**: with `--yjit` or `RUBY_YJIT_ENABLE=1`, YJIT may compile code that is only used during boot. Two options: 1. Call `RubyVM::YJIT.enable` after the application has loaded - the documentation's example is a server's after-fork hook, so each worker enables it for itself. 2. Start with tuning flags plus `--yjit-disable` (any `--yjit-*` flag otherwise starts YJIT at boot), then call `enable`. On Ruby 4.0 `RubyVM::YJIT.enable(mem_size: 96, call_threshold: 60)` sets the limits directly. `mem_size:` must be an Integer from 1 to 2048 (MiB). Keep workers alive. The documentation warns that when processes are killed too frequently, compile time can outweigh the speed-up; memory-based worker killers set with no YJIT headroom will recycle workers exactly when they get fast. ## Step 3: set and watch the memory budget | Setting or stat | Meaning | |---|---| | `--yjit-mem-size=N` | soft limit in MiB (default 128 since 3.4) on compiled code plus YJIT's metadata | | `--yjit-exec-mem-size=N` | hard limit on the executable code region only | | `runtime_stats[:code_region_size]` | bytes of machine code memory in use | | `runtime_stats[:yjit_alloc_size]` | bytes of YJIT's own metadata allocations | | `--yjit-code-gc` | discard all compiled code at the limit and start over (off by default since 3.3) | Budget per worker: with eight workers, a 128 MiB limit can add up to about a gigabyte per host. `RubyVM::YJIT.runtime_stats` returns these memory counters whenever YJIT is enabled; `--yjit-stats` adds the full counter set at a run-time cost. When the limit is reached, compilation of new code stops and the code already compiled keeps running; with `--yjit-code-gc`, all compiled code is thrown away and compiling restarts. ## Step 4: measure the result honestly - Run a **canary** set of workers with YJIT and a control set without, on the same traffic. - Compare **latency percentiles**, **CPU time per request** and **resident memory per worker**, not a single micro-benchmark. - `ratio_in_yjit`, the share of instructions executed in compiled code, should ideally approach 99%, and raising `--yjit-mem-size` often raises it. In Ruby 4.0 that stat is only present on a Ruby configured with `--enable-yjit=stats` and run with `--yjit-stats`. - Use `--yjit-stats` on a canary to see which instructions side-exit, and fix only the hottest methods. ## Step 5: roll out and keep a way back - Ship behind the environment toggle or a config flag read before `enable` is called, so rollback is a restart. - Log `RUBY_DESCRIPTION` at boot so every worker reports `+YJIT` or not. - Re-check memory after each Ruby upgrade: YJIT's defaults and counters change between versions, and its documentation says counter names are not guaranteed to stay the same. ## Summary Decide from a profile, enable after boot, size `--yjit-mem-size` against the worker count, watch `code_region_size` plus `yjit_alloc_size`, and compare a canary with a control on real traffic.

  • After enabling YJIT, per-worker memory rose by about 100 MiB and latency improved only 3%; what do you do?
    Check where the time goes: a small gain means most time is outside Ruby code, so the memory may not be worth it. Try a lower `mem_size:` and see whether latency holds; if `ratio_in_yjit` is measurable, watch how it falls. If memory is the binding constraint for the host, fewer workers with YJIT versus more without is the real comparison.
  • What happens when a worker reaches YJIT's memory limit with Ruby 4.0 defaults?
    YJIT stops compiling new code; everything already compiled keeps running as machine code and new hot code stays interpreted. Code GC is off by default since Ruby 3.3; with `--yjit-code-gc` YJIT instead discards all compiled code and starts compiling again, which frees memory but causes a performance dip.
  • Why might ratio_in_yjit be missing from runtime_stats on your Ruby 4.0 servers?
    In Ruby 4.0 it no longer works in the default build. It needs a Ruby configured with `--enable-yjit=stats` and run with `--yjit-stats`; a stock build still reports memory counters such as `code_region_size` and `yjit_alloc_size`.

saying these in an interview costs you the question

  • YJIT will speed up JSON encoding done by the json C extension
  • --yjit-mem-size caps the whole Ruby process's memory
  • Code GC is on by default, so YJIT memory can never hit its limit
  • RubyVM::YJIT.runtime_stats returns nil unless --yjit-stats is passed
  • Enable YJIT at boot so the application's startup code gets compiled
  • Restarting workers often is fine because YJIT compiles instantly