skip to content

Load Time & Memory Footprint

Bootsnap caches load-path lookups and compiled Ruby to cut boot time, while YJIT, jemalloc and per-core process counts trade memory for speed. Interviewers probe what each one costs.

part ofRuby on Railsread it →
on this pageshow

explore

questions

5

For a Rails 8.1 app on Puma, how do you choose processes per CPU core and threads per process when optimizing for throughput versus latency?

level: seniorimportance: must knowfreq 50%

answer

  1. processes for parallelism, threads for I/O
  2. default 3 threads since Rails 7.2
  3. throughput: one process per core
  4. latency: 1 thread, 1.3-1.5 processes per core
  5. RAM per core can cap processes

basics

~20 s

Processes give parallelism and threads only overlap I/O. For throughput, run one process per core and raise threads until latency suffers; for latency, keep threads low, even one per process with about 1.3 to 1.5 processes per core.

solid answer

~50 s

On CRuby each process runs Ruby on one core at a time, so **processes** are the parallelism and **threads** only fill the time a request spends waiting on I/O. Rails' tuning guide gives two directions. For **throughput**, run one process per core and add threads per process until the latency cost is no longer acceptable. For **latency**, keep threads low; at the extreme, one thread per process and 1.3 to 1.5 processes per core, so a core is not idle while one process waits. Rails 7.2 lowered the generated default from 5 threads to 3. A well-tuned app spends no more than about half its time in I/O, so extra threads mostly queue for the lock; apps making slow third-party calls inline may justify more. Memory decides the rest: if RAM per core cannot hold that many processes, the host shape, not the formula, is the limit. Then load-test.

go deeper

for a junior

Recall that processes give parallelism on CRuby, threads help while requests wait on I/O, and Rails generates 3 threads per process.

for a middle

Explain why threads beyond the app's I/O share hurt latency, and why Rails 7.2 cut the default from 5 to 3.

for a senior

Size per core for a stated goal, account for memory per core and copy-on-write, and validate each change with load tests and p95 latency.

for a principal

Choose the goal itself: cost-per-request versus user latency, and the host shapes that make either one achievable across the fleet.

## Two dials, two different resources A Rails app on Puma has two concurrency dials, and they buy different things: - **Processes** (Puma workers) give **parallelism**. Separate CRuby processes can execute Ruby on separate cores at the same moment. - **Threads** inside one process give **concurrency for I/O**. While one thread waits for the database or an HTTP response, another can run Ruby. But only one thread per process executes Ruby at any instant, so a process never uses more than about one core. That asymmetry is why sizing is a trade between three properties the Rails guide names: **throughput** (requests per second the server sustains), **latency** (how long each request takes) and **memory**. Improving one usually hurts another. ## Why more threads can hurt latency When a thread's I/O completes it must wait for the process's lock before it can continue running Ruby. With many threads busy, a request that is ready to proceed waits behind others. The guide's rule of thumb: if the app spends about 50% of its time in I/O, more than 2 or 3 threads per process may severely hurt latency, and throughput gains flatten fast. A well-crafted Rails app without slow SQL or N+1 problems rarely exceeds that 50%, which is the reasoning behind the generated default. **Rails 7.2 changed the generated `config/puma.rb` from 5 threads to 3** for this reason. The release notes describe threads spending significant time waiting for the GVL when the count is too high, and settle on 3 as a balance. Apps that call third-party APIs inline, with much higher I/O share, are the exception that may benefit from more. ## The two recipes | Goal | Processes | Threads per process | Why | |---|---|---|---| | Throughput / server utilization | one per CPU core | raise from 3 until latency is unacceptable | every core busy, threads soak up I/O waits | | Latency | about 1.3 to 1.5 per core | low, down to 1 | no queueing for the lock; spare processes cover I/O waits | On the marketplace's 8-core app servers that means, for example: 1. **Throughput-leaning:** 8 workers with 3 threads, then load-test 4 or 5 threads and stop when p95 latency degrades. 2. **Latency-leaning:** 10 to 12 single-threaded workers, accepting the extra memory of more processes. The guide notes that "one per core" counts what the application may use. On a box shared with other services, count the cores you want this app to have. ## Memory decides what is possible Processes cost far more memory than threads, because forked workers share pages only partially through copy-on-write. Threads are cheaper but not free. So the recipes run into the hardware: - If RAM per core is too low to hold one worker per core, you cannot use all the CPU you pay for. Pick a plan with more memory per core, or cut memory per worker first (allocator, YJIT limits). - Larger hosts are more memory-efficient. The guide's example: 2 servers with 32 processes each use less memory per core than 16 servers with 4 processes each, because each server's master and shared pages are amortized over more workers. - Memory per worker is measured after warm-up under real traffic, not at boot. ## Where the guide's numbers come from The 1.3 to 1.5 figure is not magic. A single-threaded process that spends, say, a quarter of its time waiting on I/O leaves its core idle for that quarter. Running a few more processes than cores lets another process use the core during those waits, without any thread contention inside a process. The exact ratio depends on the I/O share, which is why it is a range. The throughput recipe works the other way round: threads fill the waits inside each process, and the process count stays at one per core. ## Measure, then decide The guide states plainly that no default is best for everyone. The process: - Fix the goal first: a throughput target at acceptable p95, or the lowest latency at acceptable cost. - Load-test with production-like requests after warm-up, changing one dial at a time. - Watch CPU per core, request queue time, p50/p95 latency and RSS per worker together. - Re-check after big changes in I/O share, such as moving slow API calls into jobs, which makes fewer threads optimal. Applying the numbers is a matter of `WEB_CONCURRENCY` and `RAILS_MAX_THREADS` in the deploy environment, and keeping the database pool at least as large as the thread count.

  • The marketplace moves its slow payment-provider calls out of requests into background jobs; how should Puma's threads change?
    Requests now spend less of their time waiting on I/O, so extra threads have less waiting to overlap and more contention to cause. The case for more than 3 threads weakens, and fewer threads per process with the same or more processes per core usually gives better latency. Re-run the load test to confirm rather than assume.
  • Why can two large servers serve the same traffic with less total memory than many small ones?
    Each server pays a fixed cost: a Puma master and pages that preloaded workers share. With 32 workers on a server, those costs are spread over 32 processes; with 4, over just 4. The Rails guide's example is 2 servers of 32 processes using less memory per core than 16 servers of 4.

  • On CRuby, more threads per process always means more throughput with no latency cost.
  • With one thread per process you should still run exactly one process per core.
  • Rails' 3-thread default simply copies Puma's own default thread count.
  • Splitting the same processes across many smaller servers saves memory.
  • Threads are free, so only the number of processes affects memory.

Assumes Rails 8.1, whose generated config/puma.rb defaults to 3 threads per process; Rails 7.2 lowered it from 5.

open as a page

In a Rails 8.1 app, what do Bootsnap's load-path and compile caches save at boot, and why does the generated Dockerfile run bootsnap precompile?

level: middleimportance: should knowfreq 40%

basics

~20 s

Bootsnap caches where each required file lives on $LOAD_PATH and the compiled bytecode of Ruby files, plus parsed YAML. The Dockerfile runs bootsnap precompile so those caches ship inside the image and containers skip compiling at boot.

open as a page

In Rails 8.1, what does config.yjit default to in each environment, and what changes when you set it to false or a Hash?

level: middleimportance: should knowfreq 35%

basics

~10 s

Under load_defaults 8.1, config.yjit is !Rails.env.local?: off in development and test, on in production and other environments. false makes Rails skip enabling YJIT; a Hash enables it with those options, such as mem_size.

open as a page

Puma workers in a large legacy Rails marketplace grow from 400 MB to 1.2 GB daily on glibc malloc; why might the allocator be to blame, and what do jemalloc and MALLOC_ARENA_MAX=2 change?

level: seniorimportance: should knowfreq 40%

basics

~20 s

With several Puma threads per worker, glibc malloc spreads allocations over many arenas and fragments, so freed memory is not returned and RSS creeps. jemalloc, preloaded via LD_PRELOAD as Rails' Dockerfile does, fragments far less; MALLOC_ARENA_MAX=2 limits arenas but can slow Ruby.

open as a page

For a large legacy Rails marketplace's app-server fleet, how would you set a boot-time and memory budget, and which levers would you trade against each other?

level: principalimportance: nice to knowfreq 18%

basics

~20 s

Set two budgets: time from container start to serving, and resident memory per warmed worker times workers per host. Bootsnap and slimmer load paths cut boot; preloading, jemalloc, YJIT limits and host size move memory; trade them against latency.

open as a page