skip to content

What do the two numbers from Python's resource.getrlimit() actually mean?

level: juniorimportance: should knowfreq 34%

answer

  1. Kernel ceilings, one entry per resource
  2. Two numbers per limit, different jobs
  3. Soft is enforced, hard is the ceiling
  4. RLIM_INFINITY is the unlimited sentinel

basics

~20 s

resource.getrlimit returns a (soft, hard) pair for one kernel limit. The soft value is the ceiling enforced right now; the hard value is the maximum the soft one may be raised to. resource.RLIM_INFINITY means no ceiling.

solid answer

~40 s

`resource.getrlimit(resource.RLIMIT_NOFILE)` returns a two-element tuple, `(soft, hard)`. The **soft** limit is what the kernel enforces against this process at this moment; the **hard** limit is only the ceiling on how high the soft limit may be set. `resource.RLIM_INFINITY` is the sentinel for unlimited, and you compare against the constant rather than a magic number. The split lets the environment hand down a generous hard limit while the program picks its own, tighter soft limit. Limits are per process, inherited across `fork()` and preserved across `exec()`. The two that matter in practice are `resource.RLIMIT_NOFILE`, whose breach gives `OSError` with `errno.EMFILE` ("Too many open files"), and `resource.RLIMIT_AS`, whose breach gives a catchable `MemoryError`. The module is Unix-only — importing `resource` on Windows raises `ImportError`.

code

python · 5 lines
python
import resource

soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
print("soft:", soft)
print("hard:", "unlimited" if hard == resource.RLIM_INFINITY else hard)

go deeper

for a junior

Be ready to say the call returns a pair and which half is enforced now. Knowing that resource.RLIMIT_NOFILE breaches show up as "Too many open files" is enough at this level.

for a middle

Explain the mechanics: the tuple shape, RLIM_INFINITY, per-process scope, inheritance across fork and exec, and the different failure each resource produces when its soft limit is crossed.

for a senior

Show you read these at startup and assert on them, so a badly provisioned environment fails loudly at boot rather than at connection 1,024 during a traffic peak.

for a principal

Own the question of where limits should come from at all — the platform that starts the process versus the program that tightens itself — and why encoding a required floor in the application makes misprovisioning visible.

### The pair, and why there are two numbers Every Unix process carries a small table of kernel-enforced ceilings, one entry per resource. Python reads that table with `resource.getrlimit(which)`, where `which` is one of the module's RLIMIT constants, and gets back a two-element tuple `(soft, hard)`. The `resource` module is Unix-only: importing it on Windows raises `ImportError`, so portable code guards the import. **Soft is the limit the kernel enforces right now.** When the process asks for more of that resource than the soft value allows, the request fails, and the shape of the failure depends on the resource rather than on the `resource` module. **Hard is the ceiling on the soft value.** It is not checked against your allocations at all; it is the highest value this process is allowed to set its own soft limit to. An unprivileged process may move the soft limit anywhere in the range `[0, hard]`. The split exists so that a limit can be handed down once, from outside, and then tightened from inside. Whoever starts the process (a shell, a supervisor, an init system) picks a generous hard limit; the program itself then chooses the soft limit it actually wants to run under, and can drop it further for a risky phase without being able to undo the drop. That is the same "privileges go down, never up" pattern the rest of Unix uses. `resource.RLIM_INFINITY` is the sentinel for "no ceiling". On a 64-bit build it prints as a very large integer; compare against the constant, never against a hard-coded number. ### The two limits that actually bite `resource.RLIMIT_NOFILE` caps how many file descriptors the process may hold open at once — regular files, sockets, pipes, and the kernel objects behind an event loop all consume one. Crossing it produces an `OSError` whose `errno` is `errno.EMFILE` (24), with the message "Too many open files": `open()` fails, accepting a connection fails, and the failure surfaces wherever the next descriptor happened to be needed, which is usually not where the descriptors were consumed. `resource.RLIMIT_AS` caps the size of the process's virtual address space. An allocation that would cross the ceiling is refused by the kernel, and CPython turns that refusal into `MemoryError` — an ordinary, catchable Python exception raised inside the process. The rest you will meet occasionally: `resource.RLIMIT_STACK` (stack size), `resource.RLIMIT_CORE` (maximum core-dump size, where zero means no core file is written), `resource.RLIMIT_NPROC` (processes per real user id) and `resource.RLIMIT_CPU` (CPU seconds, after which the kernel sends `signal.SIGXCPU`). `resource.RLIMIT_RSS` exists in the module but modern Linux kernels no longer enforce it, which is exactly the kind of thing worth checking before you rely on a limit. ### Scope, inheritance and what you cannot read Limits are per process. They are not per thread, and they are not per user account — several processes owned by the same user each carry their own pair. A child inherits its parent's limits across `fork()`, and they survive `exec()`, so a worker starts life with whatever the parent had in force when it was spawned. `resource.getrlimit` reads only the calling process's own table; the module has no portable way to read another process's limits. ### Two things the pair is not It is not a Python-level limit. `sys.setrecursionlimit` adjusts CPython's own frame counter and has nothing to do with `resource.RLIMIT_STACK`; a recursion deep enough to run off a small thread stack takes the process down with a segmentation fault instead of raising `RecursionError`. It is also not the container's memory ceiling. A container memory cap is enforced by the kernel's control groups, not by an rlimit, so inside a container capped well below the machine's RAM `resource.getrlimit(resource.RLIMIT_AS)` will still report unlimited. Reading the pair tells you what your process is allowed to ask for; it does not tell you everything the platform may do to you. ### Using it in practice Reading and logging the pair at startup costs microseconds and answers "is this environment configured the way we assumed" without shelling out to a shell builtin and parsing text. Because both values are plain integers, they are also easy to assert on: a service that intends to hold twenty thousand concurrent sockets can compare the soft `resource.RLIMIT_NOFILE` value against that number at startup and refuse to boot with a clear message, instead of failing mysteriously at descriptor 1,024 an hour into a traffic peak. That startup check is the single highest-value use of `resource.getrlimit` in application code.

  • What happens to a Python program that exceeds the soft resource.RLIMIT_NOFILE value?
    The next descriptor-creating call fails with `OSError`, `errno` set to `errno.EMFILE` (24) and the message "Too many open files". It surfaces wherever the next descriptor was needed — an `open()`, a socket accept, a subprocess pipe — which is usually not the code that consumed the descriptors. Nothing is raised at the moment the limit is reached; only the next request fails.
  • Does resource.getrlimit show a container's memory ceiling?
    No. A container memory cap is enforced by the kernel's control groups, which are a different mechanism from rlimits. Inside a container capped far below the host's RAM, `resource.getrlimit(resource.RLIMIT_AS)` will still report unlimited unless someone explicitly set an rlimit as well. The call reports what this process may ask for, not every ceiling the platform imposes on it.

The hard limit is the credit limit on the card; the soft limit is the spending cap you set yourself in the banking app. Only the second one stops today's purchase, and you can never raise the first from inside the app.

saying these in an interview costs you the question

  • Saying getrlimit returns how many descriptors are currently open
  • Treating the hard limit as the one the kernel enforces
  • Assuming the limits are system-wide rather than per process
  • Comparing against a magic large integer instead of RLIM_INFINITY
  • Confusing sys.setrecursionlimit with resource.RLIMIT_STACK
  • Expecting the resource module to import on Windows

context