Using resource.setrlimit, which limit changes are reversible and which are not?

### The rule in one line A process may move its **soft** limit anywhere between zero and its **hard** limit, and may move its **hard** limit **down only**. Going back up requires privilege that an ordinary service process does not have, so lowering a hard limit is a one-way door for the lifetime of that process and every child it later spawns. ### The call `resource.setrlimit(which, (soft, hard))` takes the resource constant and a two-element tuple — you always write both values, even when you only mean to change one, which is why the idiom is to read the current pair first: ```python soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE) resource.setrlimit(resource.RLIMIT_NOFILE, (hard, hard)) # raise soft up to the hard ceiling ``` `resource.RLIM_INFINITY` is a legal value wherever the kernel permits it. Note that raising the soft limit to the hard limit is the single most common line of `resource` code in real services: the environment ships a low soft limit and a high hard limit, and the program opts in to the higher one. ### How CPython reports refusal Both refusals arrive as `ValueError`, which surprises people who expect a `PermissionError`: * asking for a soft limit above the hard limit gives `ValueError: current limit exceeds maximum limit` (the kernel returned `EINVAL`); * asking to raise the hard limit without privilege gives `ValueError: not allowed to raise maximum limit` (the kernel returned `EPERM`). Anything else — an unsupported resource on this platform, for instance — comes back as `OSError`; `resource.error` is simply an alias for `OSError`. Catch `ValueError` around a startup call and degrade gracefully rather than crashing a service because one environment ships a lower ceiling. ### Why the one-way rule exists, and what it costs you The hard limit is a self-imposed sandbox. A process that is about to run something it does not fully trust — parsing a hostile file, running a plugin — can lower its own hard `resource.RLIMIT_NOFILE` or `resource.RLIMIT_AS` and know the restriction cannot be lifted afterwards, by that process or by anything it forks. Raising it again needs an effective root user (on Linux, the `CAP_SYS_RESOURCE` capability), which the sandboxed code is precisely not supposed to have. The cost is that the drop is unrecoverable and it propagates. Tests are the usual casualty: a test that lowers a hard limit to prove the failure path works has permanently damaged the interpreter process running the suite, and every later test that needs the old ceiling now fails. The fix is to exercise hard-limit drops in a child process — spawn a fresh interpreter, lower the limit there, and assert on its exit status and output — and keep the parent's limits untouched. ### Where to set the limit Limits are inherited across `fork()` and preserved across `exec()`, so the *when* matters as much as the *what*: a child receives whatever its parent had in force at the moment it was spawned, and a later `resource.setrlimit` in the parent does not reach a child that already exists. Do the raise as early as possible in start-up, before any worker processes are created. Python 3.14 sharpens this. The default `multiprocessing` start method on Unix other than macOS is now `forkserver` (macOS and Windows use `spawn`, and `fork` must now be requested explicitly). With `forkserver` a helper process is started once and every later worker is forked from *that* helper — so the limits your workers see are the ones in force when the fork server was first started, not the ones in force when you called for a worker. Raise limits before the first worker is requested, or raise them inside each worker as its first action. ### The trap that raising `RLIMIT_NOFILE` does not fix A high descriptor limit does not make every API able to use it. `select.select` is bounded by the fixed `FD_SETSIZE` compiled into the C library (commonly 1024) and will fail or misbehave on a descriptor number above it, no matter what the rlimit says. Code that genuinely needs tens of thousands of descriptors must use a scalable readiness interface — `selectors.DefaultSelector` picks `epoll` or `kqueue` for you, and `asyncio` sits on the same machinery. Raising the limit is necessary, but the polling API has to be able to spend it.