A platform team must run customers' untrusted code as functions; how do you choose between microVMs, isolates and hardened containers as the sandbox?
answer
- what code do customers submit
- density against escape risk
- a warm sandbox never crosses tenants
- you own the patching afterwards
basics
~20 sChoose by the code you accept and the risk you can carry: arbitrary binaries need a microVM per tenant, single-language scripts can share isolate processes, containers need an extra kernel layer. Whatever you pick, never reuse a sandbox across tenants.
solid answer
~50 sI start from what customers submit. Arbitrary binaries or whole images need a machine boundary, so a **microVM per tenant**, or a container wrapped in one. Scripts in one language, or WebAssembly, can run in **isolates**, which are far denser and start almost instantly but make the runtime engine the boundary, so I would add process-level system-call filtering, partition risky tenants into separate processes and commit to patching the engine within hours of an advisory. Plain containers on a shared kernel are not a tenant boundary on their own. Then come the lifecycle rules that hold on every model: a warm sandbox is never reused across tenants, sandboxes have a capped lifetime, and each tenant gets egress rules and quotas. Finally the standing cost: whoever runs the sandbox owns its patching and incident response, which is often the argument for building on a hosted platform rather than operating the fleet yourself.
go deeper
Recall that untrusted code needs a strong sandbox and that a warm sandbox is never reused for another customer.
Explain what each model isolates and what customers can submit to each: anything for a microVM, scripts or WebAssembly for an isolate.
Show the lifecycle controls that hold on every model: no cross-tenant reuse, capped sandbox lifetime, egress rules and per-tenant quotas.
Own the trade: submission format, density and price against escape risk, and the patch cadence and incident response the team commits to for the product's life.
## The decision and what it commits you to Running other people's code is a different problem from running your own. Your own functions share a trust level; a customer's function may be written to attack the platform or other customers. The sandbox model is the main control, and it is a **standing** decision: it fixes the cost per tenant, what customers can submit, and which security advisories your team must answer at short notice for as long as the product lives. There is no single right answer; there is a defensible match between the code you accept, the density you need and the risk you can carry. Three questions frame it: what may customers submit, how many tenants must one host carry for the price to work, and how fast can your team patch the boundary it picks? ## Start from the code you accept | What customers submit | Fitting sandbox | Why | |---|---|---| | Arbitrary binaries, whole container images, any language | **microVM per tenant**, or a container inside one | Only a machine boundary runs anything; escaping needs a flaw in the small virtualization layer | | Scripts in one language, or WebAssembly modules | **Isolates**, with process-level hardening | Highest density and near-instant start; the engine's surface limits what code can touch | | Your own trusted code split per customer | Containers on a shared kernel can be enough | The threat is mistakes, not attacks; the kernel boundary is acceptable | Narrowing what customers may submit is often the cheapest security control available: a product that accepts only WebAssembly or one scripting language can use a denser model than one that accepts arbitrary images. ## Lifecycle rules that hold on every model The sandbox lifecycle (reuse, freeze, teardown) is where untrusted-code platforms usually leak, whatever the isolation model: 1. **Never reuse a warm sandbox across tenants.** A warm sandbox keeps memory, temporary files and open connections from its last invocation. Reuse within one tenant's function is fine; across tenants it is a data leak. 2. **Cap sandbox lifetime.** Recycle sandboxes after a bounded time or number of invocations, so a compromise or a memory leak cannot persist indefinitely. 3. **Scrub on recycle, or destroy.** Destroying and recreating is simpler to reason about than scrubbing, at the price of more starts. 4. **Restrict egress per tenant.** Deny outbound traffic by default, block the host's metadata and control-plane addresses, and allow only what the product needs. 5. **Enforce per-tenant quotas** on CPU, memory, concurrency and invocations, so one tenant cannot exhaust shared capacity or run up another's latency. ## The standing costs of each model - **microVMs.** You operate a virtual machine monitor fleet, patch guest and host kernels, and pay for a guest kernel and its memory per sandbox. Start-up is slower than an isolate, so pre-started pools are common, and pools must still respect the never-across-tenants rule. - **Isolates.** A runtime-engine bug is a cross-tenant escape, and such engines receive security fixes often, so you commit to patching within hours. You also need layers beneath the engine (system-call filtering, process partitioning, coarse timers against side channels) and accept a narrow API surface that limits what customers can build. - **Every model.** Shared hardware leaves side channels that no software boundary fully closes. The mitigations (keeping tenants off sibling hardware threads, coarse timers, dedicated hosts for the most sensitive customers) cost density, so they belong in the same price-against-risk calculation. - **Hardened containers.** Namespaces alone are not enough between hostile tenants; adding a user-space kernel or a VM wrapper brings back much of the microVM cost, plus compatibility gaps with some system calls. ## A defensible default, and when to revisit it - Prefer a **hosted platform** that already sells multi-tenant isolation if it meets your API and residency needs; operating your own sandbox fleet is a security team's full-time job. - If you build: **microVM per tenant** for arbitrary code; **isolates** for high-volume, single-language snippets where density decides the price; containers only inside a second boundary. - Keep high-risk tenants (free tiers, anonymous sign-ups) on **separate hosts or processes** from paying ones. - Revisit the choice when what customers may submit changes, when density starts to dominate cost, or when an escape advisory shows the patch cadence you committed to is not being met.
- Why is a pre-started pool of microVMs a risk on a multi-tenant platform, and how do you keep it?A pool hides start-up time, but a sandbox handed out from the pool must never have run another tenant's code. Keep the pool tenant-neutral (sandboxes that have never been assigned), assign each sandbox to one tenant for its whole life, and destroy it instead of returning it to the shared pool.
- How would you justify isolates for untrusted code to a security reviewer?By showing the layers: the engine's memory boundary, a hardened process with system-call filtering beneath it, risky tenants partitioned into separate processes, coarse timers against side channels, per-tenant CPU and memory caps, and a written commitment to ship engine security fixes within hours. Plus a narrow submission format, which shrinks what an attacker can try.
- Containers on a shared kernel are a sufficient boundary between hostile tenants.
- Warm sandboxes can be shared across tenants if the handler clears its globals.
- Isolates are unsafe for any untrusted code, so a microVM is always required.
- Once the sandbox model is chosen, tenant isolation is solved and needs no further work.
- Density is free: packing more tenants per host never changes the risk.