In Spark, how many tasks can one executor run at once, and what sets that limit?

## Slots, not cores An executor is one JVM process. Inside it Spark maintains a thread pool and a count of **task slots**, computed as `spark.executor.cores / spark.task.cpus`. `spark.task.cpus` defaults to 1, so in practice slots equal cores. Each slot holds one running task; the scheduler will not place a fifth task on a four-slot executor until one of the four finishes. The word "core" is misleading. Spark is not reserving a physical CPU: it is handing out accounting units that the cluster manager also uses for placement. If your task's user code spawns its own threads or calls into a native BLAS library that saturates every core on the box, Spark still counts it as one slot, and the machine becomes oversubscribed. `spark.task.cpus` exists precisely for that case — set it to 2 and each task consumes two slots, halving concurrency per executor. ## The arithmetic that matters Application-wide concurrency is: `concurrent tasks = executors × (spark.executor.cores / spark.task.cpus)` And the number of **waves** a stage needs is: `waves = ceil(stage task count / concurrent tasks)` A stage with 200 tasks (a common number, since post-shuffle partitioning defaults to `spark.sql.shuffle.partitions` = 200) running on ten four-core executors takes five waves. If tasks average 30 seconds, the stage takes roughly two and a half minutes plus scheduling overhead — and it takes that long regardless of how much heap sits unused. This arithmetic is the fastest sanity check available in an interview. Given a stage's task count and a cluster's slot count, you should be able to say immediately whether the job is slot-starved, well matched, or wasting a cluster. ## The two mismatches **Fewer tasks than slots** is pure waste: a stage with 12 tasks on a cluster with 200 slots leaves 188 idle, and adding executors does nothing. The fix is upstream — more input splits, or repartitioning before the expensive stage — not more machines. **Far more tasks than slots** is normally healthy, because many small waves smooth over uneven task durations. It becomes a problem only at extremes: tens of thousands of very short tasks make per-task scheduling and serialization overhead a visible fraction of runtime, and the driver becomes the bottleneck launching and collecting them. Within any wave, the wave finishes when its slowest task finishes. That is why a single task an order of magnitude slower than its peers can stall an otherwise well-provisioned stage, and why the summary metrics table on the Spark UI stage page shows min, 25th percentile, median, 75th percentile and max task duration rather than just an average. ## Threads in one JVM, and what they share Because the slots on an executor are threads in one JVM, all concurrent tasks share the same heap, the same garbage collector and the same off-heap allocations. Increasing `spark.executor.cores` therefore increases memory pressure per executor even though it costs no extra memory setting: four tasks each spilling large hash tables into the same heap is a very different situation from four single-core executors. It also increases GC contention, which is a large part of why extremely wide executors tend to underperform in practice. They also share the executor's shuffle write and read bandwidth, and the JVM's classloader and broadcast variables — the last of which is an advantage: a broadcast table is materialized once per executor, not once per task, so wider executors amortize broadcasts better. ## Locality delays the assignment Slot availability is necessary but not sufficient for a task to launch. Spark prefers to run a task where its data already is, using locality levels in descending preference: `PROCESS_LOCAL` (data in this executor's memory), `NODE_LOCAL` (on this machine), `RACK_LOCAL`, then `ANY`. When a free slot has poor locality, the scheduler waits up to `spark.locality.wait` (default 3 seconds) for a better slot before relaxing to the next level. On a busy cluster with cached data this shows up as slots that appear free while tasks sit pending — a Spark UI symptom that looks like a scheduling bug but is deliberate. ## Where the numbers come from These are submission-time settings: `--executor-cores` / `spark.executor.cores`, `--num-executors` / `spark.executor.instances`, and `spark.task.cpus`. With dynamic allocation enabled the executor count moves at runtime, so slot count is a moving target and Spark scales executors to the pending task backlog. Whichever way they are set, the two rules stay the same: tasks come from partitions, and slots come from cores.