A component keeps producing deadlocks because its operations acquire several locks. What redesigns remove the possibility entirely rather than managing it, and what does each one cost?
answer
- one coarse lock first, measure second
- shard by key so one op = one lock
- immutable / copy-on-write kills exclusion for readers
- single owner + messages: still deadlocks if handlers block
- CAS removes deadlock, adds livelock
basics
~20 sCollapse to one coarse lock; partition state so each operation touches one shard; make data immutable or copy-on-write; give state a single owner and use messages; or use lock-free atomic updates. Each trades throughput, memory, latency or complexity for the guarantee.
solid answer
~60 sRanked roughly by how much they buy and cost: - **One coarse lock.** Removes circular wait inside the component outright. Costs throughput under contention and still deadlocks if it can be held across a call into another locked subsystem. - **Partition by key.** Each operation touches exactly one shard, so no two locks are co-held. Costs a design answer for the rare cross-shard operation, which needs an ordering or a single serializer. - **Immutability / copy-on-write.** Readers need no exclusion at all, so mutual exclusion disappears for the read path. Costs allocation and makes writes coarse. - **Single owner plus messages (actor or channel).** State is mutated by one participant; others send requests. Costs latency and requires that handlers never block awaiting a reply, or two actors deadlock exactly like two mutexes. - **Lock-free updates via compare-and-swap, or optimistic versioning with retry.** No mutual exclusion, so no deadlock; livelock becomes the hazard and correctness is hard. The rule of thumb: prefer designs where the invariant cannot be violated by a stranger's code path, not designs that require everyone to remember a rule.
code
text · 8 linesbefore: op(x, y) -> lock(x); lock(y) # order matters, cycle possible
after: shard = shardOf(key)
lock(shard); apply(op); release(shard)
cross-shard op (rare):
acquire shards in ascending shard index, or
hand the whole operation to one serializergo deeper
Know that using one lock instead of several removes the multi-lock deadlock, and that immutable data needs no locking at all.
Compare coarse locking, sharding and immutability, and state the throughput cost of each.
Add message passing and optimistic or lock-free updates, and be explicit that message passing still deadlocks when handlers block on replies.
Sequence the decision - simplest correct design first, measure contention, then pay for partitioning or ownership changes - and make the invariant structural and local so no future caller can break it.
## Reframing the problem Recurring deadlocks in one component are a design signal, not a series of bugs. Every mitigation discussed elsewhere - ordering, bounded attempts, shorter sections - depends on humans continuing to obey a rule as the code grows. A redesign that removes the multi-lock requirement removes the class of defect and stops consuming review attention forever. The engineering question is which guarantee you want and what you are willing to pay. ## Option 1: collapse to a single lock Guard the whole component with one lock. Circular wait becomes impossible internally, because a thread never holds two of your locks at once. This is the cheapest correct answer, and it is dramatically undervalued: a component that is not on the hot path rarely notices the serialization, and the simplification often makes other bugs disappear too. Cost: all operations serialize, so throughput is capped by the critical section length times the arrival rate. Measure before assuming that matters. Two residual risks remain - if the single lock can be held while calling into another locked subsystem, cycles across components are still possible, and one coarse lock makes a slow operation block fast ones, turning a throughput issue into a latency issue. ## Option 2: partition the state Shard state by key so any single operation touches exactly one shard and needs exactly one lock. Concurrency scales with shard count, and no operation co-holds locks, so no cycle can form. Cost: cross-shard operations. If some operation legitimately spans two shards - a transfer between accounts in different shards - you are back to two locks and must apply an ordering there, or route all cross-shard work through a single serializing path. Choosing a shard key that makes cross-shard operations rare is the real design work, and skew can leave one hot shard doing most of the traffic. ## Option 3: immutability and copy-on-write If a structure is never mutated after publication, readers need no exclusion at all - mutual exclusion, the first Coffman condition, simply does not apply to them. Updates create a new version and swap a single reference atomically, so only writers coordinate, often with just one atomic operation. Cost: allocation and copying, which is fine for read-mostly configuration or routing tables and poor for large, write-heavy state. Writes become coarse-grained (whole-structure swaps), and readers may observe a slightly stale version, which must be acceptable semantically. Persistent (structurally shared) data structures reduce the copying cost substantially. ## Option 4: single ownership and message passing Give each piece of state exactly one owner - an actor, a task, a single-threaded event loop - and let everyone else send messages. No shared mutable state means no locks, so lock-based deadlock is gone, and the design also sidesteps the publication hazards that shared-memory models have, because state never crosses a boundary while mutable. Cost, and this is the part candidates miss: **circular waits survive**. If actor A sends a request to B and blocks awaiting the reply while B does the same toward A, they are deadlocked exactly as two mutexes are. Message-passing systems stay safe by making handlers non-blocking (send and continue, handle the reply as another message), by giving the call graph a direction so requests only flow one way, and by putting timeouts on every await. Additional costs are latency per hop, mailbox growth under overload, and the fact that a single owner is a throughput ceiling for its state. ## Option 5: lock-free and optimistic techniques Update shared state with atomic compare-and-swap: read the current value, compute the new one, swap only if it has not changed, retry if it has. With no mutual exclusion there is no deadlock, and a lock-free algorithm guarantees that *some* participant always makes progress. Cost: individual threads can retry indefinitely under heavy contention (livelock or starvation for the unlucky), correctness is genuinely difficult for anything beyond a single word, and the same idea at coarse grain - optimistic versioning with a version check and a retry, as databases do - trades wasted work under contention for freedom from blocking. Use published, tested primitives rather than hand-rolling structures. ## Choosing Ask three questions. **Where is the contention actually?** Measure before paying for anything more complex than a single lock. **What does the operation genuinely need to be atomic over?** Multi-lock designs usually come from an atomicity requirement that was never stated; write it down and often it shrinks. **Who else can violate the invariant?** A rule that depends on every future caller remembering the acquisition order is weaker than a structure where the second lock does not exist. The strongest answer is usually staged: collapse to one lock first to make it correct and simple, measure, then partition or move to single-ownership only where the numbers justify it - and keep the component's locks entirely private, never held across a call to another subsystem, so the guarantee is local and cannot be broken from outside.
- Your component now uses a single lock, yet it still deadlocks against another subsystem. What went wrong?The single lock removed cycles inside the component but not across components: some path holds your lock while calling into the other subsystem, which takes its own lock, and a reverse path exists elsewhere. The guarantee must be made local - your lock is never held across a call that leaves the component, and callers never invoke you while holding a lock you might need to call back into. If cross-component atomicity is genuinely required, define one ordering across the two subsystems and enforce it with lock levels.
- Does moving to an actor or channel-based design make deadlock impossible?No. It removes data races and lock-based deadlock, since state is owned by a single participant and never shared mutably, but a blocking wait for a reply is still hold-and-wait on a non-preemptible resource. Two actors that each await the other's response are deadlocked in exactly the same shape as two mutexes taken in opposite orders. Safety comes from non-blocking handlers, a directed request graph, and timeouts on every await.
- How do you decide between a single coarse lock and a partitioned design?By measurement, not by intuition. Look at lock wait time, hold time and arrival rate: if the coarse lock's queueing delay is invisible against end-to-end latency, keep it, because simplicity has real ongoing value. Partition when contention is demonstrably the bottleneck and the workload has a key that spreads evenly and keeps cross-shard operations rare; otherwise sharding buys you skew, cross-shard complexity, and the multi-lock problem back again.
saying these in an interview costs you the question
- Reaching for lock-free structures first, when a single coarse lock would be correct, simple and fast enough.
- Assuming actors or channels make deadlock impossible, ignoring blocking request-response cycles between them.
- Partitioning without a plan for cross-shard operations, which reintroduces two co-held locks.
- Treating fine-grained locking as automatically faster, without measuring contention or accounting for the added acquisition overhead.
- Believing a single component-wide lock guarantees safety even when it is held across calls into other locked subsystems.