skip to content

Compare the main inter-process communication mechanisms — pipes, shared memory, sockets and signals — and explain how you would choose among them for moving a high volume of data between two processes on the same machine.

level: seniorimportance: should knowfreq 45%

answer

  1. copies: pipe/socket = 2, shared memory = 0
  2. pipes give backpressure for free
  3. sockets buy location transparency
  4. signals = notification, no payload
  5. shared memory needs its own sync + wake-up channel

basics

~20 s

Pipes are simple one-way byte streams via kernel buffers. Sockets are the same idea but routable across machines. Shared memory is the only zero-copy option — mapped by both sides, needing its own synchronization. Signals carry notification, not data. For high volume on one host, use shared memory with a ring buffer, or pipes if simplicity wins.

solid answer

~60 s

Think of them along two axes: **how much copying** and **what they carry**. - **Pipes**: a one-way byte stream through a kernel buffer. Writer blocks when full, reader blocks when empty — natural flow control. Every byte is copied user→kernel→user. Simple, stream-shaped, local, no message boundaries. - **Sockets**: the same copy path plus a protocol layer, but they route across hosts and can preserve message boundaries in datagram form. Local variants drop the network stack while keeping the API, so you can move a component to another machine without rewriting the transport. - **Shared memory**: both processes map the same physical pages. Zero copies, the fastest option by a wide margin — and the only one that reintroduces classic shared-state hazards, so it needs its own synchronization and, usually, a separate channel to signal readiness. - **Signals**: asynchronous notifications with essentially no payload. Use them for control ("reload", "terminate"), never for data. For bulk local transfer, I'd start with a pipe or local socket for simplicity, measure, and move to a shared-memory ring buffer only when copy cost actually dominates.

code

text · 9 lines
text
producer:                        consumer:
  while free_slots() == 0:         while used_slots() == 0:
      wait_on(notify_pipe)             wait_on(notify_pipe)
  slot = ring[write_idx]           slot = ring[read_idx]
  fill(slot, data)                 process(slot)
  publish(write_idx + 1)           publish(read_idx + 1)
  signal(notify_pipe, 1 byte)      signal(notify_pipe, 1 byte)

# ring holds the bytes (zero copy); the pipe carries only 'something changed'

go deeper

for a junior

Name the four mechanisms and their one-line character: pipes and sockets stream copied bytes, shared memory avoids copying, signals only notify.

for a middle

Explain the copy path and blocking behaviour, why pipes give backpressure, and why shared memory needs external synchronization.

for a senior

Drive the choice from measured data rate, data shape, failure semantics and flow control, and describe a shared-memory ring with a separate wake-up channel.

for a principal

Frame it as an isolation-versus-throughput decision, including future topology (co-located or not), crash recovery of shared regions, and keeping a socket contract with a local fast path behind it.

## Why IPC exists at all Processes have isolated address spaces, so the cheap sharing threads enjoy is unavailable by construction. Every mechanism below is a way to punch a controlled hole through that isolation, and each one trades a different amount of the isolation away. ## Pipes A pipe is a kernel-managed byte buffer with a write end and a read end. Writers append, readers consume in order. When the buffer fills, the writer blocks; when it empties, the reader blocks. That blocking is free **backpressure**: a fast producer is automatically throttled by a slow consumer. Properties worth naming: unidirectional (two are needed for a dialogue), no message boundaries (the reader may get a partial or a merged write, so you must frame data yourself with lengths or delimiters), and typically inherited across process creation, which is what makes shell pipelines work. Each byte crosses the user/kernel boundary twice. ## Sockets Sockets generalize the pipe idea into a named, connection-oriented (or datagram) channel. Two important variants: - **Local/domain sockets** stay inside one machine, skipping checksums, routing and the network stack, but keep the socket programming model. - **Network sockets** carry the same API across hosts. Compared to pipes, sockets are bidirectional, addressable by name rather than by inheritance, and datagram forms preserve message boundaries. The strategic advantage is **location transparency**: a design that already talks over sockets can be split across machines without changing its communication model. The cost is protocol overhead and still one copy in each direction. ## Shared memory Both processes map the same physical pages into their address spaces. After setup, a write by one is visible to the other with **no kernel involvement and no copy** — the only mechanism that reaches memory bandwidth rather than syscall-plus-copy speed. Everything that makes it fast makes it dangerous: - There is no framing, no flow control and no ordering guarantee. You build a protocol yourself, typically a ring buffer with producer and consumer indices. - Both sides now share mutable state, so you need cross-process synchronization (shared mutexes/semaphores, or lock-free index manipulation with the appropriate memory ordering). - A crash mid-update leaves the shared region in a state the survivor must be able to recognize and recover from — a corrupted shared region can take down both parties, so the isolation benefit is partly given back. - Notification usually needs a companion channel: shared memory tells you *what*, a pipe, socket or semaphore tells you *when*, so the reader does not have to spin. ## Signals A signal is an asynchronous notification delivered to a process: a small identifier and nothing meaningful beyond it. Handlers run at arbitrary points in the target's execution, which severely restricts what they may safely do — the standard discipline is to record a flag or write one byte to a pipe and let the normal loop do the real work. Signals are for **control-plane events**: terminate, reload configuration, child exited, resize. Using them to carry data is a design error; they can coalesce, they carry no payload, and their delivery timing is not yours to control. ## Other mechanisms worth naming Message queues give named, boundary-preserving, kernel-buffered messages with optional priority; memory-mapped files behave like shared memory with a durable backing store and are a common way to share large read-mostly datasets; and the family of event/notification objects exists precisely to pair with shared memory for the wake-up problem. ## Choosing for a high-volume local transfer Walk the decision explicitly: 1. **What is the data rate, really?** Copying is fast. If you are moving tens of megabytes a second, a pipe or local socket will not be your bottleneck and simplicity wins. 2. **What shape is the data?** Streaming and framed messages fit pipes and sockets naturally. Large fixed-size buffers — video frames, captured pages, tensors — fit shared memory, because the copy is exactly the cost you are trying to remove. 3. **Do you need flow control?** Pipes and stream sockets give it for free. A shared-memory ring buffer requires you to design it, and to decide what a full buffer means: block, drop, or overwrite. 4. **What happens when one side dies?** Pipes and sockets deliver a clean end-of-stream or error. Shared memory can leave a half-written structure, so you need versioned or index-based protocols that a survivor can validate. 5. **Might these processes end up on different machines?** If yes, choose sockets now; that is the only option that survives the move unchanged. A defensible senior answer is a staged one: start with a local socket or pipe, instrument, and adopt a shared-memory ring — with a small pipe or semaphore as its wake-up channel — only when profiling shows copy and syscall cost dominating. Adding shared memory converts a copy problem into a concurrency problem, and that trade should be paid for with evidence.

  • Shared memory is the fastest option, so why is it not the default choice?
    Because it removes the very isolation that made processes attractive: both sides can corrupt the shared region, so you need cross-process synchronization, your own framing and flow control, and a recovery story for a peer that dies mid-update. Pipes and sockets give ordering, backpressure and clean end-of-stream signalling for free. Shared memory is worth it only when profiling shows copy cost dominating.
  • Why should a signal handler generally do almost nothing?
    A handler runs asynchronously at an arbitrary point in the process, possibly while the interrupted code holds a lock or is mid-update in the allocator. Doing real work there risks deadlock or corruption. The standard pattern is to set a flag or write a single byte to a pipe, then let the main loop react at a safe point.
  • Your two processes may later be split across machines. How does that change the choice?
    It argues strongly for sockets from the start, since they are the only mechanism whose model survives the move. Shared memory and pipes are host-local by nature, so adopting them bakes in co-location. If you still need local zero-copy performance, keep the socket interface as the contract and treat shared memory as an optional local fast path behind it.

Pipes are a conveyor belt through a wall — everything is handed over item by item. Shared memory is a shared warehouse with a connecting door: nothing needs carrying, but now both teams can trip over each other's forklifts.

saying these in an interview costs you the question

  • Treating signals as a way to send data between processes.
  • Assuming a pipe preserves message boundaries, so one write always equals one read.
  • Reaching for shared memory before measuring, and ignoring that it needs its own synchronization.
  • Claiming shared memory is 'just faster' without mentioning the loss of flow control, framing and crash isolation.
  • Forgetting that shared memory usually needs a separate notification channel to avoid busy-waiting.

context