skip to content

Pin and Unpin

Pin promises a value will not move, which self-referential futures need once polled. Interviewers ask what Unpin means and when you have to pin a future yourself.

on this pageshow

questions

5

In Rust, why can an `async fn` future that borrows its own local buffer across an `.await` not be moved once polled, and how does `Pin` stop that?

level: seniorimportance: must knowfreq 55%

answer

  1. locals live inside the future
  2. a borrow across .await
  3. pointer into its own state
  4. poll takes Pin<&mut Self>

basics

~20 s

Locals alive across an .await live inside the future, so a borrow of a local array points into the future itself. Future::poll takes Pin<&mut Self>, and safe code gets no &mut to a pinned !Unpin future to move it.

solid answer

~50 s

rustc compiles an `async fn` into an anonymous state machine whose fields are the locals that are alive at each `.await`. In a parser that does `let buf = [0u8; 64];`, fills it, takes `let header = &buf[..2];` and then awaits, both the array and `header` live in the future, so `header` points into the future's own memory. Moving the future after it has been polled would leave `header` pointing at the old address. Rust prevents this through the `Future::poll` signature, `self: Pin<&mut Self>`: whoever polls must first pin the future with `Box::pin`, `pin!` or `.await`, and a `Pin` to a `!Unpin` value offers no safe `&mut` (no `DerefMut`, no `get_mut`), so `mem::swap` or `mem::replace` cannot move it. Before the first poll no such borrow exists yet, which is why a fresh future can be moved freely, for example into an executor.

code

rust · 21 lines
rust
use futures::executor::block_on;
use std::mem;

async fn parse_frame(input: &[u8]) -> usize {
    let mut buf = [0u8; 64]; // stored inline in the future's state machine
    buf[..input.len()].copy_from_slice(input);
    let header = &buf[..2]; // a pointer into that same future
    std::future::ready(()).await; // `header` is held across this point
    header.len() + input.len()
}

fn main() {
    let f = parse_frame(b"HDone");
    let moved = f; // fine: never polled, so no self-reference exists yet
    let mut a = Box::pin(moved);
    let mut b = Box::pin(parse_frame(b"HDtwo!"));
    // Swapping the boxes moves two pointers; both futures stay where they are.
    mem::swap(&mut a, &mut b);
    // mem::swap(&mut *a, &mut *b); // E0596: Pin<Box<F>> gives no &mut F
    println!("{} {}", block_on(a), block_on(b));
}

go deeper

for a junior

Know that an async fn returns a state machine and that locals used after an .await are stored inside that future value.

for a middle

Explain that a borrow of such a local is a pointer into the future, so moving it after a poll would leave the pointer dangling.

for a senior

Walk through the enforcement: poll's Pin<&mut Self> receiver, no DerefMut or get_mut for !Unpin targets, async futures marked !Unpin, the guarantee lasting until drop, and why pre-poll moves are fine.

for a principal

Frame Pin as the cost of zero-allocation futures that can borrow across suspension points, and weigh it against designs that heap-allocate every coroutine frame.

## What an `async fn` turns into An `async fn` does not run when called. It returns a value of an anonymous type that implements `Future`: a **state machine** that rustc generates from the function body. Each `.await` is a point where the function may suspend, and every local variable that is still needed after that point has to survive the suspension. The only place it can survive is inside the future value itself, so the generated type holds those locals as fields, roughly one enum variant per suspension point. That is harmless for owned data. It becomes delicate when one stored local **borrows** another: ```rust async fn parse_frame(input: &[u8]) -> usize { let mut buf = [0u8; 64]; // stored inline in the future buf[..input.len()].copy_from_slice(input); let header = &buf[..2]; // a pointer into the future read_more().await; // both must survive this point header.len() + input.len() } ``` While suspended at `read_more().await`, the future contains the 64-byte array and a reference to its first two bytes, both inside the same value. The future now points into itself: it is **self-referential**. (Had `buf` been a `Vec<u8>`, `header` would point at the heap bytes rather than into the future, but rustc does not track that difference and treats every async future the same way.) The `pin` module documentation names exactly this case: the state machines generated for `async fn` are a key example of self-referential types. ## Why moving would be wrong A move in Rust is a bitwise copy to a new address, after which the old location is treated as dead. If a self-referential future were moved after it had been polled to its first `.await`, its stored reference would still hold the old address and point at freed or reused memory the next time it ran. Safe Rust has no way to fix up such pointers during a move, so the only sound rule is: **once polled, the future must stay where it is until it is dropped.** ## How `Pin` enforces it Rust enforces that rule through the type system rather than a runtime check: 1. **`Future::poll` takes `self: Pin<&mut Self>`.** Nobody can poll a future without holding a pinning pointer to it, so a future is always pinned before the first poll creates any self-reference. 2. **A pinning pointer hides `&mut` for `!Unpin` targets.** `Pin<Ptr>` implements `DerefMut` and `get_mut` only when the target is `Unpin`. Without a `&mut F`, safe code cannot call `mem::swap`, `mem::replace` or move out. Trying `mem::swap(&mut *a, &mut *b)` on two `Pin<Box<F>>` values fails with E0596, "cannot borrow data in dereference of `Pin<Box<impl Future<Output = usize>>>` as mutable". 3. **The compiler marks async futures `!Unpin`.** So the restriction in point 2 applies to them, even to `async { 1 }`, which holds no borrow. 4. **The pinning guarantee lasts until drop.** The `pin` docs promise that a pinned value stays at the same place in memory from the moment it is pinned until its `drop` runs. The documentation stresses that pinning is a **library contract**, not compiler magic: `Pin` does not change how the compiler treats the value, it only withholds the safe APIs that could move it. Getting around it needs `unsafe`, and the lesson on unsafe code covers what such code must then prove. ## What you can still do | Action | Allowed? | Why | |---|---|---| | Move a future before its first poll | Yes | no self-reference exists yet | | Move a `Pin<Box<F>>` or swap two of them | Yes | only the pointer moves; the future stays on the heap | | Get `&mut F` from `Pin<&mut F>` when `F: !Unpin` | Not safely | `get_mut` and `DerefMut` need `Unpin` | | Read through `&F` | Yes | `Pin` implements `Deref` for every pointer | This is why handing a fresh future to an executor, or returning it from a function, is fine: it moves before anyone polls it, and the executor then pins it in its own storage (typically a heap allocation) for the rest of its life. ## Why Rust needs this at all Some languages allocate every coroutine frame on the heap and move only a handle; Rust's futures are plain values that can live on the stack, inside other futures, or in a `Vec` without an allocation each. `Pin` is the price of that design: it lets borrows cross `.await` points, which the borrow checker would otherwise have to forbid, while keeping futures movable until they start. - **Self-referential structs written by hand** outside async code are a lifetimes question and are usually redesigned around owned data or indices. - **Writing a `!Unpin` type yourself** means adding a `PhantomPinned` field and reaching the data through `unsafe`, which is the unsafe branch's material.

  • If async futures may become self-referential, why can you pass an un-awaited future to a function or an executor by value?
    Before the first poll the state machine is in its initial state, holding only the captured arguments, so nothing points into it yet. Moving it is an ordinary move. `poll` requires `Pin<&mut Self>`, so by the time any self-reference can be created the future has been pinned and can no longer move safely.
  • Does `Pin` make the compiler generate different code for the pinned future?
    No. The `pin` module documents pinning as a contract between unsafe code and its users, not compiler magic: the value is laid out and dropped exactly as before. `Pin` only removes the safe ways to get `&mut` to a `!Unpin` pointee, so safe code cannot move it.
  • Why can two `Pin<Box<F>>` values be swapped when the futures inside cannot?
    Swapping the boxes exchanges two heap pointers; each future stays at its heap address. `Pin<Box<F>>` is `Unpin` for every `F`, so moving the pointer is fine, while swapping the pointees would need `&mut F`, which `Pin` withholds.

A printed report whose index cites page numbers: before the index exists you can reorder pages freely, but once it is printed, reordering makes every entry point at the wrong page. Pin is the rule that nobody reorders pages after the index is written.

  • Says Pin copies the future to a fixed heap location automatically.
  • Thinks a future cannot be moved even before it is first polled.
  • Claims async futures are self-referential only when they contain unsafe code.
  • Believes Pin changes how rustc lays out or drops the value.
  • Says the borrow checker rejects any borrow held across an .await.
open as a page

In Rust, what is the difference between pinning a future with `std::pin::pin!` and with `Box::pin`, and when do you use each?

level: juniorimportance: should knowfreq 40%

basics

~20 s

pin! (stable since Rust 1.68) pins a future in the current scope's local storage and returns Pin<&mut F>, with no allocation but no escape. Box::pin moves it to the heap and returns an owned Pin<Box<F>> you can return or store.

open as a page

In Rust, why do `futures::future::select(fut, tick)` and `(&mut fut).await` on an `async fn` future fail with E0277 "cannot be unpinned", and what fixes it?

level: middleimportance: should knowfreq 45%

basics

~10 s

futures::future::select and impl Future for &mut F require F: Unpin, and rustc never makes an async future Unpin. Pin it first with std::pin::pin! (or Box::pin to keep it) and pass fut.as_mut().

open as a page

In Rust, what does the `Unpin` auto trait mean, which types implement it, and what does `Pin<&mut T>` restrict when `T: Unpin`?

level: middleimportance: should knowfreq 45%

basics

~20 s

Unpin marks a type that does not rely on staying at one address once pinned. Almost every type gets it automatically; async futures and types holding PhantomPinned do not. For T: Unpin, Pin<&mut T> restricts nothing beyond &mut T.

open as a page

In a hand-written Rust future combinator `CountPolls<F>` wrapping an inner future, why does `self.inner.poll(cx)` fail, and how does pin projection fix it?

level: seniorimportance: nice to knowfreq 25%

basics

~20 s

poll needs Pin<&mut F>, but Pin<&mut Self> yields neither that nor &mut self.inner when F may be !Unpin (E0599). Project the pin: pin_project! with #[pin] inner gives Pin<&mut F> for it and &mut for the rest.

open as a page