Jump to content

The Pin API Explained: Why Rust’s Async Needs This Complexity

From JOHNWICK
Revision as of 00:48, 16 November 2025 by PC (talk | contribs) (Created page with "I was two days into debugging a custom Future implementation when the compiler hit me with cannot be unpinned. I stared at that error for a solid thirty minutes. What did "unpinned" even mean? The future worked fine when I .awaited it directly, but the moment I tried storing it in a struct and polling it myself, everything exploded. Turns out I’d been thinking about async Rust completely wrong. The Problem Nobody Told Me About Here’s what broke: I wanted to wrap...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

I was two days into debugging a custom Future implementation when the compiler hit me with cannot be unpinned. I stared at that error for a solid thirty minutes. What did "unpinned" even mean? The future worked fine when I .awaited it directly, but the moment I tried storing it in a struct and polling it myself, everything exploded. Turns out I’d been thinking about async Rust completely wrong. The Problem Nobody Told Me About Here’s what broke: I wanted to wrap futures with timing metrics. Simple idea — take any future, measure how long it takes to resolve, done. But storing a reference to a future while also polling it? Many futures are self-referential, which means they hold pointers to their own data. Move that future to a new memory address, and those internal pointers become dangling references pointing at garbage. Most Rust types don’t care if you move them around. An i32 at address 0x1000 works exactly the same if you copy it to 0x2000. But async functions generate state machines that need stable addresses. Rust futures are state machines that store their progress between .await points, and those state machines often end up referencing their own fields. The entire async ecosystem would be unsound without some way to prevent these moves. What I Thought Pin Did (Spoiler: Wrong) I assumed Pin was some heavyweight locking mechanism. Like it reached into the allocator and froze memory in place. Compiler magic that mortals shouldn't touch. Actually? Pin guarantees that the referent will never be moved, but not through magic. It’s just a type wrapper that restricts what safe code can do. If you have Pin<&mut T>, you can't call mem::replace or mem::swap without going unsafe. That's literally it. Most types don’t even need Pin to mean anything. The vast majority of Rust types are automatically "movable" through the Unpin auto-trait. Only types that explicitly opt out by implementing !Unpin (usually with PhantomPinned) actually get the movement restrictions. The Simplest Pin Example That Actually Works Here’s a basic future that needs pinning: use std::future::Future; use std::pin::Pin; use std::task::{Context, Poll}; use std::time::Instant;

struct Delay {

   when: Instant,

} impl Future for Delay {

   type Output = ();
   
   fn poll(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<()> {
       if Instant::now() >= self.when {
           Poll::Ready(())
       } else {
           Poll::Pending
       }
   }

} Notice poll takes self: Pin<&mut Self> instead of regular &mut self. That signature tells the compiler: "This future might be self-referential, so don't let anyone move it once polling starts." The Delay struct itself doesn't actually need pinning—it has no self-references. But the Future trait's poll method uses a pinned reference to ensure nothing unsafe happens for futures that do need it. When You Actually Hit Pin in Real Code You’ll run into this when trying to poll futures manually. Say you want to call .await on a mutable reference: async fn poll_twice() {

   let fut = async { 42 };
   
   // This works fine
   fut.await;
   
   // But this won't compile
   let mut fut = async { 42 };
   (&mut fut).await;  // Error: future cannot be unpinned

} When awaiting by value, the future is consumed, so there’s no opportunity to move it afterward. But with a reference, you could theoretically move the original after awaiting, which would violate Pin's guarantees if the future were self-referential. The fix is explicit pinning: use tokio::pin;

async fn poll_twice_fixed() {

   let fut = async { 42 };
   pin!(fut);
   
   (&mut fut).await;
   (&mut fut).await;  // Now this works

} I forgot about this pattern constantly when I started writing custom combinators. Spent twenty minutes debugging why tokio::timeout wouldn't accept my future, then realized I'd never pinned it. The Self-Reference Trap Which brings me to why this matters. Look at what async blocks actually compile to: // You write this: async fn example() {

   let data = vec![1, 2, 3];
   let slice = &data[..];
   some_async_call().await;
   println!("{:?}", slice);

}

// The compiler generates something like this state machine: enum ExampleState {

   Start,
   Waiting {
       data: Vec<i32>,
       slice: *const [i32],  // Pointer into data!
   },
   Done,

} Self-referential structs need Pin because moving them would invalidate internal pointers. If that Waiting variant moves to a new address, slice still points to the old location where data used to live. This clicked for me when I ran into a generator-based parser. The parser held a buffer and maintained pointers into that buffer for zero-copy string parsing. Every time I tried restructuring the async code, I’d hit !Unpin errors because the compiler knew those pointers couldn't survive a move. The Unpin Escape Hatch Most futures you write will implement Unpin automatically. If a type implements Unpin, Pin<&mut Self> acts exactly like a regular &mut Self, with direct access to the underlying value. The compiler only restricts movement for types that explicitly opt out. This is why simple async functions just work — they don’t actually become !Unpin unless they create self-references. The compiler figures this out during state machine generation. When implementing Future for a combinator that wraps other futures, you'll typically need structural pinning. When implementing a Future combinator, you usually need structural pinning for nested futures to call poll. The pin-project crate makes this ergonomic by auto-generating the boilerplate projection code. What Actually Confuses Everyone The thing that trips people up: Pin doesn't prevent all movement. Moving Pin<Box<T>> doesn't move the T—the pointer can be freely movable even if the T is not. The Box itself can move around; what can't move is the data it points to. I spent an hour debugging this once. Had a Pin<Box<Future>> that I moved between threads, which was fine. Then I tried extracting the future to manipulate it directly, which wasn't fine. The pin protects the data, not the pointer. Try This Tomorrow Write a tiny custom future that wraps another future and logs when it completes. Start with the example above but add a nested future field. You’ll immediately hit the projection problem and understand why pin-project exists. Or just .await something by reference instead of by value and see the compiler complain. Then fix it with tokio::pin! or Box::pin. That muscle memory matters more than memorizing the theory.