rust-async-internals

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Rust Async Internals

Rust Async 内部机制

Purpose

用途

Guide agents through Rust async/await internals: the
Future
trait and poll loop,
Pin
/
Unpin
for self-referential types, tokio's task model, diagnosing async stack traces with tokio-console, finding waker leaks, and common
select!
/
join!
pitfalls.
引导开发者了解Rust async/await的内部机制:包括
Future
trait与轮询循环、用于自引用类型的
Pin
/
Unpin
、tokio的任务模型、使用tokio-console诊断异步堆栈跟踪、查找waker泄漏,以及
select!
/
join!
宏的常见陷阱。

Triggers

触发场景

  • "How does async/await actually work in Rust?"
  • "What is Pin and Unpin in async Rust?"
  • "My async code is slow — how do I profile it?"
  • "How do I use tokio-console to debug async tasks?"
  • "I have a blocking call in async — what do I do?"
  • "How does select! work and what are the pitfalls?"
  • "Rust中的async/await实际是如何工作的?"
  • "Rust异步编程中的Pin和Unpin是什么?"
  • "我的异步代码运行缓慢——该如何分析性能?"
  • "如何使用tokio-console调试异步任务?"
  • "我的异步代码中有阻塞调用——该怎么处理?"
  • "select!的工作原理是什么,有哪些陷阱?"

Workflow

工作流程

1. The Future trait — poll model

1. Future trait — 轮询模型

rust
// std::future::Future (simplified)
pub trait Future {
    type Output;
    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}

pub enum Poll<T> {
    Ready(T),    // computation done, T is the result
    Pending,     // not ready yet, waker registered, will be polled again
}
Execution model:
  1. Calling
    .await
    calls
    poll()
    on the future
  2. If
    Pending
    : current task registers its waker and yields to the runtime
  3. When the waker is triggered (I/O ready, timer fired), the runtime re-polls
  4. If
    Ready(val)
    : the
    .await
    expression evaluates to
    val
rust
// std::future::Future (simplified)
pub trait Future {
    type Output;
    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}

pub enum Poll<T> {
    Ready(T),    // computation done, T is the result
    Pending,     // not ready yet, waker registered, will be polled again
}
执行模型:
  1. 调用
    .await
    会对future执行
    poll()
    方法
  2. 如果返回
    Pending
    :当前任务会注册其waker并让出给运行时
  3. 当waker被触发(I/O就绪、定时器到期),运行时会重新轮询该future
  4. 如果返回
    Ready(val)
    .await
    表达式的结果即为
    val

2. Implementing a simple Future

2. 实现一个简单的Future

rust
use std::{
    future::Future,
    pin::Pin,
    task::{Context, Poll},
    time::{Duration, Instant},
};

struct Delay { deadline: Instant }

impl Delay {
    fn new(dur: Duration) -> Self {
        Delay { deadline: Instant::now() + dur }
    }
}

impl Future for Delay {
    type Output = ();

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {
        if Instant::now() >= self.deadline {
            Poll::Ready(())
        } else {
            // Register the waker — runtime calls waker.wake() to re-poll
            // In production: register with I/O reactor or timer wheel
            let waker = cx.waker().clone();
            let deadline = self.deadline;
            std::thread::spawn(move || {
                let now = Instant::now();
                if deadline > now {
                    std::thread::sleep(deadline - now);
                }
                waker.wake();  // notify runtime to re-poll
            });
            Poll::Pending
        }
    }
}

// Usage
async fn main() {
    Delay::new(Duration::from_secs(1)).await;
    println!("Done");
}
rust
use std::{
    future::Future,
    pin::Pin,
    task::{Context, Poll},
    time::{Duration, Instant},
};

struct Delay { deadline: Instant }

impl Delay {
    fn new(dur: Duration) -> Self {
        Delay { deadline: Instant::now() + dur }
    }
}

impl Future for Delay {
    type Output = ();

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {
        if Instant::now() >= self.deadline {
            Poll::Ready(())
        } else {
            // Register the waker — runtime calls waker.wake() to re-poll
            // In production: register with I/O reactor or timer wheel
            let waker = cx.waker().clone();
            let deadline = self.deadline;
            std::thread::spawn(move || {
                let now = Instant::now();
                if deadline > now {
                    std::thread::sleep(deadline - now);
                }
                waker.wake();  // notify runtime to re-poll
            });
            Poll::Pending
        }
    }
}

// Usage
async fn main() {
    Delay::new(Duration::from_secs(1)).await;
    println!("Done");
}

3. Pin and Unpin

3. Pin和Unpin

Pin<P>
prevents moving the value behind pointer
P
. This matters because async state machines contain self-referential pointers (a reference into the same struct where the future lives):
rust
// Why Pin is needed: async fn compiles to a state machine struct
// that may have self-references across await points

async fn example() {
    let data = vec![1, 2, 3];
    let ref_to_data = &data;              // reference into same stack frame
    some_async_op().await;                // suspension point
    println!("{:?}", ref_to_data);       // reference still used after suspend
}
// The state machine stores both `data` and `ref_to_data`.
// If the struct were moved, `ref_to_data` would dangle.
// Pin<&mut State> prevents moving the state machine.

// Unpin: a marker trait for types that are safe to move even when pinned
// Most types implement Unpin automatically
// Futures generated by async/await do NOT implement Unpin

// Creating a Pin from Box (heap allocation → safe)
let boxed: Pin<Box<dyn Future<Output = ()>>> = Box::pin(my_future);

// Pinning to stack (unsafe, use pin! macro)
use std::pin::pin;
let fut = pin!(my_future);
fut.await;   // or poll it directly
Pin<P>
用于防止指针
P
指向的值被移动。这一点很重要,因为异步状态机包含自引用指针(指向future所在结构体内部的引用):
rust
// Why Pin is needed: async fn compiles to a state machine struct
// that may have self-references across await points

async fn example() {
    let data = vec![1, 2, 3];
    let ref_to_data = &data;              // reference into same stack frame
    some_async_op().await;                // suspension point
    println!("{:?}", ref_to_data);       // reference still used after suspend
}
// The state machine stores both `data` and `ref_to_data`.
// If the struct were moved, `ref_to_data` would dangle.
// Pin<&mut State> prevents moving the state machine.

// Unpin: a marker trait for types that are safe to move even when pinned
// Most types implement Unpin automatically
// Futures generated by async/await do NOT implement Unpin

// Creating a Pin from Box (heap allocation → safe)
let boxed: Pin<Box<dyn Future<Output = ()>>> = Box::pin(my_future);

// Pinning to stack (unsafe, use pin! macro)
use std::pin::pin;
let fut = pin!(my_future);
fut.await;   // or poll it directly

4. tokio task model

4. Tokio任务模型

rust
use tokio::task;

// Spawn a task (runs concurrently on the runtime thread pool)
let handle = tokio::spawn(async {
    // ... async work ...
    42
});
let result = handle.await.unwrap();  // wait for completion

// spawn_blocking — for CPU-bound or blocking I/O
let result = task::spawn_blocking(|| {
    // runs on a dedicated blocking thread pool
    std::fs::read_to_string("big_file.txt")
}).await.unwrap();

// yield to runtime (cooperative multitasking)
tokio::task::yield_now().await;

// LocalSet — for !Send futures (single-threaded)
let local = task::LocalSet::new();
local.run_until(async {
    task::spawn_local(async { /* !Send future */ }).await.unwrap();
}).await;
rust
use tokio::task;

// Spawn a task (runs concurrently on the runtime thread pool)
let handle = tokio::spawn(async {
    // ... async work ...
    42
});
let result = handle.await.unwrap();  // wait for completion

// spawn_blocking — for CPU-bound or blocking I/O
let result = task::spawn_blocking(|| {
    // runs on a dedicated blocking thread pool
    std::fs::read_to_string("big_file.txt")
}).await.unwrap();

// yield to runtime (cooperative multitasking)
tokio::task::yield_now().await;

// LocalSet — for !Send futures (single-threaded)
let local = task::LocalSet::new();
local.run_until(async {
    task::spawn_local(async { /* !Send future */ }).await.unwrap();
}).await;

5. tokio-console — async task inspector

5. tokio-console — 异步任务检查器

toml
undefined
toml
undefined

Cargo.toml

Cargo.toml

[dependencies] console-subscriber = "0.3" tokio = { version = "1", features = ["full", "tracing"] }

```rust
// main.rs
fn main() {
    console_subscriber::init();  // must be called before tokio runtime
    tokio::runtime::Builder::new_multi_thread()
        .enable_all()
        .build()
        .unwrap()
        .block_on(async_main());
}
bash
undefined
[dependencies] console-subscriber = "0.3" tokio = { version = "1", features = ["full", "tracing"] }

```rust
// main.rs
fn main() {
    console_subscriber::init();  // must be called before tokio runtime
    tokio::runtime::Builder::new_multi_thread()
        .enable_all()
        .build()
        .unwrap()
        .block_on(async_main());
}
bash
undefined

Install tokio-console CLI

Install tokio-console CLI

cargo install --locked tokio-console
cargo install --locked tokio-console

Run your app with tracing enabled

Run your app with tracing enabled

RUSTFLAGS="--cfg tokio_unstable" cargo run
RUSTFLAGS="--cfg tokio_unstable" cargo run

In another terminal, connect tokio-console

In another terminal, connect tokio-console

tokio-console
tokio-console

tokio-console shows:

tokio-console shows:

- Running tasks with their names, poll times, and wakeup counts

- Running tasks with their names, poll times, and wakeup counts

- Slow tasks (high poll duration = blocking in async!)

- Slow tasks (high poll duration = blocking in async!)

- Tasks that have been pending for a long time (stuck?)

- Tasks that have been pending for a long time (stuck?)

- Resource contention (mutex/semaphore wait times)

- Resource contention (mutex/semaphore wait times)

undefined
undefined

6. Blocking in async — common mistake

6. 异步上下文阻塞——常见错误

rust
// WRONG: blocking call in async context blocks entire thread
async fn bad() {
    std::thread::sleep(Duration::from_secs(1));  // blocks runtime thread!
    std::fs::read_to_string("file.txt").unwrap(); // blocking I/O blocks runtime!
}

// CORRECT: use async equivalents
async fn good() {
    tokio::time::sleep(Duration::from_secs(1)).await;   // async sleep
    tokio::fs::read_to_string("file.txt").await.unwrap(); // async I/O
}

// CORRECT: if you must block, use spawn_blocking
async fn with_blocking() {
    let content = tokio::task::spawn_blocking(|| {
        heavy_cpu_computation()   // runs on blocking thread pool
    }).await.unwrap();
}
rust
// WRONG: blocking call in async context blocks entire thread
async fn bad() {
    std::thread::sleep(Duration::from_secs(1));  // blocks runtime thread!
    std::fs::read_to_string("file.txt").unwrap(); // blocking I/O blocks runtime!
}

// CORRECT: use async equivalents
async fn good() {
    tokio::time::sleep(Duration::from_secs(1)).await;   // async sleep
    tokio::fs::read_to_string("file.txt").await.unwrap(); // async I/O
}

// CORRECT: if you must block, use spawn_blocking
async fn with_blocking() {
    let content = tokio::task::spawn_blocking(|| {
        heavy_cpu_computation()   // runs on blocking thread pool
    }).await.unwrap();
}

7. select! and join! pitfalls

7. select!和join!的陷阱

rust
use tokio::select;

// select! — complete when FIRST branch completes, cancels others
select! {
    result = fetch_a() => println!("A: {:?}", result),
    result = fetch_b() => println!("B: {:?}", result),
    // Pitfall: the LOSING branches are DROPPED immediately
    // If fetch_a wins, fetch_b's future is dropped (and its state machine cleaned up)
    // This is correct and safe — but can be surprising
}

// join! — wait for ALL to complete
let (a, b) = tokio::join!(fetch_a(), fetch_b());

// Biased select (always check first branch first)
loop {
    select! {
        biased;                        // prevents fairness, checks in order
        _ = shutdown_signal.recv() => break,
        msg = queue.recv() => process(msg),
    }
}

// select! with values from loop (use fuse)
let mut fut = some_future().fuse();   // FusedFuture: safe to poll after completion
loop {
    select! {
        val = &mut fut => { /* ... */ break; }
        _ = interval.tick() => { /* periodic work */ }
    }
}
rust
use tokio::select;

// select! — complete when FIRST branch completes, cancels others
select! {
    result = fetch_a() => println!("A: {:?}", result),
    result = fetch_b() => println!("B: {:?}", result),
    // Pitfall: the LOSING branches are DROPPED immediately
    // If fetch_a wins, fetch_b's future is dropped (and its state machine cleaned up)
    // This is correct and safe — but can be surprising
}

// join! — wait for ALL to complete
let (a, b) = tokio::join!(fetch_a(), fetch_b());

// Biased select (always check first branch first)
loop {
    select! {
        biased;                        // prevents fairness, checks in order
        _ = shutdown_signal.recv() => break,
        msg = queue.recv() => process(msg),
    }
}

// select! with values from loop (use fuse)
let mut fut = some_future().fuse();   // FusedFuture: safe to poll after completion
loop {
    select! {
        val = &mut fut => { /* ... */ break; }
        _ = interval.tick() => { /* periodic work */ }
    }
}

Related skills

相关技能

  • Use
    skills/rust/rust-debugging
    for GDB/LLDB debugging of async Rust programs
  • Use
    skills/rust/rust-profiling
    for cargo-flamegraph with async stack frames
  • Use
    skills/low-level-programming/cpp-coroutines
    for C++20 coroutine comparison
  • Use
    skills/low-level-programming/memory-model
    for memory ordering in async contexts
  • 使用
    skills/rust/rust-debugging
    对Rust异步程序进行GDB/LLDB调试
  • 使用
    skills/rust/rust-profiling
    结合cargo-flamegraph分析异步堆栈帧
  • 使用
    skills/low-level-programming/cpp-coroutines
    对比C++20协程
  • 使用
    skills/low-level-programming/memory-model
    了解异步上下文中的内存顺序