Rust Fibers

A lightweight, stackful coroutine implementation in Rust, similar in concept to Go's goroutines, but running on a single thread with cooperative multitasking.

What are Fibers?

Fibers are lightweight threads of execution that use cooperative multitasking instead of preemptive multitasking. Unlike OS threads, fibers:

• Run on a single thread
• Manually yield control to other fibers
• Have their own stack
• Can be suspended and resumed

                 ┌────────────────────┐
                 │   Operating System │
                 └──────────┬─────────┘
                            │
                            │ Single Thread
                            ▼
┌───────────────────────────────────────────────────┐
│                      Scheduler                     │
├───────────┬───────────┬───────────┬───────────────┤
│           │           │           │               │
▼           ▼           ▼           ▼               ▼
┌─────────┐ ┌─────────┐ ┌─────────┐ ┌─────────┐     ┌─────────┐
│ Fiber 1 │ │ Fiber 2 │ │ Fiber 3 │ │ Fiber 4 │ ... │ Fiber N │
└─────────┘ └─────────┘ └─────────┘ └─────────┘     └─────────┘
   2MB         2MB         2MB         2MB             2MB
   Stack       Stack       Stack       Stack           Stack

Key Features

• Stackful Coroutines: Each fiber has its own 2MB stack, allowing it to be suspended and resumed from any point in its execution, even within nested function calls.
• Cooperative Scheduling: Fibers explicitly yield control using yield_now(), allowing other fibers to execute.
• Single-Threaded: All fibers run on a single thread, taking turns to execute.
• Rust Safe: Uses Rust's type system to ensure memory safety, even when using unsafe code for stack manipulation.

Architecture

The fiber system consists of these main components:

┌───────────────────┐
│     Scheduler     │◄───┐
├───────────────────┤    │
│ - Ready Queue     │    │
│ - Fibers List     │    │
└────────┬──────────┘    │
         │               │
         │ schedules     │ yields
         ▼               │
┌───────────────────┐    │
│       Fiber       │    │
├───────────────────┤    │
│ - Stack           │────┘
│ - Callback        │
│ - State           │
└───────────────────┘

Execution Flow

┌─────────┐     ┌─────────┐     ┌─────────┐
│  Fiber  │     │ Sched-  │     │  Fiber  │
│    A    │     │  uler   │     │    B    │
└────┬────┘     └────┬────┘     └────┬────┘
     │               │               │
     │ Running       │               │ Ready
     ├───────────────┤               │
     │               │               │
     │ yield_now()   │               │
     ├──────────────►│               │
     │               │ schedule      │
     │               ├──────────────►│
     │ Suspended     │               │ Running
     │               │               │
     │               │               │
     │               │ complete/yield│
     │               │◄──────────────┤
     │               │               │
     │ schedule      │               │
     │◄──────────────┤               │
     │               │               │
     │ Running       │               │ Suspended/Completed
     │               │               │
     ▼               ▼               ▼

How to Use

Here's how to use the fiber system:

// Spawn a single fiber
spawn(|| {
    println!("Fiber running");
    
    // Yield to other fibers
    yield_now();
    
    println!("Fiber resumed after yielding");
});

// Or create a scheduler and spawn multiple fibers
let mut scheduler = Scheduler::new();

scheduler.spawn(|| {
    println!("Fiber 1 running");
    yield_now();
    println!("Fiber 1 resumed");
});

scheduler.spawn(|| {
    println!("Fiber 2 running");
    yield_now();
    println!("Fiber 2 resumed");
});

// Run all fibers
run_scheduler(scheduler);

Implementation Details

1. Stack Allocation: Each fiber has a dedicated stack of 2MB, allocated using Rust's alloc and dealloc functions.

2. Fiber States:

   • Ready: Initialized but not yet started
   • Running: Currently executing
   • Completed: Finished execution

3. Callbacks: Each fiber wraps a function, which is executed when the fiber is scheduled.

4. Yielding: When a fiber calls yield_now(), it sets a flag that the scheduler checks to determine if it should move the fiber back to the ready queue.

Memory Management

Each fiber allocates its own stack:

High Memory
        ▲
        │
┌───────┴───────┐
│   Stack Top   │
├───────────────┤
│               │
│   Free Stack  │
│     Space     │
│   (2MB max)   │
│               │
├───────────────┤
│  Stack Bottom │
└───────────────┘
        │
        ▼
Low Memory

Limitations

• Single-Threaded: All fibers run on a single thread, so they can't leverage multiple CPU cores.
• Fixed Stack Size: Each fiber allocates a fixed 2MB stack, which might be wasteful for simple fibers.
• Manual Yielding: Fibers must explicitly yield; they don't automatically yield during blocking operations.
• No Built-in Communication: Unlike Go's channels, there's no built-in mechanism for fibers to communicate.

Future Improvements

• Add a mechanism for fibers to communicate (channels)
• Implement automatic yielding during blocking operations
• Add support for dynamic stack sizes
• Explore work-stealing scheduling for better performance
• Support for running fibers across multiple threads

License

This project is available under the MIT License.