# rust-kernel A tiny hobby x86_64 kernel written in Rust, following the 2nd edition of Philipp Oppermann's [Writing an OS in Rust](https://os.phil-opp.com/). Right now it boots in QEMU, prints to the VGA text screen and the serial port (COM1), handles CPU exceptions (breakpoint, page fault, double fault on its own stack), runs the timer interrupt (uptime clock top-right), echoes what you type on the PS/2 keyboard, walks and edits its own page tables, and has a kernel heap (`Box`, `Vec`, `Rc` work) backed by a fixed-size-block allocator. It multitasks two ways: an **async/await executor** (cooperative tasks; the keyboard is one of them) and **preemptive kernel threads** switched by the timer interrupt (threads A/B/C count on the top line while you type). It has a `cargo test` suite that runs inside QEMU. Most code follows the blog_os sources (MIT / Apache-2.0) closely, with extra comments; additions are the serial mirroring, uptime clock, demo features, scripts, and the preemptive threads (`src/thread.rs`, not in the blog series). ![boot screenshot](boot.png) ## What's installed (on this box) | Tool | Version | Why | |---|---|---| | rustup | 1.29.2 | toolchain manager (in `~/.cargo/bin`, ahead of Debian's `/usr/bin/cargo`) | | Rust nightly | rustc 1.101.0-nightly (2026-10-29) | custom JSON targets + `build-std` are nightly-only | | `rust-src` component | – | source of `core`, recompiled for our bare-metal target | | `llvm-tools-preview` component | – | `llvm-objcopy` etc., used by `bootimage` | | `bootimage` (cargo subcommand) | 0.10.5 | links kernel + bootloader into a bootable disk image | | QEMU (`qemu-system-x86`, `qemu-system-gui`) | 10.0.13 | the emulator we boot in (`-gui` adds the GTK/SDL window) | `rust-toolchain.toml` pins the project to nightly with the two components, so `cargo` inside this directory picks them up automatically. Crates: `bootloader` 0.9.35 (BIOS bootloader used by the 2nd edition), `volatile` 0.2, `spin` 0.5, `x86_64` 0.15, `uart_16550` 0.3, `lazy_static` 1.5, `pic8259` 0.11 (interrupt controller), `pc-keyboard` 0.7 (scancode decoding), `linked_list_allocator` 0.9 (fallback heap allocator). Fresh setup on another machine: ```sh curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh # rustup rustup toolchain install nightly -c rust-src -c llvm-tools-preview cargo install bootimage sudo apt install qemu-system-x86 qemu-system-gui ``` ## Build and run ```sh cargo bootimage # build -> target/x86_64-rust_kernel/debug/bootimage-rust_kernel.bin cargo run # build + boot in a QEMU window; serial output also prints in the terminal ``` **Trying the keyboard:** with `cargo run`, click into the QEMU window and type; characters appear on the bottom line of the screen *and* in your terminal (via serial). Shift works; Enter starts a new line; arrows/F-keys show as e.g. ``; Backspace erases the previous character (on the current line). To get the mouse/keyboard back from QEMU press Ctrl+Alt+G; close the window or press Ctrl+C in the terminal to quit. (Over SSH without X, use `cargo run -- -display curses` to see the VGA screen right in your terminal; quit from another terminal with `pkill qemu-system`.) Headless (no window, good for SSH / scripts): ```sh ./scripts/run-headless.sh # boot, breakpoint, demos, ~5 s of threads + timer ticks, then QEMU exits (script exit 0) ./scripts/run-headless.sh stack-overflow # deliberately overflow the stack; passes if the double-fault handler catches it ./scripts/run-headless.sh page-fault # write to unmapped 0xdeadbeaf; passes if the page-fault handler catches it cargo run --features headless -- -display none # same as the first, via cargo cargo run --features headless,stack-overflow -- -display none # same as the second # (QEMU's exit status is 33 = success; cargo just reports it as non-zero) cargo run -- -display none # headless but kernel keeps running; Ctrl+C to stop ./scripts/screenshot.sh [out.png] [text] [expected] # boot, "type" text via QEMU monitor `sendkey` ("<" = Backspace; # default "Hello, kernex VGA frame, wrote "New!" through it .[heap] 256 KiB heap at 0x444444440000 (fixed-size-block allocator) [heap] Box::new(41) = 41 at 0x444444440000 [heap] Vec of 500 numbers at 0x444444441000, sum = 124750 [heap] Rc count after clone = 2 [heap] Rc count after drop = 1 [thread] spawned threads A, B, C (status on the top line) Type something! (keyboard echo, Backspace works) [async] async number: 42 [async] example task step 1 (after yield) [async] example task step 2 (after yield) [async] example task step 3 (after yield) .[thread A] count 1 .[thread B] count 1 .......[thread A] count 2 .[thread B] count 2 .....[thread C] busy loop round 1 (count = 2000000) .. 18 ticks [thread A] count 3 .[thread B] count 3 .......[thread A] count 4 .[thread B] count 4 .....[thread C] busy loop round 2 (count = 4000000) ..[thread A] count 5 .[thread B] count 5 . 36 ticks ............[thread C] busy loop round 3 (count = 6000000) ...... 54 ticks ..........[thread C] busy loop round 4 (count = 8000000) ........ 72 ticks ....[thread C] busy loop round 5 (count = 10000000) [thread C] done, exiting .............. 90 ticks [ok] timer works, exiting ``` Each `.` is a timer tick. Threads A and B only print their first 5 counts to serial (so the keyboard echo in your terminal stays readable); after that they keep counting on the VGA top line. ## Tests ```sh cargo test # everything (lib unit tests + tests/*.rs), each booted in QEMU headless cargo test --lib # just the unit tests inside src/ cargo test --test heap_allocation # one integration test ``` Each test binary is its own little kernel: `bootimage runner` boots it with the `test-args` from `Cargo.toml` (no display, serial to stdout, isa-debug-exit) and treats QEMU exit code 33 as a pass. Output looks like: ``` Running 5 tests heap_allocation::large_allocation_uses_fallback... [ok] heap_allocation::large_vec... [ok] heap_allocation::many_boxes... [ok] heap_allocation::many_boxes_long_lived... [ok] heap_allocation::simple_allocation... [ok] ``` | Test | What it checks | |---|---| | lib (`src/*.rs` `#[test_case]`s) | trivial assertion, breakpoint handler returns, VGA `println!` writes the right cells | | `tests/basic_boot.rs` | printing works straight after boot, before any `init()` | | `tests/heap_allocation.rs` | Box, big Vec, 100 K short-lived boxes (memory reuse), same with one long-lived box, >2 KiB fallback allocation | | `tests/should_panic.rs` | a failing `assert_eq!` reaches the panic handler (`harness = false`) | | `tests/stack_overflow.rs` | stack overflow lands in the double-fault handler on the IST stack (`harness = false`) | | `tests/executor.rs` | `SimpleExecutor` runs an async fn to completion; the waker-based `Executor` interleaves two tasks that `yield_now().await` (a1 b1 a2 b2 a3 b3) | | `tests/threads.rs` | two busy-looping threads that never yield both make progress while the main thread busy-waits too (only possible with preemption); a thread can exit and the others keep running | How it works: `#![feature(custom_test_frameworks)]` collects every `#[test_case]` function and passes them to `rust_kernel::test_runner` (in `src/lib.rs`), which prints each name to serial, runs it, prints `[ok]`, then exits QEMU with success. A panic goes to `test_panic_handler`, which prints `[failed]` and exits with failure. Extra args after `--` in `cargo run -- ...` are appended to the QEMU command line, e.g. `cargo run -- -d int -no-reboot` to log every interrupt and stop on triple faults, or `cargo run -- -s -S` to wait for `gdb` on port 1234. ## Project layout ``` .cargo/config.toml build-std (core, alloc) + custom target + scripts/runner.sh as the cargo runner rust-toolchain.toml pin nightly + rust-src + llvm-tools-preview x86_64-rust_kernel.json custom target: no OS, no red zone, soft-float, abort on panic, rust-lld Cargo.toml deps, features, test settings, QEMU run/test args src/lib.rs the kernel library: module list, init(), kprintln!, test framework, exit_qemu src/main.rs the bootable binary: entry_point!(kernel_main), paging + heap demos, panic handler src/gdt.rs GDT + TSS with a separate IST stack for the double-fault handler src/interrupts.rs IDT; breakpoint / page fault / double fault handlers; 8259 PIC; timer handler (uptime, ticks, calls the thread scheduler), keyboard handler (just queues the scancode) src/memory.rs OffsetPageTable setup from CR3, example mapping, BootInfoFrameAllocator src/allocator.rs heap location/size, init_heap (maps heap pages), #[global_allocator], Locked src/allocator/bump.rs bump allocator (not active, for learning) src/allocator/fixed_size_block.rs fixed-size-block allocator (ACTIVE), falls back to linked_list_allocator src/task/mod.rs async: Task (pinned boxed future + id), yield_now() src/task/simple_executor.rs first executor: polls every task in a loop (no wakers) src/task/executor.rs real executor: task queue + wakers, sleeps with hlt when idle src/task/keyboard.rs lock-free scancode queue, ScancodeStream, async keyboard echo task src/thread.rs preemptive kernel threads: spawn, round-robin schedule(), asm context switch src/vga_buffer.rs VGA text-mode writer, print!/println!, print_at (status line), backspace src/serial.rs COM1 UART writer + serial_print!/serial_println! tests/ integration tests, one QEMU-booted kernel each scripts/runner.sh `cargo run`/`cargo test` runner: `bootimage runner` + stale-image fix scripts/run-headless.sh headless boot with timeout + exit-code check scripts/screenshot.sh headless boot + typed keys + PNG screenshot via QEMU monitor boot.png screenshot of the VGA screen ``` ## What the interrupt code does (chapters 5-7) - **CPU exceptions / IDT** (`interrupts.rs`): the IDT maps vector numbers to handler functions written with `extern "x86-interrupt"` (nightly `abi_x86_interrupt`), so the compiler saves all registers and returns with `iretq`. `_start` fires an `int3` on purpose; the breakpoint handler prints the stack frame and execution continues. A page-fault handler prints the faulting address from CR2. - **Double faults** (`gdt.rs`): a stack overflow hits the guard page -> page fault -> the CPU can't push the exception frame on the broken stack -> double fault -> same again -> *triple fault* = machine reset. We load our own GDT containing a TSS whose Interrupt Stack Table entry 0 points at a separate 20 KiB static stack, and mark the double-fault IDT entry to use it. Try `./scripts/run-headless.sh stack-overflow`. - **Hardware interrupts**: the two 8259 PICs are remapped to vectors 32-47 (0-31 are CPU exceptions) and then `sti` enables interrupts. The timer (IRQ 0, ~18.2 Hz) bumps an atomic tick counter and redraws the uptime clock; the keyboard (IRQ 1) reads the scancode from port 0x60 and queues it; an async task decodes it with `pc-keyboard` (US layout, scancode set 1) and echoes it (see Multitasking below). Each handler must send End-Of-Interrupt to the PIC or no further interrupts arrive. `print!`/`serial_print!` disable interrupts while holding their lock, otherwise an interrupt handler printing while `_start` holds the lock would deadlock. The idle loop uses `hlt`, so QEMU doesn't burn a host CPU core. ## Memory (chapters 8-11) **Paging.** Every address the kernel uses is *virtual*. The CPU translates it with a 4-level page table tree (level 4 -> 3 -> 2 -> 1 -> 4 KiB page); 9 bits of the address index each level, the low 12 bits are the offset in the page. CR3 holds the *physical* address of the level-4 table (here `0x1000`). The `page-fault` feature writes to `0xdeadbeaf`, which no table maps, and the page-fault handler reports the address (from CR2) and the reason (`CAUSED_BY_WRITE`, not `PROTECTION_VIOLATION` -> the page simply isn't present). **Reaching page tables.** Page tables contain physical addresses, but we can only dereference virtual ones. With the bootloader's `map_physical_memory` feature, all of physical RAM is also mapped at `physical_memory_offset` (`0x18000000000` here), so `virt = offset + phys`. `memory::init` wraps that in the x86_64 crate's `OffsetPageTable`, which can `translate_addr` (walk the tables) and `map_to` (add entries). **Virtual memory layout** (as set up by bootloader 0.9 in this build): | Virtual address | What | |---|---| | `0xb8000` | VGA text buffer (identity-mapped: virt = phys) | | `0x200000`... | kernel code + data (the ELF; maps to phys around `0x400000`) | | `0x0100_0000_0000`... | kernel stack (80 pages, guard page below it) | | `0x0180_0000_0000`... | all physical memory (`physical_memory_offset`) | | `0x4444_4444_0000` | kernel heap, 256 KiB (also holds thread stacks) (we map it in `allocator::init_heap`) | | `0xdead_beaf_000` | demo page we map to the VGA frame (writes "New!" on screen) | The "New!" demo writes to screen row 20 through the new mapping; by the time you look, later lines have scrolled it up, so in `boot.png` it overwrote the start of the first line. **Physical frames.** `BootInfoFrameAllocator` hands out 4 KiB frames from the regions the BIOS memory map marks `Usable` (~121 MiB of QEMU's default 128 MiB). It never frees; fine for now. **Heap.** `allocator::init_heap` maps the 25 pages of `HEAP_START..+HEAP_SIZE` to fresh frames, then gives the range to the `#[global_allocator]`. From then on the `alloc` crate (`Box`, `Vec`, `Rc`, `String`, `BTreeMap`...) works. `alloc` is compiled for our target by `build-std` in `.cargo/config.toml`. **Allocator designs** (`src/allocator/`): the *bump* allocator just moves a pointer forward and can only reclaim memory when every allocation is freed (it fails the `many_boxes_long_lived` test). The `linked_list_allocator` crate keeps a list of free regions (first fit, slower, but reuses any freed memory). The active *fixed-size-block* allocator rounds each request up to 8/16/.../2048 bytes and keeps one free list per size, so alloc/free are O(1) pops/pushes; bigger requests and empty lists fall back to the linked-list allocator. Switch allocators by editing one line in `src/allocator.rs`. ## Multitasking (async/await + threads) ### Cooperative: async/await (`src/task/`, phil-opp chapter 12) An `async fn` compiles into a state machine (a `Future`). An *executor* calls `poll` on it; the future runs until it has to wait, then returns `Pending` and remembers a `Waker`. Whoever makes progress possible later (here: the keyboard interrupt) calls `waker.wake()`, which puts the task's id back into the executor's queue. When the queue is empty, the executor `hlt`s -- carefully: it disables interrupts, checks the queue, and then uses `sti; hlt` as one step (`enable_and_hlt`), so a wakeup arriving in between can't be missed. The keyboard interrupt handler now does almost nothing: read the scancode from port 0x60 and push it into a fixed-size lock-free queue (`crossbeam_queue::ArrayQueue`), then wake the keyboard task (`futures_util::task::AtomicWaker`). No locks in the interrupt handler means no deadlock if the interrupt arrives while the task holds one. The task (`print_keypresses`) is an `async` loop over a `ScancodeStream` that decodes and echoes characters. "Cooperative" means a task only gives up the CPU at an `.await`. A task that loops forever without awaiting freezes every other task. Upside: switching is cheap (just return from `poll`), and all tasks share one stack. ### Preemptive: kernel threads (`src/thread.rs`, beyond the blog series) Each thread has its own 16 KiB stack (a `Vec` on the heap). On every timer interrupt (~18 times a second) the handler sends the EOI to the PIC and then calls `thread::schedule()`, which picks the next thread round-robin and calls `switch_context(&mut old.rsp, new.rsp)` (a dozen lines of `global_asm!`): ``` push rbp, rbx, r12, r13, r14, r15 ; callee-saved regs of the OLD thread, onto its stack mov [rdi], rsp ; remember where the old stack ends mov rsp, rsi ; ---- now on the NEW thread's stack ---- pop r15, r14, r13, r12, rbx, rbp ; its callee-saved regs ret ; return to wherever IT called switch_context ``` Why only those six registers? `switch_context` is called like a normal function, so the compiler already saved the caller-saved registers, and the `x86-interrupt` handler around it saved everything the interrupted code was using (plus the CPU pushed RIP/RSP/RFLAGS). So a paused thread's complete state sits on its own stack; the scheduler only stores its `rsp`. The new thread "returns" from `schedule()`, from its timer handler, and `iretq`s back to where it was interrupted. A brand-new thread was never interrupted, so `spawn` hand-builds a stack that looks as if `switch_context` had saved it: six register slots (r12 = the thread's function) and a return address pointing to `thread_trampoline`, which moves r12 into rdi and calls `thread_start(entry)`. That re-enables interrupts (we arrive from inside the timer interrupt, where they're off), runs the function, and marks the thread finished when it returns. The code that booted the kernel becomes thread 0, "main", which runs the async executor. Preemption means even `loop {}` can't hog the CPU (thread C in the demo is a pure busy loop). The price: any code can be interrupted anywhere, so shared data needs locks, and a lock that the timer handler might also want must be taken with interrupts disabled. Here: the print macros and `print_at` already hold their spinlocks with interrupts off (`without_interrupts`), so a thread can't be switched away while holding them; the scheduler lock is only taken with interrupts off and is released *before* switching (the next thread would otherwise wait forever for a lock held by a paused thread). The heap allocator's lock is never taken in an interrupt handler, so a thread preempted while holding it only makes others spin until it runs again -- slow, not a deadlock. Known simplifications (good exercises): sleeping threads (`sleep_ticks`) just `hlt` and still get scheduled -- a real scheduler has a *blocked* state and a sleep queue; finished threads' stacks are never freed; no guard page under thread stacks (an overflow silently corrupts the heap); no priorities; a single CPU, so `spin::Mutex` is enough. ## Roadmap Done: all 12 chapters of the 2nd edition, plus preemptive kernel threads. - [x] Freestanding binary — [phil-opp](https://os.phil-opp.com/freestanding-rust-binary/) - [x] Minimal kernel booting in QEMU — [phil-opp](https://os.phil-opp.com/minimal-rust-kernel/), [OSDev: Bare Bones](https://wiki.osdev.org/Bare_Bones) - [x] VGA text mode printing — [phil-opp](https://os.phil-opp.com/vga-text-mode/), [OSDev: Printing to Screen](https://wiki.osdev.org/Printing_To_Screen) - [x] Serial output + QEMU exit — [phil-opp: Testing](https://os.phil-opp.com/testing/), [OSDev: Serial Ports](https://wiki.osdev.org/Serial_Ports) - [x] CPU exceptions / IDT — breakpoint & page fault handlers. [phil-opp: CPU Exceptions](https://os.phil-opp.com/cpu-exceptions/), [OSDev: IDT](https://wiki.osdev.org/Interrupt_Descriptor_Table), [OSDev: Exceptions](https://wiki.osdev.org/Exceptions) - [x] GDT / TSS / double faults — separate IST stack so stack overflows don't triple-fault. [phil-opp: Double Faults](https://os.phil-opp.com/double-fault-exceptions/), [OSDev: GDT](https://wiki.osdev.org/Global_Descriptor_Table), [OSDev: TSS](https://wiki.osdev.org/Task_State_Segment) - [x] Testing — `cargo test` in QEMU, custom test framework, integration tests. [phil-opp: Testing](https://os.phil-opp.com/testing/) - [x] Hardware interrupts — 8259 PIC, timer, PS/2 keyboard echo, `hlt` idle loop. [phil-opp: Hardware Interrupts](https://os.phil-opp.com/hardware-interrupts/), [OSDev: 8259 PIC](https://wiki.osdev.org/8259_PIC), [OSDev: PIT](https://wiki.osdev.org/Programmable_Interval_Timer), [OSDev: PS/2 Keyboard](https://wiki.osdev.org/PS/2_Keyboard) (later: [APIC](https://wiki.osdev.org/APIC)) - [x] Introduction to paging — CR3, page fault demo. [phil-opp](https://os.phil-opp.com/paging-introduction/), [OSDev: Paging](https://wiki.osdev.org/Paging) - [x] Paging implementation — OffsetPageTable, translation, frame allocator, example mapping. [phil-opp](https://os.phil-opp.com/paging-implementation/), [OSDev: Page Frame Allocation](https://wiki.osdev.org/Page_Frame_Allocation) - [x] Heap allocation — `alloc`, heap mapping, global allocator. [phil-opp](https://os.phil-opp.com/heap-allocation/), [OSDev: Memory Allocation](https://wiki.osdev.org/Memory_Allocation) - [x] Allocator designs — bump, linked list (crate), fixed-size block (active). [phil-opp](https://os.phil-opp.com/allocator-designs/) - [x] Async/await — Task, simple executor, waker-based executor, lock-free async keyboard task (with Backspace). [phil-opp: Async/Await](https://os.phil-opp.com/async-await/), [OSDev: Multitasking Systems](https://wiki.osdev.org/Multitasking_Systems) - [x] Preemptive kernel threads — own stacks, timer-driven round-robin, asm context switch. [OSDev: Brendan's Multi-tasking Tutorial](https://wiki.osdev.org/Brendan%27s_Multi-tasking_Tutorial), [OSDev: Context Switching](https://wiki.osdev.org/Context_Switching) Next stages (beyond the phil-opp series): 1. **User mode (ring 3) & syscalls** — ring 3 code/data segments in the GDT, set `TSS.rsp0` per thread so interrupts from user mode land on the kernel stack, enter user mode with `iretq`, then `syscall`/`sysret` (or `int 0x80` first) for a few calls (`write`, `exit`, `yield`). Per-process page tables (a new level-4 table that shares the kernel half). Threads become processes. [OSDev: Getting to Ring 3](https://wiki.osdev.org/Getting_to_Ring_3), [OSDev: System Calls](https://wiki.osdev.org/System_Calls), [OSDev: SYSENTER/SYSCALL](https://wiki.osdev.org/SYSENTER) 2. **ELF loading** — parse a static ELF (e.g. with the `xmas-elf` or `goblin` crate), map its segments into a fresh address space, jump to the entry point in ring 3. [OSDev: ELF](https://wiki.osdev.org/ELF), [OSDev: ELF Tutorial](https://wiki.osdev.org/ELF_Tutorial) 3. **Filesystem** — a VFS layer, a RAM disk / initrd first, then FAT on an ATA/virtio disk. [OSDev: File Systems](https://wiki.osdev.org/File_Systems), [OSDev: VFS](https://wiki.osdev.org/VFS), [OSDev: Initrd](https://wiki.osdev.org/Initrd), [OSDev: FAT](https://wiki.osdev.org/FAT), [OSDev: ATA PIO Mode](https://wiki.osdev.org/ATA_PIO_Mode) 4. **Shell** — line editing on top of the keyboard task, a few built-in commands (`help`, `mem`, `uptime`, `ls`, `cat`), later launching user programs. [OSDev: Shell](https://wiki.osdev.org/Shell) Small exercises along the way: a *blocked* state + sleep queue for threads, free finished threads' stacks, guard pages under thread stacks, a timer-based async `sleep` future, speed up the PIT ([OSDev: PIT](https://wiki.osdev.org/Programmable_Interval_Timer)), add handlers for divide error / general protection fault, make `BootInfoFrameAllocator` O(1), add a `mem` stat (bytes allocated) to the allocator. Other good references: [OSDev wiki main page](https://wiki.osdev.org/Expanded_Main_Page), [Intel SDM](https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdms.html), the [blog_os repo](https://github.com/phil-opp/blog_os) (one branch per post, e.g. `post-04`), and for a UEFI/framebuffer setup later, the newer [`bootloader` 0.11](https://github.com/rust-osdev/bootloader) crate. ## Notes / gotchas - The target JSON changes occasionally with new nightlies (e.g. `target-pointer-width` became an integer, `rustc-abi` was added). If a nightly update breaks the build, compare with the latest `x86_64-blog_os.json` in the blog_os repo, or pin a date in `rust-toolchain.toml` (`channel = "nightly-2026-10-29"`). - The "packages contain code that will be rejected by a future version of Rust: core" warning comes from building `core` with build-std and is harmless. - `bootimage` prints "`CARGO_MANIFEST_DIR` env variable not set" when run outside `cargo run`; also harmless. - `cargo test` with build-std breaks ("duplicate lang item in crate `core`: `sized`") if `Cargo.toml` sets `panic = "abort"` in `[profile.*]`. The target JSON already sets `"panic-strategy": "abort"`, so the profile settings were removed. - Stale disk image when switching cargo features: cargo re-uses a previously built kernel ELF with an *old* mtime, so bootimage thinks its cached image is current and boots the wrong build. `scripts/runner.sh` and the scripts `touch` the ELF first to avoid this. If you call `cargo bootimage --features ...` by hand after switching features, `touch target/x86_64-rust_kernel/debug/rust_kernel` first.