rust-kernel

@@ -32,7 +32,7 @@ Under the shell:   (wait queues) without using any CPU while they wait. - **User programs.** Small Rust ELF binaries run in **ring 3** as processes,   each with its own address space. Their memory is freed when they exit. They-  talk to the kernel through 63 `syscall`s, numbered like Linux's:+  talk to the kernel through 83 `syscall`s, numbered like Linux's:   `read`, `write`, `open`, `stat`, `getdents64`, `pipe2`, `uname`, `socket`,   `connect`, `getrandom`... A few are our own: `spawn`, `mounts`, `sysinfo`,   `resolve`, `netinfo` and `trace_me`.@@ -69,6 +69,16 @@ Under the shell:   terminal the host reaches through a socket; `getty` runs a shell on it)   and **virtio-gpu** in 2D mode (`/dev/fb0` with ioctls; `tile` draws a   tiling-window-manager mock-up on it).+- **What a C library needs (stage 15).** `poll`, `select` and **`epoll`**+  over pipes, sockets, terminals and files; **`O_NONBLOCK`** with `EAGAIN`;+  a **loopback** interface (127.0.0.1) so programs talk TCP/UDP to each+  other without a network card; **threads**: `clone(CLONE_VM |+  CLONE_THREAD ...)`, **futexes**, thread-local storage (`arch_prctl`),+  `gettid`, `tgkill`, `exit` vs `exit_group`, running on all 4 CPUs at+  once; real **ttys and ptys** with termios, canonical/raw modes, echo,+  ^C/^Z/^\\ as signals on every terminal, `/dev/tty`, `/dev/ptmx` and+  `/dev/pts/N`; plus `nanosleep`, `clock_gettime(CLOCK_REALTIME)`,+  `readv`/`writev`, `pread`/`pwrite` and `readlink /proc/self/exe`.  It has a `cargo test` suite that runs inside QEMU, and a headless self-test that drives the shell with a script.@@ -78,12 +88,12 @@ comments. Everything else is new: the serial mirroring, uptime clock, demo features and scripts, and everything past chapter 12. That covers preemptive threads, user mode, syscalls, fork/exec/mmap/signals, ELF loading, the VFS, PCI, virtio-blk, partitions, FAT32, the shell, the ext2, exFAT and btrfs drivers,-multi-core (SMP) support, the virtio-net driver and socket layer, and the+multi-core (SMP) support, the virtio-net driver and socket layer, the virtio-rng, virtio-console and virtio-gpu drivers with the framebuffer-console. The 8x16 font is [Spleen](https://github.com/fcambus/spleen) by+console, and stage 15's poll/epoll, loopback, threads, futexes and ptys. The 8x16 font is [Spleen](https://github.com/fcambus/spleen) by Frederic Cambus (BSD 2-clause, license in `src/font.rs`). -![Rustbelt 0.14.0 (stage 14: processes) on the 1024x768 virtio-gpu framebuffer: `tile` drew four tiled windows; in the left one the shell interrupts a CPU hog with Ctrl+C ("[exit code 130] (interrupted: SIGINT)"), ends a background job with `kill %1` (exit code 143), and runs cowtest and sigtest, which both report "all checks passed"](boot.png)+![Rustbelt 0.15.0 (stage 15: libc plumbing) on the 1024x768 virtio-gpu framebuffer: `tile` drew four tiled windows; in the left one the shell interrupts a CPU hog with Ctrl+C ("[exit code 130] (interrupted: SIGINT)"), `threads` sums 1..=40000000 with 4 threads and a futex mutex, `echoserver` echoes over 127.0.0.1 with epoll and poll, and `ptytest` and `nbtest` report "all checks passed"](boot.png)  ## New to Rust and C? Start here @@ -293,7 +303,7 @@ only, so the VGA screen shows just the shell session (and kernel messages such as a killed process).  ```-Rustbelt 0.14.0 -- a hobby x86_64 kernel in Rust+Rustbelt 0.15.0 -- a hobby x86_64 kernel in Rust [smp] 4 CPUs online [pci] 00:00.0 8086:1237 host bridge ... (12 PCI devices: the network card, the GPU, the rng, the console, 4 disks)@@ -326,13 +336,14 @@ Rustbelt 0.14.0 -- a hobby x86_64 kernel in Rust [virtio] hvc0: virtio-console at 00:05.0, IRQ 10 [virtio] gpu: virtio-gpu 2D at 00:03.0, display 1280x800, framebuffer 1024x768 (3072 KiB) [virtio] fbcon: text console on the framebuffer, 128x48 characters+[net] lo: loopback, 127.0.0.1/8 [net] eth0: virtio-net, MAC 52:54:00:12:34:56, IRQ 10 [net] DHCP lease: 10.0.2.15/24, gateway 10.0.2.2, DNS 10.0.2.3 [process] started pid 1 (sh), entry 0x800040ada5 A:/ $ # Rustbelt shell self-test: `cargo run --features headless` runs this with A:/ $ # `sh /mnt/disk0/scripts/selftest.sh` (B:/scripts/selftest.sh) and scripts/run-headless.sh checks the output. A:/ $ ver-Rustbelt 0.14.0 (stage 14: processes) x86_64+Rustbelt 0.15.0 (stage 15: libc plumbing) x86_64 A:/ $ drives A:  /            ramfs B:  /mnt/disk0   fat32@@ -859,7 +870,7 @@ A:/tmp $ exit 0 [ok] shell self-test finished: Exited(0) [memory] 1015 frames in use before the shell, 1015 after it and all its children exited [run-headless] frames in use returned to the baseline-[run-headless] host curl got httpd's page: Hello from Rustbelt! Rustbelt 0.14.0 (stage 14: processes)+[run-headless] host curl got httpd's page: Hello from Rustbelt! Rustbelt 0.15.0 (stage 15: libc plumbing) [run-headless] all 67+12 expected shell and hvc0 lines found [run-headless] kernel exited OK ```@@ -876,7 +887,7 @@ E:/ $ getty -1 /dev/hvc0  Rustbelt getty: shell on this terminal (type `exit` to end it) E:/ $ ver-Rustbelt 0.14.0 (stage 14: processes) x86_64+Rustbelt 0.15.0 (stage 15: libc plumbing) x86_64 E:/ $ echo hi over hvc0 hi over hvc0 E:/ $ random -d 20 3@@ -967,6 +978,7 @@ heap_allocation::simple_allocation...               [ok] | `tests/virtio.rs` | **the virtual drivers**: 2 x 4 KiB from virtio-rng differ, aren't constant, have 15384-17384 one bits out of 32768 and contain all 256 byte values; the `random` program (hex, dice, a usage error); hvc0 receives exactly the bytes QEMU feeds in from `tests/hvc0-input.txt`, and 8 KiB written to it go out with nothing dropped; virtio-gpu answers `GET_DISPLAY_INFO`, refuses a flush of a resource that doesn't exist with `ERR_INVALID_RESOURCE_ID`, full and clipped flushes work, a pattern written into the framebuffer reads back; the text console is on the framebuffer (128x48) and printing lights up pixels; `/dev` lists `fb0 hvc0 null random urandom`; `tile` draws the expected colours at known pixels, moves the console to 61x46 and `tile -r` back, with no frames leaked. `scripts/gpu-check.sh` checks what QEMU actually displays | | `tests/guard_pages.rs` | the guard page sits right below each kernel stack, and the user stack's 64 KiB guard zone has the right bounds; a kernel thread that recurses forever is killed with "kernel stack overflow" three times in a row (the rest of the kernel carries on, and its stack is freed); the user `overflow` program ends with `Killed(SIGSEGV, "stack overflow")` and leaves no frames behind | | `tests/process.rs` | **the process model** (stage 14), mostly by running the user test programs: `cowtest` (globals, heap and stack stay separate after fork in both directions, three generations, `MAP_SHARED` vs `MAP_PRIVATE`); `forktest` (fork/wait4/getppid, 10 children reaped with `wait4(-1)`, `ECHILD`, execve errors, vfork + exec, `pipe2(O_CLOEXEC)`, fork + dup2 + exec through a pipe, a child dying of SIGSEGV); **exec argv/envp/auxv** (`forktest args "two words" ""` with `FOO=bar EMPTY=`: argc, every argv and envp string, `AT_PAGESZ`, `AT_ENTRY` = `_start`, `AT_PHDR` covering the entry, `AT_RANDOM`, `AT_EXECFN`, stack position); `mmaptest` (zeroed anonymous memory, 64 MiB that costs nothing until touched, SIGSEGV handlers that see `si_addr` and `SEGV_ACCERR`/`SEGV_MAPERR` and fix the fault so the access is retried, `PROT_NONE`, file-backed private mappings, `brk`, `EINVAL`s, memory back to the start); an unhandled fault is `Killed(SIGSEGV)` with status 11 and exit code 139; `sigtest` (handler round trip through sigreturn with registers intact, masks and pending signals, SIGKILL can't be caught, default actions, SIGCHLD with `si_pid`/`si_status`, SIGPIPE, stop/continue seen by `wait4(WUNTRACED/WCONTINUED)`, `EINTR` vs `SA_RESTART`, `kill` errors, signal 0, `kill(-pgid)`); wait-status encoding; a **fork bomb** stops at `EAGAIN`, is killed and reaped, and leaves frames and the process count unchanged; **20 fork/exec/exit loops leak no frames** and leave no shared frames; **Ctrl+C** (`tty::input_char('\x03')`) kills `spin` in the foreground group with SIGINT (exit code 130), and does nothing when no group is in the foreground |+| `tests/libc.rs` | **libc plumbing** (stage 15), by running four user test programs twice each (a warm-up, then a run that must leave the thread, process, futex-waiter, pty, socket and frame counts exactly as they were): `threads` (4 threads sum 1..=40,000,000 on several CPUs; a futex mutex keeps a shared counter exact while an unlocked one loses updates; each thread sees its own TLS value through `fs:`; `gettid`; `FUTEX_WAIT` with a stale value is `EAGAIN`, times out, `FUTEX_WAKE` with no waiters; `CLONE_CHILD_CLEARTID` + futex wake for join; `exit` ends one thread, `exit_group` all; `tgkill` and `ESRCH`); `echoserver` (an epoll server and a poll server on 127.0.0.1, three forked clients each, plus UDP over loopback); `ptytest` (canonical mode holds a half line back, backspace editing, CR -> NL, echo, ONLCR; raw mode passes bytes through without echo; `TCGETS`/`TCSETS`; ^C and ^\\ kill the child that made the pty its controlling terminal with SIGINT/SIGQUIT and echo "^C"; `/dev/tty` with and without a controlling terminal; closing the master gives the slave EOF); `nbtest` (`EAGAIN` from pipes, sockets and ptys; 64 KiB pipe capacity; poll bits; `fcntl(F_SETFL, O_NONBLOCK)` and `FIONBIO`; nonblocking `connect` is `EINPROGRESS` then `POLLOUT`; `select`, `ppoll`, `readv`/`writev`, `pread`/`pwrite`, `getcwd`, `uname`, `CLOCK_REALTIME`, `nanosleep`, `readlink /proc/self/exe`) |  How it works: `#![feature(custom_test_frameworks)]` collects every `#[test_case]` function and passes them to `rustbelt::test_runner` (in `src/lib.rs`), which@@ -1026,9 +1038,12 @@ src/block/virtio_blk.rs   legacy virtio-blk driver: one virtqueue, polled, bounc src/block/partition.rs    MBR + GPT parsing, Partition = a BlockDevice window onto its disk src/elf.rs                minimal ELF64 loader (PT_LOAD segments -> regions; program headers for AT_PHDR) src/errno.rs              Linux errno numbers-src/tty.rs                console line discipline (cooked: echo + line editing; raw: every key, arrows-                          as escape sequences; Ctrl+C/Ctrl+Z signal the foreground process group)-                          + Console (the File behind fds 0/1/2)+src/tty.rs                ttys (stage 15): the console, hvc0 and ptys share one line discipline+                          (termios, canonical/raw, echo, ^C/^Z/^\\ -> signals), sessions and+                          controlling terminals, /dev/tty, /dev/ptmx + /dev/pts/N+src/poll.rs               poll, ppoll, select, pselect6 and epoll (one global "something changed" counter)+src/futex.rs              futex WAIT/WAKE (+ bitsets, private flag, timeouts): what libc mutexes and+                          pthread_join sleep on src/fs/mod.rs             VFS: Inode + FileSystem traits, mount table, path normalize/lookup/create/remove,                           symlink following (ELOOP after 8), init (boot files),                           mount_disks (ext2, exFAT, btrfs or FAT partitions/disks -> /mnt/diskN)@@ -1072,12 +1087,15 @@ user/sysinfo.rs           CPUs, per-CPU load bars, threads, processes, memory (t user/overflow.rs          recurses forever: killed with "stack overflow" by the user stack guard zone src/time.rs               microsecond clock: the TSC, calibrated against the PIT at boot src/net/mod.rs            networking: the smoltcp Interface, DHCP + DNS, the `netd` thread, IRQ handling-src/net/virtio_net.rs     legacy virtio-net driver: RX/TX virtqueues, MAC, smoltcp's Device trait+src/net/virtio_net.rs     legacy virtio-net driver: RX/TX virtqueues, MAC+src/net/device.rs         smoltcp's Device: frames for our own MAC/IP (127.x) loop back, the rest go to the NIC src/net/socket.rs         BSD-style sockets (TCP, UDP, ICMP) on smoltcp, blocking on a wait queue user/{ifconfig,ping,host,fetch,httpd}.rs  the network programs user/{random,getty,tile}.rs  stage 13: getrandom, a shell on hvc0, the framebuffer tiling demo user/{forktest,cowtest,mmaptest,sigtest}.rs  stage 14: the process-model checks (each prints ok/FAILED                           lines and "all checks passed"; tests/process.rs runs them)+user/{threads,echoserver,ptytest,nbtest}.rs  stage 15: threads + futexes, poll/epoll over loopback,+                          ptys, nonblocking I/O (tests/libc.rs runs them) scripts/hvc0-session.txt  the commands run-headless.sh feeds into hvc0 (for `getty -1 /dev/hvc0`) scripts/gpu-check.sh      types `tile`, dumps the virtio-gpu screen (PPM), compares pixels scripts/http-reply.sh     the host side of `fetch http://10.0.2.100/`: QEMU runs it per connection@@ -1091,7 +1109,7 @@ src/virtio/console.rs     virtio-console (hvc0): 8 RX + 8 TX buffers, IRQ-driven src/virtio/gpu.rs         virtio-gpu 2D: display info, resource, backing, scanout, transfer + flush src/fbcon.rs              text console on the framebuffer: cells, cursor, scrolling, dirty rows, a viewport src/font.rs               the 8x16 bitmap font (Spleen, BSD-2), Latin-1 + box drawing-src/dev.rs                devfs at /dev: null, random, urandom, hvc0, fb0 (the Framebuffer file)+src/dev.rs                devfs at /dev: null, random, urandom, console, tty, ptmx, pts/, hvc0, fb0 src/serial.rs             COM1 UART writer + serial_print!/serial_println! tests/                    integration tests, one QEMU-booted kernel each scripts/runner.sh         `cargo run`/`cargo test` runner: `bootimage runner` + stale-image fix@@ -1249,8 +1267,9 @@ and allocates, so `GlobalAlloc::alloc`/`dealloc` now take the heap lock inside someone wakes it. Two ways to be woken:  - `sleep_ticks(n)` marks the thread sleeping until tick `now + n`; the PIT-  handler wakes it once that tick has passed (the `sleep_ms` syscall rounds-  up to 55 ms ticks).+  handler wakes it once that tick has passed (`nanosleep` sleeps whole+  55 ms ticks this way, then yields until the TSC clock says the time is+  really up). - A `WaitQueue` (a list of waiting thread ids). `wait_until(|| condition)`   turns interrupts off, checks the condition, and if it's false puts the   thread on the queue, marks it blocked and switches away; when woken it checks@@ -1502,6 +1521,7 @@ Numbers are Linux's wherever Linux has the same call, so man pages and | 3 | `close` | fd | | | 4 | `stat` | path, path_len, *Stat | follows symlinks | | 5 | `fstat` | fd, *Stat | |+| 7 | `poll` | *pollfd, n, timeout_ms | **stage 15**: -1 = forever; pipes, sockets, ttys, files, epoll fds | | 8 | `lseek` | fd, offset, whence | `SEEK_SET/CUR/END` = 0/1/2; `ESPIPE` on console/pipes | | 9 | `mmap` | addr, len, prot, flags, fd, offset | MAP_PRIVATE/SHARED, MAP_ANONYMOUS, MAP_FIXED; files: private only | | 10 | `mprotect` | addr, len, prot | PROT_READ 1, PROT_WRITE 2, PROT_EXEC 4 |@@ -1510,13 +1530,18 @@ Numbers are Linux's wherever Linux has the same call, so man pages and | 13 | `rt_sigaction` | sig, *new, *old, 8 | kernel `struct sigaction`; handlers need SA_RESTORER | | 14 | `rt_sigprocmask` | how, *set, *old, 8 | SIG_BLOCK 0 / SIG_UNBLOCK 1 / SIG_SETMASK 2 | | 15 | `rt_sigreturn` |  | only from a signal handler's restorer |-| 16 | `ioctl` | fd, request, arg | console: `TTY_SET_RAW` (1): raw mode on (arg 1) / off (0); returns the old mode. `/dev/fb0`: `FB_GET_INFO` (0x4600) writes `[width, height, bytes per row, 32]` to `*arg`; `FB_FLUSH` (0x46f0) shows `*[x, y, w, h]` (arg 0: the rows this fd wrote); `FB_FILL` (0x46f1) fills `*[x, y, w, h, 0xRRGGBB]`; `FB_SET_CONSOLE` (0x46f2) moves the text console into `*[x, y, w, h]` (arg 0: full screen) and returns `cols << 16 \| rows`. Console, stage 14: `TIOCGPGRP`/`TIOCSPGRP` (0x540f/0x5410) get/set the foreground process group (Ctrl+C goes to it), `TIOCGWINSZ` (0x5413) the size in characters. Others: `ENOTTY` |+| 16 | `ioctl` | fd, request, arg | any fd, **stage 15**: `FIONBIO` (0x5421). Ttys (console, hvc0, ptys), **stage 15**: `TCGETS`/`TCSETS`/`TCSETSW`/`TCSETSF` (0x5401-0x5404, the 36-byte kernel `struct termios`), `TIOCSCTTY`, `TIOCNOTTY`, `TIOCSWINSZ`, `TIOCGPTN`/`TIOCSPTLCK` on `/dev/ptmx`. Console: `TTY_SET_RAW` (1): raw mode on (arg 1) / off (0); returns the old mode. `/dev/fb0`: `FB_GET_INFO` (0x4600) writes `[width, height, bytes per row, 32]` to `*arg`; `FB_FLUSH` (0x46f0) shows `*[x, y, w, h]` (arg 0: the rows this fd wrote); `FB_FILL` (0x46f1) fills `*[x, y, w, h, 0xRRGGBB]`; `FB_SET_CONSOLE` (0x46f2) moves the text console into `*[x, y, w, h]` (arg 0: full screen) and returns `cols << 16 \| rows`. Console, stage 14: `TIOCGPGRP`/`TIOCSPGRP` (0x540f/0x5410) get/set the foreground process group (Ctrl+C goes to it), `TIOCGWINSZ` (0x5413) the size in characters. Others: `ENOTTY` |+| 17 | `pread64` | fd, buf, len, offset | **stage 15**: regular files; leaves the file position alone |+| 18 | `pwrite64` | fd, buf, len, offset | **stage 15** |+| 19 | `readv` | fd, *iovec, n | **stage 15**: `struct iovec {base, len}`, up to 1024 |+| 20 | `writev` | fd, *iovec, n | **stage 15** | | 22 | `pipe` | *[u32; 2] | [read end, write end]; `pipe2` (293) also takes `O_CLOEXEC` |+| 23 | `select` | n, *rd, *wr, *ex, *timeval | **stage 15**: fd bitmaps (n <= 1024) | | 24 | `sched_yield` | | | | 32 | `dup` | fd | lowest free fd | | 33 | `dup2` | old, new | closes `new` first if open | | 34 | `pause` |  | sleeps until a signal; returns EINTR |-| 35 | `sleep_ms` | ms | **ours**: Linux nr 35 is `nanosleep(timespec)`. A signal ends it early with `EINTR` |+| 35 | `nanosleep` | *timespec, *remaining | **Linux's `nanosleep` since stage 15** (stages 2-14 had our own `sleep_ms(ms)` here). Sleeps whole PIT ticks, then yields until the TSC clock says the time is up; a signal ends it early with `EINTR` and the time left in `*remaining` | | 39 | `getpid` | | | | 41 | `socket` | domain, type, protocol | `AF_INET` (2) only; `SOCK_STREAM` (1) = TCP, `SOCK_DGRAM` (2) = UDP, or `SOCK_DGRAM` + `IPPROTO_ICMP` (1) = ping socket (like Linux's unprivileged ping sockets). Returns an fd; `close` closes it | | 42 | `connect` | fd, *sockaddr_in, len | TCP: blocks for the handshake (10 s, `ETIMEDOUT`; `ECONNREFUSED` on a reset). UDP: sets the default peer |@@ -1526,28 +1551,44 @@ Numbers are Linux's wherever Linux has the same call, so man pages and | 49 | `bind` | fd, *sockaddr_in, len | local port (address must be 0 or ours) | | 50 | `listen` | fd, backlog | TCP; up to 4 connections can be waiting | | 54 | `setsockopt` | fd, level, name, *val, len | only `SOL_SOCKET`/`SO_RCVTIMEO` (a `timeval`) |-| 56 | `clone` | flags, stack, ... | only fork-like flags: SIGCHLD + optional CLONE_VFORK (else ENOSYS) |+| 56 | `clone` | flags, stack, *ptid, *ctid, tls | fork-like (SIGCHLD, optional CLONE_VFORK), or since **stage 15** a thread: `CLONE_VM|FS|FILES|SIGHAND|THREAD|SYSVSEM` plus optional `SETTLS`, `PARENT_SETTID`, `CHILD_SETTID`, `CHILD_CLEARTID`. The child starts on `stack` with rax = 0 | | 57 | `fork` |  | copy-on-write; parent gets the pid, child 0 | | 58 | `vfork` |  | fork + the parent waits until the child execs or exits | | 59 | `execve` | path, argv, envp (C strings) | Linux's stack layout: argc, argv, envp, auxv |-| 60 | `exit` | code | ends the process (one thread each, so the same as `exit_group`, 231) |+| 60 | `exit` | code | since **stage 15** ends only the calling *thread*; `exit_group` (231) ends the whole process | | 61 | `wait4` | pid, *status, options, *rusage | **Linux's `wait4` since stage 14** (stage 13 had a simpler `wait`): pid > 0 one child, -1 any, 0 / < -1 a process group; `*status` gets the packed status (`WIFEXITED`, `WEXITSTATUS`, `WIFSIGNALED`, `WTERMSIG`, `WIFSTOPPED`...); options `WNOHANG` 1 (returns 0 if nobody is done), `WUNTRACED` 2, `WCONTINUED` 8; reaps the child; `ECHILD` if none; rusage must be 0 | | 62 | `kill` | pid, sig | pid 0 = my group, -1 = all, < -1 = group -pid |-| 63 | `uname` | *utsname | Linux's struct: 6 NUL-padded 65-byte fields: "Rustbelt", "qemu", "0.14.0", "stage 14: processes", "x86_64", "(none)" |-| 72 | `fcntl` | fd, cmd, arg | F_DUPFD 0, F_GETFD 1, F_SETFD 2, F_GETFL 3, F_SETFL 4, F_DUPFD_CLOEXEC 1030 |-| 79 | `getcwd` | buf, len | returns the length (no NUL); `ERANGE` if too small |+| 63 | `uname` | *utsname | Linux's struct: 6 NUL-padded 65-byte fields: "Rustbelt", "qemu", "0.15.0", "stage 15: libc plumbing", "x86_64", "(none)" |+| 72 | `fcntl` | fd, cmd, arg | F_DUPFD 0, F_GETFD 1, F_SETFD 2, F_GETFL 3, F_SETFL 4 (`O_NONBLOCK` on pipes, sockets, ttys), F_DUPFD_CLOEXEC 1030 |+| 79 | `getcwd` | buf, len | like Linux since stage 15: writes a NUL and returns the length including it; `ERANGE` if too small | | 80 | `chdir` | path, path_len | per-process cwd, inherited by forked and spawned children | | 83 | `mkdir` | path, path_len | | | 84 | `rmdir` | path, path_len | empty directories only (`ENOTEMPTY`); `EBUSY` on a mount point | | 87 | `unlink` | path, path_len | files and symlinks (the link, not its target); directories give `EISDIR` |-| 89 | `readlink` | path, path_len, buf, len | a symlink's target (no NUL); returns its length, cut to `len` like Linux; `EINVAL` if not a symlink |+| 89 | `readlink` | path, path_len, buf, len | a symlink's target (no NUL); returns its length, cut to `len` like Linux; `EINVAL` if not a symlink. Stage 15: `/proc/self/exe` is the running program's path | | 109 | `setpgid` | pid, pgid | 0 = myself / a new group | | 110 | `getppid` |  | 0 if started by the kernel or orphaned | | 111 | `getpgrp` |  |  |+| 112 | `setsid` |  | **stage 15**: new session + group, no controlling tty | | 121 | `getpgid` | pid |  |+| 124 | `getsid` | pid | **stage 15** |+| 158 | `arch_prctl` | code, addr | **stage 15**: ARCH_SET_FS 0x1002 / ARCH_GET_FS 0x1003: the TLS pointer |+| 186 | `gettid` |  | **stage 15**: thread id (= pid for the first thread) |+| 200 | `tkill` | tid, sig | **stage 15** |+| 202 | `futex` | *u32, op, val, *timespec, *u32, val3 | **stage 15**: FUTEX_WAIT 0, WAKE 1, WAIT_BITSET 9, WAKE_BITSET 10 (+ PRIVATE 128) |+| 213 | `epoll_create` | size | **stage 15**: = epoll_create1(0) | | 217 | `getdents64` | fd, buf, len | real Linux `struct linux_dirent64` records (ino, off, reclen, type, name\0) |-| 228 | `clock_gettime` | clock, *timespec | microsecond resolution (TSC calibrated against the PIT); any clock id counts from boot |-| 231 | `exit_group` | code | = exit (one thread per process) |+| 218 | `set_tid_address` | *u32 | **stage 15**: cleared + futex-woken when this thread exits; returns the tid |+| 228 | `clock_gettime` | clock, *timespec | microsecond resolution (TSC calibrated against the PIT). `CLOCK_MONOTONIC` (1) counts from boot; since stage 15 `CLOCK_REALTIME` (0) is real Unix time (the CMOS clock read once at boot, plus uptime) |+| 231 | `exit_group` | code | ends the process: every thread (stage 15; `exit`, 60, ends one thread) |+| 232 | `epoll_wait` | epfd, *epoll_event, max, timeout_ms | **stage 15**: level-triggered; EPOLLONESHOT supported |+| 233 | `epoll_ctl` | epfd, op, fd, *epoll_event | **stage 15**: ADD 1, DEL 2, MOD 3 |+| 234 | `tgkill` | tgid, tid, sig | **stage 15**: a signal for one thread |+| 270 | `pselect6` | n, *rd, *wr, *ex, *timespec, *{*mask, 8} | **stage 15** |+| 271 | `ppoll` | *pollfd, n, *timespec, *mask, 8 | **stage 15**: sets the mask while waiting |+| 281 | `epoll_pwait` | epfd, *events, max, timeout, *mask, 8 | **stage 15** |+| 288 | `accept4` | fd, *addr, *len, flags | **stage 15**: flags: SOCK_NONBLOCK, SOCK_CLOEXEC |+| 291 | `epoll_create1` | flags | **stage 15**: EPOLL_CLOEXEC | | 292 | `dup3` | old, new, flags | flags: O_CLOEXEC | | 293 | `pipe2` | *[u32; 2], flags | flags: O_CLOEXEC | | 318 | `getrandom` | buf, len, flags | random bytes from virtio-rng (the host's `/dev/urandom`), up to 64 KiB per call; never blocks, flags ignored. Without virtio-rng: a TSC-seeded xorshift (not secure) |@@ -1902,6 +1943,7 @@ Programs in `B:/bin` (`user/*.rs`, each 30-150 lines): | `tile [1-6]` / `tile -r` | draw a tiling-window-manager mock-up on `/dev/fb0` and move the console into the first window; `-r` gives the console the whole screen back | | `hello`, `demo`, `evil`, `spin`, `child` | the stage 5/6 test programs: `run demo` runs all the stage 6 syscall checks | | `forktest`, `cowtest`, `mmaptest`, `sigtest` | the stage 14 checks of fork/exec/wait4, copy-on-write, mmap and signals; each ends with "all checks passed" (`forktest bomb` is a bounded fork bomb, `mmaptest crash` dies of SIGSEGV) |+| `threads [n]`, `echoserver [port]`, `ptytest`, `nbtest` | the stage 15 checks: a parallel sum on `n` threads with a futex mutex; an epoll + poll echo server over 127.0.0.1 (`echoserver 7070` just serves forever); ptys and the line discipline; nonblocking I/O and the libc extras. Each ends with "all checks passed" |  Built-in aliases are the DOS names: `dir` = `ls`, `type` = `cat`, `del` = `rm`, `md` = `mkdir`, `copy` = `cp`; and for networking,@@ -3243,7 +3285,8 @@ time a child exits, stops or continues. ### What it does and doesn't do  - One thread per process: no `CLONE_VM`/`CLONE_THREAD`, no futexes yet-  (they are next on the roadmap, because libc threads need them). `clone`+  (they are next on the roadmap, because libc threads need them).+  *Stage 15 added both; see the next chapter.* `clone`   accepts only fork-style flags (`SIGCHLD`, optionally `CLONE_VFORK`). - `vfork` is fork + "the parent waits until the child execs or exits". It   is correct for every legal use of vfork, just not faster than fork.@@ -3255,6 +3298,217 @@ time a child exits, stops or continues. - Ctrl+C and the foreground group work on the main console. hvc0's   `getty` shell runs programs without a foreground group, so there Ctrl+C   is just a character. Proper ttys and ptys are on the roadmap.+  *Stage 15: hvc0 is a real tty now, and `getty` makes it the shell's+  controlling terminal, so Ctrl+C works there too.*++## Libc plumbing (stage 15)++Stage 14 gave Rustbelt processes. A real C library (musl, glibc) expects+quite a lot more from a kernel before ordinary programs work: a way to+wait on many files at once, files that say "not now" instead of+blocking, threads, a way for threads to sleep on a lock, per-thread+variables, and terminals that behave like terminals. Stage 15 adds all of+that, still with Rustbelt's own Rust programs as the users. This chapter+explains each idea from scratch.++### Event loops: poll, select and epoll++Imagine a chat server with three clients. If it calls `read` on client 1,+it blocks until client 1 says something, while clients 2 and 3 wait. One+fix is a thread per client (below). The other, used by nearly every+server, GUI toolkit and terminal emulator, is an **event loop**:++```text+loop {+    ready = wait_until_any_is_ready([listener, client1, client2, ...])+    for fd in ready { accept / read / write -- none of these block now }+}+```++The kernel offers three versions of `wait_until_any_is_ready`:++- **`poll(fds, n, timeout_ms)`**: an array of `{fd, events, revents}`.+  You set `events` (`POLLIN` = "tell me when I can read", `POLLOUT` =+  "...write"), the kernel fills in `revents`, including `POLLHUP` ("the+  other end hung up") and `POLLERR`. `ppoll` is the same with a+  nanosecond timeout and a signal mask.+- **`select(n, readfds, writefds, exceptfds, timeout)`**: the 1983 BSD+  original, with bitmaps instead of an array (`fd_set`: bit `fd` set means+  "watch this fd"). `pselect6` is its `ppoll`.+- **`epoll`**: `epoll_create1` makes an epoll instance (itself an fd);+  `epoll_ctl(ADD/MOD/DEL, fd, events, data)` registers fds *once*;+  `epoll_wait` returns only the ready ones, each with your `data` (usually+  the fd or a pointer to your client struct). With 10 000 connections,+  poll copies and checks 10 000 entries per call; epoll doesn't.++How Rustbelt does it (`src/poll.rs`): every open file can answer "what's+ready right now?" (`File::poll`). Whatever might change that (bytes into a+pipe, a key, a packet, a pty write, a close) calls `poll::notify()`,+which bumps one global counter and wakes every sleeping poller. A poller+checks its fds; if none is ready it sleeps until the counter moves or the+timeout ends. Simple and it can't miss an event, at the cost of waking+pollers that didn't need waking. epoll here is *level-triggered*: a fd is+reported as long as it stays ready (Linux's default; `EPOLLET`, "only+report changes", is accepted but behaves the same).++Try `echoserver` (it runs the same echo test with an epoll server and a+poll server) or `echoserver 7070` to keep one running.++### Nonblocking I/O and EAGAIN++An event loop must never block, so its fds are put in **nonblocking**+mode: `fcntl(fd, F_SETFL, O_NONBLOCK)`, or `ioctl(fd, FIONBIO, &1)`, or+`O_NONBLOCK` at `open`/`pipe2`/`socket` time. A read that would have to+wait now returns at once with **`EAGAIN`** ("try again later"; also+called `EWOULDBLOCK`), and so does a write to a full pipe (Rustbelt's pipes+hold 64 KiB, like Linux's). A nonblocking `connect` returns+`EINPROGRESS`; poll for `POLLOUT` to learn when it's connected. `nbtest`+checks all of these.++### Loopback++`127.0.0.1` is "this machine". Programs use it to talk TCP/UDP to each+other (a database and its client, a compositor and its apps, X11) with no+network card involved. Rustbelt now has an `lo` interface: frames sent to+127.x (or to our own eth0 address) are handed straight back to smoltcp's+receive side instead of the virtio-net card (`src/net/device.rs`). It is+there even with no network card at all, so `echoserver` works offline.++### Threads vs processes++A **process** is an address space plus open files, a cwd, signal+handlers and so on. A **thread** is "a place where the CPU is executing":+registers, a stack, a signal mask. Stage 14 had exactly one thread per+process. Linux has one call for both, `clone(flags, ...)`, and the flags+say what to *share* with the parent:++| Flag | Shares |+|---|---|+| `CLONE_VM` | the address space (same memory) |+| `CLONE_FILES`, `CLONE_FS` | the fd table; the cwd |+| `CLONE_SIGHAND` | signal handlers |+| `CLONE_THREAD` | the process itself: same pid, the new thread gets a new *tid* |+| `CLONE_SETTLS` | start the new thread with this TLS pointer (below) |+| `CLONE_PARENT_SETTID` / `CLONE_CHILD_SETTID` | write the new tid here |+| `CLONE_CHILD_CLEARTID` | when the thread exits, write 0 here and futex-wake it (that's `pthread_join`) |++`fork` is just `clone` with none of them. `pthread_create` is all of+them. `getpid` gives the process id, `gettid` the thread id (the first+thread's tid equals the pid). `exit` (60) ends *one thread*;+`exit_group` (231, what libc's `exit()` calls) ends them all. `tgkill`+sends a signal to one thread. Threads of one process run on different+CPUs at the same time: `threads` sums 1..=40,000,000 on 4 threads and+prints how many CPUs took part.++Sharing memory means two threads can update the same variable at once.+`counter += 1` is a load, an add and a store; two threads interleaving+them lose updates. `threads` shows exactly that, then fixes it with a+lock.++### Futexes: how locks sleep++A lock is just an integer. Taking a free lock is one atomic+compare-and-swap in user space, no kernel at all. The kernel is only+needed when a thread must *wait*, and that's `futex` ("fast userspace+mutex"):++- `futex(&word, FUTEX_WAIT, expected)`: "sleep, but only if `word` still+  equals `expected`"+- `futex(&word, FUTEX_WAKE, n)`: "wake up to n threads sleeping on `word`"++The "only if still equal" check, done under the kernel's lock, closes the+race where the holder unlocks between our check and our sleep: then WAIT+sees the new value and returns `EAGAIN` immediately. The mutex in+`user/threads.rs` is the classic one from Ulrich Drepper's paper+*Futexes Are Tricky*: the word is 0 (free), 1 (locked) or 2 (locked and+someone may be waiting). Lock: CAS 0 -> 1; if that fails, swap in 2 and+`FUTEX_WAIT` while it isn't 0. Unlock: swap in 0; only if the old value+was 2 call `FUTEX_WAKE`. With no contention, neither side ever enters+the kernel.++The kernel keeps no per-lock state, just a list of sleepers, each tagged+with the word it waits on: (address space, virtual address), or the+physical address for `MAP_SHARED` memory (`src/futex.rs`). Timeouts,+`FUTEX_WAIT_BITSET`/`WAKE_BITSET`, `REQUEUE` and the private flag are+there too, because libc uses them.++### Thread-local storage++`errno` is a global variable... but each thread needs its own, or one+thread's failing `read` would overwrite another's. Such per-thread+variables are **thread-local storage** (TLS). On x86_64 the trick is a+segment register: every thread has its own **FS base** address, and code+reads its TLS at `fs:offset`. libc allocates a TLS block per thread and+sets FS with `arch_prctl(ARCH_SET_FS, addr)` (or `CLONE_SETTLS`). The+kernel's whole job is to save and restore that address on each thread+switch (it lives in the `IA32_FS_BASE` MSR, `src/thread.rs`).+`set_tid_address` registers the `CLONE_CHILD_CLEARTID` word for the+first thread.++### ttys and ptys, and why terminals are weird++Terminals are strange because they're a 1970s design that never went+away. A terminal was a keyboard and screen at the end of a serial line;+the kernel sat between the line and the programs, and that middle layer,+the **line discipline**, is still in every Unix:++- **Canonical mode** (the default): what you type collects in a line+  buffer you can edit (Backspace, ^U erases the line) and `read` gets+  nothing until Enter. ^D at the start of a line is end-of-file.+  Programs that want every key (editors, Rustbelt's shell line editor)+  switch to **raw** (non-canonical) mode.+- **Echo**: the *kernel* prints what you type, not the program. That's+  why `cat` shows your typing before it prints the line back, and why a+  password prompt turns `ECHO` off.+- **Signals**: ^C sends `SIGINT`, ^\ `SIGQUIT`, ^Z `SIGTSTP`, not to "the+  program" but to the terminal's **foreground process group**, the job+  the shell put in front (`TIOCSPGRP`). That's why ^C kills `cat` and not+  the shell waiting for it.+- **Output processing**: `ONLCR` turns `\n` into `\r\n`, because a+  teletype needed a carriage return to move back to column 0.++All of it is configured by one struct, **termios**, read and written with+the `TCGETS`/`TCSETS` ioctls (libc: `tcgetattr`/`tcsetattr`).++**Sessions.** `setsid()` starts a session (roughly, a login). The first+terminal it opens becomes its **controlling terminal**; `/dev/tty` always+means "my controlling terminal", whatever its real name. When the+session leader exits, the foreground job gets `SIGHUP` ("hang up", from+the modem days). `getty` on hvc0 is now a classic one: it forks, the+child calls `setsid`, opens `/dev/hvc0` (which becomes its controlling+terminal) and `exec`s the shell, so ^C on hvc0 interrupts programs there+just like on the console.++**Pseudo-terminals.** A terminal window has no serial line; a program+plays the terminal instead. Opening **`/dev/ptmx`** gives that program+the **master** end and creates **`/dev/pts/N`**, the **slave** end, which+the shell gets as its stdin/out/err. Bytes written to the master are+"typed" into the slave, through the line discipline (so echo and ^C+work); what the shell writes to the slave comes out of the master to be+drawn. xterm, ssh, tmux and the future compositor's terminal all work+this way. In Rustbelt the console, hvc0 and every pty share one `Tty`+(`src/tty.rs`). `ptytest` drives one from both ends.++### What stage 15 does and doesn't do++Does: everything above, at `-smp 4`, checked by `tests/libc.rs`, which+runs each test program twice and requires the thread, process, futex,+pty, socket and memory-frame counts to come back exactly where they were+(no leaks). Also `nanosleep` (now exact to the microsecond on top of the+55 ms PIT tick), `clock_gettime(CLOCK_REALTIME)` (the CMOS clock), `uname`,+`getcwd`, `readlink /proc/self/exe`, `readv`/`writev`, `pread`/`pwrite`.++Doesn't (yet):++- No `SIGTTIN`/`SIGTTOU`: a background job may read and write the+  terminal.+- epoll is level-triggered only; no `EPOLLONESHOT`, no `signalfd`,+  `eventfd` or `timerfd`.+- One global "something changed" counter wakes every poller.+- Timeouts are rounded up to 55 ms PIT ticks (never early).+- No FPU/SSE state saving (Rustbelt's programs are soft-float). musl+  needs it, so it comes first in the next stage.+- Paths are still passed as pointer + length, not C strings.  ## Roadmap @@ -3265,7 +3519,9 @@ memory cleanup, the shell, hand-written ext2, exFAT and (read-only) btrfs drivers, multi-core support with a sleep/wake scheduler and guard pages, networking (virtio-net + smoltcp + sockets), and the virtual drivers (virtio-rng, virtio-console, virtio-gpu 2D with a-framebuffer console).+framebuffer console), the process model (fork/exec/mmap/signals), and+the kernel side of a libc (poll/epoll, nonblocking I/O, loopback,+threads and futexes, ttys and ptys).  - [x] Freestanding binary — [phil-opp](https://os.phil-opp.com/freestanding-rust-binary/) - [x] Minimal kernel booting in QEMU — [phil-opp](https://os.phil-opp.com/minimal-rust-kernel/), [OSDev: Bare Bones](https://wiki.osdev.org/Bare_Bones)@@ -3300,8 +3556,8 @@ framebuffer console). - [x] Networking (stage 12): virtio-net (legacy, interrupt-driven), smoltcp for ARP/IPv4/ICMP/UDP/TCP/DHCP/DNS, BSD-style socket syscalls with blocking on wait queues, `ifconfig`/`ping`/`host`/`fetch`/`httpd` — [OSDev: Virtio](https://wiki.osdev.org/Virtio), [smoltcp](https://docs.rs/smoltcp), [OSDev: Networking](https://wiki.osdev.org/Category:Networking), [Beej's Guide to Network Programming](https://beej.us/guide/bgnet/) - [x] Virtual drivers (stage 13): a modern (1.0) virtio transport shared by virtio-rng (`getrandom`, `/dev/random`, smoltcp's seed), virtio-console (`/dev/hvc0`, `getty`) and virtio-gpu 2D (a 1024x768 framebuffer, a bitmap-font console, `/dev/fb0`, `tile`) — [virtio 1.2 spec](https://docs.oasis-open.org/virtio/virtio/v1.2/virtio-v1.2.html), [OSDev: Virtio](https://wiki.osdev.org/Virtio), [Linux virtio-gpu driver](https://github.com/torvalds/linux/tree/master/drivers/gpu/drm/virtio) - [x] Processes (stage 14): `fork` with copy-on-write (refcounted frames), `execve` with Linux's argv/envp/auxv stack, `mmap`/`munmap`/`mprotect`/`brk` with demand paging, signals with Linux signal frames and `sigreturn`, `wait4`, `dup2`/`pipe2`/`fcntl`, process groups, Ctrl+C and job control in the shell — [man 2 fork](https://man7.org/linux/man-pages/man2/fork.2.html), [man 2 execve](https://man7.org/linux/man-pages/man2/execve.2.html), [man 7 signal](https://man7.org/linux/man-pages/man7/signal.7.html), [LWN: getauxval() and the auxiliary vector](https://lwn.net/Articles/519085/), [OSDev: Paging](https://wiki.osdev.org/Paging)-- [ ] Kernel-side groundwork for libc: `poll`/`select`, non-blocking sockets, loopback, futexes and threads, ttys and ptys (next)-- [ ] The OS split and a musl port (after that)+- [x] Libc plumbing (stage 15): `poll`/`ppoll`/`select`/`pselect6`/`epoll`, `O_NONBLOCK` + `EAGAIN`, a loopback interface, `futex`, `clone` threads with TLS (`arch_prctl`), `set_tid_address`, `gettid`/`tgkill`, `exit` vs `exit_group`, ttys and ptys with termios, `/dev/tty`, `/dev/ptmx` + `/dev/pts/N`, a fork+exec `getty`, `nanosleep`, `CLOCK_REALTIME`, `readv`/`writev`, `pread`/`pwrite` — [man 7 epoll](https://man7.org/linux/man-pages/man7/epoll.7.html), [man 2 futex](https://man7.org/linux/man-pages/man2/futex.2.html), [Drepper: Futexes Are Tricky](https://www.akkadia.org/drepper/futex.pdf), [The TTY demystified](https://www.linusakesson.net/programming/tty/), [man 7 pty](https://man7.org/linux/man-pages/man7/pty.7.html)+- [ ] The kernel/OS split and a musl port (next; waits for Tyler's go) - [ ] A Hyprland-style tiling compositor on the framebuffer (later)  ### Filesystem roadmap (Tyler's plan)@@ -3360,7 +3616,13 @@ Next, in this order:    with demand paging, signals, `wait4`, process groups and job control.    Left over for later: `MAP_SHARED` file mappings (needs a page cache),    `sigaltstack`, queued real-time signals, users and permissions.-5. **The rest of the kernel side of a libc** (next), roughly in this order:+5. ~~**The rest of the kernel side of a libc**~~ — done (stage 15): all+   of the list below except `openat` with C strings, the bigger `stat` and+   `getuid`, which come with the musl port (they are easier to get right+   with musl's own test programs). Left over: `SIGTTIN`/`SIGTTOU`+   (background jobs touching the terminal), edge-triggered epoll, IPv6+   loopback, per-file wait queues instead of one global poll counter.+   The plan as it was:    - `poll`/`select` (and later `epoll`), so one program can wait on      several fds: the keyboard, a pipe and a socket at once;    - **non-blocking sockets** (`O_NONBLOCK`, `EAGAIN`, `connect` in@@ -3375,12 +3637,21 @@ Next, in this order:      control and a terminal emulator becomes possible;    - smaller things musl expects: `nanosleep`, `readv`/`writev`, `openat`      with C strings, a bigger `stat`, `getuid` and friends.-6. **The OS split and a musl port** (after that): user space moves into its+6. **The kernel/OS split and a musl port** (next; this is the OS stage and+   waits for Tyler's go): user space moves into its    own tree (libc, toolchain, programs), and [musl](https://musl.libc.org/)    is ported, so ordinary C programs recompiled against Rustbelt's    syscalls (numbered like Linux's already) can run. `mmap` of `/dev/fb0`    comes with it, so programs can draw without a syscall per rectangle.-7. **The compositor** (after that): see below; `tile` is its first sketch.+   One kernel job comes first: **saving FPU/SSE state** (`XSAVE`/`FXSAVE`+   on every switch and in signal frames). Rustbelt's own programs are+   built soft-float, so the kernel has never needed it, but musl and every+   ordinary C compiler output use SSE registers freely; without saving+   them two threads would silently corrupt each other's floating point.+   Also: paths as C strings (`openat`), Linux's full `struct stat`.+7. **The Hyprland-style compositor** (after that): see below; `tile` is+   its first sketch, and stage 15's ptys are what its terminal windows+   will run shells on.  **Long term: a graphical desktop.** The goal is a **Hyprland-style tiling compositor**: windows tile automatically, with keyboard-driven layouts and@@ -3506,6 +3777,30 @@ and for a UEFI/framebuffer setup later, the newer [`bootloader` 0.11](https://gi   - QEMU's default network card is an e1000. Adding `-netdev`/`-device     virtio-net-pci` replaces it, so the PCI device count stayed 10 (stage 13's     GPU, rng and console make it 12).+- **Pitfalls hit in stage 15:**+  - *`write` must return what the caller asked for, not what went out.*+    With ONLCR a 6-byte `"hello\n"` becomes 7 bytes on the wire. The tty+    returned 7; the shell's `write_all` then sliced past the end of its+    buffer, panicked, and its panic handler overflowed the stack. Now the+    tty returns the caller's count (and maps a partial write back).+  - *Sleeping is only as fine as the timer.* `nanosleep(1 ms)` and a+    50 ms futex timeout could return early or late by up to one 55 ms+    PIT tick, depending on where in the tick they started. Deadlines now+    add a tick (never early), and `nanosleep` tops up by yielding until+    the TSC clock says the time has really passed.+  - *A closed slave means hang-up.* `ptytest` closed its copy of the+    slave after forking; for a moment no one had it open, the master+    reported `POLLHUP`, and the test read EOF. Keep the slave open in the+    parent (Linux behaves the same).+  - *Read one byte at a time in raw mode if you might `exec`.* The shell's+    line editor read 64 bytes at once; when commands were pasted into+    hvc0, the lines after the first were sitting in its buffer and were+    lost when it ran the first. Reading one byte per call leaves the rest+    in the tty, where the next reader finds it.+  - *Lazily started kernel threads look like leaks.* The hvc0 input+    thread started on first use, so its stack (16 frames) appeared+    between the headless self-test's two frame counts. It is now started+    at boot when a virtio console is present. - **Pitfalls hit in stage 14:**   - *The kernel wrote straight through copy-on-write.* Ring 0 ignores     read-only page flags unless `CR0.WP` is set, so a `read()` into a

Binary file changed.

@@ -24,27 +24,26 @@ SESSION="${2:-}" shift $(( $# > 2 ? 2 : $# )) if [ -z "$SESSION" ]; then     # The default session (what boot.png shows): the tiling demo moves the-    # text console into the left window of the virtio-gpu framebuffer, then-    # stage 14's process model at work: a CPU hog (output to a file) stopped-    # with Ctrl+C, another one in the background ended with `kill`, two test-    # programs. "vx{BS}er" types a typo and fixes it.+    # text console into the left window of the virtio-gpu framebuffer, a+    # CPU hog stopped with Ctrl+C (the tty turns ^C into SIGINT), then+    # stage 15's libc plumbing: threads summing on all CPUs, the epoll/poll+    # echo server over 127.0.0.1, the pty and nonblocking-I/O tests.+    # "vx{BS}er" types a typo and fixes it.     SESSION=$(mktemp /tmp/session.XXXXXX)     cat > "$SESSION" <<'TXT' tile vx{BS}er spin 1000 > /tmp/hog.txt {^C}-wc -l /tmp/hog.txt-start spin 1000 > /tmp/bg.txt-jobs-kill %1-wait-cowtest | grep passed-sigtest | grep passed+threads | grep threads+echoserver | grep echoes+ptytest | grep passed+nbtest | grep passed TXT-    set -- "Rustbelt 0.14.0 (stage 14: processes) x86_64" "tile: 4 windows on 1024x768, console 61x46" \-        "^C" "(interrupted: SIGINT)" "(start spin 1000 > /tmp/bg.txt), exit code 143" \-        "[cowtest] all checks passed" "[sigtest] all checks passed"+    set -- "Rustbelt 0.15.0 (stage 15: libc plumbing) x86_64" "tile: 4 windows on 1024x768, console 61x46" \+        "^C" "(interrupted: SIGINT)" "threads: 4 threads summed 1..=40000000 = 800000020000000" \+        "echoserver: epoll: 3 clients, 9 echoes" "[threads] all checks passed" \+        "[ptytest] all checks passed" "[nbtest] all checks passed" fi SOCK=$(mktemp -u /tmp/qemu-mon.XXXXXX) SERIAL_LOG=$(mktemp /tmp/serial.XXXXXX)