rust-kernel
Rustbelt: a hobby x86_64 kernel written in Rust
140876 bytes · raw
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 | # Rustbelt
**Rustbelt** is Tyler's hobby x86_64 kernel written in Rust. It started from
the 2nd edition of Philipp Oppermann's
[Writing an OS in Rust](https://os.phil-opp.com/) and keeps going past the end
of it. It boots in QEMU straight into **its own shell**: a user-mode program
that has a Unix feel with DOS touches. It offers `ls`, `cat`, pipes and
redirection, plus `dir`, `type`, `ver`, and drive letters from `A:` to
`E:`.
Under the shell:
- **VGA text and serial output.** The kernel prints to the VGA text screen and
to the serial port (COM1).
- **CPU exceptions.** It handles breakpoints, page faults, and double faults
(on their own stack).
- **Several CPUs.** It starts every CPU core QEMU gives it (`-smp 4`)
through the local APIC, and its scheduler runs threads and programs on
all of them at once.
- **Timers and keyboard.** The PIT drives an uptime clock and wakes
sleeping threads; each CPU's local APIC timer drives preemption. The
PS/2 keyboard feeds a tty with cooked and raw modes.
- **Memory.** It walks and edits its own page tables. It has a kernel heap
with a fixed-size-block allocator, and frees every frame a process or
thread used when it exits. Every stack has an unmapped **guard page**
under it, so an overflow is caught and reported instead of silently
corrupting memory.
- **Multitasking.** It multitasks two ways: an **async/await executor** and
**preemptive kernel threads**. Threads can sleep and wait for events
(wait queues) without using any CPU while they wait.
- **User programs.** Small Rust ELF binaries run in **ring 3** as processes,
each with its own address space. Their memory is freed when they exit. They
talk to the kernel through about 27 `syscall`s, numbered like Linux's:
`read`, `write`, `open`, `stat`, `getdents64`, `pipe`, `uname`... A few
are our own: `spawn`, `wait`, `mounts` and `sysinfo`.
- **Files.** Files live behind a Linux-style **VFS**: a ramfs at `/` and
`/tmp`, a **FAT32** partition mounted at `/mnt/disk0`, an **ext2**
disk at `/mnt/disk1`, an **exFAT** disk at `/mnt/disk2` and a **btrfs**
disk at `/mnt/disk3`. The ext2 and exFAT drivers are hand-written and
read and write their disks (ext2: permissions, symlinks, indirect blocks;
exFAT: long Unicode names, case-insensitive lookup, FAT chains). The real
`e2fsck` and `fsck.exfat` check what they write. The btrfs driver, also
hand-written, is **read-only**: copy-on-write B-trees, chunk mapping
(single and DUP), crc32c checksums on every node and data block,
zlib/lzo/zstd compression, holes, symlinks and subvolumes.
The disks are **virtio-blk** devices found by **PCI** enumeration, with
their **GPT/MBR** partitions parsed. The shell and its programs are
loaded from the FAT disk.
It has a `cargo test` suite that runs inside QEMU, and a headless self-test
that drives the shell with a script.
Most code follows the blog_os sources (MIT / Apache-2.0) closely, with extra
comments. Everything else is new: the serial mirroring, uptime clock, demo
features and scripts, and everything past chapter 12. That covers preemptive
threads, user mode, syscalls, ELF loading, the VFS, PCI, virtio-blk,
partitions, FAT32, the shell, the ext2, exFAT and btrfs drivers, and
multi-core (SMP) support.
## New to Rust and C? Start here
Rustbelt is meant to be read. You don't need to know Rust or C well to
follow it; here is a path through it, and the handful of ideas that make
kernel code look strange at first.
**A reading order.** Each file starts with a comment explaining what it is
for. Read them roughly in the order the kernel uses them:
1. `src/main.rs` -- `kernel_main` is the first Rust code that runs. It
calls the `init` functions below in order, then starts the shell.
2. `src/vga_buffer.rs`, `src/serial.rs` -- printing text: the screen is just
memory at address `0xb8000`, the serial port is a few I/O ports.
3. `src/interrupts.rs`, `src/gdt.rs` -- what happens when the CPU is
interrupted (a key press, a timer, a crash).
4. `src/memory.rs`, `src/allocator.rs` -- page tables (how addresses are
translated) and the heap (`Box`, `Vec`).
5. `src/thread.rs` -- threads, the scheduler and sleeping. `src/smp.rs` and
`src/apic.rs` -- how the other CPUs get started.
6. `src/process.rs`, `src/syscall.rs`, `src/elf.rs` -- user programs.
7. `user/hello.rs`, then `user/sh.rs` -- what a program on Rustbelt looks like.
8. `src/fs/` -- the file systems, once the rest makes sense.
**Rust in kernel code, in five ideas:**
- **`#![no_std]`**: there is no operating system underneath, so Rust's
standard library (files, threads, `println!`) isn't available. Only
`core` (basic types) and, once there's a heap, `alloc` (`Box`, `Vec`,
`String`). Rustbelt defines its own `println!`.
- **`unsafe`**: Rust normally guarantees memory safety; inside an `unsafe {}`
block *you* promise it instead. Kernels need it to touch hardware (raw
addresses, port I/O, CPU registers). Rustbelt keeps these blocks small and
says in a comment why each one is OK.
- **Ownership and `Drop`**: every value has one owner, and when the owner
goes away Rust runs its `Drop`. Rustbelt uses that for cleanup: dropping a
`KernelStack` unmaps it and frees its memory (`src/stack.rs`); dropping
the last `Arc<Process>` frees the whole address space.
- **`Mutex` and `Arc`**: `spin::Mutex<T>` is a lock: `.lock()` gives you the
`T` and unlocks automatically when you're done. `Arc<T>` is a shared
pointer with a reference count, so several threads can hold the same
process.
- **`global_asm!` / `asm!`**: a few things (switching stacks, entering ring
3, starting a CPU) need raw assembly. Each block is commented line by line.
**C ideas you'll see** (the hardware docs and OSDev wiki are written in C
terms): a *pointer* is just an address; `#[repr(C)]` lays out a struct the
way C would, so it matches what the CPU or a disk format expects; *volatile*
reads and writes tell the compiler a memory access really has to happen
(it's a device, not ordinary memory).
**Kernel words, briefly:** a *page* is 4 KiB of memory; a *frame* is the
physical memory behind it. An *interrupt* stops the CPU and runs a handler.
*Ring 0* is the kernel, *ring 3* user programs. A *syscall* is a user program
asking the kernel for something. *SMP* means several CPUs sharing memory; the
first one to boot is the *BSP*, the others *APs*.
**Try things.** `cargo run`, then in the shell: `help`, `ver`, `sysinfo`,
`spin 200 > A:/tmp/spin.txt &` followed by `sysinfo` (watch one CPU go to 100%), `overflow`
(a stack overflow, caught). Change something small -- the message in
`user/overflow.rs`, the bar width in `user/sysinfo.rs` -- and run
`cargo test` to check nothing broke. When something crashes, the `[kernel]`
lines on the serial output tell you which handler caught it and where.
## What's installed (on this box)
| Tool | Version | Why |
|---|---|---|
| rustup | 1.29.2 | toolchain manager (in `~/.cargo/bin`, ahead of Debian's `/usr/bin/cargo`) |
| Rust nightly | rustc 1.101.0-nightly (2026-10-29) | custom JSON targets + `build-std` are nightly-only |
| `rust-src` component | – | source of `core`, recompiled for our bare-metal target |
| `llvm-tools-preview` component | – | `llvm-objcopy` etc., used by `bootimage` |
| `bootimage` (cargo subcommand) | 0.10.5 | links kernel + bootloader into a bootable disk image |
| QEMU (`qemu-system-x86`, `qemu-system-gui`) | 10.0.13 | the emulator we boot in (`-gui` adds the GTK/SDL window) |
| `dosfstools`, `mtools`, `fdisk` | Debian packages | `build.rs` makes the FAT32 test disk with `sfdisk` + `mkfs.vfat` + `mmd`/`mcopy` |
| `e2fsprogs`, `exfatprogs` | Debian packages | the ext2 and exFAT images, and `e2fsck` / `fsck.exfat` to check Rustbelt's writes |
| `btrfs-progs` | 6.14 | `mkfs.btrfs --rootdir` and `btrfs-convert` make the btrfs images; `btrfs check --readonly` checks them |
`rust-toolchain.toml` pins the project to nightly with the two components, so
`cargo` inside this directory picks them up automatically. It also installs
the built-in `x86_64-unknown-none` target, used to compile the user programs.
Crates: `bootloader` 0.9.35 (BIOS bootloader used by the 2nd edition),
`volatile` 0.2, `spin` 0.5, `x86_64` 0.15, `uart_16550` 0.3, `lazy_static` 1.5,
`pic8259` 0.11 (interrupt controller), `pc-keyboard` 0.7 (scancode decoding),
`linked_list_allocator` 0.9 (fallback heap allocator), `fatfs` (git, `no_std` +
`alloc`; FAT12/16/32 reading and writing), `miniz_oxide` 0.8 (zlib inflate)
and `ruzstd` 0.8 (zstd decoding) for btrfs's compressed extents; LZO is
hand-written (`src/fs/lzo.rs`).
Fresh setup on another machine:
```sh
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh # rustup
rustup toolchain install nightly -c rust-src -c llvm-tools-preview -t x86_64-unknown-none
cargo install bootimage
sudo apt install qemu-system-x86 qemu-system-gui dosfstools mtools fdisk e2fsprogs exfatprogs btrfs-progs
```
## Build and run
```sh
cargo bootimage # build -> target/x86_64-rust_kernel/debug/bootimage-rustbelt.bin
cargo run # build + boot in a QEMU window; serial output also prints in the terminal
```
**Using it:** `cargo run` opens a QEMU window. The kernel prints a few boot
lines, then you're at the shell prompt `A:/ $`. Click into the window and type,
for example `help`, `cd B:`, `ls` or `cat readme.txt | wc`, or `ls -l C:/`
for the ext2 disk, `ls -l D:/` for the exFAT one, or `ls -l E:/` for the
read-only btrfs one. See
[The shell](#the-shell-stage-7) for everything it can do. Everything on the
screen is also mirrored to your terminal over serial. Typing `exit` ends the
shell, and the kernel starts a fresh one.
To get the mouse and keyboard back from QEMU, press Ctrl+Alt+G. To quit, close
the window or press Ctrl+C in the terminal. Over SSH without X, use
`cargo run -- -display curses` to see the VGA screen right in your terminal,
and quit from another terminal with `pkill qemu-system`.
Headless (no window, good for SSH / scripts):
```sh
./scripts/run-headless.sh # boot, the shell runs disk/scripts/selftest.sh, QEMU exits;
# passes if the shell exits 0 AND the expected lines appear
./scripts/run-headless.sh stack-overflow # deliberately overflow the stack; passes if the double-fault handler catches it
./scripts/run-headless.sh page-fault # write to unmapped 0xdeadbeaf; passes if the page-fault handler catches it
EXT2_IMG=copy.img ./scripts/run-headless.sh # use copy.img as the ext2 disk, writable (no snapshot)
EXFAT_IMG=copy.img ./scripts/run-headless.sh # the same for the exFAT disk
BTRFS_IMG=other.img ./scripts/run-headless.sh # another btrfs image as E: (always snapshot=on)
./scripts/ext2-check.sh # Rustbelt writes to copies of the ext2 images, then the
# host's e2fsck -fn checks them (see "ext2 (stage 8)")
./scripts/exfat-check.sh # the same for exFAT with fsck.exfat -n (see "exFAT (stage 9)")
./scripts/btrfs-check.sh # btrfs check --readonly on the images, then every file Rustbelt
# reads is compared with the host's copy (see "btrfs (stage 10)")
cargo run --features headless -- -display none # the self-test via cargo
# (QEMU's exit status is 33 = success; cargo just reports it as non-zero)
cargo run --features demos # the earlier stages' demos first, then the shell
./scripts/screenshot.sh [out.png] [session.txt] [expected...]
# boot, type a shell session via QEMU monitor `sendkey`
# (one command per line; {BS} = Backspace, {UP} = up arrow),
# check the expected texts appear, save the VGA screen
# (default: the session in boot.png); exits 1 if text is missing
```
Cargo features (in `Cargo.toml`):
| Feature | Effect |
|---|---|
| (none) | boot, start the shell on the console (the default) |
| `headless` | the shell runs `/mnt/disk0/scripts/selftest.sh` instead of reading the keyboard; its exit code becomes QEMU's (0 -> success); a 60 s timer safety net fails the run if it hangs; panics exit with failure |
| `demos` | before the shell: the breakpoint, paging and heap demos, threads A/B/C, the async example, and the `hello`/`evil`/`spin`/`demo` programs of stages 2-6 |
| `stack-overflow` | after boot, recurse forever to test the double-fault handler (with `headless`: exit success when caught) |
| `page-fault` | after boot, write to unmapped address `0xdeadbeaf` (with `headless`: exit success when caught) |
Expected serial output of `./scripts/run-headless.sh`. Boot is quick now
that the demos are behind a feature; most of the output is the shell running
`disk/scripts/selftest.sh`. Each command is echoed after its prompt. The
`[process] started/exited` lines are the kernel's own log. They go to serial
only, so the VGA screen shows just the shell session (and kernel messages
such as a killed process).
```
Rustbelt 0.11.0 -- a hobby x86_64 kernel in Rust
[smp] 4 CPUs online
[pci] 00:00.0 8086:1237 host bridge
... (10 PCI devices)
[pci] 10 devices on the PCI bus
[disk] vda: virtio-blk, 131072 sectors (64 MiB)
[disk] vda: GPT partition table, 1 partition(s)
[disk] vda1: start 2048 sectors 126976 type Microsoft basic data (FAT/exFAT/NTFS) name "fat32"
[disk] vdb: virtio-blk, 65536 sectors (32 MiB)
[disk] vdb: no partition table (NoSignature)
[disk] vdc: virtio-blk, 65536 sectors (32 MiB)
[disk] vdc: no partition table (WholeDiskExfat)
[disk] vdd: virtio-blk, 262144 sectors (128 MiB)
[disk] vdd: no partition table (NoSignature)
[disk] vda1: FAT32 volume "TYLERDISK" mounted at /mnt/disk0
[disk] vdb: ext2 volume "RUSTBELT" (rev 1, 1 KiB blocks, 4 groups) mounted at /mnt/disk1
[disk] vdc: exFAT volume "TYLEREXFAT" (4 KiB clusters, 7680 clusters) mounted at /mnt/disk2
[disk] vdd: btrfs volume "TYLERBTRFS" (16 KiB nodes, metadata DUP, data single, 2 subvolumes, read-only) mounted at /mnt/disk3
[vfs] mounted ramfs at /
[vfs] mounted ramfs at /tmp
[vfs] mounted fat32 at /mnt/disk0
[vfs] mounted ext2 at /mnt/disk1
[vfs] mounted exfat at /mnt/disk2
[vfs] mounted btrfs at /mnt/disk3
[process] started pid 1 (sh), entry 0x8000406b60
A:/ $ # Rustbelt shell self-test: `cargo run --features headless` runs this with
A:/ $ # `sh /mnt/disk0/scripts/selftest.sh` (B:/scripts/selftest.sh) and scripts/run-headless.sh checks the output.
A:/ $ ver
Rustbelt 0.11.0 (stage 11: core polish) x86_64
A:/ $ drives
A: / ramfs
B: /mnt/disk0 fat32
C: /mnt/disk1 ext2
D: /mnt/disk2 exfat
E: /mnt/disk3 btrfs
A:/ $ pwd
/
A:/ $ cd B:
B:/ $ ls
[process] started pid 2 (ls), entry 0x8000401880
bin/ notes/ readme.txt scripts/
[process] pid 2 (ls) exited with code 0
B:/ $ ls -l notes
[process] started pid 3 (ls), entry 0x8000401880
125 todo.txt
[process] pid 3 (ls) exited with code 0
B:/ $ type B:/readme.txt
[process] started pid 4 (cat), entry 0x8000400260
Hello from the FAT32 disk! This file lives on a virtio-blk disk, in a GPT
partition, in a FAT32 file system -- read through the kernel's VFS.
In the Rustbelt shell this disk is drive B: -- try `cd B:` and `ls`.
[process] pid 4 (cat) exited with code 0
B:/ $ cat b:notes/todo.txt | grep -i EXT | wc -l
[process] started pid 5 (cat), entry 0x8000400260
[process] started pid 6 (grep), entry 0x8000400610
[process] started pid 7 (wc), entry 0x8000400260
[process] pid 5 (cat) exited with code 0
[process] pid 6 (grep) exited with code 0
1
[process] pid 7 (wc) exited with code 0
B:/ $ dir bin | wc -l
[process] started pid 8 (ls), entry 0x8000401880
[process] started pid 9 (wc), entry 0x8000400260
[process] pid 8 (ls) exited with code 0
15
[process] pid 9 (wc) exited with code 0
B:/ $ echo first line > A:/tmp/out.txt
B:/ $ echo second line >> /tmp/out.txt
B:/ $ cat < /tmp/out.txt
[process] started pid 10 (cat), entry 0x8000400260
first line
second line
[process] pid 10 (cat) exited with code 0
B:/ $ md /tmp/work
[process] started pid 11 (mkdir), entry 0x8000400100
[process] pid 11 (mkdir) exited with code 0
B:/ $ copy B:/notes/todo.txt /tmp/work
[process] started pid 12 (cp), entry 0x80004006b0
[process] pid 12 (cp) exited with code 0
B:/ $ touch /tmp/work/empty.txt
[process] started pid 13 (touch), entry 0x8000400100
[process] pid 13 (touch) exited with code 0
B:/ $ ls -l A:/tmp/work
[process] started pid 14 (ls), entry 0x8000401880
0 empty.txt
125 todo.txt
[process] pid 14 (ls) exited with code 0
B:/ $ del /tmp/work/todo.txt /tmp/work/empty.txt /tmp/work
[process] started pid 15 (rm), entry 0x8000400100
[process] pid 15 (rm) exited with code 0
B:/ $ grep "B:" readme.txt
[process] started pid 16 (grep), entry 0x8000400610
In the Rustbelt shell this disk is drive B: -- try `cd B:` and `ls`.
[process] pid 16 (grep) exited with code 0
B:/ $ alias ll="ls -l"
B:/ $ ll A:/tmp
[process] started pid 17 (ls), entry 0x8000401880
23 out.txt
[process] pid 17 (ls) exited with code 0
B:/ $ run --trace cat B:/notes/todo.txt
[process] started pid 18 (cat), entry 0x8000400260
[trace 18] open("/mnt/disk0/notes/todo.txt", 0o0) = 3
[trace 18] read(3, "Done: the shell (cd, ls, cat, run, start"..., 512) = 125
Done: the shell (cd, ls, cat, run, start, pipes, redirection, drive letters).
Next: more file systems -- ext2, exFAT, btrfs.
[trace 18] write(1, "Done: the shell (cd, ls, cat, run, start"..., 125) = 125
[trace 18] read(3, "", 512) = 0
[trace 18] close(3) = 0
[trace 18] exit(0)
[process] pid 18 (cat) exited with code 0
B:/ $ start sleep 1
[process] started pid 19 (sleep), entry 0x8000400000
[pid 19] sleep started in the background
B:/ $ jobs
[pid 19] running start sleep 1
B:/ $ wait
[process] pid 19 (sleep) exited with code 0
[done] pid 19 (start sleep 1), exit code 0
B:/ $ cd C:
C:/ $ ls
[process] started pid 20 (ls), entry 0x8000401880
big.txt docs/ hello-link@ hello.txt lost+found/ readme.txt
sparse.bin
[process] pid 20 (ls) exited with code 0
C:/ $ cat hello-link
[process] started pid 21 (cat), entry 0x8000400260
Hello from the ext2 disk! This file lives on a second virtio disk, in an
ext2 file system that Rustbelt reads and writes with its own driver.
[process] pid 21 (cat) exited with code 0
C:/ $ grep "line 12345" big.txt
[process] started pid 22 (grep), entry 0x8000400610
big.txt line 12345
[process] pid 22 (grep) exited with code 0
C:/ $ cp B:/notes/todo.txt C:/todo.txt
[process] started pid 23 (cp), entry 0x80004006b0
[process] pid 23 (cp) exited with code 0
C:/ $ cat C:/todo.txt | grep -i ext2
[process] started pid 24 (cat), entry 0x8000400260
[process] started pid 25 (grep), entry 0x8000400610
[process] pid 24 (cat) exited with code 0
Next: more file systems -- ext2, exFAT, btrfs.
[process] pid 25 (grep) exited with code 0
C:/ $ md C:/made-here
[process] started pid 26 (mkdir), entry 0x8000400100
[process] pid 26 (mkdir) exited with code 0
C:/ $ echo written by Rustbelt > C:/made-here/note.txt
C:/ $ type C:/made-here/note.txt
[process] started pid 27 (cat), entry 0x8000400260
written by Rustbelt
[process] pid 27 (cat) exited with code 0
C:/ $ ls -l C:/
[process] started pid 28 (ls), entry 0x8000401880
-rw-r--r-- 304000 big.txt
drwxr-xr-x <DIR> docs/
lrwxrwxrwx hello-link -> hello.txt
-rw-r--r-- 142 hello.txt
drwx------ <DIR> lost+found/
drwxr-xr-x <DIR> made-here/
-rw-r--r-- 284 readme.txt
-rw-r--r-- 69206016 sparse.bin
-rw-r--r-- 125 todo.txt
[process] pid 28 (ls) exited with code 0
C:/ $ cd D:
D:/ $ ls
[process] started pid 29 (ls), entry 0x8000401880
Long File Name With Spaces.txt
MixedCase.TXT
Photos/
a-really-long-file-name-that-takes-several-file-name-directory-entries-in-exfat-xxxxxxxxxxxxxxxxxxxx.txt
frag-chain.txt
numbers.txt
readme.txt
Ünïcödé café.txt
[process] pid 29 (ls) exited with code 0
D:/ $ cat "long file name with spaces.txt"
[process] started pid 30 (cat), entry 0x8000400260
This file has a long name with spaces in it. exFAT stores names of up
to 255 UTF-16 characters, 15 per "File Name" directory entry.
[process] pid 30 (cat) exited with code 0
D:/ $ cp B:/notes/todo.txt D:/
[process] started pid 31 (cp), entry 0x80004006b0
[process] pid 31 (cp) exited with code 0
D:/ $ cp C:/hello.txt "Copied From Ext2.txt"
[process] started pid 32 (cp), entry 0x80004006b0
[process] pid 32 (cp) exited with code 0
D:/ $ cp mixedcase.txt C:/from-exfat.txt
[process] started pid 33 (cp), entry 0x80004006b0
[process] pid 33 (cp) exited with code 0
D:/ $ cat C:/from-exfat.txt
[process] started pid 34 (cat), entry 0x8000400260
MixedCase.TXT: exFAT keeps the case you wrote but ignores it when looking names up.
[process] pid 34 (cat) exited with code 0
D:/ $ md "New Folder"
[process] started pid 35 (mkdir), entry 0x8000400100
[process] pid 35 (mkdir) exited with code 0
D:/ $ echo hi from exFAT > "New Folder/Note.txt"
D:/ $ type "new folder/NOTE.TXT"
[process] started pid 36 (cat), entry 0x8000400260
hi from exFAT
[process] pid 36 (cat) exited with code 0
D:/ $ ls -l D:/
[process] started pid 37 (ls), entry 0x8000401880
142 Copied From Ext2.txt
132 Long File Name With Spaces.txt
84 MixedCase.TXT
<DIR> New Folder/
<DIR> Photos/
88 a-really-long-file-name-that-takes-several-file-name-directory-entries-in-exfat-xxxxxxxxxxxxxxxxxxxx.txt
38000 frag-chain.txt
340000 numbers.txt
363 readme.txt
125 todo.txt
61 Ünïcödé café.txt
[process] pid 37 (ls) exited with code 0
D:/ $ cd E:
E:/ $ ls -l
[process] started pid 38 (ls), entry 0x8000401880
drwxr-xr-x <DIR> Music/
drwxr-xr-x <DIR> archive/
-rw-r--r-- 2160000 big.txt
drwxr-xr-x <DIR> docs/
-rw-r--r-- 0 empty.txt
lrwxrwxrwx hello-link -> hello.txt
-rw-r--r-- 78 hello.txt
drwxr-xr-x <DIR> projects/
-rw-r--r-- 1500000 random.bin
-rw-r--r-- 637 readme.txt
-rw-r--r-- 1048576 zeros.bin
[process] pid 38 (ls) exited with code 0
E:/ $ cat hello.txt
[process] started pid 39 (cat), entry 0x8000400260
Hello from btrfs! (an inline extent: these bytes live inside the B-tree leaf)
[process] pid 39 (cat) exited with code 0
E:/ $ cat hello-link
[process] started pid 40 (cat), entry 0x8000400260
Hello from btrfs! (an inline extent: these bytes live inside the B-tree leaf)
[process] pid 40 (cat) exited with code 0
E:/ $ cat docs/btrfs/cow.txt
[process] started pid 41 (cat), entry 0x8000400260
Copy-on-write: btrfs never overwrites a block in place. To change a leaf
it writes a new copy somewhere free, then a new copy of its parent that
points to it, and so on up to the root. The superblock is the last thing
written; until then the old tree is still complete on disk.
[process] pid 41 (cat) exited with code 0
E:/ $ wc big.txt
[process] started pid 42 (wc), entry 0x8000400260
120000 360000 2160000 big.txt
[process] pid 42 (wc) exited with code 0
E:/ $ grep "line 119999" big.txt
[process] started pid 43 (grep), entry 0x8000400610
btrfs line 119999
[process] pid 43 (grep) exited with code 0
E:/ $ cat projects/rustbelt/stage10.txt
[process] started pid 44 (cat), entry 0x8000400260
Stage 10: a read-only btrfs driver. Next: core polish (SMP, memory
cleanup, guard pages, a proper scheduler), then networking.
[process] pid 44 (cat) exited with code 0
E:/ $ cp docs/btrfs/cow.txt D:/cow.txt
[process] started pid 45 (cp), entry 0x80004006b0
[process] pid 45 (cp) exited with code 0
E:/ $ type D:/COW.TXT | wc -l
[process] started pid 46 (cat), entry 0x8000400260
[process] started pid 47 (wc), entry 0x8000400260
[process] pid 46 (cat) exited with code 0
4
[process] pid 47 (wc) exited with code 0
E:/ $ cp docs/btrfs/chunks.txt C:/chunks.txt
[process] started pid 48 (cp), entry 0x80004006b0
[process] pid 48 (cp) exited with code 0
E:/ $ cat C:/chunks.txt | wc -l
[process] started pid 49 (cat), entry 0x8000400260
[process] started pid 50 (wc), entry 0x8000400260
[process] pid 49 (cat) exited with code 0
3
[process] pid 50 (wc) exited with code 0
E:/ $ echo hi > new.txt
sh: new.txt: Read-only file system
E:/ $ rm hello.txt
[process] started pid 51 (rm), entry 0x8000400100
rm: hello.txt: Read-only file system
[process] pid 51 (rm) exited with code 1
[exit code 1]
E:/ $ mkdir newdir
[process] started pid 52 (mkdir), entry 0x8000400100
mkdir: newdir: Read-only file system
[process] pid 52 (mkdir) exited with code 1
[exit code 1]
E:/ $ run evil
[process] started pid 53 (evil), entry 0x80004000f0
[evil] pid 53: reading kernel memory at 0x200000...
[kernel] user page fault: address 0x200000, error PageFaultErrorCode(PROTECTION_VIOLATION | USER_MODE)
[kernel] page fault in user mode at rip 0x800040017e
[process] pid 53 (evil) killed: page fault
[exit code 139]
E:/ $ # Stage 11: a stack overflow hits the guard zone and is killed cleanly;
E:/ $ # then two CPU hogs in the background, and sysinfo shows the load per CPU.
E:/ $ run overflow
[process] started pid 54 (overflow), entry 0x8000400130
[overflow] pid 54: recursing until the stack runs out...
[kernel] user page fault: address 0xffffffae88, error PageFaultErrorCode(CAUSED_BY_WRITE | USER_MODE)
[kernel] stack overflow in user mode at rip 0x8000400021
[process] pid 54 (overflow) killed: stack overflow
[exit code 139]
E:/ $ start spin
[process] started pid 55 (spin), entry 0x8000400400
[pid 55] spin started in the background
[spin] pid 55: spinning without syscalls...
E:/ $ start spin
[process] started pid 56 (spin), entry 0x8000400400
[pid 56] spin started in the background
[spin] pid 56: spinning without syscalls...
E:/ $ sysinfo
[process] started pid 57 (sysinfo), entry 0x8000400420
[spin] round 1 done (count = 30000000)
[spin] round 1 done (count = 30000000)
[spin] round 2 done (count = 60000000)
[spin] round 2 done (count = 60000000)
4 CPUs online (this program ran on CPU 1), up 11 s
10 threads, 4 processes, 1296 KiB of memory frames in use
[spin] round 3 done (count = 90000000)
[process] pid 55 (spin) exited with code 0
CPU load over the last quarter second:
CPU0 [####################] 100%
CPU1 [....................] 0%
CPU2 [####################] 100%
CPU3 [....................] 0%
[process] pid 57 (sysinfo) exited with code 0
[done] pid 55 (start spin), exit code 0
E:/ $ wait
...
E:/ $ nosuchprogram
sh: nosuchprogram: command not found (try `help`)
E:/ $ cd Q:
cd: there is no drive Q: (see `drives`)
E:/ $ cd A:/tmp
A:/tmp $ pwd
/tmp
A:/tmp $ exit 0
[process] pid 1 (sh) exited with code 0
[ok] shell self-test finished: Exited(0)
[memory] 171 frames in use before the shell, 171 after it and all its children exited
[run-headless] frames in use returned to the baseline
[run-headless] all 45+4 expected shell lines found
[run-headless] kernel exited OK
```
(Which CPU each program lands on, and the exact order of lines printed by
programs running at the same time, change from run to run.)
The run ends with the same number of frames in use before and after the
shell, even though 57 processes have come and gone; `run-headless.sh`
fails if they differ. Exited processes give back their page tables, code,
user stacks and kernel stacks. (The 171 at boot are mostly the four virtio
disks' queues and buffers, the kernel stack area and the other CPUs' stacks.)
The kernel boots from the bootimage on the first (IDE) disk; the scripts and
`Cargo.toml` attach a **second** disk, `target/disk.img` (GPT + FAT32), a
**third**, `target/ext2.img` (raw ext2), a **fourth**,
`target/exfat.img` (raw exFAT), and a **fifth**, `target/btrfs.img` (raw
btrfs), as `-drive file=target/disk.img,format=raw,if=virtio,snapshot=on`
and so on. `snapshot=on` makes QEMU throw away writes when it exits, so
every boot and every test sees the same fresh image even though the kernel
writes to it. The tests attach more images *after* those five, so the
drive letters B: to E: are the same in `cargo run` and `cargo test`: two
more ext2 images (4 KiB blocks, revision 0, `/mnt/disk4` and `disk5`), one
more exFAT image (512-byte clusters, `disk6`), and four more btrfs images
(lzo, zstd, converted from ext2: `disk7` to `disk9`; and one with sha256
checksums that is refused, so it gets no `/mnt/diskN`). Drive letters
follow mount order, so a new user-visible disk always goes right after the
last one `cargo run` attaches, never after the test-only ones.
## Tests
```sh
cargo test # everything (lib unit tests + tests/*.rs), each booted in QEMU headless
cargo test --lib # just the unit tests inside src/
cargo test --test heap_allocation # one integration test
```
Each test binary is its own little kernel: `bootimage runner` boots it with the
`test-args` from `Cargo.toml` (no display, serial to stdout, isa-debug-exit) and
treats QEMU exit code 33 as a pass. Output looks like:
```
Running 5 tests
heap_allocation::large_allocation_uses_fallback... [ok]
heap_allocation::large_vec... [ok]
heap_allocation::many_boxes... [ok]
heap_allocation::many_boxes_long_lived... [ok]
heap_allocation::simple_allocation... [ok]
```
| Test | What it checks |
|---|---|
| lib (`src/*.rs` `#[test_case]`s) | trivial assertion, breakpoint handler returns, VGA `println!` writes the right cells; btrfs: crc32c check value and the `"default"` name hash, a hand-built prealloc extent and an explicit hole, an LZO literal run + overlapping match |
| `tests/basic_boot.rs` | printing works straight after boot, before any `init()` |
| `tests/heap_allocation.rs` | Box, big Vec, 100 K short-lived boxes (memory reuse), same with one long-lived box, >2 KiB fallback allocation |
| `tests/should_panic.rs` | a failing `assert_eq!` reaches the panic handler (`harness = false`) |
| `tests/stack_overflow.rs` | stack overflow lands in the double-fault handler on the IST stack (`harness = false`) |
| `tests/executor.rs` | `SimpleExecutor` runs an async fn to completion; the waker-based `Executor` interleaves two tasks that `yield_now().await` (a1 b1 a2 b2 a3 b3) |
| `tests/shell.rs` | **the shell**: tokenizer (operators without spaces, quotes, comments), parser (pipelines, `<` `>` `>>`, `&`, error cases), drive-letter assignment from a mount table, `B:/x` -> `/mnt/disk0/x` translation and back for the prompt; then the real `/bin/sh` runs a script (cd with a drive letter, ls, a pipe, `>`/`>>`/`<`, `ver`, `exit 3`) with its stdout in a pipe and the output is checked; background jobs (`start`, `jobs`, `wait`); 20 processes started and reaped leave `frames_in_use` unchanged |
| `tests/user_mode.rs` | programs loaded from the FAT disk: `hello` exits 0, `demo` passes all its checks (exit 0), `evil` is killed with code 139, `spin` is preempted; spawn with arguments + trace + exit code 7, then reaped; a child's stdout into a pipe; spawn errors (`ENOENT`, `ENOEXEC`, `EISDIR`, `ECHILD`); `wait` with `WNOHANG` returns `EAGAIN` while the child runs; ELF loader rejects garbage; user pointers give `EFAULT` (length 0 is fine); `sleep` blocks for the right number of ticks; a `WaitQueue` wakes its waiter |
| `tests/vfs.rs` | read `/hello.txt` to EOF, path normalisation (`.`, `..`, relative to a cwd), `ENOENT`/`ENOTDIR`/`EINVAL`/`EISDIR`/`EEXIST`, `/tmp` is a separate ramfs, longest-prefix mounts, sparse writes zero-fill, listing, remove (`ENOTEMPTY`, `EBUSY` on mount points), seek/stat/truncate, `getdents64` record layout, pipes pass bytes and report EOF / `EPIPE` |
| `tests/block.rs` | PCI scan finds the virtio disk; sector 1 of the image is the GPT header; multi-sector reads equal single-sector reads; write + read back; out-of-range and partial-sector requests fail; the GPT partition is registered as `vda1` (with a FAT32 boot sector); MBR parsing on a RAM disk |
| `tests/fat.rs` | the FAT32 volume is mounted at `/mnt/disk0` with the expected files (`bin`, `notes`, `readme.txt`, `scripts`); read `readme.txt`; every program in `bin/` is a valid ELF; create/write/read/truncate/remove a 10 KB file; directories with long names, `ENOTEMPTY`; the `BlockIo` adapter handles unaligned reads and writes |
| `tests/ext2.rs` | the three ext2 images are mounted (`/mnt/disk1`, `disk4`, `disk5`); permissions and case-sensitive names; `big.txt` line by line and in one read (direct, single and double indirect blocks); `sparse.bin`'s data at 0 / 40 MiB / 65 MiB (triple indirect) and its holes; fast and slow symlinks, `readlink`, removing a link keeps its target; 4 KiB blocks and revision 0; on each image: mkdir, create, a 300 KB write, append, overwrite across a block boundary, a write at 70 MiB (hole + triple indirect), truncate, `ENOTEMPTY`/`EISDIR`/`ENOTDIR`, unlink, rmdir, and the superblock's free counts are back where they started; 200 files grow a directory; copy from FAT to ext2. `scripts/ext2-check.sh` then checks the same kernel's writes with the host's `e2fsck` |
| `tests/exfat.rs` | both exFAT images are mounted (`/mnt/disk2`, `/mnt/disk6`), label and cluster size; long, mixed-case and Unicode names listed as written; case-insensitive lookup (`ÜNÏCÖDÉ CAFÉ.TXT`, nested paths); `numbers.txt` at a cluster boundary and at its end; the FAT-chained `frag-chain.txt` on 4 KiB and 512-byte clusters; bad names (`EINVAL`, `ENAMETOOLONG`, `EEXIST` for a name in another case); on each image: mkdir, create, write across clusters, append, truncate, a write past the end (zero gap), `ENOTEMPTY`, unlink, rmdir, and the bitmap and PercentInUse are back where they started; a contiguous file turns into a FAT chain; 200 files grow a directory; the root grows; copies between FAT, ext2 and exFAT; a last test leaves files for `scripts/exfat-check.sh`'s `fsck.exfat` |
| `tests/btrfs.rs` | the four btrfs images are mounted (`/mnt/disk3`, `disk7`-`disk9`) and described (label, node size, DUP/single, subvolume count); **every file on every image is read and its size and CRC-32 compared with build.rs's manifest of the host's tree** (zlib, lzo with a two-level tree, zstd with 64 KiB nodes, and the converted image); reads at odd offsets across compressed and plain extent boundaries; holes in the converted `sparse.bin`; `ext2_saved/image` starts like an ext2 disk; hash lookups are exact (`HELLO.TXT` is `ENOENT`); symlinks, including one leaving a subvolume; subvolumes listed with ids and read-only flags; the sha256 image is refused with a reason; create/write/truncate/mkdir/remove give `EROFS`; and, last, damage on purpose: copy 0 of a DUP node (copy 1 is used), a data sector (`EIO` for that block only), then both copies (`EIO`). `scripts/btrfs-check.sh` runs `btrfs check --readonly` and compares the same kernel's output with the host's files |
| `tests/threads.rs` | two busy-looping threads that never yield both make progress while the main thread busy-waits too (only possible with preemption); a thread can exit and the others keep running |
| `tests/smp.rs` | **multi-core and the scheduler**: all 4 CPUs are online and take timer ticks; 4 busy threads run on 2+ CPUs *at the same time* (per-CPU busy ticks); 3 `spin` processes run on 2+ CPUs; 4 threads on 4 CPUs add 200,000 to a locked counter without losing one; `sleep_ticks(10)` lasts 10-12 ticks; sleepers wake in deadline order (3, 6, 9); while 4 threads sleep the CPUs are idle (<10% busy, so no busy-polling); two threads ping-pong 1,000 times through a `WaitQueue`; **leak checks**: 200 threads and 40 processes spawned and exited leave `frames_in_use`, the thread count, the process count and the number of kernel stacks exactly where they were |
| `tests/guard_pages.rs` | the guard page sits right below each kernel stack, and the user stack's 64 KiB guard zone has the right bounds; a kernel thread that recurses forever is killed with "kernel stack overflow" three times in a row (the rest of the kernel carries on, and its stack is freed); the user `overflow` program ends with `Killed("stack overflow")` and leaves no frames behind |
How it works: `#![feature(custom_test_frameworks)]` collects every `#[test_case]`
function and passes them to `rustbelt::test_runner` (in `src/lib.rs`), which
prints each name to serial, runs it, prints `[ok]`, then exits QEMU with success.
A panic goes to `test_panic_handler`, which prints `[failed]` and exits with failure.
Extra args after `--` in `cargo run -- ...` are appended to the QEMU command
line, e.g. `cargo run -- -d int -no-reboot` to log every interrupt and stop on
triple faults, or `cargo run -- -s -S` to wait for `gdb` on port 1234.
## Project layout
```
.cargo/config.toml build-std (core, alloc) + custom target + scripts/runner.sh as the cargo runner
rust-toolchain.toml pin nightly + rust-src + llvm-tools-preview
x86_64-rust_kernel.json custom target: no OS, no red zone, soft-float, abort on panic, rust-lld
Cargo.toml deps, features, test settings, QEMU run/test args
src/lib.rs the kernel library: module list, init(), kprintln!, test framework, exit_qemu
src/main.rs the bootable binary: kernel_main (init, then a thread that runs the shell and
restarts it when it exits), the `demos` feature's demos, panic handler
src/gdt.rs GDT (kernel + user segments) + TSS (double-fault IST stack, rsp0 per thread);
one GDT + TSS per CPU
src/interrupts.rs IDT; breakpoint / page fault / GPF / invalid opcode / double fault handlers
(faults from ring 3 kill the process; guard-page hits kill with
"stack overflow"); 8259 PIC; local APIC timer, reschedule IPI;
PIT handler (uptime, ticks, wakes sleeping threads),
keyboard handler (just queues the scancode)
src/memory.rs OffsetPageTable setup from CR3, example mapping, BootInfoFrameAllocator,
global frame allocator + free list (free_frame), per-process address spaces
(and freeing them), user pointer checks, kernel stack mappings
src/allocator.rs heap location/size, init_heap (maps heap pages), #[global_allocator], Locked<A>
src/allocator/bump.rs bump allocator (not active, for learning)
src/allocator/fixed_size_block.rs fixed-size-block allocator (ACTIVE), falls back to linked_list_allocator
src/task/mod.rs async: Task (pinned boxed future + id), yield_now()
src/task/simple_executor.rs first executor: polls every task in a loop (no wakers)
src/task/executor.rs real executor: task queue + wakers, sleeps with hlt when idle
src/task/keyboard.rs lock-free scancode queue, ScancodeStream, async keyboard task (keys -> tty)
src/thread.rs threads + the multi-core scheduler: spawn, one shared run queue, per-CPU idle
threads, asm context switch, sleep/wake, WaitQueue, join, stack-overflow kills
src/stack.rs kernel stacks: 64 KiB each with an unmapped guard page below, in their own
region (L4 slot 200), slots reused when threads exit
src/smp.rs multi-core bring-up: AP trampoline (real mode -> long mode, in assembly),
INIT/SIPI, per-CPU data (swapgs), CPU numbering, waking idle CPUs
src/apic.rs local APIC: enable, timer (calibrated against the PIT), EOI, IPIs
src/acpi.rs finds the ACPI RSDP/RSDT/MADT to list the CPUs
src/syscall.rs syscall/sysret MSR setup, asm entry stub, dispatch + all syscalls (table at the top)
src/process.rs Process (pid, parent, cwd, args, fd table, exit status), spawn from a path or
ELF bytes, iretq into ring 3, exit/kill, wait (reaping)
src/pci.rs PCI bus scan through config ports 0xCF8/0xCFC, device names
src/block/mod.rs BlockDevice trait, device registry (vda, vda1...), RamDisk (for tests)
src/block/virtio_blk.rs legacy virtio-blk driver: one virtqueue, polled, bounce buffer
src/block/partition.rs MBR + GPT parsing, Partition = a BlockDevice window onto its disk
src/elf.rs minimal ELF64 loader (PT_LOAD segments -> user pages)
src/errno.rs Linux errno numbers
src/tty.rs console line discipline (cooked: echo + line editing; raw: every key, arrows
as escape sequences) + Console (the File behind fds 0/1/2)
src/fs/mod.rs VFS: Inode + FileSystem traits, mount table, path normalize/lookup/create/remove,
symlink following (ELOOP after 8), init (boot files),
mount_disks (ext2, exFAT, btrfs or FAT partitions/disks -> /mnt/diskN)
src/fs/file.rs File trait (what an fd points to), InodeFile (inode + offset), Stat, getdents64
src/fs/ramfs.rs in-memory filesystem
src/fs/fat.rs FAT32 via the fatfs crate: BlockIo adapter + FatFs/FatInode for the VFS
src/fs/ext2.rs hand-written ext2: superblock, groups, bitmaps, inodes, indirect blocks,
directories, symlinks; read + write (the layout is explained at the top)
src/fs/exfat.rs hand-written exFAT: boot checksum, bitmap, FAT chains + NoFatChain, up-case
table, entry sets + SetChecksum; read + write (layout explained at the top)
src/fs/btrfs.rs hand-written read-only btrfs: superblock, chunk tree (single/DUP), root and FS
trees, crc32c on nodes and data, inline/regular/compressed extents, holes,
symlinks, subvolumes (the format is explained at the top)
src/fs/lzo.rs LZO1X decompressor for btrfs's lzo extents
src/fs/pipe.rs pipes: a byte queue with a blocking read end and a write end
build.rs compiles user/*.rs to ELF files, then builds target/disk.img (GPT + FAT32)
and the ext2 images target/ext2.img, ext2-4k.img, ext2-rev0.img (mke2fs -d),
and the exFAT images target/exfat.img, exfat-512.img (mkfs.exfat + populator),
and the btrfs images target/btrfs*.img (mkfs.btrfs --rootdir, btrfs-convert)
with manifests of their files (target/btrfs-*.manifest)
build/exfat_populate.rs build.rs's own little exFAT writer: copies a tree into a fresh mkfs.exfat image
disk/ files copied onto the FAT32 disk (readme.txt, notes/todo.txt,
scripts/selftest.sh = the headless shell session)
disk2/ files copied onto the ext2 disk (hello.txt, readme.txt, docs/...);
build.rs adds big.txt, sparse.bin and two symlinks
disk3/ files copied onto the exFAT disk (long, mixed-case, Unicode names, Photos/...);
build.rs adds numbers.txt and frag-chain.txt
disk4/ files copied onto the btrfs disks (hello.txt, docs/, Music/, projects/ and
archive/ become subvolumes); build.rs adds big.txt, random.bin, zeros.bin,
empty.txt and symlinks
user/ulib.rs tiny user-space "libc": syscall wrappers, println!/eprintln!, a 64 KiB heap
(Vec/String), line reading, strerror, _start, panic handler
user/link.ld linker script for user programs (base 0x80_0040_0000)
user/sh.rs the shell: line editor, builtins, aliases, PATH, pipes, redirection, jobs
user/shparse.rs the shell's parser + drive letters (also compiled into tests/shell.rs)
user/{ls,cat,wc,grep,mkdir,rm,touch,cp,sleep}.rs the shell's tools
user/{hello,evil,spin,demo,child}.rs test/demo programs (all programs go to /mnt/disk0/bin)
user/sysinfo.rs CPUs, per-CPU load bars, threads, processes, memory (the sysinfo syscall)
user/overflow.rs recurses forever: killed with "stack overflow" by the user stack guard zone
src/vga_buffer.rs VGA text-mode writer, print!/println!, print_at (status line), backspace
src/serial.rs COM1 UART writer + serial_print!/serial_println!
tests/ integration tests, one QEMU-booted kernel each
scripts/runner.sh `cargo run`/`cargo test` runner: `bootimage runner` + stale-image fix
scripts/run-headless.sh headless boot + shell self-test, timeout, exit-code and output checks
scripts/screenshot.sh headless boot + a typed shell session + PNG screenshot via QEMU monitor
scripts/ext2-check.sh Rustbelt writes to copies of the ext2 images, then host e2fsck -fn + debugfs
scripts/exfat-check.sh Rustbelt writes to copies of the exFAT images, then host fsck.exfat -n + dump.exfat
scripts/btrfs-check.sh btrfs check --readonly, then tests/btrfs.rs's file CRCs vs. the host's files
boot.png screenshot of the VGA screen (a shell session)
```
## What the interrupt code does (chapters 5-7)
- **CPU exceptions / IDT** (`interrupts.rs`): the IDT maps vector numbers to handler
functions written with `extern "x86-interrupt"` (nightly `abi_x86_interrupt`), so
the compiler saves all registers and returns with `iretq`. `_start` fires an
`int3` on purpose; the breakpoint handler prints the stack frame and execution
continues. A page-fault handler prints the faulting address from CR2.
- **Double faults** (`gdt.rs`): a stack overflow hits the guard page -> page fault ->
the CPU can't push the exception frame on the broken stack -> double fault ->
same again -> *triple fault* = machine reset. We load our own GDT containing a
TSS whose Interrupt Stack Table entry 0 points at a separate 20 KiB static stack,
and mark the double-fault IDT entry to use it. Try `./scripts/run-headless.sh stack-overflow`.
- **Hardware interrupts**: the two 8259 PICs are remapped to vectors 32-47 (0-31 are
CPU exceptions) and then `sti` enables interrupts. The timer (IRQ 0, ~18.2 Hz)
bumps an atomic tick counter and redraws the uptime clock; the keyboard (IRQ 1)
reads the scancode from port 0x60 and queues it; an async task decodes it with
`pc-keyboard` (US layout, scancode set 1) and echoes it (see Multitasking below). Each handler must send End-Of-Interrupt to the
PIC or no further interrupts arrive. `print!`/`serial_print!` disable interrupts
while holding their lock, otherwise an interrupt handler printing while
`_start` holds the lock would deadlock. The idle loop uses `hlt`, so QEMU
doesn't burn a host CPU core.
## Memory (chapters 8-11)
**Paging.** Every address the kernel uses is *virtual*. The CPU translates it with
a 4-level page table tree (level 4 -> 3 -> 2 -> 1 -> 4 KiB page); 9 bits of the
address index each level, the low 12 bits are the offset in the page. CR3 holds the
*physical* address of the level-4 table (here `0x1000`). The `page-fault` feature
writes to `0xdeadbeaf`, which no table maps, and the page-fault handler reports
the address (from CR2) and the reason (`CAUSED_BY_WRITE`, not `PROTECTION_VIOLATION`
-> the page simply isn't present).
**Reaching page tables.** Page tables contain physical addresses, but we can only
dereference virtual ones. With the bootloader's `map_physical_memory` feature, all of
physical RAM is also mapped at `physical_memory_offset` (`0x18000000000` here), so
`virt = offset + phys`. `memory::init` wraps that in the x86_64 crate's
`OffsetPageTable`, which can `translate_addr` (walk the tables) and `map_to` (add entries).
**Virtual memory layout** (as set up by bootloader 0.9 in this build):
| Virtual address | What |
|---|---|
| `0xb8000` | VGA text buffer (identity-mapped: virt = phys) |
| `0x200000`... | kernel code + data (the ELF; maps to phys around `0x400000`) |
| `0x0100_0000_0000`... | kernel stack (80 pages, guard page below it) |
| `0x0180_0000_0000`... | all physical memory (`physical_memory_offset`) |
| `0x4444_4444_0000` | kernel heap, 4 MiB (also holds thread stacks) (we map it in `allocator::init_heap`) |
| `0xdead_beaf_000` | demo page we map to the VGA frame (writes "New!" on screen) |
The "New!" demo writes to screen row 20 through the new mapping; by the time you look,
later lines have scrolled it up, so in `boot.png` it overwrote the start of the first line.
**Physical frames.** `BootInfoFrameAllocator` hands out 4 KiB frames from the regions
the BIOS memory map marks `Usable` (~121 MiB of QEMU's default 128 MiB). It never frees;
fine for now.
**Heap.** `allocator::init_heap` maps the 1024 pages (4 MiB) of `HEAP_START..+HEAP_SIZE` to fresh
frames, then gives the range to the `#[global_allocator]`. From then on the `alloc`
crate (`Box`, `Vec`, `Rc`, `String`, `BTreeMap`...) works. `alloc` is compiled for our
target by `build-std` in `.cargo/config.toml`.
**Allocator designs** (`src/allocator/`): the *bump* allocator just moves a pointer
forward and can only reclaim memory when every allocation is freed (it fails the
`many_boxes_long_lived` test). The `linked_list_allocator` crate keeps a list of free
regions (first fit, slower, but reuses any freed memory). The active
*fixed-size-block* allocator rounds each request up to 8/16/.../2048 bytes and keeps
one free list per size, so alloc/free are O(1) pops/pushes; bigger requests and empty
lists fall back to the linked-list allocator. Switch allocators by editing one line in
`src/allocator.rs`.
## Multitasking (async/await + threads)
### Cooperative: async/await (`src/task/`, phil-opp chapter 12)
An `async fn` compiles into a state machine (a `Future`). An *executor* calls
`poll` on it; the future runs until it has to wait, then returns `Pending` and
remembers a `Waker`. Whoever makes progress possible later (here: the keyboard
interrupt) calls `waker.wake()`, which puts the task's id back into the
executor's queue. When the queue is empty, the executor `hlt`s -- carefully:
it disables interrupts, checks the queue, and then uses `sti; hlt` as one step
(`enable_and_hlt`), so a wakeup arriving in between can't be missed.
The keyboard interrupt handler now does almost nothing: read the scancode from
port 0x60 and push it into a fixed-size lock-free queue (`crossbeam_queue::ArrayQueue`),
then wake the keyboard task (`futures_util::task::AtomicWaker`). No locks in the
interrupt handler means no deadlock if the interrupt arrives while the task
holds one. The task (`print_keypresses`) is an `async` loop over a
`ScancodeStream` that decodes and echoes characters.
"Cooperative" means a task only gives up the CPU at an `.await`. A task that
loops forever without awaiting freezes every other task. Upside: switching is
cheap (just return from `poll`), and all tasks share one stack.
### Preemptive: kernel threads (`src/thread.rs`, beyond the blog series)
Each thread has its own 64 KiB stack (a `KernelStack`, with a guard page
under it). On every local APIC timer interrupt (~100 times a second, on
each CPU) the handler sends the EOI and then calls `thread::schedule()`,
which picks the next ready thread round-robin and calls
`switch_context(&mut old.rsp, new.rsp)` (a dozen lines of `global_asm!`):
```
push rbp, rbx, r12, r13, r14, r15 ; callee-saved regs of the OLD thread, onto its stack
mov [rdi], rsp ; remember where the old stack ends
mov rsp, rsi ; ---- now on the NEW thread's stack ----
pop r15, r14, r13, r12, rbx, rbp ; its callee-saved regs
ret ; return to wherever IT called switch_context
```
Why only those six registers? `switch_context` is called like a normal
function, so the compiler already saved the caller-saved registers, and the
`x86-interrupt` handler around it saved everything the interrupted code was
using (plus the CPU pushed RIP/RSP/RFLAGS). So a paused thread's complete
state sits on its own stack; the scheduler only stores its `rsp`. The new
thread "returns" from `schedule()`, from its timer handler, and `iretq`s back to
where it was interrupted.
A brand-new thread was never interrupted, so `spawn` hand-builds a stack that
looks as if `switch_context` had saved it: six register slots (r12 = the
thread's function) and a return address pointing to `thread_trampoline`, which
moves r12 into rdi and calls `thread_start(entry)`. That re-enables interrupts
(we arrive from inside the timer interrupt, where they're off), runs the
function, and marks the thread finished when it returns. The code that booted
the kernel becomes thread 0, "main", which runs the async executor.
Preemption means even `loop {}` can't hog the CPU (thread C in the demo is a
pure busy loop). The price: any code can be interrupted anywhere, so shared
data needs locks, and a lock that the timer handler might also want must be
taken with interrupts disabled. Here: the print macros and `print_at` already
hold their spinlocks with interrupts off (`without_interrupts`), so a thread
can't be switched away while holding them; the scheduler lock is only taken
with interrupts off and is released *before* switching (the next thread would
otherwise wait forever for a lock held by a paused thread).
**The locking rule:** a lock that is *ever* taken with interrupts off must
*always* be taken with interrupts off. Otherwise: thread T takes the lock with
interrupts on, gets preempted, and some code running with interrupts off (so
it can never be preempted back to T) spins on that lock forever. Stage 6 hit
exactly this with the heap allocator: wait-queue code runs with interrupts off
and allocates, so `GlobalAlloc::alloc`/`dealloc` now take the heap lock inside
`without_interrupts`.
**Blocking (stage 6).** A thread can be *blocked*: the scheduler skips it until
someone wakes it. Two ways to be woken:
- `sleep_ticks(n)` marks the thread sleeping until tick `now + n`; the PIT
handler wakes it once that tick has passed (the `sleep_ms` syscall rounds
up to 55 ms ticks).
- A `WaitQueue` (a list of waiting thread ids). `wait_until(|| condition)`
turns interrupts off, checks the condition, and if it's false puts the
thread on the queue, marks it blocked and switches away; when woken it checks
again. `wake_all()` unblocks everyone on the queue. Checking with interrupts
off is what prevents a *lost wakeup* (the event firing between "check" and
"sleep"). Users: the console (`read(0)` waits for Enter; the keyboard path
wakes it), pipes (reader waits for data or EOF) and `wait(pid)` (parent
waits on the child's `exited` queue).
If every thread is blocked, the scheduler idles with `sti; hlt` until an
interrupt makes something runnable.
**Stack size.** Stage 6 first used 16 KiB stacks and crashed with a ring-0
jump to address 0: the syscall -> VFS -> FAT -> ELF loader path in an
unoptimized debug build used more than that, overflowed into the heap and
smashed a neighbour. Stacks are 64 KiB now, and since stage 11 each has a
real guard page under it (see "Multiple CPUs" below), which replaced the
old canary check.
Stage 11 rewrote the scheduler for several CPUs; the next section explains
what changed.
## Multiple CPUs, sleep/wake and guard pages (stage 11)
Until stage 10, Rustbelt used one CPU. QEMU now runs with `-smp 4`, and the
kernel starts all four.
### Starting the other CPUs (`src/smp.rs`, `src/apic.rs`, `src/acpi.rs`)
At power-on only one CPU runs: the *bootstrap processor* (BSP, CPU 0). The
others (*application processors*, APs) wait, halted, until the BSP sends
them messages through its **local APIC**, a small interrupt controller
built into every CPU core:
1. **Which CPUs exist?** The firmware describes them in ACPI tables. `acpi.rs`
finds the RSDP (a signature in the BIOS area), follows it to the RSDT, then
to the MADT, which lists one entry per CPU with its APIC id.
2. **Trampoline.** An AP starts in 16-bit *real mode*, like an 8086, at an
address below 1 MiB. So `smp.rs` copies a small assembly trampoline to
physical address `0x8000`, plus parameters: the page table (CR3), the
control registers, a fresh kernel stack, the CPU number and the Rust
function to call. The trampoline jumps straight from real mode into
64-bit long mode using the BSP's settings.
3. **INIT, then STARTUP.** The BSP sends an *INIT* inter-processor interrupt
(IPI), waits ~110 ms (two PIT ticks), then a *STARTUP* IPI whose vector
(`0x08`) means "start at `0x8000`"; a second STARTUP if the AP didn't
answer. Then it waits for the AP to say it's ready, one CPU at a time.
4. **AP setup** (`ap_entry`): its own GDT and TSS (with its own double-fault
stack), the shared IDT, the syscall MSRs, its local APIC and timer,
and then it becomes an idle thread in the scheduler.
**Per-CPU data.** Some things must exist once per CPU: the TSS (it holds the
stack used when an interrupt arrives from ring 3), the "current thread", the
syscall entry's scratch slots. `cpu_id()` reads the local APIC id and maps it
to 0..3. The syscall entry can't call Rust before it has a stack, so it uses
`swapgs`: the `KERNEL_GS_BASE` MSR of each CPU points at that CPU's
`PerCpu { syscall_kernel_rsp, syscall_user_rsp }`, and `swapgs` makes
`gs:[0]`/`gs:[8]` reach them.
**Timers.** The old PIT (on the 8259 PIC) still runs, on CPU 0 only: it
keeps the uptime and wakes sleeping threads. Each CPU's local APIC timer,
calibrated against the PIT to ~10 ms, preempts whatever that CPU runs.
**No I/O APIC**: it routes device interrupts to chosen CPUs, but
Rustbelt's devices are polled (virtio-blk) or happy on CPU 0 (keyboard,
PIT), so the 8259 is kept and wired to CPU 0's local APIC (LINT0 in
"ExtINT" mode). Moving to the I/O APIC is a good exercise once
interrupt-driven virtio needs it.
### The scheduler (`src/thread.rs`)
- **One shared ready queue** (a `VecDeque`) under one lock. Each CPU has its
own *current* thread and its own *idle* thread, which runs `sti; hlt` when
there's nothing to do. Simple and fair; per-CPU queues with work stealing
would scale better, but four CPUs don't need them.
- **Thread states:** `Ready`, `Running`, `Blocked` (on a wait queue),
`Sleeping(until tick)`, `Finished`.
- **Handing a thread to another CPU safely.** When CPU 0 switches away from
thread T, T stays marked `on_cpu` until CPU 0 has fully left T's stack
(`finish_switch`, right after the switch). Until then no other CPU may
pick T, or two CPUs would run on the same stack.
- **Sleeping without polling.** `sleep_ticks(n)` marks the thread
`Sleeping(now + n)` and switches away; it's not on the ready queue, so it
costs nothing. The PIT handler wakes threads whose time has come
(`wake_sleepers`) and, if some CPU is idle, sends it a *reschedule IPI*
(vector `0x41`) so it picks the thread up at once instead of at its next
timer tick.
- **Wait queues.** `queue.wait_until(|| condition)` blocks until the
closure returns `Some`; `queue.wake_all()` wakes the waiters. The
classic *lost wake-up* (the event happens between "check" and "block",
possibly on another CPU) is prevented by joining the queue *before*
checking, plus a `woken` flag: a wake that arrives early makes the block
return immediately. Console input, pipes, `wait`, `join` and the async
executor (when it has nothing to do) all use wait queues.
- **Statistics.** Each local APIC tick counts as *busy* or *idle* for that
CPU; the `sysinfo` syscall (502) returns these, and `user/sysinfo.rs`
turns two samples into load bars.
**The locking rule now matters more:** a lock that is ever taken with
interrupts off must always be taken with interrupts off. On one CPU breaking
it deadlocks only sometimes; on four, a thread spinning with interrupts off
on CPU 1 can wait forever for a lock held by a preempted thread. The
scheduler, frame allocator, heap, process table and kernel page-table locks
all follow it.
### Memory cleanup
Everything a thread or process had is given back when it ends:
- A finished thread can't free the stack it's running on, so the *next*
thread on that CPU frees it (`finish_switch`), before removing the thread
from the table. So when `join` returns, the memory really is free.
- A process's address space (its page tables at every level, code, data,
heap, user stack) is freed in `Drop for Process`, when the last `Arc` to it
goes away. `wait` joins the process's thread before reaping, so the parent
holds the last reference.
- Children whose parent exits first are *orphaned* and clean themselves up
when they exit.
`memory::frames_in_use()` counts frames; the headless run and `tests/smp.rs`
check it returns to exactly the same number after hundreds of threads and
processes.
### Guard pages (`src/stack.rs`, `src/process.rs`)
A stack grows down. If it grows past its end it would silently overwrite
whatever lies below. A **guard page** is an address range under the stack
that is deliberately *not mapped*: touching it causes a page fault, so the
overflow is caught at the first byte.
- **Kernel stacks** live in their own region of the address space (L4 slot
200, `0x0000_6400_0000_0000`): 4 KiB guard page, 64 KiB stack, guard page,
stack, ... When a kernel stack overflows, the CPU can't even push the
page-fault frame (it would land in the guard page too), so it raises a
*double fault*, which runs on its own IST stack. That handler sees the
faulting address is a guard page and prints
`[kernel] kernel stack overflow in thread 'NAME' (hit guard page 0x...)`,
then kills that thread (or its process) and keeps going.
- **Stack probes.** A function with a big local array could jump *over* a
4 KiB guard page. `"stack-probes": {"kind": "inline"}` in the target JSON
makes the compiler touch each page of a big frame in order, so it can't.
- **User stacks** (16 KiB) get a 64 KiB never-mapped zone underneath, and
the ELF loader refuses programs whose segments reach into it. A fault
there kills the program with "stack overflow" (exit code 139). Try
`overflow` in the shell.
Kernel stacks are reused: a freed stack's slot goes on a free list and its
page tables stay, so they never need freeing. **Why no TLB shootdown?** When
a stack is freed, other CPUs may still have its old addresses cached in their
TLB (the CPU's cache of page translations). Normally the freeing CPU would
send them all an IPI to flush it. Here nothing can use a freed slot until a
*new* thread gets it, so each thread instead flushes its stack's pages
(`invlpg`) on each CPU the first time it runs there. Cheaper and simpler,
and only possible because nobody else ever touches a thread's stack.
Known simplifications (good exercises): no priorities; one shared run
queue; no CPU affinity; device interrupts all go to CPU 0.
## User mode, syscalls and processes
The user programs in `user/` run in **ring 3** as separate processes, each
with its own address space and file-descriptor table. The shell and its
tools are described in [The shell](#the-shell-stage-7). These are the test
and demo programs from stages 5 and 6 (run them from the shell: `run hello`,
`run evil`...):
| Program | What it does |
|---|---|
| `hello` | prints "Hello from user mode!", `getpid`, reads `/hello.txt` from the ramfs, creates and reads back `/tmp/note.txt`, checks error cases (`ENOENT`, `EBADF`, `EFAULT`, `ENOSYS`), exits with the number of failed checks (0) |
| `evil` | reads kernel memory at `0x200000` -> page fault in ring 3 -> the kernel kills it and keeps going |
| `spin` | busy-loops ~2 s with no syscalls; it still gets preempted (threads A/B/C keep printing) |
| `demo` | stage 6 checks: cwd, reads `/mnt/disk0/readme.txt`, `stat`/`fstat`/`lseek`, lists `bin/` with `getdents64`, `chdir` errors, `mkdir`/`unlink`/`rmdir` on ramfs and FAT, a pipe from a child, arguments, `wait` exit codes (42, 139 for killed `evil`), `ECHILD`/`ENOENT`/`ENOEXEC`, `sleep_ms`; exits with the number of failed checks |
| `child` | helper for `demo` and tests: `child pipe` writes a line to stdout, `child cat` copies stdin to stdout, `child 42` exits with 42, otherwise prints its arguments |
All of them are read from `/mnt/disk0/bin/`, on the FAT32 partition of the
virtio disk; they're not compiled into the kernel any more.
### Rings in one paragraph
The CPU's *current privilege level* (CPL) is the low 2 bits of the CS register:
0 = kernel, 3 = user. In ring 3 the CPU refuses privileged instructions (`hlt`,
`cli`, writing CR3, `in`/`out`...) and refuses to touch any page whose page-table
entries lack the `USER_ACCESSIBLE` bit. User code can only get back into ring 0
through doors the kernel set up: interrupts/exceptions (via the IDT) and the
`syscall` instruction (via the LSTAR MSR). Every such door switches to a kernel
stack, so a user program can't corrupt kernel state through its own stack pointer.
### GDT and TSS (`src/gdt.rs`)
```
0x00 null
0x08 kernel code (ring 0) 0x10 kernel data (ring 0)
0x18 user data (ring 3) |3 0x20 user code (ring 3) |3
0x28 TSS (16 bytes: two slots)
```
The odd order (user *data* before user *code*) is forced by `sysret`: it loads
SS = STAR[63:48] + 8 and CS = STAR[63:48] + 16, so the two user segments must
be adjacent in that order. The TSS has two jobs: IST[0] is the double-fault
stack (stage 2), and **`rsp0`** is the stack the CPU switches to when an
interrupt arrives while in ring 3. Each thread has its own kernel stack, so the
scheduler writes the next thread's stack top into `TSS.rsp0` on every switch
(`gdt::set_kernel_stack`). Since stage 11 every CPU has its own GDT and TSS
(the TSS is per CPU by nature: it says where *this* CPU's interrupts land).
### The syscall path (`src/syscall.rs`)
`syscall` was chosen over `int 0x80`: it's what Linux uses on x86_64, it's
faster, and the setup is only four MSRs. The catch is that, unlike an
interrupt, `syscall` does **not** switch stacks -- the entry code must.
Setup (`syscall::init`): `EFER.SCE = 1` (enable), `STAR` = the selectors above,
`LSTAR` = address of `syscall_entry`, `SFMASK = IF|DF|TF` (so we enter with
interrupts off until we're on a kernel stack).
One call, step by step:
1. User code puts the number in `rax`, args in `rdi, rsi, rdx` (Linux ABI; we
also save `r10, r8, r9`) and executes `syscall`.
2. CPU: `rcx` = user RIP, `r11` = user RFLAGS, CS/SS = kernel, RFLAGS &= ~SFMASK,
RIP = LSTAR. RSP is still the *user* stack.
3. `syscall_entry` (asm): `swapgs` to reach this CPU's `PerCpu` data,
stash user RSP there, load this thread's kernel stack top (set by the
scheduler), push user RSP, `swapgs` back, push `rcx`, `r11` and the args.
4. Shuffle into the C ABI and call `dispatch(nr, a1..a5)` in Rust with
interrupts re-enabled (so a long syscall, e.g. a blocking `read`, can be preempted).
5. `dispatch` validates user pointers (`user_slice`: range must be inside the
user half *and* mapped with `USER_ACCESSIBLE` in the current page table),
runs the call, and returns a value or `-errno` in `rax`.
6. `cli`, pop everything back, restore user RSP, `sysretq` (RIP = rcx,
RFLAGS = r11, CPL = 3).
With several CPUs, the "current kernel stack" and "saved user RSP" can't be
globals (two CPUs can be in a syscall at once), so they're per-CPU, reached
with `swapgs` (see "Multiple CPUs" below).
Numbers are Linux's wherever Linux has the same call, so man pages and
`strace` habits mostly apply. Args 4 and 5 go in `r10` and `r8` like Linux.
| nr | name | args | notes |
|---|---|---|---|
| 0 | `read` | fd, buf, len | console: blocks until Enter; pipe: blocks until data or EOF |
| 1 | `write` | fd, buf, len | console = VGA + serial; pipe: `EPIPE` once the reader is gone |
| 2 | `open` | path, path_len, flags | `O_CREAT` 0o100, `O_TRUNC` 0o1000, `O_APPEND` 0o2000 (every write goes to the end: `>>`); directories can be opened (for `getdents64`) |
| 3 | `close` | fd | |
| 4 | `stat` | path, path_len, *Stat | follows symlinks |
| 5 | `fstat` | fd, *Stat | |
| 8 | `lseek` | fd, offset, whence | `SEEK_SET/CUR/END` = 0/1/2; `ESPIPE` on console/pipes |
| 16 | `ioctl` | fd, request, arg | only `TTY_SET_RAW` (1): console raw mode on (arg 1) / off (0); returns the old mode, `ENOTTY` if fd isn't the console |
| 22 | `pipe` | *[u32; 2] | [read end, write end] |
| 24 | `sched_yield` | | |
| 35 | `sleep_ms` | ms | **ours**: Linux nr 35 is `nanosleep(timespec)` |
| 39 | `getpid` | | |
| 60 | `exit` | code | |
| 61 | `wait` | pid, flags | **simpler than Linux `wait4`**: blocks, returns the exit code (139 = killed, like a shell shows SIGSEGV), reaps the child (freeing its memory); `ECHILD` if not your child; flags `WNOHANG` (1): don't block, `EAGAIN` while it's still running |
| 63 | `uname` | *utsname | Linux's struct: 6 NUL-padded 65-byte fields: "Rustbelt", "qemu", "0.11.0", "stage 11: core polish", "x86_64", "(none)" |
| 79 | `getcwd` | buf, len | returns the length (no NUL); `ERANGE` if too small |
| 80 | `chdir` | path, path_len | per-process cwd, inherited by spawned children |
| 83 | `mkdir` | path, path_len | |
| 84 | `rmdir` | path, path_len | empty directories only (`ENOTEMPTY`); `EBUSY` on a mount point |
| 87 | `unlink` | path, path_len | files and symlinks (the link, not its target); directories give `EISDIR` |
| 89 | `readlink` | path, path_len, buf, len | a symlink's target (no NUL); returns its length, cut to `len` like Linux; `EINVAL` if not a symlink |
| 217 | `getdents64` | fd, buf, len | real Linux `struct linux_dirent64` records (ino, off, reclen, type, name\0) |
| 500 | `spawn` | path, path_len, argv, argv_len, *SpawnOptions | **ours**: start a program, returns its pid |
| 501 | `mounts` | buf, len | **ours** (Linux reads `/proc/mounts`): the mount table as text, one `path fstype` line per mount, in mount order; returns the length, `ERANGE` if too small. The shell assigns drive letters from it |
| 502 | `sysinfo` | *SysInfo | **ours**: CPU count, the CPU you're on, thread/process/frame counts, uptime ticks, and per-CPU timer ticks and busy ticks (sample twice for the load) |
Deviations from Linux worth knowing:
- **Paths are pointer + length**, not NUL-terminated C strings (easier to
validate, and Rust `&str`s aren't NUL-terminated). Relative paths start at
the process's cwd; `.` and `..` are resolved in the kernel (`fs::normalize`).
- **`Stat` is tiny**: `#[repr(C)] struct Stat { kind: u64, size: u64, mode: u64 }`,
kind 1 = file, 2 = directory, 3 = console, 4 = pipe, 5 = symlink; `mode`
is the permission bits (`0o644`), or 0 on file systems without them
(ramfs, FAT). No inode numbers, owners or times yet.
- **spawn instead of fork + exec.** `fork` copies the whole address space (or
needs copy-on-write page tables to be fast) and `exec` immediately throws the
copy away. `spawn` does the useful part in one step, like Windows
`CreateProcess` or POSIX `posix_spawn`. What Unix shells do between fork and
exec -- pointing fds 0/1/2 at a pipe or a file -- is spawn's `SpawnOptions`:
`{ stdin, stdout, stderr: i64 (one of *my* fds, -1 = same as mine), flags }`.
`flags = SPAWN_TRACE` (1) prints every syscall the child makes, as
`[trace pid] open("notes/todo.txt", 0o0) = 3` in strace style (strings
shown, errors as `-2 ENOENT`), which is what the shell's `run --trace`
uses. fork + exec could still be added later.
- **argv** is passed as one buffer with NUL-separated arguments
(`"child\0with\0three args"`; empty = just the program name). The kernel
copies the strings onto the new process's stack and starts `_start` with
`rdi = argc`, `rsi = pointer to argc [ptr, len] pairs` (Rust-friendly
slices instead of C's `char **argv`). `ulib` wraps that as `argc()`/`arg(i)`.
Errors are negative errno values like Linux (`-2` ENOENT, `-9` EBADF, `-14`
EFAULT, `-38` ENOSYS...). Unknown numbers return `-ENOSYS`.
### Address spaces (`src/memory.rs`, `src/process.rs`)
Each process gets a fresh level-4 page table. The kernel's L4 entries are
**copied** into it (so the kernel, heap, physical-memory map and thread stacks
are mapped in every address space and the kernel keeps working after a CR3
switch), and L4 slot 1 is the process's own:
```
0x0000_0000_0020_0000 kernel code (bootloader 0.9 puts it here) shared, ring 0 only
0x0000_0080_0000_0000 USER_START (L4 slot 1) per process
0x0000_0080_0040_0000 program (.text R-X, .rodata R--, .data/.bss RW-)
0x0000_00ff_ffff_b000 user stack (16 KiB, RW-, grows down)
0x0000_0100_0000_0000 USER_END
0x0000_0100_0020_0000 boot stack (L4 slot 2) shared
0x0000_0180_0000_0000 all physical memory (phys_mem_offset, slot 3) shared
0x0000_4444_4444_0000 kernel heap (thread kernel stacks live here) shared
```
Real kernels put the kernel in the upper half (`0xffff_8000_...`) and give
user space the whole lower half. With bootloader 0.9 the kernel sits low, so
here user space is one 512 GiB slot that the kernel doesn't use. Only user
pages have `USER_ACCESSIBLE`, which is why `evil` faults: the page at
`0x200000` *is* mapped, just not for ring 3 (`PROTECTION_VIOLATION | USER_MODE`).
`.text` is not writable and everything else is `NO_EXECUTE` (when EFER.NXE is on).
A **process** = a kernel thread (`thread.rs`, it already has a kernel stack and
gets preempted) + a CR3 value + a `Process` struct (pid, name, fd table, exit
status). The scheduler, on each switch, sets `TSS.rsp0` and the syscall kernel
stack to the next thread's stack and writes CR3 if it differs. A new process's
thread starts in the kernel, then `iretq`s into ring 3 with a hand-built frame
(SS=user data, RSP=user stack, RFLAGS=IF, CS=user code, RIP=ELF entry).
When ring-3 code faults (page fault with `USER_MODE`, general protection fault
or invalid opcode with CS RPL 3), the handler calls `process::exit_current`
instead of panicking: the process is marked `Killed`, its thread exits, the
scheduler picks someone else. A fault in ring 0 still panics -- that's a kernel bug.
### ELF loading and building user programs (`src/elf.rs`, `build.rs`, `user/`)
`build.rs` compiles each `user/*.rs` with plain `rustc` for the built-in
`x86_64-unknown-none` target (no build-std needed: it ships a precompiled
`core` and uses soft-float, so no SSE state to save). Flags worth knowing:
`-C relocation-model=static` (no dynamic loader), `-C code-model=large`
(the program lives above 4 GiB; the default model assumes 32-bit addresses
and fails with "relocation overflow"), `-T user/link.ld` (one PT_LOAD
segment per permission, page-aligned). The large code model names its
sections `.ltext`/`.lrodata`/`.ldata`/`.lbss`, so the linker script lists
those too -- forgetting that put the code into a non-executable segment and
every program died at its first instruction. The ELFs (~15-20 KB) used to be
embedded in the kernel with `include_bytes!`; since stage 6 `build.rs` copies
them onto the FAT32 disk image instead and `process::spawn_path` reads them
through the VFS (see "Disks" below).
`user/ulib.rs` is a tiny libc-equivalent included by each program: the
`syscall` inline asm, wrappers, `print!/println!/eprintln!`, `_start` (calls
`main`, then `exit` with its return value) and a panic handler (`exit(101)`).
Since stage 7 it also has a **heap**: a 64 KiB array in `.bss` with a
power-of-two size-class allocator (free list per size) as the
`#[global_allocator]`. A program that says `extern crate alloc;` can use
`Vec`, `String` and `format!`; the `x86_64-unknown-none` target ships a
precompiled `alloc` too.
The loader (~160 lines, no crate) checks the header (magic, 64-bit, little
endian, x86_64, `ET_EXEC`), and for every `PT_LOAD` segment: checks it's
inside the user range, maps zeroed frames with flags from the segment
(`W` -> writable, no `X` -> NX), copies `filesz` bytes through the
physical-memory map and leaves the rest (`.bss`) zero. It refuses (`ENOEXEC`)
a file where two segments with different permissions share a page. A page
has one set of permissions, so the second segment would silently get the
first one's. This happened for real in stage 7: `sleep` has no `.data`, so
the linker put its `.bss` on the read-only `.rodata` page, and `sleep` died
on its first write to a static. `user/link.ld` now aligns the location
counter before `.data`, not just the (empty, hence dropped) section.
### VFS (`src/fs/`) and file descriptors
Designed like Linux's so real filesystems can plug in later:
- `trait Inode` (`fs/mod.rs`): a file, directory or symlink: `kind`, `size`, `mode`, `readlink`,
`read_at(offset, buf)`, `write_at`, `truncate`, `lookup(name)`,
`create(name, kind)`, `list` (names + kinds), `remove(name)`.
Methods have default impls returning an error, so a read-only fs implements
only what it supports.
- `trait FileSystem`: `name()` and `root() -> Arc<dyn Inode>`. A disk filesystem
will hold its block device and parsed superblock.
- **Mount table**: `mount("/tmp", fs)`; `lookup(path)` picks the longest mount
point that prefixes the path, then walks the rest with `Inode::lookup`.
Boot mounts a ramfs on `/` (`/hello.txt`, `/etc/motd`, `/tmp`, `/mnt`), a
second, separate ramfs on `/tmp`, and each FAT partition at `/mnt/diskN`.
`fs::normalize(cwd, path)` turns any path into a clean absolute one first.
- `trait File` (`fs/file.rs`): what a file descriptor points to: `read(buf)`,
`write(buf)`, `seek`, `stat`, `getdents`. `InodeFile` = an inode + its own offset (two opens of the same
file have separate offsets, like Linux's `struct file`). `tty::Console` is
another `File`: writes go to VGA+serial, reads come from the keyboard line
buffer. `fs/pipe.rs` gives two more: a pipe's read and write ends.
- **fd table**: `Vec<Option<Arc<dyn File>>>` per process (max 16), fds 0-2 =
Console. `open` = `fs::lookup` (or `fs::create` with `O_CREAT`) + `InodeFile::open`
+ first free slot.
Keyboard input for `read(0)` goes through a small tty line discipline
(`src/tty.rs`), like Linux's termios:
- **Cooked mode** (the default): the kernel echoes the characters and
Backspace edits the current line. Enter makes the line, with its `\n`,
available to readers. Programs like `cat` with no arguments see this.
- **Raw mode** (`ioctl(0, TTY_SET_RAW, 1)`): every key goes to the reader at
once and nothing is echoed. Backspace is `\x08`, and the arrows, Home, End
and Delete arrive as ANSI escape sequences (`ESC [ A` = up...). The shell
uses this while you edit a line, so it can do the echo and history itself.
It switches back to cooked before running a program. Raw mode is also
switched off automatically if its owner process exits.
The console understands two control characters on output. `\x08` moves back
one column and blanks it, and `\x0c` (form feed) clears the screen, which is
what `cls` writes.
**Process cleanup.** When `wait` reaps a process, `memory::free_address_space`
walks L4 slot 1 (the user half) and frees every page, page table and the L4
table. Freed frames go on a free list that `alloc_frame` uses first. The list
is an intrusive linked list: each free frame stores the address of the next
one, written through the physical-memory map, so it needs no heap. The
exiting process disables interrupts before publishing its exit status. That
way the parent can't free the address space while the dying thread might
still run on it.
Known simplifications (exercises): a process nobody waits for keeps its memory; orphans aren't reparented; no signals; user pointer
checks walk the page table but don't guard against another thread unmapping
the page mid-syscall (there's no `munmap` yet, so that's fine for now).
## Disks: PCI, virtio-blk, partitions, FAT32 (stage 6)
```
/mnt/disk0/readme.txt
VFS (fs/mod.rs) longest mount prefix -> FatFs, rest "readme.txt"
FatInode (fs/fat.rs) fatfs crate: clusters, FAT chains, directory entries
BlockIo adapter byte offsets -> whole 512-byte sectors
Partition vda1 sector + 2048, bounds-checked
VirtioBlk vda virtqueue request to QEMU
PCI found the device and its I/O port at boot
```
**PCI** (`src/pci.rs`). Every PCI device has a 256-byte configuration space
(vendor/device id, class code, BARs = where its registers live). On a PC you
read it by writing `bus/device/function/offset` to port `0xCF8` and reading
port `0xCFC`. `pci::scan()` tries every bus/device/function, keeps the ones
whose vendor id isn't `0xFFFF`, and prints them with a class name. QEMU's
default machine has 7: host bridge, ISA/IDE/ACPI parts of the PIIX chipset,
VGA, the e1000 network card, and our disk, `1af4:1001`.
**Why virtio-blk and not ATA PIO?** ATA PIO is simpler (a few port writes,
then 256 `in` instructions per sector) but it's a 1980s interface that only
emulators and old hardware have. virtio is what QEMU/KVM and cloud VMs
actually use, and it teaches the pattern every modern device uses: the driver
and device share **queues of descriptors in RAM**. The legacy (virtio 0.9.5)
interface is used, through I/O port BAR0, because it needs no MMIO mapping or
PCI capability parsing; QEMU's device is "transitional" and speaks both.
**virtio-blk in one paragraph** (`src/block/virtio_blk.rs`, ~280 lines).
Reset the device, set ACKNOWLEDGE + DRIVER, accept no optional features,
read the queue size, allocate the virtqueue (descriptor table + available
ring + used ring, physically contiguous: `memory::alloc_contiguous_frames`)
and give its page number to the device, set DRIVER_OK. A request is a chain
of three descriptors: a 16-byte header (`IN` = read or `OUT` = write, sector
number), the data buffer, and a 1-byte status the device fills in. Put the
chain's head in the available ring, write the queue number to the notify
port, then **poll** the used ring until the device hands it back. Data goes
through a one-page **bounce buffer** whose physical address we know (heap
buffers are virtually contiguous but not physically). Interrupts are switched
off for the device (`VRING_AVAIL_F_NO_INTERRUPT`); polling is fine because QEMU
completes requests almost instantly. Exercises: IRQ-driven completion (block
the thread on a wait queue), several requests in flight, virtio 1.0 "modern"
MMIO.
**`BlockDevice`** (`src/block/mod.rs`): `sector_count()`,
`read_sectors(lba, buf)`, `write_sectors(lba, buf)`, buffers a whole number of
512-byte sectors. Devices get Linux-style names in a registry
(`block::devices()`, `block::get("vda1")`). `RamDisk` is a `Vec<u8>` disk used
by tests.
**Partitions** (`src/block/partition.rs`). Sector 0 is the MBR: four 16-byte
entries at offset 446 and the `55 AA` signature. A single entry of type `0xEE`
is a "protective MBR" meaning *look for a GPT*: sector 1 has the `EFI PART`
header, which says where the partition entry array is (128 bytes each: type
GUID, unique GUID, first/last LBA, UTF-16 name). Each partition becomes a
`Partition` -- itself a `BlockDevice` that adds its start sector and refuses
to go past its end -- registered as `vda1`, `vda2`... CRCs aren't checked yet.
**FAT32 via the `fatfs` crate** (`src/fs/fat.rs`). `fatfs` does the
filesystem logic; it wants something implementing its `Read + Write + Seek`
traits on *bytes*. `BlockIo` is that adapter: it keeps a byte position, sends
whole aligned sectors straight to the device, and handles partial sectors
through a one-sector buffer (read the sector, change the bytes, write it back).
Writes go straight through, so `flush` has nothing to do. (`fatfs` 0.3 from
crates.io depends on the abandoned `core_io` crate, which doesn't build on
current nightlies, so `Cargo.toml` pins a git revision of the 0.4 branch with
its own `no_std` IO traits.) For the VFS, `FatFs` holds the `fatfs::FileSystem`
in an `Arc<Mutex<..>>`, and a `FatInode` is just *(that filesystem, a path, a
kind)*: each call reopens its path from the root. Slower than caching
directory entries, but simple and never stale. Lookups are case-insensitive,
like FAT itself. No timestamps (`NullTimeProvider`) and long file names on.
**The test disk** (`build.rs`). After compiling the user programs,
`build.rs` makes `target/disk.img` (64 MiB): `sfdisk` writes a GPT with one
"Microsoft basic data" partition at sector 2048, `mkfs.vfat -F 32 -s 1 -n
TYLERDISK` formats a separate partition image (1 sector per cluster so 62 MiB
is enough clusters for FAT32), `mmd`/`mcopy` (mtools, which edit FAT images
without mounting, so no root) copy in `disk/` and `bin/<programs>`, and the
partition image is spliced into the disk at sector 2048. If a tool is missing
the build stops with `sudo apt install dosfstools mtools fdisk`
(`mkfs.vfat` lives in `/usr/sbin`, which isn't in a normal user's `PATH` on
Debian, so `build.rs` looks there too). Look inside it from Linux:
```sh
/usr/sbin/sfdisk -l target/disk.img # not in a normal user's PATH on Debian
export MTOOLS_SKIP_CHECK=1 # mtools: don't complain about the small-cluster FAT32
mdir -i target/disk.img@@1M ::/bin # @@1M = partition offset (2048 * 512)
mtype -i target/disk.img@@1M ::/readme.txt
```
## The shell (stage 7)
`/bin/sh` on the disk (`user/sh.rs`, about 550 lines) is an ordinary ring-3
program. It uses nothing but the syscalls above. Its design is a **hybrid**:
Unix navigation and pipes, with DOS-style program running and drive letters
on top. At boot a kernel thread starts it on the console and waits for it.
When it exits, the thread starts a new one, the way `init` respawns a login
shell.
```
Rustbelt shell. Type `help` for commands. Drives: A: = /, B: = /mnt/disk0, C: = /mnt/disk1
A:/ $ cd b:
B:/ $ ls
bin/ notes/ readme.txt scripts/
B:/ $ type readme.txt | grep -i rustbelt
In the Rustbelt shell this disk is drive B: -- try `cd B:` and `ls`.
B:/ $ ls bin | wc -l
15
B:/ $ echo first > A:/tmp/a.txt
B:/ $ echo second >> A:/tmp/a.txt
B:/ $ wc < A:/tmp/a.txt
2 2 13
B:/ $ run --trace cat notes/todo.txt
[trace 8] open("notes/todo.txt", 0o0) = 3
[trace 8] read(3, "Done: the shell (cd, ls, cat, run, start"..., 512) = 125
Done: the shell (cd, ls, cat, run, start, pipes, redirection, drive letters).
Next: more file systems -- ext2, exFAT, btrfs.
[trace 8] write(1, "Done: the shell (cd, ls, cat, run, start"..., 125) = 125
[trace 8] read(3, "", 512) = 0
[trace 8] close(3) = 0
[trace 8] exit(0)
B:/ $ start sleep 5
[pid 9] sleep started in the background
B:/ $ jobs
[pid 9] running start sleep 5
B:/ $ run evil
[evil] pid 10: reading kernel memory at 0x200000...
[process] pid 10 (evil) killed: page fault
[exit code 139]
B:/ $ ver
Rustbelt 0.8.0 (stage 8: ext2) x86_64
B:/ $ <- Enter on an empty line, a few seconds later
[done] pid 9 (start sleep 5), exit code 0
B:/ $
```
### Commands
| Command | What it does |
|---|---|
| `cd [dir]` | change directory (`cd` alone = `A:/`); `cd B:` goes to the root of drive B |
| `pwd` | print the current directory as a Unix path (the prompt shows the drive form) |
| `echo text...` | print the arguments |
| `ver` | kernel name and version, from the `uname` syscall |
| `drives` | the drive letters and what they map to |
| `alias [name=value]` | list aliases or define one: `alias ll="ls -l"` |
| `history` | the commands typed so far |
| `jobs` / `wait [pid]` | list background jobs / wait for them (all, or one pid) |
| `cls`, `clear` | clear the screen |
| `help` | the command list |
| `exit [code]` | leave the shell (with the last command's exit code, or `code`) |
| `run [--trace] prog args` | run a program in the foreground; `--trace` prints every syscall it makes |
| `start prog args` | run it in the background and print its pid (same as `prog args &`) |
| `prog args` | a bare name is looked up in each drive's `bin` directory, in order: `A:/bin`, `B:/bin`... |
Programs in `B:/bin` (`user/*.rs`, each 30-150 lines):
| Program | What it does |
|---|---|
| `ls [-l] [path...]` | list a directory, sorted; in columns on the console, one name per line into a pipe or file (like real `ls`); symlinks get an `@`; `-l` shows sizes and `<DIR>`, plus permissions (`-rw-r--r--`) and `link -> target` on ext2 |
| `cat [file...]` | print files, or stdin (also for `-`) |
| `wc [-l] [file...]` | lines, words, bytes |
| `grep [-i] [-v] [-n] text [file...]` | lines containing `text` (plain text, no regexes); exit code 0 = found, 1 = not found |
| `mkdir dir...` | create directories |
| `rm path...` | delete files or empty directories |
| `touch file...` | create empty files |
| `cp src dest` | copy a file (into `dest` if it's a directory) |
| `sleep N` / `sleep 250ms` | wait (blocks in the kernel, no busy loop) |
| `hello`, `demo`, `evil`, `spin`, `child` | the stage 5/6 test programs: `run demo` runs all the stage 6 syscall checks |
Built-in aliases are the DOS names: `dir` = `ls`, `type` = `cat`,
`del` = `rm`, `md` = `mkdir`, `copy` = `cp`.
**Line editing**: Backspace, and Up/Down to browse the history. Left/Right
and Tab aren't supported yet. **Scripts**: `sh file` runs the commands in a
file and echoes each one after its prompt. If stdin isn't the console
(`sh < file`, `cat file | sh`), the shell reads commands from it the same
way. `#` starts a comment.
### Pipes and redirection
`a | b | c`, `> file` (create or truncate), `>> file` (append) and `< file`
can be combined: `cat < in.txt | grep x | wc -l > count.txt`. The operators
don't need spaces around them (`ls|wc`). Quotes (`'...'` or `"..."`) group
words and hide operators. There are no variables or escapes.
How a pipeline runs (`Shell::run_pipeline`): for each stage, the shell picks
its stdin and stdout and spawns it.
- **stdin**: the `< file`, else the read end of the previous stage's pipe,
else the shell's own stdin.
- **stdout**: a new `pipe()` to the next stage, overridden by `> file`, else
the shell's own stdout.
- **Spawning**: the stage starts with `spawn(path, argv, SpawnOptions { stdin,
stdout, ... })`.
Then the shell **closes its own copies** of all pipe ends and files. A reader
only sees end-of-file once every write end is closed, and the shell holds
one. Last, it waits for every stage; the last stage's exit code is the
pipeline's. A builtin can be a stage too (`help | grep run`): it just writes
to the pipe. Pipes have no size limit here, so that can't block.
### Drive letters
Drive letters are only shell syntax. The kernel keeps one Unix tree:
- **Why start at A:?** In DOS and Windows the system drive is `C:` only for
historical reasons. `A:` and `B:` were reserved for floppy drives, so the
first hard disk got `C:`, and Windows still keeps that habit. Rustbelt has
no floppies, so its letters simply start at the front of the alphabet.
(The first version of the shell used `C:` for `/`.)
- **Mapping**: `A:` is `/`. `B:`, `C:`... go to the mounts under `/mnt`,
in the order they were mounted. The shell reads them from the `mounts`
syscall before every prompt, so a newly mounted disk would get a letter
right away. Other mounts, such as `/tmp`, stay under A: (`A:/tmp`).
- **Translation**: every unquoted word that starts with a known letter is
translated before any syscall sees it (`shparse::translate`):
- `B:/notes/todo.txt` becomes `/mnt/disk0/notes/todo.txt`.
- `b:` becomes `/mnt/disk0` (any case).
- `B:notes` becomes `/mnt/disk0/notes`, since drives don't have their own
current directory.
Quote a word to keep it literal: `grep "B:" readme.txt`.
- **Prompt**: the prompt goes the other way (`shparse::display`). The longest
matching drive path wins, so `/mnt/disk0/bin` shows as `B:/bin`.
- **Unknown letters**: these are left alone, so `cd Q:` says
`there is no drive Q:`.
- **Stable letters**: since letters follow mount order, `Cargo.toml` and
the scripts attach the user-visible disks first (B: FAT32, C: ext2,
D: exFAT, E: btrfs) and the test-only images after them. A disk that
fails to mount (like the sha256 btrfs test image) gets no `/mnt/diskN`
and no letter, so it can't shift the others either.
### How the pieces fit
- `user/shparse.rs`: the tokenizer, the parser (`Pipeline` of `Command`s,
each with words and redirections) and the drive-letter functions. All of
them are pure functions with no syscalls. The same file is compiled into
the shell *and* into `tests/shell.rs` (`#[path = "../user/shparse.rs"]`),
so the parser is unit-tested inside the kernel test suite.
- `user/sh.rs`: the main loop runs these steps: refresh the drives, report
finished background jobs (`wait` with `WNOHANG`), print the prompt, read a
line, then run it. Before running, each command goes through `prepare()`:
it translates drive letters, expands aliases, and strips `run` /
`run --trace` / `start` prefixes. The command then runs as a builtin or is
spawned with the right fds.
- **Kernel support added for the shell**:
- `ioctl` (raw mode), `uname`, `mounts`, `wait` flags and `O_APPEND`.
- Freeing process memory, so a long session doesn't run out of frames.
- strace-style `--trace` output.
- The `\x08` and `\x0c` console controls.
- A fix for the `|` key (see Notes).
## ext2 (stage 8)
The third disk, `target/ext2.img`, holds an **ext2** file system that
Rustbelt reads and writes with its own driver, `src/fs/ext2.rs` (about 1100
lines including a long explanation at the top, no crates). It is mounted at `/mnt/disk1`,
which is drive **`C:`** in the shell:
```
A:/ $ drives
A: / ramfs
B: /mnt/disk0 fat32
C: /mnt/disk1 ext2
A:/ $ cd c:
C:/ $ ls -l
-rw-r--r-- 304000 big.txt
drwxr-xr-x <DIR> docs/
lrwxrwxrwx hello-link -> hello.txt
-rw-r--r-- 142 hello.txt
drwx------ <DIR> lost+found/
-rw-r--r-- 284 readme.txt
-rw-r--r-- 69206016 sparse.bin
C:/ $ cat hello-link
Hello from the ext2 disk! This file lives on a second virtio disk, in an
ext2 file system that Rustbelt reads and writes with its own driver.
C:/ $ cp B:/notes/todo.txt C:/
C:/ $ cat todo.txt | grep -i ext2
Next: more file systems -- ext2, exFAT, btrfs.
```
### The on-disk layout, for a learner
ext2 (1993) was Linux's main file system through the 1990s, and ext3 and
ext4 grew out of it. It is the classic file system to learn from, because
everything on disk is an **array of fixed-size records** that you find with
plain arithmetic. You can follow along with `dumpe2fs target/ext2.img` and
`debugfs target/ext2.img` (both in `/usr/sbin`).
- **Blocks and groups.** The disk is cut into blocks of 1, 2 or 4 KiB, and
the blocks into **block groups** (8192 blocks each with 1 KiB blocks).
Our 32 MiB disk has 1 KiB blocks, so it has 4 groups. Each group keeps
its own bitmaps and inode table, so a file's inode and its data usually
sit close together.
```
byte 0 1024 2048 (block 2)
| boot block | superblock | group descriptors | reserved | block bitmap | inode bitmap | inode table | data... <- group 0
| superblock + descriptor backups | block bitmap | inode bitmap | inode table | data... <- group 1
```
- **Superblock** (always at byte 1024, whatever the block size). It holds
the totals: block size (`1024 << s_log_block_size`), blocks and inodes
per group, the free counts, the magic number `0xEF53`, the revision and
three sets of **feature flags**. `probe()` reads just that magic, and
`mount_disks` tries ext2 first, then FAT.
- **Feature flags.** There are three sets, and each set tells an old driver
what to do with a flag it doesn't know:
- **compat**: ignore it.
- **ro_compat**: mount read-only (Rustbelt then says `read-only` in the
boot log, and writes give `EROFS`).
- **incompat**: don't mount at all. ext4's extents are one, so Rustbelt
refuses ext4 instead of misreading it.
- **Group descriptors** (32 bytes per group, in the block after the
superblock). Each says where that group's block bitmap, inode bitmap and
inode table are, and how many blocks and inodes are free there.
- **Bitmaps.** One bit per block or per inode, 1 = in use. Allocating is
"find a 0 bit, set it, decrement the free count in the group descriptor
*and* the superblock". Rustbelt allocates a file's blocks in the group of
its inode first.
- **Inodes.** Inode N is at index `(N-1) % inodes_per_group` of group
`(N-1) / inodes_per_group`'s table. Inode 2 is the root directory, and 11
is `lost+found`. An inode has the mode (type + permissions), owner,
size, times, link count and **15 block pointers**:
```
i_block[0..12] data blocks 0-11 1 KiB blocks: the first 12 KiB
i_block[12] -> 256 block numbers single indirect, up to 268 KiB
i_block[13] -> 256 single-indirect blocks double indirect, up to ~64 MiB
i_block[14] -> 256 double-indirect blocks triple indirect, up to ~16 GiB
```
`Volume::bmap` turns "block 40 960 of this file" into a disk block by
working out which level it falls in and walking down the pointers. A 0
pointer is a **hole**, which reads as zeros. That is how `sparse.bin`
is 66 MiB while using only a few blocks. With 4 KiB blocks, each indirect
block holds 1024 numbers.
- **Directories** are files full of variable-length entries:
`inode (u32), rec_len (u16), name_len (u8), file_type (u8), name`.
`rec_len` is the distance to the next entry. To delete an entry,
Rustbelt adds its length to the previous entry's `rec_len`. To add one,
it looks for an entry with spare room behind it, and if there is none,
it gives the directory a new block. Revision 0 has no `file_type` byte,
so the driver reads the inode to tell files from directories.
- **Symlinks.** A target shorter than 60 bytes is stored right inside
`i_block` (a *fast* symlink, like `hello-link`). Longer targets get a
data block (a *slow* symlink, like `docs/long-link`, 74 bytes). The VFS
follows up to 8 links in a path, then gives `ELOOP`. `stat` follows
them, `rm` doesn't, and the new `readlink` syscall (89) returns the
target, which `ls -l` prints as `name -> target`.
### What the driver does and doesn't do
- **Reads:**
- The revision 0 and revision 1 formats, with 1 KiB and 4 KiB blocks
(2 KiB should work too, but isn't tested).
- Direct, single, double and triple indirect blocks, and holes.
- Fast and slow symlinks.
- Directories with `dir_index` hash trees. The index hides inside
entries that plain ext2 code skips.
- **Writes:**
- Creating files and directories (mode 0644 and 0755), writing,
appending, writing past the end (which leaves a hole), truncating to 0,
`unlink` and `rmdir`.
- The bitmaps, the free counts and the parent's link count stay
consistent, so `e2fsck` is happy. Changing a hash-tree directory
clears its index flag, like Linux's ext2 does.
- **Design:**
- No cache: every operation re-reads what it needs, and every write goes
straight to the disk.
- One `Mutex` per volume.
- An `Ext2Inode` is just *(volume, inode number, kind)*.
- **Not yet:**
- Creating symlinks, hard links, rename, and `chmod`/`chown`.
- Deleting a file that has an extended-attribute block (gives
`EOPNOTSUPP`).
- Writes past 2 GiB.
- Real timestamps: there is no clock yet, so new files get the time the
image was made.
- The backup superblocks are never updated (Linux doesn't either), and
there is no journal: a crash in the middle of an operation leaves work
for `e2fsck`, just like real ext2.
### The test images (`build.rs`)
`build.rs` copies `disk2/` (text files, nested directories) to
`target/ext2-tree/` and adds a few generated files:
- `big.txt`: 16 000 numbered lines, 304 000 bytes, which reaches the double
indirect blocks.
- `sparse.bin`: 66 MiB with data only at 0, 40 MiB and 65 MiB, so the
last one sits in a **triple** indirect block.
- The two symlinks.
Then `mke2fs -t ext2 -d <tree>` builds the images without root or mounting:
| Image | Size | Block size | Notes | Attached as |
|---|---|---|---|---|
| `target/ext2.img` | 32 MiB | 1 KiB | revision 1, label RUSTBELT, the full tree | `vdb` -> `/mnt/disk1` = `C:` (run and tests) |
| `target/ext2-4k.img` | 8 MiB | 4 KiB | label FOURK, small tree | `vdd` -> `/mnt/disk3`, tests only |
| `target/ext2-rev0.img` | 4 MiB | 1 KiB | **revision 0** (`-E revision=0`), small tree | `vde` -> `/mnt/disk4`, tests only |
**Raw, not partitioned.** The FAT disk has a GPT because that's what real
disks have. The ext2 image is the bare file system, with no partition table,
so the host's `e2fsck`, `debugfs` and `dumpe2fs` work on the file directly.
`mount_disks` mounts a whole disk if it has no partitions, and its
partitions if it does. The disks are numbered in device order: `vda1` =
disk0, `vdb` = disk1, `vdc` (exFAT) = disk2.
### Checking Rustbelt's writes with the real e2fsck
All QEMU disks normally use `snapshot=on`, so nothing reaches the image
files. `scripts/ext2-check.sh` does the opposite. It copies the images to
`target/fsck/` and boots Rustbelt on the **writable** copies twice:
- once for the shell self-test (`EXT2_IMG=... ./scripts/run-headless.sh`),
which does `cp B: -> C:`, `md` and `echo >` on C:.
- once for the `tests/ext2.rs` kernel, which creates, writes, truncates
and deletes on all three images.
Then it runs `e2fsck -fn` on every copy (`-f` = check even if marked clean,
`-n` = read-only, change nothing) and shows the root directory with
`debugfs`:
```sh
cargo test --test ext2 && ./scripts/ext2-check.sh
```
```
== e2fsck -fn target/fsck/selftest.img (after Rustbelt wrote to it)
e2fsck 1.47.2 (1-Jan-2025)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
RUSTBELT: 25/8192 files (0.0% non-contiguous), 2777/32768 blocks
...
== debugfs: ls -l / on the self-test copy
23 100644 (1) 0 0 125 30-Oct-2026 01:45 todo.txt
24 40755 (2) 0 0 1024 30-Oct-2026 01:45 made-here
== debugfs: cat /made-here/note.txt
written by Rustbelt
== all images are clean
```
Pass 5 is the one that catches allocation bugs: it recounts every bitmap
and compares the result with the free counts in the group descriptors and
the superblock.
## exFAT (stage 9)
The fourth disk, `target/exfat.img`, holds an **exFAT** file system:
the format on SD cards over 32 GB and most USB sticks, which Windows,
macOS, Linux, cameras and phones all understand. Rustbelt reads and writes
it with its own driver, `src/fs/exfat.rs`: about 1000 lines including a
long explanation at the top, written from
[Microsoft's exFAT specification](https://learn.microsoft.com/en-us/windows/win32/fileio/exfat-specification),
with no crates. It is mounted at `/mnt/disk2`, which is drive **`D:`**:
```
A:/ $ cd d:
D:/ $ ls -l
132 Long File Name With Spaces.txt
84 MixedCase.TXT
<DIR> Photos/
88 a-really-long-file-name-that-takes-several-file-name-directory-entries-in-exfat-xxxxxxxxxxxxxxxxxxxx.txt
38000 frag-chain.txt
340000 numbers.txt
363 readme.txt
61 Ünïcödé café.txt
D:/ $ cat "long file name with spaces.txt"
This file has a long name with spaces in it. exFAT stores names of up
to 255 UTF-16 characters, 15 per "File Name" directory entry.
D:/ $ cp C:/hello.txt "From Ext2.txt"
D:/ $ type "FROM EXT2.TXT"
Hello from the ext2 disk! This file lives on a second virtio disk, in an
ext2 file system that Rustbelt reads and writes with its own driver.
```
Names keep the case you typed but are found in any case, like on Windows.
Quote names with spaces. Inside quotes the shell doesn't translate drive
letters, so `cd D:` first (or write `/mnt/disk2/...`). exFAT has no
permissions, so `ls -l` shows sizes only, like on the FAT disk.
### The on-disk layout, for a learner
exFAT is FAT's successor, redesigned for big flash cards. If you know
FAT32, three things are new: a **bitmap** says which clusters are free,
contiguous files can skip the FAT entirely, and a file's directory entry
is a **set** of 32-byte entries with a checksum.
```text
sector 0 1-8 9-10 11 12-23 FatOffset ClusterHeapOffset
| boot sector | ext. boot | OEM | checksum | backup of 0-11 | ... | FAT | ... | cluster 2 | cluster 3 | ...
```
- **Boot sector.** Says `EXFAT ` at byte 3; that's how `mount_disks`
recognizes it. Then the geometry, all in sectors: where the FAT and the
**cluster heap** start, how many clusters there are, the root directory's
first cluster, and the sector and cluster sizes as powers of two (`9` =
512 bytes). Byte 112 is **PercentInUse**, which we keep up to date.
- **Boot checksum.** Sector 11 is the 32-bit "rotate right, add the next
byte" checksum of sectors 0-10, repeated to fill the sector. It skips
the three bytes that change while the volume is in use (VolumeFlags and
PercentInUse), so updating those doesn't mean rewriting the checksum. We
check it at mount.
- **FAT.** One 32-bit entry per cluster: the next cluster of the file, or
`0xFFFFFFFF` for the end. Clusters are numbered from 2, and cluster N
starts at `ClusterHeapOffset + (N - 2) * cluster size`.
- **Everything else lives in clusters**, even the "system files":
- **Allocation bitmap**: 1 bit per cluster. This, not the FAT, says
what's free. We keep it in memory (960 bytes for our disk) and write
each changed byte straight through.
- **Up-case table**: the upper-case form of every UTF-16 character. Name
lookups compare up-cased names. Because the table is on the disk, every
implementation agrees on what "the same name" means, even for `é`/`É`.
It's stored compressed (`0xFFFF, n` = "the next n characters map to
themselves") and has its own checksum, which we verify. We keep only
the characters that change, in a sorted `Vec`, and binary-search it.
- **Root directory**: like any directory, except that it has no entry
of its own, so its size is the length of its FAT chain.
**Directories** are files of 32-byte entries. Byte 0 is the type, and bit 7
of it means "in use". Deleting just clears that bit, and type `0x00` marks
the end. The root also holds a volume label entry (`0x83`), the bitmap's
(`0x81`) and the up-case table's (`0x82`). A file or directory is an
**entry set**:
```text
0x85 File SecondaryCount, SetChecksum, attributes (0x10 = directory), 3 timestamps
0xC0 Stream Extension flags, NameLength, NameHash, ValidDataLength, FirstCluster, DataLength
0xC1 File Name 15 UTF-16 characters <- repeated: a 104-character name needs 7
```
- **SetChecksum** is a 16-bit "rotate right, add" over every byte of the
set except itself. Change a file's size and you must recompute it, or
other systems treat the file as corrupt.
- **NameHash** is the same checksum over the up-cased name, so a lookup can
skip most entries without comparing names.
- **NoFatChain** (stream flag `0x02`): "my clusters are consecutive; don't
read the FAT". Most files are contiguous, so most files never touch the
FAT. When a file must grow and the next cluster is taken, the driver
writes FAT links for the clusters it already has, clears the flag, and
carries on wherever there's room (`Volume::grow`). Directories work the
same way.
- **ValidDataLength** <= **DataLength**: bytes past it read as zeros
(Windows uses this to create big files quickly). We read it that way and
always write the two equal.
- Directories have no `.` or `..` entries, and their size is always whole
clusters.
### What the driver does and doesn't do
- **Reading:** contiguous and FAT-chained files, long UTF-16 names,
case-insensitive lookup through the volume's own up-case table, nested
directories, the volume label, and both cluster sizes we test (4 KiB and
512 bytes). Damaged entry sets (bad SetChecksum) are skipped.
- **Writing:** create, write, append, truncate to 0, mkdir, unlink and
rmdir (`ENOTEMPTY` if it isn't empty). That includes:
- allocating from the bitmap, preferring the cluster right after the
file's last one;
- growing directories when a new entry set doesn't fit, the root too;
- converting NoFatChain files to FAT chains, freeing chains, and
recomputing SetChecksums, NameHashes and PercentInUse.
- Names are checked: 1-255 UTF-16 characters, no control characters or
`" * / : < > ? \ |`. A name that differs only in case is `EEXIST`.
- **Not supported:**
- rename and symlinks (exFAT has no symlinks);
- real timestamps: there's no clock driver, so everything is 1 Jan 2026;
- a second FAT (TexFAT): such a volume mounts read-only;
- the VolumeDirty flag (writes go straight through);
- shrinking directories (nobody does).
- **Like the ext2 driver**, there's no block cache: only the bitmap and
one FAT sector are cached. An inode is just the path from the root,
resolved again on every call, because exFAT has no inode numbers (a file
*is* its directory entry).
### The test images, and why there's a populator (`build.rs`)
`mkfs.exfat` (exfatprogs) formats an image, but unlike `mke2fs -d` it can't
copy files in, and mtools doesn't speak exFAT. The usual way is to mount the
image, but that needs root and a loop device (or FUSE, which doesn't work on
this box either). So `build/exfat_populate.rs` is a **small exFAT writer of
its own** (~230 lines) that `build.rs` runs after `mkfs.exfat`. It only
handles a freshly formatted volume, which makes it simple: clusters are
handed out in order, each file is one contiguous NoFatChain run, and files
named `frag-*` get every other cluster, linked through the FAT, so the
kernel has a fragmented file to read. It deliberately shares no code with
the kernel driver, so a misreading of the spec in one doesn't hide in the
other, and the host's `fsck.exfat` checks its output on every run of
`scripts/exfat-check.sh`.
The tree is `disk3/` (long names with spaces, mixed case, non-ASCII names, a
104-character name, `Photos/2026 Trip/trip notes.txt`) plus two generated
files: `numbers.txt` (20 000 lines, 340 000 bytes, contiguous) and
`frag-chain.txt` (38 000 bytes, fragmented).
| Image | Size | Clusters | Notes | Attached as |
|---|---|---|---|---|
| `target/exfat.img` | 32 MiB | 4 KiB | label TYLEREXFAT, the full tree | `vdc` -> `/mnt/disk2` = `D:` (run and tests) |
| `target/exfat-512.img` | 8 MiB | 512 bytes | label SMALLCLUST, small tree: every file spans many clusters | `vdf` -> `/mnt/disk5`, tests only |
Like the ext2 disk, it's a bare file system with no partition table, so
`fsck.exfat` and `dump.exfat` work on the file directly. An exFAT boot
sector also ends in `55 AA`, like an MBR, so the partition scanner checks
for `EXFAT ` and reports `no partition table (WholeDiskExfat)` instead of
"MBR, 0 partitions".
### Checking Rustbelt's writes with the real fsck.exfat
`scripts/exfat-check.sh` works like `ext2-check.sh`. It copies the images to
`target/fsck/` and boots Rustbelt on the **writable** copies: once for the
shell self-test (`EXFAT_IMG=... ./scripts/run-headless.sh`), and once for the
`tests/exfat.rs` kernel. Its last test, `written_files_stay_for_fsck`,
deletes nothing, so fsck sees the tricky cases on disk: 300 files in a grown
directory that became a FAT chain, deleted entries between live ones, and a
file converted from NoFatChain to a FAT chain. Then it runs `fsck.exfat -n`
(report, change nothing) on every copy and reads a new file back with
`dump.exfat` and `dd`:
```sh
cargo test --test exfat && ./scripts/exfat-check.sh
```
```
== fsck.exfat -n target/exfat.img (pristine, as built)
target/exfat.img: clean. directories 3, files 8
== fsck.exfat -n target/exfat-512.img (pristine, as built)
target/exfat-512.img: clean. directories 3, files 4
13 exFAT tests ok
== fsck.exfat -n target/fsck/selftest-exfat.img (after Rustbelt wrote to it)
target/fsck/selftest-exfat.img: clean. directories 4, files 11
== fsck.exfat -n target/fsck/tests-exfat.img (after Rustbelt wrote to it)
target/fsck/tests-exfat.img: clean. directories 4, files 212
== fsck.exfat -n target/fsck/tests-exfat-512.img (after Rustbelt wrote to it)
target/fsck/tests-exfat-512.img: clean. directories 4, files 32
== dump.exfat -s "/New Folder": the entry set Rustbelt wrote for Note.txt
Path: /New Folder/Note.txt
Entrytype: 0x85
SetChecksum: 0xAF76
Entrytype: 0xC0
GeneralSecondaryFlags: 0x03
NameLength: 8
NameHash: 0x2C4C
ValidDataLength: 14
FirstCluster: 19
DataLength: 14
...
== its 14 bytes at cluster 19, read with dd:
hi from exFAT
== all images are clean
```
`fsck.exfat` recomputes the bitmap from every file's clusters, and checks
each SetChecksum and the boot checksum. A flipped bitmap bit or one changed
byte in an entry set gives "files corrupted 1". It doesn't check
PercentInUse, so the tests check that themselves: their `usage()` helper
reads the bitmap and byte 112 straight off the disk with a few lines of
independent parsing.
## btrfs (stage 10)
`E:` is a **btrfs** volume, read by a hand-written, **read-only** driver
(`src/fs/btrfs.rs`, about 1 200 lines with comments; `src/fs/lzo.rs` for
LZO). btrfs is what most of Tyler's Linux machines will use, so this is
the file system Rustbelt most needs to *read*. Writing comes much later
(see below).
```
E:/ $ ls -l
drwxr-xr-x <DIR> Music/
drwxr-xr-x <DIR> archive/ <- a read-only subvolume (id 256)
-rw-r--r-- 2160000 big.txt <- 17 zlib-compressed extents
drwxr-xr-x <DIR> docs/
-rw-r--r-- 0 empty.txt
lrwxrwxrwx hello-link -> hello.txt
-rw-r--r-- 78 hello.txt <- an inline extent, inside a leaf
drwxr-xr-x <DIR> projects/ <- a subvolume (id 257)
-rw-r--r-- 1500000 random.bin <- incompressible: stored as is
-rw-r--r-- 637 readme.txt
-rw-r--r-- 1048576 zeros.bin
E:/ $ echo hi > new.txt
sh: new.txt: Read-only file system
```
### The ideas, for a learner
**Everything is a B-tree item.** ext2 and exFAT have fixed tables (inode
tables, a FAT, bitmaps) at fixed places. btrfs has almost none: just the
superblock at 64 KiB (magic `_BHRfS_M`). Everything else is an *item* in a
B-tree, sorted by a 17-byte key `(objectid, type, offset)`. A file is a
handful of items that sit next to each other because they share an
objectid (the inode number):
```
(257 INODE_ITEM 0) size, mode, link count, times, flags
(257 INODE_REF 256) "my name in directory 256 is docs"
(258 DIR_ITEM <hash>) a directory entry, found by crc32c(name)
(258 DIR_INDEX <n>) the same entry again, in creation order (readdir)
(259 EXTENT_DATA 0) file bytes 0.. : inline data, or "see disk extent X"
(259 EXTENT_DATA 131072) file bytes 131072.. : the next extent
```
Leaves hold items; internal nodes hold `(key, child pointer)` pairs.
Looking something up is "walk down from the root to the last key <= the
one you want" (`Volume::search_le`); listing a directory or a file's
extents is a range walk (`Volume::range`). `lookup` uses DIR_ITEM (one
search by name hash), and `ls` uses DIR_INDEX (a range, in creation order).
**Copy-on-write B-trees.** btrfs never overwrites a live block. To change
a leaf it writes a *new* copy elsewhere, then a new copy of the parent
pointing to it, and so on up to the root; the superblock is written last
and switches everything over at once. Until then the old tree is still
complete, so a crash just leaves the old version. The same trick makes
snapshots nearly free: a snapshot is a second root pointing at the same
nodes. For a *reader* it means every node is immutable once written, and
each one records the transaction (generation) that wrote it.
**Chunks: logical addresses.** Tree pointers and extent addresses are
*logical* byte numbers, not disk offsets. The **chunk tree** maps logical
ranges to (device, physical offset) "stripes". With **single** there is one
stripe; with **DUP** (the default for metadata on one disk) there are two
copies on the same disk, and the driver reads the second when the first
fails its checksum. RAID profiles would spread stripes over several
devices; the driver refuses those with a message. But the chunk tree is
itself stored at logical addresses, so how do you read it? The superblock
carries a small bootstrap copy of the SYSTEM chunks (`sys_chunk_array`),
just enough to find the chunk tree. Open order: superblock, then
sys_chunk_array, then chunk tree, then root tree, then FS tree.
**Trees of trees.** The **root tree** lists every other tree with a
ROOT_ITEM (where its root node is, its level). Tree 5 is the top-level
**FS tree** (files and directories); tree 7 is the **csum tree**
(checksums of data blocks); others (extent tree, device tree...) only
matter for writing. The "default" entry in the root tree's directory 6
says which subvolume to show; the driver honours it.
**Subvolumes** are separate FS trees (ids 256 and up) with their own inode
numbers, so `(subvolume, inode)` names a file (`Loc` in the driver). A
directory entry that points at a subvolume has a ROOT_ITEM key instead of
an INODE_ITEM key; following it moves to that tree's top directory. The
driver lists them from the root tree's ROOT_REF items with their
read-only flag (`BtrfsFs::subvolumes`; on E: `archive` is 256, read-only,
and `projects` is 257), and `cd E:/projects` just walks into one. Snapshots
are subvolumes too, so they would work the same way.
**Checksums everywhere.** Every node starts with a checksum of the rest of
the node, plus its own logical address and the file system's id, so a
misplaced or stale block is caught as well as a corrupt one. Data blocks
are checksummed per sector in the csum tree, keyed by logical address. The
driver checks both (crc32c: the Castagnoli CRC, a few lines in software);
a bad block is `EIO` with a log line, and with DUP it tries the other copy
first. Newer btrfs can use xxhash64, sha256 or blake2b instead; those are
refused at mount time: `btrfs, but can't mount it: checksum type 2 (sha256)
is not supported, only crc32c`. Directory names are found by
`crc32c(name)` too, with seed `~1` and no final inversion (the "default"
hash 2378154706 is a unit test).
**Extents.** A file's data comes as EXTENT_DATA items, each covering a
range of the file:
| Kind | What it means | Driver |
|---|---|---|
| inline | the bytes are inside the item, in the leaf (small files, symlink targets) | copied, or decompressed |
| regular | `disk_bytenr` + `offset` + `num_bytes`: a slice of an on-disk extent, which may be shared with other files or snapshots | read + checksummed |
| compressed | a regular or inline extent with zlib, lzo or zstd: the whole extent is decompressed, then sliced | `miniz_oxide`, `src/fs/lzo.rs`, `ruzstd` |
| prealloc | reserved by `fallocate`, never written | zeros |
| hole | `disk_bytenr` 0, or simply no item (the `no_holes` feature) | zeros |
btrfs's lzo is not plain LZO: it's a 4-byte total length, then per segment
a 4-byte length plus an LZO1X block, and a segment header never crosses a
4 KiB sector boundary (padding instead). zstd extents can be followed by
zero padding, so the driver decodes frame by frame instead of "all
input".
### Why read-only first, and what writing would take
Reading btrfs only needs the trees as they are on disk. Writing needs
everything copy-on-write implies:
- **Allocation**: the extent tree and free-space tree say which logical
ranges are used, with *reference counts* (an extent can be shared by
files, snapshots and reflinks). Every new node or data block needs a
free range, and every freed one a decrement.
- **COW updates**: changing one item means new copies of the leaf and of
every node up to the root, in each affected tree (FS tree, extent tree,
csum tree, root tree...), plus splitting and merging nodes to stay
balanced.
- **Transactions**: batch all of that, write it, flush, then write the
superblock (all copies) with the new generation, in the right order
for crash safety. Also the log tree for `fsync`.
- **Checksums and compression on the way out**, inode/dir item updates,
back-references (INODE_REF, ROOT_BACKREF...).
That's a few thousand careful lines where a bug corrupts the volume, versus
about a thousand for reading. Read-only first gets Tyler's disks readable,
and it builds the parts writing needs anyway (key order, tree walking,
chunk mapping, checksums). A first write step could be overwriting
existing data blocks in place on a NODATACOW file.
### What the driver does and doesn't do
- Does: superblock (crc32c, feature flags, single device), sys_chunk_array
+ chunk tree with single and DUP, root tree, the default subvolume, FS
and csum trees; node checks (checksum, bytenr, fsid, level); inode items,
INODE_REF, DIR_ITEM (hash lookup), DIR_INDEX (readdir); inline, regular,
prealloc and hole extents, zlib/lzo/zstd, shared extents (slices into a
bigger extent, as btrfs-convert makes them); data checksums; symlinks;
subvolumes (listed, entered). A 256 KiB node cache and a one-extent
decompression cache keep `cat big.txt` from re-reading and re-inflating.
- Doesn't: write anything (`EROFS` for write, truncate, create, mkdir,
rm); RAID profiles or multiple devices; checksum types other than crc32c;
encryption; extended attributes, hard-link counts and timestamps aren't
shown; the extent tree isn't read (not needed to read files).
### The test images (`build.rs`)
`build.rs` copies `disk4/` to a staging tree, adds `big.txt` (120 000 lines,
2.16 MB), `random.bin` (1.5 MB of noise), `zeros.bin`, `empty.txt` and three
symlinks, and makes 128 MiB images with `mkfs.btrfs --rootdir`:
| Image | Options | Why |
|---|---|---|
| `btrfs.img` (E:) | `--compress zlib -u projects -u ro:archive`, defaults: 16 KiB nodes, metadata DUP | the main one |
| `btrfs-lzo.img` | `--compress lzo -n 4096 -m single` | lzo; small nodes make a 2-level FS tree |
| `btrfs-zstd.img` | `--compress zstd -n 65536` | zstd; 64 KiB nodes |
| `btrfs-convert.img` | `mke2fs -d` (4 KiB blocks), then `btrfs-convert` | real holes and shared extents, an `ext2_saved` subvolume |
| `btrfs-sha256.img` | `--csum sha256` | must be refused |
`mkfs.btrfs --rootdir` never makes holes or prealloc extents (sparse
files are stored as data), so holes come from the converted image and
prealloc from a hand-made item in a unit test. Each tree gets a manifest
(`target/btrfs-tree.manifest`, `btrfs-convert.manifest`): one line per
directory, file (size + CRC-32) and symlink (target).
### Checking against the host
`./scripts/btrfs-check.sh` (after `cargo test --test btrfs`):
```
== btrfs check --readonly target/btrfs.img
found 2056192 bytes used, no error found
... (all five images)
== run the tests/btrfs.rs kernel
11 btrfs tests ok
== compare what Rustbelt read with the host's files (zlib.crc32)
/mnt/disk3: 22 entries (12 files, 4710275 bytes) match
/mnt/disk7: 22 entries (12 files, 4710275 bytes) match
/mnt/disk8: 22 entries (12 files, 4710275 bytes) match
/mnt/disk9: 7 entries (4 files, 5442941 bytes) match
== all good
```
The test kernel itself already compares every file with the manifests; the
script checks again independently, from the host's files with Python's
`zlib.crc32`, using the `[btrfs-file]` lines the kernel prints.
## Roadmap
Done: all 12 chapters of the 2nd edition, plus preemptive kernel threads,
user mode + syscalls, ELF loading, a VFS with a ramfs, PCI, virtio-blk,
MBR/GPT partitions, FAT32, blocking threads, pipes, spawn/wait, process
memory cleanup, the shell, hand-written ext2, exFAT and (read-only)
btrfs drivers, and multi-core support with a sleep/wake scheduler and
guard pages.
- [x] Freestanding binary — [phil-opp](https://os.phil-opp.com/freestanding-rust-binary/)
- [x] Minimal kernel booting in QEMU — [phil-opp](https://os.phil-opp.com/minimal-rust-kernel/), [OSDev: Bare Bones](https://wiki.osdev.org/Bare_Bones)
- [x] VGA text mode printing — [phil-opp](https://os.phil-opp.com/vga-text-mode/), [OSDev: Printing to Screen](https://wiki.osdev.org/Printing_To_Screen)
- [x] Serial output + QEMU exit — [phil-opp: Testing](https://os.phil-opp.com/testing/), [OSDev: Serial Ports](https://wiki.osdev.org/Serial_Ports)
- [x] CPU exceptions / IDT — breakpoint & page fault handlers. [phil-opp: CPU Exceptions](https://os.phil-opp.com/cpu-exceptions/), [OSDev: IDT](https://wiki.osdev.org/Interrupt_Descriptor_Table), [OSDev: Exceptions](https://wiki.osdev.org/Exceptions)
- [x] GDT / TSS / double faults — separate IST stack so stack overflows don't triple-fault. [phil-opp: Double Faults](https://os.phil-opp.com/double-fault-exceptions/), [OSDev: GDT](https://wiki.osdev.org/Global_Descriptor_Table), [OSDev: TSS](https://wiki.osdev.org/Task_State_Segment)
- [x] Testing — `cargo test` in QEMU, custom test framework, integration tests. [phil-opp: Testing](https://os.phil-opp.com/testing/)
- [x] Hardware interrupts — 8259 PIC, timer, PS/2 keyboard echo, `hlt` idle loop. [phil-opp: Hardware Interrupts](https://os.phil-opp.com/hardware-interrupts/), [OSDev: 8259 PIC](https://wiki.osdev.org/8259_PIC), [OSDev: PIT](https://wiki.osdev.org/Programmable_Interval_Timer), [OSDev: PS/2 Keyboard](https://wiki.osdev.org/PS/2_Keyboard) (later: [APIC](https://wiki.osdev.org/APIC))
- [x] Introduction to paging — CR3, page fault demo. [phil-opp](https://os.phil-opp.com/paging-introduction/), [OSDev: Paging](https://wiki.osdev.org/Paging)
- [x] Paging implementation — OffsetPageTable, translation, frame allocator, example mapping. [phil-opp](https://os.phil-opp.com/paging-implementation/), [OSDev: Page Frame Allocation](https://wiki.osdev.org/Page_Frame_Allocation)
- [x] Heap allocation — `alloc`, heap mapping, global allocator. [phil-opp](https://os.phil-opp.com/heap-allocation/), [OSDev: Memory Allocation](https://wiki.osdev.org/Memory_Allocation)
- [x] Allocator designs — bump, linked list (crate), fixed-size block (active). [phil-opp](https://os.phil-opp.com/allocator-designs/)
- [x] Async/await — Task, simple executor, waker-based executor, lock-free async keyboard task (with Backspace). [phil-opp: Async/Await](https://os.phil-opp.com/async-await/), [OSDev: Multitasking Systems](https://wiki.osdev.org/Multitasking_Systems)
- [x] Preemptive kernel threads — own stacks, timer-driven round-robin, asm context switch. [OSDev: Brendan's Multi-tasking Tutorial](https://wiki.osdev.org/Brendan%27s_Multi-tasking_Tutorial), [OSDev: Context Switching](https://wiki.osdev.org/Context_Switching)
- [x] User mode (ring 3) & syscalls — GDT user segments, TSS.rsp0 per thread, `syscall`/`sysret`, per-process page tables, user faults kill the process. [OSDev: Getting to Ring 3](https://wiki.osdev.org/Getting_to_Ring_3), [OSDev: System Calls](https://wiki.osdev.org/System_Calls), [OSDev: SYSENTER/SYSCALL](https://wiki.osdev.org/SYSENTER)
- [x] ELF loading — hand-written PT_LOAD loader, user programs built by `build.rs`. [OSDev: ELF](https://wiki.osdev.org/ELF), [OSDev: ELF Tutorial](https://wiki.osdev.org/ELF_Tutorial)
- [x] VFS skeleton + ramfs — Inode/FileSystem/File traits, mount table, fd table, `open/read/write/close`. [OSDev: VFS](https://wiki.osdev.org/VFS), [Linux VFS overview](https://www.kernel.org/doc/html/latest/filesystems/vfs.html)
- [x] PCI enumeration + virtio-blk driver + `BlockDevice` trait — [OSDev: PCI](https://wiki.osdev.org/PCI), [OSDev: Virtio](https://wiki.osdev.org/Virtio), [virtio spec (legacy interface)](https://docs.oasis-open.org/virtio/virtio/v1.2/virtio-v1.2.html)
- [x] MBR + GPT partitions — [OSDev: MBR](https://wiki.osdev.org/MBR_(x86)), [OSDev: GPT](https://wiki.osdev.org/GPT)
- [x] FAT32 through `fatfs`, mounted at `/mnt/disk0`; programs loaded from disk — [OSDev: FAT](https://wiki.osdev.org/FAT)
- [x] Blocking threads + wait queues, pipes, `spawn`/`wait`, cwd, `stat`/`lseek`/`getdents64` — [OSDev: Blocking Process](https://wiki.osdev.org/Blocking_Process)
- [x] Shell as a user program: line editing, builtins, PATH, pipes, redirection, background jobs, drive letters, `run --trace` — [OSDev: Shell](https://wiki.osdev.org/Shell)
- [x] Process cleanup: frame free list, address spaces freed on reap
- [x] ext2, hand-written: read + write, indirect blocks, symlinks, 1/4 KiB blocks, revision 0/1, checked with `e2fsck` — [OSDev: Ext2](https://wiki.osdev.org/Ext2), [The Second Extended File System (Dave Poirier)](https://www.nongnu.org/ext2-doc/ext2.html)
- [x] exFAT, hand-written: read + write, entry sets + checksums, up-case table, NoFatChain and FAT chains, checked with `fsck.exfat` — [Microsoft exFAT specification](https://learn.microsoft.com/en-us/windows/win32/fileio/exfat-specification)
- [x] SMP (stage 11): ACPI MADT, local APIC + timer, INIT/SIPI trampoline, per-CPU GDT/TSS/`swapgs` data, a scheduler for 4 CPUs with sleep/wake and wait queues, frames freed on every exit, guard pages under kernel and user stacks — [OSDev: SMP](https://wiki.osdev.org/Symmetric_Multiprocessing), [OSDev: APIC](https://wiki.osdev.org/APIC), [OSDev: MADT](https://wiki.osdev.org/MADT), [OSDev: APIC timer](https://wiki.osdev.org/APIC_Timer)
- [x] btrfs, hand-written, read-only: chunk tree (single/DUP), root/FS/csum trees, crc32c everywhere, zlib/lzo/zstd, holes, subvolumes, checked against `btrfs check` and the host's files — [btrfs on-disk format](https://btrfs.readthedocs.io/en/latest/dev/On-disk-format.html)
### Filesystem roadmap (Tyler's plan)
Everything below plugs into the VFS above: a filesystem is a `FileSystem`
whose `root()` returns `Inode`s; mounting it is one `fs::mount` call.
1. **ramfs / tmpfs** — done (`src/fs/ramfs.rs`).
2. **Block device driver** — done: `BlockDevice` trait + legacy virtio-blk
(stage 6). Still to do: a block cache, IRQ-driven completion, maybe an
ATA PIO driver for comparison ([OSDev: ATA PIO Mode](https://wiki.osdev.org/ATA_PIO_Mode)).
3. **Partition tables** — done: MBR + GPT (no CRC checks yet).
4. **FAT32** — done via the `fatfs` crate (`src/fs/fat.rs`). A good exercise
later: a hand-written read-only FAT32 to see what the crate does.
5. **ext2** — done (stage 8, `src/fs/ext2.rs`): written by hand, read and
write: superblock, block groups, bitmaps, inode table, direct/indirect
blocks, directory entries, symlinks. Still to do: symlink creation,
rename, hard links, a block cache, real timestamps once there's an RTC
driver. Later, ext4's extents would be a nice follow-up. [OSDev: Ext2](https://wiki.osdev.org/Ext2),
[The Second Extended File System (Dave Poirier)](https://www.nongnu.org/ext2-doc/ext2.html)
6. **exFAT** — done (stage 9, `src/fs/exfat.rs`): written by hand from
Microsoft's spec, read and write: boot checksum, allocation bitmap, FAT
chains and NoFatChain runs, up-case table, entry sets with SetChecksum
and NameHash. Chosen because it works on literally anything (cameras,
phones, Windows, macOS, Linux), so Rustbelt can share USB sticks and SD
cards with every other machine. Still to do: rename, timestamps,
TexFAT. [Microsoft exFAT specification](https://learn.microsoft.com/en-us/windows/win32/fileio/exfat-specification)
7. **btrfs, read-only** — done (stage 10, `src/fs/btrfs.rs`): because most
of Tyler's Linux systems will use btrfs going forward, so this kernel
can read their disks. Copy-on-write B-trees, the chunk tree (logical ->
physical, single and DUP), root/FS/csum trees, crc32c checksums,
zlib/lzo/zstd, holes, subvolumes. Still to do: writing (see "Why
read-only first"), other checksum types, RAID/multi-device, xattrs.
[btrfs on-disk format](https://btrfs.readthedocs.io/en/latest/dev/On-disk-format.html),
[btrfs docs](https://btrfs.readthedocs.io/)
8. **NTFS** — last, maybe never.
Next, in this order:
1. ~~**Core polish**~~ — done (stage 11): SMP on 4 CPUs, a scheduler with
sleep/wake and wait queues, memory freed on every exit, guard pages.
Left over for later: priorities, per-CPU run queues, an I/O APIC, a
bigger or growable heap.
2. **Networking** (next): a virtio-net driver plus [smoltcp](https://github.com/smoltcp-rs/smoltcp)
for TCP/IP. [virtio spec](https://docs.oasis-open.org/virtio/virtio/v1.2/virtio-v1.2.html)
3. **More virtio devices**: virtio-rng (entropy), virtio-console, and
virtio-gpu (a framebuffer instead of VGA text mode).
4. **Running Linux software**: `fork`, `exec`, `mmap` and signals, then a
[musl](https://musl.libc.org/) port, so ordinary C programs recompiled
against Rustbelt's syscalls (numbered like Linux's already) can run.
**Long term: a graphical desktop.** The goal is a **Hyprland-style tiling
compositor**: windows tile automatically, with keyboard-driven layouts and
smooth animations. It will draw **natively on the virtio-gpu framebuffer**
first, with Rustbelt's own programs as clients, so it needs nothing beyond
step 3. **Wayland client support** comes later, once step 4 can run
ordinary Linux programs: the compositor then speaks the Wayland protocol, so
existing Wayland apps can open windows on it.
Out of scope for now: **drivers for physical hardware** (real disks, NICs,
USB). Rustbelt targets QEMU's virtio devices. A **Raspberry Pi / ARM port**
is the planned future hardware target. The split into a separate
**userland and OS** (its own repo layout, libc, toolchain) comes later.
Other ideas, any time:
- **Shell extras**: Left/Right cursor movement, Tab completion of file
names, `$?` and environment variables, Ctrl+C to kill the foreground job
(that needs signals or a kill syscall).
- **More programs**: `more`/`less` (needs raw mode and screen size), an editor.
Small exercises along the way: a timer-based
async `sleep` future, speed up the PIT
([OSDev: PIT](https://wiki.osdev.org/Programmable_Interval_Timer)), add handlers for
divide error / general protection fault, make `BootInfoFrameAllocator` O(1), add a
`mem` stat (bytes allocated) to the allocator.
Other good references: [OSDev wiki main page](https://wiki.osdev.org/Expanded_Main_Page),
[Intel SDM](https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdms.html),
the [blog_os repo](https://github.com/phil-opp/blog_os) (one branch per post, e.g. `post-04`),
and for a UEFI/framebuffer setup later, the newer [`bootloader` 0.11](https://github.com/rust-osdev/bootloader) crate.
## Notes / gotchas
- The target JSON changes occasionally with new nightlies (e.g. `target-pointer-width`
became an integer, `rustc-abi` was added). If a nightly update breaks the build,
compare with the latest `x86_64-blog_os.json` in the blog_os repo, or pin a date in
`rust-toolchain.toml` (`channel = "nightly-2026-10-29"`).
- The "packages contain code that will be rejected by a future version of Rust: core"
warning comes from building `core` with build-std and is harmless.
- `bootimage` prints "`CARGO_MANIFEST_DIR` env variable not set" when run outside
`cargo run`; also harmless.
- `cargo test` with build-std breaks ("duplicate lang item in crate `core`: `sized`")
if `Cargo.toml` sets `panic = "abort"` in `[profile.*]`. The target JSON already
sets `"panic-strategy": "abort"`, so the profile settings were removed.
- Stale disk image when switching cargo features: cargo re-uses a previously built
kernel ELF with an *old* mtime, so bootimage thinks its cached image is current
and boots the wrong build. `scripts/runner.sh` and the scripts `touch` the ELF
first to avoid this. If you call `cargo bootimage --features ...` by hand after
switching features, `touch target/x86_64-rust_kernel/debug/rustbelt` first.
- `build.rs` reruns when `user/`, `disk/`, `disk2/`, `disk3/`, `disk4/`, `build/` or
`build.rs` change and rebuilds `target/disk.img` and the ext2, exFAT and btrfs images. QEMU opens them with
`snapshot=on`, so the kernel's writes (the tests create and delete files)
never reach the files on disk (except in `scripts/ext2-check.sh` and
`scripts/exfat-check.sh`).
- e2fsprogs: `sudo apt install e2fsprogs`. `mke2fs`, `e2fsck`, `debugfs` and
`dumpe2fs` live in `/usr/sbin`, which isn't on a normal user's `PATH`
(`build.rs` and the script look there). Newer `mke2fs` dropped `-r 0`;
revision 0 is `-E revision=0`. `mke2fs -d` stores all-zero blocks as
holes, so `sparse.bin`'s data chunks are deliberately non-zero.
- exfatprogs: `sudo apt install exfatprogs`. `mkfs.exfat`, `fsck.exfat` and
`dump.exfat` live in `/usr/sbin` too. Mounting an exFAT image to fill it
needs root and a loop device, and `exfat-fuse` can't mount here either
(`fusermount3: unknown option 'user=box'`; with sudo, "Block device
required"), hence `build/exfat_populate.rs`. `dump.exfat -d PATH` in
exfatprogs 1.2.9 can't find files inside some directories (e.g.
`/NewFolder/Note.txt`), even on images that only mkfs and the populator
touched; `dump.exfat -s DIR` lists them fine, so `exfat-check.sh` uses that.
- btrfs-progs: `sudo apt install btrfs-progs` (6.14 here; `mkfs.btrfs`,
`btrfs`, `btrfs-convert` are in `/usr/bin`). `mkfs.btrfs --rootdir`
writes sparse files as real data (no holes, no prealloc) and numbers
`-u` subvolumes in its own order (`-u projects -u ro:archive` gives
`archive` 256 and `projects` 257). Compressed extents are at most
128 KiB, uncompressed ones 1 MiB, and data that doesn't compress is
stored plain. `btrfs-convert` refuses ext2 with 1 KiB blocks and fails
*silently* on a 64 MiB image (256 MiB works). `btrfs inspect-internal
dump-tree` and `dump-super` on an image show exactly what the driver
parses; they were the main debugging tool.
- The VGA console is ASCII-only: `Ünïcödé café.txt` shows as `■n■c■d■
caf■.txt` on screen (serial shows it right). Mapping UTF-8 to code page
437 in `vga_buffer.rs` would be a small fix.
- `docs/long-link` points through lots of `..` and `.`, but along a path
that also works on Linux (`cat` it after `debugfs -R "rdump /docs /tmp/x"`).
The VFS resolves `..` lexically, so a target like `file.txt/..` would
work in Rustbelt but not on Linux; the test image avoids that.
- Debug builds use a lot of stack: deep call chains (syscall -> VFS -> fatfs
-> ELF loader) needed more than 16 KiB. If you see "kernel stack overflow
in thread", that's the guard page catching it: make the stack bigger
(`STACK_SIZE` in `src/stack.rs`) or the call chain shallower.
- **SMP pitfalls hit in stage 11:**
- *`join` returned before the memory was free.* The dead thread was
removed from the table first and its stack freed a moment later, on
another CPU, so the leak test saw 7 extra frames now and then. Now the
stack and process reference are dropped first, then the thread is removed.
- *Mixed-up output.* With two programs printing on two CPUs, lines came
out as `count = CPU900000000`: `println!` sent each formatted piece as
its own `write`, and the console printed one write piece by piece. Now
`ulib`'s `println!` formats a whole line into a buffer first, and the
console holds a lock for the whole of one write.
- *A lost wake-up through the back door.* If a thread blocks but there's
nothing else to run, it halts in place; a wake from another CPU then
marks it Ready *and* queues it while it's still running here. Another
CPU could then pick it up too. `done_waiting` takes it back off the queue.
- *A process exiting before its spawner stored its thread id.* On one CPU
that couldn't happen; on four it can. `reap` waits for the id.
- Freeing a kernel stack's page tables would need every CPU to flush its
paging-structure caches; keeping them and reusing the slot avoids it.
- **The `\ |` key**: pc-keyboard 0.7's scancode set 1 decodes it (scancode
0x2B) as `KeyCode::Oem7`, but its US 104-key layout only maps `Oem5` to
`\` and `|`. So that key typed nothing, which went unnoticed until the
shell needed `|`. `task/keyboard.rs` renames `Oem7` to `Oem5` before
decoding.
- User programs: the linker drops empty output sections, and their
`ALIGN(4K)` with them. A program without `.data` then had its `.bss` on the
read-only `.rodata` page (see "ELF loading"). `user/link.ld` uses
`. = ALIGN(4K);` before `.data`, and the loader now rejects such files.
|