foundry

# Foundry OS: decisions and plan

## Tyler's answers to the proposal (2026-10-31)

| # | Question | Decision |
|---|---|---|
| 1 | Name, place, history | The OS is **Foundry OS**, in `/workspace/foundry` (local git only: no remote, no push). The kernel's `user/` and `disk/` histories were kept with `git subtree split`, done in a scratch clone at a55b41d (`/workspace/foundry-scratch/rk`), never in `/workspace/rust-kernel`. |
| 2 | What stays in the kernel | The ABI test programs stay as kernel fixtures: `hello evil spin demo child overflow forktest cowtest mmaptest sigtest threads echoserver ptytest nbtest fputest pathtest libctest coretest unixtest rawabi` and the musl ones `vdsotest piehello piethreads dynhello remaptest`. The shell, the tools and the net/sys programs moved here. |
| 3 | Which shell | The Rust shell stays the interactive login shell, renamed **rbsh**, keeping its DOS commands (`run`/`start`, `dir`/`type`/`cls`...) and drive letters. **dash** (later busybox `ash`) becomes `/bin/sh` for POSIX scripts once the root is ext2. Under contract v1 rbsh is still installed as `/bin/sh`, because that is the path the kernel runs. |
| 4 | Root file system | An **ext2 root**, so `A:` = `/` is the OS disk. The FAT32 disk stays as a **data disk** (`B:`). Plus an **exFAT share** (`C:`, `/mnt/share`) for files that go to Windows/macOS. Built in M3 (contract v2). |
| 5 | Name clashes | The Rust tools stay the default. busybox/sbase applets go under **`/usr/lib/busybox`**, off the default `PATH` (call them by full path, or add the directory to `PATH` yourself). |
| 6 | Tarballs | **Not in git.** A git-ignored download cache (`cache/`), each file pinned by SHA256 (`toolchain/musl/musl.sha256` + `fetch.sh`; `ports/<name>/<name>.sha256`). |
| 7 | Which kernel | A **pinned commit**: `KERNEL_REV` in `config.mk` (now `50476f4`, Rustbelt 0.21.0; before that `5291332`, 0.20.0), bumped deliberately: see [bumping-the-kernel.md](bumping-the-kernel.md). |
| 8 | Box packages | Small build packages are OK (`make`, `bzip2`, `musl-tools`, `autoconf`). |

## Disk layout (contract v2, built in M3)

The details and the reasons are in [boot-contract.md](boot-contract.md).
One disk, three partitions:

```
foundry.img (GPT, 90 MiB)
├── 1  ext2   "Linux root (x86-64)"   → /           (A:)  the OS, from root/
├── 2  FAT32  "Microsoft basic data"  → /mnt/disk0  (B:)  data disk + the kernel's ABI test programs, from rootfs/
└── 3  exFAT  "Microsoft basic data"  → /mnt/share  (C:)  the share, from share/
```

Inside the ext2 root:

```
/sbin/init         Foundry OS init (rust/init)
/bin               rbsh, dash (also /bin/sh), the Rust tools (ls cat grep ...), our C programs
/etc               passwd group hosts resolv.conf motd   (root/etc/)
/scripts           the self-test init runs headless
/usr/lib/busybox   busybox + applet symlinks, off the default PATH (M5)
```

## Milestones

| | Goal | Status |
|---|---|---|
| **M1** | Foundry OS builds the same image; the kernel's self-test passes on it | **done**: `make test` passes; `make compare` shows the same 44 programs (47 since the 0.20.0 pin) and identical non-program files |
| M1b | `rbrt` becomes a real library crate; rbsh's parser tests run on the host | next |
| **M2** | musl hello + probes in `/bin`, built static-PIE with `toolchain/foundry-cc` | **done**: `hello-musl` and `fprobe` (40 checks incl. timerfd) in `/bin`, checked by `make test` |
| M2b | dash built from ports/ as `/bin/dash`, with script checks in `make test` | **done** |
| **M3** | dash as `/bin/sh`, ext2 root (contract v2), Rust `/sbin/init`, exFAT share | **done** on Rustbelt 0.21.0: `make test` (v2) passes, plus host fsck of all three file systems; `make test-v1` still passes |
| M4 | sbase tools | needs testing per tool (docs/syscall-gaps.md) |
| M5 | busybox in `/usr/lib/busybox` | `bzip2` for the tarball |

## Long-term track: a self-hosted Rust toolchain (after M5, not started)

**The goal:** write a Rust program *on* Foundry OS, compile it there, and
run it there, without the box. Today every program is compiled on the box
(Linux) and copied into the image.

Words used below:

- **Compiler, toolchain.** A compiler turns source code (`main.rs`,
  `hello.c`) into a program the CPU can run. A *toolchain* is the compiler
  plus its helpers: the linker (glues the pieces into one file), the
  standard library, and for Rust, `cargo` (the build tool).
- **Cross-compiling.** Compiling on one system for another one. That is
  what we do now: the box's compilers make programs for Foundry OS.
- **Self-hosted.** A system that can build its own programs (in the end,
  its own compiler) on itself. Linux is self-hosted; Foundry OS isn't yet.
- **Bootstrap.** Getting the first compiler onto a system that has none:
  you cross-compile a compiler on another machine, copy it over, and from
  then on it can build things locally, even newer versions of itself.
- **Dynamic linker (`ld.so`).** Most Linux programs don't carry their
  libraries inside them. They name shared libraries (`.so` files), and a
  small program, the dynamic linker, loads those into memory when the
  program starts. Rustbelt has none today: it refuses such programs
  (`PT_INTERP`), so everything we run is *static* (all in one file).

**Hard parts, honestly:**

- **Size.** rustc is big: the compiler itself (most of it is LLVM, a
  huge C++ library) is about 150-200 MB, and the standard library for one
  target another 100-300 MB. Our whole root partition is 32 MiB today.
- **Memory.** LLVM is hungry: a hello world needs a few hundred MB of RAM,
  a real crate 1 GB or more. Our QEMU runs use QEMU's default of 128 MiB.
- **QEMU speed.** Our tests run QEMU without KVM, i.e. by emulating
  every instruction in software ("TCG"), roughly 5-20x slower than the
  box. With KVM (hardware virtualization) it's near native. Then add the
  kernel's own speed: Rustbelt's FAT driver re-walks the path on every
  read, for example, and nobody has measured page-fault or disk speed.
- **64 file descriptors per process** (`MAX_FDS` in the kernel). cargo,
  rustc and the linker can open more than that at once (many `.rlib`
  files, jobserver pipes, threads).
- **No `root=` option**, and only ext2 roots. A toolchain partition can't
  be chosen at boot. It has to sit on the one ext2 root (so the root must
  grow to a few GB) or be mounted elsewhere, e.g. as another ext2 disk
  that appears at `/mnt/diskN`.

### Stage 1: a C compiler that runs on Foundry OS

*In plain words:* first prove that "compile on the target" works at all,
with a small compiler. **tcc** (Tiny C Compiler, ~300 KB, compiles and
links in one program, needs no separate assembler or linker) is the
easiest first step. Then a real one: **gcc** (with binutils' `as` and
`ld`) or **clang/lld**, built static against our musl.

- *Size:* **M** for tcc (one port, like dash, plus its own small libc
  headers path); **L** for gcc/clang (big builds; gcc needs `as`, `ld`,
  `cc1` and lots of processes; clang is one big LLVM program, a good
  rehearsal for rustc).
- *Done when:* `make test` runs `tcc -static hello.c -o /tmp/hello &&
  /tmp/hello` on Foundry OS and prints the line; later the same with
  `gcc`, and on-target gcc builds `c/fprobe.c` and it passes 40/40.
- *OS-side:* ports/tcc, ports/binutils + ports/gcc (or clang), the musl
  headers and `crt*.o` installed into the root (`/usr/include`,
  `/usr/lib`), a bigger root partition.
- *Kernel (light, maybe nothing):* whatever the ports trip over, found
  the usual way (fprobe-style checks, docs/syscall-gaps.md).

### Stage 2: what rustc and cargo need from the system

*In plain words:* rustc and cargo are ordinary Linux programs, but big
and demanding ones. Before porting them, make sure the system can carry
them: room on disk, enough memory, fast enough files and memory mapping,
lots of threads and child processes, and a way to get libraries
(*crates*).

| Need | Why | Kernel or OS |
|---|---|---|
| **Either** a dynamic linker **or** a fully static rustc | Official rustc builds load `librustc_driver-*.so` at run time. Option A: Rustbelt loads `PT_INTERP` programs and we ship musl's `ld.so` (musl's `libc.so` is its own dynamic linker). Option B: build rustc statically (no `dylib` driver, `crt-static`, LLVM linked in). B needs no kernel work but a custom rustc build. | A: **kernel** (load the interpreter, pass `AT_BASE`/aux vector) + OS (ship `ld.so`). B: OS only |
| RAM: 1-2 GB | LLVM | **kernel**: confirm/raise how much RAM Rustbelt can use; OS: `-m 2G` in our QEMU runs |
| Disk: a few GB of ext2 | toolchain + `target/` dirs | OS (bigger root or a second ext2 disk); **kernel**: ext2 performance and large-file/large-dir behaviour |
| `mmap` of big files, `mremap` | rustc mmaps `.rlib`/metadata files; musl's `realloc` tries `mremap` (25, still ENOSYS, K10) | **kernel** (file-backed `mmap` speed, `mremap`) |
| More than 64 fds | rustc + linker + jobserver | **kernel** (raise `MAX_FDS`) |
| Threads, jobserver pipes | rustc uses threads; cargo hands out build slots through a pipe (`jobserver`) | mostly there (K-tests pass); check `poll` on pipes under load |
| Spawning many processes | cargo starts rustc, rustc starts the linker (`cc`/`ld`); Rust's `Command` uses musl `posix_spawn`, i.e. `clone(CLONE_VM \| CLONE_VFORK)` | **kernel** (check that clone flag set; today's table lists fork-like `SIGCHLD [+ CLONE_VFORK]` and threads) |
| `/proc/self/exe` and more `/proc` | rustc finds its sysroot through `current_exe()`; cargo reads `/proc/cpuinfo`-style info for job counts | `/proc/self/exe` exists; **kernel** for the rest (K7) |
| Crates | `cargo build` downloads from crates.io over **HTTPS** | Option A: TLS (an OS-side port, e.g. rustls in a tool or BearSSL) + DNS, both on our network stack. Option B (simpler first): an **offline vendored registry** (`cargo vendor` on the box, copied in, `--offline`). OS only |

- *Size:* **XL**. Each row is small to medium, but together they are a
  lot, and some (RAM, fds, the dynamic linker) are design changes in the
  kernel.
- *Done when:* a stress program (C or Rust, cross-built) on Foundry OS
  maps a 200 MB file, runs 16 threads, opens 200 fds, spawns 500
  children through `posix_spawn`, and the frame count is flat afterwards.
  Plus `free` shows ≥ 1.5 GB, and `df` shows ≥ 3 GB on the toolchain disk.
- *Kernel requests for Edward (kept light):* RAM limit, `MAX_FDS`,
  `mremap`, `CLONE_VM|CLONE_VFORK`, and (only if we choose option A) the
  `PT_INTERP` loader. Everything else is OS-side.

### Stage 3: rustc and cargo on Foundry OS

*In plain words:* cross-compile rustc and cargo on the box for
`x86_64-unknown-linux-musl` (the same Linux ABI Foundry OS speaks), copy
them onto a Foundry OS disk, and use them there. Our own `no_std`
programs (rbsh, the tools) need only `core`/`alloc` and our `rbrt`, so
they are a good first real test.

Cheaper wins on the way, each its own milestone:

1. `rustc --version` runs on Foundry OS (proves loading, size, memory).
2. `rustc hello.rs` (no cargo) builds a static hello that runs on Foundry
   OS. Needs rustc to find a linker: our on-target gcc/`ld` from stage 1,
   or rustc's bundled `rust-lld`.
3. `cargo build --offline` of a one-file crate.
4. **End:** `cargo build --offline --release` of `rust/rbsh` (our shell)
   **on Foundry OS**, and the resulting `rbsh` runs the v2 self-test.

- *Size:* **XL**. The cross-build of rustc itself is long (hours on the
  box, many GB of disk), and every failure on target is slow to debug
  under TCG.
- *Done when:* step 4 passes in a `make test-selfhost` (allowed to be
  slow, outside the normal `make test`).
- *OS-side:* the cross-build recipe (`ports/rust`, pinned like musl),
  the toolchain disk, the vendored registry, `make test-selfhost`.
- *Kernel:* nothing new beyond stage 2, plus whatever stage 3 finds.