foundry

# Foundry OS: decisions and plan

## Tyler's answers to the proposal (2026-10-31)

| # | Question | Decision |
|---|---|---|
| 1 | Name, place, history | The OS is **Foundry OS**, in `/workspace/foundry` (local git only: no remote, no push). The kernel's `user/` and `disk/` histories were kept with `git subtree split`, done in a scratch clone at a55b41d (`/workspace/foundry-scratch/rk`), never in `/workspace/rust-kernel`. |
| 2 | What stays in the kernel | The ABI test programs stay as kernel fixtures: `hello evil spin demo child overflow forktest cowtest mmaptest sigtest threads echoserver ptytest nbtest fputest pathtest libctest coretest unixtest rawabi` and the musl ones `vdsotest piehello piethreads dynhello remaptest`. The shell, the tools and the net/sys programs moved here. |
| 3 | Which shell | The Rust shell stays the interactive login shell, renamed **rbsh**, keeping its DOS commands (`run`/`start`, `dir`/`type`/`cls`...) and drive letters. **dash** (later busybox `ash`) becomes `/bin/sh` for POSIX scripts once the root is ext2. Under contract v1 rbsh is still installed as `/bin/sh`, because that is the path the kernel runs. |
| 4 | Root file system | An **ext2 root**, so `A:` = `/` is the OS disk. The FAT32 disk stays as a **data disk** (`B:`). Plus an **exFAT share** (`C:`, `/mnt/share`) for files that go to Windows/macOS. Built in M3 (contract v2). |
| 5 | Name clashes | The Rust tools stay the default. busybox/sbase applets go under **`/usr/lib/busybox`**, off the default `PATH` (call them by full path, or add the directory to `PATH` yourself). |
| 6 | Tarballs | **Not in git.** A git-ignored download cache (`cache/`), each file pinned by SHA256 (`toolchain/musl/musl.sha256` + `fetch.sh`; `ports/<name>/<name>.sha256`). |
| 7 | Which kernel | A **pinned commit**: `KERNEL_REV` in `config.mk` (now `50476f4`, Rustbelt 0.21.0; before that `5291332`, 0.20.0), bumped deliberately: see [bumping-the-kernel.md](bumping-the-kernel.md). |
| 8 | Box packages | Small build packages are OK (`make`, `bzip2`, `musl-tools`, `autoconf`). |

## Disk layout (contract v2, built in M3)

The details and the reasons are in [boot-contract.md](boot-contract.md).
One disk, three partitions:

```
foundry.img (GPT, 90 MiB)
├── 1  ext2   "Linux root (x86-64)"   → /           (A:)  the OS, from root/
├── 2  FAT32  "Microsoft basic data"  → /mnt/disk0  (B:)  data disk + the kernel's ABI test programs, from rootfs/
└── 3  exFAT  "Microsoft basic data"  → /mnt/share  (C:)  the share, from share/
```

Inside the ext2 root:

```
/sbin/init         Foundry OS init (rust/init)
/bin               rbsh, dash (also /bin/sh), the Rust tools (ls cat grep ...), our C programs
/etc               passwd group hosts resolv.conf motd   (root/etc/)
/scripts           the self-test init runs headless
/usr/lib/busybox   busybox + applet symlinks, off the default PATH (M5)
```

## Milestones

| | Goal | Status |
|---|---|---|
| **M1** | Foundry OS builds the same image; the kernel's self-test passes on it | **done**: `make test` passes; `make compare` shows the same 44 programs (47 since the 0.20.0 pin) and identical non-program files |
| M1b | `rbrt` becomes a real library crate; rbsh's parser tests run on the host (`make unit`); Foundry OS branding in rbsh and getty | done |
| **M2** | musl hello + probes in `/bin`, built static-PIE with `toolchain/foundry-cc` | **done**: `hello-musl` and `fprobe` (40 checks incl. timerfd) in `/bin`, checked by `make test` |
| M2b | dash built from ports/ as `/bin/dash`, with script checks in `make test` | **done** |
| **M3** | dash as `/bin/sh`, ext2 root (contract v2), Rust `/sbin/init`, exFAT share | **done** on Rustbelt 0.21.0: `make test` (v2) passes, plus host fsck of all three file systems; `make test-v1` still passes |
| M4 | sbase tools | needs testing per tool (docs/syscall-gaps.md) |
| M5 | busybox in `/usr/lib/busybox` | `bzip2` for the tarball |
| M6 | User management: passwords, `login`, users and groups, real file owners and modes ([below](#m6-user-management-approved-not-started)) | approved; needs kernel K13 |
| M7 | A bootable Raspberry Pi 3 build ([below](#m7-raspberry-pi-3-bootable-build-placeholder)) | placeholder; Edward's arm64/Raspberry Pi port is in progress on the kernel side |

## M6: user management (approved, not started)

**In plain words.** Today everything on Foundry OS runs as **root**, the
all-powerful user (user id, *uid*, 0): any program can read or delete any
file. On a real multi-user system each person has an **account**: a name,
a number (uid), a **group** (gid), a home folder and a password. The
kernel then checks, on every file access, whether *this* uid may read,
write or run *that* file, using the file's owner and its **mode bits**
(`rwxr-x---`: what the owner, the group and everyone else may do). M6
makes Foundry OS work like that: you log in as `tyler`, your shell runs
as uid 1000 in `/home/tyler`, and you need root's password (`su`) for
system changes.

Words used below:

- **`/etc/passwd`**: one line per account (`tyler:x:1000:1000:Tyler:/home/tyler:/bin/rbsh`:
  name, password placeholder, uid, gid, full name, home, shell). Everyone
  may read it, because programs like `ls -l` turn uids into names with it.
- **`/etc/group`**: the groups and who is in them (`wheel:x:10:tyler`).
- **`/etc/shadow`**: the password **hashes**, readable by root only. A
  hash is a one-way scramble of the password: easy to compute from the
  password, practically impossible to turn back into it. `login` hashes
  what you type and compares. We use **SHA-512 crypt** (`$6$...` in the
  file, done by musl's `crypt()`), with a random *salt* per password so
  two equal passwords look different. **yescrypt** (`$y$`), which is slower
  to brute-force, comes later.
- **setuid bit**: a mode bit (`rws` instead of `rwx`) that makes a program
  run as the file's *owner* rather than as whoever started it. That is how
  `su` and `passwd`, owned by root, can read or change `/etc/shadow` when
  `tyler` runs them. They are therefore the most security-sensitive
  programs on the system and must stay small.
- **getty → login → shell**: `getty` opens a terminal and asks for a name,
  `login` checks the password, switches to the user's uid/gid/groups,
  goes to the home folder and starts the user's shell (rbsh).

**Size: L.** The OS-side programs are small, and busybox brings most of
them. But every piece depends on the kernel really enforcing ids and
permissions (K13, not assumed to work), and on the image build getting
owners and modes exactly right. The security pieces (first-boot setup,
no default password, setuid programs) need care and tests, not just code.

### Scope

1. **Account files**: `/etc/passwd`, `/etc/group` (root, wheel, tyler's
   group, ...), and `/etc/shadow` with **mode 0600 root:root** (or 0640
   root:shadow if a `shadow` group is needed for a non-root helper).
   Hashes are SHA-512 crypt through musl; yescrypt later.
2. **Login chain**: init starts `getty` on each terminal (instead of rbsh
   directly), `getty` runs `login`, `login` runs **rbsh as the user in
   their home dir** with `HOME`, `USER`, `LOGNAME`, `SHELL` and `PATH` set.
   A wrong password gives "Login incorrect" and a short delay.
3. **Becoming root**: **`su`** now (busybox `su`, setuid root). **`sudo`**
   or a small Rust **`doas`** later.
4. **Account tools**: `passwd`, `adduser`/`deluser`, `addgroup`, `chmod`,
   `chown`: **busybox first**, Rust replacements (in `rust/`) later.
5. **First-boot setup (no default password, ever)**: the image ships with
   root's password field **locked** (`!` or `*` in `/etc/shadow`: no
   password can match) and no user account. On the first interactive boot
   init sees `/etc/foundry/setup-needed` and runs a setup program on the
   console *before* any login prompt: it asks for a new root password
   (twice), and a user name + password, creates the user and their home
   folder, then deletes the marker.

   *Why this matters:* a Raspberry Pi is usually on a home network, often
   reachable by other machines (and, through port forwarding or an
   attacker on the same Wi-Fi, by the internet). A built-in password like
   `root`/`root` is the first thing every scanning bot tries: the Mirai
   botnet took over hundreds of thousands of devices with exactly such
   factory passwords. Anything that ever listens on the network (`httpd`
   today, a future ssh) is only as safe as the weakest password. So the
   image must not contain *any* password that works, not even "just for
   testing".
6. **Test credentials for headless `make test`, without shipping them**:
   - The image in `out/foundry.img` stays locked (setup-needed).
   - `make test` makes its own copy (it already boots `out/test-v2.img`)
     and writes test accounts **into that copy only**, from the host,
     with e2fsprogs (`debugfs -w`): a `/etc/shadow` with hashes of
     throwaway passwords generated for this run (`openssl passwd -6`,
     random each time, kept in `out/`), and the setup marker removed.
   - The guest test script gets the passwords on its standard input from
     the host-side feed (the same way `scripts/hvc0-session.txt` already
     feeds the hvc0 console), never from a file in the image.
   - A check in `make test` fails if the release image has any shadow
     entry that is not locked.
7. **Owners and modes set by the image build** (`mke2fs -d` copies modes
   but takes owners from `root_owner`; per-file fixes go through
   `debugfs -w` `set_inode_field` or an ext2 build step that reads a
   small table like `root/permissions.txt`):

   | path | owner | mode |
   |---|---|---|
   | `/etc/shadow` | root:root | 0600 (or root:shadow 0640) |
   | `/etc/passwd`, `/etc/group` | root:root | 0644 |
   | `/bin/*`, `/sbin/*` | root:root | 0755 |
   | `/bin/su`, `/bin/passwd` (busybox links: the busybox binary they point to, or a separate setuid busybox with only these applets) | root:root | **4755** (setuid) |
   | `/root` | root:root | 0700 |
   | `/home/tyler` (test image) | tyler:tyler | 0700 |
   | `/tmp` | root:root | 1777 (sticky: you can only delete your own files) |
   | `/etc/foundry/setup-needed` | root:root | 0600 |

   FAT32 and exFAT (B:, C:) have no owners at all; what they *show* is
   up to the kernel's mount options (K13 (j)).

### Done when (checks `make test` can run)

On the test copy of the image, logged in through `getty → login` on hvc0
with the throwaway credentials:

1. Host side, before boot: `debugfs` shows `/etc/shadow` mode 0600 uid 0,
   `/bin/su` mode 4755 uid 0, `/home/tyler` uid 1000; the **release** image
   has only locked shadow entries and the setup marker present.
2. A wrong password prints `Login incorrect`; the right one gives a
   prompt, and `id` prints `uid=1000(tyler) gid=1000(tyler) groups=...`,
   `pwd` prints `/home/tyler`.
3. As tyler: `cat /etc/shadow` fails with `Permission denied`; writing to
   `/etc/passwd`, `/bin/ls` and `/root` fails; creating a file in
   `/home/tyler` works and `ls -l` shows it as `tyler tyler`, with the
   mode umask 022 gives (`-rw-r--r--`); a file chmodded 0600 by tyler is
   unreadable by a second test user.
4. `su` with root's test password gives uid 0 (`id -u` = 0); `su` with a
   wrong password fails; `exit` returns to tyler.
5. As root: `adduser`/`passwd` create a third user; the new `/etc/shadow`
   line starts with `$6$`; that user can log in on the console; `deluser`
   removes them and their login fails.
6. `chown` by tyler on a file of tyler's fails with `Operation not
   permitted`; by root it works.
7. First boot, simulated: booting the release image copy headless with the
   setup answers fed on the console leads to the setup program, then to a
   working login; booting again does not run setup again.

### Kernel prerequisites

All in [kernel-requests.md](kernel-requests.md), **K13** (open). Nothing
there is assumed to work until Edward's audit says so and our checks
pass: earlier probes showed ids and modes being *stored* but not
*enforced*.

## M7: Raspberry Pi 3 bootable build (placeholder)

Edward's arm64 / Raspberry Pi port of Rustbelt is in progress on the
kernel side. Once it boots, this milestone makes Foundry OS build an SD
card image for a Pi 3: `aarch64` builds of our Rust and C programs (a
second target for `foundry-cc` and Cargo), the Pi's boot partition
(firmware + kernel), and the same ext2 root / data / share layout. M6
comes first on purpose: a Pi on a network must not boot with a default
password. Details once the kernel's boot contract for the Pi is known.

## Long-term track: a self-hosted Rust toolchain (after M5, not started)

**The goal:** write a Rust program *on* Foundry OS, compile it there, and
run it there, without the box. Today every program is compiled on the box
(Linux) and copied into the image.

Words used below:

- **Compiler, toolchain.** A compiler turns source code (`main.rs`,
  `hello.c`) into a program the CPU can run. A *toolchain* is the compiler
  plus its helpers: the linker (glues the pieces into one file), the
  standard library, and for Rust, `cargo` (the build tool).
- **Cross-compiling.** Compiling on one system for another one. That is
  what we do now: the box's compilers make programs for Foundry OS.
- **Self-hosted.** A system that can build its own programs (in the end,
  its own compiler) on itself. Linux is self-hosted; Foundry OS isn't yet.
- **Bootstrap.** Getting the first compiler onto a system that has none:
  you cross-compile a compiler on another machine, copy it over, and from
  then on it can build things locally, even newer versions of itself.
- **Dynamic linker (`ld.so`).** Most Linux programs don't carry their
  libraries inside them. They name shared libraries (`.so` files), and a
  small program, the dynamic linker, loads those into memory when the
  program starts. Rustbelt has none today: it refuses such programs
  (`PT_INTERP`), so everything we run is *static* (all in one file).

**Hard parts, honestly:**

- **Size.** rustc is big: the compiler itself (most of it is LLVM, a
  huge C++ library) is about 150-200 MB, and the standard library for one
  target another 100-300 MB. Our whole root partition is 32 MiB today.
- **Memory.** LLVM is hungry: a hello world needs a few hundred MB of RAM,
  a real crate 1 GB or more. Our QEMU runs use QEMU's default of 128 MiB.
- **QEMU speed.** Our tests run QEMU without KVM, i.e. by emulating
  every instruction in software ("TCG"), roughly 5-20x slower than the
  box. With KVM (hardware virtualization) it's near native. Then add the
  kernel's own speed: Rustbelt's FAT driver re-walks the path on every
  read, for example, and nobody has measured page-fault or disk speed.
- **64 file descriptors per process** (`MAX_FDS` in the kernel). cargo,
  rustc and the linker can open more than that at once (many `.rlib`
  files, jobserver pipes, threads).
- **No `root=` option**, and only ext2 roots. A toolchain partition can't
  be chosen at boot. It has to sit on the one ext2 root (so the root must
  grow to a few GB) or be mounted elsewhere, e.g. as another ext2 disk
  that appears at `/mnt/diskN`.

### Stage 1: a C compiler that runs on Foundry OS

*In plain words:* first prove that "compile on the target" works at all,
with a small compiler. **tcc** (Tiny C Compiler, ~300 KB, compiles and
links in one program, needs no separate assembler or linker) is the
easiest first step. Then a real one: **gcc** (with binutils' `as` and
`ld`) or **clang/lld**, built static against our musl.

- *Size:* **M** for tcc (one port, like dash, plus its own small libc
  headers path); **L** for gcc/clang (big builds; gcc needs `as`, `ld`,
  `cc1` and lots of processes; clang is one big LLVM program, a good
  rehearsal for rustc).
- *Done when:* `make test` runs `tcc -static hello.c -o /tmp/hello &&
  /tmp/hello` on Foundry OS and prints the line; later the same with
  `gcc`, and on-target gcc builds `c/fprobe.c` and it passes 40/40.
- *OS-side:* ports/tcc, ports/binutils + ports/gcc (or clang), the musl
  headers and `crt*.o` installed into the root (`/usr/include`,
  `/usr/lib`), a bigger root partition.
- *Kernel (light, maybe nothing):* whatever the ports trip over, found
  the usual way (fprobe-style checks, docs/syscall-gaps.md).

### Stage 2: what rustc and cargo need from the system

*In plain words:* rustc and cargo are ordinary Linux programs, but big
and demanding ones. Before porting them, make sure the system can carry
them: room on disk, enough memory, fast enough files and memory mapping,
lots of threads and child processes, and a way to get libraries
(*crates*).

| Need | Why | Kernel or OS |
|---|---|---|
| **Either** a dynamic linker **or** a fully static rustc | Official rustc builds load `librustc_driver-*.so` at run time. Option A: Rustbelt loads `PT_INTERP` programs and we ship musl's `ld.so` (musl's `libc.so` is its own dynamic linker). Option B: build rustc statically (no `dylib` driver, `crt-static`, LLVM linked in). B needs no kernel work but a custom rustc build. | A: **kernel** (load the interpreter, pass `AT_BASE`/aux vector) + OS (ship `ld.so`). B: OS only |
| RAM: 1-2 GB | LLVM | **kernel**: confirm/raise how much RAM Rustbelt can use; OS: `-m 2G` in our QEMU runs |
| Disk: a few GB of ext2 | toolchain + `target/` dirs | OS (bigger root or a second ext2 disk); **kernel**: ext2 performance and large-file/large-dir behaviour |
| `mmap` of big files, `mremap` | rustc mmaps `.rlib`/metadata files; musl's `realloc` tries `mremap` (25, still ENOSYS, K10) | **kernel** (file-backed `mmap` speed, `mremap`) |
| More than 64 fds | rustc + linker + jobserver | **kernel** (raise `MAX_FDS`) |
| Threads, jobserver pipes | rustc uses threads; cargo hands out build slots through a pipe (`jobserver`) | mostly there (K-tests pass); check `poll` on pipes under load |
| Spawning many processes | cargo starts rustc, rustc starts the linker (`cc`/`ld`); Rust's `Command` uses musl `posix_spawn`, i.e. `clone(CLONE_VM \| CLONE_VFORK)` | **kernel** (check that clone flag set; today's table lists fork-like `SIGCHLD [+ CLONE_VFORK]` and threads) |
| `/proc/self/exe` and more `/proc` | rustc finds its sysroot through `current_exe()`; cargo reads `/proc/cpuinfo`-style info for job counts | `/proc/self/exe` exists; **kernel** for the rest (K7) |
| Crates | `cargo build` downloads from crates.io over **HTTPS** | Option A: TLS (an OS-side port, e.g. rustls in a tool or BearSSL) + DNS, both on our network stack. Option B (simpler first): an **offline vendored registry** (`cargo vendor` on the box, copied in, `--offline`). OS only |

- *Size:* **XL**. Each row is small to medium, but together they are a
  lot, and some (RAM, fds, the dynamic linker) are design changes in the
  kernel.
- *Done when:* a stress program (C or Rust, cross-built) on Foundry OS
  maps a 200 MB file, runs 16 threads, opens 200 fds, spawns 500
  children through `posix_spawn`, and the frame count is flat afterwards.
  Plus `free` shows ≥ 1.5 GB, and `df` shows ≥ 3 GB on the toolchain disk.
- *Kernel requests for Edward (kept light):* RAM limit, `MAX_FDS`,
  `mremap`, `CLONE_VM|CLONE_VFORK`, and (only if we choose option A) the
  `PT_INTERP` loader. Everything else is OS-side.

### Stage 3: rustc and cargo on Foundry OS

*In plain words:* cross-compile rustc and cargo on the box for
`x86_64-unknown-linux-musl` (the same Linux ABI Foundry OS speaks), copy
them onto a Foundry OS disk, and use them there. Our own `no_std`
programs (rbsh, the tools) need only `core`/`alloc` and our `rbrt`, so
they are a good first real test.

Cheaper wins on the way, each its own milestone:

1. `rustc --version` runs on Foundry OS (proves loading, size, memory).
2. `rustc hello.rs` (no cargo) builds a static hello that runs on Foundry
   OS. Needs rustc to find a linker: our on-target gcc/`ld` from stage 1,
   or rustc's bundled `rust-lld`.
3. `cargo build --offline` of a one-file crate.
4. **End:** `cargo build --offline --release` of `rust/rbsh` (our shell)
   **on Foundry OS**, and the resulting `rbsh` runs the v2 self-test.

- *Size:* **XL**. The cross-build of rustc itself is long (hours on the
  box, many GB of disk), and every failure on target is slow to debug
  under TCG.
- *Done when:* step 4 passes in a `make test-selfhost` (allowed to be
  slow, outside the normal `make test`).
- *OS-side:* the cross-build recipe (`ports/rust`, pinned like musl),
  the toolchain disk, the vendored registry, `make test-selfhost`.
- *Kernel:* nothing new beyond stage 2, plus whatever stage 3 finds.