foundry

@@ -1,4 +1,88 @@ # Foundry OS -The operating system (userland) that runs on the Rustbelt kernel.-Full README follows in a later commit.+**Foundry OS** is a small operating system that runs on the **Rustbelt**+kernel, the way Ubuntu runs on Linux. The kernel (in its own repo,+`/workspace/rust-kernel`) drives the hardware; Foundry OS is everything+else: the shell `rbsh`, the tools (`ls`, `cat`, `grep`...), the network and+system programs, and the disk image they live on.++This repo builds **one file**, `out/foundry.img`, a virtual hard disk. The+kernel boots and runs the shell from it.++## Quick start++```sh+make            # build out/foundry.img (the same as `make image`)+make test       # boot it headless in QEMU and run the shell self-test+make run        # boot it in a QEMU window and play+make help       # every target+```++The first `make test` also builds the pinned kernel (a few minutes). You need+the packages the kernel needs plus `make` (all already on the box):+`rustup` (nightly), `qemu-system-x86`, `dosfstools`, `mtools`, `fdisk`,+`e2fsprogs`, `exfatprogs`, `btrfs-progs`, `musl-tools`, `gcc`.++## Make targets++| Target | What it does |+|---|---|+| `make image` | builds the Rust programs, the kernel's test fixtures, assembles `out/staging/`, and writes `out/foundry.img` |+| `make rust` | `cargo build --release` in `rust/` |+| `make fixtures` | builds the kernel's ABI test programs from the pinned kernel source |+| `make kernel` | builds the pinned Rustbelt kernel into `out/kernel-target` |+| `make run` | boots the image in a QEMU window (`NOGRAPHIC=1` for serial only) |+| `make test` | headless boot, self-test, ~90 expected-output checks; exit 0 = pass |+| `make test-reference` | the same test on the kernel's *own* disk image (the baseline) |+| `make compare` | compares our image with the kernel's, file by file |+| `make musl` | downloads (into `cache/`) and builds musl into `out/sysroot` (for M2) |+| `make clean` | `rm -rf out` (`make distclean` also empties `cache/`) |++## What is where++```+foundry/+├── Makefile, config.mk   the build; config.mk pins the kernel commit+├── rust/                 the Rust programs (a Cargo workspace)+│   ├── rbsh/             the shell (installed as /bin/sh for now)+│   ├── coreutils-rs/     ls cat wc grep mkdir rm touch cp sleep+│   ├── net/              ifconfig ping host fetch httpd+│   ├── sys/              getty tile random sysinfo+│   ├── rbrt/             the runtime every program includes (ulib.rs) + shared build.rs+│   └── link.ld           puts programs at address 0x80_0040_0000+├── rootfs/               files copied onto the disk as they are (readme, notes, the self-test)+│   └── etc/              passwd, hosts... for the later ext2 root (not in the image yet)+├── toolchain/musl/       musl C library: pinned by SHA256, downloaded into cache/+├── c/                    our own C programs (M2)+├── ports/                other people's C programs, one directory each (dash first)+├── scripts/              mkimage, build-kernel, run-qemu, run-headless, compare-image...+├── docs/                 how it all fits together (start with docs/glossary.md)+├── out/                  everything built (git-ignored)+└── cache/                downloads (git-ignored)+```++## How it fits with the kernel++The kernel repo and this repo meet in exactly one place: the **disk image**.+[docs/boot-contract.md](docs/boot-contract.md) says what the image must+contain. QEMU gets our image as its first virtio disk, the kernel mounts it at+`/mnt/disk0` (the shell calls it drive `B:`) and runs `/mnt/disk0/bin/sh`.++We build against **one exact kernel commit** (`KERNEL_REV` in `config.mk`,+now `a55b41d` = Rustbelt 0.19.0). The build exports that commit from the+kernel repo's git store with `git archive` and compiles it in `out/`, so the+kernel repo itself is never changed. To move to a newer kernel, change+`KERNEL_REV`, run `make test`, and commit.++## Where this came from++The programs were in the kernel repo's `user/` folder. They were brought here+with **`git subtree split`**, which keeps their full history: try+`git log --follow rust/coreutils-rs/src/bin/ls.rs`. The kernel keeps its ABI+test programs (`forktest`, `rawabi`...), and for now the image still carries+them because the self-test runs them. See [docs/plan.md](docs/plan.md) for+what comes next and [docs/proposal.md](docs/proposal.md) for the full+background.++New to these words? **[docs/glossary.md](docs/glossary.md)** explains+sysroot, static-PIE, subtree, GPT, musl and the rest.

@@ -0,0 +1,52 @@+# The boot contract: what the kernel expects from the Foundry OS image++The Rustbelt kernel and Foundry OS live in different repos. This page is the+whole agreement between them. The kernel never needs to know *how* the+image was built, only that it follows the contract.++## Contract v1: "today, written down" (in use now)++Pinned kernel: Rustbelt 0.19.0, commit a55b41d. No kernel change needed.++1. **Where.** The OS image is the **first virtio-blk disk** QEMU is given.+   The kernel mounts it at `/mnt/disk0` (drive `B:` in rbsh).+2. **Format.** A **GPT** partition table with one partition starting at+   sector 2048 (1 MiB), type "Microsoft basic data", formatted **FAT32**,+   volume label `TYLERDISK` (kept from the kernel for now). 64 MiB total.+3. **Contents.**+   - **`/bin/sh`**: the program the kernel starts. In Foundry OS this is+     **rbsh** (crate `rust/rbsh`, binary `rbsh`), copied under the name `sh`.+   - `/scripts/selftest.sh`: the self-test, for headless kernels.+   - Programs live in `/bin`. Each is a **static** ELF: either `ET_EXEC`+     linked at or above `0x80_0040_0000`, or (since 0.19.0) a+     **static-PIE** (`ET_DYN` without `PT_INTERP`). Dynamic programs are+     refused.+4. **What the kernel provides.** `/` is a ramfs the kernel builds itself+   (with `/tmp`, `/dev`, `/proc`, `/mnt`). It runs `/mnt/disk0/bin/sh`+   with `PATH=/bin:/mnt/disk0/bin HOME=/ TERM=linux`, and restarts it when+   it exits. A headless kernel (`--features headless`) instead runs+   `sh /mnt/disk0/scripts/selftest.sh` once and powers QEMU off with the+   shell's status: QEMU exit code **33 = pass**, 35 = fail.+5. **Other disks belong to the kernel.** The ext2, exFAT and btrfs test+   disks are built by the kernel's build into `out/kernel-target`. Our run+   scripts attach them after our disk, so `C:`, `D:`, `E:` keep their letters.++Under v1 the image must also carry the kernel's ABI test programs+(`forktest`, `rawabi`, ...), because the self-test runs them. They are+built from the pinned kernel source (`make fixtures`); see+[kernel-requests.md](kernel-requests.md) for how that dependency goes away.++## Contract v2: a real root file system (planned, needs kernel request K5)++1. The OS image is GPT with a partition of type **Linux root (x86-64)**,+   `4F68BCE3-E8CD-4DB1-96E7-FBCAF984B709` (the freedesktop *Discoverable+   Partitions* rule), formatted **ext2**. The kernel mounts it at **`/`**,+   so `A:` = `/` is the OS disk.+2. The kernel keeps a ramfs at `/tmp`, devfs at `/dev`, and mounts the other+   disks at `/mnt/diskN`.+3. The kernel runs the first that exists of `/sbin/init`, `/bin/init`,+   `/bin/sh` as **pid 1**, does not respawn it, and hands orphans to it.+   Headless: `["init", "selftest"]`; pid 1's exit status ends QEMU.+4. Fallback: no root partition → behave as v1.++See [plan.md](plan.md) for the disk layout we chose for v2.

@@ -0,0 +1,28 @@+# Glossary: the words used in Foundry OS++| Word | What it means here |+|---|---|+| **kernel** | The program that owns the hardware: memory, CPUs, disks, network. Ours is **Rustbelt**, in its own repo. |+| **userland** | Everything that is not the kernel: the shell, `ls`, the C library. It runs in the CPU's unprivileged mode ("ring 3"). Foundry OS *is* a userland. |+| **syscall** | A request a program makes to the kernel ("open this file", "start this program"). Rustbelt uses Linux's syscall numbers, plus a few of its own (500-505). |+| **disk image** | A plain file holding every byte of a virtual hard disk. QEMU uses it as a disk. Ours is `out/foundry.img`. |+| **GPT** | The partition table at the start of the disk: it says where each partition starts and what it is for. Ours has one partition, at sector 2048 (1 MiB). |+| **FAT32 / ext2 / exFAT** | File system formats. FAT32 is what the image uses today (simple, no permissions or symlinks). ext2 is the Unix-style format we will move the OS to. exFAT is for sharing files with Windows/macOS. |+| **mtools** | Programs (`mcopy`, `mmd`) that write files *into* a FAT image file directly, so building the image needs no root and no mounting. |+| **QEMU** | A program that pretends to be a whole PC, so we can boot Rustbelt + Foundry OS without real hardware. "Headless" means no window: output goes to the terminal and a log. |+| **boot contract** | The short agreement between kernel and OS about what the disk image must contain: [boot-contract.md](boot-contract.md). |+| **pinned commit** | One exact version of the kernel (`KERNEL_REV` in `config.mk`). We build against it until we change it on purpose. |+| **`no_std`** | Rust without its standard library (no files, threads or `println!` from Rust itself). Our Rust programs bring their own small runtime, `rbrt`. |+| **Cargo workspace** | Several Rust packages built together with one `cargo build` and one `Cargo.lock` (`rust/Cargo.toml`). |+| **ELF** | The executable file format of Linux and Rustbelt. `ET_EXEC` = a program fixed at one address; `ET_DYN` = position-independent (a shared library or a PIE). |+| **static binary** | A program with everything it needs copied into the one file: no `.so` libraries, no dynamic loader at run time. Rustbelt has no dynamic loader, so all our programs are static. |+| **PIE / static-PIE** | **P**osition-**I**ndependent **E**xecutable: a program that can be loaded at any address. A *static-PIE* is static **and** position-independent: it fixes up its own addresses when it starts. Rustbelt 0.19.0 can load these, so C programs can be built with the ordinary `musl-gcc -static-pie` and no special link tricks (the old `rb-cc` wrapper is not needed any more). |+| **libc** | The C standard library (`printf`, `malloc`, `fopen`). It turns those calls into syscalls. |+| **musl** | A small, clean libc (Alpine Linux uses it). Easy to read, easy to link statically. Our C programs will use it. |+| **sysroot** | A folder that looks like the root of the *target* system as far as the compiler cares: `include/` (headers like `stdio.h`) and `lib/` (`libc.a`, start files `crt1.o`...). The compiler reads those instead of the build machine's own. `make musl` makes `out/sysroot`. |+| **cross toolchain** | A compiler setup that builds programs for a different system than the one it runs on (here: for Rustbelt, on Debian). |+| **tarball** | A `.tar.gz` archive of a project's source code, as released. We never commit them: they are downloaded into `cache/` and checked by **SHA256**, a fingerprint of the exact bytes. |+| **port** | Someone else's program (dash, busybox) adapted to build for our system: `ports/<name>/`. |+| **fixture** | A test program that checks the *kernel* (fork, signals, raw syscalls). The kernel repo owns them; they are not part of Foundry OS. |+| **git subtree split** | A git command that takes one folder of a repo (here the kernel's `user/`) and makes a new history containing only that folder, with all its old commits. We merged that history in here, so `git log --follow` still shows how each file grew. |+| **drive letters** | `A:`, `B:`... are **rbsh** syntax only (DOS style): `A:` = `/`, `B:` = `/mnt/disk0` (our disk), `C:`-`E:` = the other disks. The kernel knows nothing about them. |

@@ -0,0 +1,109 @@+# Requests to the kernel side (Rustbelt)++The kernel is Edward's lane; Foundry OS never edits `/workspace/rust-kernel`.+These are requests. The full list (K0-K11) with background is in+[proposal.md](proposal.md), section 8.++## How M1 works without any kernel change++The kernel's `build.rs` always builds *all* programs in its `user/` and its+own `target/disk.img`. That does **not** stop us: the disk image is not+inside the kernel's boot image. It is just a `-drive` argument to QEMU. So+`make test`/`make run` boot the kernel's unchanged boot image with **our**+`out/foundry.img` as the first virtio disk. The kernel's own `disk.img`+(built into our `out/kernel-target`) is only used as the reference for+`make compare` and `make test-reference`.++No workaround in the kernel build was needed. The only costs today:++- the kernel build still compiles the moved programs (wasted seconds), and+- the kernel's own `cargo run`/`cargo test` use the kernel's `disk.img`, not+  Foundry OS's. `Cargo.toml`'s `run-args`/`test-args` say+  `file=target/disk.img`, a relative path that also assumes the default+  target dir.++## K1: `RUSTBELT_DISK0` (small, optional for M1, wanted before the kernel deletes the moved programs)++**Request:** in `build.rs`, if the environment variable `RUSTBELT_DISK0` is+set to an image file, copy that file to `$CARGO_TARGET_DIR/disk.img` (or+`target/disk.img`) instead of building the FAT32 disk. Nothing changes when+it is unset.++```sh+RUSTBELT_DISK0=/workspace/foundry/out/foundry.img cargo test   # kernel tests on the Foundry OS image+```++The exact change, 16 added lines in `build.rs` only. Tested in a scratch+clone at a55b41d (`/workspace/foundry-scratch/rk`, branch `k1-experiment`,+own target dir): `cargo build` with the variable set produced a+`disk.img` byte-identical to `out/foundry.img`.++```diff+diff --git a/build.rs b/build.rs+index cc5505b..1f432dc 100644+--- a/build.rs++++ b/build.rs+@@ -130,6 +130,9 @@ fn main() {+     println!("cargo:rerun-if-changed=vdso");+ +     build_vdso(&manifest, &out_dir);++    // K1: an OS image from outside (Foundry OS's `make image`) as disk 0.++    println!("cargo:rerun-if-env-changed=RUSTBELT_DISK0");++    let external_disk0 = env::var_os("RUSTBELT_DISK0").map(PathBuf::from);+     for program in PROGRAMS {+         build_program(&user_dir, &out_dir, program);+     }+@@ -139,7 +142,10 @@ fn main() {+     for &(program, kind) in MUSL_PROGRAMS {+         build_musl_program(&user_dir, &out_dir, program, kind);+     }+-    build_disk(&manifest, &out_dir);++    match &external_disk0 {++        Some(img) => use_external_disk(&manifest, img),++        None => build_disk(&manifest, &out_dir),++    }+     build_ext2_disks(&manifest, &out_dir);+     build_exfat_disks(&manifest, &out_dir);+     build_btrfs_disks(&manifest, &out_dir);+@@ -263,6 +269,16 @@ fn run(cmd: &mut Command, stdin: Option<&str>) {+     assert!(status.success(), "{cmd:?} failed");+ }+ ++/// K1: copy an OS image built elsewhere to target/disk.img (instead of++/// building one), so `cargo run`/`cargo test` boot it as disk 0.++fn use_external_disk(manifest: &Path, img: &Path) {++    println!("cargo:rerun-if-changed={}", img.display());++    let target_dir = env::var("CARGO_TARGET_DIR").map(PathBuf::from).unwrap_or_else(|_| manifest.join("target"));++    fs::create_dir_all(&target_dir).unwrap();++    fs::copy(img, target_dir.join("disk.img"))++        .unwrap_or_else(|e| panic!("RUSTBELT_DISK0={}: {e}", img.display()));++}+++ fn build_disk(manifest: &Path, out_dir: &Path) {+     let target_dir = env::var("CARGO_TARGET_DIR").map(PathBuf::from).unwrap_or_else(|_| manifest.join("target"));+     fs::create_dir_all(&target_dir).unwrap();+```++Follow-ups, in order (each optional):++1. When `RUSTBELT_DISK0` is set, also skip `build_program` for the moved+   programs (`sh ls cat wc grep mkdir rm touch cp sleep ifconfig ping host+   fetch httpd getty tile random sysinfo`). Their binaries only go into the+   disk the kernel no longer builds.+2. Proposal step 9: once both repos have been green for a while, delete+   the moved programs from `user/`. The kernel keeps its fixtures and a+   frozen `ulib.rs`. `tests/shell.rs` includes `user/shparse.rs`; it would+   either keep a copy or test only through the booted shell.+3. K5 (contract v2: ext2 root, `/sbin/init` as pid 1). That is what makes+   Foundry OS the owner of what runs at boot.++## What Foundry OS still takes from the kernel repo++| What | How | Goes away when |+|---|---|---|+| the kernel itself | `git archive a55b41d` → `out/kernel-src`, built in `out/kernel-target` | never (that's the point), but bumps are deliberate |+| the 25 ABI fixtures in `/bin` | compiled from `out/kernel-src/user` | the self-test stops running them (they move to the kernel's own tests) or contract v2 |+| the C:/D:/E: test disks | `ext2.img`, `exfat.img`, `btrfs.img` from the kernel build | stays: they are kernel-owned fixtures |+| the expected self-test lines | copied into `scripts/run-headless.sh` (same 76 + 12 checks) | — (ours now; kernel-version lines must be updated on a bump) |+| `scripts/http-reply.sh`, `scripts/hvc0-session.txt` | copied at a55b41d | — (ours now) |+| Debian's `musl-gcc` | builds the 5 musl fixtures, as the kernel does | the fixtures leave the image |

@@ -0,0 +1,43 @@+# Foundry OS: decisions and plan++## Tyler's answers to the proposal (2026-10-31)++| # | Question | Decision |+|---|---|---|+| 1 | Name, place, history | The OS is **Foundry OS**, in `/workspace/foundry` (local git only: no remote, no push). The kernel's `user/` and `disk/` histories were kept with `git subtree split`, done in a scratch clone at a55b41d (`/workspace/foundry-scratch/rk`), never in `/workspace/rust-kernel`. |+| 2 | What stays in the kernel | The ABI test programs stay as kernel fixtures: `hello evil spin demo child overflow forktest cowtest mmaptest sigtest threads echoserver ptytest nbtest fputest pathtest libctest coretest unixtest rawabi` and the musl ones `vdsotest piehello piethreads dynhello remaptest`. The shell, the tools and the net/sys programs moved here. |+| 3 | Which shell | The Rust shell stays the interactive login shell, renamed **rbsh**, keeping its DOS commands (`run`/`start`, `dir`/`type`/`cls`...) and drive letters. **dash** (later busybox `ash`) becomes `/bin/sh` for POSIX scripts once the root is ext2. Under contract v1 rbsh is still installed as `/bin/sh`, because that is the path the kernel runs. |+| 4 | Root file system | Later: an **ext2 root**, so `A:` = `/` is the OS disk. The FAT32 disk stays as a **data disk** (`B:`). Plus an **exFAT data partition** for sharing files with Windows/macOS. M1 keeps contract v1 (today's layout). |+| 5 | Name clashes | The Rust tools stay the default. busybox/sbase applets go under **`/usr/lib/busybox`**, off the default `PATH` (call them by full path, or add the directory to `PATH` yourself). |+| 6 | Tarballs | **Not in git.** A git-ignored download cache (`cache/`), each file pinned by SHA256 (`toolchain/musl/musl.sha256` + `fetch.sh`; `ports/<name>/<name>.sha256`). |+| 7 | Which kernel | A **pinned commit**: `KERNEL_REV=a55b41d` (Rustbelt 0.19.0) in `config.mk`, bumped deliberately. |+| 8 | Box packages | Small build packages are OK (`make`, `bzip2`, `musl-tools`, `autoconf`). |++## Disk layout later (contract v2, not built yet)++```+foundry.img (GPT)+├── 1  ext2   "Linux root (x86-64)" type   → mounted at /       (A:)  the OS: /bin /sbin /etc /usr ...+└── 2  exFAT  "Microsoft basic data" type  → /mnt/diskN          data shared with other computers+data.img (separate disk, today's FAT32)    → /mnt/disk0          (B:)  a plain data disk+```++Inside the ext2 root:++```+/bin       rbsh, the Rust tools (ls cat grep ...), dash as /bin/sh+/sbin      init (Rust, M3)+/etc       passwd group hosts resolv.conf motd profile rc   (rootfs/etc/)+/usr/lib/busybox   busybox + applet symlinks, off the default PATH (M5)+```++## Milestones++| | Goal | Status |+|---|---|---|+| **M1** | Foundry OS builds the same image; the kernel's self-test passes on it | **done**: `make test` passes; `make compare` shows the same 44 programs and identical non-program files |+| M1b | `rbrt` becomes a real library crate; rbsh's parser tests run on the host | next |+| M2 | musl hello + probes in `/bin`, built with plain `musl-gcc -static-pie` | the toolchain pieces exist (`make musl`, `c/hello-musl.c`) |+| M3 | dash as `/bin/sh`, ext2 root (contract v2), Rust `/sbin/init` | needs kernel K5 |+| M4 | sbase tools | needs testing per tool (docs/syscall-gaps.md) |+| M5 | busybox in `/usr/lib/busybox` | `bzip2` for the tarball |

@@ -0,0 +1,48 @@+# Porting a C program to Foundry OS++*Status: groundwork only (milestone M2 adds the first C programs to the image).*++## The toolchain++Rustbelt 0.19.0 loads **static-PIE** programs, so C programs need no special+tricks any more. With Debian's musl (`musl-tools`):++```sh+musl-gcc -O2 -static-pie c/hello-musl.c -o out/hello-musl+```++That makes an `ET_DYN` file with no `PT_INTERP`: a static-PIE. It runs on the+Linux host too, which is handy (configure scripts run their test programs).++The old `rb-cc` wrapper from the proposal (crt1.o, `-Ttext-segment`,+`--defsym=_DYNAMIC`, `--no-relax`) was only needed for `ET_EXEC` at+Rustbelt's high user base. Don't use it for new programs. The kernel keeps+one program built that way (`vdsotest`) as a loader test.++## Our own musl (`make musl`)++`make musl` downloads musl into `cache/` (SHA256-checked, see+`toolchain/musl/`) and installs a static-only musl into `out/sysroot`+(`include/`, `lib/libc.a`, the `crt*.o` start files, and a `bin/musl-gcc`+wrapper).++**Known gap (M2 to-do):** musl's own generated `musl-gcc` wrapper does not+handle `-static-pie`. Its specs file always adds the dynamic-loader path, so+the result gets a `PT_INTERP` and is refused. Debian's `musl-gcc` handles+this properly, so use that until `toolchain/musl/` ships its own+`musl-gcc.specs` (or a small `foundry-cc` wrapper) with the `static-pie` case.++## Adding a port++```+ports/<name>/<name>.sha256   "SHA256  <tarball>"+ports/<name>/<name>.url      where to download it+ports/<name>/patches/        0001-*.patch ...+ports/<name>/build.sh        unpack from cache/, patch, CC=musl-gcc ./configure --host=x86_64-linux-musl,+                             make, install into out/staging (busybox/sbase: under /usr/lib/busybox)+```++Then boot it (`make test` / `make run`), try it, and note any missing+syscall in [syscall-gaps.md](syscall-gaps.md). `run --trace prog` in rbsh+shows every syscall, and `syscall_N(...) = -38 ENOSYS` marks the ones+Rustbelt doesn't have.

@@ -0,0 +1,18 @@+# Syscall gaps: which program needs what the kernel lacks++A living list. When a port hits a missing or wrong syscall, add a line here+and ask the kernel side (see kernel-requests.md). Status from Clarke's tests+up to Rustbelt 0.18.0 (proposal section 2.6); 0.19.0 added the vDSO,+static-PIE and `mremap`.++| Gap | Who notices | Status |+|---|---|---|+| `AF_INET6` (EAFNOSUPPORT) | IPv6-only tools | open |+| `ITIMER_VIRTUAL`/`ITIMER_PROF`, CPU-clock timers | profilers, CPU watchdogs | open |+| `timerfd_*` 283/286/287 | event-loop programs | open |+| `/proc/<pid>`, `/proc/mounts`, `/proc/meminfo` | `ps`, `top`, `mount`, `free`, `df` | open |+| `reboot` 169, `mount`/`umount` | `poweroff`, busybox init, `mount` | open |+| permissions stored, not enforced | `test -x` always true | accepted for now |+| no `/etc/passwd` | `getpwuid(0)` fails, `ls -l` owner names | Foundry OS job: `rootfs/etc/` (ships with contract v2) |+| `mremap` 25 | big `realloc`s copied | fixed in 0.19.0 (to re-check) |+| `ttyname()`, fd-message merging, `/proc/self/cwd` (K11) | `tty` | fixed in b0a701d (to re-check) |