foundry

# Requests to the kernel side (Rustbelt)

The kernel is Edward's lane; Foundry OS never edits `/workspace/rust-kernel`.
These are requests. The full list (K0-K11) with background is in
[proposal.md](proposal.md), section 8. K12 and K13 (below) are new since M3.

## How M1 works without any kernel change

The kernel's `build.rs` always builds *all* programs in its `user/` and its
own `target/disk.img`. That does **not** stop us: the disk image is not
inside the kernel's boot image. It is just a `-drive` argument to QEMU. So
`make test`/`make run` boot the kernel's unchanged boot image with **our**
`out/foundry.img` as the first virtio disk. The kernel's own `disk.img`
(built into our `out/kernel-target`) is only used as the reference for
`make compare` and `make test-reference`.

No workaround in the kernel build was needed. The only costs today:

- the kernel build still compiles the moved programs (wasted seconds), and
- the kernel's own `cargo run`/`cargo test` use the kernel's `disk.img`, not
  Foundry OS's. `Cargo.toml`'s `run-args`/`test-args` say
  `file=target/disk.img`, a relative path that also assumes the default
  target dir.

## K1: `RUSTBELT_DISK0` (small, optional for M1, wanted before the kernel deletes the moved programs)

> **Status: landed in Rustbelt 0.21.0 (50476f4).** Edward took this patch
> and went further: `CARGO_TARGET_DIR` is honored everywhere (his scripts
> and `Cargo.toml` paths, via the new `scripts/qemu.sh` runner), and the 19
> moved programs aren't even built when `RUSTBELT_DISK0` is set.
> `make k1-check` builds the pinned kernel that way in its own target dir
> (`out/kernel-k1-target`) and checks both: the kernel's `disk.img` is
> byte-identical to `out/foundry.img`, and no moved program was built.
> We still build the reference kernel *without* the variable, because
> `make test-reference` and `make compare` need the kernel's own disk.img.

**Request:** in `build.rs`, if the environment variable `RUSTBELT_DISK0` is
set to an image file, copy that file to `$CARGO_TARGET_DIR/disk.img` (or
`target/disk.img`) instead of building the FAT32 disk. Nothing changes when
it is unset.

```sh
RUSTBELT_DISK0=/workspace/foundry/out/foundry.img cargo test   # kernel tests on the Foundry OS image
```

The exact change, 16 added lines in `build.rs` only. Tested in a scratch
clone at a55b41d (`/workspace/foundry-scratch/rk`, branch `k1-experiment`,
own target dir): `cargo build` with the variable set produced a
`disk.img` byte-identical to `out/foundry.img`.

```diff
diff --git a/build.rs b/build.rs
index cc5505b..1f432dc 100644
--- a/build.rs
+++ b/build.rs
@@ -130,6 +130,9 @@ fn main() {
     println!("cargo:rerun-if-changed=vdso");
 
     build_vdso(&manifest, &out_dir);
+    // K1: an OS image from outside (Foundry OS's `make image`) as disk 0.
+    println!("cargo:rerun-if-env-changed=RUSTBELT_DISK0");
+    let external_disk0 = env::var_os("RUSTBELT_DISK0").map(PathBuf::from);
     for program in PROGRAMS {
         build_program(&user_dir, &out_dir, program);
     }
@@ -139,7 +142,10 @@ fn main() {
     for &(program, kind) in MUSL_PROGRAMS {
         build_musl_program(&user_dir, &out_dir, program, kind);
     }
-    build_disk(&manifest, &out_dir);
+    match &external_disk0 {
+        Some(img) => use_external_disk(&manifest, img),
+        None => build_disk(&manifest, &out_dir),
+    }
     build_ext2_disks(&manifest, &out_dir);
     build_exfat_disks(&manifest, &out_dir);
     build_btrfs_disks(&manifest, &out_dir);
@@ -263,6 +269,16 @@ fn run(cmd: &mut Command, stdin: Option<&str>) {
     assert!(status.success(), "{cmd:?} failed");
 }
 
+/// K1: copy an OS image built elsewhere to target/disk.img (instead of
+/// building one), so `cargo run`/`cargo test` boot it as disk 0.
+fn use_external_disk(manifest: &Path, img: &Path) {
+    println!("cargo:rerun-if-changed={}", img.display());
+    let target_dir = env::var("CARGO_TARGET_DIR").map(PathBuf::from).unwrap_or_else(|_| manifest.join("target"));
+    fs::create_dir_all(&target_dir).unwrap();
+    fs::copy(img, target_dir.join("disk.img"))
+        .unwrap_or_else(|e| panic!("RUSTBELT_DISK0={}: {e}", img.display()));
+}
+
 fn build_disk(manifest: &Path, out_dir: &Path) {
     let target_dir = env::var("CARGO_TARGET_DIR").map(PathBuf::from).unwrap_or_else(|_| manifest.join("target"));
     fs::create_dir_all(&target_dir).unwrap();
```

Follow-ups, in order (each optional):

1. When `RUSTBELT_DISK0` is set, also skip `build_program` for the moved
   programs (`sh ls cat wc grep mkdir rm touch cp sleep ifconfig ping host
   fetch httpd getty tile random sysinfo`). Their binaries only go into the
   disk the kernel no longer builds.
2. Proposal step 9: once both repos have been green for a while, delete
   the moved programs from `user/`. The kernel keeps its fixtures and a
   frozen `ulib.rs`. `tests/shell.rs` includes `user/shparse.rs`; it would
   either keep a copy or test only through the booted shell.
3. K5 (contract v2: ext2 root, `/sbin/init` as pid 1). **Landed in
   0.21.0** and used since M3 (docs/boot-contract.md). That is what makes
   Foundry OS the owner of what runs at boot.

## K12: clean shutdown when pid 1 exits (DONE: K12.5, Rustbelt 0.21.1)

> **Status: done in Rustbelt 0.21.1 (6bb2b25), Edward's "K12.5".** When
> pid 1 exits, `boot::shutdown()` syncs and unmounts every file system,
> newest mount first, before QEMU quits, and prints one line per mount:
>
> ```
> [shutdown] unmounted /mnt/share (exfat)
> [shutdown] unmounted /mnt/disk0 (fat32)
> [shutdown] unmounted /proc (proc)
> [shutdown] unmounted /dev (devfs)
> [shutdown] unmounted /tmp (ramfs)
> [shutdown] unmounted / (ext2)
> ```
>
> FAT32's unmount clears the dirty bit, so after `make test` the host's
> `fsck.fat -n` on B: is clean without any help. scripts/check-v2-image.sh
> now treats a set dirty bit as a hard **FAIL** (it was a WARN), and
> scripts/expect-v2.txt expects the `[shutdown]` lines.
>
> Still open, for later: `reboot` (syscall 169) is not implemented yet.
> When K7 adds it, `reboot(RB_POWER_OFF)` is meant to call the same
> `boot::shutdown()` and then end the QEMU run with an exit status we
> choose, so a `poweroff` command leaves the disks just as clean.

What we asked for (kept for the record):


Found by `make test` (v2), which boots a writable copy of our image and
checks it with `fsck` afterwards. ext2 (A:) and exFAT (C:) come back
clean. The FAT32 data disk (B:) comes back with its **dirty bit set**:

```
$ fsck.fat -n B-partition-after-the-run
There are differences between boot sector and its backup.
  65:01/00
Dirty bit is set. Fs was not properly unmounted and some data may be corrupt.
B-partition-after-the-run: 33 files, 2676/80628 clusters
```

The image before the run is clean (`fsck.fat -n` exit 0), and only byte 65
of the boot sector differs in that area. The `fatfs` crate sets this bit on
the first write and clears it in `FileSystem::unmount()` (or on drop);
Rustbelt never unmounts, and `run_init_v2` calls `exit_qemu_with` right
after pid 1 exits. Everything else on the volume is consistent (fsck is
clean once the bit is cleared), so no data is lost. But Windows, macOS and
Linux will all offer to "repair" the disk.

**Request:** when pid 1 exits (and in `reboot`/power-off later, K7),
unmount or flush the mounted file systems before QEMU exits. For FAT that
means calling fatfs's `unmount()`. Our host check
(scripts/check-v2-image.sh) printed a WARN instead of failing until 0.21.1.

## K13: users and permissions, enforced (OPEN; needed for M6)

**Status: open.** Edward has a read-only audit of this queued; until it
reports and our own checks pass, Foundry OS assumes **none** of it works.
What earlier probes showed (up to 0.18.0; not re-checked on 0.21.0):

- Modes, uid and gid are **stored** (`chmod`, the `chown` family read back
  on ramfs; new files are 0:0), and the id syscalls keep track of
  real/effective/saved ids with the right rules in themselves
  (`setresuid(1000,1000,0)` then `setuid(0)` works; after `setuid(1000)`
  going back is `EPERM`).
- But **permissions are not enforced**: `test -x` is always true and
  `access()` always returns 0 (proposal.md section 8: "permissions/ids not
  enforced"). The K6 list only asked for the setuid family as uid-0
  no-ops, so nothing so far was built or tested for a non-root user.
- FAT accepts `chown` to root:root and gives `EPERM` for 5:5.

What M6 (user management, docs/plan.md) needs. **Now** = M6 can't pass
without it; **nice** = can wait.

| | Item | Needed |
|---|---|---|
| (a) | Real per-process credentials: uid, gid, euid, egid, saved uid/gid, supplementary groups; inherited by `fork`, kept by `execve` (except setuid exec) | **now** |
| (b) | `setuid`/`setgid`/`setgroups`/`setresuid`/`setresgid`/`setreuid`/`setregid` that really change the credentials, with Linux's rules (only root may set arbitrary ids; others may only swap between their real/effective/saved ids; `setgroups` root only) | **now** |
| (c) | Permission checks from the mode bits, owner and groups (root bypasses read/write, and exec needs at least one `x` bit) on: `open` (read/write/`O_CREAT`/`O_TRUNC`), `execve`, `unlink`/`rmdir`/`rename` (write + search on the parent, sticky bit on `/tmp`), `mkdir`, `chdir`, path lookup (search `x` on every directory), `truncate` | **now** |
| (d) | **setuid / setgid bits on `execve`**: the program runs with the file owner's uid (gid) as effective and saved id; ignored on a `nosuid` mount and for scripts | **now** (for `su`, `passwd`) |
| (e) | `chown` only by root (a non-root owner may only change the group to one of their own groups); `chmod` only by the owner or root; a write by a non-root user clears setuid/setgid | **now** |
| (f) | ext2 stores uid, gid and the full mode (incl. setuid/setgid/sticky) **persistently**: after a reboot, and as written by `mke2fs -d`/`debugfs` on the host | **now** |
| (g) | `access`/`faccessat` (and `faccessat2`) answer from the **real** uid/gid, as Linux does | **now** |
| (h) | `umask` applied to the mode of new files and directories (`open`, `mkdir`, `mknod`) | **now** |
| (i) | `/dev/console`, `/dev/hvc0`, ptys: owner and mode that `login` can `chown`/`chmod` to the user (and `getty`/`login` can open the tty as root first) | **now** |
| (j) | FAT32 and exFAT (no owners on disk): show a fixed owner and mode, chosen per mount like Linux's `uid=`, `gid=`, `umask=`/`fmask=`/`dmask=` (even if only as kernel build-time options); refuse or ignore `chown` consistently | nice (B: and C: can stay root-owned and world-writable at first) |
| (k) | `/proc/<pid>/status` `Uid:`/`Gid:` lines (for `ps`, `id` checks in tests) | nice |
| (l) | Signals: a non-root process may only `kill` processes with the same real/effective uid | nice (needed before untrusted users) |

## K14: small gaps found by sbase (light; all nice-to-have)

M4 ran 86 checks of suckless sbase 0.1 (`root/scripts/sbase-test.sh`,
part of `make test`) on Rustbelt 0.21.1. 79 pass: sorting, sed, find,
xargs, dd, `ls -l`, `chmod`, links, `touch`, `cp -p`, checksums, `kill`,
`nice`, `date`... Only these need the kernel. None blocks anything today;
the test runs each one as an expected failure ("xfail") and turns red when
it starts working, so we notice.

| | Gap | Evidence (Rustbelt 0.21.1, 6bb2b25) | Who notices |
|---|---|---|---|
| (a) | `mknod`/`mknodat` (133/259) missing, so no named pipes (FIFOs) | probe: `mknod(S_IFIFO)` on /tmp and `mkfifo` on ext2 both `-1 errno 38 (ENOSYS)` (Linux: 0); `sbase/mkfifo fifo` → "Function not implemented"; no `mknod` in `syscall.rs`'s dispatch | `mkfifo`, shell scripts that use FIFOs, busybox `mkfifo` |
| (b) | `chroot` (161) missing | probe: `chroot("/")` = `-1 errno 38` (Linux as root: 0); `sbase/chroot / true` fails | `chroot`; later, build sandboxes |
| (c) | `getdents64` never returns `.` and `..`; a directory's `st_size` is its entry count (ext2 /etc: `st_size` 6, `st_blocks` 1; Linux: 1024 or 4096) | probe: `readdir(/etc)` gives 6 entries, none of them `.`/`..` (Linux: 2 of them); `fs/mod.rs:114` "All children of this directory (without "." and "..")"; `ls -a` shows no dot entries | `ls -a`, `find`-style tools that expect them; `du` on directories counts a bit low |
| (d) | FAT32/exFAT keep no file times: `stat` shows 0 (1970) and `utimensat` is accepted and forgotten | `touch -t 200001020304` on `/mnt/disk0` exits 0 but `ls -l` still shows `Jan 01 1970`; `fs/mod.rs` `set_times` default "FAT and exFAT have time fields, but our drivers don't write them yet" | `ls -l`, `make`, `cp -p` to B:/C:, files copied to a PC |

Not kernel issues (ours, or just sbase): `df` and `id` are not in sbase 0.1
(busybox at M5 has them; `statfs` already exists); `logname` fails because
nothing sets `LOGNAME` yet (M6's `login` will); sbase has no `sed -i`,
`head -c`, `od -c` or fractional `sleep`. An SIGINT sent to a background
`sleep` is ignored, which looked like a bug but is right: dash sets SIGINT
to "ignore" for background jobs and that survives `exec` (probe: a
`SIG_IGN` child's `nanosleep(3s)` returns 0 after 3.00 s, a handler's
returns EINTR with `rem` 2.70 s, `SIG_DFL` is killed by signal 2, same as
Linux).

## What Foundry OS still takes from the kernel repo

| What | How | Goes away when |
|---|---|---|
| the kernel itself | `git archive $(KERNEL_REV)` (now 6bb2b25) → `out/kernel-src`, built in `out/kernel-target` | never (that's the point), but bumps are deliberate |
| the 28 ABI fixtures (v2: in `B:/bin`, i.e. `/mnt/disk0/bin`, last on PATH) | compiled from `out/kernel-src/user` | the self-test stops running them (they move to the kernel's own tests) |
| the C:/D:/E: test disks (v1 tests only) | `ext2.img`, `exfat.img`, `btrfs.img` from the kernel build | stays: they are kernel-owned fixtures |
| the expected self-test lines | v1: copied into `scripts/run-headless.sh` (same 76 + 12 checks); v2: ours, in `scripts/expect-v2.txt` | — (ours now; kernel-version lines must be updated on a bump) |
| `scripts/http-reply.sh`, `scripts/hvc0-session.txt` | copied at a55b41d | — (ours now) |
| Debian's `musl-gcc` | builds the 5 musl fixtures, as the kernel does | the fixtures leave the image |