# Developing This chapter is the working manual for anyone who builds KDOS from source or changes it. It covers what a development machine needs, how to get from a fresh clone to a bootable ISO, how to rebuild one small thing instead of everything, how to boot and inspect the result, what can be checked with no build at all, and how a release is cut. It assumes you know what KDOS is; for the build told as one continuous story from `git clone` to a bootable image, read [How KDOS is built](how-kdos-is-built.md) first. The mechanism behind `make build` is in [The build system](build-system.md), the recipe format in [Writing ports](writing-ports.md), and the test harnesses in [Testing](testing.md). Terms such as *port*, *pack* and *box* are defined in the [glossary](../06-reference/glossary.md). The path from a clone to a running system has five steps: 1. Clone the repository and enable the pre-push check. 2. Run `make fetch` to download every upstream source. It is the only step the build needs the network for; apart from building container images, `make build` downloads nothing. 3. Run `make build` to compile everything into `build/iso-build/kdos.iso`. 4. Run `make run` to boot the ISO in a virtual machine. 5. After a change, rebuild only what changed, using the table in [Rebuilding one thing](#rebuilding-one-thing). ## What a development machine needs | Need | For | Notes | |---|---|---| | Docker | `make build`, `make run-hw` | The build image carries every compiler the build uses | | Network access for image builds | The first `make build`, the first `make run-hw` | Building the `os-dev` image pulls Alpine and its packages, again whenever the `Dockerfile` changes; the `run-hw` image is built from Ubuntu on first use | | Docker or podman | Generating a vendor bundle during `make fetch` | Only when the archive and the cache lack one; see [Where sources come from](#where-sources-come-from) | | `curl`, `sha256sum` | `make fetch` | All ordinary downloading happens on your machine | | `git`, `curl`, and network access to the source archive | The pre-push hook | The hook reads hashes with `git grep` and checks each one against the archive with `curl` | | A C compiler (`$CC`, default `cc`) | `make fetch-check`, `make snapshots`, `ports/fetch --tree`, `ports/publish`, `testing/preflight.sh`, `testing/selftest.sh`, `make updates` | Optional for `make fetch`: without one, the whole fetch runs inside the `kdos-fetch` container. `make updates` (`ports/update`) always runs `cc` and ignores `$CC` | | `fuser` (from psmisc) | `check-iso-free`, which `make build` runs first | Without it the check is skipped, and nothing stops a build rewriting an ISO a virtual machine is booted from | | QEMU, OVMF firmware at `/usr/share/ovmf/OVMF.fd`, and `/dev/kvm` | `make run`, `make rundisk` | Only to boot the result. Both pass `-enable-kvm -cpu host`, so without `/dev/kvm` they print a warning and QEMU then refuses to start | | Disk space | Everything | See below | No language toolchain (Rust, Go, Node, Python, Haskell) is needed on your machine. The one step that uses them, generating a vendor bundle, runs them inside a container. No step needs you to log in as root or use `sudo`, but two things amount to root access. Permission to use Docker is equivalent to root on that machine, since a container can mount any part of it. And `build/fs`, the target root filesystem, is owned by root, so reading or deleting it takes root or a container; see [Where the build puts things](#where-the-build-puts-things). **Disk.** The upstream sources for the current tree take about 41.5 GB (38.6 GiB): the 2,448 distinct files the recipes fetch, which is what `ports/.srccache/` holds after a complete `make fetch`. Each distinct file is held once in the cache and hard-linked into every port directory that names it. Budget tens of gigabytes more for `build/`, and more again for phase snapshots: the first of the eleven phases from `00_cross` to `50_desktop` archives the target tree compressed, each later one a layer of what it added, changed or rewrote, and `60_kernel` and `70_image` take none. On a synthetic ten-phase tree the layers came to 23% of what full copies take; a complete set from a real build has not been measured. Snapshots are optional, per run and per phase; see [Snapshots](build-system.md#snapshots). ## Getting the source ```sh git clone kdos && cd kdos git config core.hooksPath script/hooks # the pre-push check; once per clone make fetch # downloads every source ``` A clone carries recipes, patches and KDOS's own code. The upstream source archives the recipes name are not in git. `make fetch` downloads each one, checks it against the `sha256 =` line in its recipe, and puts it in its port directory. After that, the build itself runs with no network (the orchestrator's container is started with `--network none`) for as long as the recipes do not change. Only building the `os-dev` image, the first time or after the `Dockerfile` changes, downloads Alpine packages. Run `make fetch` again after pulling or switching branches; it downloads only what is new. `make fetch-check` confirms, offline, that nothing is missing or corrupt. `git config core.hooksPath script/hooks` points git at `script/hooks/`. Its `pre-push` hook refuses a push whose recipes name a source the archive does not hold, so that a recipe you publish can be fetched by everyone else. Where each source is kept, how `make fetch` finds it and what the hook checks are described in [Where sources come from](#where-sources-come-from). ## The first build ```sh make build # compile everything; the orchestrator runs with no network ``` `make build` builds the `os-dev` container image from the repository's `Dockerfile` (Alpine 3.23 with GCC, musl, bash and the other tools the orchestrator needs), then runs the orchestrator inside it with `--network none`, `--privileged` and `--cpu-shares=256`, a weight that lets an idle host give the build every thread; `KDOS_JOBS=N` also caps it with `--cpus`. The repository is mounted with `build/` writable and `src/`, `fs/`, `script/` and `ports/` read-only. The current commit and whether the working tree is dirty are passed in and recorded with each snapshot. Your user and group ids are passed in as well, and when the orchestrator exits it hands everything under `build/` back to you, except `build/fs`; see [Where the build puts things](#where-the-build-puts-things). The orchestrator is `kdosbuild`, compiled from `src/devtools/kdosbuild/` by `script/kdosbuild.sh` at the start of a build when its sources, flags or compiler changed. On a terminal it first opens a picker that asks whether to start fresh or restore a snapshot, and whether to write snapshots as it goes; see [The startup picker](build-system.md#the-startup-picker). Without a terminal it prints plain lines instead, so a build can be logged to a file. A full build from scratch takes many hours. The finished ISO is `build/iso-build/kdos.iso`. Two things to know before the first run: - **The ISO carries no applications.** The medium ships the application [catalogue](../06-reference/glossary.md), and an application is built by podman on the machine that wants it, from the *Applications* window in the Start menu ([Applications](../02-user-guide/applications.md#the-store)) or with `kdos app install`. An exported set of packs imports offline. - **The build refuses to overwrite an ISO a virtual machine is using.** QEMU reads the image lazily, so rewriting it under a running guest turns every block the guest has not cached into an I/O error. Shut the guest down first, or override with `make build ALLOW_ISO_IN_USE=1`. These variables change how much of the host the build takes, or how much it rebuilds, and nothing about what it produces: | Variable | Effect | |---|---| | `KDOS_JOBS=N` | The job count for every phase: `MAKEFLAGS=-jN`, `CMAKE_BUILD_PARALLEL_LEVEL`, `CARGO_BUILD_JOBS` and `-jN` on every ninja call that names no job count, and a `--cpus` cap of N on the build container, no higher than the host's thread count, so cargo and go follow it. Unset, `script/env/common.env` takes the host's thread count clamped to one job per 2 GiB of memory. Lower it when a large C++ port is OOM-killed | | `KDOS_CPU_SHARES=N` | The build container's CPU weight, default `256`. It is not a cap: an idle host still gives the build every thread, and it only orders the build against other containers and system services | | `KDOS_CCACHE=0` | Turns off the compiler cache, default `1`: CMake ports inside the chroot compile through ccache into `build/ccache`, and a cached object is byte-identical to a compiled one (see [The compiler cache](how-kdos-is-built.md#the-compiler-cache)). Forwarded into the chroot by `script/chroot/exec.sh` | | `KDOS_PKG_STORE=1` | Turns on the package store, default `0`: a port whose recipe, environment and dependencies' exact bytes match a package built before is installed from `build/pkgstore` instead of being built, and every package built is stored. `check` builds every hit anyway and logs any difference to `build/logs/pkgstore-check.log`. Use `0` or `check` for a release. See [The package store](../03-architecture/packaging.md#the-package-store) | | `KDOS_PKG_STORE_MAX=SIZE` | The size `70_image` evicts the store down to, least recently used first, default `60G` | Four opt-in variables change what the build produces. Each is passed by the `Makefile` and forwarded into the chroot by `script/chroot/exec.sh`: | Variable | Effect | |---|---| | `KDOS_ISO_SOURCES=1` | Copies the tree onto the ISO under `/sources`, beside `system.sfs` rather than inside it: `src/`, `script/`, `fs/`, and `ports/` with every recipe and fetched source but without its caches, with a `SOURCES` stamp giving the port count, the size and the build time. The `Makefile` and `Dockerfile` are not mounted into the build, so neither is on the medium. It roughly doubles the size of the image | | `KDOS_PACK_KDOS=1` | Also packs the root filesystem as the base pack `kdos`, in `build/kdos-base/kdos.kpack`, and puts it on the ISO under `/packs`. `kdos-box create ports base=pack:kdos` then gives a running KDOS a clean KDOS to build ports in. Without the flag, a pack left from an earlier build is deleted | | `KDOS_ISO_COMP=zstd:3` | Compresses `system.sfs` with zstd at level 3 instead of the default level 15: a faster `70_image` and an image about 10% larger, for the rig, never a release. `xz` and `zstd:1` to `zstd:22` are accepted; any other value stops `110_iso.sh` | | `KDOS_MAKE_BINHOST=1` | Keeps every package the build makes and writes them into a signed binhost in `build/binhost/`, which `kpkg binhost` can install from. Each run adds what it built, so a complete binhost needs one `--fresh` build with the flag set | For example, `make build KDOS_ISO_SOURCES=1`. ## The phases The orchestrator runs thirteen phases, each a directory under `script/phases/`, in sorted order. A phase can be named by its directory or by its short name, the word after the number: | Directory | Short name | Title | |---|---|---| | `00_cross` | `cross` | Cross Toolchain | | `10_bootstrap` | `bootstrap` | Base Userland | | `20_selfhost` | `selfhost` | Self-Hosting Bootstrap | | `30_foundation` | `foundation` | Build Foundation | | `31_compilers` | `compilers` | Compilers | | `40_lang` | `lang` | Languages | | `41_system` | `system` | System | | `42_graphics` | `graphics` | Graphics Stack | | `43_toolkits` | `toolkits` | Toolkits | | `44_apps` | `apps` | Applications | | `50_desktop` | `desktop` | Desktop | | `60_kernel` | `kernel` | Kernel | | `70_image` | `image` | Image | `00_cross`, `10_bootstrap` and `70_image` are directories of numbered scripts; the other ten install the ports their package list names and the dependencies those need. From `30_foundation` on, each list names exactly the ports its phase installs; `20_selfhost`'s nine names pull in six more. The numbers leave gaps so that a phase can be added between two others without renaming any. What each phase contains, and how one runs, is in [The build system](build-system.md#phases). ## Make targets | Target | Does | Needs | |---|---|---| | `all` | The default target; the same as `build` | | | `build` | The whole build, in the container. Runs `check-iso-free` first | Docker | | `fetch` | Fetch every port's sources into `ports/core`, generating vendor bundles the archive lacks. `ports/fetch ` narrows it | Network; a container only to generate | | `fetch-check` | List, offline, every archived source that is missing or fails its hash | A C compiler | | `updates` | Check every port for a newer upstream release | Network, `curl`, `git`, `cc` | | `publish` | Upload every source a recipe names that the archive lacks, each into its shelf's volume `sources-` (or the lowest-numbered volume with room when that one is full); `PORTS` narrows it | Network, the archive token | | `publish-dry` | Print what `publish` would upload, under which names and into which volumes | A C compiler | | `publish-check` | List every source the archive lacks, by anonymous requests | Network | | `publish-plan` | Give every shelf in `ports/shelves` that has no volume one, packing them in sorted order to about 650 assets a volume; never changes an assigned shelf. `PUBLISH_ARGS=--dry-run` only prints the plan | A C compiler | | `publish-describe` | Rewrite every archive release's title and notes from `ports/sources.idx` | Network, the archive token | | `publish-rehome` | Move each archived file that is not in its shelf's volume under its `--` name there; run it, push the index, and run it again to delete the old copies | Network, the archive token | | `publish-retire` | Delete every `src-` release of the older one-release-per-shelf layout that holds no asset, and its tag; refuse any that still holds assets | Network, the archive token | | `publish-orphans` | List index entries no recipe names any more | | | `freeze` | Attach `sources.sha256` for `TAG` to that system release | Network, the archive token | | `release` | Create or update `TAG`'s system release as a draft: the ISO, `SHA256SUMS`, its signature and `sources.sha256` | Network, the archive token, a finished build | | `release-publish` | Make `TAG`'s release public and Latest | Network, the archive token | | `snapshots` | List the phase snapshots, compiling `build/.kdosbuild` with your machine's compiler when your compiler did not build it | A C compiler | | `run` | Boot the ISO in a virtual machine, with `build/kdos.qcow2` attached as a disk (created if missing as a sparse qcow2 of `KDOS_DISK`, 96 GB by default, above the roughly 53 GB install `kinstall` checks the disk against) | QEMU, OVMF | | `rundisk` | Boot `build/kdos.qcow2` instead of the ISO | QEMU, OVMF, an existing disk image | | `run-hw` | Boot the ISO with hardware-accelerated graphics | Docker, the NVIDIA Container Toolkit | | `rundisk-hw` | The same, from the disk image | Docker, the NVIDIA Container Toolkit | | `debug-boot` | Boot the built kernel and initramfs directly, with the serial console on your terminal, for early-boot debugging | QEMU, a finished build | | `check-iso-free` | Refuse to rewrite an ISO a process has open | `fuser`; without it the check is skipped | | `check-hw` | Check the accelerated-graphics setup: an error without Docker, warnings without the `nvidia` runtime or `/dev/udmabuf` | | | `cleandisk` | Replace `build/kdos.qcow2` with a new, empty, sparse disk image of `KDOS_DISK` (96 GB by default) | QEMU | | `cleanbuild` | Delete everything in `build/` except `snapshots`, `ccache`, `pkgstore` and `keys`, and except what only root can remove (see below) | | | `clean` | Delete everything in `build/` except `keys`, and except what only root can remove | | | `help` | List every target with a one-line description | | | `check` | Run `preflight`, `selftest`, `docscheck` and `phaseclosure` in turn | | | `preflight`, `selftest`, `docscheck`, `phaseclosure` | Run that check alone; see [Working without a build at all](#working-without-a-build-at-all) | A C compiler; Python 3 for `phaseclosure` | | `selftest-asan` | `selftest` compiled with `SELFTEST_CC`, by default AddressSanitizer and UBSan | A compiler with the sanitizers | | `depdrift` | Read `build/fs` for undeclared same-phase link dependencies; advisory | Read access to `build/fs`: a container or root; Python 3 | | `debuginfo` | Fail when `build/fs` ships more than 1 MiB of DWARF outside the allowlist (ghc's `-debug` runtimes); lists it by package | Read access to `build/fs`; Python 3 | | `unowned` | List the files in `build/fs` no package and no `fs/` path provides | Read access to `build/fs`; Python 3 | | `linkcheck`, `entrycheck` | Fail on an undeclared `DT_NEEDED` owner, an interpreter that does not resolve or OpenSSL 3 and 4 in one process; or on a dangling link, a dead desktop or D-Bus `Exec=`, a missing `Icon=` or udev program. `70_image` runs both and stops on a failure | Read access to `build/fs`; Python 3 | | `quick` | The fast loop: rebuild the ports named by `QUICK` and patch them into a booted ISO; `QUICK_ARGS` reaches the rig | The rig image, a finished ISO | | `rig-image`, `devdeps-image` | Build the rig container, or the development container and run the suite in it | Docker, network once | Both cleans run on your machine as your user. `build/fs` is owned by root, so they cannot remove it, and the target ignores the failure: after `make clean` the old root filesystem is still there, and the next build runs on it. Remove it from a container, or as root: ```sh docker run --rm -v "$PWD/build:/b" os-dev rm -rf /b/fs ``` The `os-dev` image exists once `make build` has run. Arguments reach the orchestrator through `BUILD_ARGS` and the version checker through `PORTUP_ARGS`: ```sh make build BUILD_ARGS="--continue-from 41_system" make updates PORTUP_ARGS="--check curl" ``` The archive targets take `PORTS` (a space-separated port list), `PUBLISH_ARGS` (passed to `ports/publish` unchanged) and `TAG` (the system tag `freeze` and the release targets act on): ```sh make publish PORTS="zstd lz4" make release TAG=v0.2 PUBLISH_ARGS=--dry-run ``` The orchestrator options used most often are these; the full list is in [The build system](build-system.md#flags): | Option | Effect | |---|---| | `--fresh` | Skip the startup picker and run every phase on the existing tree | | `--restore PHASE` | Restore a snapshot and continue after it (`PHASE` is a name, a directory, a 1-based index or `latest`) | | `--continue-from PHASE` | Resume at `PHASE` on the existing tree, restoring nothing | | `--phases LIST` | Run only these phases | | `--steps LIST` | Run only these scripts, each written `PHASE:script.sh` | | `--rebuild LIST` | Rebuild these ports even though they are installed | | `--no-snapshot` | Write no snapshots during this build | | `--full-snapshots` | Write every snapshot as a whole tree, never as a layer on the one before | | `--port-jobs N` | Build up to `N` ports of a package phase at once, by dependency level; 1, the default, builds one at a time | | `--plain`, `--json` | No interface: plain lines, or one JSON object per event | **Building ports side by side.** `make build BUILD_ARGS="--port-jobs 3"` runs each package phase level by level: the ports whose dependencies are all installed build together, each with `KDOS_JOBS` divided between them and on its own window of CPUs, and one commit step then installs the level in the serial order. What each port builds against is decided by the tree, not by timing, so the result matches a serial build as long as every recipe declares its dependencies. It pays where a phase has many ports too small to use every CPU, the configure, install and packaging time of a userland phase above all; a phase ruled by one large port gains little. 2 or 3 is the useful range: each running port has its own compilers and linkers, and a new port starts beside running ones only while a quarter of the memory is free, so on a machine with little memory to spare lower `KDOS_JOBS` as well or leave it at 1. How a level is built, and what a failure in one does, is in [Building a package phase by level](build-system.md#building-a-package-phase-by-level). `ports/update --check` exits 1 when it finds an update, so `make updates` treats status 1 as success and fails only on status 2 or higher. ## Running the result ```sh make run # plain graphics: the desktop works, the phosphor pass does not make run-hw # accelerated: the phosphor pass is on ``` The *phosphor pass* is the compositor's CRT-style post-processing shader. `make run` uses a plain virtual display (`virtio-vga` with QEMU's GTK window), on which the compositor falls back to software rendering, and the phosphor pass turns itself off there because a full-screen shader in software is far too slow. `make run-hw` runs a containerised QEMU 10 with GPU-accelerated graphics (virgl) through `testing/qemu-hw/run.sh`, which is the configuration where the shader is visible. `make run` and `make rundisk` give the guest 4 GB of memory and as many CPUs as your machine has, put the serial console on your terminal, and use QEMU's user-mode networking. `make run` also attaches a USB tablet, so the pointer tracks the window without being grabbed. Every run target asks `testing/qemu-audio.sh` for a sound card: it probes which audio backend the local QEMU can use and adds an Intel HDA controller when there is one, or nothing when the host cannot play sound. All run targets come up at 1920x1080. `KDOS_RES` changes that: ```sh make run KDOS_RES=2560x1440 ``` KDOS never asks for a display mode; the desktop takes the one the virtual screen says it prefers. For QEMU's virtio GPU that is whatever its `xres` and `yres` properties say, and left unset they default to 1280x800, which is too small to lay out the Start menu's three columns. `KDOS_RES` sets those two properties, the `Makefile` passes it to `testing/qemu-hw/run.sh`, and `testing/vnc-shot.py` sets the same properties from its own `--size` (whose default is 1280x800). The accelerated run scales the picture to fit its window instead of resizing the guest (`zoom-to-fit=on`). Otherwise QEMU's window would report its own size to the guest, and the desktop would follow whatever size the window happened to open at. `KDOS_QEMU_DISPLAY` overrides the display option: `KDOS_QEMU_DISPLAY=gtk,gl=es` makes the guest follow the window, and `KDOS_QEMU_DISPLAY=egl-headless` runs with no window at all. `make debug-boot` boots `build/fs/boot/vmlinuz-kdos` and `build/fs/boot/initramfs.cpio.gz` directly, with the ISO attached and the kernel's console on both the screen and the serial port (`-serial stdio`), so the serial console prints in your terminal. The kernel command line carries `quiet loglevel=3`, the same as the default Limine entry, so only kernel messages at error level or above appear there. For every kernel message, boot the `KDOS Live (verbose)` entry from the Limine menu under `make run`, or edit the `-append` line of the `debug-boot` target. The target uses no KVM and no firmware. ## Where the build puts things | Path | Holds | Notes | |---|---|---| | `build/fs` | The target root filesystem | Owned by root on purpose; see below | | `build/iso-build/kdos.iso` | The ISO | | | `build/logs//` | One log per step | The first thing to read when a build fails | | `build/logs/chroot.log` | Warnings from `script/chroot/exec.sh`: a cgroup tree it could not bind read-only, an open-files limit it could not raise | | | `build/snapshots//` | Phase snapshots | Kept by `cleanbuild` | | `build/snapshots/.held/@/` | Snapshots replaced or deleted while later snapshots still layer on them | Deleted by the orchestrator once nothing needs them | | `build/.snap-lineage/` | One index per snapshot path, describing the tree as the last snapshot or restore left it; the next snapshot is a layer of the difference | Removed by `cleanbuild`, which makes the next snapshot full | | `build/ccache/` | The compiler cache CMake ports compile through, up to 20 GB | Kept by `cleanbuild`, removed by `clean`. Written as root; remove it from a container | | `build/pkgstore/` | The package store, `//` holding one package and its `META`, up to `KDOS_PKG_STORE_MAX` | Written only with `KDOS_PKG_STORE` on. Kept by `cleanbuild`, removed by `clean` | | `build/logs/pkgstore-check.log` | What `KDOS_PKG_STORE=check` found: one line per port whose rebuild differed from its stored package, then the members that differ | See [Testing](testing.md#reading-pkgstore-checklog) | | `build/mark/` | The early phases' "already done" markers | See [Building from scratch](#building-from-scratch) | | `build/cross/` | The cross toolchain (binutils and GCC for `x86_64-kdos-linux-musl`) that the first two phases compile with | | | `build/tmp/` | Scratch space for the early phases | Emptied at the start of every step of those phases | | `build/keys/` | Signing keys: `kdos-release.key` and `kdos-release.pub`, which `ports/publish --release` signs a release's `SHA256SUMS` with, and `linux-module-signing.pem`, which signs every kernel module | Kept by both cleans. `kpkg keygen build/keys/kdos-release` writes the release key, once; the `linux` port writes the module-signing key the first time it builds without one | | `build/kdos.qcow2` | The virtual machine's disk | Created by `make run` | | `build/.kdosbuild` | The compiled orchestrator, with `build/.kdosbuild.sum` beside it | Recompiled at the start of a build when its sources, flags or compiler changed | | `build/.devplan.json` | The build plan in force | | | `build/kdos-base/` | The base pack, with `KDOS_PACK_KDOS=1` | | | `build/image-ports/` | The recipe-only ports tree `70_image/105_ports.sh` stages, which `110_iso.sh` appends to `system.sfs` as `/ports` | Rewritten by every image build | | `build/fetch-home` | The fetch container's home and toolchain caches | Written as your user | | `build/freeze/` | `sources-.sha256`, written by `ports/publish --freeze` | | | `build/release//` | What `ports/publish --release` uploads besides the ISO: `SHA256SUMS`, `SHA256SUMS.sig`, `kdos-release.pub`, `sources.sha256` and the notes | The ISO's parts are cut here and deleted when the run ends | `build/` is ignored by git in its entirety. Some tools keep compiled helpers and data under `ports/`, all ignored by git: | Path | Holds | |---|---| | `ports/.srccache/` | The source cache. Plain data; survives both cleans | | `ports/.kpkgbin/` | The recipe reader `ports/fetch` and `ports/publish` compile, recompiled whenever its sources are newer than the binary | | `ports/.portup` | The version checker `ports/update` compiles, recompiled when a `.c` or `.h` file under `src/devtools/kdos-portup/` or the three libraries it links (`libkbase`, `libkpkg`, `libkbuild`) is newer than the binary | | `ports/.portup-tools/` | The version checker's own copy of the recipe reader, compiled when missing or when it cannot run; a change to its sources does not trigger it | | `ports/.update-cache.json` | The version checker's results, kept for 24 hours | Those helpers are compiled against whichever C library ran them last: musl in an Alpine container, usually glibc on your machine. A binary built against one cannot run under the other, and the failure does not say so: `ports/fetch` and `ports/publish` report every recipe as unreadable (`cannot read its kpkgbuild`, `kpkg cannot read its recipe`), and `ports/update` fails to start its checker with the shell's `No such file or directory`. The version checker's own recipe reader under `ports/.portup-tools` is tested before each use and rebuilt when it exits with status 127, so it recovers from a C-library switch by itself; the other two do not. `make build` mounts `ports/` read-only and `make fetch`'s container keeps its own recipe reader under `build/fetch-home`, so neither touches them. Running `ports/update`, `ports/fetch` or `ports/publish` yourself inside a container that mounts the repository read-write does, for example any Alpine-based development container. Delete `ports/.portup`, `ports/.portup-tools` and `ports/.kpkgbin` afterwards and let the next run recompile them. A container that runs as root leaves those paths owned by root, so remove them from a container rather than with `sudo`. The source cache, `ports/.srccache`, is plain data and does not need clearing. `build/fs` is deliberately not handed back to your user. Changing ownership across that tree would clear every setuid bit (the password checker, the resource helper, and the two user-namespace mapping helpers without which no container can start), and would leave the account and privilege files owned by an ordinary user. Reading it from your machine needs a container or root. ## Where sources come from Every source file that git does not carry is stored in the repository `kunaldawn/kdos` as a release asset. The archive is a run of numbered volumes, releases tagged `sources-1`, `sources-2` and on, each titled `Sources N: … ` after the shelves it holds. The second field of a shelf's line in `ports/shelves` is its volume, and a volume holds the files of its shelves as `--`; a file only old history names is `attic--` in the highest volume. These releases are pre-releases and never the latest release, so the repository's latest release is always a KDOS system release. The committed file `ports/sources.idx` says which release and which asset hold each hash, one line per file, sorted by hash, after a header whose first line is `# kdos-sources-index 2`. A line looks like this: ```text f7ef3ae8a22e521f289803fe93543eb64c329b58aa73a9e224dfd915a2a5f4f7 sources-3 net-libs--curl-8.22.0.tar.xz curl/curl-8.22.0.tar.xz ``` so that file's address is ```text https://github.com/kunaldawn/kdos/releases/download/sources-3/net-libs--curl-8.22.0.tar.xz ``` The second field is the volume, which is the shelf's own unless that volume was full when the file was stored; the file then went to the lowest-numbered volume with room, or opened the next one. A line whose tag is `src-` names a file still in the older layout of one release per shelf; it fetches the same way until `ports/publish --rehome` moves it. The third field is the name GitHub stored, which is what the address uses; the fourth is the file's own name. The asset name is `--`, and differs further when GitHub rewrote a character outside `[A-Za-z0-9._-]` (`libsigc++` is stored as `libsigc..`), or when that name was already taken in the volume and the file was stored as `----` or `------`. A file larger than 1900 MiB is stored in parts, `.part01` … `.partNN`, each under GitHub's 2 GiB limit per asset, and its line ends in `parts=:

,…,`, the hash of each part in order. A name is only an address. The recipe hash and the digest GitHub computes for the asset are the same string, so a file that verifies is the file the recipe meant, whichever route it came by. The archive is append-only by default: an asset whose digest matches its index line is never replaced or deleted, so an old checkout can still find the exact bytes it was written against after upstream has moved on or gone. The two deliberate exceptions are `ports/publish --rehome`, which moves a file to the volume and name of the shelf its port now sits on and deletes the old copy only after an index naming the new one is pushed, and `ports/publish --orphans --prune=yes-delete`, which deletes files no current recipe and no recipe at any `v*` tag names. Each volume's notes on GitHub hold a section per shelf, its description and then every file of it the volume holds in a table, with links, so a file can also be found by browsing the Releases page. `ports/srclib.sh` holds this addressing, and `ports/fetch`, `ports/publish` and the pre-push hook all read it from there, so the three cannot disagree about where a hash lives. An index whose first line is not the format-2 header is not read at all: `ports/fetch` warns once and goes upstream for everything. `ports/fetch` resolves each file a `sha256 =` line names, other than files git tracks itself (such as patches), by trying these locations in order and stopping at the first copy that verifies: | | Location | |---|---| | 1 | The port directory: a file already there | | 2 | The cache, `ports/.srccache/sha256-XX/` (`XX` is the hash's first two hex digits) | | 3 | The archive, `$KDOS_SOURCES_BASE//` from the hash's line in `ports/sources.idx`; for a file in parts, every part, each checked against its own hash and then joined and checked as a whole. A hash the index does not name skips this step | | 4 | The recipe's `source =` URL for that file | | 5 | Generation, only for the port's own `-vendor-.tar.xz` | A copy that fails its hash, or is empty, is removed and the next location is tried. If the archive cannot be reached, the fetch falls through to upstream. The archive is asked before upstream because it does not disappear when an upstream host does. **The cache.** Every copy that verifies is entered into the cache, and the port directory holds the same file. Both are hard links when the cache and the checkout share a filesystem, so the cache costs no extra space; otherwise each is a copy, so a cache on another disk holds a second copy of every source. Either way, switching branches downloads nothing already fetched. A cache entry whose bytes do not hash to its name is deleted rather than served. The cache holds each file under its hash, split into `sha256-XX/` directories by the first two hex digits only to keep each directory small. It is ignored by git and survives `make clean`; deleting it costs a full re-download and nothing else. It belongs to one checkout. A second checkout or a fresh clone starts with an empty cache unless `KDOS_SRCCACHE` points both at a shared path, or unless it is fetched with `ports/fetch --tree ` from a checkout that already has the cache. **Unhashed files.** A file named by a recipe with no hash yet (for example, just after bumping `version =`) is downloaded from upstream only, with a warning, and never from the archive or the cache. `KDOS_ALLOW_UNVERIFIED=1` silences the warning. A vendor bundle with no hash is generated. **Leftover archives.** Source files are not tracked by git, so a pull or branch switch that moves a port to another version leaves the old file in the port directory. Nothing the build reads names it; `testing/preflight.sh` lists it as a stale fetched archive, and it is safe to delete. **Interrupted downloads.** A download goes to `.part` and is renamed into place only once it verifies. The next run resumes the partial file. If a resumed download fails its hash, it is fetched once more from the beginning from the same URL before the next location is tried, because the partial file may have held the start of different bytes. **Vendor bundles.** Ports written in Rust, Go, Python and Haskell build offline from a *vendor bundle*: a tarball of their dependencies, generated by the language's own package manager. For Rust, Go and Haskell that toolchain is the version this tree compiles with; Python bundles are resolved by the `python3` and `pip` of the fetch image's Debian release, not by the tree's own Python. A recipe asks for one with `vendoring =` (`rust`, `go`, `node`, `haskell` or `python`) or with `pypackages =`. In the current tree 71 recipes use `rust`, 38 `go`, 45 `python`, 4 `haskell` and 1 `node`. A default `make fetch` runs in two passes. Your machine resolves everything it can from the first four locations with `curl` and `sha256sum`. Only a port whose vendor bundle is still missing is handed to the `kdos-fetch` container, which generates it; a clone whose archive is complete never builds that image. A port missing any other file is not handed over, because its bundle cannot be generated without its source, and fails on your machine. The `kdos-fetch` image is built from `ports/Containerfile.fetch` on Debian trixie, with the Rust, Go, Node, GHC and cabal versions read from the `rust`, `go`, `nodejs`, `ghc` and `cabal-install` recipes. It runs with host networking and as your own user and group, so everything it writes is yours; its home directory and each toolchain's cache live in `build/fetch-home`. A generated bundle is held to its recipe hash like a download. Bundles are built reproducibly (sorted names, a fixed timestamp, owner 0, `xz -9 -T1`), so the same inputs give the same bytes. | Command | Does | |---|---| | `ports/fetch [port…]` | Fetch every port, or only the named ones | | `ports/fetch --check [port…]` | Offline: list each archived file that is missing or fails its hash, and each recipe that cannot be read; exit 1 if any. `make fetch-check` runs it over every port | | `ports/fetch --tree [port…]` | Fetch for the `ports/core` of another checkout ``, using this tree's archive logic, recipe reader, cache and index. Never generates a vendor bundle | A port is always named by its bare name, never by its shelf: `ports/fetch pv` fetches `ports/core/cli/pv`. A named port that does not exist fails the run, in every mode, so a typo is not mistaken for a port whose sources are all in place; `ports/publish ` behaves the same way. A checkout with no `ports/srclib.sh` cannot fetch from the archive by itself, and this tree's `ports/fetch` cannot be copied into it, because it needs `srclib.sh` beside it. Fetch for it from a checkout that has one instead: ```sh ports/fetch --tree ../other-checkout ``` This works because the newest `ports/sources.idx` names every file an older checkout's recipes can ask for, wherever it now sits, except a file pruned as an orphan, which comes from upstream. In this mode, a file the other checkout tracks through a Git LFS filter counts as an archive to fetch, because without LFS its working copy is only a pointer. | Variable | Default | Effect | |---|---|---| | `KDOS_SOURCES_REPO` | `kunaldawn/kdos` | The archive repository | | `KDOS_SOURCES_BASE` | `https://github.com/$KDOS_SOURCES_REPO/releases/download` | The archive's download base. Set it empty to use upstream only | | `KDOS_SOURCES_INDEX` | `ports/sources.idx` | The index to read | | `KDOS_SRCCACHE` | `ports/.srccache` | The cache. A path outside the repository is mounted into the fetch container | | `KDOS_FETCH_HOST` | unset | `1`: one pass on your machine, generating vendor bundles with its own toolchains | | `FETCH_JOBS` | `1` | How many ports `ports/fetch` works on at once, `--check` included; `make fetch` and `make fetch-check` pass it through. Above 1, each port's lines print together when it finishes, and a run that generates vendor bundles still generates them one at a time | | `KDOS_ALLOW_UNVERIFIED` | unset | `1`: no warning for a file its recipe gives no hash | | `ENGINE_OUT` | `docker` if installed, else `podman` | The container engine for generating vendor bundles | | `CC` | `cc` | The compiler for the recipe reader, `ports/.kpkgbin/kpkg` | ### The pre-push check The `pre-push` hook makes two checks, in this order. **The ports layout.** At the tip of each pushed ref that no remote already holds, read from git's own tree rather than your working copy, the hook requires every `kpkgbuild` under `ports/core` to be exactly at `ports/core///kpkgbuild`, with `` listed in that commit's `ports/shelves`; every bare port name to be unique across `ports/core` and the `src/` areas; and no listed shelf to be named `libs` or `core`, to share its name with a port, or to use anything but lowercase letters, digits and `-`. It needs no network. Only the tip is checked, because only the tip is built: an intermediate commit whose breach a later commit repairs passes. A refusal starts `pre-push: the ports layout is broken at :` and lists each breach. `testing/preflight.sh` checks the same and more; the hook holds when preflight was not run. Set `KDOS_SKIP_LAYOUT_CHECK=1` to skip it. **The archive.** The hook then enforces the archive's side of fetching. For each ref pushed, it takes the hashes the pushed commit's recipes name and the remote's current commit does not, and requires each to be listed in `ports/sources.idx` as of the pushed commit and present at the archive address that line gives, every part of a file stored in parts. A file git itself carries, such as a patch, is not checked. The hook also refuses the push when it cannot reach the archive, or the archive answers anything other than 200 or 404, since it then cannot prove the sources are there. Set `KDOS_SKIP_PUBLISH_CHECK=1` for a push you know is safe; it skips this check only, not the layout check. Setting `core.hooksPath` replaces `.git/hooks` entirely, including any hooks git-lfs installed. How a new source reaches the archive is in [Publishing sources](writing-ports.md#publishing-sources). ## Rebuilding one thing A full build takes hours, and almost no change needs one. Find what you changed in this table: | Changed | Run | |---|---| | Something under `fs/` | `make build BUILD_ARGS="--phases 10_bootstrap,70_image --steps 10_bootstrap:000_file_system.sh"` | | One port's recipe | `make build BUILD_ARGS="--phases ,70_image --rebuild "`, for example `--phases 41_system,70_image --rebuild pv` | | A desktop program | `make build BUILD_ARGS="--phases 50_desktop --rebuild "` | | The book under `docs/` | `make build BUILD_ARGS="--phases 41_system,70_image"`; see below | | A library under `src/libs/` | `make build BUILD_ARGS="--phases 10_bootstrap,41_system,42_graphics,44_apps,50_desktop,70_image --steps 10_bootstrap:120_kpkg.sh,10_bootstrap:130_kinstall.sh"`; see below | | The installer (`src/system/kdos-installer`, or `kdos-appbox`'s `catalogue.c`) | `make build BUILD_ARGS="--phases 10_bootstrap,70_image --steps 10_bootstrap:130_kinstall.sh"`; see below | | Only packaging | `make build BUILD_ARGS="--phases 70_image"` | | Only packaging, for the rig | `make build KDOS_ISO_COMP=zstd:3 BUILD_ARGS="--phases 70_image"` | | Nothing; resuming an interrupted run | `make build BUILD_ARGS="--continue-from "` | | Every phase, on the existing tree, skipping the startup picker | `make build BUILD_ARGS="--fresh --no-snapshot"` | Use the phase that installs the port. Add `70_image` whenever you want a new ISO from the change. A new port is built only once a phase list names it. In `40_lang` to `44_apps` the list is a directory, `packages.d/`, whose files are each named for a shelf (`.txt`) or a `src/` area (`src-.txt`) and name only ports filed there; `41_system` also carries `00-order.txt` for the runs whose order a comment pins, the one file in a `packages.d/` that may name a port an earlier phase installs. Every other package phase has a single `packages.txt`. [Which phase lists a port](writing-ports.md#which-phase-lists-a-port) says which phase a new port belongs to. [The ports catalogue](../06-reference/ports-catalogue.md) lists every port by shelf with its phase. To find the phase from the tree: ```sh grep -lx '' script/phases/*/packages.txt script/phases/*/packages.d/*.txt # the lists that name it ls build/logs/*/*_.install.log # the phase whose logs hold its install ``` From `30_foundation` on, every port a phase installs is named in that phase's list, so for almost every port the first command prints one file, and the phase is the directory in its path. 8 ports are named in more than one list: `20_selfhost`'s `musl`, `gcc`, `binutils`, `m4`, `tar`, `diffutils`, `gawk` and `zlib` are named again in `30_foundation`'s pinned run. A later list names a port again so that it is rebuilt there when its recipe has changed, before the ports after it use it. For these the first command prints several files. Pass only one of those phases: `--rebuild` forces the port in every selected phase whose resolved install order contains it, as a listed port or as a dependency, so selecting two such phases compiles it twice. The latest phase that names it is the usual choice, because a port rebuilt under a phase is built with that phase's environment, and the latest one is the environment it was last built with. The environments differ: `20_selfhost` names no compiler, while every later phase sets `CC=gcc` and `CXX=g++`. `PORT_REPO` differs too: `src/system` and `src/art` are on it from `40_lang` on, and `src/desktop` and `src/daemons` only in `50_desktop`. Four ports in `ports/core` are named in no list: `20_selfhost`'s list depends on them (`gmp`, `mpfr`, `mpc` and `xxhash`), so that phase installs them, and the second command finds it. Every other recipe is named by a list, and `testing/bookcounts.py` fails on one that no phase installs. **Editing a library rebuilds every port of KDOS's own**, not only the ones that use it. KDOS's own programs are recipes under `src/system/`, `src/art/`, `src/desktop/` and `src/daemons/`. Such a recipe has no upstream tarball, so its recipe hash covers its own directory and the whole of `src/libs`; working out which libraries a `build.sh` actually compiles would need a shell parser inside the package manager. Each of these ports takes seconds to compile. Upstream ports are unaffected. The phase lists install every one but `kdos-installer`, in `41_system`, `42_graphics`, `44_apps` and `50_desktop`, and their changed hash is enough to rebuild them, so they need no `--rebuild`. `kdos-installer` is in no phase list, so `--rebuild kdos-installer` reaches no phase and rebuilds nothing; the installer on the image comes from the `10_bootstrap` step `130_kinstall.sh` instead, which also compiles in `catalogue.c` from `src/system/kdos-appbox`. An edit to that file rebuilds the `kdos-appbox` port through its directory hash but leaves `kinstall` as it was until the step is named, as in the installer row above. **The book is a port too.** `src/system/kdos-doc` renders `docs/kdos` to HTML under `/usr/share/doc/kdos`, writes `kdos(7)`, which says where the book is and lists its chapters, and writes one manual page for every `kdos-*` command in [the command index](../06-reference/command-index.md), whose DESCRIPTION is the book section that command's row links to. Its directory holds `docs`, a symbolic link to the repository's `docs/`, and the recipe hash follows it, so an edit anywhere in the book changes `kdos-doc`'s hash and the next `41_system` run rebuilds it with no `--rebuild`. The build container mounts `docs/` read-only, and `script/chroot/exec.sh` binds it read-only at `/kdos/docs`. The generator, `mkbook.py`, fails the build when a row of the index links a section the book does not have, and leaves out a command whose own port ships its manual page (`kdos-bb.1`). Three programs link the libraries outside any recipe. The orchestrator is recompiled at the start of every build. The package manager `kpkg` and the installer `kinstall` are compiled by the `10_bootstrap` steps `120_kpkg.sh` and `130_kinstall.sh`, which is why the command above names them. `120_kpkg.sh` recompiles whenever its sources' hash differs from the one it recorded; `130_kinstall.sh` exits at once when its completion marker under `build/mark/` exists, unless the [build plan](build-system.md#build-plans) names it; that section also explains what a plan suppresses and why only the named ports are forced. Leave the `10_bootstrap` part out when your change touches neither program's libraries. For a desktop program there is a faster loop still: `testing/quick.sh` builds only the named ports into `build/fs`, with no packaging, and patches the files they own into the RAM overlay of a booted ISO. It cannot carry anything under `fs/`, a new port, or a program that owns a D-Bus name, and what it shows is not evidence about the shipped image. See [Testing](testing.md#the-fast-loop). ### Building from scratch `--fresh`, and *start fresh* in the picker, run every phase on the tree already in `build/`. They do not empty it: each `00_cross` and `10_bootstrap` script exits at once when its marker under `build/mark/` exists, and the build skips every port already installed from the same recipe. To build from nothing, empty the three trees that record that progress first. `build/fs` belongs to root, so do it from a container: ```sh docker run --rm -v "$PWD/build:/b" os-dev rm -rf /b/fs /b/mark /b/cross make build BUILD_ARGS=--fresh ``` Budget most of a day for the build that follows. If it writes snapshots, budget the disk as well: `00_cross` archives `build/fs` compressed, and each later phase to `50_desktop` a layer of what it changed. A complete set has not been measured; see [Snapshots](build-system.md#snapshots), which also says how a phase opts out of its snapshot. ## Things to avoid while a build runs - **Do not edit a port's sources while it is being rebuilt.** The recipe hash is taken when the build starts, so an edit in the middle is rebuilt on the next run, but the package being installed now was compiled from a mix of the two trees. - **Do not re-run an early phase on a tree that is already past it while snapshots are being written**, as they are by `--fresh`, by `--continue-from` and by a plan run with `--snapshot`. The early phase's snapshot would be overwritten with the later tree, filed under its name. A [build plan](build-system.md#build-plans) that narrows the run with `--phases` or `--steps`, as the commands in [Rebuilding one thing](#rebuilding-one-thing) do, writes no snapshots and is safe; `--rebuild` on its own does not narrow, so it keeps them. To resume an interrupted run, use `--continue-from` with the phase it stopped in, not an earlier one. - **Do not switch between running the `ports/` helpers in a container and on your machine without clearing them.** A helper compiled against the other C library fails without saying why; see [Where the build puts things](#where-the-build-puts-things) for which ones recover by themselves. - **Run long builds where they will survive your shell.** A build started from a shell that exits dies with it; use `tmux`, `screen` or similar. ## Working without a build at all Much of the system can be checked on your development machine, with no container and no virtual machine: ```sh testing/preflight.sh # the wiring: a few minutes testing/selftest.sh # libraries and their consumers: a few minutes make snapshots # compiles build/.kdosbuild for this machine if needed build/.kdosbuild --preview build 132x43 vt # a build screen, drawn as text ``` `testing/preflight.sh` checks the wiring a full build would otherwise fail on: that the ports tree is shelved and every name is one port, that every package phase from `30_foundation` on installs exactly the ports its list names, that every recipe parses, that the phase scripts are valid shell, and similar. `testing/selftest.sh` compiles `kdosbuild`, `kinstall` and the other programs it tests into a temporary directory, which it deletes when it exits, and runs their assertions. Among other things it resolves the installer's plan over the shipped catalogue (`kinstall --dump plan`) and, where the Wayland libraries are installed, draws the resource monitor's pages as text from recorded `/proc` trees (`kdos-res --fixture testing/fixtures/res --dump`). Neither program is installed on a development machine, so those forms are what the self-test runs, not commands to type. On one development machine each script took about two and a half minutes. What each harness proves is in [Testing](testing.md#what-each-tool-proves). `build/.kdosbuild` runs on your machine only when your machine's compiler built it, which `make snapshots` does. `make build` replaces it with a musl binary compiled in the build image, which a glibc machine cannot run; the shell then reports `No such file or directory` without mentioning the missing musl loader. `build/.kdosbuild.sum` names the compiler as well as the sources, so each side recompiles when the other built the binary last. Run `make snapshots` again after a build, or keep a separate copy with `KDOSBUILD_BIN=build/.kdosbuild-host script/kdosbuild.sh --list`. `--preview` draws one of the screens `build`, `activity`, `failure`, `pinned`, `complete`, `startup`, `plan` or `packages` at the given size, in one of the [glyph tiers](../06-reference/glossary.md) `rich`, `vt` or `ascii` (which set of drawing characters the screen may use). The window model needs no display at all. `libkwm` links nothing but the C library, so the self-test replays `testing/fixtures/wm/geometry.txt` against it on any machine in milliseconds. It is the fastest way to answer a question about where a window lands, what a tiled window becomes, or which edge a moving edge stops against. ## When a build fails Work through these in order: 1. The failing step's log, under `build/logs//`. The orchestrator names it; in the panel it opens when a step fails, `O` opens the log and `C` copies its path (see [Keys](build-system.md#keys)). 2. [Build troubleshooting](build-troubleshooting.md), which lists the failures that recur, by symptom. Check it before debugging from scratch. 3. The port's own `config.log`, if a configure script failed. `kpkg` leaves a failed port's unpacked source in place at `build/fs/var/cache/kpkg/work//-/`. `build/fs` is owned by root, so read it from a container or as root. `C compiler cannot create executables` rarely means what it says; the real error is in the log, next to the test program that failed. 4. `testing/preflight.sh`, which catches wiring mistakes (a missing dependency, an unlisted port) before a build that runs for hours finds them. ## Cutting a release A KDOS release is a git tag and the GitHub release of that tag on `kunaldawn/kdos`, which carries the ISO, a list of every source hash the tag's recipes name, the tag's recipe tree signed for `kdos update fetch`, and a signed checksum list over all of them. The source list says what the release needs to build; the hashes themselves are pinned by the tag, whose recipes git cannot change without changing the tag. The release key is made before the build, because the build copies its public half into the image's `/etc/kdos/keys/tree`: that is how an installed machine knows which trees to accept. An image built with no key takes no tree. ```sh kpkg keygen build/keys/kdos-release # once; build/keys survives both cleans git tag && git push origin git switch --detach # build from the tag, not the working tree make fetch && make build make release TAG= PUBLISH_ARGS=--dry-run # the plan: no network, no token make release TAG= # a DRAFT release with every asset make release-publish TAG= # make it public and Latest ``` The three are `ports/publish --release ` with `--dry-run`, with nothing, and with `--publish`. `ports/publish --release `: 1. Requires the tag to exist locally and, by an anonymous `git ls-remote`, on `origin` at the same commit, and exits 1 otherwise: a release names a tree everyone else must be able to check out. 2. Runs the `--freeze` list below for the tag, and stops unless every source it names is archived. 3. Takes the ISO from `--iso `, by default `build/iso-build/kdos.iso`, as the asset `kdos-.iso`. An ISO larger than `$KDOS_PART_SIZE` bytes (1,900 MiB) is cut into `kdos-.iso.part01`, `.part02`, … under `build/release//`, since GitHub refuses an asset of 2 GiB. The parts are deleted when the run ends and cut again on the next. 4. Makes `build/release//kdos-ports.tar.xz`, the tag's recipe tree as an image's `/ports` holds it, from `git archive`: `ports/core`, `ports/shelves`, `ports/sources.idx`, `src/` and `docs/`, and a `TREE` stamp naming the commit and its time, as a reproducible tar compressed with `xz -9 -T1`. It signs the tree with the release key as `kdos-ports.tar.xz.sig` and checks that signature against the public key. 5. Writes `build/release//SHA256SUMS` in `sha256sum` format. It names the whole ISO even when only its parts are uploaded, each part, `sources.sha256`, the tree, its signature and `kdos-release.pub`. 6. Signs it with `kpkg sign`, which writes `SHA256SUMS.sig`. The key is `--key `, else `$KDOS_RELEASE_KEY`, else `build/keys/kdos-release.key`, and its public half is the `.pub` beside it, uploaded as `kdos-release.pub`. The signature is checked against that public key before anything is uploaded. A missing key stops the run; `--unsigned` releases without the signatures and the key, and no machine then takes its tree. 7. Finds the release among all of the repository's releases, drafts included, and creates it as a **draft** named `KDOS ` when there is none. While it is a draft its notes are rewritten on every run: the download, the joining of the parts, the check and the signature check. 8. Uploads the ISO or its parts, `sources.sha256`, `kdos-ports.tar.xz` and its `.sig`, `SHA256SUMS`, `SHA256SUMS.sig` and `kdos-release.pub`, hashing each again immediately before its upload. An asset already there with the same bytes is kept, so an interrupted run is resumed by running it again. One with other bytes stops the run with exit 2; delete it by hand, and only while the release is still a draft. 9. With `--publish`, makes the release public and Latest, then asks GitHub anonymously which release is the latest, and exits 1 with a warning unless the answer is ``. The draft is visible only to the repository's writers, so the assets go out together when you publish it. Once it is Latest, every installed machine's nightly `kdos update check --fetch` takes its tree. The archive's `sources-` volumes are pre-releases created with `make_latest` false, so none of them ever takes Latest from a system release. Someone who downloads the release checks it with: ```sh cat kdos-.iso.part* > kdos-.iso # only when the ISO is in parts sha256sum -c SHA256SUMS --ignore-missing KPKG_KEYRING= kpkg verify-pkg SHA256SUMS ``` A public key downloaded beside the list it signs proves only that the two agree; the key id that `kpkg keygen` and `kpkg sign` print, and the release notes repeat, is what to compare against a copy obtained another way. `ports/publish --freeze `, which `--release` runs as its step 2: 1. Extracts the tag's recipes with `git archive`, so the list reflects the tag and not your working tree. 2. Writes `build/freeze/sources-.sha256` in `sha256sum` format, one ` /` line per source, sorted. 3. Checks every hash against the archive, and exits 1 listing any that are missing. Publish those with `ports/publish` first. 4. On its own, attaches the list as `sources.sha256` to the release `` on `$KDOS_REPO` (default `kunaldawn/kdos`), creating a draft release pointing at the tag when none exists. It never replaces an existing `sources.sha256` with different contents; it exits 2 instead. With `--check` it stops after step 3 and says what it would attach; with `--dry-run`, as with `--release --dry-run`, it stops after step 2, since it uses no network. Uploading needs a token in `$KDOS_SOURCES_TOKEN` or `~/.config/kdos/sources-token`, which is refused when anyone but you can read it; see [Publishing sources](writing-ports.md#publishing-sources). Leave GitHub's immutable releases **off** on `kunaldawn/kdos`: the source archive's `sources-` volumes live in the same repository, the setting applies to all of them, and it would freeze each at its first publication, after which no new source could be added to it. ## See also - [How KDOS is built](how-kdos-is-built.md): the whole build as one story, from clone to ISO - [How KDOS differs](../01-philosophy/how-kdos-differs.md): why the sources are pinned and the build is self-hosting, compared with other distributions - [The ports catalogue](../06-reference/ports-catalogue.md): every port, by shelf and phase - [The build system](build-system.md): phases, snapshots, build plans, the chroot - [Writing ports](writing-ports.md): adding or changing a recipe, and publishing its sources - [Build troubleshooting](build-troubleshooting.md): recurring failures, by symptom - [Testing](testing.md): what to run before believing a change - [The C libraries](c-libraries.md): what KDOS programs are built on - [Getting started](../02-user-guide/getting-started.md): the same build, from a user's side --- *Part V — Building and developing, chapter 31.* Previous: [30. How KDOS is built](how-kdos-is-built.md) · [Contents](../README.md) · Next: [32. The build system](build-system.md)