initramfs+modules: robust copy aliasing, curated stage1 + PHYs, firmware policy via firmware.conf, runtime readiness, build ID; docs sync
Some checks failed
Build Zero OS Initramfs / build (push) Has been cancelled
Build Zero OS Initramfs / test-matrix (qemu, basic) (push) Has been cancelled
Build Zero OS Initramfs / test-matrix (qemu, serial) (push) Has been cancelled

Summary of changes (with references):\n\nModules + PHY coverage\n- Curated and normalized stage1 list in [config.modules.conf](config/modules.conf:1):\n  - Boot-critical storage, core virtio, common NICs (Intel/Realtek/Broadcom), overlay/fuse, USB HCD/HID.\n  - Added PHY drivers required by NIC MACs:\n    * realtek (for r8169, etc.)\n    * broadcom families: broadcom, bcm7xxx, bcm87xx, bcm_phy_lib, bcm_phy_ptp\n- Robust underscore↔hyphen aliasing during copy so e.g. xhci_pci → xhci-pci.ko, hid_generic → hid-generic.ko:\n  - [bash.initramfs_copy_resolved_modules()](scripts/lib/initramfs.sh:990)\n\nFirmware policy and coverage\n- Firmware selection now authoritative via [config/firmware.conf](config/firmware.conf:1); ignore modules.conf firmware hints:\n  - [bash.initramfs_setup_modules()](scripts/lib/initramfs.sh:229)\n  - Count from firmware.conf for reporting; remove stale required-firmware.list.\n- Expanded NIC firmware set (bnx2, bnx2x, tigon, intel, realtek, rtl_nic, qlogic, e100) in [config.firmware.conf](config/firmware.conf:1).\n- Installer enforces firmware.conf source-of-truth in [bash.alpine_install_firmware()](scripts/lib/alpine.sh:392).\n\nEarly input & build freshness\n- Write a runtime build stamp to /etc/zero-os-build-id for embedded initramfs verification:\n  - [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:568)\n- Minor init refinements in [config.init](config/init:1) (ensures /home, consistent depmod path).\n\nRebuild helper improvements\n- [scripts/rebuild-after-zinit.sh](scripts/rebuild-after-zinit.sh:1):\n  - Added --verify-only; container-aware execution; selective marker clears only.\n  - Prints stage status before/after; avoids --rebuild-from; resolves full kernel version for diagnostics.\n\nRemote flist readiness + zinit\n- Init scripts now probe BASE_URL readiness and accept FLISTS_BASE_URL/FLIST_BASE_URL; firmware target is /lib/firmware:\n  - [sh.firmware.sh](config/zinit/init/firmware.sh:1)\n  - [sh.modules.sh](config/zinit/init/modules.sh:1)\n\nContainer, docs, and utilities\n- Stream container build logs by calling runtime build directly in [bash.docker_build_container()](scripts/lib/docker.sh:56).\n- Docs updated to reflect firmware policy, runtime readiness, rebuild helper, early input, and GRUB USB:\n  - [docs.NOTES.md](docs/NOTES.md)\n  - [docs.PROMPT.md](docs/PROMPT.md)\n  - [docs.review-rfs-integration.md](docs/review-rfs-integration.md)\n- Added GRUB USB creator (referenced in docs): [scripts/make-grub-usb.sh](scripts/make-grub-usb.sh)\n\nCleanup\n- Removed legacy/duplicated config trees under configs/ and config/zinit.old/.\n- Minor newline and ignore fixes: [.gitignore](.gitignore:1)\n\nNet effect\n- Runtime now has correct USB HCDs/HID-generic and NIC+PHY coverage (Realtek/Broadcom), with matching firmware installed in initramfs.\n- Rebuild workflow is minimal and host/container-aware; docs are aligned with implemented behavior.\n
This commit is contained in:
2025-09-23 14:03:01 +02:00
parent 2fba2bd4cd
commit ad0a06e267
87 changed files with 833 additions and 17307 deletions

33
AGENTS.md Normal file
View File

@@ -0,0 +1,33 @@
# AGENTS.md
This file provides guidance to agents when working with code in this repository.
Non-obvious build/run facts (read from scripts):
- Always build inside the containerized toolchain; ./scripts/build.sh will spawn a transient container, or use the persistent dev container via ./scripts/dev-container.sh start then ./scripts/dev-container.sh build.
- The incremental pipeline is stage-driven with .build-stages markers; list status via ./scripts/build.sh --show-stages and force a subset with --rebuild-from=<stage>.
- Outputs are anchored to PROJECT_ROOT (normalized in [bash.common.sh](scripts/lib/common.sh:236)):
- Kernel: dist/vmlinuz.efi (created by [bash.kernel_build_with_initramfs()](scripts/lib/kernel.sh:174))
- Initramfs archive: dist/initramfs.cpio.xz (created by [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:688))
- Kernel embeds the initramfs (CONFIG_INITRAMFS_SOURCE set by [bash.kernel_modify_config_for_initramfs()](scripts/lib/kernel.sh:130)); no separate initrd is required in normal flow.
Fast iteration helpers (init/zinit/modules.conf):
- Use ./scripts/rebuild-after-zinit.sh to re-copy /init, re-apply zinit, re-resolve/copy modules, and recreate the cpio (initramfs-only by default).
- Flags: --with-kernel (also re-embed kernel; forces --rebuild-from=initramfs_create), --refresh-container-mods (rebuild container /lib/modules if missing), --verify-only (report changes since last cpio).
- DEBUG=1 shows full safe_execute logs and stage timings.
Critical conventions (avoid breakage):
- Use logging/safety helpers from [bash.common.sh](scripts/lib/common.sh:1): log_info/warn/error/debug, safe_execute, section_header.
- Paths must be anchored to PROJECT_ROOT (already normalized after sourcing config) to avoid CWD drift (kernel builds cd into kernel/current).
- Do not edit /etc/shadow directly; passwordless root is applied by chroot ${initramfs_dir} passwd -d root in [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:575).
- initdebug=true on kernel cmdline opens an early shell from [config/init](config/init) even without /init-debug.
Modules and firmware (non-obvious flow):
- Container /lib/modules/<FULL_VERSION> is the authoritative source for dependency resolution and copying into initramfs (ensure kernel_modules stage ran at least once).
- RFS firmware pack now installs all linux-firmware* into container and packs from /lib/firmware; runtime overmount targets /lib/firmware (not /usr/lib/firmware).
Troubleshooting gotchas:
- If “Initramfs directory not found: initramfs” or kernel output in wrong place, path anchoring patches exist in [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:688), [bash.initramfs_validate()](scripts/lib/initramfs.sh:799), and [bash.kernel_build_with_initramfs()](scripts/lib/kernel.sh:174); ensure DEBUG logs show “... anchored to PROJECT_ROOT”.
- On host, the rebuild helper delegates to ./scripts/dev-container.sh so you dont need to manually start the container.
Utilities:
- Create BIOS+UEFI USB with embedded-initramfs kernel: sudo ./scripts/make-grub-usb.sh /dev/sdX --kparams "console=ttyS0 initdebug=true" (use --with-initrd only if you want a separate initrd on ESP).

View File

@@ -2,12 +2,15 @@
# Alpine Linux provides separate firmware packages for hardware support # Alpine Linux provides separate firmware packages for hardware support
# Format: FIRMWARE_PACKAGE:DESCRIPTION # Format: FIRMWARE_PACKAGE:DESCRIPTION
# Essential network firmware packages # Essential network firmware packages (wired NICs matching stage1 drivers)
linux-firmware-bnx2:Broadcom NetXtreme firmware linux-firmware-bnx2:Broadcom NetXtreme (bnx2) firmware
linux-firmware-bnx2x:Broadcom NetXtreme II (bnx2x) firmware
linux-firmware-tigon:Broadcom tg3 (Tigon) firmware
linux-firmware-e100:Intel PRO/100 firmware linux-firmware-e100:Intel PRO/100 firmware
linux-firmware-intel:Intel network and WiFi firmware (includes e1000e, igb, ixgbe, i40e, ice) linux-firmware-intel:Intel NIC firmware (covers e1000e, igb, ixgbe, i40e, ice)
linux-firmware-realtek:Realtek network firmware (r8169, etc.) linux-firmware-rtl_nic:Realtek NIC firmware (r8169, etc.)
linux-firmware-qlogic:QLogic network firmware linux-firmware-realtek:Realtek NIC firmware (meta)
linux-firmware-qlogic:QLogic NIC firmware
# Storage firmware (if needed) # Storage firmware (if needed)
# linux-firmware-marvell:Marvell storage/network firmware (not available in Alpine 3.22) # linux-firmware-marvell:Marvell storage/network firmware (not available in Alpine 3.22)

View File

@@ -1,4 +1,4 @@
#!/bin/sh -x #!/bin/sh
# Alpine-based Zero-OS Init Script # Alpine-based Zero-OS Init Script
# Maintains identical flow to original busybox version # Maintains identical flow to original busybox version
@@ -24,11 +24,8 @@ mount -t devtmpfs devtmpfs /dev
mkdir -p /dev/pts mkdir -p /dev/pts
mount -t devpts devpts /dev/pts mount -t devpts devpts /dev/pts
# Early debug shell if kernel parameter initdebug=true is present # Re-initialize modules dependencies for basic init
if grep -qw "initdebug=true" /proc/cmdline; then depmod -a
echo "[+] initdebug=true detected; starting emergency shell. Type 'exit' to continue boot."
/bin/sh -l
fi
echo "[+] building ram filesystem" echo "[+] building ram filesystem"
@@ -52,6 +49,8 @@ echo " copying /var..."
cp -ar /var $target cp -ar /var $target
echo " copying /run..." echo " copying /run..."
cp -ar /run $target cp -ar /run $target
echo " creating /home"
mkdir $target/home
# Create essential directories # Create essential directories
mkdir -p $target/dev mkdir -p $target/dev
@@ -77,45 +76,41 @@ if [ -x /sbin/udevd ]; then
echo " starting udevd..." echo " starting udevd..."
udevd --daemon udevd --daemon
# Preload keyboard input modules early so console works before zinit and rfs mounts
echo "[+] preloading keyboard input modules"
for m in i8042 atkbd usbhid hid hid_generic evdev xhci_pci xhci_hcd ehci_pci ehci_hcd ohci_pci ohci_hcd uhci_hcd; do
modprobe "$m"
done
echo "[+] loading essential drivers"
# Load core drivers for storage and network
/bin/busybox sh -l
modprobe btrfs 2>/dev/null || true
modprobe fuse 2>/dev/null || true
modprobe overlay 2>/dev/null || true
# Load storage drivers
modprobe ahci 2>/dev/null || true
modprobe nvme 2>/dev/null || true
modprobe virtio_blk 2>/dev/null || true
modprobe virtio_scsi 2>/dev/null || true
modprobe virtio_pci 2>/dev/null || true
# Load network drivers
modprobe virtio_net 2>/dev/null || true
echo " triggering device discovery..." echo " triggering device discovery..."
udevadm trigger --action=add --type=subsystems udevadm trigger --action=add --type=subsystems
udevadm trigger --action=add --type=devices udevadm trigger --action=add --type=devices
udevadm trigger --action=add
udevadm settle udevadm settle
echo " stopping udevd..." echo " stopping udevd..."
kill $(pidof udevd) || true kill $(pidof udevd) || true
else else
echo " warning: udevd not found, skipping hardware detection" echo " warning: udevd not found, skipping hardware detection"
fi fi
echo "[+] loading essential drivers"
# Load core drivers for storage and network
modprobe btrfs 2>/dev/null || true
modprobe fuse 2>/dev/null || true
modprobe overlay 2>/dev/null || true
# Load storage drivers
modprobe ahci 2>/dev/null || true
modprobe nvme 2>/dev/null || true
modprobe virtio_blk 2>/dev/null || true
modprobe virtio_scsi 2>/dev/null || true
modprobe virtio_pci 2>/dev/null || true
# Load network drivers
modprobe virtio_net 2>/dev/null || true
modprobe e1000 2>/dev/null || true
modprobe e1000e 2>/dev/null || true
echo "[+] loading USB keyboard support"
modprobe usbcore
modprobe ehci-hcd
modprobe xhci-hcd
modprobe usbhid
modprobe evdev
# Unmount init filesystems
umount /proc 2>/dev/null || true
umount /sys 2>/dev/null || true
echo "[+] debug hook: initdebug=true or /init-debug" echo "[+] debug hook: initdebug=true or /init-debug"
if grep -qw "initdebug=true" /proc/cmdline; then if grep -qw "initdebug=true" /proc/cmdline; then
@@ -124,16 +119,23 @@ if grep -qw "initdebug=true" /proc/cmdline; then
sh /init-debug sh /init-debug
else else
echo " initdebug=true: starting interactive shell (no /init-debug). Type 'exit' to continue." echo " initdebug=true: starting interactive shell (no /init-debug). Type 'exit' to continue."
/bin/sh -l debug="-d"
/bin/busybox sh
fi fi
elif [ -x /init-debug ]; then elif [ -x /init-debug ]; then
echo " executing /init-debug ..." echo " executing /init-debug ..."
sh /init-debug sh /init-debug
fi fi
echo "[+] switching root" # Unmount init filesystems
mkdir /root/home umount /proc 2>/dev/null || true
echo " exec switch_root /mnt/root /sbin/zinit init" umount /sys 2>/dev/null || true
exec switch_root /mnt/root /sbin/zinit -d init
echo "[+] switching root"
echo " exec switch_root /mnt/root /sbin/zinit init"
exec switch_root /mnt/root /sbin/zinit ${debug} init
# switch_root failed, drop into shell
/bin/busybox sh
## ##

View File

@@ -1,6 +1,6 @@
# #
# Automatically generated file; DO NOT EDIT. # Automatically generated file; DO NOT EDIT.
# Linux/x86 6.12.42 Kernel Configuration # Linux/x86 6.12.44 Kernel Configuration
# #
CONFIG_CC_VERSION_TEXT="gcc (Alpine 14.2.0) 14.2.0" CONFIG_CC_VERSION_TEXT="gcc (Alpine 14.2.0) 14.2.0"
CONFIG_CC_IS_GCC=y CONFIG_CC_IS_GCC=y
@@ -11,8 +11,8 @@ CONFIG_AS_VERSION=24400
CONFIG_LD_IS_BFD=y CONFIG_LD_IS_BFD=y
CONFIG_LD_VERSION=24400 CONFIG_LD_VERSION=24400
CONFIG_LLD_VERSION=0 CONFIG_LLD_VERSION=0
CONFIG_RUSTC_VERSION=108900 CONFIG_RUSTC_VERSION=109000
CONFIG_RUSTC_LLVM_VERSION=200107 CONFIG_RUSTC_LLVM_VERSION=200108
CONFIG_CC_CAN_LINK=y CONFIG_CC_CAN_LINK=y
CONFIG_CC_CAN_LINK_STATIC=y CONFIG_CC_CAN_LINK_STATIC=y
CONFIG_CC_HAS_ASM_GOTO_OUTPUT=y CONFIG_CC_HAS_ASM_GOTO_OUTPUT=y
@@ -22,7 +22,7 @@ CONFIG_CC_HAS_ASM_INLINE=y
CONFIG_CC_HAS_NO_PROFILE_FN_ATTR=y CONFIG_CC_HAS_NO_PROFILE_FN_ATTR=y
CONFIG_LD_CAN_USE_KEEP_IN_OVERLAY=y CONFIG_LD_CAN_USE_KEEP_IN_OVERLAY=y
CONFIG_RUSTC_HAS_UNNECESSARY_TRANSMUTES=y CONFIG_RUSTC_HAS_UNNECESSARY_TRANSMUTES=y
CONFIG_PAHOLE_VERSION=0 CONFIG_PAHOLE_VERSION=130
CONFIG_IRQ_WORK=y CONFIG_IRQ_WORK=y
CONFIG_BUILDTIME_TABLE_SORT=y CONFIG_BUILDTIME_TABLE_SORT=y
CONFIG_THREAD_INFO_IN_TASK=y CONFIG_THREAD_INFO_IN_TASK=y
@@ -235,7 +235,7 @@ CONFIG_CHECKPOINT_RESTORE=y
CONFIG_SCHED_AUTOGROUP=y CONFIG_SCHED_AUTOGROUP=y
CONFIG_RELAY=y CONFIG_RELAY=y
CONFIG_BLK_DEV_INITRD=y CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE="../../root" CONFIG_INITRAMFS_SOURCE="/workspace/dist/initramfs.cpio.xz"
CONFIG_INITRAMFS_ROOT_UID=0 CONFIG_INITRAMFS_ROOT_UID=0
CONFIG_INITRAMFS_ROOT_GID=0 CONFIG_INITRAMFS_ROOT_GID=0
# CONFIG_RD_GZIP is not set # CONFIG_RD_GZIP is not set
@@ -2142,7 +2142,9 @@ CONFIG_FW_LOADER_SYSFS=y
CONFIG_EXTRA_FIRMWARE="" CONFIG_EXTRA_FIRMWARE=""
CONFIG_FW_LOADER_USER_HELPER=y CONFIG_FW_LOADER_USER_HELPER=y
# CONFIG_FW_LOADER_USER_HELPER_FALLBACK is not set # CONFIG_FW_LOADER_USER_HELPER_FALLBACK is not set
# CONFIG_FW_LOADER_COMPRESS is not set CONFIG_FW_LOADER_COMPRESS=y
CONFIG_FW_LOADER_COMPRESS_XZ=y
CONFIG_FW_LOADER_COMPRESS_ZSTD=y
CONFIG_FW_CACHE=y CONFIG_FW_CACHE=y
# CONFIG_FW_UPLOAD is not set # CONFIG_FW_UPLOAD is not set
# end of Firmware loader # end of Firmware loader

View File

@@ -1,57 +1,74 @@
# Module loading specification for Zero-OS Alpine initramfs # Module loading specification for Zero-OS Alpine initramfs
# Format: STAGE:MODULE_NAME:FIRMWARE_PACKAGE (optional) # Format: STAGE:MODULE
# Focus on most common NIC modules to ensure networking works on most hardware # Firmware selection is authoritative in config/firmware.conf; do not add firmware hints here.
# Stage 1: Core subsystems + networking and essential boot modules # Stage 1: boot-critical storage, core virtio, networking, overlay, and input (USB/HID/keyboard)
# All core subsystems and NICs must be loaded BEFORE network can come up
stage1:virtio:none # Core virtio subsystem (REQUIRED)
stage1:virtio_ring:none # Virtio ring buffer (REQUIRED)
stage1:virtio_pci:none # Virtio PCI bus
stage1:virtio_net:none # Virtio network (VMs, cloud)
stage1:virtio_scsi:none # Virtio SCSI (VMs, cloud)
stage1:virtio_blk:none # Virtio block (VMs, cloud)
stage1:e1000:linux-firmware-intel # Intel E1000 (very common)
stage1:e1000e:linux-firmware-intel # Intel E1000E (very common)
stage1:r8169:linux-firmware-realtek # Realtek (most common desktop/server)
stage1:realtek:linux-firmware-realtek # Realtek (most common desktop/server)
stage1:igb:linux-firmware-intel # Intel Gigabit (servers)
stage1:ixgbe:linux-firmware-intel # Intel 10GbE (servers)
stage1:i40e:linux-firmware-intel # Intel 40GbE (modern servers)
stage1:ice:linux-firmware-intel # Intel E800 series (latest)
stage1:8139too:none # Realtek 8139 (legacy)
stage1:8139cp:none # Realtek 8139C+ (legacy)
stage1:bnx2:linux-firmware-bnx2 # Broadcom NetXtreme
stage1:bnx2x:linux-firmware-bnx2 # Broadcom NetXtreme II
stage1:tg3:none # Broadcom Tigon3
stage1:b44:none # Broadcom 44xx
stage1:atl1:none # Atheros L1
stage1:atl1e:none # Atheros L1E
stage1:atl1c:none # Atheros L1C
stage1:alx:none # Atheros Alx
stage1:libata:none # Core ATA subsystem (REQUIRED)
stage1:scsi_mod:none # SCSI subsystem
stage1:sd_mod:none # SCSI disk support
stage1:ahci:none # SATA AHCI
stage1:nvme_core:none # Core NVMe subsystem (REQUIRED)
stage1:nvme:none # NVMe storage
stage1:tun:none # TUN/TAP for networking
stage1:overlay:none # OverlayFS for containers
stage1:fuse:none # OverlayFS for containers
# Stage 1: USB keyboard support (host controllers + HID) # Storage
stage1:xhci_pci:none stage1:libata
stage1:xhci_hcd:none stage1:libahci
stage1:ehci_pci:none stage1:ahci
stage1:ehci_hcd:none stage1:scsi_mod
stage1:ohci_pci:none stage1:sd_mod
stage1:ohci_hcd:none stage1:nvme_core
stage1:uhci_hcd:none stage1:nvme
stage1:usbhid:none stage1:virtio_blk
stage1:hid_generic:none stage1:virtio_scsi
stage1:hid:none
stage1:atkbd:none
stage1:libps2:none
stage1:i2c_smbus:none
stage1:serio:none
stage1:i8042i:none
# Core virtio
stage1:virtio
stage1:virtio_ring
stage1:virtio_pci
stage1:virtio_pci_legacy_dev
stage1:virtio_pci_modern_dev
# Networking (common NICs)
stage1:virtio_net
stage1:e1000
stage1:e1000e
stage1:igb
stage1:ixgbe
stage1:igc
stage1:i40e
stage1:ice
stage1:r8169
stage1:8139too
stage1:8139cp
stage1:bnx2
stage1:bnx2x
stage1:tg3
stage1:tun
# PHY drivers
stage1:realtek
# Broadcom PHY families (required for many Broadcom NICs)
stage1:broadcom
stage1:bcm7xxx
stage1:bcm87xx
stage1:bcm_phy_lib
stage1:bcm_phy_ptp
# Filesystems / overlay
stage1:overlay
stage1:fuse
# USB host controllers and HID/keyboard input
stage1:xhci_pci
stage1:xhci_hcd
stage1:ehci_pci
stage1:ehci_hcd
stage1:ohci_pci
stage1:ohci_hcd
stage1:uhci_hcd
stage1:usbhid
stage1:hid_generic
stage1:hid
stage1:atkbd
stage1:libps2
stage1:i8042
stage1:evdev
stage1:serio_raw
stage1:serio
# Keep stage2 empty; we only use stage1 in this build
# stage2: (intentionally unused)

View File

@@ -1,2 +0,0 @@
exec: sh /etc/zinit/init/cgroup.sh
oneshot: true

View File

@@ -1 +0,0 @@
exec: depmod -a

View File

@@ -1,2 +0,0 @@
exec: /sbin/getty -L 115200 ttyS0 vt100
restart: always

View File

@@ -1,2 +0,0 @@
exec: /sbin/getty console linux
restart: always

View File

@@ -1,2 +0,0 @@
exec: haveged -w 1024 -d 32 -i 32 -v 1
oneshot: true

View File

@@ -1,6 +0,0 @@
#!/bin/bash
echo "start ash terminal"
while true; do
getty -l /bin/ash -n 19200 tty2
done

View File

@@ -1,10 +0,0 @@
set -x
mount -t tmpfs cgroup_root /sys/fs/cgroup
subsys="pids cpuset cpu cpuacct blkio memory devices freezer net_cls perf_event net_prio hugetlb"
for sys in $subsys; do
mkdir -p /sys/fs/cgroup/$sys
mount -t cgroup $sys -o $sys /sys/fs/cgroup/$sys/
done

View File

@@ -1,10 +0,0 @@
#!/bin/bash
modprobe fuse
modprobe btrfs
modprobe tun
modprobe br_netfilter
echo never > /sys/kernel/mm/transparent_hugepage/enabled
ulimit -n 524288

View File

@@ -1,10 +0,0 @@
#!/bin/sh
ntp_flags=$(grep -o 'ntp=.*' /proc/cmdline | sed 's/^ntp=//')
params=""
if [ -n "$ntp_flags" ]; then
params=$(echo "-p $ntp_flags" | sed s/,/' -p '/g)
fi
exec ntpd -n $params

View File

@@ -1,4 +0,0 @@
#!/bin/bash
echo "Enable ip forwarding"
echo 1 > /proc/sys/net/ipv4/ip_forward

View File

@@ -1,3 +0,0 @@
#!/bin/sh
mkdir /dev/shm
mount -t tmpfs shm /dev/shm

View File

@@ -1,15 +0,0 @@
#!/bin/ash
if [ -f /etc/ssh/ssh_host_rsa_key ]; then
# ensure existing file permissions
chown root:root /etc/ssh/ssh_host_*
chmod 600 /etc/ssh/ssh_host_*
exit 0
fi
echo "Setting up sshd"
mkdir -p /run/sshd
ssh-keygen -f /etc/ssh/ssh_host_rsa_key -N '' -t rsa
ssh-keygen -f /etc/ssh/ssh_host_dsa_key -N '' -t dsa
ssh-keygen -f /etc/ssh/ssh_host_ecdsa_key -N '' -t ecdsa -b 521
ssh-keygen -f /etc/ssh/ssh_host_ed25519_key -N '' -t ed25519

View File

@@ -1,4 +0,0 @@
#!/bin/sh
udevadm trigger --action=add
udevadm settle

View File

@@ -1,2 +0,0 @@
exec: ip l set lo up
oneshot: true

View File

@@ -1,2 +0,0 @@
exec: sh /etc/zinit/init/modprobe.sh
oneshot: true

View File

@@ -1,6 +0,0 @@
exec: /usr/bin/mycelium --key-file /tmp/mycelium_priv_key.bin
--tun-name my0 --silent --peers tcp://188.40.132.242:9651 tcp://136.243.47.186:9651
tcp://185.69.166.7:9651 tcp://185.69.166.8:9651 tcp://65.21.231.58:9651 tcp://65.109.18.113:9651
tcp://209.159.146.190:9651 tcp://5.78.122.16:9651 tcp://5.223.43.251:9651 tcp://142.93.217.194:9651
after:
- network

View File

@@ -1,5 +0,0 @@
exec: dhcpcd eth0
after:
- depmod
- udevd
- udev-trigger

View File

@@ -1,3 +0,0 @@
exec: sh /etc/zinit/init/ntpd.sh
after:
- network

View File

@@ -1,2 +0,0 @@
exec: sh /etc/zinit/init/routing.sh
oneshot: true

View File

@@ -1,2 +0,0 @@
exec: /etc/zinit/init/shm.sh
oneshot: true

View File

@@ -1,2 +0,0 @@
exec: sh /etc/zinit/init/sshd-setup.sh
oneshot: true

View File

@@ -1,3 +0,0 @@
exec: /usr/sbin/sshd -D -e
after:
- sshd-setup

View File

@@ -1,6 +0,0 @@
exec: sh /etc/zinit/init/udev.sh
oneshot: true
after:
- depmod
- udevmon
- udevd

View File

@@ -1 +0,0 @@
exec: udevd

View File

@@ -1 +0,0 @@
exec: udevadm monitor

View File

@@ -1,33 +0,0 @@
# Main zinit configuration for Zero OS Alpine
# This replaces OpenRC completely
# Logging configuration
log_level: debug
log_file: /var/log/zinit/zinit.log
# Initialization phases
init:
# Phase 1: Critical system setup
- stage1-modules
- udevd
- depmod
# Phase 2: Extended hardware and networking
- stage2-modules
- network
- lo
# Phase 3: System services
- routing
- ntp
- haveged
# Phase 4: User services
- sshd-setup
- sshd
- getty
- console
- gettyconsole
# Service dependencies and ordering managed by individual service files
# All services are defined in the services/ subdirectory

View File

@@ -1,2 +0,0 @@
exec: /sbin/getty -L 9600 console
restart: always

View File

@@ -1,2 +1,2 @@
exec: /sbin/agetty --noclear -a root tty1 linux exec: /sbin/agetty --noclear -a root console linux
restart: always restart: always

View File

@@ -0,0 +1,2 @@
exec: /sbin/agetty --noclear -a root tty2 vt100
restart: always

View File

@@ -1,2 +1,5 @@
exec: haveged -w 1024 -d 32 -i 32 -v 1 exec: haveged -w 1024 -d 32 -i 32 -v 1
oneshot: true oneshot: true
after:
- shm

View File

@@ -8,7 +8,38 @@ log() { echo "[rfs-firmware] $*"; }
RFS_BIN="${RFS_BIN:-rfs}" RFS_BIN="${RFS_BIN:-rfs}"
TARGET="/lib/firmware" TARGET="/lib/firmware"
BASE_URL="${FLISTS_BASE_URL:-https://zos.grid.tf/store/flists}"
# Accept both FLISTS_BASE_URL and FLIST_BASE_URL (alias); prefer FLISTS_BASE_URL
if [ -n "${FLISTS_BASE_URL:-}" ]; then
BASE_URL="${FLISTS_BASE_URL}"
elif [ -n "${FLIST_BASE_URL:-}" ]; then
BASE_URL="${FLIST_BASE_URL}"
else
BASE_URL="https://zos.grid.tf/store/flists"
fi
# HTTP readiness helper: wait until BASE_URL responds to HTTP(S)
wait_for_http() {
url="$1"
tries="${2:-60}"
delay="${3:-2}"
i=0
while [ "$i" -lt "$tries" ]; do
if command -v wget >/dev/null 2>&1; then
if wget -q --spider "$url"; then
return 0
fi
elif command -v busybox >/dev/null 2>&1; then
if busybox wget -q --spider "$url"; then
return 0
fi
fi
i=$((i+1))
log "waiting for $url (attempt $i/$tries)"
sleep "$delay"
done
return 1
}
# Allow override via env # Allow override via env
if [ -n "${FIRMWARE_FLIST:-}" ] && [ -f "${FIRMWARE_FLIST}" ]; then if [ -n "${FIRMWARE_FLIST:-}" ] && [ -f "${FIRMWARE_FLIST}" ]; then
@@ -29,11 +60,18 @@ else
fi fi
if [ -z "${FL:-}" ]; then if [ -z "${FL:-}" ]; then
# Try remote fetch as a fallback # Try remote fetch as a fallback (but first ensure BASE_URL is reachable)
mkdir -p /etc/rfs mkdir -p /etc/rfs
FL="/etc/rfs/firmware-latest.fl" FL="/etc/rfs/firmware-latest.fl"
URL="${BASE_URL}/firmware-latest.fl" URL="${BASE_URL%/}/firmware-latest.fl"
log "firmware-latest.fl not found locally; fetching ${URL}" log "firmware-latest.fl not found locally; will fetch ${URL}"
# Probe BASE_URL root so DNS/HTTP are ready before fetch
ROOT_URL="${BASE_URL%/}/"
if ! wait_for_http "$ROOT_URL" 60 2; then
log "BASE_URL not reachable yet: ${ROOT_URL}; continuing to attempt fetch anyway"
fi
log "fetching ${URL}"
if command -v wget >/dev/null 2>&1; then if command -v wget >/dev/null 2>&1; then
wget -q -O "${FL}" "${URL}" || true wget -q -O "${FL}" "${URL}" || true

View File

@@ -3,7 +3,17 @@
modprobe fuse modprobe fuse
modprobe btrfs modprobe btrfs
modprobe tun modprobe tun
modprobe br_netfilter modprobe
modprobe usbhid
modprobe hid_generic
modprobe hid
modprobe atkbd
modprobe libps2
modprobe i2c_smbus
modprobe serio
modprobe i8042i
echo never > /sys/kernel/mm/transparent_hugepage/enabled echo never > /sys/kernel/mm/transparent_hugepage/enabled

View File

@@ -9,7 +9,37 @@ log() { echo "[rfs-modules] $*"; }
RFS_BIN="${RFS_BIN:-rfs}" RFS_BIN="${RFS_BIN:-rfs}"
KVER="$(uname -r)" KVER="$(uname -r)"
TARGET="/lib/modules/${KVER}" TARGET="/lib/modules/${KVER}"
BASE_URL="${FLISTS_BASE_URL:-https://zos.grid.tf/store/flists}" # Accept both FLISTS_BASE_URL and FLIST_BASE_URL (alias); prefer FLISTS_BASE_URL
if [ -n "${FLISTS_BASE_URL:-}" ]; then
BASE_URL="${FLISTS_BASE_URL}"
elif [ -n "${FLIST_BASE_URL:-}" ]; then
BASE_URL="${FLIST_BASE_URL}"
else
BASE_URL="https://zos.grid.tf/store/flists"
fi
# HTTP readiness helper: wait until BASE_URL responds to HTTP(S)
wait_for_http() {
url="$1"
tries="${2:-60}"
delay="${3:-2}"
i=0
while [ "$i" -lt "$tries" ]; do
if command -v wget >/dev/null 2>&1; then
if wget -q --spider "$url"; then
return 0
fi
elif command -v busybox >/dev/null 2>&1; then
if busybox wget -q --spider "$url"; then
return 0
fi
fi
i=$((i+1))
log "waiting for $url (attempt $i/$tries)"
sleep "$delay"
done
return 1
}
# Allow override via env # Allow override via env
if [ -n "${MODULES_FLIST:-}" ] && [ -f "${MODULES_FLIST}" ]; then if [ -n "${MODULES_FLIST:-}" ] && [ -f "${MODULES_FLIST}" ]; then
@@ -33,8 +63,15 @@ if [ -z "${FL:-}" ]; then
# Try remote fetch as a fallback (modules-<uname -r>-Zero-OS.fl) # Try remote fetch as a fallback (modules-<uname -r>-Zero-OS.fl)
mkdir -p /etc/rfs mkdir -p /etc/rfs
FL="/etc/rfs/modules-${KVER}.fl" FL="/etc/rfs/modules-${KVER}.fl"
URL="${BASE_URL}/modules-${KVER}-Zero-OS.fl" URL="${BASE_URL%/}/modules-${KVER}-Zero-OS.fl"
log "modules-${KVER}.fl not found locally; fetching ${URL}" log "modules-${KVER}.fl not found locally; will fetch ${URL}"
# Probe BASE_URL root so DNS/HTTP are ready before fetch
ROOT_URL="${BASE_URL%/}/"
if ! wait_for_http "$ROOT_URL" 60 2; then
log "BASE_URL not reachable yet: ${ROOT_URL}; continuing to attempt fetch anyway"
fi
log "fetching ${URL}"
if command -v wget >/dev/null 2>&1; then if command -v wget >/dev/null 2>&1; then
wget -q -O "${FL}" "${URL}" || true wget -q -O "${FL}" "${URL}" || true

View File

@@ -1,4 +1,4 @@
exec: sh /etc/zinit/init/modules.sh exec: sh /etc/zinit/init/modules.sh
restart: always restart: always
after: after:
- network - rfs-modules

View File

@@ -1,5 +1,5 @@
exec: /etc/zinit/init/shm.sh exec: /etc/zinit/init/shm.sh
oneshot: true oneshot: true
after: after:
- firmware - rfs-modules
- modules - rfs-firmware

View File

@@ -1,3 +1,4 @@
exec: /usr/sbin/sshd -D -e exec: /usr/sbin/sshd -D -e
after: after:
- sshd-setup - sshd-setup
- shm

View File

@@ -1,5 +1,4 @@
exec: /bin/sh -c "udevadm control --reload; udevadm trigger --action=add --type=subsystems; udevadm trigger --action=add --type=devices; udevadm settle" exec: sh -c "sleep 3 ; /etc/zinit/init/udev.sh"
oneshot: true oneshot: true
after: after:
- rfs-modules - rfs-modules
- rfs-firmware

View File

@@ -1,106 +0,0 @@
#!/bin/sh -x
# Alpine-based Zero-OS Init Script
# Maintains identical flow to original busybox version
echo ""
echo "============================================"
echo "== ZERO-OS ALPINE INITRAMFS =="
echo "============================================"
echo "[+] creating ram filesystem"
target="/mnt/root"
mkdir -p $target
mount -t proc proc /proc
mount -t sysfs sysfs /sys
mount -t tmpfs tmpfs /mnt/root -o size=1024M
mount -t devtmpfs devtmpfs /dev
echo "[+] building ram filesystem"
# Copy Alpine filesystem to tmpfs (same as original)
echo " copying /bin..."
cp -ar /bin $target
echo " copying /etc..."
cp -ar /etc $target
echo " copying /lib..."
cp -ar /lib* $target
echo " copying /usr..."
cp -ar /usr $target
echo " copying /root..."
cp -ar /root $target
echo " copying /sbin..."
cp -ar /sbin $target
echo " copying /tmp..."
cp -ar /tmp $target
echo " copying /var..."
cp -ar /var $target
echo " copying /run..."
cp -ar /run $target
# Create essential directories
mkdir -p $target/dev
mkdir -p $target/sys
mkdir -p $target/proc
mkdir -p $target/mnt
# Mount filesystems in tmpfs
mount -t proc proc $target/proc
mount -t sysfs sysfs $target/sys
mount -t devtmpfs devtmpfs $target/dev
# Mount devpts for terminals
mkdir -p $target/dev/pts
mount -t devpts devpts $target/dev/pts
echo "[+] setting environment"
export PATH
echo "[+] probing drivers"
# Use Alpine's udev instead of busybox udevadm
if [ -x /sbin/udevd ]; then
echo " starting udevd..."
udevd --daemon
echo " triggering device discovery..."
udevadm trigger --action=add --type=subsystems
udevadm trigger --action=add --type=devices
udevadm settle
echo " stopping udevd..."
kill $(pidof udevd) || true
else
echo " warning: udevd not found, skipping hardware detection"
fi
echo "[+] loading essential drivers"
# Load core drivers for storage and network
modprobe btrfs 2>/dev/null || true
modprobe fuse 2>/dev/null || true
modprobe overlay 2>/dev/null || true
# Load storage drivers
modprobe ahci 2>/dev/null || true
modprobe nvme 2>/dev/null || true
modprobe virtio_blk 2>/dev/null || true
modprobe virtio_scsi 2>/dev/null || true
# Load network drivers
modprobe virtio_net 2>/dev/null || true
modprobe e1000 2>/dev/null || true
modprobe e1000e 2>/dev/null || true
# Unmount init filesystems
umount /proc 2>/dev/null || true
umount /sys 2>/dev/null || true
echo "[+] checking for debug files"
if [ -e /init-debug ]; then
echo " executing debug script..."
sh /init-debug
fi
echo "[+] switching root"
echo " exec switch_root /mnt/root /sbin/zinit init"
exec switch_root /mnt/root /sbin/zinit -d init
##

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -1,59 +0,0 @@
# Essential kernel modules for Zero-OS Alpine initramfs
# This file contains a curated list of essential modules for network and storage functionality
# Comments are supported (lines starting with #)
# Network drivers - Intel
e1000
e1000e
igb
ixgbe
i40e
ice
# Network drivers - Realtek
r8169
8139too
8139cp
# Network drivers - Broadcom
bnx2
bnx2x
tg3
b44
# Network drivers - Atheros
atl1
atl1e
atl1c
alx
# VirtIO drivers
virtio_net
virtio_scsi
virtio_blk
virtio_pci
# Tunnel and container support
tun
overlay
# Storage subsystem (essential only)
scsi_mod
sd_mod
# Control Groups (cgroups v1 and v2) - essential for container management
cgroup_pids
cgroup_freezer
cgroup_perf_event
cgroup_device
cgroup_cpuset
cgroup_bpf
cgroup_debug
memcg
blkio_cgroup
cpu_cgroup
cpuacct
hugetlb_cgroup
net_cls_cgroup
net_prio_cgroup
devices_cgroup

View File

@@ -1,46 +0,0 @@
# MINIMAL Alpine packages for Zero-OS embedded initramfs
# Target: ~50MB total (not 700MB!)
# Core system (essential only)
alpine-baselayout
busybox
musl
# Module loading & hardware detection
eudev
eudev-hwids
eudev-libs
eudev-netifnames
kmod
# Console/terminal management
util-linux
# Essential networking (for Zero-OS connectivity)
iproute2
ethtool
# Filesystem support (minimal)
btrfs-progs
dosfstools
# Essential libraries only
zlib
# Network utilities (minimal)
dhcpcd
tcpdump
bmon
# Random number generation (for crypto/security)
haveged
# SSH access and terminal multiplexer
openssh-server
zellij
# Essential debugging and monitoring tools included
# NO development tools, NO curl/wget, NO python, NO redis
# NO massive linux-firmware package
# Other tools will be loaded from RFS after network connectivity

View File

@@ -1,10 +0,0 @@
# sources.conf - Components to download and build for initramfs
# Format: TYPE:NAME:URL:VERSION:BUILD_FUNCTION[:EXTRA_OPTIONS]
# Git repositories to clone and build
git:zinit:https://github.com/threefoldtech/zinit:master:build_zinit
git:mycelium:https://github.com/threefoldtech/mycelium:0.6.1:build_mycelium
git:rfs:https://github.com/threefoldtech/rfs:development:build_rfs
# Pre-built releases to download
release:corex:https://github.com/threefoldtech/corex/releases/download/2.1.4/corex-2.1.4-amd64-linux-static:2.1.4:install_corex:rename=corex

View File

@@ -1,2 +0,0 @@
exec: sh /etc/zinit/init/cgroup.sh
oneshot: true

View File

@@ -1,2 +0,0 @@
exec: /sbin/getty -L 9600 console
restart: always

View File

@@ -1 +0,0 @@
exec: depmod -a

View File

@@ -1,2 +0,0 @@
exec: /sbin/getty -L 115200 ttyS0 vt100
restart: always

View File

@@ -1,2 +0,0 @@
exec: /sbin/getty console linux
restart: always

View File

@@ -1,2 +0,0 @@
exec: haveged -w 1024 -d 32 -i 32 -v 1
oneshot: true

View File

@@ -1,6 +0,0 @@
#!/bin/bash
echo "start ash terminal"
while true; do
getty -l /bin/ash -n 19200 tty2
done

View File

@@ -1,10 +0,0 @@
set -x
mount -t tmpfs cgroup_root /sys/fs/cgroup
subsys="pids cpuset cpu cpuacct blkio memory devices freezer net_cls perf_event net_prio hugetlb"
for sys in $subsys; do
mkdir -p /sys/fs/cgroup/$sys
mount -t cgroup $sys -o $sys /sys/fs/cgroup/$sys/
done

View File

@@ -1,10 +0,0 @@
#!/bin/bash
modprobe fuse
modprobe btrfs
modprobe tun
modprobe br_netfilter
echo never > /sys/kernel/mm/transparent_hugepage/enabled
ulimit -n 524288

View File

@@ -1,10 +0,0 @@
#!/bin/sh
ntp_flags=$(grep -o 'ntp=.*' /proc/cmdline | sed 's/^ntp=//')
params=""
if [ -n "$ntp_flags" ]; then
params=$(echo "-p $ntp_flags" | sed s/,/' -p '/g)
fi
exec ntpd -n $params

View File

@@ -1,4 +0,0 @@
#!/bin/bash
echo "Enable ip forwarding"
echo 1 > /proc/sys/net/ipv4/ip_forward

View File

@@ -1,3 +0,0 @@
#!/bin/sh
mkdir /dev/shm
mount -t tmpfs shm /dev/shm

View File

@@ -1,15 +0,0 @@
#!/bin/ash
if [ -f /etc/ssh/ssh_host_rsa_key ]; then
# ensure existing file permissions
chown root:root /etc/ssh/ssh_host_*
chmod 600 /etc/ssh/ssh_host_*
exit 0
fi
echo "Setting up sshd"
mkdir -p /run/sshd
ssh-keygen -f /etc/ssh/ssh_host_rsa_key -N '' -t rsa
ssh-keygen -f /etc/ssh/ssh_host_dsa_key -N '' -t dsa
ssh-keygen -f /etc/ssh/ssh_host_ecdsa_key -N '' -t ecdsa -b 521
ssh-keygen -f /etc/ssh/ssh_host_ed25519_key -N '' -t ed25519

View File

@@ -1,4 +0,0 @@
#!/bin/sh
udevadm trigger --action=add
udevadm settle

View File

@@ -1,2 +0,0 @@
exec: ip l set lo up
oneshot: true

View File

@@ -1,2 +0,0 @@
exec: sh /etc/zinit/init/modprobe.sh
oneshot: true

View File

@@ -1,6 +0,0 @@
exec: /usr/bin/mycelium --key-file /tmp/mycelium_priv_key.bin
--tun-name my0 --silent --peers tcp://188.40.132.242:9651 tcp://136.243.47.186:9651
tcp://185.69.166.7:9651 tcp://185.69.166.8:9651 tcp://65.21.231.58:9651 tcp://65.109.18.113:9651
tcp://209.159.146.190:9651 tcp://5.78.122.16:9651 tcp://5.223.43.251:9651 tcp://142.93.217.194:9651
after:
- network

View File

@@ -1,5 +0,0 @@
exec: dhcpcd eth0
after:
- depmod
- udevd
- udev-trigger

View File

@@ -1,3 +0,0 @@
exec: sh /etc/zinit/init/ntpd.sh
after:
- network

View File

@@ -1,2 +0,0 @@
exec: sh /etc/zinit/init/routing.sh
oneshot: true

View File

@@ -1,2 +0,0 @@
exec: /etc/zinit/init/shm.sh
oneshot: true

View File

@@ -1,2 +0,0 @@
exec: sh /etc/zinit/init/sshd-setup.sh
oneshot: true

View File

@@ -1,3 +0,0 @@
exec: /usr/sbin/sshd -D -e
after:
- sshd-setup

View File

@@ -1,6 +0,0 @@
exec: sh /etc/zinit/init/udev.sh
oneshot: true
after:
- depmod
- udevmon
- udevd

View File

@@ -1 +0,0 @@
exec: udevd

View File

@@ -1 +0,0 @@
exec: udevadm monitor

View File

@@ -86,7 +86,7 @@ Initramfs Assembly Key Functions
- Components copy: [bash.initramfs_copy_components()](scripts/lib/initramfs.sh:101) - Components copy: [bash.initramfs_copy_components()](scripts/lib/initramfs.sh:101)
- Installs built components (zinit/rfs/mycelium/corex) into proper locations, strips/UPX where applicable. - Installs built components (zinit/rfs/mycelium/corex) into proper locations, strips/UPX where applicable.
- Modules setup: [bash.initramfs_setup_modules()](scripts/lib/initramfs.sh:229) - Modules setup: [bash.initramfs_setup_modules()](scripts/lib/initramfs.sh:229)
- Reads [config/modules.conf](config/modules.conf), resolves deps via [bash.initramfs_resolve_module_dependencies()](scripts/lib/initramfs.sh:318), generates stage1 list with firmware correlation. - Reads [config/modules.conf](config/modules.conf), resolves deps via [bash.initramfs_resolve_module_dependencies()](scripts/lib/initramfs.sh:318), generates stage1 list (firmware hints in modules.conf are ignored; firmware.conf is authoritative).
- Create module scripts: [bash.initramfs_create_module_scripts()](scripts/lib/initramfs.sh:427) - Create module scripts: [bash.initramfs_create_module_scripts()](scripts/lib/initramfs.sh:427)
- Writes /etc/zinit/init/stage1-modules.sh and stage2-modules.sh for zinit to load modules. - Writes /etc/zinit/init/stage1-modules.sh and stage2-modules.sh for zinit to load modules.
- Binary size optimization: [bash.initramfs_strip_and_upx()](scripts/lib/initramfs.sh:491) - Binary size optimization: [bash.initramfs_strip_and_upx()](scripts/lib/initramfs.sh:491)
@@ -110,10 +110,19 @@ RFS Flists (modules/firmware)
- Packing scripts: - Packing scripts:
- Modules: [bash.pack-modules.sh](scripts/rfs/pack-modules.sh:1) - Modules: [bash.pack-modules.sh](scripts/rfs/pack-modules.sh:1)
- Firmware: [bash.pack-firmware.sh](scripts/rfs/pack-firmware.sh:1) - Firmware: [bash.pack-firmware.sh](scripts/rfs/pack-firmware.sh:1)
- Firmware policy:
- For initramfs: [config/firmware.conf](config/firmware.conf) is the single source of truth for preinstalled firmware; modules.conf hints are ignored.
- For RFS: install all Alpine linux-firmware* packages into the build container and pack from /lib/firmware (full set for runtime).
- Integrated in stage_rfs_flists: - Integrated in stage_rfs_flists:
- Embeds /etc/rfs/modules-<FULL_KERNEL_VERSION>.fl - Embeds /etc/rfs/modules-<FULL_KERNEL_VERSION>.fl
- Embeds /etc/rfs/firmware-latest.fl (or tagged by FIRMWARE_TAG) - Embeds /etc/rfs/firmware-latest.fl (or tagged by FIRMWARE_TAG)
- See [bash.main_build_process() — stage_rfs_flists](scripts/build.sh:298) - See [bash.main_build_process() — stage_rfs_flists](scripts/build.sh:298)
- Runtime mount/readiness:
- Firmware flist mounts over /lib/firmware (overmount hides any initramfs firmware).
- Modules flist mounts at /lib/modules/$(uname -r).
- Init scripts probe BASE_URL reachability (accepts FLISTS_BASE_URL or FLIST_BASE_URL) and wait for HTTP(S) before fetching:
- Firmware: [sh.firmware.sh](config/zinit/init/firmware.sh:1)
- Modules: [sh.modules.sh](config/zinit/init/modules.sh:1)
Branding Behavior (Passwordless Root, motd/issue) Branding Behavior (Passwordless Root, motd/issue)
- Finalization hook: [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:575) - Finalization hook: [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:575)
@@ -126,10 +135,12 @@ Branding Behavior (Passwordless Root, motd/issue)
- Branding also updates /etc/motd and /etc/issue to Zero-OS. - Branding also updates /etc/motd and /etc/issue to Zero-OS.
Console and getty Console and getty
- Early keyboard and debug:
- [config/init](config/init) preloads input/HID and USB HCD modules (i8042, atkbd, usbhid, hid, hid_generic, evdev, xhci/ehci/ohci/uhci) so console input works before zinit/rfs.
- Kernel cmdline initdebug=true opens an early interactive shell; if /init-debug exists and is executable, it runs preferentially.
- Serial and console getty configs (zinit service YAML): - Serial and console getty configs (zinit service YAML):
- [config/zinit/getty.yaml](config/zinit/getty.yaml) - [config/zinit/getty-tty1.yaml](config/zinit/getty-tty1.yaml)
- [config/zinit/gettyconsole.yaml](config/zinit/gettyconsole.yaml) - [config/zinit/getty-console.yaml](config/zinit/getty-console.yaml)
- [config/zinit/console.yaml](config/zinit/console.yaml)
- Optional ash login loop (not enabled unless referenced): - Optional ash login loop (not enabled unless referenced):
- [bash.ashloging.sh](config/zinit/init/ashloging.sh:1) - [bash.ashloging.sh](config/zinit/init/ashloging.sh:1)
@@ -156,14 +167,23 @@ How to Verify Passwordless Root
Stage System and Incremental Rebuilds Stage System and Incremental Rebuilds
- Stage markers stored in .build-stages/ (one file per stage). - Stage markers stored in .build-stages/ (one file per stage).
- To minimally rebuild: - Minimal rebuild helper (host or container):
- Remove relevant .done files, e.g.: - [scripts/rebuild-after-zinit.sh](scripts/rebuild-after-zinit.sh) clears only: modules_setup, modules_copy, init_script, zinit_setup, validation, initramfs_create, initramfs_test (kernel_build only with --with-kernel; kernel_modules only with --refresh-container-mods).
- initramfs_create.done initramfs_test.done validation.done - Flags:
- --with-kernel (also rebuild kernel; ensures cpio is recreated right before embedding)
- --refresh-container-mods (rebuild container /lib/modules for fresh containers)
- --verify-only (report changed files and stage status; no rebuild)
- Shows stage status before/after marker removal; no --rebuild-from is passed by default (relies on markers only).
- Manual minimal rebuild:
- Remove relevant .done files, e.g.: initramfs_create.done initramfs_test.done validation.done
- Rerun: DEBUG=1 ./scripts/build.sh --skip-tests - Rerun: DEBUG=1 ./scripts/build.sh --skip-tests
- Show status: - Show status:
- ./scripts/build.sh --show-stages - ./scripts/build.sh --show-stages
Key Decisions (current) Key Decisions (current)
- Firmware selection for initramfs comes exclusively from [config/firmware.conf](config/firmware.conf); firmware hints in modules.conf are ignored to avoid duplication/mismatch.
- Runtime firmware flist overmounts /lib/firmware after network readiness; init scripts wait for FLISTS_BASE_URL/FLIST_BASE_URL HTTP reachability before fetching.
- Early keyboard and debug shell added to [config/init](config/init) as described above.
- Branding enforces passwordless root via passwd -d -R inside initramfs finalization, avoiding direct edits of passwd/shadow files. - Branding enforces passwordless root via passwd -d -R inside initramfs finalization, avoiding direct edits of passwd/shadow files.
- Directory paths normalized to absolute after loading config to avoid CWD-sensitive behavior. - Directory paths normalized to absolute after loading config to avoid CWD-sensitive behavior.
- Container image contains shadow suite to ensure passwd/chpasswd availability; perl removed. - Container image contains shadow suite to ensure passwd/chpasswd availability; perl removed.
@@ -172,11 +192,11 @@ File Pointers (quick jump)
- Orchestrator: [scripts/build.sh](scripts/build.sh) - Orchestrator: [scripts/build.sh](scripts/build.sh)
- Common and config loading: [bash.common.sh](scripts/lib/common.sh:1) - Common and config loading: [bash.common.sh](scripts/lib/common.sh:1)
- Finalization hook: [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:575) - Finalization hook: [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:575)
- Passwordless deletion: [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:592)
- Validation entry: [bash.initramfs_validate()](scripts/lib/initramfs.sh:799) - Validation entry: [bash.initramfs_validate()](scripts/lib/initramfs.sh:799)
- CPIO creation: [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:688) - CPIO creation: [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:688)
- Kernel embed config: [bash.kernel_modify_config_for_initramfs()](scripts/lib/kernel.sh:130) - Kernel embed config: [bash.kernel_modify_config_for_initramfs()](scripts/lib/kernel.sh:130)
- RFS packers: [bash.pack-modules.sh](scripts/rfs/pack-modules.sh:1), [bash.pack-firmware.sh](scripts/rfs/pack-firmware.sh:1) - RFS packers: [bash.pack-modules.sh](scripts/rfs/pack-modules.sh:1), [bash.pack-firmware.sh](scripts/rfs/pack-firmware.sh:1)
- USB creator: [scripts/make-grub-usb.sh](scripts/make-grub-usb.sh)
Change Log Change Log
- 2025-09-09: - 2025-09-09:

View File

@@ -34,6 +34,8 @@ Repository map (jump-points)
- RFS flists tooling: - RFS flists tooling:
- Modules packer: [bash.pack-modules.sh](scripts/rfs/pack-modules.sh:1) - Modules packer: [bash.pack-modules.sh](scripts/rfs/pack-modules.sh:1)
- Firmware packer: [bash.pack-firmware.sh](scripts/rfs/pack-firmware.sh:1) - Firmware packer: [bash.pack-firmware.sh](scripts/rfs/pack-firmware.sh:1)
- Boot media utility:
- GRUB USB creator: [scripts/make-grub-usb.sh](scripts/make-grub-usb.sh)
High-priority behaviors and policies High-priority behaviors and policies
1) Branding passwordless root (shadow-aware) 1) Branding passwordless root (shadow-aware)
@@ -53,18 +55,20 @@ High-priority behaviors and policies
- Pre-CPIO essential check includes “home”: - Pre-CPIO essential check includes “home”:
- [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:680) - [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:680)
4) Remote flist fallback (modules + firmware) 4) Remote flist fallback + readiness (modules + firmware)
- When local manifests are missing, fetch from zos.grid.tf and mount via rfs: - When local manifests are missing, fetch from zos.grid.tf and mount via rfs:
- Firmware fallback: [sh.firmware.sh](config/zinit/init/firmware.sh:1) - Firmware: [sh.firmware.sh](config/zinit/init/firmware.sh:1)
- Default BASE_URL: https://zos.grid.tf/store/flists - BASE_URL from FLISTS_BASE_URL (or FLIST_BASE_URL alias), default https://zos.grid.tf/store/flists
- Fetch path: ${BASE_URL}/firmware-latest.fl to /etc/rfs/firmware-latest.fl - Probes BASE_URL for HTTP(S) readiness (wget --spider) before fetching firmware-latest.fl
- Modules fallback: [sh.modules.sh](config/zinit/init/modules.sh:1) - Fetch path: ${BASE_URL%/}/firmware-latest.fl to /etc/rfs/firmware-latest.fl
- Fetch path: ${BASE_URL}/modules-$(uname -r)-Zero-OS.fl to /etc/rfs/modules-$(uname -r).fl - Modules: [sh.modules.sh](config/zinit/init/modules.sh:1)
- Env overrides: - BASE_URL from FLISTS_BASE_URL (or FLIST_BASE_URL alias)
- FIRMWARE_FLIST, MODULES_FLIST: use local file if provided - Probes BASE_URL for HTTP(S) readiness before fetching modules-$(uname -r)-Zero-OS.fl
- RFS_BIN: defaults to rfs - Env overrides:
- FLISTS_BASE_URL: overrides base URL - FIRMWARE_FLIST, MODULES_FLIST: use local file if provided
- wget is available (initramfs includes it); scripts prefer wget, fallback to busybox wget if needed. - RFS_BIN: defaults to rfs
- FLISTS_BASE_URL or FLIST_BASE_URL: override base URL
- wget is available (initramfs includes it); scripts prefer wget, fallback to busybox wget if needed.
5) Incremental build guards 5) Incremental build guards
- Kernel build now defaults INITRAMFS_ARCHIVE if unset (fix for unbound var on incremental runs): - Kernel build now defaults INITRAMFS_ARCHIVE if unset (fix for unbound var on incremental runs):
@@ -72,6 +76,12 @@ High-priority behaviors and policies
- Initramfs test stage already guards INITRAMFS_ARCHIVE: - Initramfs test stage already guards INITRAMFS_ARCHIVE:
- [bash.stage_initramfs_test()](scripts/build.sh:385) - [bash.stage_initramfs_test()](scripts/build.sh:385)
6) Early keyboard input and debug shell
- Early HID/input and USB HCD modules are preloaded before zinit to ensure console usability:
- [config.init](config/init:80)
- Debug hook: kernel cmdline initdebug=true runs /init-debug if present or drops to a shell:
- [config.init](config/init:115)
Flags and config Flags and config
- Config file: [config/build.conf](config/build.conf) - Config file: [config/build.conf](config/build.conf)
- Branding flags: - Branding flags:
@@ -85,6 +95,9 @@ Flags and config
- COMPONENTS_DIR="components" - COMPONENTS_DIR="components"
- KERNEL_DIR="kernel" - KERNEL_DIR="kernel"
- DIST_DIR="dist" - DIST_DIR="dist"
- Firmware policies:
- Initramfs: [config/firmware.conf](config/firmware.conf) is authoritative; modules.conf firmware hints are ignored.
- RFS: full Alpine firmware set is installed into container and packed from /lib/firmware (see [bash.pack-firmware.sh](scripts/rfs/pack-firmware.sh:1)).
- Firmware flist naming tag: - Firmware flist naming tag:
- FIRMWARE_TAG (env > config > “latest”) - FIRMWARE_TAG (env > config > “latest”)
- Container image tools (podman rootless OK) defined by [Dockerfile](Dockerfile): - Container image tools (podman rootless OK) defined by [Dockerfile](Dockerfile):
@@ -113,7 +126,7 @@ Common tasks and commands
- Show stage status: - Show stage status:
- ./scripts/build.sh --show-stages - ./scripts/build.sh --show-stages
Checklists Checklists and helpers
A) Diagnose “passwordless root not working” A) Diagnose “passwordless root not working”
- Confirm branding flags are loaded: - Confirm branding flags are loaded:
@@ -130,17 +143,34 @@ B) Fix “Initramfs directory not found: initramfs (resolved: /workspace/kernel/
- Confirm validation prints “Validation debug:” with resolved absolute path: - Confirm validation prints “Validation debug:” with resolved absolute path:
- [bash.initramfs_validate()](scripts/lib/initramfs.sh:799) - [bash.initramfs_validate()](scripts/lib/initramfs.sh:799)
C) INITRAMFS_ARCHIVE unbound during kernel build stage C) Minimal rebuild after zinit/init/modules.conf changes
- Use the helper (works from host or container):
- scripts/rebuild-after-zinit.sh
- Defaults: initramfs-only; clears only modules_setup, modules_copy, init_script, zinit_setup, validation, initramfs_create, initramfs_test
- Flags:
- --with-kernel: also rebuild kernel; cpio is recreated immediately before embedding
- --refresh-container-mods: rebuild container /lib/modules for fresh containers
- --verify-only: report changed files and stage status; no rebuild
- Stage status is printed before/after marker removal; the helper avoids --rebuild-from by default to prevent running early stages.
- Manual fallback:
- rm -f .build-stages/initramfs_create.done .build-stages/initramfs_test.done .build-stages/validation.done
- DEBUG=1 ./scripts/build.sh --skip-tests
D) INITRAMFS_ARCHIVE unbound during kernel build stage
- stage_kernel_build now defaults INITRAMFS_ARCHIVE if unset: - stage_kernel_build now defaults INITRAMFS_ARCHIVE if unset:
- [bash.stage_kernel_build()](scripts/build.sh:398) - [bash.stage_kernel_build()](scripts/build.sh:398)
- If error persists, ensure stage_initramfs_create ran or that defaulting logic sees dist/initramfs.cpio.xz. - If error persists, ensure stage_initramfs_create ran or that defaulting logic sees dist/initramfs.cpio.xz.
D) Modules/firmware not found by rfs init scripts E) Modules/firmware not found by rfs init scripts
- Confirm local manifests under /etc/rfs or remote fallback working: - Confirm local manifests under /etc/rfs or remote fallback:
- Firmware: [sh.firmware.sh](config/zinit/init/firmware.sh:1) - Firmware: [sh.firmware.sh](config/zinit/init/firmware.sh:1)
- Modules: [sh.modules.sh](config/zinit/init/modules.sh:1) - Modules: [sh.modules.sh](config/zinit/init/modules.sh:1)
- For remote:
- Set FLISTS_BASE_URL or FLIST_BASE_URL; default is https://zos.grid.tf/store/flists
- Scripts probe BASE_URL readiness (wget --spider) before fetch
- Firmware target is /lib/firmware; modules target is /lib/modules/$(uname -r)
- Confirm uname -r matches remote naming “modules-$(uname -r)-Zero-OS.fl” - Confirm uname -r matches remote naming “modules-$(uname -r)-Zero-OS.fl”
- Confirm wget present (it is in initramfs), or busybox fallback. - Confirm wget present (or busybox wget)
Project conventions Project conventions
- Edit policy: - Edit policy:
@@ -157,8 +187,9 @@ Key files to keep in sync with behavior decisions
- Validation diagnostics: [bash.initramfs_validate()](scripts/lib/initramfs.sh:799) - Validation diagnostics: [bash.initramfs_validate()](scripts/lib/initramfs.sh:799)
- Archive creation (pre-CPIO checks): [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:688) - Archive creation (pre-CPIO checks): [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:688)
- Path normalization after config: [bash.common.sh](scripts/lib/common.sh:236) - Path normalization after config: [bash.common.sh](scripts/lib/common.sh:236)
- Modules/firmware remote fallback: [sh.modules.sh](config/zinit/init/modules.sh:1), [sh.firmware.sh](config/zinit/init/firmware.sh:1) - Modules/firmware remote fallback + readiness: [sh.modules.sh](config/zinit/init/modules.sh:1), [sh.firmware.sh](config/zinit/init/firmware.sh:1)
- Kernel stage defaulting for archive: [bash.stage_kernel_build()](scripts/build.sh:398) - Kernel stage defaulting for archive: [bash.stage_kernel_build()](scripts/build.sh:398)
- GRUB USB creator: [scripts/make-grub-usb.sh](scripts/make-grub-usb.sh)
- Operational notes: [docs/NOTES.md](docs/NOTES.md) - Operational notes: [docs/NOTES.md](docs/NOTES.md)
When in doubt When in doubt

BIN
docs/img_1758452705037.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 230 KiB

View File

@@ -124,7 +124,7 @@ Directory: scripts/rfs
## Future runtime units (deferred) ## Future runtime units (deferred)
Will be added as new zinit units once flist generation is validated: Will be added as new zinit units once flist generation is validated:
- Mount firmware flist read-only at /usr/lib/firmware - Mount firmware flist read-only at /lib/firmware (overmount to hide initramfs firmware beneath)
- Mount modules flist read-only at /lib/modules/<FULL_VERSION> - Mount modules flist read-only at /lib/modules/<FULL_VERSION>
- Run depmod -a <FULL_VERSION> - Run depmod -a <FULL_VERSION>
- Run udev coldplug sequence (reload, trigger add, settle) - Run udev coldplug sequence (reload, trigger add, settle)

View File

@@ -20,10 +20,13 @@ iproute2
kmod kmod
libc-utils libc-utils
linux-firmware-bnx2 linux-firmware-bnx2
linux-firmware-bnx2x
linux-firmware-e100 linux-firmware-e100
linux-firmware-intel linux-firmware-intel
linux-firmware-qlogic linux-firmware-qlogic
linux-firmware-realtek linux-firmware-realtek
linux-firmware-rtl_nic
linux-firmware-tigon
musl musl
nftables nftables
openssh-server openssh-server

View File

@@ -1,4 +1,4 @@
root::20349:0::::: root::20354:0:::::
bin:!::0::::: bin:!::0:::::
daemon:!::0::::: daemon:!::0:::::
lp:!::0::::: lp:!::0:::::

View File

@@ -395,43 +395,35 @@ function alpine_install_firmware() {
section_header "Installing Required Firmware Packages (Selective)" section_header "Installing Required Firmware Packages (Selective)"
# Use smart firmware selection from module analysis if available # Policy: firmware.conf is the authoritative source for initramfs firmware
local firmware_packages=() local firmware_packages=()
if [[ ! -f "$firmware_conf" ]]; then
log_warn "Firmware configuration not found: ${firmware_conf}; skipping firmware installation to initramfs"
return 0
fi
# Ignore any REQUIRED_FIRMWARE_PACKAGES hints to avoid duplication/mismatch with modules.conf
if [[ -n "${REQUIRED_FIRMWARE_PACKAGES:-}" ]]; then if [[ -n "${REQUIRED_FIRMWARE_PACKAGES:-}" ]]; then
log_info "Using intelligent firmware selection based on COPIED modules only" log_info "Firmware selection: ignoring REQUIRED_FIRMWARE_PACKAGES; using ${firmware_conf} as authoritative source"
read -ra firmware_packages <<< "$REQUIRED_FIRMWARE_PACKAGES" fi
log_info "Required firmware packages (${#firmware_packages[@]}):" # Read firmware packages from config (excluding comments and empty lines)
for package in "${firmware_packages[@]}"; do while IFS=: read -r package description; do
log_info "${package}" # Skip comments and empty lines
done if [[ "$package" =~ ^[[:space:]]*# ]] || [[ -z "${package// }" ]]; then
else continue
log_info "Falling back to firmware configuration file"
if [[ ! -f "$firmware_conf" ]]; then
log_warn "No firmware configuration found and no module requirements"
log_info "Skipping firmware installation"
return 0
fi fi
# Read firmware packages from config (excluding comments and empty lines) # Trim whitespace
while IFS=: read -r package description; do package=$(echo "$package" | xargs)
# Skip comments and empty lines description=$(echo "$description" | xargs)
if [[ "$package" =~ ^[[:space:]]*# ]] || [[ -z "${package// }" ]]; then
continue
fi
# Trim whitespace if [[ -n "$package" ]]; then
package=$(echo "$package" | xargs) firmware_packages+=("$package")
description=$(echo "$description" | xargs) log_info " - ${package}: ${description}"
fi
if [[ -n "$package" ]]; then done < "$firmware_conf"
firmware_packages+=("$package")
log_info " - ${package}: ${description}"
fi
done < "$firmware_conf"
fi
if [[ ${#firmware_packages[@]} -eq 0 ]]; then if [[ ${#firmware_packages[@]} -eq 0 ]]; then
log_warn "No firmware packages to install" log_warn "No firmware packages to install"

View File

@@ -56,7 +56,7 @@ function docker_build_container() {
fi fi
log_info "Building container image: ${tag}" log_info "Building container image: ${tag}"
safe_execute ${CONTAINER_RUNTIME} build -t "${tag}" -f "${dockerfile_path}" "${PROJECT_ROOT}" ${CONTAINER_RUNTIME} build -t "${tag}" -f "${dockerfile_path}" "${PROJECT_ROOT}"
log_info "Container image built successfully: ${tag}" log_info "Container image built successfully: ${tag}"
} }

View File

@@ -250,16 +250,9 @@ function initramfs_setup_modules() {
local stage1_modules=() local stage1_modules=()
local stage1_firmware=() local stage1_firmware=()
while IFS=: read -r stage module firmware_line; do while IFS=: read -r stage module _ignored_fw; do
if [[ "$stage" == "stage1" && -n "$module" ]]; then if [[ "$stage" == "stage1" && -n "$module" ]]; then
stage1_modules+=("$module") stage1_modules+=("$module")
# Extract firmware package name (before any comment)
local firmware=$(echo "$firmware_line" | sed 's/[[:space:]]*#.*//' | tr -d ' ')
if [[ -n "$firmware" && "$firmware" != "none" ]]; then
stage1_firmware+=("$firmware")
required_firmware+=("$firmware")
log_debug "Module $module requires firmware: $firmware"
fi
fi fi
done < <(grep "^stage1:" "$modules_conf") done < <(grep "^stage1:" "$modules_conf")
@@ -286,32 +279,32 @@ function initramfs_setup_modules() {
local stage2_with_deps=() local stage2_with_deps=()
# Create firmware requirements list (remove duplicates) # Create firmware requirements list (remove duplicates)
local unique_firmware=($(printf '%s\n' "${required_firmware[@]}" | sort -u)) # Firmware selection policy: use config/firmware.conf as the authoritative source.
if [[ ${#unique_firmware[@]} -gt 0 ]]; then # Ignore firmware hints in modules.conf (third column) to avoid duplication/mismatch.
printf '%s\n' "${unique_firmware[@]}" > "${modules_dir}/required-firmware.list" if [[ -f "${PROJECT_ROOT}/config/firmware.conf" ]]; then
log_info "Created firmware requirements list: ${#unique_firmware[@]} packages" log_info "Firmware selection: using config/firmware.conf (ignoring modules.conf firmware hints)"
for fw in "${unique_firmware[@]}"; do
log_info " Required firmware: $fw"
done
# Export for use by firmware installation
export REQUIRED_FIRMWARE_PACKAGES="${unique_firmware[*]}"
else else
log_info "No firmware packages required" log_warn "Firmware selection: config/firmware.conf not found; no firmware will be installed into initramfs"
export REQUIRED_FIRMWARE_PACKAGES=""
fi fi
# Remove any stale list created by previous versions
rm -f "${modules_dir}/required-firmware.list" 2>/dev/null || true
# Do NOT export REQUIRED_FIRMWARE_PACKAGES anymore; alpine_install_firmware reads the config file directly.
# Create module loading scripts # Create module loading scripts
initramfs_create_module_scripts "$initramfs_dir" "$kernel_version" initramfs_create_module_scripts "$initramfs_dir" "$kernel_version"
# Report final counts # Report final counts
local stage1_count=${#stage1_with_deps[@]} local stage1_count=${#stage1_with_deps[@]}
local firmware_count=${#unique_firmware[@]} # Count firmware entries from authoritative firmware.conf (non-empty, non-comment lines)
local firmware_count=0
if [[ -f "${PROJECT_ROOT}/config/firmware.conf" ]]; then
firmware_count=$(grep -v '^[[:space:]]*#' "${PROJECT_ROOT}/config/firmware.conf" | sed '/^[[:space:]]*$/d' | wc -l | tr -d ' ')
fi
log_info "Module configuration complete:" log_info "Module configuration complete:"
log_info " Stage1 (critical + deps): ${stage1_count} modules" log_info " Stage1 (critical + deps): ${stage1_count} modules"
log_info " Stage2: disabled (only using stage1)" log_info " Stage2: disabled (only using stage1)"
log_info " Required firmware packages: ${firmware_count}" log_info " Firmware packages (from firmware.conf): ${firmware_count}"
log_info " Total modules: ${stage1_count}" log_info " Total modules: ${stage1_count}"
} }
@@ -631,6 +624,17 @@ EOF
log_info "Branding disabled: leaving /etc/motd, /etc/issue and root password unchanged" log_info "Branding disabled: leaving /etc/motd, /etc/issue and root password unchanged"
fi fi
# Write a unique build stamp to verify runtime image freshness
# Allows confirming that the booted VM uses the latest embedded initramfs
local _rand=""
if command_exists "openssl"; then _rand=$(openssl rand -hex 6 2>/dev/null || true); fi
local _build_id="ZOS-BUILD $(date -u +%Y-%m-%dT%H:%M:%SZ) ${_rand}"
# Write via tee to ensure creation + visible logging, avoid shell redirection edge cases
safe_mkdir "${initramfs_dir}/etc"
printf "%s\n" "${_build_id}" | tee "${initramfs_dir}/etc/zero-os-build-id" >/dev/null
safe_execute chmod 644 "${initramfs_dir}/etc/zero-os-build-id"
log_info "Build ID: ${_build_id} (written to /etc/zero-os-build-id)"
# Ensure essential Linux filesystem directories exist (defensive), including /home # Ensure essential Linux filesystem directories exist (defensive), including /home
# Some earlier stages or cleanups may have run in previous builds; enforce presence now. # Some earlier stages or cleanups may have run in previous builds; enforce presence now.
log_info "Ensuring essential directories exist (including /home)" log_info "Ensuring essential directories exist (including /home)"
@@ -1070,8 +1074,39 @@ function initramfs_copy_resolved_modules() {
for module in "${all_modules[@]}"; do for module in "${all_modules[@]}"; do
local module_found=false local module_found=false
# Try to find module file # Try to find module file with robust underscore/hyphen aliasing
local module_file=$(find "$container_modules_path" -name "${module}.ko*" -type f | head -1) # modprobe treats '_' and '-' as equivalent, but our file search must do both.
local query="${module}"
local module_file=""
# 1) Exact match (as given)
module_file=$(find "$container_modules_path" -name "${query}.ko*" -type f | head -1)
# 2) Fallback: underscores -> hyphens (e.g., xhci_pci -> xhci-pci)
if [[ -z "$module_file" || ! -f "$module_file" ]]; then
local alt="${query//_/-}"
if [[ "$alt" != "$query" ]]; then
module_file=$(find "$container_modules_path" -name "${alt}.ko*" -type f | head -1)
if [[ -n "$module_file" && -f "$module_file" ]]; then
log_info "Resolved module alias: ${query} -> ${alt} (hyphen)"
query="$alt"
fi
fi
fi
# 3) Fallback: hyphens -> underscores (defensive; modinfo sometimes reports hyphen)
if [[ -z "$module_file" || ! -f "$module_file" ]]; then
local alt2="${query//-/_}"
if [[ "$alt2" != "$query" ]]; then
local mf2
mf2=$(find "$container_modules_path" -name "${alt2}.ko*" -type f | head -1)
if [[ -n "$mf2" && -f "$mf2" ]]; then
log_info "Resolved module alias: ${query} -> ${alt2} (underscore)"
query="$alt2"
module_file="$mf2"
fi
fi
fi
if [[ -n "$module_file" && -f "$module_file" ]]; then if [[ -n "$module_file" && -f "$module_file" ]]; then
# Preserve directory structure # Preserve directory structure
@@ -1081,13 +1116,13 @@ function initramfs_copy_resolved_modules() {
safe_mkdir "$target_dir" safe_mkdir "$target_dir"
safe_execute cp "$module_file" "${initramfs_modules_dir}/${rel_path}" safe_execute cp "$module_file" "${initramfs_modules_dir}/${rel_path}"
log_debug "Copied module: $module from $rel_path" log_debug "Copied module: ${query} from ${rel_path}"
((copied_count++)) ((copied_count++))
module_found=true module_found=true
fi fi
if [[ "$module_found" == "false" ]]; then if [[ "$module_found" == "false" ]]; then
log_warn "Module not found in container: $module" log_warn "Module not found in container (after alias fallbacks): ${module}"
((failed_count++)) ((failed_count++))
fi fi
done done

256
scripts/make-grub-usb.sh Normal file
View File

@@ -0,0 +1,256 @@
#!/usr/bin/env bash
# Create a GRUB USB disk that boots on both BIOS (legacy) and UEFI.
# It copies kernel (vmlinuz) and initramfs, installs GRUB for i386-pc and x86_64-efi,
# and writes a minimal grub.cfg with optional kernel parameters.
#
# DANGEROUS: This script will repartition and format the target device (e.g., /dev/sdX).
#
# Usage:
# sudo scripts/make-grub-usb.sh /dev/sdX \
# --kernel dist/vmlinuz.efi \
# --initrd dist/initramfs.cpio.xz \
# --kparams "console=ttyS0 initdebug=true" \
# --label ZOSBOOT \
# --no-confirm
#
# Defaults (resolved relative to repo root):
# --kernel dist/vmlinuz.efi
# --initrd dist/initramfs.cpio.xz
# --label ZOS
# --kparams "" (none)
#
# Requirements on host:
# - grub-install (supports --target=i386-pc and --target=x86_64-efi)
# - parted, sfdisk (optional), dosfstools (mkfs.vfat), util-linux (lsblk, partprobe)
# - run as root
#
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
DEVICE="${1:-}"
shift || true
KERNEL="${PROJECT_ROOT}/dist/vmlinuz.efi"
# Default: no separate initrd because initramfs is embedded in kernel. Enable with --with-initrd or --initrd.
INITRD=""
KPARAMS=""
LABEL="ZOS"
NO_CONFIRM=0
ESP_SIZE_MB=512
# Request including separate initrd
WANT_INITRD=0
error() { echo "[ERROR] $*" >&2; }
info() { echo "[INFO] $*"; }
warn() { echo "[WARN] $*"; }
die() { error "$*"; exit 1; }
require_root() {
[[ "$EUID" -eq 0 ]] || die "This script must be run as root."
}
command_exists() { command -v "$1" &>/dev/null; }
parse_args() {
while [[ $# -gt 0 ]]; do
case "$1" in
--kernel)
KERNEL="$(realpath -m "${2:-}")"; shift 2 ;;
--initrd)
INITRD="$(realpath -m "${2:-}")"; shift 2 ;;
--with-initrd)
WANT_INITRD=1; shift ;;
--kparams)
KPARAMS="${2:-}"; shift 2 ;;
--label)
LABEL="${2:-ZOS}"; shift 2 ;;
--no-confirm)
NO_CONFIRM=1; shift ;;
--esp-size-mb)
ESP_SIZE_MB="${2:-512}"; shift 2 ;;
-*)
die "Unknown option: $1" ;;
*)
die "Unexpected argument: $1" ;;
esac
done
}
confirm_dangerous() {
info "About to wipe and repartition device: ${DEVICE}"
lsblk -o NAME,SIZE,TYPE,MOUNTPOINT "${DEVICE}"
if [[ $NO_CONFIRM -eq 1 ]]; then
info "--no-confirm provided; proceeding without interactive confirmation"
return
fi
echo
read -r -p "Type the DEVICE path to CONFIRM (e.g., ${DEVICE}) or 'abort' to cancel: " ans
[[ "$ans" == "$DEVICE" ]] || die "Confirmation mismatch or aborted."
}
check_prereqs() {
command_exists parted || die "parted not found"
command_exists mkfs.vfat || die "mkfs.vfat (dosfstools) not found"
command_exists grub-install || die "grub-install not found"
command_exists lsblk || die "lsblk not found"
command_exists partprobe || die "partprobe not found"
}
resolve_defaults() {
[[ -b "$DEVICE" ]] || die "Device not found or not a block device: ${DEVICE}"
[[ -f "$KERNEL" ]] || die "Kernel file not found: ${KERNEL}"
if [[ $WANT_INITRD -eq 1 ]]; then
# If user asked for separate initrd but none specified, use default path
if [[ -z "${INITRD}" ]]; then
INITRD="${PROJECT_ROOT}/dist/initramfs.cpio.xz"
fi
[[ -f "$INITRD" ]] || die "Requested --with-initrd but initramfs not found: ${INITRD}"
else
# By default, no separate initrd (initramfs is embedded in kernel)
INITRD=""
fi
}
umount_partitions() {
info "Unmounting any mounted partitions of ${DEVICE}"
local kids
kids="$(lsblk -nr -o NAME "${DEVICE}" | tail -n +2 || true)"
if [[ -n "${kids}" ]]; then
while read -r name; do
[[ -z "$name" ]] && continue
local path="/dev/${name}"
if mountpoint -q -- "/dev/${name}" 2>/dev/null; then
umount -f "/dev/${name}" || true
fi
# also try mounts by path
local mp
mp="$(lsblk -nr -o MOUNTPOINT "$path" || true)"
if [[ -n "$mp" ]]; then
for m in $mp; do
[[ -n "$m" ]] && umount -f "$m" || true
done
fi
done <<< "${kids}"
fi
}
partition_device_gpt() {
info "Creating GPT with BIOS boot + ESP on ${DEVICE}"
# Create a new GPT
parted -s "${DEVICE}" mklabel gpt
# Create BIOS boot partition 1MiB..3MiB
parted -s "${DEVICE}" mkpart biosgrub 1MiB 3MiB
parted -s "${DEVICE}" set 1 bios_grub on
# Create ESP (FAT32) from 3MiB to 3MiB + ESP_SIZE_MB
local esp_end="$((3 + ESP_SIZE_MB))MiB"
parted -s "${DEVICE}" mkpart esp fat32 3MiB "${esp_end}"
parted -s "${DEVICE}" set 2 esp on
# Inform kernel
partprobe "${DEVICE}"
sleep 1
}
format_esp() {
local part="${DEVICE}2"
[[ -b "$part" ]] || die "ESP partition not found: ${part}"
info "Formatting ESP ${part} as FAT32 (label=${LABEL})"
mkfs.vfat -F32 -n "${LABEL}" "$part"
}
mount_esp() {
ESP_MNT="$(mktemp -d)"
info "Mounting ESP at ${ESP_MNT}"
mount "${DEVICE}2" "${ESP_MNT}"
}
install_grub() {
info "Installing GRUB (BIOS i386-pc) to ${DEVICE}"
grub-install --target=i386-pc --boot-directory="${ESP_MNT}/boot" --recheck "${DEVICE}"
info "Installing GRUB (UEFI x86_64-efi) to ESP"
mkdir -p "${ESP_MNT}/EFI/BOOT"
grub-install --target=x86_64-efi --efi-directory="${ESP_MNT}" --boot-directory="${ESP_MNT}/boot" --removable --recheck
}
copy_kernel_initrd() {
mkdir -p "${ESP_MNT}/boot"
info "Copying kernel to ESP /boot/vmlinuz"
cp -f "${KERNEL}" "${ESP_MNT}/boot/vmlinuz"
if [[ -n "${INITRD}" ]]; then
info "Copying initramfs to ESP /boot/initramfs.cpio.xz"
cp -f "${INITRD}" "${ESP_MNT}/boot/initramfs.cpio.xz"
fi
}
write_grub_cfg() {
local cfg="${ESP_MNT}/boot/grub/grub.cfg"
info "Writing GRUB config: ${cfg}"
mkdir -p "$(dirname "$cfg")"
cat > "$cfg" <<'EOF'
set default=0
set timeout=3
if [ "${grub_platform}" = "efi" ]; then
set gfxpayload=keep
fi
menuentry "Zero-OS" {
linux /boot/vmlinuz __KPARAMS__
__INITRD_LINE__
}
EOF
local initrd_line=""
if [[ -n "${INITRD}" ]]; then
initrd_line=" initrd /boot/initramfs.cpio.xz"
fi
# Substitute placeholders
sed -i \
-e "s|__KPARAMS__|${KPARAMS}|g" \
-e "s|__INITRD_LINE__|${initrd_line}|g" \
"$cfg"
# For removable media on UEFI, ensure fallback bootloader path exists
if [[ -f "/usr/lib/grub/x86_64-efi/monolithic/grubx64.efi" ]]; then
cp -f "/usr/lib/grub/x86_64-efi/monolithic/grubx64.efi" "${ESP_MNT}/EFI/BOOT/BOOTX64.EFI" || true
elif [[ -f "${ESP_MNT}/EFI/BOOT/grubx64.efi" ]]; then
cp -f "${ESP_MNT}/EFI/BOOT/grubx64.efi" "${ESP_MNT}/EFI/BOOT/BOOTX64.EFI" || true
fi
}
cleanup() {
set +e
if mountpoint -q "${ESP_MNT}" 2>/dev/null; then
info "Unmounting ESP ${ESP_MNT}"
umount "${ESP_MNT}"
fi
[[ -n "${ESP_MNT:-}" ]] && rmdir "${ESP_MNT}" 2>/dev/null || true
}
main() {
require_root
[[ -n "${DEVICE}" ]] || die "Usage: $0 /dev/sdX [--kernel path] [--initrd path] [--kparams \"...\"] [--label ZOS] [--no-confirm]"
parse_args "$@"
resolve_defaults
check_prereqs
confirm_dangerous
trap cleanup EXIT INT TERM
umount_partitions
partition_device_gpt
format_esp
mount_esp
install_grub
copy_kernel_initrd
write_grub_cfg
info "Done. You can now boot from the USB on BIOS and UEFI systems."
}
main "$@"

View File

@@ -5,6 +5,7 @@
# scripts/rebuild-after-zinit.sh --run-tests # include boot tests (still no kernel rebuild by default) # scripts/rebuild-after-zinit.sh --run-tests # include boot tests (still no kernel rebuild by default)
# scripts/rebuild-after-zinit.sh --with-kernel # also rebuild kernel (re-embed updated initramfs) # scripts/rebuild-after-zinit.sh --with-kernel # also rebuild kernel (re-embed updated initramfs)
# scripts/rebuild-after-zinit.sh --refresh-container-mods # rebuild container /lib/modules if missing (kernel modules stage) # scripts/rebuild-after-zinit.sh --refresh-container-mods # rebuild container /lib/modules if missing (kernel modules stage)
# scripts/rebuild-after-zinit.sh --verify-only # only report detected changes, do not rebuild
# scripts/rebuild-after-zinit.sh -- ... # pass extra args to build.sh # scripts/rebuild-after-zinit.sh -- ... # pass extra args to build.sh
set -euo pipefail set -euo pipefail
@@ -19,6 +20,7 @@ run_tests=0
extra_args=() extra_args=()
rebuild_kernel=0 rebuild_kernel=0
refresh_container_mods=0 refresh_container_mods=0
verify_only=0
# Parse flags; pass through any remaining args to build.sh after -- # Parse flags; pass through any remaining args to build.sh after --
while [[ $# -gt 0 ]]; do while [[ $# -gt 0 ]]; do
@@ -35,6 +37,10 @@ while [[ $# -gt 0 ]]; do
refresh_container_mods=1 refresh_container_mods=1
shift shift
;; ;;
--verify-only)
verify_only=1
shift
;;
--) --)
shift shift
extra_args=("$@") extra_args=("$@")
@@ -51,6 +57,108 @@ done
log "PROJECT_ROOT=${PROJECT_ROOT}" log "PROJECT_ROOT=${PROJECT_ROOT}"
log "STAGES_DIR=${STAGES_DIR}" log "STAGES_DIR=${STAGES_DIR}"
# Show current stage status before any changes (host-safe; does not require container)
log "Stage status (before):"
("${PROJECT_ROOT}/scripts/build.sh" --show-stages) || true
# ------------------------------------------------------------
# Container detection helper
# ------------------------------------------------------------
in_container() {
[[ -f /.dockerenv ]] || [[ -f /run/.containerenv ]] || grep -q 'container' /proc/1/cgroup 2>/dev/null
}
# ------------------------------------------------------------
# Change detection (verify what changed since last archive build)
# ------------------------------------------------------------
marker_init="${STAGES_DIR}/initramfs_create.done"
marker_time=0
if [[ -f "$marker_init" ]]; then
marker_time=$(stat -c %Y "$marker_init" 2>/dev/null || echo 0)
fi
log "Detecting changes since last initramfs_create marker: ${marker_init:-<none>}"
check_dir_changed() {
local path="$1"
local cutoff="$2"
local count
count=$(find "$path" -type f -printf '%T@ %p\n' 2>/dev/null | awk -v c="$cutoff" '$1 > c {n++} END {print n+0}')
echo "${count:-0}"
}
list_some_changes() {
local path="$1"
local cutoff="$2"
# list up to 5 example files
find "$path" -type f -printf '%T@ %p\n' 2>/dev/null | awk -v c="$cutoff" '$1 > c {print $2}' | head -n 5
}
zinit_dir="${PROJECT_ROOT}/config/zinit"
init_file="${PROJECT_ROOT}/config/init"
modules_conf="${PROJECT_ROOT}/config/modules.conf"
zinit_changed=0
init_changed=0
modules_changed=0
if [[ -d "$zinit_dir" ]]; then
zinit_changed=$(check_dir_changed "$zinit_dir" "$marker_time")
fi
if [[ -f "$init_file" ]]; then
if [[ $(stat -c %Y "$init_file" 2>/dev/null || echo 0) -gt $marker_time ]]; then init_changed=1; fi
fi
if [[ -f "$modules_conf" ]]; then
if [[ $(stat -c %Y "$modules_conf" 2>/dev/null || echo 0) -gt $marker_time ]]; then modules_changed=1; fi
fi
log "Changes since last archive:"
log " - config/zinit: ${zinit_changed} file(s) changed"
if [[ "$zinit_changed" -gt 0 ]]; then
list_some_changes "$zinit_dir" "$marker_time" | sed 's/^/ * /' || true
fi
log " - config/init: $([[ $init_changed -eq 1 ]] && echo changed || echo unchanged)"
log " - config/modules.conf: $([[ $modules_changed -eq 1 ]] && echo changed || echo unchanged)"
if [[ "$verify_only" -eq 1 ]]; then
log "verify-only set; exiting without rebuild"
exit 0
fi
# ------------------------------------------------------------
# Container /lib/modules/<FULL_VERSION> presence diagnostics
# (we never clear kernel_modules unless --refresh-container-mods is given)
# ------------------------------------------------------------
compute_full_kver() {
# Read from configs without sourcing (safe in any shell)
local build_conf="${PROJECT_ROOT}/config/build.conf"
local kcfg="${PROJECT_ROOT}/config/kernel.config"
local base_ver=""
local localver=""
if [[ -f "$build_conf" ]]; then
base_ver="$(grep -E '^KERNEL_VERSION=' "$build_conf" | head -1 | cut -d= -f2 | tr -d '\"')"
fi
if [[ -f "$kcfg" ]]; then
localver="$(grep -E '^CONFIG_LOCALVERSION=' "$kcfg" | head -1 | cut -d'\"' -f2)"
fi
echo "${base_ver}${localver}"
}
modules_dir_for_full() {
local full="$1"
echo "/lib/modules/${full}"
}
full_kver="$(compute_full_kver)"
container_modules_dir="$(modules_dir_for_full "$full_kver")"
log "Container modules version: ${full_kver:-<unknown>}"
if [[ -d "$container_modules_dir" ]]; then
before_count=$(find "$container_modules_dir" -type f -name '*.ko*' 2>/dev/null | wc -l | tr -d ' ')
log "Before build: ${container_modules_dir} exists with ${before_count} module file(s)"
else
log "Before build: ${container_modules_dir} not present (fresh container scenario)"
fi
# Minimal set of stages to clear when zinit changes: # Minimal set of stages to clear when zinit changes:
# - zinit_setup: recopy zinit YAML and init scripts into initramfs # - zinit_setup: recopy zinit YAML and init scripts into initramfs
# - validation: re-check initramfs contents # - validation: re-check initramfs contents
@@ -87,6 +195,7 @@ if [[ "$rebuild_kernel" -eq 1 ]]; then
fi fi
# Remove completion markers to force incremental rebuild of those stages # Remove completion markers to force incremental rebuild of those stages
log "Planned markers to clear: ${stages_to_clear[*]}"
for s in "${stages_to_clear[@]}"; do for s in "${stages_to_clear[@]}"; do
marker="${STAGES_DIR}/${s}.done" marker="${STAGES_DIR}/${s}.done"
if [[ -f "$marker" ]]; then if [[ -f "$marker" ]]; then
@@ -97,20 +206,39 @@ for s in "${stages_to_clear[@]}"; do
fi fi
done done
# Show stage status after marker removal (still host-safe)
log "Stage status (after marker removal):"
("${PROJECT_ROOT}/scripts/build.sh" --show-stages) || true
# Build # Build
log "Starting incremental rebuild (zinit changes)" log "Starting incremental rebuild (zinit changes)"
# If we plan to rebuild the kernel, force the pipeline to run from initramfs_create # IMPORTANT: Do NOT pass --rebuild-from or --force-rebuild; that would force ALL stages to run.
# so the cpio archive is recreated before kernel_build (ignoring prior .done markers). # We rely exclusively on removed markers to minimally re-run only the necessary stages.
build_from_args=() build_from_args=()
if [[ "$rebuild_kernel" -eq 1 ]]; then
build_from_args=(--rebuild-from=initramfs_create)
log "Rebuild-from: initramfs_create (guarantee cpio is recreated before kernel_build)"
fi
if [[ "$run_tests" -eq 1 ]]; then if in_container; then
log "Including boot tests" # Run directly when already inside the dev/build container
DEBUG=1 "${PROJECT_ROOT}/scripts/build.sh" "${build_from_args[@]}" "${extra_args[@]}" if [[ "$run_tests" -eq 1 ]]; then
log "Including boot tests (in-container)"
DEBUG=1 "${PROJECT_ROOT}/scripts/build.sh" "${build_from_args[@]}" "${extra_args[@]}"
else
log "Skipping boot tests (in-container)"
DEBUG=1 "${PROJECT_ROOT}/scripts/build.sh" --skip-tests "${build_from_args[@]}" "${extra_args[@]}"
fi
else else
log "Skipping boot tests (use --run-tests to include)" # Not in container: delegate to dev-container manager which ensures container exists and is running
DEBUG=1 "${PROJECT_ROOT}/scripts/build.sh" --skip-tests "${build_from_args[@]}" "${extra_args[@]}" devctl="${PROJECT_ROOT}/scripts/dev-container.sh"
if [[ ! -x "$devctl" ]]; then
log "[ERROR] Dev container manager not found: ${devctl}"
log "[HINT] Run ./scripts/build.sh directly (it can start a transient container), or start the dev container via ./scripts/dev-container.sh start"
exit 1
fi
if [[ "$run_tests" -eq 1 ]]; then
log "Including boot tests via dev-container"
"$devctl" build "${build_from_args[@]}" "${extra_args[@]}"
else
log "Skipping boot tests via dev-container"
"$devctl" build --skip-tests "${build_from_args[@]}" "${extra_args[@]}"
fi
fi fi