forked from tfgrid/zosbuilder
initramfs+modules: robust copy aliasing, curated stage1 + PHYs, firmware policy via firmware.conf, runtime readiness, build ID; docs sync
Summary of changes (with references):\n\nModules + PHY coverage\n- Curated and normalized stage1 list in [config.modules.conf](config/modules.conf:1):\n - Boot-critical storage, core virtio, common NICs (Intel/Realtek/Broadcom), overlay/fuse, USB HCD/HID.\n - Added PHY drivers required by NIC MACs:\n * realtek (for r8169, etc.)\n * broadcom families: broadcom, bcm7xxx, bcm87xx, bcm_phy_lib, bcm_phy_ptp\n- Robust underscore↔hyphen aliasing during copy so e.g. xhci_pci → xhci-pci.ko, hid_generic → hid-generic.ko:\n - [bash.initramfs_copy_resolved_modules()](scripts/lib/initramfs.sh:990)\n\nFirmware policy and coverage\n- Firmware selection now authoritative via [config/firmware.conf](config/firmware.conf:1); ignore modules.conf firmware hints:\n - [bash.initramfs_setup_modules()](scripts/lib/initramfs.sh:229)\n - Count from firmware.conf for reporting; remove stale required-firmware.list.\n- Expanded NIC firmware set (bnx2, bnx2x, tigon, intel, realtek, rtl_nic, qlogic, e100) in [config.firmware.conf](config/firmware.conf:1).\n- Installer enforces firmware.conf source-of-truth in [bash.alpine_install_firmware()](scripts/lib/alpine.sh:392).\n\nEarly input & build freshness\n- Write a runtime build stamp to /etc/zero-os-build-id for embedded initramfs verification:\n - [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:568)\n- Minor init refinements in [config.init](config/init:1) (ensures /home, consistent depmod path).\n\nRebuild helper improvements\n- [scripts/rebuild-after-zinit.sh](scripts/rebuild-after-zinit.sh:1):\n - Added --verify-only; container-aware execution; selective marker clears only.\n - Prints stage status before/after; avoids --rebuild-from; resolves full kernel version for diagnostics.\n\nRemote flist readiness + zinit\n- Init scripts now probe BASE_URL readiness and accept FLISTS_BASE_URL/FLIST_BASE_URL; firmware target is /lib/firmware:\n - [sh.firmware.sh](config/zinit/init/firmware.sh:1)\n - [sh.modules.sh](config/zinit/init/modules.sh:1)\n\nContainer, docs, and utilities\n- Stream container build logs by calling runtime build directly in [bash.docker_build_container()](scripts/lib/docker.sh:56).\n- Docs updated to reflect firmware policy, runtime readiness, rebuild helper, early input, and GRUB USB:\n - [docs.NOTES.md](docs/NOTES.md)\n - [docs.PROMPT.md](docs/PROMPT.md)\n - [docs.review-rfs-integration.md](docs/review-rfs-integration.md)\n- Added GRUB USB creator (referenced in docs): [scripts/make-grub-usb.sh](scripts/make-grub-usb.sh)\n\nCleanup\n- Removed legacy/duplicated config trees under configs/ and config/zinit.old/.\n- Minor newline and ignore fixes: [.gitignore](.gitignore:1)\n\nNet effect\n- Runtime now has correct USB HCDs/HID-generic and NIC+PHY coverage (Realtek/Broadcom), with matching firmware installed in initramfs.\n- Rebuild workflow is minimal and host/container-aware; docs are aligned with implemented behavior.\n
This commit is contained in:
@@ -34,6 +34,8 @@ Repository map (jump-points)
|
||||
- RFS flists tooling:
|
||||
- Modules packer: [bash.pack-modules.sh](scripts/rfs/pack-modules.sh:1)
|
||||
- Firmware packer: [bash.pack-firmware.sh](scripts/rfs/pack-firmware.sh:1)
|
||||
- Boot media utility:
|
||||
- GRUB USB creator: [scripts/make-grub-usb.sh](scripts/make-grub-usb.sh)
|
||||
|
||||
High-priority behaviors and policies
|
||||
1) Branding passwordless root (shadow-aware)
|
||||
@@ -53,18 +55,20 @@ High-priority behaviors and policies
|
||||
- Pre-CPIO essential check includes “home”:
|
||||
- [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:680)
|
||||
|
||||
4) Remote flist fallback (modules + firmware)
|
||||
4) Remote flist fallback + readiness (modules + firmware)
|
||||
- When local manifests are missing, fetch from zos.grid.tf and mount via rfs:
|
||||
- Firmware fallback: [sh.firmware.sh](config/zinit/init/firmware.sh:1)
|
||||
- Default BASE_URL: https://zos.grid.tf/store/flists
|
||||
- Fetch path: ${BASE_URL}/firmware-latest.fl to /etc/rfs/firmware-latest.fl
|
||||
- Modules fallback: [sh.modules.sh](config/zinit/init/modules.sh:1)
|
||||
- Fetch path: ${BASE_URL}/modules-$(uname -r)-Zero-OS.fl to /etc/rfs/modules-$(uname -r).fl
|
||||
- Env overrides:
|
||||
- FIRMWARE_FLIST, MODULES_FLIST: use local file if provided
|
||||
- RFS_BIN: defaults to rfs
|
||||
- FLISTS_BASE_URL: overrides base URL
|
||||
- wget is available (initramfs includes it); scripts prefer wget, fallback to busybox wget if needed.
|
||||
- Firmware: [sh.firmware.sh](config/zinit/init/firmware.sh:1)
|
||||
- BASE_URL from FLISTS_BASE_URL (or FLIST_BASE_URL alias), default https://zos.grid.tf/store/flists
|
||||
- Probes BASE_URL for HTTP(S) readiness (wget --spider) before fetching firmware-latest.fl
|
||||
- Fetch path: ${BASE_URL%/}/firmware-latest.fl to /etc/rfs/firmware-latest.fl
|
||||
- Modules: [sh.modules.sh](config/zinit/init/modules.sh:1)
|
||||
- BASE_URL from FLISTS_BASE_URL (or FLIST_BASE_URL alias)
|
||||
- Probes BASE_URL for HTTP(S) readiness before fetching modules-$(uname -r)-Zero-OS.fl
|
||||
- Env overrides:
|
||||
- FIRMWARE_FLIST, MODULES_FLIST: use local file if provided
|
||||
- RFS_BIN: defaults to rfs
|
||||
- FLISTS_BASE_URL or FLIST_BASE_URL: override base URL
|
||||
- wget is available (initramfs includes it); scripts prefer wget, fallback to busybox wget if needed.
|
||||
|
||||
5) Incremental build guards
|
||||
- Kernel build now defaults INITRAMFS_ARCHIVE if unset (fix for unbound var on incremental runs):
|
||||
@@ -72,6 +76,12 @@ High-priority behaviors and policies
|
||||
- Initramfs test stage already guards INITRAMFS_ARCHIVE:
|
||||
- [bash.stage_initramfs_test()](scripts/build.sh:385)
|
||||
|
||||
6) Early keyboard input and debug shell
|
||||
- Early HID/input and USB HCD modules are preloaded before zinit to ensure console usability:
|
||||
- [config.init](config/init:80)
|
||||
- Debug hook: kernel cmdline initdebug=true runs /init-debug if present or drops to a shell:
|
||||
- [config.init](config/init:115)
|
||||
|
||||
Flags and config
|
||||
- Config file: [config/build.conf](config/build.conf)
|
||||
- Branding flags:
|
||||
@@ -85,6 +95,9 @@ Flags and config
|
||||
- COMPONENTS_DIR="components"
|
||||
- KERNEL_DIR="kernel"
|
||||
- DIST_DIR="dist"
|
||||
- Firmware policies:
|
||||
- Initramfs: [config/firmware.conf](config/firmware.conf) is authoritative; modules.conf firmware hints are ignored.
|
||||
- RFS: full Alpine firmware set is installed into container and packed from /lib/firmware (see [bash.pack-firmware.sh](scripts/rfs/pack-firmware.sh:1)).
|
||||
- Firmware flist naming tag:
|
||||
- FIRMWARE_TAG (env > config > “latest”)
|
||||
- Container image tools (podman rootless OK) defined by [Dockerfile](Dockerfile):
|
||||
@@ -113,7 +126,7 @@ Common tasks and commands
|
||||
- Show stage status:
|
||||
- ./scripts/build.sh --show-stages
|
||||
|
||||
Checklists
|
||||
Checklists and helpers
|
||||
|
||||
A) Diagnose “passwordless root not working”
|
||||
- Confirm branding flags are loaded:
|
||||
@@ -130,17 +143,34 @@ B) Fix “Initramfs directory not found: initramfs (resolved: /workspace/kernel/
|
||||
- Confirm validation prints “Validation debug:” with resolved absolute path:
|
||||
- [bash.initramfs_validate()](scripts/lib/initramfs.sh:799)
|
||||
|
||||
C) INITRAMFS_ARCHIVE unbound during kernel build stage
|
||||
C) Minimal rebuild after zinit/init/modules.conf changes
|
||||
- Use the helper (works from host or container):
|
||||
- scripts/rebuild-after-zinit.sh
|
||||
- Defaults: initramfs-only; clears only modules_setup, modules_copy, init_script, zinit_setup, validation, initramfs_create, initramfs_test
|
||||
- Flags:
|
||||
- --with-kernel: also rebuild kernel; cpio is recreated immediately before embedding
|
||||
- --refresh-container-mods: rebuild container /lib/modules for fresh containers
|
||||
- --verify-only: report changed files and stage status; no rebuild
|
||||
- Stage status is printed before/after marker removal; the helper avoids --rebuild-from by default to prevent running early stages.
|
||||
- Manual fallback:
|
||||
- rm -f .build-stages/initramfs_create.done .build-stages/initramfs_test.done .build-stages/validation.done
|
||||
- DEBUG=1 ./scripts/build.sh --skip-tests
|
||||
|
||||
D) INITRAMFS_ARCHIVE unbound during kernel build stage
|
||||
- stage_kernel_build now defaults INITRAMFS_ARCHIVE if unset:
|
||||
- [bash.stage_kernel_build()](scripts/build.sh:398)
|
||||
- If error persists, ensure stage_initramfs_create ran or that defaulting logic sees dist/initramfs.cpio.xz.
|
||||
|
||||
D) Modules/firmware not found by rfs init scripts
|
||||
- Confirm local manifests under /etc/rfs or remote fallback working:
|
||||
E) Modules/firmware not found by rfs init scripts
|
||||
- Confirm local manifests under /etc/rfs or remote fallback:
|
||||
- Firmware: [sh.firmware.sh](config/zinit/init/firmware.sh:1)
|
||||
- Modules: [sh.modules.sh](config/zinit/init/modules.sh:1)
|
||||
- For remote:
|
||||
- Set FLISTS_BASE_URL or FLIST_BASE_URL; default is https://zos.grid.tf/store/flists
|
||||
- Scripts probe BASE_URL readiness (wget --spider) before fetch
|
||||
- Firmware target is /lib/firmware; modules target is /lib/modules/$(uname -r)
|
||||
- Confirm uname -r matches remote naming “modules-$(uname -r)-Zero-OS.fl”
|
||||
- Confirm wget present (it is in initramfs), or busybox fallback.
|
||||
- Confirm wget present (or busybox wget)
|
||||
|
||||
Project conventions
|
||||
- Edit policy:
|
||||
@@ -157,8 +187,9 @@ Key files to keep in sync with behavior decisions
|
||||
- Validation diagnostics: [bash.initramfs_validate()](scripts/lib/initramfs.sh:799)
|
||||
- Archive creation (pre-CPIO checks): [bash.initramfs_create_cpio()](scripts/lib/initramfs.sh:688)
|
||||
- Path normalization after config: [bash.common.sh](scripts/lib/common.sh:236)
|
||||
- Modules/firmware remote fallback: [sh.modules.sh](config/zinit/init/modules.sh:1), [sh.firmware.sh](config/zinit/init/firmware.sh:1)
|
||||
- Modules/firmware remote fallback + readiness: [sh.modules.sh](config/zinit/init/modules.sh:1), [sh.firmware.sh](config/zinit/init/firmware.sh:1)
|
||||
- Kernel stage defaulting for archive: [bash.stage_kernel_build()](scripts/build.sh:398)
|
||||
- GRUB USB creator: [scripts/make-grub-usb.sh](scripts/make-grub-usb.sh)
|
||||
- Operational notes: [docs/NOTES.md](docs/NOTES.md)
|
||||
|
||||
When in doubt
|
||||
|
||||
Reference in New Issue
Block a user