forked from tfgrid/zosbuilder
334821dacf93481c22d039bdbd7a14124c0f354c
2 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
| ad0a06e267 |
initramfs+modules: robust copy aliasing, curated stage1 + PHYs, firmware policy via firmware.conf, runtime readiness, build ID; docs sync
Summary of changes (with references):\n\nModules + PHY coverage\n- Curated and normalized stage1 list in [config.modules.conf](config/modules.conf:1):\n - Boot-critical storage, core virtio, common NICs (Intel/Realtek/Broadcom), overlay/fuse, USB HCD/HID.\n - Added PHY drivers required by NIC MACs:\n * realtek (for r8169, etc.)\n * broadcom families: broadcom, bcm7xxx, bcm87xx, bcm_phy_lib, bcm_phy_ptp\n- Robust underscore↔hyphen aliasing during copy so e.g. xhci_pci → xhci-pci.ko, hid_generic → hid-generic.ko:\n - [bash.initramfs_copy_resolved_modules()](scripts/lib/initramfs.sh:990)\n\nFirmware policy and coverage\n- Firmware selection now authoritative via [config/firmware.conf](config/firmware.conf:1); ignore modules.conf firmware hints:\n - [bash.initramfs_setup_modules()](scripts/lib/initramfs.sh:229)\n - Count from firmware.conf for reporting; remove stale required-firmware.list.\n- Expanded NIC firmware set (bnx2, bnx2x, tigon, intel, realtek, rtl_nic, qlogic, e100) in [config.firmware.conf](config/firmware.conf:1).\n- Installer enforces firmware.conf source-of-truth in [bash.alpine_install_firmware()](scripts/lib/alpine.sh:392).\n\nEarly input & build freshness\n- Write a runtime build stamp to /etc/zero-os-build-id for embedded initramfs verification:\n - [bash.initramfs_finalize_customization()](scripts/lib/initramfs.sh:568)\n- Minor init refinements in [config.init](config/init:1) (ensures /home, consistent depmod path).\n\nRebuild helper improvements\n- [scripts/rebuild-after-zinit.sh](scripts/rebuild-after-zinit.sh:1):\n - Added --verify-only; container-aware execution; selective marker clears only.\n - Prints stage status before/after; avoids --rebuild-from; resolves full kernel version for diagnostics.\n\nRemote flist readiness + zinit\n- Init scripts now probe BASE_URL readiness and accept FLISTS_BASE_URL/FLIST_BASE_URL; firmware target is /lib/firmware:\n - [sh.firmware.sh](config/zinit/init/firmware.sh:1)\n - [sh.modules.sh](config/zinit/init/modules.sh:1)\n\nContainer, docs, and utilities\n- Stream container build logs by calling runtime build directly in [bash.docker_build_container()](scripts/lib/docker.sh:56).\n- Docs updated to reflect firmware policy, runtime readiness, rebuild helper, early input, and GRUB USB:\n - [docs.NOTES.md](docs/NOTES.md)\n - [docs.PROMPT.md](docs/PROMPT.md)\n - [docs.review-rfs-integration.md](docs/review-rfs-integration.md)\n- Added GRUB USB creator (referenced in docs): [scripts/make-grub-usb.sh](scripts/make-grub-usb.sh)\n\nCleanup\n- Removed legacy/duplicated config trees under configs/ and config/zinit.old/.\n- Minor newline and ignore fixes: [.gitignore](.gitignore:1)\n\nNet effect\n- Runtime now has correct USB HCDs/HID-generic and NIC+PHY coverage (Realtek/Broadcom), with matching firmware installed in initramfs.\n- Rebuild workflow is minimal and host/container-aware; docs are aligned with implemented behavior.\n |
|||
| afd4f4c6f9 |
feat(rfs): flist pack to S3 + read-only route embedding + zinit mount scripts; docs; dev-container tooling
Summary
- Implemented plain S3-only flist workflow (no web endpoint). rfs pack uploads blobs using write creds; flist route.url is patched to embed read-only S3 credentials so rfs mount reads directly from S3.
Changes
1) New RFS tooling (scripts/rfs/)
- common.sh:
- Compute FULL_KERNEL_VERSION from configs (no uname).
- Load S3 config and construct pack store URI.
- Build read-only S3 route URL and patch flist (sqlite).
- Helpers to locate modules/firmware trees and rfs binary.
- pack-modules.sh:
- Pack /lib/modules/<FULL_KERNEL_VERSION> to dist/flists/modules-<FULL_KERNEL_VERSION>.fl
- Patch flist route to s3://READ:READ@host:port/ROUTE_PATH?region=ROUTE_REGION (default /blobs, garage).
- Optional upload of .fl using MinIO client (mcli/mc).
- pack-firmware.sh:
- Source firmware from $PROJECT_ROOT/firmware (fallback to initramfs/lib/firmware).
- Pack to dist/flists/firmware-<TAG_OR_DATE>.fl (FIRMWARE_TAG or YYYYMMDD).
- Patch flist route to read-only S3; optional .fl upload via mcli/mc.
- verify-flist.sh:
- rfs flist inspect/tree; optional mount test (best effort).
- patch-stores.sh:
- Helper to patch stores (kept though not used by default).
2) Dev-container (Dockerfile)
- Added sqlite and MinIO client package for manifest patching/upload (expect mcli binary at runtime; scripts support both mcli/mc).
- Retains rustup and musl target for building rfs/zinit/mycelium.
3) Config and examples
- config/rfs.conf.example:
- S3_ENDPOINT/S3_REGION/S3_BUCKET/S3_PREFIX
- S3_ACCESS_KEY/S3_SECRET_KEY (write)
- READ_ACCESS_KEY/READ_SECRET_KEY (read-only)
- ROUTE_ENDPOINT (defaults to S3_ENDPOINT), ROUTE_PATH=/blobs, ROUTE_REGION=garage
- MANIFESTS_SUBPATH, UPLOAD_MANIFESTS (mcli upload optional)
- config/rfs.conf updated by user with real values (not committed here; example included).
- config/modules.conf minor tweak (staged).
4) Zinit mount scripts (config/zinit/init/)
- firmware.sh:
- Mounts firmware-latest.fl over /usr/lib/firmware using rfs mount (env override FIRMWARE_FLIST supported).
- modules.sh:
- Mounts modules-$(uname -r).fl over /lib/modules/$(uname -r) (env override MODULES_FLIST supported).
- Both skip if target already mounted and respect RFS_BIN env.
5) Documentation
- docs/rfs-flists.md:
- End-to-end flow, S3-only route URL patching, mcli upload notes.
- docs/review-rfs-integration.md:
- Integration points, build flow, and post-build standalone usage.
- docs/depmod-behavior.md:
- depmod reads .modinfo; recommend prebuilt modules.*(.bin); use depmod -A only on mismatch.
6) Utility
- scripts/functionlist.md synced with current functions.
Behavioral details
- Pack (write):
s3://S3_ACCESS_KEY:S3_SECRET_KEY@HOST:PORT/S3_BUCKET/S3_PREFIX?region=REGION
- Flist route (read, post-patch):
s3://READ_ACCESS_KEY:READ_SECRET_KEY@HOST:PORT/ROUTE_PATH?region=ROUTE_REGION
Defaults: ROUTE_PATH=/blobs, ROUTE_REGION=garage; ROUTE_ENDPOINT derived from S3_ENDPOINT if not set.
Runtime mount examples
- Modules:
rfs mount -m dist/flists/modules-6.12.44-Zero-OS.fl /lib/modules/6.12.44-Zero-OS
- Firmware:
rfs mount -m dist/flists/firmware-YYYYMMDD.fl /usr/lib/firmware
Notes
- FUSE policy: If "allow_other" error occurs, enable user_allow_other in /etc/fuse.conf or run mounts as root.
- WEB_ENDPOINT rewrite is disabled by default (set WEB_ENDPOINT=""). Plain S3 route is embedded in flists.
- MinIO client binary in dev-container is mcli; scripts support mcli (preferred) and mc (fallback).
Files added/modified
- Added: scripts/rfs/{common.sh,pack-modules.sh,pack-firmware.sh,verify-flist.sh,patch-stores.sh}
- Added: config/zinit/init/{firmware.sh,modules.sh}
- Added: docs/{rfs-flists.md,review-rfs-integration.md,depmod-behavior.md}
- Added: config/rfs.conf.example
- Modified: Dockerfile, scripts/functionlist.md, config/modules.conf, config/zinit/sshd-setup.yaml, .gitignore
|
