feat: first-draft preview-capable zosstorage

- CLI: add topology selection (-t/--topology), preview flags (--show/--report), and removable policy override (--allow-removable) (src/cli/args.rs)
- Config: built-in sensible defaults; deterministic overlays for logging, fstab, removable, topology (src/config/loader.rs)
- Device: discovery via /proc + /sys with include/exclude regex and removable policy (src/device/discovery.rs)
- Idempotency: detection via blkid; safe emptiness checks (src/idempotency/mod.rs)
- Partition: topology-driven planning (Single, DualIndependent, BtrfsRaid1, SsdHddBcachefs) (src/partition/plan.rs)
- FS: planning + creation (mkfs.vfat, mkfs.btrfs, bcachefs format) and UUID capture via blkid (src/fs/plan.rs)
- Orchestrator: pre-flight with preview JSON (disks, partition_plan, filesystems_planned, mount scheme). Skips emptiness in preview; supports stdout+file (src/orchestrator/run.rs)
- Util/Logging/Types/Errors: process execution, tracing, shared types (src/util/mod.rs, src/logging/mod.rs, src/types.rs, src/errors.rs)
- Docs: add README with exhaustive usage and preview JSON shape (README.md)

Builds and unit tests pass: discovery, util, idempotency helpers, and fs parser tests.
This commit is contained in:
2025-09-29 11:37:07 +02:00
commit 507bc172c2
38 changed files with 6558 additions and 0 deletions

101
PROMPT.md Normal file
View File

@@ -0,0 +1,101 @@
You are GPT-5 Codex paired with KILO coder. Produce only what is requested. Do not improvise.
Objective
- Implement `zosstorage`, a Rust binary compiled for static `musl`, embedded in an Alpine Linux initramfs (x86_64 only).
- Purpose: one-shot disk initializer invoked during the first boot of a fresh node. Idempotent; if rerun on a system already provisioned, it must perform no changes and exit success.
- Absolutely never destroy, overwrite, or repartition devices containing existing data. Development/testing uses pristine virtual disks only. Abort immediately if a target device is not empty.
Execution Context
- Runs inside initramfs on Alpine Linux (busybox environment). No reliance on system services or long-running daemons.
- No Cargo.toml manual edits; dependencies managed via `cargo add`. All code must compile with stable Rust toolchains available in the build system.
- Avoid stdout spam. Implement structured logging/tracing (details TBD) but no stray `println!`.
Development Methodology
- Compartmentalize the codebase into clear modules from the outset.
- Begin by proposing the repository layout (directories, modules, tests, docs).
- Define public APIs first: traits, structs, enums, function signatures, and associated documentation comments.
- Only after obtaining approval on the API surface may you proceed to fill in function bodies.
- Use `todo!()` or explanatory comments as temporary placeholders until behavior is agreed upon.
- Preserve this iterative approach for every major module: outline first, implementation after review.
Device Discovery
- Enumerate candidate block devices under `/dev` and filter out all pseudodevices (`/dev/ram*`, `/dev/zram*`, `/dev/fd*`, `/dev/loop*`, etc.). The filtering rules must be configurable for future allowlists (e.g., removable media).
- Default device classes include `/dev/sd*`, `/dev/nvme*`, `/dev/vd*`. If no eligible disks are found, return a well-defined error.
Partitioning Requirements
- Use GPT exclusively. Honor 1MiB alignment boundaries.
- For BIOS compatibility, create a small `bios_boot` partition (exact size TBD—assume 1MiB for now, placed first).
- Create a 512MiB FAT32 ESP on each disk, label `ZOSBOOT`. Each ESP is independent; synchronization will be handled by another tool (out of scope). Ensure unique partition UUIDs while keeping identical labels.
- Remaining disk capacity is provisioned per configuration (see below).
- Before making changes, verify the device has no existing partitions or filesystem signatures; abort otherwise.
Filesystem Provisioning
- All data mounts are placed somewhere under `/var/cache`. Precise mountpoints and subvolume strategies are configurable.
- Supported backends:
* Single disk: default to `btrfs`, label `ZOSDATA`.
* Two disks/NVMe: default to individual `btrfs` filesystems per disk, each labeled `ZOSDATA`, mounted under `/var/cache/<UUID>` (exact path pattern TBD). Optional support for `btrfs` RAID1 or `bcachefs` RAID1 if requested.
* Mixed SSD/NVMe + HDD: default to `bcachefs` with SSD as cache/promote and HDD as backing store, label resulting filesystem `ZOSDATA`. Alternative mode: separate `btrfs` per device (label `ZOSDATA`).
- Reserved filesystem labels: `ZOSBOOT` (ESP), `ZOSDATA` (all data filesystems). GPT partition names: `zosboot` (bios_boot and ESP), `zosdata` (data), `zoscache` (cache).
- Filesystem tuning options (compression, RAID profile, etc.) must be configurable; define sensible defaults and provide extension points.
Configuration Input
- Accept configuration via:
* Kernel command line parameter (name TBD, e.g., `zosstorage.config=`) pointing to a YAML configuration descriptor.
* Optional CLI flags when run in user space (must mirror kernel cmdline semantics).
* On-disk YAML config file (default path TBD, e.g., `/etc/zosstorage/config.yaml`).
- Establish clear precedence: kernel cmdline overrides CLI arguments, which override config file defaults. No interactive prompts inside initramfs.
- YAML schema must at least describe disk selection rules, desired filesystem layout, boot partition preferences, filesystem options, mount targets, and logging verbosity. Document the schema and provide validation.
State Reporting
- After successful provisioning, emit a JSON state report (path TBD, e.g., `/run/zosstorage/state.json`) capturing:
* Enumerated disks and their roles,
* Created partitions with identifiers,
* Filesystems, labels (`ZOSBOOT`, `ZOSDATA`, `ZOSCACHE`), mountpoints,
* Overall status and timestamp.
- Ensure the report is machine-readable and versioned.
Logging
- Integrate a structured logging/tracing backend (e.g., `tracing` crate). Provide log levels (error, warn, info, debug) and allow configuration through CLI/config/cmdline.
- By default, logs go to stderr; design for optional redirection to a file (path TBD). Avoid using `println!`.
System Integration
- Decide whether to generate `/etc/fstab` entries; if enabled, produce deterministic ordering and documentation. Otherwise, document alternative mount management.
- After provisioning, ensure the initramfs can mount the new filesystems (e.g., call `udevadm settle` if necessary). No external services are invoked.
- No responsibility for updating `vmlinuz.efi`; another subsystem handles kernel updates.
Failure Handling
- If any target disk fails validation (non-empty, filtered out, or errors occur), abort the entire run with a descriptive error message. Provide a `--force` flag stub for future use, but keep it non-functional for now (must return “unimplemented”).
Testing & Validation (initial expectations)
- Provide integration test scaffolding targeting QEMU/KVM scenarios (e.g., single virtio disk 40GiB, dual NVMe 40GiB each, SSD+HDD mix). Tests can be smoke-level initially but must compile.
- Document manual testing steps for developers to reproduce in VMs.
- VM test matrix using virtio disks (/dev/vd?) to validate topologies:
* 1 disk (/dev/vda): single topology → create btrfs on the data partition labeled ZOSDATA.
* 2 disks (/dev/vda, /dev/vdb):
- dual_independent: btrfs per disk (two independent ZOSDATA filesystems).
- bcachefs cache/backing: treat /dev/vda as cache (SSD-like) and /dev/vdb as backing (HDD-like); create one bcachefs labeled ZOSDATA.
- btrfs_raid1: mirrored btrfs across the two data partitions labeled ZOSDATA.
* 3 disks (/dev/vda, /dev/vdb, /dev/vdc):
- bcachefs: cache on /dev/vda; backing on /dev/vdb and /dev/vdc with two replicas (two copies), labeled ZOSDATA.
- Ensure device discovery includes /dev/vd* by default and filters pseudodevices.
Documentation & Deliverables
- Produce comprehensive README including: overview, prerequisites, configuration schema, example YAML, command-line usage, JSON report format, filesystem label semantics (`ZOSBOOT`, `ZOSDATA`, `ZOSCACHE`), limitations, and roadmap.
- Ensure Rust code contains module-level and public API documentation (/// doc comments). Implement `--help` output mirroring README usage.
- Include architectural notes describing module boundaries (device discovery, partitioning, filesystem provisioning, config parsing, logging, reporting).
Open Items (call out explicitly)
- Exact sizes and ordering for `bios_boot` partition awaiting confirmation; note assumptions in code and documentation.
- Mount point naming scheme under `/var/cache` (per-UUID vs. config-defined) still to be finalized.
- Filesystem-specific tuning parameters (compression, RAID values, `bcachefs` options) require explicit defaults from stakeholders.
- Path/location for YAML config, kernel cmdline key, JSON report path, and optional log file path need final confirmation.
- Decision whether `/etc/fstab` is generated remains pending.
Implementation Constraints
- Stick to clear module boundaries. Provide unit tests where possible (e.g., config parsing, device filtering).
- Maintain strict idempotency: detect when provisioning already occurred (e.g., presence of `ZOSBOOT` partitions and expected filesystem labels `ZOSDATA`/`ZOSCACHE`) and exit gracefully.
- Write clean, production-quality Rust adhering to idiomatic practices and Clippy.
Deliverables
- Repository layout proposal (src modules, tests directory, docs). Highlight major components and their responsibilities.
- API skeletons (traits, structs, function signatures) with doc comments, using `todo!()` placeholders for bodies until approved.
- After API approval, progressively fill in implementations, preserving the compartmentalized structure and documenting assumptions.