zosstorage/docs/SPECS.md

# zosstorage Detailed Specifications

This document finalizes core specifications required before code skeleton implementation. It complements [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) and [docs/SCHEMA.md](docs/SCHEMA.md), and references the API declarations listed in [docs/API.md](docs/API.md).

Linked modules and functions
- Logging module: [src/logging/mod.rs](src/logging/mod.rs)
  - [fn init_logging(opts: &LogOptions) -> Result<()>](src/logging/mod.rs:1)
- Report module: [src/report/state.rs](src/report/state.rs)
  - [const REPORT_VERSION: &str](src/report/state.rs:1)
  - [fn build_report(...) -> StateReport](src/report/state.rs:1)
  - [fn write_report(report: &StateReport) -> Result<()>](src/report/state.rs:1)
- Device module: [src/device/discovery.rs](src/device/discovery.rs)
  - [fn discover(filter: &DeviceFilter) -> Result<Vec<Disk>>](src/device/discovery.rs:1)
- Partitioning module: [src/partition/plan.rs](src/partition/plan.rs)
  - [fn plan_partitions(disks: &[Disk], cfg: &Config) -> Result<PartitionPlan>](src/partition/plan.rs:1)
  - [fn apply_partitions(plan: &PartitionPlan) -> Result<Vec<PartitionResult>>](src/partition/plan.rs:1)
- Filesystems module: [src/fs/plan.rs](src/fs/plan.rs)
  - [fn plan_filesystems(disks: &[Disk], parts: &[PartitionResult], cfg: &Config) -> Result<FsPlan>](src/fs/plan.rs:1)
  - [fn make_filesystems(plan: &FsPlan) -> Result<Vec<FsResult>>](src/fs/plan.rs:1)
- Mount module: [src/mount/ops.rs](src/mount/ops.rs)
  - [fn plan_mounts(fs_results: &[FsResult], cfg: &Config) -> Result<MountPlan>](src/mount/ops.rs:1)
  - [fn apply_mounts(plan: &MountPlan) -> Result<Vec<MountResult>>](src/mount/ops.rs:1)
  - [fn maybe_write_fstab(mounts: &[MountResult], cfg: &Config) -> Result<()>](src/mount/ops.rs:1)
- Idempotency module: [src/idempotency/mod.rs](src/idempotency/mod.rs)
  - [fn detect_existing_state() -> Result<Option<StateReport>>](src/idempotency/mod.rs:1)
  - [fn is_empty_disk(disk: &Disk) -> Result<bool>](src/idempotency/mod.rs:1)
- CLI module: [src/cli/args.rs](src/cli/args.rs)
  - [fn from_args() -> Cli](src/cli/args.rs:1)
- Orchestrator: [src/orchestrator/run.rs](src/orchestrator/run.rs)
  - [fn run(ctx: &Context) -> Result<()>](src/orchestrator/run.rs:1)

---

## 1. Logging and tracing

Goals
- Structured, low-noise logging compatible with initramfs.
- Defaults to stderr. Optional file at /run/zosstorage/zosstorage.log controlled by config or CLI.

Configuration
- Levels: error, warn, info, debug (default info).
- Propagation: single global initialization via [fn init_logging](src/logging/mod.rs:1). Subsequent calls must be no-ops.

Implementation notes
- Use tracing and tracing-subscriber.
- Format: compact, with fields level, target, message, and optional module path. Avoid timestamps if writing to stderr in initramfs; include timestamps when logging to file.

Example behavior
- CLI --log-level debug sets level to debug.
- CLI --log-to-file or config.logging.to_file true enables file layer at /run/zosstorage/zosstorage.log.

---

## 2. JSON state report schema v1

Location
- Default output: /run/zosstorage/state.json

Versioning
- Include a top-level string field version equal to [REPORT_VERSION](src/report/state.rs:1). Start with v1.

Schema example

```json
{
  "version": "v1",
  "timestamp": "2025-09-25T12:00:00Z",
  "status": "success",
  "disks": [
    {
      "path": "/dev/nvme0n1",
      "size_bytes": 40007973632,
      "rotational": false,
      "model": "QEMU NVMe Ctrl",
      "serial": "nvme-1234",
      "selected": true,
      "roles": ["esp", "data"]
    }
  ],
  "partitions": [
    {
      "disk": "/dev/nvme0n1",
      "number": 1,
      "role": "bios_boot",
      "gpt_name": "zosboot",
      "uuid": "11111111-1111-1111-1111-111111111111",
      "start_mib": 1,
      "size_mib": 1
    },
    {
      "disk": "/dev/nvme0n1",
      "number": 2,
      "role": "esp",
      "gpt_name": "zosboot",
      "uuid": "22222222-2222-2222-2222-222222222222",
      "start_mib": 2,
      "size_mib": 512,
      "fs_label": "ZOSBOOT"
    },
    {
      "disk": "/dev/nvme0n1",
      "number": 3,
      "role": "data",
      "gpt_name": "zosdata",
      "uuid": "33333333-3333-3333-3333-333333333333",
      "start_mib": 514,
      "size_mib": 39000
    }
  ],
  "filesystems": [
    {
      "kind": "vfat",
      "device": "/dev/nvme0n1p2",
      "uuid": "AAAA-BBBB",
      "label": "ZOSBOOT",
      "mountpoint": null
    },
    {
      "kind": "btrfs",
      "device": "/dev/nvme0n1p3",
      "uuid": "aaaaaaaa-bbbb-cccc-dddd-eeeeffffffff",
      "label": "ZOSDATA",
      "mountpoint": "/var/cache/aaaaaaaa-bbbb-cccc-dddd-eeeeffffffff"
    }
  ],
  "mounts": [
    {
      "source": "/dev/nvme0n1p3",
      "target": "/var/cache/aaaaaaaa-bbbb-cccc-dddd-eeeeffffffff",
      "fstype": "btrfs",
      "options": "defaults,ssd,compress=zstd:3"
    }
  ]
}
```

Notes
- UUID formats follow tool output: VFAT UUID short form allowed.
- Status values: success, already_provisioned, error. On error, add error field with reason.

---

## 3. Device discovery and filtering rules

Default include patterns
- ^/dev/sd\\w+$
- ^/dev/nvme\\w+n\\d+$
- ^/dev/vd\\w+$

Default exclude patterns
- ^/dev/ram\\d+$
- ^/dev/zram\\d+$
- ^/dev/loop\\d+$
- ^/dev/fd\\d+$

Selection policy
- Compile include and exclude regex into [DeviceFilter](src/device/discovery.rs).
- Enumerate device candidates and apply:
  - Must match at least one include.
  - Must not match any exclude.
  - Must be larger than min_size_gib (default 10).
- Probing
  - Gather size, rotational flag, model, serial when available.
  - Expose via [struct Disk](src/device/discovery.rs:1).

No eligible disks
- Return a specific error variant in [enum Error](src/errors.rs:1).

---

## 4. Partitioning plan

Constraints
- GPT only; enforce 1 MiB alignment.
- Abort immediately if any target disk is non-empty when require_empty_disks is true.

Layout defaults
- BIOS boot: 1 MiB first, role BiosBoot, GPT name zosboot, no filesystem.
- ESP: 512 MiB FAT32, GPT name zosboot; filesystem label ZOSBOOT.
- Data: remainder, GPT name zosdata.
- Cache partitions (only in ssd_hdd_bcachefs): GPT name zoscache on SSD.

Per-topology specifics
- single: All roles on the single disk.
- dual_independent: Each disk gets BIOS boot + ESP + data.
- ssd_hdd_bcachefs: SSD gets BIOS boot + ESP + zoscache, HDD gets BIOS boot + ESP + zosdata.

Safety checks
- Ensure unique partition UUIDs.
- Verify no pre-existing partitions or signatures. Use blkid or similar via [run_cmd_capture](src/util/mod.rs:1).
- After partition creation, run udev settle via [udev_settle](src/util/mod.rs:1).

Application
- Utilize sgdisk helpers in [apply_partitions](src/partition/plan.rs:1).

---

## 5. Filesystem provisioning strategies

Kinds
- Vfat for ESP, label ZOSBOOT.
- Btrfs for data on single and dual_independent.
- Bcachefs for ssd_hdd_bcachefs (SSD cache, HDD backing).
- All data filesystems use label ZOSDATA.

Defaults
- btrfs: compression zstd:3, raid_profile none unless explicitly set to raid1 in btrfs_raid1 mode.
- bcachefs: cache_mode promote, compression zstd, checksum crc32c.
- vfat: ESP label ZOSBOOT.

Planning and execution
- Decide mapping of [PartitionResult](src/partition/plan.rs:1) to [FsSpec](src/fs/plan.rs:1) in [plan_filesystems](src/fs/plan.rs:1).
- Create filesystems in [make_filesystems](src/fs/plan.rs:1) through wrapped mkfs tools.
- Capture resulting identifiers (fs uuid, label) in [FsResult](src/fs/plan.rs:1).

---

## 6. Mount scheme and fstab policy

Scheme
- per_uuid under /var/cache: directories named as filesystem UUIDs.

Mount options
- btrfs: ssd when non-rotational underlying device, compress from config, defaults otherwise.
- vfat: defaults, utf8.

fstab
- Disabled by default.
- When enabled, [maybe_write_fstab](src/mount/ops.rs:1) writes deterministic entries sorted by target path.

---

## 7. Idempotency detection

Signals for already-provisioned system
- Expected GPT names found: zosboot, zosdata, and zoscache when applicable.
- Filesystems with labels ZOSBOOT for ESP and ZOSDATA for all data filesystems.
- When consistent with selected topology, [detect_existing_state](src/idempotency/mod.rs:1) returns a StateReport and orchestrator exits success without changes.

Disk emptiness
- [is_empty_disk](src/idempotency/mod.rs:1) checks for absence of partitions and FS signatures before any modification.

---

## 8. CLI flags and help text outline

Flags mirrored by [struct Cli](src/cli/args.rs:1) parsed via [from_args](src/cli/args.rs:1)
- --config PATH
- --log-level LEVEL    error | warn | info | debug
- --log-to-file
- --fstab              enable fstab generation
- --force              present but returns unimplemented error

Kernel cmdline
- zosstorage.config= accepts a path or file: URI or data: URL as described in [docs/SCHEMA.md](docs/SCHEMA.md).

Help text sections
- NAME, SYNOPSIS, DESCRIPTION
- CONFIG PRECEDENCE
- TOPOLOGIES: single, dual_independent, ssd_hdd_bcachefs, btrfs_raid1
- SAFETY AND IDEMPOTENCY
- REPORTS
- EXIT CODES: 0 success or already_provisioned, non-zero on error

---

## 9. Integration testing plan (QEMU KVM)

Scenarios to scaffold in [tests/](tests/)
- Single disk 40 GiB virtio: validates single topology end-to-end smoke.
- Dual NVMe 40 GiB each: validates dual_independent topology.
- SSD NVMe + HDD virtio: validates ssd_hdd_bcachefs topology.
- Negative: no eligible disks, or non-empty disk should abort.

Test strategy
- Tests will be staged as integration test scaffolds that compile and document manual steps or automated harness placeholders.
- Mocks
  - Provide a test DeviceProvider to simulate discovery when running without QEMU.
  - Wrap external tools via utility trait to enable command capture in dry-runs.

Artifacts to validate
- Presence of expected partition GPT names.
- Filesystems created with correct labels.
- Mountpoints under /var/cache/<UUID> when running in a VM.
- JSON report validates against v1 schema.

---

## 10. Documentation deliverables

- [README.md](README.md)
  - Overview, quickstart, config precedence, example YAML, topology walkthroughs, usage, report format, safety, limitations, roadmap.
- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)
- [docs/SCHEMA.md](docs/SCHEMA.md)
- [docs/API.md](docs/API.md)
- Release notes template and CHANGELOG policy.

---

## 11. Build and packaging for static musl and Alpine initramfs

Rust build
- Target: x86_64-unknown-linux-musl
- Avoid glibc-only dependencies.

Binary constraints
- No reliance on services; suitable for busybox initramfs.

Embedding in initramfs
- Place the statically linked binary in initramfs.
- Ensure required external tools (sgdisk, blkid, mkfs.vfat, mkfs.btrfs, mkfs.bcachefs, udevadm) are present in the same initramfs environment.

Runtime notes
- Minimal stdout use; all status via tracing.
- Exit codes:
  - 0 on success and on already provisioned.
  - Non-zero on any validation or execution error.

---

## 12. Open items carried forward

- Confirm exact BIOS boot partition requirements across target platforms; currently set to 1 MiB first.
- Finalize btrfs and bcachefs tuning defaults after stakeholder review.
- Decide if/when to enable fstab generation by default in future.
- Allow removable media policies and additional device classes in configuration.

---

## 13. Next steps

- Proceed to code-mode to scaffold modules and types as declared in [docs/API.md](docs/API.md).
- Add dependencies via cargo add as listed in [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
- Implement bodies with todo!() placeholders and exhaustive doc comments before enabling functional behavior.

End of specifications.