feat: first-draft preview-capable zosstorage

- CLI: add topology selection (-t/--topology), preview flags (--show/--report), and removable policy override (--allow-removable) (src/cli/args.rs)
- Config: built-in sensible defaults; deterministic overlays for logging, fstab, removable, topology (src/config/loader.rs)
- Device: discovery via /proc + /sys with include/exclude regex and removable policy (src/device/discovery.rs)
- Idempotency: detection via blkid; safe emptiness checks (src/idempotency/mod.rs)
- Partition: topology-driven planning (Single, DualIndependent, BtrfsRaid1, SsdHddBcachefs) (src/partition/plan.rs)
- FS: planning + creation (mkfs.vfat, mkfs.btrfs, bcachefs format) and UUID capture via blkid (src/fs/plan.rs)
- Orchestrator: pre-flight with preview JSON (disks, partition_plan, filesystems_planned, mount scheme). Skips emptiness in preview; supports stdout+file (src/orchestrator/run.rs)
- Util/Logging/Types/Errors: process execution, tracing, shared types (src/util/mod.rs, src/logging/mod.rs, src/types.rs, src/errors.rs)
- Docs: add README with exhaustive usage and preview JSON shape (README.md)

Builds and unit tests pass: discovery, util, idempotency helpers, and fs parser tests.
This commit is contained in:
2025-09-29 11:37:07 +02:00
commit 507bc172c2
38 changed files with 6558 additions and 0 deletions

758
docs/API-SKELETONS.md Normal file
View File

@@ -0,0 +1,758 @@
# zosstorage API Skeletons (Proposed)
Purpose
- This document proposes concrete Rust module skeletons with public API surface and doc comments.
- Bodies intentionally use todo!() or are omitted for approval-first workflow.
- After approval, these will be created in the src tree in Code mode.
Index
- [src/lib.rs](src/lib.rs)
- [src/errors.rs](src/errors.rs)
- [src/main.rs](src/main.rs)
- [src/cli/args.rs](src/cli/args.rs)
- [src/logging/mod.rs](src/logging/mod.rs)
- [src/types.rs](src/types.rs)
- [src/config/loader.rs](src/config/loader.rs)
- [src/device/discovery.rs](src/device/discovery.rs)
- [src/partition/plan.rs](src/partition/plan.rs)
- [src/fs/plan.rs](src/fs/plan.rs)
- [src/mount/ops.rs](src/mount/ops.rs)
- [src/report/state.rs](src/report/state.rs)
- [src/orchestrator/run.rs](src/orchestrator/run.rs)
- [src/idempotency/mod.rs](src/idempotency/mod.rs)
- [src/util/mod.rs](src/util/mod.rs)
Conventions
- Shared [type Result<T>](src/errors.rs:1) and [enum Error](src/errors.rs:1).
- No stdout prints; use tracing only.
- External tools invoked via [util](src/util/mod.rs) wrappers.
---
## Crate root
References
- [src/lib.rs](src/lib.rs)
- [type Result<T> = std::result::Result<T, Error>](src/errors.rs:1)
Skeleton (for later implementation in code mode)
```rust
//! Crate root for zosstorage: one-shot disk provisioning utility for initramfs.
pub mod cli;
pub mod logging;
pub mod config;
pub mod device;
pub mod partition;
pub mod fs;
pub mod mount;
pub mod report;
pub mod orchestrator;
pub mod idempotency;
pub mod util;
pub mod errors;
pub use errors::{Error, Result};
/// Crate version constants.
pub const VERSION: &str = env!("CARGO_PKG_VERSION");
```
---
## Errors
References
- [enum Error](src/errors.rs:1)
- [type Result<T>](src/errors.rs:1)
Skeleton
```rust
use thiserror::Error as ThisError;
/// Top-level error for zosstorage.
#[derive(Debug, ThisError)]
pub enum Error {
#[error("configuration error: {0}")]
Config(String),
#[error("validation error: {0}")]
Validation(String),
#[error("device discovery error: {0}")]
Device(String),
#[error("partitioning error: {0}")]
Partition(String),
#[error("filesystem error: {0}")]
Filesystem(String),
#[error("mount error: {0}")]
Mount(String),
#[error("report error: {0}")]
Report(String),
#[error("external tool '{tool}' failed with status {status}: {stderr}")]
Tool {
tool: String,
status: i32,
stderr: String,
},
#[error("unimplemented: {0}")]
Unimplemented(&'static str),
#[error(transparent)]
Other(#[from] anyhow::Error),
}
pub type Result<T> = std::result::Result<T, Error>;
```
---
## Entrypoint
References
- [fn main()](src/main.rs:1)
- [fn run(ctx: &Context) -> Result<()>](src/orchestrator/run.rs:1)
Skeleton
```rust
use zosstorage::{cli, config, logging, orchestrator, Result};
fn main() {
// Initialize minimal logging early (fallback).
// Proper logging config will be applied after CLI parsing.
// No stdout printing.
if let Err(e) = real_main() {
// In initramfs, emit minimal error to stderr and exit non-zero.
eprintln!("error: {e}");
std::process::exit(1);
}
}
fn real_main() -> Result<()> {
let cli = cli::from_args();
let log_opts = logging::LogOptions::from_cli(&cli);
logging::init_logging(&log_opts)?;
let cfg = config::load_and_merge(&cli)?;
config::validate(&cfg)?;
let ctx = orchestrator::Context::new(cfg, log_opts);
orchestrator::run(&ctx)
}
```
---
## CLI
References
- [struct Cli](src/cli/args.rs:1)
- [fn from_args() -> Cli](src/cli/args.rs:1)
Skeleton
```rust
use clap::{Parser, ValueEnum};
/// zosstorage - one-shot disk initializer for initramfs.
#[derive(Debug, Parser)]
#[command(name = "zosstorage", disable_help_subcommand = true)]
pub struct Cli {
/// Path to YAML configuration (mirrors kernel cmdline key 'zosstorage.config=')
#[arg(long = "config")]
pub config: Option<String>,
/// Log level: error, warn, info, debug
#[arg(long = "log-level", default_value = "info")]
pub log_level: String,
/// Also log to /run/zosstorage/zosstorage.log
#[arg(long = "log-to-file", default_value_t = false)]
pub log_to_file: bool,
/// Enable writing /etc/fstab entries
#[arg(long = "fstab", default_value_t = false)]
pub fstab: bool,
/// Present but non-functional; returns unimplemented error
#[arg(long = "force")]
pub force: bool,
}
/// Parse CLI arguments (non-interactive; suitable for initramfs).
pub fn from_args() -> Cli {
Cli::parse()
}
```
---
## Logging
References
- [struct LogOptions](src/logging/mod.rs:1)
- [fn init_logging(opts: &LogOptions) -> Result<()>](src/logging/mod.rs:1)
Skeleton
```rust
use crate::Result;
/// Logging options resolved from CLI and/or config.
#[derive(Debug, Clone)]
pub struct LogOptions {
pub level: String, // "error" | "warn" | "info" | "debug"
pub to_file: bool, // when true, logs to /run/zosstorage/zosstorage.log
}
impl LogOptions {
pub fn from_cli(cli: &crate::cli::Cli) -> Self {
Self { level: cli.log_level.clone(), to_file: cli.log_to_file }
}
}
/// Initialize tracing subscriber according to options.
/// Must be idempotent when called once in process lifetime.
pub fn init_logging(opts: &LogOptions) -> Result<()> {
todo!("set up tracing for stderr and optional file layer")
}
```
---
## Configuration types
References
- [struct Config](src/types.rs:1)
- [enum Topology](src/types.rs:1)
- [struct DeviceSelection](src/types.rs:1)
- [struct Partitioning](src/types.rs:1)
- [struct FsOptions](src/types.rs:1)
- [struct MountScheme](src/types.rs:1)
- [struct ReportOptions](src/types.rs:1)
Skeleton
```rust
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct LoggingConfig {
pub level: String, // default "info"
pub to_file: bool, // default false
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct DeviceSelection {
pub include_patterns: Vec<String>,
pub exclude_patterns: Vec<String>,
pub allow_removable: bool,
pub min_size_gib: u64,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum Topology {
Single,
DualIndependent,
SsdHddBcachefs,
BtrfsRaid1,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct BiosBootSpec {
pub enabled: bool,
pub size_mib: u64,
pub gpt_name: String, // "zosboot"
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct EspSpec {
pub size_mib: u64,
pub label: String, // "ZOSBOOT"
pub gpt_name: String, // "zosboot"
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct DataSpec {
pub gpt_name: String, // "zosdata"
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CacheSpec {
pub gpt_name: String, // "zoscache"
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Partitioning {
pub alignment_mib: u64,
pub require_empty_disks: bool,
pub bios_boot: BiosBootSpec,
pub esp: EspSpec,
pub data: DataSpec,
pub cache: CacheSpec,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct BtrfsOptions {
pub label: String, // "ZOSDATA"
pub compression: String, // "zstd:3"
pub raid_profile: String // "none" | "raid1"
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct BcachefsOptions {
pub label: String, // "ZOSDATA"
pub cache_mode: String, // "promote" | "writeback?"
pub compression: String, // "zstd"
pub checksum: String, // "crc32c"
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct VfatOptions {
pub label: String, // "ZOSBOOT"
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct FsOptions {
pub btrfs: BtrfsOptions,
pub bcachefs: BcachefsOptions,
pub vfat: VfatOptions,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum MountSchemeKind {
PerUuid,
Custom, // reserved
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct MountScheme {
pub base_dir: String, // "/var/cache"
pub scheme: MountSchemeKind,
pub fstab_enabled: bool,
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ReportOptions {
pub path: String, // "/run/zosstorage/state.json"
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Config {
pub version: u32,
pub logging: LoggingConfig,
pub device_selection: DeviceSelection,
pub topology: Topology,
pub partitioning: Partitioning,
pub filesystem: FsOptions,
pub mount: MountScheme,
pub report: ReportOptions,
}
```
---
## Configuration I/O
References
- [fn load_and_merge(cli: &Cli) -> Result<Config>](src/config/loader.rs:1)
- [fn validate(cfg: &Config) -> Result<()>](src/config/loader.rs:1)
Skeleton
```rust
use crate::{cli::Cli, Result};
/// Load defaults, merge file config, overlay CLI, and finally kernel cmdline.
pub fn load_and_merge(cli: &Cli) -> Result<crate::config::types::Config> {
todo!("implement precedence: file < CLI < kernel cmdline key 'zosstorage.config='")
}
/// Validate semantic correctness of the configuration.
pub fn validate(cfg: &crate::config::types::Config) -> Result<()> {
todo!("ensure device filters, sizes, topology combinations are valid")
}
```
---
## Device discovery
References
- [struct Disk](src/device/discovery.rs:1)
- [struct DeviceFilter](src/device/discovery.rs:1)
- [trait DeviceProvider](src/device/discovery.rs:1)
- [fn discover(filter: &DeviceFilter) -> Result<Vec<Disk>>](src/device/discovery.rs:1)
Skeleton
```rust
use crate::Result;
/// Eligible block device.
#[derive(Debug, Clone)]
pub struct Disk {
pub path: String,
pub size_bytes: u64,
pub rotational: bool,
pub model: Option<String>,
pub serial: Option<String>,
}
/// Compiled device filters from config patterns.
#[derive(Debug, Clone)]
pub struct DeviceFilter {
pub include: Vec<regex::Regex>,
pub exclude: Vec<regex::Regex>,
pub min_size_gib: u64,
}
/// Abstract provider for devices to enable testing with doubles.
pub trait DeviceProvider {
fn list_block_devices(&self) -> Result<Vec<Disk>>;
fn probe_properties(&self, disk: &mut Disk) -> Result<()>;
}
/// Discover eligible disks according to the filter policy.
pub fn discover(filter: &DeviceFilter) -> Result<Vec<Disk>> {
todo!("enumerate /dev, apply include/exclude, probe properties")
}
```
---
## Partitioning
References
- [enum PartRole](src/partition/plan.rs:1)
- [struct PartitionSpec](src/partition/plan.rs:1)
- [struct PartitionPlan](src/partition/plan.rs:1)
- [struct PartitionResult](src/partition/plan.rs:1)
- [fn plan_partitions(disks: &[Disk], cfg: &Config) -> Result<PartitionPlan>](src/partition/plan.rs:1)
- [fn apply_partitions(plan: &PartitionPlan) -> Result<Vec<PartitionResult>>](src/partition/plan.rs:1)
Skeleton
```rust
use crate::{config::types::Config, device::Disk, Result};
#[derive(Debug, Clone, Copy)]
pub enum PartRole {
BiosBoot,
Esp,
Data,
Cache,
}
#[derive(Debug, Clone)]
pub struct PartitionSpec {
pub role: PartRole,
pub size_mib: Option<u64>, // None means "remainder"
pub gpt_name: String, // zosboot | zosdata | zoscache
}
#[derive(Debug, Clone)]
pub struct DiskPlan {
pub disk: Disk,
pub parts: Vec<PartitionSpec>,
}
#[derive(Debug, Clone)]
pub struct PartitionPlan {
pub alignment_mib: u64,
pub disks: Vec<DiskPlan>,
pub require_empty_disks: bool,
}
#[derive(Debug, Clone)]
pub struct PartitionResult {
pub disk: String,
pub part_number: u32,
pub role: PartRole,
pub gpt_name: String,
pub uuid: String,
pub start_mib: u64,
pub size_mib: u64,
pub device_path: String, // e.g., /dev/nvme0n1p2
}
/// Compute GPT-only plan per topology and constraints.
pub fn plan_partitions(disks: &[Disk], cfg: &Config) -> Result<PartitionPlan> {
todo!("layout bios boot, ESP, data (and cache for SSD/HDD); align to 1 MiB")
}
/// Apply plan using sgdisk and verify via blkid; require empty disks if configured.
pub fn apply_partitions(plan: &PartitionPlan) -> Result<Vec<PartitionResult>> {
todo!("shell out to sgdisk, trigger udev settle, collect partition GUIDs")
}
```
---
## Filesystems
References
- [enum FsKind](src/fs/plan.rs:1)
- [struct FsSpec](src/fs/plan.rs:1)
- [struct FsPlan](src/fs/plan.rs:1)
- [struct FsResult](src/fs/plan.rs:1)
- [fn plan_filesystems(...)](src/fs/plan.rs:1)
- [fn make_filesystems(...)](src/fs/plan.rs:1)
Skeleton
```rust
use crate::{partition::PartitionResult, config::types::Config, Result};
#[derive(Debug, Clone, Copy)]
pub enum FsKind {
Vfat,
Btrfs,
Bcachefs,
}
#[derive(Debug, Clone)]
pub struct FsSpec {
pub kind: FsKind,
pub devices: Vec<String>, // 1 device for vfat/btrfs; 2 for bcachefs (cache + backing)
pub label: String, // "ZOSBOOT" (vfat) or "ZOSDATA" (data)
}
#[derive(Debug, Clone)]
pub struct FsPlan {
pub specs: Vec<FsSpec>,
}
#[derive(Debug, Clone)]
pub struct FsResult {
pub kind: FsKind,
pub devices: Vec<String>,
pub uuid: String,
pub label: String,
}
/// Decide which partitions get which filesystem based on topology.
pub fn plan_filesystems(
parts: &[PartitionResult],
cfg: &Config,
) -> Result<FsPlan> {
todo!("map ESP to vfat, data to btrfs or bcachefs according to topology")
}
/// Create the filesystems and return identity info (UUIDs, labels).
pub fn make_filesystems(plan: &FsPlan) -> Result<Vec<FsResult>> {
todo!("invoke mkfs tools with configured options via util::run_cmd")
}
```
---
## Mounting
References
- [struct MountPlan](src/mount/ops.rs:1)
- [struct MountResult](src/mount/ops.rs:1)
- [fn plan_mounts(...)](src/mount/ops.rs:1)
- [fn apply_mounts(...)](src/mount/ops.rs:1)
- [fn maybe_write_fstab(...)](src/mount/ops.rs:1)
Skeleton
```rust
use crate::{fs::FsResult, config::types::Config, Result};
#[derive(Debug, Clone)]
pub struct MountPlan {
pub entries: Vec<(String /* source */, String /* target */, String /* fstype */, String /* options */)>,
}
#[derive(Debug, Clone)]
pub struct MountResult {
pub source: String,
pub target: String,
pub fstype: String,
pub options: String,
}
/// Build mount plan under /var/cache/<UUID> by default.
pub fn plan_mounts(fs_results: &[FsResult], cfg: &Config) -> Result<MountPlan> {
todo!("create per-UUID directories and mount mapping")
}
/// Apply mounts using syscalls (nix), ensuring directories exist.
pub fn apply_mounts(plan: &MountPlan) -> Result<Vec<MountResult>> {
todo!("perform mount syscalls and return results")
}
/// Optionally generate /etc/fstab entries in deterministic order.
pub fn maybe_write_fstab(mounts: &[MountResult], cfg: &Config) -> Result<()> {
todo!("when enabled, write fstab entries")
}
```
---
## Reporting
References
- [const REPORT_VERSION: &str](src/report/state.rs:1)
- [struct StateReport](src/report/state.rs:1)
- [fn build_report(...)](src/report/state.rs:1)
- [fn write_report(...)](src/report/state.rs:1)
Skeleton
```rust
use serde::{Serialize, Deserialize};
use crate::{device::Disk, partition::PartitionResult, fs::FsResult, mount::MountResult, Result};
pub const REPORT_VERSION: &str = "v1";
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct StateReport {
pub version: String,
pub timestamp: String, // RFC3339
pub status: String, // "success" | "already_provisioned" | "error"
pub disks: Vec<serde_json::Value>,
pub partitions: Vec<serde_json::Value>,
pub filesystems: Vec<serde_json::Value>,
pub mounts: Vec<serde_json::Value>,
pub error: Option<String>,
}
/// Build the machine-readable state report.
pub fn build_report(
disks: &[Disk],
parts: &[PartitionResult],
fs: &[FsResult],
mounts: &[MountResult],
status: &str,
) -> StateReport {
todo!("assemble structured report in v1 format")
}
/// Write the state report to the configured path (default /run/zosstorage/state.json).
pub fn write_report(report: &StateReport, path: &str) -> Result<()> {
todo!("serialize to JSON and persist atomically")
}
```
---
## Orchestrator
References
- [struct Context](src/orchestrator/run.rs:1)
- [fn run(ctx: &Context) -> Result<()>](src/orchestrator/run.rs:1)
Skeleton
```rust
use crate::{config::types::Config, logging::LogOptions, Result};
/// Execution context holding resolved configuration and environment flags.
#[derive(Debug, Clone)]
pub struct Context {
pub cfg: Config,
pub log: LogOptions,
}
impl Context {
pub fn new(cfg: Config, log: LogOptions) -> Self { Self { cfg, log } }
}
/// High-level one-shot flow; aborts on any validation failure.
/// Returns Ok(()) on success and also on no-op when already provisioned.
pub fn run(ctx: &Context) -> Result<()> {
todo!("idempotency check, discovery, planning, application, reporting")
}
```
---
## Idempotency
References
- [fn detect_existing_state() -> Result<Option<StateReport>>](src/idempotency/mod.rs:1)
- [fn is_empty_disk(disk: &Disk) -> Result<bool>](src/idempotency/mod.rs:1)
Skeleton
```rust
use crate::{device::Disk, report::StateReport, Result};
/// Return existing state if system is already provisioned; otherwise None.
pub fn detect_existing_state() -> Result<Option<StateReport>> {
todo!("probe GPT names (zosboot, zosdata, zoscache) and FS labels (ZOSBOOT, ZOSDATA)")
}
/// Determine if a disk is empty (no partitions and no known FS signatures).
pub fn is_empty_disk(disk: &Disk) -> Result<bool> {
todo!("use blkid and partition table inspection to declare emptiness")
}
```
---
## Utilities
References
- [struct CmdOutput](src/util/mod.rs:1)
- [fn which_tool(name: &str) -> Result<Option<String>>](src/util/mod.rs:1)
- [fn run_cmd(args: &[&str]) -> Result<()>](src/util/mod.rs:1)
- [fn run_cmd_capture(args: &[&str]) -> Result<CmdOutput>](src/util/mod.rs:1)
- [fn udev_settle(timeout_ms: u64) -> Result<()>](src/util/mod.rs:1)
Skeleton
```rust
use crate::Result;
/// Captured output from an external tool invocation.
#[derive(Debug, Clone)]
pub struct CmdOutput {
pub status: i32,
pub stdout: String,
pub stderr: String,
}
/// Locate the absolute path to required tool if available.
pub fn which_tool(name: &str) -> Result<Option<String>> {
todo!("use 'which' crate or manual PATH scanning")
}
/// Run a command and return Ok if the exit status is zero.
pub fn run_cmd(args: &[&str]) -> Result<()> {
todo!("spawn process, log stderr on failure, map to Error::Tool")
}
/// Run a command and capture stdout/stderr for parsing (e.g., blkid).
pub fn run_cmd_capture(args: &[&str]) -> Result<CmdOutput> {
todo!("spawn process and collect output")
}
/// Call udevadm settle with a timeout; warn if unavailable, then no-op.
pub fn udev_settle(timeout_ms: u64) -> Result<()> {
todo!("invoke udevadm settle when present")
}
```
---
Approval gate
- This API skeleton is ready for implementation as source files with todo!() bodies.
- Upon approval, we will:
- Create the src/ files as outlined.
- Add dependencies via cargo add.
- Ensure all modules compile with placeholders.
- Add initial tests scaffolding and example configs.
References summary
- [fn main()](src/main.rs:1)
- [fn from_args()](src/cli/args.rs:1)
- [fn init_logging(opts: &LogOptions)](src/logging/mod.rs:1)
- [fn load_and_merge(cli: &Cli)](src/config/loader.rs:1)
- [fn validate(cfg: &Config)](src/config/loader.rs:1)
- [fn discover(filter: &DeviceFilter)](src/device/discovery.rs:1)
- [fn plan_partitions(disks: &[Disk], cfg: &Config)](src/partition/plan.rs:1)
- [fn apply_partitions(plan: &PartitionPlan)](src/partition/plan.rs:1)
- [fn plan_filesystems(parts: &[PartitionResult], cfg: &Config)](src/fs/plan.rs:1)
- [fn make_filesystems(plan: &FsPlan)](src/fs/plan.rs:1)
- [fn plan_mounts(fs_results: &[FsResult], cfg: &Config)](src/mount/ops.rs:1)
- [fn apply_mounts(plan: &MountPlan)](src/mount/ops.rs:1)
- [fn maybe_write_fstab(mounts: &[MountResult], cfg: &Config)](src/mount/ops.rs:1)
- [fn build_report(...)](src/report/state.rs:1)
- [fn write_report(report: &StateReport)](src/report/state.rs:1)
- [fn detect_existing_state()](src/idempotency/mod.rs:1)
- [fn is_empty_disk(disk: &Disk)](src/idempotency/mod.rs:1)
- [fn which_tool(name: &str)](src/util/mod.rs:1)
- [fn run_cmd(args: &[&str])](src/util/mod.rs:1)
- [fn run_cmd_capture(args: &[&str])](src/util/mod.rs:1)
- [fn udev_settle(timeout_ms: u64)](src/util/mod.rs:1)

217
docs/API.md Normal file
View File

@@ -0,0 +1,217 @@
# zosstorage Public API Skeletons
This document defines the initial public API surface for all modules. It lists traits, structs, enums, constants, and functions with responsibilities and behavioral contracts. Implementations are intentionally deferred until approval. Function bodies will be added later and guarded by todo placeholders.
Conventions
- All modules return a shared Result alias and Error enum.
- No interactive prompts; APIs are deterministic and suitable for initramfs use.
- External tooling calls are mediated via utility wrappers.
Module index
- [src/main.rs](src/main.rs)
- [src/lib.rs](src/lib.rs)
- [src/errors.rs](src/errors.rs)
- [src/cli/args.rs](src/cli/args.rs)
- [src/logging/mod.rs](src/logging/mod.rs)
- [src/types.rs](src/types.rs)
- [src/config/loader.rs](src/config/loader.rs)
- [src/device/discovery.rs](src/device/discovery.rs)
- [src/partition/plan.rs](src/partition/plan.rs)
- [src/fs/plan.rs](src/fs/plan.rs)
- [src/mount/ops.rs](src/mount/ops.rs)
- [src/report/state.rs](src/report/state.rs)
- [src/orchestrator/run.rs](src/orchestrator/run.rs)
- [src/idempotency/mod.rs](src/idempotency/mod.rs)
- [src/util/mod.rs](src/util/mod.rs)
Common errors and result
- [enum Error](src/errors.rs:1)
- Top-level error type covering parse/validation errors, device discovery errors, partitioning failures, filesystem mkfs errors, mount errors, report write errors, and external tool invocation failures with stderr capture.
- [type Result<T> = std::result::Result<T, Error>](src/errors.rs:1)
- Shared result alias across modules.
Crate root
- [src/lib.rs](src/lib.rs)
- Exposes crate version constants, the prelude, and re-exports common types for consumers of the library (tests/integration). No heavy logic.
Entrypoint
- [fn main()](src/main.rs:1)
- Initializes logging based on CLI defaults, parses CLI flags and kernel cmdline, loads and validates configuration, and invokes the orchestrator run sequence. Avoids stdout; logs via tracing only.
Orchestrator
- [struct Context](src/orchestrator/run.rs:1)
- Aggregates resolved configuration, logging options, and environment flags suited for initramfs execution.
- [fn run(ctx: &Context) -> Result<()>](src/orchestrator/run.rs:1)
- High-level one-shot flow:
- Idempotency detection
- Device discovery
- Partition planning and application
- Filesystem planning and creation
- Mount planning and application
- Report generation and write
- Aborts the entire run on any validation or execution failure. Returns Ok on successful no-op if already provisioned.
CLI
- [struct Cli](src/cli/args.rs:1)
- Mirrors kernel cmdline semantics with flags:
- --config PATH
- --log-level LEVEL
- --log-to-file
- --fstab
- --force (present, returns unimplemented error)
- [fn from_args() -> Cli](src/cli/args.rs:1)
- Parses argv without side effects; suitable for initramfs.
Logging
- [struct LogOptions](src/logging/mod.rs:1)
- Holds level and optional file target (/run/zosstorage/zosstorage.log).
- [fn init_logging(opts: &LogOptions) -> Result<()>](src/logging/mod.rs:1)
- Configures tracing-subscriber for stderr by default and optional file layer when enabled. Must be idempotent.
Configuration types
- [struct Config](src/types.rs:1)
- The validated configuration used by the orchestrator, containing logging, device selection rules, topology, partitioning, filesystem options, mount scheme, and report path.
- [enum Topology](src/types.rs:1)
- Values: single, dual_independent, ssd_hdd_bcachefs, btrfs_raid1 (opt-in).
- [struct DeviceSelection](src/types.rs:1)
- Include and exclude regex patterns, minimum size, removable policy.
- [struct Partitioning](src/types.rs:1)
- Alignment, emptiness requirement, bios_boot, esp, data, cache GPT names and sizes where applicable.
- [struct BtrfsOptions](src/types.rs:1)
- Compression string and raid profile (none or raid1).
- [struct BcachefsOptions](src/types.rs:1)
- Cache mode (promote or writeback), compression, checksum.
- [struct VfatOptions](src/types.rs:1)
- Reserved for ESP mkfs options; includes label ZOSBOOT.
- [struct FsOptions](src/types.rs:1)
- Aggregates BtrfsOptions, BcachefsOptions, VfatOptions and shared defaults such as ZOSDATA label.
- [enum MountSchemeKind](src/types.rs:1)
- Values: per_uuid, custom (future).
- [struct MountScheme](src/types.rs:1)
- Base directory (/var/cache), scheme kind, fstab enabled flag.
- [struct ReportOptions](src/types.rs:1)
- Output path (/run/zosstorage/state.json).
Configuration IO
- [fn load_and_merge(cli: &Cli) -> Result<Config>](src/config/loader.rs:1)
- Loads built-in defaults, optionally merges on-disk config, overlays CLI flags, and finally overlays kernel cmdline via zosstorage.config=. Validates the YAML against types and constraints.
- [fn validate(cfg: &Config) -> Result<()>](src/config/loader.rs:1)
- Ensures structural and semantic validity (e.g., disk selection rules not empty, sizes non-zero, supported topology combinations).
Device discovery
- [struct Disk](src/device/discovery.rs:1)
- Represents an eligible block device with path, size, rotational flag, and identifiers (serial, model if available).
- [struct DeviceFilter](src/device/discovery.rs:1)
- Derived from DeviceSelection; compiled regexes and size thresholds for efficient filtering.
- [trait DeviceProvider](src/device/discovery.rs:1)
- Abstraction for listing /dev and probing properties, enabling test doubles.
- [fn discover(filter: &DeviceFilter) -> Result<Vec<Disk>>](src/device/discovery.rs:1)
- Returns eligible disks or a well-defined error if none are found.
Partitioning
- [enum PartRole](src/partition/plan.rs:1)
- Roles: BiosBoot, Esp, Data, Cache.
- [struct PartitionSpec](src/partition/plan.rs:1)
- Declarative spec for a single partition: role, optional size_mib, gpt_name (zosboot, zosdata, zoscache), and reserved filesystem label when role is Esp (ZOSBOOT).
- [struct DiskPlan](src/partition/plan.rs:1)
- The planned set of PartitionSpec instances for a single Disk in the chosen topology.
- [struct PartitionPlan](src/partition/plan.rs:1)
- Combined plan across all target disks, including alignment rules and safety checks.
- [struct PartitionResult](src/partition/plan.rs:1)
- Result of applying a DiskPlan: device path of each created partition, role, partition GUID, and gpt_name.
- [fn plan_partitions(disks: &[Disk], cfg: &Config) -> Result<PartitionPlan>](src/partition/plan.rs:1)
- Produces a GPT-only plan with 1 MiB alignment, bios boot first (1 MiB), ESP 512 MiB, data remainder, and zoscache on SSD for ssd_hdd_bcachefs.
- [fn apply_partitions(plan: &PartitionPlan) -> Result<Vec<PartitionResult>>](src/partition/plan.rs:1)
- Executes the plan via sgdisk and related utilities. Aborts if target disks are not empty or if signatures are detected.
Filesystems
- [enum FsKind](src/fs/plan.rs:1)
- Values: Vfat, Btrfs, Bcachefs.
- [struct FsSpec](src/fs/plan.rs:1)
- Maps PartitionResult to desired filesystem kind and label (ZOSBOOT for ESP; ZOSDATA for all data filesystems including bcachefs).
- [struct FsPlan](src/fs/plan.rs:1)
- Plan of mkfs operations across all partitions and devices given the topology.
- [struct FsResult](src/fs/plan.rs:1)
- Output of mkfs: device path(s), fs uuid, label, and kind.
- [fn plan_filesystems(disks: &[Disk], parts: &[PartitionResult], cfg: &Config) -> Result<FsPlan>](src/fs/plan.rs:1)
- Determines which partitions receive vfat, btrfs, or bcachefs, and aggregates tuning options.
- [fn make_filesystems(plan: &FsPlan) -> Result<Vec<FsResult>>](src/fs/plan.rs:1)
- Invokes mkfs.vfat, mkfs.btrfs, mkfs.bcachefs accordingly via utility wrappers and returns filesystem identities.
Mounting
- [struct MountPlan](src/mount/ops.rs:1)
- Derived from FsResult entries: creates target directories under /var/cache/<UUID> and the mounts required for the current boot.
- [struct MountResult](src/mount/ops.rs:1)
- Actual mount operations performed (source, target, fstype, options).
- [fn plan_mounts(fs_results: &[FsResult], cfg: &Config) -> Result<MountPlan>](src/mount/ops.rs:1)
- Translates filesystem identities to mount targets and options.
- [fn apply_mounts(plan: &MountPlan) -> Result<Vec<MountResult>>](src/mount/ops.rs:1)
- Performs mounts using syscalls (nix crate) with minimal dependencies. Ensures directories exist.
- [fn maybe_write_fstab(mounts: &[MountResult], cfg: &Config) -> Result<()>](src/mount/ops.rs:1)
- When enabled, generates /etc/fstab entries in deterministic order. Disabled by default.
Reporting
- [const REPORT_VERSION: &str](src/report/state.rs:1)
- Version string for the JSON payload schema.
- [struct StateReport](src/report/state.rs:1)
- Machine-readable state describing discovered disks, created partitions, filesystems, labels, mountpoints, status, and timestamp.
- [fn build_report(disks: &[Disk], parts: &[PartitionResult], fs: &[FsResult], mounts: &[MountResult], status: &str) -> StateReport](src/report/state.rs:1)
- Constructs a StateReport matching REPORT_VERSION.
- [fn write_report(report: &StateReport) -> Result<()>](src/report/state.rs:1)
- Writes JSON to /run/zosstorage/state.json (configurable).
Idempotency
- [fn detect_existing_state() -> Result<Option<StateReport>>](src/idempotency/mod.rs:1)
- Probes for expected GPT names (zosboot, zosdata, zoscache where applicable) and filesystem labels (ZOSBOOT, ZOSDATA). If present and consistent, returns a StateReport; orchestrator exits success without changes.
- [fn is_empty_disk(disk: &Disk) -> Result<bool>](src/idempotency/mod.rs:1)
- Determines disk emptiness: absence of partitions and known filesystem signatures.
Utilities
- [struct CmdOutput](src/util/mod.rs:1)
- Captures status, stdout, stderr from external tool invocations.
- [fn which_tool(name: &str) -> Result<Option<String>>](src/util/mod.rs:1)
- Locates a required system utility in PATH, returning its absolute path if available.
- [fn run_cmd(args: &[&str]) -> Result<()>](src/util/mod.rs:1)
- Executes a command (args[0] is binary) and returns Ok when exit status is zero; logs stderr on failure.
- [fn run_cmd_capture(args: &[&str]) -> Result<CmdOutput>](src/util/mod.rs:1)
- Executes a command and returns captured output for parsing (e.g., blkid).
- [fn udev_settle(timeout_ms: u64) -> Result<()>](src/util/mod.rs:1)
- Calls udevadm settle with a timeout when available; otherwise no-ops with a warning.
Behavioral notes and contracts
- Safety and idempotency:
- Partitioning strictly aborts on any pre-existing partition or filesystem signature when require_empty_disks is true.
- Filesystem creation never proceeds if partitioning validation fails.
- The orchestrator treats any partial progress as failure; state report is only written on success.
- Labels and GPT names:
- ESP partitions are labeled ZOSBOOT (filesystem) and named zosboot (GPT).
- Data filesystems use label ZOSDATA regardless of backend kind.
- Cache partitions in bcachefs topology use GPT name zoscache.
- Topology-specific behavior:
- single: one data filesystem (btrfs) on the sole disk.
- dual_independent: two separate btrfs filesystems, one per disk.
- ssd_hdd_bcachefs: bcachefs spanning SSD (cache/promote) and HDD (backing), labeled ZOSDATA.
- btrfs_raid1: only when explicitly requested; otherwise default to independent btrfs.
Module dependency overview
```mermaid
flowchart LR
cli --> config
config --> orchestrator
logging --> orchestrator
orchestrator --> idempotency
orchestrator --> device
device --> partition
partition --> fs
fs --> mount
mount --> report
orchestrator --> report
util --> partition
util --> fs
util --> mount
util --> idempotency
```
Status
- This API surface is ready for code-mode skeleton implementation with todo placeholders in function bodies and comprehensive doc comments.

237
docs/ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,237 @@
# zosstorage Architecture
This document defines the repository layout, module boundaries, public API surface (signatures only), defaults, and high-level execution flow for the initramfs-only provisioning utility.
Baseline decisions and labels
- External tools inside initramfs are allowed and will be wrapped via helpers: sgdisk, blkid, mkfs.vfat, mkfs.btrfs, mkfs.bcachefs, udevadm.
- Kernel cmdline key: zosstorage.config=
- Default config path: /etc/zosstorage/config.yaml
- JSON state report path: /run/zosstorage/state.json
- Optional log file path: /run/zosstorage/zosstorage.log
- fstab generation: disabled by default
- GPT partition names: zosboot, zosdata, zoscache
- Filesystem labels:
- ESP: ZOSBOOT
- Data filesystems including bcachefs: ZOSDATA
Repository layout
Top level
- [Cargo.toml](Cargo.toml)
- [PROMPT.md](PROMPT.md)
- [README.md](README.md)
- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)
- [docs/SCHEMA.md](docs/SCHEMA.md)
- [examples/config/minimal.yaml](examples/config/minimal.yaml)
- [examples/config/dual-btrfs.yaml](examples/config/dual-btrfs.yaml)
- [examples/config/ssd-hdd-bcachefs.yaml](examples/config/ssd-hdd-bcachefs.yaml)
- [tests/](tests/)
- [tests/integration_single_disk.rs](tests/integration_single_disk.rs)
- [tests/integration_dual_disk.rs](tests/integration_dual_disk.rs)
- [tests/integration_ssd_hdd.rs](tests/integration_ssd_hdd.rs)
Crate sources
- [src/main.rs](src/main.rs)
- [src/lib.rs](src/lib.rs)
- [src/errors.rs](src/errors.rs)
- [src/cli/args.rs](src/cli/args.rs)
- [src/logging/mod.rs](src/logging/mod.rs)
- [src/config/loader.rs](src/config/loader.rs)
- [src/types.rs](src/types.rs)
- [src/device/discovery.rs](src/device/discovery.rs)
- [src/partition/plan.rs](src/partition/plan.rs)
- [src/fs/plan.rs](src/fs/plan.rs)
- [src/mount/ops.rs](src/mount/ops.rs)
- [src/report/state.rs](src/report/state.rs)
- [src/orchestrator/run.rs](src/orchestrator/run.rs)
- [src/idempotency/mod.rs](src/idempotency/mod.rs)
- [src/util/mod.rs](src/util/mod.rs)
Module responsibilities
- [src/main.rs](src/main.rs)
- Entrypoint. Parse CLI, initialize logging, load and merge configuration per precedence, call orchestrator. No stdout spam.
- [src/lib.rs](src/lib.rs)
- Crate exports, prelude, version constants, Result alias.
- [src/errors.rs](src/errors.rs)
- Common error enum and Result alias via thiserror.
- [src/cli/args.rs](src/cli/args.rs)
- CLI definition mirroring kernel cmdline semantics; provide non-interactive interface. Stub --force returns unimplemented.
- [src/logging/mod.rs](src/logging/mod.rs)
- Initialize tracing; levels error, warn, info, debug; default to stderr; optional file target.
- [src/config/loader.rs](src/config/loader.rs) and [src/types.rs](src/types.rs)
- YAML schema types, validation, loading, and merging with CLI and kernel cmdline.
- [src/device/discovery.rs](src/device/discovery.rs)
- Device discovery under /dev with filters and allowlist; probe emptiness safely.
- [src/partition/plan.rs](src/partition/plan.rs)
- GPT-only planning and application; 1 MiB alignment; create bios boot, ESP, data and cache partitions with strict safety checks.
- [src/fs/plan.rs](src/fs/plan.rs)
- Filesystem provisioning: vfat for ESP, btrfs for ZOSDATA, bcachefs for SSD+HDD mode; all data filesystems labeled ZOSDATA.
- [src/mount/ops.rs](src/mount/ops.rs)
- Mount per-UUID under /var/cache/<UUID>. Optional fstab writing, disabled by default.
- [src/report/state.rs](src/report/state.rs)
- Build and write JSON state report with version field.
- [src/orchestrator/run.rs](src/orchestrator/run.rs)
- One-shot flow orchestration with abort-on-any-validation-error policy.
- [src/idempotency/mod.rs](src/idempotency/mod.rs)
- Detect prior provisioning via GPT names and labels; return success-without-changes.
- [src/util/mod.rs](src/util/mod.rs)
- Shell-out, udev settle, and helpers.
Public API surface (signatures; implementation to follow after approval)
Entrypoint and orchestrator
- [fn main()](src/main.rs:1)
- [struct Context](src/orchestrator/run.rs:1)
- [fn run(ctx: &Context) -> Result<()>](src/orchestrator/run.rs:1)
CLI
- [struct Cli](src/cli/args.rs:1)
- [fn from_args() -> Cli](src/cli/args.rs:1)
Logging
- [struct LogOptions](src/logging/mod.rs:1)
- [fn init_logging(opts: &LogOptions) -> Result<()>](src/logging/mod.rs:1)
Config
- [struct Config](src/types.rs:1)
- [enum Topology](src/types.rs:1)
- [struct DeviceSelection](src/types.rs:1)
- [struct FsOptions](src/types.rs:1)
- [struct MountScheme](src/types.rs:1)
- [fn load_and_merge(cli: &Cli) -> Result<Config>](src/config/loader.rs:1)
- [fn validate(cfg: &Config) -> Result<()>](src/config/loader.rs:1)
Device discovery
- [struct Disk](src/device/discovery.rs:1)
- [struct DeviceFilter](src/device/discovery.rs:1)
- [trait DeviceProvider](src/device/discovery.rs:1)
- [fn discover(filter: &DeviceFilter) -> Result<Vec<Disk>>](src/device/discovery.rs:1)
Partitioning
- [struct PartitionSpec](src/partition/plan.rs:1)
- [struct PartitionPlan](src/partition/plan.rs:1)
- [struct PartitionResult](src/partition/plan.rs:1)
- [fn plan_partitions(disks: &[Disk], cfg: &Config) -> Result<PartitionPlan>](src/partition/plan.rs:1)
- [fn apply_partitions(plan: &PartitionPlan) -> Result<Vec<PartitionResult>>](src/partition/plan.rs:1)
Filesystems
- [enum FsKind](src/fs/plan.rs:1)
- [struct FsSpec](src/fs/plan.rs:1)
- [struct FsPlan](src/fs/plan.rs:1)
- [struct FsResult](src/fs/plan.rs:1)
- [fn plan_filesystems(disks: &[Disk], parts: &[PartitionResult], cfg: &Config) -> Result<FsPlan>](src/fs/plan.rs:1)
- [fn make_filesystems(plan: &FsPlan) -> Result<Vec<FsResult>>](src/fs/plan.rs:1)
Mounting
- [struct MountPlan](src/mount/ops.rs:1)
- [struct MountResult](src/mount/ops.rs:1)
- [fn plan_mounts(fs_results: &[FsResult], cfg: &Config) -> Result<MountPlan>](src/mount/ops.rs:1)
- [fn apply_mounts(plan: &MountPlan) -> Result<Vec<MountResult>>](src/mount/ops.rs:1)
- [fn maybe_write_fstab(mounts: &[MountResult], cfg: &Config) -> Result<()>](src/mount/ops.rs:1)
Reporting
- [const REPORT_VERSION: &str](src/report/state.rs:1)
- [struct StateReport](src/report/state.rs:1)
- [fn build_report(...) -> StateReport](src/report/state.rs:1)
- [fn write_report(report: &StateReport) -> Result<()>](src/report/state.rs:1)
Idempotency
- [fn detect_existing_state() -> Result<Option<StateReport>>](src/idempotency/mod.rs:1)
- [fn is_empty_disk(disk: &Disk) -> Result<bool>](src/idempotency/mod.rs:1)
Errors and Result
- [enum Error](src/errors.rs:1)
- [type Result<T> = std::result::Result<T, Error>](src/errors.rs:1)
Execution flow
```mermaid
flowchart TD
A[Start] --> B[Initialize logging]
B --> C[Parse CLI and kernel cmdline]
C --> D[Load and validate config]
D --> E[Idempotency detection]
E -->|already provisioned| Z[Exit success]
E -->|not provisioned| F[Discover devices]
F --> G[Plan partitions]
G --> H[Apply partitions]
H --> I[Plan filesystems]
I --> J[Create filesystems]
J --> K[Plan mounts]
K --> L[Apply mounts]
L --> M[Write state report]
M --> N[Finalize]
N --> Z[Exit success]
```
Configuration precedence
- Kernel cmdline key zosstorage.config= overrides CLI and file
- CLI flags override config file
- Config file provides defaults at /etc/zosstorage/config.yaml
- No interactive prompts in initramfs
Device discovery and filtering
- Include device classes by default: /dev/sd*, /dev/nvme*, /dev/vd*
- Exclude pseudodevices: /dev/ram*, /dev/zram*, /dev/fd*, /dev/loop*, etc.
- Allow future allowlists and removable media policies via configuration
- If no eligible disks are found, return a well-defined error
Partitioning plan
- GPT exclusively with 1 MiB alignment
- bios boot partition first, 1 MiB
- ESP 512 MiB FAT32, label ZOSBOOT, GPT name zosboot
- Data partition consumes remainder, GPT name zosdata
- When cache is requested, create GPT name zoscache partitions as needed
- Abort if any pre-existing partitions or filesystem signatures are detected
- Ensure unique partition UUIDs and identical labels where required
Filesystem provisioning defaults
- Single disk: btrfs labeled ZOSDATA
- Two disks: btrfs per disk labeled ZOSDATA, no RAID by default
- SSD plus HDD: bcachefs with SSD as cache or promote and HDD as backing, filesystem label ZOSDATA
- Filesystem tuning options configurable with sensible defaults and extension points
Mount scheme and fstab policy
- Mount under /var/cache/<UUID> using filesystem UUID to create stable subdirectories
- Optional /etc/fstab generation disabled by default; when enabled, produce deterministic order with documentation
Idempotency detection
- Consider the system provisioned when expected GPT names and filesystem labels are present and consistent
- On a provisioned system, exit success without making any changes
Reporting
- Emit machine-readable JSON state report at /run/zosstorage/state.json
- Report includes enumerated disks and roles, created partitions with identifiers, filesystems with labels and mountpoints, overall status and timestamp
- Version the report payload via REPORT_VERSION
Logging
- Use tracing with levels error, warn, info, debug
- Default to stderr; optionally log to file at /run/zosstorage/zosstorage.log
- Avoid println and stdout spam
External tooling policy
- Invoke system utilities via wrappers that check for tool availability, capture stderr, and emit structured logs
- Provide udev settle helper; avoid reliance on long-running services
Planned dependencies to add via cargo add
- clap
- serde, serde_yaml, serde_json
- thiserror
- anyhow or eyre
- tracing, tracing-subscriber
- nix
- regex
- uuid
- which
- time
- tempfile
Open items and assumptions to track
- Exact BIOS boot size and placement pending confirmation; currently 1 MiB first
- Final mount naming scheme under /var/cache may evolve
- Filesystem tuning defaults for btrfs and bcachefs require stakeholder input
- Paths for config, report, and log file may be adjusted later
- fstab generation remains disabled by default pending decision
Next steps after approval
- Formalize configuration schema and validation rules in [docs/SCHEMA.md](docs/SCHEMA.md)
- Define detailed doc comments for all listed types and functions
- Prepare code-mode implementation skeletons with todo placeholders and add dependencies via cargo add

123
docs/DEV_WORKFLOW.md Normal file
View File

@@ -0,0 +1,123 @@
# Modular Development Workflow and Traceability
Goal
- Enable incremental implementation without re-reading the entire codebase each time.
- Make module changes discoverable via grep and predictable locations.
- Keep a single source of truth for the API surface, invariants, and extension points.
Core Principles
1) Contract-first per module
- API signatures and responsibilities are documented in [docs/API-SKELETONS.md](docs/API-SKELETONS.md) and mirrored by crate modules:
- [src/types.rs](src/types.rs)
- [fn load_and_merge()](src/config/loader.rs:1), [fn validate()](src/config/loader.rs:1)
- [fn from_args()](src/cli/args.rs:1)
- [struct LogOptions](src/logging/mod.rs:1), [fn init_logging()](src/logging/mod.rs:1)
- [fn discover()](src/device/discovery.rs:1)
- [fn plan_partitions()](src/partition/plan.rs:1), [fn apply_partitions()](src/partition/plan.rs:1)
- [fn plan_filesystems()](src/fs/plan.rs:1), [fn make_filesystems()](src/fs/plan.rs:1)
- [fn plan_mounts()](src/mount/ops.rs:1), [fn apply_mounts()](src/mount/ops.rs:1), [fn maybe_write_fstab()](src/mount/ops.rs:1)
- [const REPORT_VERSION](src/report/state.rs:1), [fn build_report()](src/report/state.rs:1), [fn write_report()](src/report/state.rs:1)
- [struct Context](src/orchestrator/run.rs:1), [fn run()](src/orchestrator/run.rs:1)
- [fn detect_existing_state()](src/idempotency/mod.rs:1), [fn is_empty_disk()](src/idempotency/mod.rs:1)
- [struct CmdOutput](src/util/mod.rs:1), [fn which_tool()](src/util/mod.rs:1), [fn run_cmd()](src/util/mod.rs:1), [fn run_cmd_capture()](src/util/mod.rs:1), [fn udev_settle()](src/util/mod.rs:1)
2) Grep-able region markers in code
- Every module contains the following optional annotated regions:
- // REGION: API
- // REGION: EXTENSION_POINTS
- // REGION: SAFETY
- // REGION: ERROR_MAPPING
- // REGION: TODO
- Example snippet to add near top of a module:
// REGION: API
// api: device::discover(filter: &DeviceFilter) -> Result<Vec<Disk>>
// api: device::DeviceProvider
// REGION: API-END
- These must be kept concise, one-liners per function/trait/struct; they act as a quick index for search.
3) Stable identifiers for cross-references
- Use short identifiers in comments to reference public items:
- api: module::item
- ext: module::hook_or_trait
- safety: module::invariant_name
- errmap: module::error_path
- This allows quick discovery via regex: grep -R "api: device::" src/
4) Single source of truth for API surface
- Keep high-level API in [docs/API-SKELETONS.md](docs/API-SKELETONS.md) as canonical index. After adding/removing a public function or type, update this file.
- Add a short note in [docs/SPECS.md](docs/SPECS.md) if behavior or invariants change.
5) Architectural decisions recorded as ADRs
- Use docs/adr/NNNN-title.md to document decisions (context, decision, consequences).
- Start with [docs/adr/0001-modular-workflow.md](docs/adr/0001-modular-workflow.md) (added alongside this doc).
- Link ADRs from [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) when they supersede or refine prior guidance.
6) Module ownership and boundaries
- Add a “Module Responsibilities” section in each modules header doc comment summarizing scope and non-goals.
- Example references:
- [src/device/discovery.rs](src/device/discovery.rs)
- [src/partition/plan.rs](src/partition/plan.rs)
- [src/fs/plan.rs](src/fs/plan.rs)
- [src/mount/ops.rs](src/mount/ops.rs)
- [src/report/state.rs](src/report/state.rs)
7) Invariants and safety notes
- For code that must uphold safety or idempotency invariants, annotate with:
// SAFETY: explanation
// IDEMPOTENCY: explanation
- Example locations:
- [fn apply_partitions()](src/partition/plan.rs:1) must enforce empty-disks rule when configured.
- [fn make_filesystems()](src/fs/plan.rs:1) must not run if partitioning failed.
8) Error mapping consistency
- Centralize conversions to [enum Error](src/errors.rs:1). When calling external tools, wrap failures into Error::Tool with stderr captured.
- Annotate mapping areas with:
// ERROR: mapping external failure to Error::Tool
9) Module-local CHANGELOG entries
- Keep a single CHANGELOG in the repo root, plus module-local “Changes” sections appended on each module top comment (short bullets).
- Cross-link to the ADR when relevant.
10) Task-level breadcrumbs
- For multi-step features, add a short progress marker at top of relevant modules:
// TODO(KILO): feature-X step 2/4 parsing args done; next implement validation
- Summary of active tasks can also live in docs/SPECS.md under “In-Progress Work”.
11) Example configs and fixtures
- Keep comprehensive examples in:
- [config/zosstorage.example.yaml](config/zosstorage.example.yaml)
- Add minimal example variants if needed for tests (future):
- examples/config/minimal.yaml
- examples/config/dual-btrfs.yaml
- examples/config/ssd-hdd-bcachefs.yaml
12) “Golden paths” for resuming work
- If resuming work later:
- Read module headers for responsibilities
- Grep for REGION markers: API / TODO / SAFETY / ERROR_MAPPING
- Check [docs/API-SKELETONS.md](docs/API-SKELETONS.md) for contract changes
- Check latest ADRs in docs/adr/
- Check [docs/SPECS.md](docs/SPECS.md) and the “In-Progress Work” section (if used)
Checklist for adding a new feature
- Update contracts:
- Add or modify function/type signatures in code and reflect in [docs/API-SKELETONS.md](docs/API-SKELETONS.md)
- Add REGION: API one-liners for the new items
- Update invariants:
- Add REGION: SAFETY notes if needed
- Update specs:
- Document behavior in [docs/SPECS.md](docs/SPECS.md)
- If it is a long-term decision, add an ADR under docs/adr/
- Add examples if config or output formats change
- Update [config/zosstorage.example.yaml](config/zosstorage.example.yaml) or add a new example file
- Keep error mapping and logging consistent:
- Ensure any external tool calls map errors to [enum Error](src/errors.rs:1)
- Run cargo build and update any broken references
Optional automation (future)
- A simple “index check” script (cargo xtask) could validate:
- All public items referenced under REGION: API appear in [docs/API-SKELETONS.md](docs/API-SKELETONS.md)
- No broken module references
- REGION markers are well-formed
By following these conventions, changes stay localized and discoverable. A contributor (human or assistant) can quickly locate relevant areas by scanning module headers, REGION markers, and the centralized API/docs without re-reading the entire tree.

200
docs/SCHEMA.md Normal file
View File

@@ -0,0 +1,200 @@
# zosstorage Configuration Schema
This document defines the YAML configuration for the initramfs-only disk provisioning utility and the exact precedence rules between configuration sources. It complements [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
Canonical paths and keys
- Kernel cmdline key: zosstorage.config=
- Default config file path: /etc/zosstorage/config.yaml
- JSON state report path: /run/zosstorage/state.json
- Optional log file path: /run/zosstorage/zosstorage.log
- fstab generation: disabled by default
- Reserved filesystem labels: ZOSBOOT (ESP), ZOSDATA (all data filesystems)
- GPT partition names: zosboot, zosdata, zoscache
Precedence and merge strategy
1. Start from built-in defaults documented here.
2. Merge in the on-disk config file if present at /etc/zosstorage/config.yaml.
3. Merge CLI flags next; these override file values.
4. Merge kernel cmdline last; zosstorage.config= overrides CLI and file.
5. No interactive prompts are permitted.
The kernel cmdline key zosstorage.config= accepts:
- A path to a YAML file inside the initramfs root (preferred).
- A file: absolute path (e.g., file:/run/config/zos.yaml).
- A data: URL containing base64 YAML (optional extension).
Top-level YAML structure
```yaml
version: 1
logging:
level: info # one of: error, warn, info, debug
to_file: false # default false; when true, logs to /run/zosstorage/zosstorage.log
device_selection:
include_patterns: # default: ["^/dev/sd\\w+$", "^/dev/nvme\\w+n\\d+$", "^/dev/vd\\w+$"]
- "^/dev/sd\\w+$"
- "^/dev/nvme\\w+n\\d+$"
- "^/dev/vd\\w+$"
exclude_patterns: # default excludes: ram, zram, loop, fd, dm-crypt mappings not matching include, etc.
- "^/dev/ram\\d+$"
- "^/dev/zram\\d+$"
- "^/dev/loop\\d+$"
- "^/dev/fd\\d+$"
allow_removable: false # future option; default false
min_size_gib: 10 # ignore devices smaller than this (default 10)
topology: # desired overall layout; see values below
mode: single # single | dual_independent | ssd_hdd_bcachefs | btrfs_raid1 (optional)
partitioning:
alignment_mib: 1 # GPT alignment in MiB
require_empty_disks: true # abort if any partition or FS signatures exist
bios_boot:
enabled: true
size_mib: 1
gpt_name: zosboot # name for the tiny BIOS boot partition (non-FS)
esp:
size_mib: 512
label: ZOSBOOT
gpt_name: zosboot
data:
gpt_name: zosdata
cache:
gpt_name: zoscache
filesystem:
btrfs:
label: ZOSDATA
compression: zstd:3 # string passed to -O/compress option handling
raid_profile: none # none | raid1
bcachefs:
label: ZOSDATA
cache_mode: promote # promote | writeback (if supported during mkfs; default promote)
compression: zstd
checksum: crc32c
vfat:
label: ZOSBOOT
mount:
base_dir: /var/cache
scheme: per_uuid # per_uuid | custom
fstab:
enabled: false
report:
path: /run/zosstorage/state.json
```
Topology modes
- single: One eligible disk. Create BIOS boot (if enabled), ESP 512 MiB, remainder as data. Make a btrfs filesystem labeled ZOSDATA on the data partition.
- dual_independent: Two eligible disks. On each disk, create BIOS boot (if enabled) + ESP + data. Create a separate btrfs filesystem labeled ZOSDATA on each data partition. No RAID by default.
- ssd_hdd_bcachefs: One SSD/NVMe and one HDD. Create BIOS boot (if enabled) + ESP on both as required. Create cache (on SSD) and data/backing (on HDD) partitions named zoscache and zosdata respectively. Make a bcachefs filesystem across both with label ZOSDATA, using SSD as cache/promote and HDD as backing.
- btrfs_raid1: Optional mode if explicitly requested. Create mirrored btrfs across two disks for the data role with raid1 profile. Not enabled by default.
Validation rules
- Abort if no eligible disks are found after filtering.
- Abort if any target disk is not empty when require_empty_disks: true. Emptiness is determined by absence of partitions and known FS signatures.
- Never modify devices outside include_patterns or inside exclude_patterns.
- Ensure unique GPT partition UUIDs. ESP labels on different disks may be identical (ZOSBOOT), but partition UUIDs must differ.
- Filesystem labels must follow reserved semantics: ESP uses ZOSBOOT, all data filesystems use ZOSDATA.
Logging section
- logging.level: error | warn | info | debug. Default info.
- logging.to_file: when true, logs also go to /run/zosstorage/zosstorage.log. Default false.
Device selection section
- include_patterns: array of regex patterns (Rust-style) matched against absolute device paths.
- exclude_patterns: array of regex patterns that are removed after inclusion.
- allow_removable: future toggle for including removable media. Default false.
- min_size_gib: minimum size to consider a disk eligible; default 10 GiB.
Partitioning section
- alignment_mib: default 1 (MiB boundaries).
- require_empty_disks: true by default to guarantee safety.
- bios_boot: enabled true, size_mib 1, gpt_name zosboot. Used for BIOS-bootable GPT where needed.
- esp: size_mib 512, label ZOSBOOT, gpt_name zosboot.
- data: gpt_name zosdata.
- cache: gpt_name zoscache, only created in ssd_hdd_bcachefs mode.
Filesystem section
- btrfs: label ZOSDATA; compression string such as zstd:3; raid_profile none|raid1 (only applied when topology permits).
- bcachefs: label ZOSDATA; cache_mode promote|writeback; compression; checksum. Exact tuning defaults remain open items.
- vfat: label ZOSBOOT used for ESP.
Mount section
- base_dir: default /var/cache.
- scheme:
- per_uuid: mount data filesystems at /var/cache/<FS-UUID>
- custom: reserved for future mapping-by-config, not yet implemented.
- fstab.enabled: default false. When true, zosstorage will generate fstab entries in deterministic order.
Report section
- path: default /run/zosstorage/state.json.
- The report content schema is defined separately in [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) and the reporting module.
Configuration examples
Minimal single-disk btrfs
```yaml
version: 1
topology:
mode: single
```
Dual independent btrfs (two disks)
```yaml
version: 1
topology:
mode: dual_independent
filesystem:
btrfs:
compression: zstd:5
```
SSD + HDD with bcachefs
```yaml
version: 1
topology:
mode: ssd_hdd_bcachefs
partitioning:
cache:
gpt_name: zoscache
filesystem:
bcachefs:
cache_mode: promote
compression: zstd
checksum: crc32c
```
CLI flags (to mirror kernel cmdline)
- --config PATH: path to YAML config (mirrors zosstorage.config=)
- --log-level LEVEL: error|warn|info|debug
- --log-to-file: enables logging to /run/zosstorage/zosstorage.log
- --fstab: enable writing fstab entries
- --force: present but non-functional; returns unimplemented
Kernel cmdline examples
- zosstorage.config=/etc/zosstorage/config.yaml
- zosstorage.config=file:/run/zos.yaml
- zosstorage.config=data:application/x-yaml;base64,PHZlcnNpb246IDEK...
Notes on idempotency
- If expected GPT names (zosboot, zosdata, zoscache where applicable) and filesystem labels (ZOSBOOT, ZOSDATA) are found consistent with the chosen topology, zosstorage exits successfully without changes.
Future extensions
- Removable media allowlist policies
- btrfs RAID profiles beyond raid1
- bcachefs extended tuning options
- Custom mount mapping schemes
- Multiple topology groups on multi-disk systems
Reference modules
- [src/types.rs](src/types.rs)
- [src/config/loader.rs](src/config/loader.rs)
- [src/cli/args.rs](src/cli/args.rs)
- [src/orchestrator/run.rs](src/orchestrator/run.rs)
- [src/partition/plan.rs](src/partition/plan.rs)
- [src/fs/plan.rs](src/fs/plan.rs)
- [src/mount/ops.rs](src/mount/ops.rs)
- [src/report/state.rs](src/report/state.rs)
- [src/idempotency/mod.rs](src/idempotency/mod.rs)
Change log
- v1: Initial draft of schema and precedence rules
End of document

336
docs/SPECS.md Normal file
View File

@@ -0,0 +1,336 @@
# zosstorage Detailed Specifications
This document finalizes core specifications required before code skeleton implementation. It complements [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) and [docs/SCHEMA.md](docs/SCHEMA.md), and references the API declarations listed in [docs/API.md](docs/API.md).
Linked modules and functions
- Logging module: [src/logging/mod.rs](src/logging/mod.rs)
- [fn init_logging(opts: &LogOptions) -> Result<()>](src/logging/mod.rs:1)
- Report module: [src/report/state.rs](src/report/state.rs)
- [const REPORT_VERSION: &str](src/report/state.rs:1)
- [fn build_report(...) -> StateReport](src/report/state.rs:1)
- [fn write_report(report: &StateReport) -> Result<()>](src/report/state.rs:1)
- Device module: [src/device/discovery.rs](src/device/discovery.rs)
- [fn discover(filter: &DeviceFilter) -> Result<Vec<Disk>>](src/device/discovery.rs:1)
- Partitioning module: [src/partition/plan.rs](src/partition/plan.rs)
- [fn plan_partitions(disks: &[Disk], cfg: &Config) -> Result<PartitionPlan>](src/partition/plan.rs:1)
- [fn apply_partitions(plan: &PartitionPlan) -> Result<Vec<PartitionResult>>](src/partition/plan.rs:1)
- Filesystems module: [src/fs/plan.rs](src/fs/plan.rs)
- [fn plan_filesystems(disks: &[Disk], parts: &[PartitionResult], cfg: &Config) -> Result<FsPlan>](src/fs/plan.rs:1)
- [fn make_filesystems(plan: &FsPlan) -> Result<Vec<FsResult>>](src/fs/plan.rs:1)
- Mount module: [src/mount/ops.rs](src/mount/ops.rs)
- [fn plan_mounts(fs_results: &[FsResult], cfg: &Config) -> Result<MountPlan>](src/mount/ops.rs:1)
- [fn apply_mounts(plan: &MountPlan) -> Result<Vec<MountResult>>](src/mount/ops.rs:1)
- [fn maybe_write_fstab(mounts: &[MountResult], cfg: &Config) -> Result<()>](src/mount/ops.rs:1)
- Idempotency module: [src/idempotency/mod.rs](src/idempotency/mod.rs)
- [fn detect_existing_state() -> Result<Option<StateReport>>](src/idempotency/mod.rs:1)
- [fn is_empty_disk(disk: &Disk) -> Result<bool>](src/idempotency/mod.rs:1)
- CLI module: [src/cli/args.rs](src/cli/args.rs)
- [fn from_args() -> Cli](src/cli/args.rs:1)
- Orchestrator: [src/orchestrator/run.rs](src/orchestrator/run.rs)
- [fn run(ctx: &Context) -> Result<()>](src/orchestrator/run.rs:1)
---
## 1. Logging and tracing
Goals
- Structured, low-noise logging compatible with initramfs.
- Defaults to stderr. Optional file at /run/zosstorage/zosstorage.log controlled by config or CLI.
Configuration
- Levels: error, warn, info, debug (default info).
- Propagation: single global initialization via [fn init_logging](src/logging/mod.rs:1). Subsequent calls must be no-ops.
Implementation notes
- Use tracing and tracing-subscriber.
- Format: compact, with fields level, target, message, and optional module path. Avoid timestamps if writing to stderr in initramfs; include timestamps when logging to file.
Example behavior
- CLI --log-level debug sets level to debug.
- CLI --log-to-file or config.logging.to_file true enables file layer at /run/zosstorage/zosstorage.log.
---
## 2. JSON state report schema v1
Location
- Default output: /run/zosstorage/state.json
Versioning
- Include a top-level string field version equal to [REPORT_VERSION](src/report/state.rs:1). Start with v1.
Schema example
```json
{
"version": "v1",
"timestamp": "2025-09-25T12:00:00Z",
"status": "success",
"disks": [
{
"path": "/dev/nvme0n1",
"size_bytes": 40007973632,
"rotational": false,
"model": "QEMU NVMe Ctrl",
"serial": "nvme-1234",
"selected": true,
"roles": ["esp", "data"]
}
],
"partitions": [
{
"disk": "/dev/nvme0n1",
"number": 1,
"role": "bios_boot",
"gpt_name": "zosboot",
"uuid": "11111111-1111-1111-1111-111111111111",
"start_mib": 1,
"size_mib": 1
},
{
"disk": "/dev/nvme0n1",
"number": 2,
"role": "esp",
"gpt_name": "zosboot",
"uuid": "22222222-2222-2222-2222-222222222222",
"start_mib": 2,
"size_mib": 512,
"fs_label": "ZOSBOOT"
},
{
"disk": "/dev/nvme0n1",
"number": 3,
"role": "data",
"gpt_name": "zosdata",
"uuid": "33333333-3333-3333-3333-333333333333",
"start_mib": 514,
"size_mib": 39000
}
],
"filesystems": [
{
"kind": "vfat",
"device": "/dev/nvme0n1p2",
"uuid": "AAAA-BBBB",
"label": "ZOSBOOT",
"mountpoint": null
},
{
"kind": "btrfs",
"device": "/dev/nvme0n1p3",
"uuid": "aaaaaaaa-bbbb-cccc-dddd-eeeeffffffff",
"label": "ZOSDATA",
"mountpoint": "/var/cache/aaaaaaaa-bbbb-cccc-dddd-eeeeffffffff"
}
],
"mounts": [
{
"source": "/dev/nvme0n1p3",
"target": "/var/cache/aaaaaaaa-bbbb-cccc-dddd-eeeeffffffff",
"fstype": "btrfs",
"options": "defaults,ssd,compress=zstd:3"
}
]
}
```
Notes
- UUID formats follow tool output: VFAT UUID short form allowed.
- Status values: success, already_provisioned, error. On error, add error field with reason.
---
## 3. Device discovery and filtering rules
Default include patterns
- ^/dev/sd\\w+$
- ^/dev/nvme\\w+n\\d+$
- ^/dev/vd\\w+$
Default exclude patterns
- ^/dev/ram\\d+$
- ^/dev/zram\\d+$
- ^/dev/loop\\d+$
- ^/dev/fd\\d+$
Selection policy
- Compile include and exclude regex into [DeviceFilter](src/device/discovery.rs).
- Enumerate device candidates and apply:
- Must match at least one include.
- Must not match any exclude.
- Must be larger than min_size_gib (default 10).
- Probing
- Gather size, rotational flag, model, serial when available.
- Expose via [struct Disk](src/device/discovery.rs:1).
No eligible disks
- Return a specific error variant in [enum Error](src/errors.rs:1).
---
## 4. Partitioning plan
Constraints
- GPT only; enforce 1 MiB alignment.
- Abort immediately if any target disk is non-empty when require_empty_disks is true.
Layout defaults
- BIOS boot: 1 MiB first, role BiosBoot, GPT name zosboot, no filesystem.
- ESP: 512 MiB FAT32, GPT name zosboot; filesystem label ZOSBOOT.
- Data: remainder, GPT name zosdata.
- Cache partitions (only in ssd_hdd_bcachefs): GPT name zoscache on SSD.
Per-topology specifics
- single: All roles on the single disk.
- dual_independent: Each disk gets BIOS boot + ESP + data.
- ssd_hdd_bcachefs: SSD gets BIOS boot + ESP + zoscache, HDD gets BIOS boot + ESP + zosdata.
Safety checks
- Ensure unique partition UUIDs.
- Verify no pre-existing partitions or signatures. Use blkid or similar via [run_cmd_capture](src/util/mod.rs:1).
- After partition creation, run udev settle via [udev_settle](src/util/mod.rs:1).
Application
- Utilize sgdisk helpers in [apply_partitions](src/partition/plan.rs:1).
---
## 5. Filesystem provisioning strategies
Kinds
- Vfat for ESP, label ZOSBOOT.
- Btrfs for data on single and dual_independent.
- Bcachefs for ssd_hdd_bcachefs (SSD cache, HDD backing).
- All data filesystems use label ZOSDATA.
Defaults
- btrfs: compression zstd:3, raid_profile none unless explicitly set to raid1 in btrfs_raid1 mode.
- bcachefs: cache_mode promote, compression zstd, checksum crc32c.
- vfat: ESP label ZOSBOOT.
Planning and execution
- Decide mapping of [PartitionResult](src/partition/plan.rs:1) to [FsSpec](src/fs/plan.rs:1) in [plan_filesystems](src/fs/plan.rs:1).
- Create filesystems in [make_filesystems](src/fs/plan.rs:1) through wrapped mkfs tools.
- Capture resulting identifiers (fs uuid, label) in [FsResult](src/fs/plan.rs:1).
---
## 6. Mount scheme and fstab policy
Scheme
- per_uuid under /var/cache: directories named as filesystem UUIDs.
Mount options
- btrfs: ssd when non-rotational underlying device, compress from config, defaults otherwise.
- vfat: defaults, utf8.
fstab
- Disabled by default.
- When enabled, [maybe_write_fstab](src/mount/ops.rs:1) writes deterministic entries sorted by target path.
---
## 7. Idempotency detection
Signals for already-provisioned system
- Expected GPT names found: zosboot, zosdata, and zoscache when applicable.
- Filesystems with labels ZOSBOOT for ESP and ZOSDATA for all data filesystems.
- When consistent with selected topology, [detect_existing_state](src/idempotency/mod.rs:1) returns a StateReport and orchestrator exits success without changes.
Disk emptiness
- [is_empty_disk](src/idempotency/mod.rs:1) checks for absence of partitions and FS signatures before any modification.
---
## 8. CLI flags and help text outline
Flags mirrored by [struct Cli](src/cli/args.rs:1) parsed via [from_args](src/cli/args.rs:1)
- --config PATH
- --log-level LEVEL error | warn | info | debug
- --log-to-file
- --fstab enable fstab generation
- --force present but returns unimplemented error
Kernel cmdline
- zosstorage.config= accepts a path or file: URI or data: URL as described in [docs/SCHEMA.md](docs/SCHEMA.md).
Help text sections
- NAME, SYNOPSIS, DESCRIPTION
- CONFIG PRECEDENCE
- TOPOLOGIES: single, dual_independent, ssd_hdd_bcachefs, btrfs_raid1
- SAFETY AND IDEMPOTENCY
- REPORTS
- EXIT CODES: 0 success or already_provisioned, non-zero on error
---
## 9. Integration testing plan (QEMU KVM)
Scenarios to scaffold in [tests/](tests/)
- Single disk 40 GiB virtio: validates single topology end-to-end smoke.
- Dual NVMe 40 GiB each: validates dual_independent topology.
- SSD NVMe + HDD virtio: validates ssd_hdd_bcachefs topology.
- Negative: no eligible disks, or non-empty disk should abort.
Test strategy
- Tests will be staged as integration test scaffolds that compile and document manual steps or automated harness placeholders.
- Mocks
- Provide a test DeviceProvider to simulate discovery when running without QEMU.
- Wrap external tools via utility trait to enable command capture in dry-runs.
Artifacts to validate
- Presence of expected partition GPT names.
- Filesystems created with correct labels.
- Mountpoints under /var/cache/<UUID> when running in a VM.
- JSON report validates against v1 schema.
---
## 10. Documentation deliverables
- [README.md](README.md)
- Overview, quickstart, config precedence, example YAML, topology walkthroughs, usage, report format, safety, limitations, roadmap.
- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)
- [docs/SCHEMA.md](docs/SCHEMA.md)
- [docs/API.md](docs/API.md)
- Release notes template and CHANGELOG policy.
---
## 11. Build and packaging for static musl and Alpine initramfs
Rust build
- Target: x86_64-unknown-linux-musl
- Avoid glibc-only dependencies.
Binary constraints
- No reliance on services; suitable for busybox initramfs.
Embedding in initramfs
- Place the statically linked binary in initramfs.
- Ensure required external tools (sgdisk, blkid, mkfs.vfat, mkfs.btrfs, mkfs.bcachefs, udevadm) are present in the same initramfs environment.
Runtime notes
- Minimal stdout use; all status via tracing.
- Exit codes:
- 0 on success and on already provisioned.
- Non-zero on any validation or execution error.
---
## 12. Open items carried forward
- Confirm exact BIOS boot partition requirements across target platforms; currently set to 1 MiB first.
- Finalize btrfs and bcachefs tuning defaults after stakeholder review.
- Decide if/when to enable fstab generation by default in future.
- Allow removable media policies and additional device classes in configuration.
---
## 13. Next steps
- Proceed to code-mode to scaffold modules and types as declared in [docs/API.md](docs/API.md).
- Add dependencies via cargo add as listed in [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
- Implement bodies with todo!() placeholders and exhaustive doc comments before enabling functional behavior.
End of specifications.

130
docs/VIBE_HOWTO.md Normal file
View File

@@ -0,0 +1,130 @@
# VIBE HOWTO — Work Fast Without Re-reading the Tree
Purpose
- Ship features incrementally, with minimal context reload.
- Use grep-friendly region markers, a single API index, and clear responsibilities per module.
- Keep safety and error conventions explicit and consistent.
Key Ideas
- Contracts live in [docs/API-SKELETONS.md](docs/API-SKELETONS.md).
- Every module starts with REGION blocks that act like a 10-second summary.
- Architectural decisions go in ADRs under [docs/adr/0001-modular-workflow.md](docs/adr/0001-modular-workflow.md).
- Examples live at [config/zosstorage.example.yaml](../config/zosstorage.example.yaml).
Quickstart
- Build:
- cargo build
- Inspect CLI:
- cargo run -- --help
- Run with config and debug logs:
- cargo run -- --config ./config/zosstorage.example.yaml --log-level debug
- Default config file location (initramfs/user space):
- /etc/zosstorage/config.yaml (copy from [config/zosstorage.example.yaml](../config/zosstorage.example.yaml))
What the REGION markers mean
- REGION: API — one-liners for public items; grep target
- REGION: RESPONSIBILITIES — scope and non-goals
- REGION: EXTENSION_POINTS — how to extend without rewriting modules
- REGION: SAFETY — invariants; when to abort; idempotency rules
- REGION: ERROR_MAPPING — exact mapping to Error variants
- REGION: TODO — the next concrete steps to implement
Where to start (by responsibility)
- Entrypoint/binary:
- [src/main.rs](../src/main.rs)
- CLI parsing:
- [src/cli/args.rs](../src/cli/args.rs)
- Logging initialization:
- [src/logging/mod.rs](../src/logging/mod.rs)
- Config load/merge/validate:
- [src/config/loader.rs](../src/config/loader.rs)
- [src/types.rs](../src/types.rs)
- Device discovery:
- [src/device/discovery.rs](../src/device/discovery.rs)
- Partitioning planning/apply:
- [src/partition/plan.rs](../src/partition/plan.rs)
- Filesystem planning/create:
- [src/fs/plan.rs](../src/fs/plan.rs)
- Mount planning/apply + fstab:
- [src/mount/ops.rs](../src/mount/ops.rs)
- Reporting JSON:
- [src/report/state.rs](../src/report/state.rs)
- Orchestration:
- [src/orchestrator/run.rs](../src/orchestrator/run.rs)
- Idempotency/emptiness probes:
- [src/idempotency/mod.rs](../src/idempotency/mod.rs)
- External tools and shell-out:
- [src/util/mod.rs](../src/util/mod.rs)
Daily workflow (feature implementation)
1) Find the module
- Open the file and read the REGION header block for a 10-second overview.
- If you dont know where to look, grep by region:
- grep -R "REGION: TODO" src/
- grep -R "REGION: API" src/
2) Implement only the declared APIs first
- Keep signatures stable (as in [docs/API-SKELETONS.md](docs/API-SKELETONS.md)).
- Replace todo!() bodies one by one.
- Add safety and error notes under REGION: SAFETY and REGION: ERROR_MAPPING.
3) Keep docs in sync
- If you change a signature, update [docs/API-SKELETONS.md](docs/API-SKELETONS.md).
- If behavior or invariants change, update [docs/SPECS.md](docs/SPECS.md).
- If its an architectural choice, add or edit an ADR under [docs/adr/0001-modular-workflow.md](docs/adr/0001-modular-workflow.md).
4) Validate and build
- cargo build
- Use [config/zosstorage.example.yaml](../config/zosstorage.example.yaml) to drive configs.
- For VM testing guidance see [PROMPT.md](../PROMPT.md) Testing & Validation.
Error policy (quick rules)
- Map external tool failures to Error::Tool with status and stderr (see [src/errors.rs](../src/errors.rs)).
- Validation failures should be Error::Validation with clear messages.
- Config IO/parse issues are Error::Config; reporting IO errors are Error::Report.
- Use Error::Other(anyhow) sparingly (last-resort wrapping).
Safety and idempotency rules
- Never change a disk that isnt empty when require_empty_disks=true.
- Partitioning must guarantee GPT, 1 MiB alignment, reserved GPT names, and no preexisting signatures.
- Filesystems must use reserved labels: ZOSBOOT (ESP) and ZOSDATA (all data filesystems).
- Orchestrator must do nothing on already-provisioned systems and exit success.
Suggested first implementation tasks
- Utilities (unblocks everything else)
- Implement which_tool, run_cmd, run_cmd_capture, udev_settle in [src/util/mod.rs](../src/util/mod.rs)
- Kernel cmdline data: URL support
- Add base64 YAML support to config loading in [src/config/loader.rs](../src/config/loader.rs)
- Logging init
- Implement tracing setup in [src/logging/mod.rs](../src/logging/mod.rs), idempotent setup, optional file target
VM test matrix (virtio /dev/vd?)
- 1 disk (/dev/vda):
- single → btrfs on data, label ZOSDATA
- 2 disks (/dev/vda, /dev/vdb):
- dual_independent → btrfs per disk (two ZOSDATA)
- bcachefs cache/backing → /dev/vda cache (SSD-like), /dev/vdb backing (HDD-like), label ZOSDATA
- btrfs_raid1 → mirrored btrfs across both, label ZOSDATA
- 3 disks (/dev/vda, /dev/vdb, /dev/vdc):
- bcachefs → cache on /dev/vda; backing on /dev/vdb and /dev/vdc with two replicas; label ZOSDATA
Continuity checklist (resume after a break)
- Open the target module and read the REGION block.
- grep -R "REGION: TODO" src/ to find pending items.
- Verify the function signatures in [docs/API-SKELETONS.md](docs/API-SKELETONS.md).
- Check recent ADRs under [docs/adr/0001-modular-workflow.md](docs/adr/0001-modular-workflow.md).
- Skim [docs/SPECS.md](docs/SPECS.md) for behavior notes and in-progress sections.
Commit discipline
- Small commits per module/functionality.
- Keep REGION markers accurate (especially API and TODO).
- When changing behavior, update docs and/or ADRs in the same PR.
Reference index
- Architecture: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)
- Schema: [docs/SCHEMA.md](docs/SCHEMA.md)
- API Index: [docs/API-SKELETONS.md](docs/API-SKELETONS.md)
- Specs: [docs/SPECS.md](docs/SPECS.md)
- Dev Workflow (detailed): [docs/DEV_WORKFLOW.md](docs/DEV_WORKFLOW.md)
- ADR: [docs/adr/0001-modular-workflow.md](docs/adr/0001-modular-workflow.md)
- Example config: [config/zosstorage.example.yaml](../config/zosstorage.example.yaml)

View File

@@ -0,0 +1,74 @@
# ADR 0001: Modular Development Workflow with Grep-able Contracts and Regions
Status
- Accepted
- Date: 2025-09-25
Context
- The project will be developed iteratively with frequent returns and feature additions. We need a workflow that avoids re-reading the entire tree to find context.
- APIs should remain stable and discoverable, with a single source of truth for contracts and invariants.
- Contributors should be able to quickly grep the repository and see where to implement behavior, extend via hooks, and understand safety/error mapping.
Decision
- Adopt a modular development workflow comprising:
1) Contract-first documentation
- Keep canonical API declarations in [docs/API-SKELETONS.md](docs/API-SKELETONS.md).
- Mirror these contracts in module headers and doc comments.
2) Grep-friendly REGION markers in source files
- Each module (where applicable) starts with compact region markers:
- REGION: API — one-liners listing public items and signatures
- REGION: RESPONSIBILITIES — scope/non-goals
- REGION: EXTENSION_POINTS — how to extend without rewriting
- REGION: SAFETY — invariants and safety/idempotency constraints
- REGION: ERROR_MAPPING — standard mapping to error types
- REGION: TODO — active work or future steps
- These markers enable quick discovery via grep without deep reading.
3) ADRs for architectural decisions
- Document significant structural or process choices in docs/adr/.
- Cross-link in [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) when decisions are refined/superseded.
4) Examples and fixtures
- Maintain comprehensive example configs in config/, beginning with [config/zosstorage.example.yaml](config/zosstorage.example.yaml).
5) Golden path to resume work
- Read module headers, grep REGION markers, consult [docs/API-SKELETONS.md](docs/API-SKELETONS.md) and recent ADRs, and check [docs/SPECS.md](docs/SPECS.md) for in-progress notes.
Consequences
- Pros
- Fast onboarding and re-entry; contributors quickly find affected code by grepping markers or function names.
- Clear boundaries and invariants reduce coupling across modules.
- Documentation remains synchronized with code via consistent, enforceable patterns.
- Cons
- Slight overhead to keep REGION markers and API docs updated.
- Requires discipline to maintain concise one-liners and avoid duplication.
Implementation Notes
- Region markers have been added to key modules:
- [src/config/loader.rs](src/config/loader.rs)
- [src/orchestrator/run.rs](src/orchestrator/run.rs)
- [src/cli/args.rs](src/cli/args.rs)
- [src/device/discovery.rs](src/device/discovery.rs)
- [src/partition/plan.rs](src/partition/plan.rs)
- [src/fs/plan.rs](src/fs/plan.rs)
- [src/mount/ops.rs](src/mount/ops.rs)
- [src/report/state.rs](src/report/state.rs)
- Remaining modules will follow the same pattern as needed (e.g., util, idempotency, main/lib if helpful).
Related Documents
- Architecture: [docs/ARCHITECTURE.md](../ARCHITECTURE.md)
- API Index: [docs/API-SKELETONS.md](../API-SKELETONS.md)
- Specs: [docs/SPECS.md](../SPECS.md)
- Dev Workflow: [docs/DEV_WORKFLOW.md](../DEV_WORKFLOW.md)
- Example Config: [config/zosstorage.example.yaml](../../config/zosstorage.example.yaml)
Alternatives Considered
- Heavy code generation for interface docs — rejected due to complexity and limited incremental value for a small codebase.
- Relying solely on doc comments — insufficient discoverability without grep-able structured markers.
Adoption Plan
- Keep REGION markers up to date with each change to public APIs and module scope.
- Update [docs/API-SKELETONS.md](../API-SKELETONS.md) when signatures change.
- Add new ADRs when we introduce significant architectural adjustments.

53
docs/adr/README.md Normal file
View File

@@ -0,0 +1,53 @@
# Architectural Decision Records (ADR) Index
Purpose
- Central place to list and navigate all ADRs.
- Naming scheme: `NNNN-title.md` (zero-padded sequence).
Index
- [0001: Modular Development Workflow with Grep-able Contracts and Regions](0001-modular-workflow.md)
Conventions
- Location: [docs/adr/](.)
- Create new ADRs incrementally with next number (e.g., `0002-<short-title>.md`).
- Each ADR should include:
- Status (Proposed/Accepted/Deprecated)
- Context
- Decision
- Consequences
- Links to related docs (e.g., [docs/ARCHITECTURE.md](../ARCHITECTURE.md), [docs/API-SKELETONS.md](../API-SKELETONS.md))
Workflow
1) Propose an ADR when a design decision impacts architecture or module boundaries.
2) Add the ADR file under [docs/adr/](.) with the next sequence number.
3) Append the ADR to this index under “Index”.
4) Optionally add a short “Architectural Decisions” section to [docs/ARCHITECTURE.md](../ARCHITECTURE.md) summarizing the latest ADRs.
Template
```md
# ADR NNNN: Title
Status
- Proposed | Accepted | Deprecated
- Date: YYYY-MM-DD
Context
- Brief background and forces at play.
Decision
- The choice made; bullet any alternatives considered.
Consequences
- Pros/cons, impacts on modules, tests, build, etc.
Links
- [docs/ARCHITECTURE.md](../ARCHITECTURE.md)
- Related ADRs
```
Validation ideas (optional future work)
- Add a CI step or `cargo xtask adr-check` to verify:
- Files match `NNNN-title.md`
- All ADRs are listed in this README
- Sequence numbers are contiguous