lhumina_code/hero_skills
4
0
Fork 0
Code Issues 34 Pull requests Projects Releases 1 Packages 1 Wiki Activity Actions

service_compute.nu — hero_compute lifecycle module #154

New issue
Closed
opened 2026-04-28 09:35:11 +00:00 by mahmoud · 5 comments
mahmoud commented 2026-04-28 09:35:11 +00:00
Owner
Copy link

Add service_compute.nu per the tracker in #75. Module exposes install | start [--reset] | stop | status for the hero_compute stack.

Scope

  • Module file: tools/modules/services/service_compute.nu
  • Follow the standard pattern documented in the nu_service and nu_service_use skills.
  • Template to copy from: service_codescalers.nu — full-featured, supports --root. Same shape as the hero_compute manager.
  • Source repo: hero_compute (canonical layout already applied; reference repo for /hero_rust_repo_create).

Service-specific notes

  • Pure Rust workspace at 0.1.7+; build via cargo build --release.
  • Four binaries to install to ~/hero/bin/ (per scripts/buildenv.sh::BINARIES): hero_compute, hero_compute_server, hero_compute_ui, hero_compute_explorer.
  • hero_compute is the lifecycle manager — selfstart pattern, registers and starts the other three with hero_proc. See hero_proc_service_selfstart.
  • Requires --root — KVM device access, raw networking, Mycelium overlay membership. Match service_mycelium.nu / service_codescalers.nu for the privileged-service shape.
  • Multi-node modes: local (default), master (explorer hub), worker (joins a master). The repo's Makefile drives this via make start MODE=master MASTER_IP=<ip>. Surface this in the nu module's start flags or document it in the start-output test plan.
  • Runtime dependencies (versions pinned in the repo's scripts/buildenv.sh):
    • cloud-hypervisor (VMM backend)
    • my_hypervisor (VM frontend)
    • mycelium (IPv6 overlay)
    • hero_proc (process supervisor — already required by every service)
  • Uses hero_compute_registry for VM images at runtime — install step does not need to clone the registry; the running stack pulls images on demand.
  • The repo's scripts/install.sh shows the full system-deps + binaries flow you'll port into the nu module — use it as the donor for the install subcommand.
  • System packages needed (Debian/Ubuntu): libssl-dev, pkg-config, iproute2, busybox-static, virtiofsd, e2fsprogs. Reuse installers.nu::install_base where possible.

Acceptance criteria

  1. use services/mod.nu * makes service_compute available.
  2. On a target host (root for --root mode):
    • service_compute install --root clones the repo, installs system deps + cloud-hypervisor + my_hypervisor + mycelium, runs cargo build --release, copies the four binaries to ~/hero/bin/.
    • service_compute start --root registers with hero_proc and becomes healthy. The hero_compute manager handles the selfstart of _server / _ui / _explorer.
    • service_compute status --root reports state.
    • service_compute stop --root cleanly unregisters.
  3. The start output prints sockets / UI URL / a short test plan, per the nu_service_use skill.
  4. (Optional) Multi-node start: service_compute start --root --mode master and --mode worker --master-ip <ip> mirror the Makefile.

References

  • Parent tracker: #75
  • Pattern skills: nu_service, nu_service_use
  • Self-start pattern: hero_proc_service_selfstart
  • Reference repo for canonical layout: hero_rust_repo_create (uses hero_compute as donor)
Add `service_compute.nu` per the tracker in #75. Module exposes `install | start [--reset] | stop | status` for the `hero_compute` stack. ## Scope - Module file: `tools/modules/services/service_compute.nu` - Follow the standard pattern documented in the [`nu_service`](../../src/branch/development/claude/skills/nu_service/SKILL.md) and [`nu_service_use`](../../src/branch/development/claude/skills/nu_service_use/SKILL.md) skills. - Template to copy from: `service_codescalers.nu` — full-featured, supports `--root`. Same shape as the hero_compute manager. - Source repo: [`hero_compute`](https://forge.ourworld.tf/lhumina_code/hero_compute) (canonical layout already applied; reference repo for `/hero_rust_repo_create`). ## Service-specific notes - Pure Rust workspace at `0.1.7`+; build via `cargo build --release`. - Four binaries to install to `~/hero/bin/` (per `scripts/buildenv.sh::BINARIES`): `hero_compute`, `hero_compute_server`, `hero_compute_ui`, `hero_compute_explorer`. - `hero_compute` is the lifecycle manager — selfstart pattern, registers and starts the other three with hero_proc. See [`hero_proc_service_selfstart`](../../src/branch/development/claude/skills/hero_proc_service_selfstart/SKILL.md). - **Requires `--root`** — KVM device access, raw networking, Mycelium overlay membership. Match `service_mycelium.nu` / `service_codescalers.nu` for the privileged-service shape. - Multi-node modes: `local` (default), `master` (explorer hub), `worker` (joins a master). The repo's Makefile drives this via `make start MODE=master MASTER_IP=<ip>`. Surface this in the nu module's `start` flags or document it in the start-output test plan. - Runtime dependencies (versions pinned in the repo's `scripts/buildenv.sh`): - `cloud-hypervisor` (VMM backend) - `my_hypervisor` (VM frontend) - `mycelium` (IPv6 overlay) - `hero_proc` (process supervisor — already required by every service) - Uses [`hero_compute_registry`](https://forge.ourworld.tf/lhumina_code/hero_compute_registry) for VM images at runtime — install step does **not** need to clone the registry; the running stack pulls images on demand. - The repo's [`scripts/install.sh`](https://forge.ourworld.tf/lhumina_code/hero_compute/src/branch/development/scripts/install.sh) shows the full system-deps + binaries flow you'll port into the nu module — use it as the donor for the `install` subcommand. - System packages needed (Debian/Ubuntu): `libssl-dev`, `pkg-config`, `iproute2`, `busybox-static`, `virtiofsd`, `e2fsprogs`. Reuse `installers.nu::install_base` where possible. ## Acceptance criteria 1. `use services/mod.nu *` makes `service_compute` available. 2. On a target host (root for `--root` mode): - `service_compute install --root` clones the repo, installs system deps + cloud-hypervisor + my_hypervisor + mycelium, runs `cargo build --release`, copies the four binaries to `~/hero/bin/`. - `service_compute start --root` registers with hero_proc and becomes healthy. The `hero_compute` manager handles the selfstart of `_server` / `_ui` / `_explorer`. - `service_compute status --root` reports state. - `service_compute stop --root` cleanly unregisters. 3. The `start` output prints sockets / UI URL / a short test plan, per the `nu_service_use` skill. 4. (Optional) Multi-node start: `service_compute start --root --mode master` and `--mode worker --master-ip <ip>` mirror the Makefile. ## References - Parent tracker: #75 - Pattern skills: `nu_service`, `nu_service_use` - Self-start pattern: `hero_proc_service_selfstart` - Reference repo for canonical layout: `hero_rust_repo_create` (uses hero_compute as donor)
mahmoud self-assigned this 2026-04-28 09:35:11 +00:00
mahmoud removed their assignment 2026-04-28 10:03:57 +00:00
mahmoud commented 2026-04-28 10:08:19 +00:00
Author
Owner
Copy link

Updated scope after research on the actual repo. Hero_compute is at 0.1.7+ in the canonical layout; the four binaries (hero_compute manager + _server, _ui, _explorer) are in scripts/buildenv.sh::BINARIES. --root confirmed needed for KVM and Mycelium. Multi-node modes and runtime deps (cloud-hypervisor, my_hypervisor, mycelium) added. The repo's scripts/install.sh is the donor for the install flow.

Updated scope after research on the actual repo. Hero_compute is at 0.1.7+ in the canonical layout; the four binaries (`hero_compute` manager + `_server`, `_ui`, `_explorer`) are in `scripts/buildenv.sh::BINARIES`. `--root` confirmed needed for KVM and Mycelium. Multi-node modes and runtime deps (cloud-hypervisor, my_hypervisor, mycelium) added. The repo's `scripts/install.sh` is the donor for the install flow.
fatmaebrahim commented 2026-04-29 07:17:31 +00:00
Member
Copy link

Implementation Spec for Issue #154

Objective

Add tools/modules/services/service_compute.nu — a privileged Hero-service lifecycle module for the hero_compute stack — exposing install, start [--reset], stop, status (plus reset), modeled on service_codescalers.nu and matching the privileged-service shape of service_mycelium.nu. The module registers four binaries (hero_compute manager + hero_compute_server + hero_compute_ui + hero_compute_explorer) with hero_proc, mirroring what the upstream Rust manager does internally, so existing nu_service_use workflows ( service_compute start --root --reset ) work end-to-end.

Requirements

  • New file: tools/modules/services/service_compute.nu. No edits to existing service modules.
  • Re-export the new module in tools/modules/services/mod.nu so use services/mod.nu * exposes service_compute.
  • Follow the donor template service_codescalers.nu exactly — same constants, helper layout (svx_*), and section banners — adapted to four binaries instead of three.
  • Match the privileged-service shape from service_mycelium.nu: --root is the documented default; svc_require_sudo is called early; binaries are copied to /root/hero/bin/; system-package and runtime-binary install steps run via ^sudo.
  • All commands accept --root(-r), --update(-u) where applicable, and --reset. start additionally accepts --mode (local|master|worker) and --master-ip <ip> matching the upstream Makefile.
  • Reuse where present:
    • services/lib.nu helpers (svc_require_sudo, svc_require_proc, svc_install, svc_install_binary, svc_drop_registration, svc_stop_service, svc_service_status, svc_start_preflight, svc_server_timing, svc_ui_timing, svc_server_health_policy, svc_ui_health_policy, svc_service_config, svc_require_binary, svc_bins_ok, svc_sock_base, svc_bin, svc_need_sudo).
    • tools/modules/platform.nu::pkg_ensure for Debian package install (matches installers.nu::install_base style).
    • service_mycelium install --root for the mycelium runtime dep (already in tree).
  • New local helpers (private to service_compute.nu, named with svx_ prefix per donor convention) for runtime-only deps not already wrapped:
    • svx_install_system_packages — apt-get installs libssl-dev, pkg-config, iproute2, busybox-static, virtiofsd, e2fsprogs, iptables, build-essential, musl-tools, qemu-utils. (Mirrors hero_compute scripts/configure.sh::install_system_packages.)
    • svx_install_cloud_hypervisor — download cloud-hypervisor-static from https://github.com/cloud-hypervisor/cloud-hypervisor/releases/download/v43.0/cloud-hypervisor-static to /usr/local/bin/cloud-hypervisor, chmod +x. Skip when already at the right version.
    • svx_install_my_hypervisor — download my_hypervisor (and my_hypervisor-init) release artifacts from https://forge.ourworld.tf/geomind_code/my_hypervisor/releases/download/v0.1.2/... to /usr/local/bin/my_hypervisor and ~/.my_hypervisor/bin/my_hypervisor-init, then run my_hypervisor kernel extract-host if no kernel is present.
    • svx_kvm_check — verify /dev/kvm exists; print a warning (non-fatal) if not (the hero_compute manager will fail at runtime with a clearer diagnostic).
  • start output must end with the standard summary block (per nu_service_use): service state, RPC sockets, UI URL (port 9001 by default), explorer URL (port 9002), worker RPC port (9003), Mycelium IPv6 (when available), and a copy/paste test plan with proc service status, proc logs tail hero_compute_server, proc job list --service hero_compute.
  • Every action spec must include kill_other (socket and/or port cleanup) and at least one health check, per hero_proc_service_selfstart.
  • cargo build must use --release (issue mandates) and limit to the four binaries via --bin flags or --workspace --release (donor uses per-bin --bin via svc_cargo_install); pass release: true through svc_install.

Files to Modify/Create

  • tools/modules/services/service_compute.nu — NEW. Full lifecycle module, ~450-550 lines, mirroring service_codescalers.nu.
  • tools/modules/services/mod.nu — MODIFY. Add export use service_compute.nu line in alphabetical/dependency order (next to service_codescalers.nu is fine).
  • tools/modules/services/packages.nu — OPTIONAL MODIFY. Add a service_compute entry to service_install_all only if the bundle install needs it (the issue does not mandate; can be skipped — note in PR).

Implementation Plan

Step 1: Add the module file with constants and helpers (header + naming + paths)

Files: tools/modules/services/service_compute.nu (NEW)

  • Add the module-doc comment block (mirror service_codescalers.nu lines 1-58 in shape) describing:
    • hero_compute as a four-binary stack (manager + server + ui + explorer).
    • Requires --root: KVM device access, raw networking, mycelium overlay membership. Document this prominently in the docstring (same callout style as service_mycelium.nu lines 1-58).
    • Multi-node modes (local|master|worker) and the --master-ip flag.
    • Runtime deps: cloud-hypervisor v43.0, my_hypervisor v0.1.2, mycelium v0.7.3, hero_proc (already managed by service_proc).
    • Notes that the runtime registry (hero_compute_registry) is fetched lazily by the manager — install does NOT clone it.
  • Add use ../clients/proc.nu *, use ./lib.nu *, use ../platform.nu [pkg_ensure], use ./service_mycelium.nu (for the mycelium install reuse).
  • Add constants block: 
    const SVX_SERVICE_NAME = "hero_compute"
const SVX_FORGE_LOC    = "lhumina_code/hero_compute"
const SVX_BINARIES     = ["hero_compute" "hero_compute_server" "hero_compute_ui" "hero_compute_explorer"]
const SVX_ACTIONS      = ["hero_compute_server" "hero_compute_ui" "hero_compute_explorer"]
const SVX_DESCRIPTION  = "Hero Compute — slice-based VM manager"
const SVX_CLOUD_HYPERVISOR_VERSION = "v43.0"
const SVX_MY_HYPERVISOR_VERSION    = "v0.1.2"
const SVX_UI_PORT        = 9001
const SVX_EXPLORER_PORT  = 9002
const SVX_WORKER_RPC_PORT = 9003
  • The manager hero_compute binary is in SVX_BINARIES (gets installed to bin dir) but is NOT a hero_proc action — same shape as hero_codescalers in the donor where the eponymous CLI binary is a binary but not an action.
  • No --instance support in v1 (issue does not require it). The donor's instance-aware naming helpers (svx_svc_name, svx_sock_dir) can be simplified to constants. Keep parameter slots for future symmetry but always pass 0.
    Dependencies: none.

Step 2: Add system-dep + runtime-binary helpers

Files: tools/modules/services/service_compute.nu

  • Add def svx_install_system_packages [] { ... }:
    • Skip on non-apt platforms (mirrors installers.nu::install_chrome short-circuit).
    • For each of ["build-essential" "libssl-dev" "pkg-config" "musl-tools" "iproute2" "iptables" "busybox-static" "virtiofsd" "e2fsprogs" "qemu-utils"], call pkg_ensure $pkg (or pkg_ensure_dev for -dev libs).
    • On Debian, after install, symlink /usr/lib/qemu/virtiofsd → /usr/local/bin/virtiofsd if needed (matches upstream configure.sh::install_system_packages).
  • Add def svx_install_cloud_hypervisor [] { ... }:
    • Check ^cloud-hypervisor --version and skip if it already reports the pinned version.
    • Else ^sudo curl -sfL <url> -o /usr/local/bin/cloud-hypervisor && ^sudo chmod +x ... where url is https://github.com/cloud-hypervisor/cloud-hypervisor/releases/download/v43.0/cloud-hypervisor-static.
  • Add def svx_install_my_hypervisor [] { ... }:
    • Download my_hypervisor and my_hypervisor-init from forge releases. Use the central-config URL form https://forge.ourworld.tf/geomind_code/my_hypervisor/releases/download/<version>/my_hypervisor-x86_64-unknown-linux-musl and the -init suffix variant. Place the main binary at /usr/local/bin/my_hypervisor, the init binary at ~/.my_hypervisor/bin/my_hypervisor-init.
    • Call my_hypervisor kernel list; if empty, call ^sudo my_hypervisor kernel extract-host and warn-only on failure (matches upstream configure.sh::check_kernel).
    • Call ^my_hypervisor doctor and print a warning summary listing any NOT FOUND items.
  • Add def svx_kvm_check [] { ... }:
    • Print warning if /dev/kvm is missing. Do not error out — the user may be running the install on a build host with no KVM.
  • All three helpers are idempotent: skip when the right version is already in place.
    Dependencies: Step 1.

Step 3: Add action spec builders for the three daemon binaries

Files: tools/modules/services/service_compute.nu

  • Each action follows the donor svx_server_action / svx_ui_action shape: name, script (path to binary), interpreter: "exec", is_process: true, RUST_LOG: "info", kill_other (socket + port), health check, retry policy from svc_server_timing or svc_ui_timing.
  • svx_server_action [root: bool] for hero_compute_server:
    • bin = (svc_bin "hero_compute_server" $root).
    • sock = $"($sock_base)/hero_compute/rpc.sock".
    • timing: svc_server_timing. Health: openrpc_socket on the rpc.sock with svc_server_health_policy.
    • kill_other: socket: [$sock_path], port: [$SVX_WORKER_RPC_PORT] (server binds the worker RPC port in master/worker mode).
    • env: RUST_LOG, plus pass through HERO_COMPUTE_MODE, HERO_COMPUTE_MASTER_IP, and MYCELIUM_IP (read from $env — start sets these before registering when --mode/--master-ip are provided).
  • svx_ui_action [root: bool] for hero_compute_ui:
    • bin = (svc_bin "hero_compute_ui" $root).
    • The UI binds TCP port 9001 (per install.sh "Open http://${ip}:9001").
    • kill_other: port: [$SVX_UI_PORT], socket: [$"($sock_base)/hero_compute/ui.sock"].
    • Health: prefer http_url: $"http://localhost:($SVX_UI_PORT)/" since the UI is a dual TCP+UDS service; fall back to openrpc_socket if no /health is reachable yet (use svc_ui_health_policy).
    • timing: svc_ui_timing (3 retries, 5s stop).
  • svx_explorer_action [root: bool] for hero_compute_explorer:
    • bin = (svc_bin "hero_compute_explorer" $root).
    • The explorer hub binds TCP port 9002.
    • kill_other: port: [$SVX_EXPLORER_PORT], socket: [$"($sock_base)/hero_compute/explorer.sock"].
    • Health: http_url: $"http://localhost:($SVX_EXPLORER_PORT)/" with svc_ui_health_policy.
    • timing: svc_ui_timing.
  • svx_service_config [] returns (svc_service_config $SVX_SERVICE_NAME $SVX_ACTIONS $SVX_DESCRIPTION).
    Dependencies: Step 1.

Step 4: Implement install (clone + system deps + runtime deps + cargo build + copy)

Files: tools/modules/services/service_compute.nu

  • Signature mirrors donor: export def install [--root(-r), --update(-u), --reset, --release].
  • Body order (each step idempotent):
    1. if $root { svc_require_sudo } (issue mandates --root).
    2. svx_kvm_check — warning only.
    3. svx_install_system_packages (apt + virtiofsd symlink).
    4. svx_install_cloud_hypervisor (download release binary).
    5. svx_install_my_hypervisor (download + kernel extract + doctor).
    6. service_mycelium install --root=$root --update=$update --reset=$reset (delegate; reuses existing module).
    7. svc_install $SVX_SERVICE_NAME $SVX_FORGE_LOC $SVX_BINARIES $root $update $reset $release — clones via forge_ensure_local, runs cargo build --release --bin hero_compute --bin hero_compute_server --bin hero_compute_ui --bin hero_compute_explorer, copies the four binaries to ~/hero/bin/ (and /root/hero/bin/ when --root).
    • Note for the CI/donor parity: pass release: true always, since hero_compute's Makefile builds release-only and the runtime depends on the optimized build for VM I/O throughput. We can keep the --release flag exposed in case a developer wants debug, defaulting to true instead of donor's false (minor deviation — call out in ### Notes).
  • After install, print a "Next steps" block matching the upstream installer:
    • service_compute start --root — single-node local mode.
    • service_compute start --root --mode master — explorer hub.
    • service_compute start --root --mode worker --master-ip <ip> — worker.
      Dependencies: Steps 2 + 3.

Step 5: Implement start (preflight + register + start + summary)

Files: tools/modules/services/service_compute.nu

  • Signature: export def start [--reset, --root(-r), --update(-u), --mode: string = "local", --master-ip: string = ""].
  • Validate: $mode in ["local" "master" "worker"]; if $mode == "worker" then $master_ip must be non-empty (else error make).
  • Body sequence — copy donor start exactly with these adaptations:
    1. if $root { svc_require_sudo } (always needed for hero_compute).
    2. svc_start_preflight $SVX_SERVICE_NAME "service_compute" $reset $update $root — early-exit if running and no --reset/--update.
    3. svc_drop_registration $SVX_SERVICE_NAME $SVX_ACTIONS $root.
    4. Call install --root=$root --update=$update --reset=$reset --release to ensure binaries.
    5. svc_require_binary "hero_compute_server" $root, same for _ui, _explorer, manager hero_compute.
    6. Build action specs with mode-aware env: temporarily set $env.HERO_COMPUTE_MODE = $mode, $env.HERO_COMPUTE_MASTER_IP = $master_ip before constructing (svx_server_action ...), OR pass mode/master_ip as parameters into the action builders so the env block bakes them in. (Cleaner: extend builder signatures to [root: bool, mode: string, master_ip: string].)
    7. proc action set for each of the three actions, then proc service set (svx_service_config), then proc service start $SVX_SERVICE_NAME --root=$root.
    8. Settle (sleep 3sec), then run the donor's "running? / svc_status / why blocked? / failed jobs / startup log" diagnostic block — copy from service_codescalers.nu lines 281-346 verbatim, adjusted for the action name hero_compute_server.
    9. Final summary block (matches nu_service_use requirements): 
      === hero_compute summary ===
  mode      : <mode>
  master_ip : <master_ip or "(none)">
  state     : running ✓
  rpc sock  : .../hero_compute/rpc.sock
  ui  sock  : .../hero_compute/ui.sock
  ui  url   : http://<host>:9001/
  explorer  : http://<host>:9002/   (master mode)
  worker rpc: <host>:9003           (master/worker)
  mycelium  : <ipv6>                (read from ~/hero/var/hero_compute.env if present)
  commands  :
    proc service status hero_compute --root
    proc logs tail hero_compute_server --root
    proc job list --service hero_compute --root
  • Read mycelium IP via svc_mycelium_address $root (already in lib.nu) for the summary.
    Dependencies: Steps 3 + 4.

Step 6: Implement stop and status

Files: tools/modules/services/service_compute.nu

  • export def stop [--root(-r)]: thin wrapper around svc_stop_service $SVX_SERVICE_NAME $SVX_ACTIONS $root (mirrors donor lines 378-383).
  • export def status [--root(-r)]: svc_service_status $SVX_SERVICE_NAME "service_compute" $root (mirrors donor lines 393-398).
    Dependencies: Step 1.

Step 7: (Optional) Implement reset

Files: tools/modules/services/service_compute.nu

  • Copy donor reset (service_codescalers.nu lines 418-496) verbatim, replacing hero_codescalers with hero_compute and the binary list. Strict additive — does NOT remove cloud-hypervisor / my_hypervisor / mycelium runtime deps (they are shared).
  • Print a final "Note: cloud-hypervisor, my_hypervisor, mycelium binaries left in place — remove them manually if desired."
    Dependencies: Step 1.

Step 8: Wire into mod.nu

Files: tools/modules/services/mod.nu

  • Add export use service_compute.nu line after export use service_codescalers.nu (line 7) so use services/mod.nu * exposes service_compute.
    Dependencies: Step 1.

Step 9: Smoke test (manual, by the executing agent)

Files: none (verification only)

  • use services/mod.nu * from a fresh nu shell — confirm service_compute install --help lists the flags.
  • On a KVM-enabled root host: service_compute install --root runs end-to-end (clone, system deps, runtime binaries, cargo build, copy, mycelium install delegation).
  • service_compute start --root → reads the printed summary, opens http://<host>:9001, runs proc service status hero_compute --root.
  • service_compute start --root --mode master → confirms explorer is reachable on :9002.
  • service_compute stop --root → unregistration succeeds; sockets are gone.
  • service_compute status --root → reports state cleanly.
    Dependencies: Steps 1-8.

Acceptance Criteria

  • use services/mod.nu * makes service_compute available (Step 8).
  • service_compute install --root clones the repo, installs system deps + cloud-hypervisor + my_hypervisor + mycelium, runs cargo build --release, copies the four binaries to ~/hero/bin/ (Steps 2 + 4).
  • service_compute start --root registers with hero_proc and starts the manager, server, UI, and explorer (Step 5).
  • service_compute status --root reports state (Step 6).
  • service_compute stop --root cleanly unregisters (Step 6).
  • start output prints sockets / UI URL / a short test plan per nu_service_use (Step 5 final summary block).
  • (Optional) start --root --mode master and --mode worker --master-ip <ip> mirror the upstream Makefile (Step 5 mode handling).
  • Each action spec includes kill_other (port + socket) and at least one health check (Step 3).
  • --root is documented as required in the module docstring; non-root start either errors (preferred) or prints a clear warning (Step 1 docstring + Step 5 preflight).

Notes

  • Donor deviation: --release defaults to true. The donor (service_codescalers.nu) defaults --release to false (debug for fast iteration). hero_compute's runtime is heavily I/O-bound (cloud-hypervisor + virtiofsd) and the upstream Makefile builds release-only. Default to release; keep the flag exposed for power users.
  • Donor deviation: no --instance support in v1. The donor uses an instance-numbered naming scheme. hero_compute does not support multiple instances on one host (the manager owns /dev/kvm and the mycelium TUN). Skip the helper functions and the parameter; revisit only if a real multi-instance need surfaces.
  • Mycelium reuse, not duplication. The service_mycelium install --root call inside service_compute install is the cleanest path — it already pulls/builds/installs the binaries and leaves service_mycelium start --root available to bring the daemon up. The hero_compute manager will pick up the daemon via the standard mycelium socket. Document this in the module header.
  • System packages helper duplication trade-off. installers.nu::install_base exists but installs a wide superset (incl. LibreOffice, ONNX, Docker). For hero_compute we want a narrow KVM-focused subset — duplicate the relevant pkg_ensure calls in svx_install_system_packages rather than calling install_base. The list (~10 packages) is short enough that duplication is cheaper than refactoring install_base into smaller pieces.
  • Runtime binary downloads use ^sudo curl. They go to /usr/local/bin/. The donor uses svc_install_binary for cargo build outputs; that helper assumes the source is local and runs through invoking-user permissions. For pre-built downloads we shell out directly with ^sudo curl ... and ^sudo chmod +x ... — same pattern as installers.nu::install_chrome (lines 230-241).
  • No hero_compute_registry clone. The issue and scripts/install.sh confirm the registry is fetched lazily by the manager binary at runtime — the install step does NOT need to clone or vendor it. Document this in the module header so future maintainers do not "fix" it.
  • The manager binary hero_compute is installed but not registered as an action. It is in SVX_BINARIES but NOT in SVX_ACTIONS. Same shape as hero_codescalers (binary hero_codescalers in BINARIES, not in ACTIONS — only _server and _ui are actions). The manager is a CLI entry point, not a daemon.
  • svc_install may rebuild more than four binaries. svc_cargo_install invokes cargo build --bin per name in SVX_BINARIES. Verify the upstream workspace does not have a binary named hero_compute outside the main crate that would clash — if so, fall back to cargo build --workspace --release like the upstream Makefile and adjust svc_cargo_install callers, OR add a thin wrapper helper. Defer the decision to implementation time; the four---bin form is preferred and should work given the upstream BINARIES="hero_compute hero_compute_server hero_compute_ui hero_compute_explorer" declaration.
  • mod.nu ordering. The new export goes after service_codescalers.nu (line 7) — alphabetical-ish order is loose but adjacency to its sibling makes diff review easy.
  • packages.nu update is OPTIONAL. The issue does not require including service_compute in the bulk install (service_install_all). Many hosts cannot run hero_compute (no KVM). Leave it out of the bulk installer; the user explicitly opts in via service_compute install --root.
## Implementation Spec for Issue #154 ### Objective Add `tools/modules/services/service_compute.nu` — a privileged Hero-service lifecycle module for the `hero_compute` stack — exposing `install`, `start [--reset]`, `stop`, `status` (plus `reset`), modeled on `service_codescalers.nu` and matching the privileged-service shape of `service_mycelium.nu`. The module registers four binaries (`hero_compute` manager + `hero_compute_server` + `hero_compute_ui` + `hero_compute_explorer`) with hero_proc, mirroring what the upstream Rust manager does internally, so existing `nu_service_use` workflows ( `service_compute start --root --reset` ) work end-to-end. ### Requirements - New file: `tools/modules/services/service_compute.nu`. No edits to existing service modules. - Re-export the new module in `tools/modules/services/mod.nu` so `use services/mod.nu *` exposes `service_compute`. - Follow the donor template `service_codescalers.nu` exactly — same constants, helper layout (`svx_*`), and section banners — adapted to four binaries instead of three. - Match the privileged-service shape from `service_mycelium.nu`: `--root` is the documented default; `svc_require_sudo` is called early; binaries are copied to `/root/hero/bin/`; system-package and runtime-binary install steps run via `^sudo`. - All commands accept `--root(-r)`, `--update(-u)` where applicable, and `--reset`. `start` additionally accepts `--mode (local|master|worker)` and `--master-ip <ip>` matching the upstream Makefile. - Reuse where present: - `services/lib.nu` helpers (`svc_require_sudo`, `svc_require_proc`, `svc_install`, `svc_install_binary`, `svc_drop_registration`, `svc_stop_service`, `svc_service_status`, `svc_start_preflight`, `svc_server_timing`, `svc_ui_timing`, `svc_server_health_policy`, `svc_ui_health_policy`, `svc_service_config`, `svc_require_binary`, `svc_bins_ok`, `svc_sock_base`, `svc_bin`, `svc_need_sudo`). - `tools/modules/platform.nu::pkg_ensure` for Debian package install (matches `installers.nu::install_base` style). - `service_mycelium install --root` for the mycelium runtime dep (already in tree). - New local helpers (private to `service_compute.nu`, named with `svx_` prefix per donor convention) for runtime-only deps not already wrapped: - `svx_install_system_packages` — apt-get installs `libssl-dev`, `pkg-config`, `iproute2`, `busybox-static`, `virtiofsd`, `e2fsprogs`, `iptables`, `build-essential`, `musl-tools`, `qemu-utils`. (Mirrors hero_compute `scripts/configure.sh::install_system_packages`.) - `svx_install_cloud_hypervisor` — download `cloud-hypervisor-static` from `https://github.com/cloud-hypervisor/cloud-hypervisor/releases/download/v43.0/cloud-hypervisor-static` to `/usr/local/bin/cloud-hypervisor`, chmod +x. Skip when already at the right version. - `svx_install_my_hypervisor` — download `my_hypervisor` (and `my_hypervisor-init`) release artifacts from `https://forge.ourworld.tf/geomind_code/my_hypervisor/releases/download/v0.1.2/...` to `/usr/local/bin/my_hypervisor` and `~/.my_hypervisor/bin/my_hypervisor-init`, then run `my_hypervisor kernel extract-host` if no kernel is present. - `svx_kvm_check` — verify `/dev/kvm` exists; print a warning (non-fatal) if not (the hero_compute manager will fail at runtime with a clearer diagnostic). - `start` output must end with the standard summary block (per `nu_service_use`): service state, RPC sockets, UI URL (port 9001 by default), explorer URL (port 9002), worker RPC port (9003), Mycelium IPv6 (when available), and a copy/paste test plan with `proc service status`, `proc logs tail hero_compute_server`, `proc job list --service hero_compute`. - Every action spec must include `kill_other` (socket and/or port cleanup) and at least one health check, per `hero_proc_service_selfstart`. - `cargo build` must use `--release` (issue mandates) and limit to the four binaries via `--bin` flags or `--workspace --release` (donor uses per-bin `--bin` via `svc_cargo_install`); pass `release: true` through `svc_install`. ### Files to Modify/Create - `tools/modules/services/service_compute.nu` — NEW. Full lifecycle module, ~450-550 lines, mirroring `service_codescalers.nu`. - `tools/modules/services/mod.nu` — MODIFY. Add `export use service_compute.nu` line in alphabetical/dependency order (next to `service_codescalers.nu` is fine). - `tools/modules/services/packages.nu` — OPTIONAL MODIFY. Add a `service_compute` entry to `service_install_all` only if the bundle install needs it (the issue does not mandate; can be skipped — note in PR). ### Implementation Plan #### Step 1: Add the module file with constants and helpers (header + naming + paths) Files: `tools/modules/services/service_compute.nu` (NEW) - Add the module-doc comment block (mirror `service_codescalers.nu` lines 1-58 in shape) describing: - hero_compute as a four-binary stack (manager + server + ui + explorer). - **Requires `--root`**: KVM device access, raw networking, mycelium overlay membership. Document this prominently in the docstring (same callout style as `service_mycelium.nu` lines 1-58). - Multi-node modes (`local|master|worker`) and the `--master-ip` flag. - Runtime deps: `cloud-hypervisor` v43.0, `my_hypervisor` v0.1.2, `mycelium` v0.7.3, `hero_proc` (already managed by `service_proc`). - Notes that the runtime registry (`hero_compute_registry`) is fetched lazily by the manager — install does NOT clone it. - Add `use ../clients/proc.nu *`, `use ./lib.nu *`, `use ../platform.nu [pkg_ensure]`, `use ./service_mycelium.nu` (for the mycelium install reuse). - Add constants block: ``` const SVX_SERVICE_NAME = "hero_compute" const SVX_FORGE_LOC = "lhumina_code/hero_compute" const SVX_BINARIES = ["hero_compute" "hero_compute_server" "hero_compute_ui" "hero_compute_explorer"] const SVX_ACTIONS = ["hero_compute_server" "hero_compute_ui" "hero_compute_explorer"] const SVX_DESCRIPTION = "Hero Compute — slice-based VM manager" const SVX_CLOUD_HYPERVISOR_VERSION = "v43.0" const SVX_MY_HYPERVISOR_VERSION = "v0.1.2" const SVX_UI_PORT = 9001 const SVX_EXPLORER_PORT = 9002 const SVX_WORKER_RPC_PORT = 9003 ``` - The manager `hero_compute` binary is in `SVX_BINARIES` (gets installed to bin dir) but is NOT a hero_proc action — same shape as `hero_codescalers` in the donor where the eponymous CLI binary is a binary but not an action. - No `--instance` support in v1 (issue does not require it). The donor's instance-aware naming helpers (`svx_svc_name`, `svx_sock_dir`) can be simplified to constants. Keep parameter slots for future symmetry but always pass `0`. Dependencies: none. #### Step 2: Add system-dep + runtime-binary helpers Files: `tools/modules/services/service_compute.nu` - Add `def svx_install_system_packages [] { ... }`: - Skip on non-apt platforms (mirrors `installers.nu::install_chrome` short-circuit). - For each of `["build-essential" "libssl-dev" "pkg-config" "musl-tools" "iproute2" "iptables" "busybox-static" "virtiofsd" "e2fsprogs" "qemu-utils"]`, call `pkg_ensure $pkg` (or `pkg_ensure_dev` for `-dev` libs). - On Debian, after install, symlink `/usr/lib/qemu/virtiofsd → /usr/local/bin/virtiofsd` if needed (matches upstream `configure.sh::install_system_packages`). - Add `def svx_install_cloud_hypervisor [] { ... }`: - Check `^cloud-hypervisor --version` and skip if it already reports the pinned version. - Else `^sudo curl -sfL <url> -o /usr/local/bin/cloud-hypervisor && ^sudo chmod +x ...` where url is `https://github.com/cloud-hypervisor/cloud-hypervisor/releases/download/v43.0/cloud-hypervisor-static`. - Add `def svx_install_my_hypervisor [] { ... }`: - Download `my_hypervisor` and `my_hypervisor-init` from forge releases. Use the central-config URL form `https://forge.ourworld.tf/geomind_code/my_hypervisor/releases/download/<version>/my_hypervisor-x86_64-unknown-linux-musl` and the `-init` suffix variant. Place the main binary at `/usr/local/bin/my_hypervisor`, the init binary at `~/.my_hypervisor/bin/my_hypervisor-init`. - Call `my_hypervisor kernel list`; if empty, call `^sudo my_hypervisor kernel extract-host` and warn-only on failure (matches upstream `configure.sh::check_kernel`). - Call `^my_hypervisor doctor` and print a warning summary listing any `NOT FOUND` items. - Add `def svx_kvm_check [] { ... }`: - Print warning if `/dev/kvm` is missing. Do not error out — the user may be running the install on a build host with no KVM. - All three helpers are **idempotent**: skip when the right version is already in place. Dependencies: Step 1. #### Step 3: Add action spec builders for the three daemon binaries Files: `tools/modules/services/service_compute.nu` - Each action follows the donor `svx_server_action` / `svx_ui_action` shape: name, script (path to binary), `interpreter: "exec"`, `is_process: true`, `RUST_LOG: "info"`, kill_other (socket + port), health check, retry policy from `svc_server_timing` or `svc_ui_timing`. - `svx_server_action [root: bool]` for `hero_compute_server`: - bin = `(svc_bin "hero_compute_server" $root)`. - sock = `$"($sock_base)/hero_compute/rpc.sock"`. - timing: `svc_server_timing`. Health: `openrpc_socket` on the rpc.sock with `svc_server_health_policy`. - kill_other: `socket: [$sock_path]`, `port: [$SVX_WORKER_RPC_PORT]` (server binds the worker RPC port in master/worker mode). - env: `RUST_LOG`, plus pass through `HERO_COMPUTE_MODE`, `HERO_COMPUTE_MASTER_IP`, and `MYCELIUM_IP` (read from `$env` — `start` sets these before registering when `--mode`/`--master-ip` are provided). - `svx_ui_action [root: bool]` for `hero_compute_ui`: - bin = `(svc_bin "hero_compute_ui" $root)`. - The UI binds TCP port 9001 (per `install.sh` "Open http://${ip}:9001"). - kill_other: `port: [$SVX_UI_PORT]`, `socket: [$"($sock_base)/hero_compute/ui.sock"]`. - Health: prefer `http_url: $"http://localhost:($SVX_UI_PORT)/"` since the UI is a dual TCP+UDS service; fall back to `openrpc_socket` if no /health is reachable yet (use `svc_ui_health_policy`). - timing: `svc_ui_timing` (3 retries, 5s stop). - `svx_explorer_action [root: bool]` for `hero_compute_explorer`: - bin = `(svc_bin "hero_compute_explorer" $root)`. - The explorer hub binds TCP port 9002. - kill_other: `port: [$SVX_EXPLORER_PORT]`, `socket: [$"($sock_base)/hero_compute/explorer.sock"]`. - Health: `http_url: $"http://localhost:($SVX_EXPLORER_PORT)/"` with `svc_ui_health_policy`. - timing: `svc_ui_timing`. - `svx_service_config []` returns `(svc_service_config $SVX_SERVICE_NAME $SVX_ACTIONS $SVX_DESCRIPTION)`. Dependencies: Step 1. #### Step 4: Implement `install` (clone + system deps + runtime deps + cargo build + copy) Files: `tools/modules/services/service_compute.nu` - Signature mirrors donor: `export def install [--root(-r), --update(-u), --reset, --release]`. - Body order (each step idempotent): 1. `if $root { svc_require_sudo }` (issue mandates `--root`). 2. `svx_kvm_check` — warning only. 3. `svx_install_system_packages` (apt + virtiofsd symlink). 4. `svx_install_cloud_hypervisor` (download release binary). 5. `svx_install_my_hypervisor` (download + kernel extract + doctor). 6. `service_mycelium install --root=$root --update=$update --reset=$reset` (delegate; reuses existing module). 7. `svc_install $SVX_SERVICE_NAME $SVX_FORGE_LOC $SVX_BINARIES $root $update $reset $release` — clones via `forge_ensure_local`, runs `cargo build --release --bin hero_compute --bin hero_compute_server --bin hero_compute_ui --bin hero_compute_explorer`, copies the four binaries to `~/hero/bin/` (and `/root/hero/bin/` when `--root`). - Note for the CI/donor parity: pass `release: true` always, since hero_compute's Makefile builds release-only and the runtime depends on the optimized build for VM I/O throughput. We can keep the `--release` flag exposed in case a developer wants debug, defaulting to `true` instead of donor's `false` (minor deviation — call out in `### Notes`). - After install, print a "Next steps" block matching the upstream installer: - `service_compute start --root` — single-node local mode. - `service_compute start --root --mode master` — explorer hub. - `service_compute start --root --mode worker --master-ip <ip>` — worker. Dependencies: Steps 2 + 3. #### Step 5: Implement `start` (preflight + register + start + summary) Files: `tools/modules/services/service_compute.nu` - Signature: `export def start [--reset, --root(-r), --update(-u), --mode: string = "local", --master-ip: string = ""]`. - Validate: `$mode in ["local" "master" "worker"]`; if `$mode == "worker"` then `$master_ip` must be non-empty (else `error make`). - Body sequence — copy donor `start` exactly with these adaptations: 1. `if $root { svc_require_sudo }` (always needed for hero_compute). 2. `svc_start_preflight $SVX_SERVICE_NAME "service_compute" $reset $update $root` — early-exit if running and no `--reset`/`--update`. 3. `svc_drop_registration $SVX_SERVICE_NAME $SVX_ACTIONS $root`. 4. Call `install --root=$root --update=$update --reset=$reset --release` to ensure binaries. 5. `svc_require_binary "hero_compute_server" $root`, same for `_ui`, `_explorer`, manager `hero_compute`. 6. Build action specs with mode-aware env: temporarily set `$env.HERO_COMPUTE_MODE = $mode`, `$env.HERO_COMPUTE_MASTER_IP = $master_ip` before constructing `(svx_server_action ...)`, OR pass `mode`/`master_ip` as parameters into the action builders so the env block bakes them in. (Cleaner: extend builder signatures to `[root: bool, mode: string, master_ip: string]`.) 7. `proc action set` for each of the three actions, then `proc service set (svx_service_config)`, then `proc service start $SVX_SERVICE_NAME --root=$root`. 8. Settle (`sleep 3sec`), then run the donor's "running? / svc_status / why blocked? / failed jobs / startup log" diagnostic block — copy from `service_codescalers.nu` lines 281-346 verbatim, adjusted for the action name `hero_compute_server`. 9. Final summary block (matches `nu_service_use` requirements): ``` === hero_compute summary === mode : <mode> master_ip : <master_ip or "(none)"> state : running ✓ rpc sock : .../hero_compute/rpc.sock ui sock : .../hero_compute/ui.sock ui url : http://<host>:9001/ explorer : http://<host>:9002/ (master mode) worker rpc: <host>:9003 (master/worker) mycelium : <ipv6> (read from ~/hero/var/hero_compute.env if present) commands : proc service status hero_compute --root proc logs tail hero_compute_server --root proc job list --service hero_compute --root ``` - Read mycelium IP via `svc_mycelium_address $root` (already in `lib.nu`) for the summary. Dependencies: Steps 3 + 4. #### Step 6: Implement `stop` and `status` Files: `tools/modules/services/service_compute.nu` - `export def stop [--root(-r)]`: thin wrapper around `svc_stop_service $SVX_SERVICE_NAME $SVX_ACTIONS $root` (mirrors donor lines 378-383). - `export def status [--root(-r)]`: `svc_service_status $SVX_SERVICE_NAME "service_compute" $root` (mirrors donor lines 393-398). Dependencies: Step 1. #### Step 7: (Optional) Implement `reset` Files: `tools/modules/services/service_compute.nu` - Copy donor `reset` (`service_codescalers.nu` lines 418-496) verbatim, replacing `hero_codescalers` with `hero_compute` and the binary list. Strict additive — does NOT remove cloud-hypervisor / my_hypervisor / mycelium runtime deps (they are shared). - Print a final "Note: cloud-hypervisor, my_hypervisor, mycelium binaries left in place — remove them manually if desired." Dependencies: Step 1. #### Step 8: Wire into mod.nu Files: `tools/modules/services/mod.nu` - Add `export use service_compute.nu` line after `export use service_codescalers.nu` (line 7) so `use services/mod.nu *` exposes `service_compute`. Dependencies: Step 1. #### Step 9: Smoke test (manual, by the executing agent) Files: none (verification only) - `use services/mod.nu *` from a fresh nu shell — confirm `service_compute install --help` lists the flags. - On a KVM-enabled root host: `service_compute install --root` runs end-to-end (clone, system deps, runtime binaries, cargo build, copy, mycelium install delegation). - `service_compute start --root` → reads the printed summary, opens `http://<host>:9001`, runs `proc service status hero_compute --root`. - `service_compute start --root --mode master` → confirms explorer is reachable on `:9002`. - `service_compute stop --root` → unregistration succeeds; sockets are gone. - `service_compute status --root` → reports state cleanly. Dependencies: Steps 1-8. ### Acceptance Criteria - [ ] `use services/mod.nu *` makes `service_compute` available (Step 8). - [ ] `service_compute install --root` clones the repo, installs system deps + cloud-hypervisor + my_hypervisor + mycelium, runs `cargo build --release`, copies the four binaries to `~/hero/bin/` (Steps 2 + 4). - [ ] `service_compute start --root` registers with hero_proc and starts the manager, server, UI, and explorer (Step 5). - [ ] `service_compute status --root` reports state (Step 6). - [ ] `service_compute stop --root` cleanly unregisters (Step 6). - [ ] `start` output prints sockets / UI URL / a short test plan per `nu_service_use` (Step 5 final summary block). - [ ] (Optional) `start --root --mode master` and `--mode worker --master-ip <ip>` mirror the upstream Makefile (Step 5 mode handling). - [ ] Each action spec includes `kill_other` (port + socket) and at least one health check (Step 3). - [ ] `--root` is documented as required in the module docstring; non-root `start` either errors (preferred) or prints a clear warning (Step 1 docstring + Step 5 preflight). ### Notes - **Donor deviation: `--release` defaults to true.** The donor (`service_codescalers.nu`) defaults `--release` to false (debug for fast iteration). hero_compute's runtime is heavily I/O-bound (cloud-hypervisor + virtiofsd) and the upstream Makefile builds release-only. Default to release; keep the flag exposed for power users. - **Donor deviation: no `--instance` support in v1.** The donor uses an instance-numbered naming scheme. hero_compute does not support multiple instances on one host (the manager owns `/dev/kvm` and the mycelium TUN). Skip the helper functions and the parameter; revisit only if a real multi-instance need surfaces. - **Mycelium reuse, not duplication.** The `service_mycelium install --root` call inside `service_compute install` is the cleanest path — it already pulls/builds/installs the binaries and leaves `service_mycelium start --root` available to bring the daemon up. The hero_compute manager will pick up the daemon via the standard mycelium socket. Document this in the module header. - **System packages helper duplication trade-off.** `installers.nu::install_base` exists but installs a wide superset (incl. LibreOffice, ONNX, Docker). For hero_compute we want a narrow KVM-focused subset — duplicate the relevant `pkg_ensure` calls in `svx_install_system_packages` rather than calling `install_base`. The list (~10 packages) is short enough that duplication is cheaper than refactoring `install_base` into smaller pieces. - **Runtime binary downloads use `^sudo curl`.** They go to `/usr/local/bin/`. The donor uses `svc_install_binary` for cargo build outputs; that helper assumes the source is local and runs through invoking-user permissions. For pre-built downloads we shell out directly with `^sudo curl ...` and `^sudo chmod +x ...` — same pattern as `installers.nu::install_chrome` (lines 230-241). - **No `hero_compute_registry` clone.** The issue and `scripts/install.sh` confirm the registry is fetched lazily by the manager binary at runtime — the install step does NOT need to clone or vendor it. Document this in the module header so future maintainers do not "fix" it. - **The manager binary `hero_compute` is installed but not registered as an action.** It is in `SVX_BINARIES` but NOT in `SVX_ACTIONS`. Same shape as `hero_codescalers` (binary `hero_codescalers` in BINARIES, not in ACTIONS — only `_server` and `_ui` are actions). The manager is a CLI entry point, not a daemon. - **`svc_install` may rebuild more than four binaries.** `svc_cargo_install` invokes `cargo build --bin` per name in `SVX_BINARIES`. Verify the upstream workspace does not have a binary named `hero_compute` outside the main crate that would clash — if so, fall back to `cargo build --workspace --release` like the upstream Makefile and adjust `svc_cargo_install` callers, OR add a thin wrapper helper. Defer the decision to implementation time; the four-`--bin` form is preferred and should work given the upstream `BINARIES="hero_compute hero_compute_server hero_compute_ui hero_compute_explorer"` declaration. - **mod.nu ordering.** The new export goes after `service_codescalers.nu` (line 7) — alphabetical-ish order is loose but adjacency to its sibling makes diff review easy. - **packages.nu update is OPTIONAL.** The issue does not require including `service_compute` in the bulk install (`service_install_all`). Many hosts cannot run hero_compute (no KVM). Leave it out of the bulk installer; the user explicitly opts in via `service_compute install --root`.
fatmaebrahim commented 2026-04-29 07:32:40 +00:00
Member
Copy link

Test Results

This PR adds a Nushell module; there is no automated test suite that exercises the install/start/stop/status/reset lifecycle without root + KVM. The verification below is what can be run statically.

Parse check

  • service_compute.nu parse: ok
  • services/mod.nu * re-export parse: ok

Help-surface check

  • service_compute install — signature loads: ok
  • service_compute start — signature loads: ok
  • service_compute stop — signature loads: ok
  • service_compute status — signature loads: ok
  • service_compute reset — signature loads: ok

Runtime exercising

End-to-end install / start / stop / status verification requires:

  • KVM-enabled host (/dev/kvm available)
  • --root privileges
  • network access to forge.ourworld.tf for the hero_compute clone + my_hypervisor release download
  • network access to github.com for the cloud-hypervisor v43.0 release download

These cannot be run from the development environment. Acceptance criteria 2-7 in the spec are covered by manual smoke testing on a target host (issue spec, Step 9).

CI hooks

The existing CI workflow (.forgejo/workflows/build.yaml) only runs bash scripts/test.sh, which validates SKILL.md frontmatter under $SKILLS_DIR (claude/skills). It does not parse or load anything under tools/modules/, so a syntax regression in service_compute.nu or services/mod.nu would not be caught by current CI. The repo's only other test script, scripts/test_forge_identity.nu, is unrelated (covers forge_secrets_toml_get and forge_git_ensure_identity). Recommend adding nu -c 'use ./tools/modules/services/mod.nu *' as a smoke step in a follow-up so the nu module surface is gated by CI.

## Test Results This PR adds a Nushell module; there is no automated test suite that exercises the install/start/stop/status/reset lifecycle without root + KVM. The verification below is what can be run statically. ### Parse check - `service_compute.nu` parse: ok - `services/mod.nu *` re-export parse: ok ### Help-surface check - `service_compute install` — signature loads: ok - `service_compute start` — signature loads: ok - `service_compute stop` — signature loads: ok - `service_compute status` — signature loads: ok - `service_compute reset` — signature loads: ok ### Runtime exercising End-to-end install / start / stop / status verification requires: - KVM-enabled host (`/dev/kvm` available) - `--root` privileges - network access to forge.ourworld.tf for the `hero_compute` clone + my_hypervisor release download - network access to github.com for the cloud-hypervisor v43.0 release download These cannot be run from the development environment. Acceptance criteria 2-7 in the spec are covered by manual smoke testing on a target host (issue spec, Step 9). ### CI hooks The existing CI workflow (`.forgejo/workflows/build.yaml`) only runs `bash scripts/test.sh`, which validates `SKILL.md` frontmatter under `$SKILLS_DIR` (`claude/skills`). It does not parse or load anything under `tools/modules/`, so a syntax regression in `service_compute.nu` or `services/mod.nu` would not be caught by current CI. The repo's only other test script, `scripts/test_forge_identity.nu`, is unrelated (covers `forge_secrets_toml_get` and `forge_git_ensure_identity`). Recommend adding `nu -c 'use ./tools/modules/services/mod.nu *'` as a smoke step in a follow-up so the nu module surface is gated by CI.
fatmaebrahim commented 2026-04-29 07:35:29 +00:00
Member
Copy link

Implementation Summary

Files

  • tools/modules/services/service_compute.nu — new, 766 lines. Lifecycle module for the hero_compute four-binary stack (hero_compute manager + hero_compute_server + hero_compute_ui + hero_compute_explorer). Modeled on service_codescalers.nu, with the privileged-service shape from service_mycelium.nu.
  • tools/modules/services/mod.nu — one-line addition: export use service_compute.nu placed after service_codescalers.nu for diff readability.
  • tools/modules/services/packages.nu — intentionally NOT modified. Bulk install is opt-in only; many hosts cannot run hero_compute (no KVM).

What it does

  • service_compute install [--root] [--update] [--reset] [--debug] — installs the host system packages (libssl-dev, pkg-config, iproute2, busybox-static, virtiofsd, e2fsprogs, iptables, build-essential, musl-tools, qemu-utils), downloads cloud-hypervisor v43.0 to /usr/local/bin/, downloads my_hypervisor v0.1.2 (main + init binaries) and runs kernel extract-host + doctor, delegates the mycelium runtime install to service_mycelium install --root, then runs cargo build --release for the four hero_compute binaries via svc_install. All steps are idempotent.
  • service_compute start [--root] [--reset] [--update] [--mode <local|master|worker>] [--master-ip <ip>] — preflight, drop existing registration, ensure binaries via install, register the three daemon actions and the service with hero_proc, start, settle, then print a summary block (mode, master_ip, state, sockets, UI URL, explorer URL, worker RPC, mycelium IPv6, copy/paste test plan). Mode/master-ip are threaded into the per-action env (HERO_COMPUTE_MODE, HERO_COMPUTE_MASTER_IP).
  • service_compute stop [--root] — wraps svc_stop_service.
  • service_compute status [--root] — wraps svc_service_status.
  • service_compute reset [--root] — stops everything, removes the four hero_compute binaries from the bin dir; intentionally leaves cloud-hypervisor / my_hypervisor / mycelium runtime binaries in place.

Action specs

Each daemon action (hero_compute_server, hero_compute_ui, hero_compute_explorer) carries:

  • kill_other covering its socket and TCP port (rpc.sock + 9003 for the server, ui.sock + 9001 for the UI, explorer.sock + 9002 for the explorer)
  • A health check using svc_server_health_policy (RPC) or svc_ui_health_policy (HTTP)
  • Retry/timing from svc_server_timing / svc_ui_timing
  • RUST_LOG=info,..., HERO_SOCKET_DIR, HERO_COMPUTE_SOCK_NAME, mode, master_ip, mycelium IP

Deviations from the spec (called out for review)

  1. The --release flag became --debug (inverted polarity). Nushell does not allow type annotations on switch flags, so a default-true --release is not expressible. --debug defaults absent (release build) and is documented in install's docstring.
  2. --instance support omitted entirely (per spec note). hero_compute owns /dev/kvm and the mycelium TUN — multiple instances on one host is not a real use case.
  3. --debug is NOT threaded through start → install. A debug build requires explicit service_compute install --debug first, then service_compute start --root.

Verification

  • nu -c 'use ./tools/modules/services/service_compute.nu; print "parse ok"' — clean.
  • nu -c 'use ./tools/modules/services/mod.nu *; print "mod parse ok"' — clean.
  • help service_compute install/start/stop/status/reset — all signatures render cleanly.

End-to-end verification (install → start → status → stop) requires /dev/kvm, --root, and network access to forge.ourworld.tf + github.com. That is the manual smoke-testing scope from Step 9 of the spec.

Notes for the reviewer

  • The CI workflow (.forgejo/workflows/build.yaml) only runs scripts/test.sh, which checks SKILL.md frontmatter — it does NOT parse-check the tools/modules/services/*.nu files. A regression in this module would slip past CI. Worth adding a nu -c 'use ./tools/modules/services/mod.nu *' smoke step in a follow-up.
  • The hero_compute_registry runtime tree is fetched lazily by the manager binary at start time — install does NOT clone it. This is documented in the module's header comment.
## Implementation Summary ### Files - `tools/modules/services/service_compute.nu` — new, 766 lines. Lifecycle module for the hero_compute four-binary stack (`hero_compute` manager + `hero_compute_server` + `hero_compute_ui` + `hero_compute_explorer`). Modeled on `service_codescalers.nu`, with the privileged-service shape from `service_mycelium.nu`. - `tools/modules/services/mod.nu` — one-line addition: `export use service_compute.nu` placed after `service_codescalers.nu` for diff readability. - `tools/modules/services/packages.nu` — intentionally NOT modified. Bulk install is opt-in only; many hosts cannot run hero_compute (no KVM). ### What it does - `service_compute install [--root] [--update] [--reset] [--debug]` — installs the host system packages (libssl-dev, pkg-config, iproute2, busybox-static, virtiofsd, e2fsprogs, iptables, build-essential, musl-tools, qemu-utils), downloads cloud-hypervisor v43.0 to `/usr/local/bin/`, downloads my_hypervisor v0.1.2 (main + init binaries) and runs `kernel extract-host` + `doctor`, delegates the mycelium runtime install to `service_mycelium install --root`, then runs `cargo build --release` for the four hero_compute binaries via `svc_install`. All steps are idempotent. - `service_compute start [--root] [--reset] [--update] [--mode <local|master|worker>] [--master-ip <ip>]` — preflight, drop existing registration, ensure binaries via `install`, register the three daemon actions and the service with hero_proc, start, settle, then print a summary block (mode, master_ip, state, sockets, UI URL, explorer URL, worker RPC, mycelium IPv6, copy/paste test plan). Mode/master-ip are threaded into the per-action env (`HERO_COMPUTE_MODE`, `HERO_COMPUTE_MASTER_IP`). - `service_compute stop [--root]` — wraps `svc_stop_service`. - `service_compute status [--root]` — wraps `svc_service_status`. - `service_compute reset [--root]` — stops everything, removes the four hero_compute binaries from the bin dir; intentionally leaves cloud-hypervisor / my_hypervisor / mycelium runtime binaries in place. ### Action specs Each daemon action (`hero_compute_server`, `hero_compute_ui`, `hero_compute_explorer`) carries: - `kill_other` covering its socket and TCP port (rpc.sock + 9003 for the server, ui.sock + 9001 for the UI, explorer.sock + 9002 for the explorer) - A health check using `svc_server_health_policy` (RPC) or `svc_ui_health_policy` (HTTP) - Retry/timing from `svc_server_timing` / `svc_ui_timing` - `RUST_LOG=info,...`, `HERO_SOCKET_DIR`, `HERO_COMPUTE_SOCK_NAME`, mode, master_ip, mycelium IP ### Deviations from the spec (called out for review) 1. The `--release` flag became `--debug` (inverted polarity). Nushell does not allow type annotations on switch flags, so a default-`true` `--release` is not expressible. `--debug` defaults absent (release build) and is documented in `install`'s docstring. 2. `--instance` support omitted entirely (per spec note). hero_compute owns `/dev/kvm` and the mycelium TUN — multiple instances on one host is not a real use case. 3. `--debug` is NOT threaded through `start → install`. A debug build requires explicit `service_compute install --debug` first, then `service_compute start --root`. ### Verification - `nu -c 'use ./tools/modules/services/service_compute.nu; print "parse ok"'` — clean. - `nu -c 'use ./tools/modules/services/mod.nu *; print "mod parse ok"'` — clean. - `help service_compute install/start/stop/status/reset` — all signatures render cleanly. End-to-end verification (install → start → status → stop) requires `/dev/kvm`, `--root`, and network access to forge.ourworld.tf + github.com. That is the manual smoke-testing scope from Step 9 of the spec. ### Notes for the reviewer - The CI workflow (`.forgejo/workflows/build.yaml`) only runs `scripts/test.sh`, which checks SKILL.md frontmatter — it does NOT parse-check the `tools/modules/services/*.nu` files. A regression in this module would slip past CI. Worth adding a `nu -c 'use ./tools/modules/services/mod.nu *'` smoke step in a follow-up. - The `hero_compute_registry` runtime tree is fetched lazily by the manager binary at start time — `install` does NOT clone it. This is documented in the module's header comment.
fatmaebrahim commented 2026-04-29 07:37:18 +00:00
Member
Copy link

Pull request opened: #169

This PR implements the changes discussed in this issue.

Pull request opened: https://forge.ourworld.tf/lhumina_code/hero_skills/pulls/169 This PR implements the changes discussed in this issue.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
development
development_timur
docs/setup-bootstrap-adr
main
lab-latest
latest
Labels
Clear labels   
prio_critical

   
prio_low

   
type_bug

   
type_contact

   
type_issue

   
type_lead

   
type_question

   
type_story

   
type_task
No labels
prio_critical
prio_low
type_bug
type_contact
type_issue
type_lead
type_question
type_story
type_task
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
fatmaebrahim
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lhumina_code/hero_skills#154
No description provided.