lab service --start --ephemeral leaks PATH_VAR/PATH_BUILD/PATH_CODE into the child + acquire chatter pollutes --json stdout #300

New issue

Closed

opened 2026-05-25 10:29:32 +00:00 by timur · 1 comment

timur commented

2026-05-25 10:29:32 +00:00

Owner

Context

lab service <name> --start --ephemeral --json overrides PATH_ROOT on the spawned child (so hero_var_dir() lands under scratch) but leaves PATH_VAR / PATH_BUILD / PATH_CODE inherited from the parent. herolib_core::base::path_var() checks PATH_VAR first and only falls back to $PATH_ROOT/var, so resolve_socket_dir() resolves to $HOME/hero/var/sockets — the shared path — even though OSIS storage correctly lands under scratch.

Result: the child happily binds $HOME/hero/var/sockets/<svc>/rpc.sock while lab waits for <scratch>/var/sockets/<svc>/rpc.sock, hits the 5s deadline, and bails with stderr tail: <empty> (the banner that would reveal the mismatch only ever lands in stdout.log, which lab doesn't surface).

Reproducible by following the post-#138 / hero_service#12 E2E flow on a contributor box that exports PATH_VAR=$HOME/hero/var (the canonical lab user init setup):

cd hero_service
PATH=$HOME/.local/bin:$PATH \
  PATH_ROOT=$HOME/hero PATH_VAR=$HOME/hero/var \
  PATH_BUILD=$HOME/hero/build PATH_CODE=$HOME/hero/code \
  cargo test --workspace --test greeter_e2e -- --test-threads=1

Each test panics with ephemeral socket <scratch>/var/sockets/hero_service/rpc.sock did not come up within 5s.

A second, smaller bug compounds the first: acquire_binary writes its using installed … / installed from Forge → … progress lines to stdout, so even when the spawn succeeds the JSON envelope lab --json emits is preceded by chatter — serde_json::from_slice on the test driver side rejects it.

Goals

crates/lab/src/service/ephemeral.rs::spawn_inner clears PATH_VAR / PATH_BUILD / PATH_CODE (alongside the existing HERO_SOCKET_DIR / INTEGRATIONTEST removal) so the child's path resolvers honour the single PATH_ROOT pivot.
Spawn-timeout failure messages tail both stdout.log and stderr.log (the startup banner that fingerprints env-pivot bugs lives in stdout).
acquire_binary's progress chatter (using installed, installed from Forge) routes to stderr so --json stdout stays a single line.
cargo test --workspace --test greeter_e2e (hero_service @ fe8c78ad) goes 5/5 green on a stock contributor environment.

## Context `lab service <name> --start --ephemeral --json` overrides `PATH_ROOT` on the spawned child (so `hero_var_dir()` lands under scratch) but leaves `PATH_VAR` / `PATH_BUILD` / `PATH_CODE` inherited from the parent. `herolib_core::base::path_var()` checks `PATH_VAR` *first* and only falls back to `$PATH_ROOT/var`, so `resolve_socket_dir()` resolves to `$HOME/hero/var/sockets` — the shared path — even though OSIS storage correctly lands under scratch. Result: the child happily binds `$HOME/hero/var/sockets/<svc>/rpc.sock` while lab waits for `<scratch>/var/sockets/<svc>/rpc.sock`, hits the 5s deadline, and bails with `stderr tail: <empty>` (the banner that would reveal the mismatch only ever lands in `stdout.log`, which lab doesn't surface). Reproducible by following the post-#138 / hero_service#12 E2E flow on a contributor box that exports `PATH_VAR=$HOME/hero/var` (the canonical `lab user init` setup): ``` cd hero_service PATH=$HOME/.local/bin:$PATH \ PATH_ROOT=$HOME/hero PATH_VAR=$HOME/hero/var \ PATH_BUILD=$HOME/hero/build PATH_CODE=$HOME/hero/code \ cargo test --workspace --test greeter_e2e -- --test-threads=1 ``` Each test panics with `ephemeral socket <scratch>/var/sockets/hero_service/rpc.sock did not come up within 5s`. A second, smaller bug compounds the first: `acquire_binary` writes its ` using installed …` / ` installed from Forge → …` progress lines to **stdout**, so even when the spawn succeeds the JSON envelope `lab --json` emits is preceded by chatter — `serde_json::from_slice` on the test driver side rejects it. ## Goals - `crates/lab/src/service/ephemeral.rs::spawn_inner` clears `PATH_VAR` / `PATH_BUILD` / `PATH_CODE` (alongside the existing `HERO_SOCKET_DIR` / `INTEGRATIONTEST` removal) so the child's path resolvers honour the single `PATH_ROOT` pivot. - Spawn-timeout failure messages tail **both** `stdout.log` and `stderr.log` (the startup banner that fingerprints env-pivot bugs lives in stdout). - `acquire_binary`'s progress chatter (`using installed`, `installed from Forge`) routes to stderr so `--json` stdout stays a single line. - `cargo test --workspace --test greeter_e2e` (hero_service @ `fe8c78ad`) goes 5/5 green on a stock contributor environment.

timur referenced this issue from a commit

2026-05-25 10:31:44 +00:00

fix(lab): clear PATH_VAR/BUILD/CODE on ephemeral child + route acquire chatter to stderr

timur referenced this issue

2026-05-25 10:32:05 +00:00

fix(lab): clear PATH_VAR/BUILD/CODE on ephemeral child + route acquire chatter to stderr #301

timur commented

2026-05-25 10:33:01 +00:00

Author

Owner

Completed

Fixed in PR #301 (squash-merged into development).

crates/lab/src/service/ephemeral.rs::spawn_inner now clears PATH_VAR / PATH_BUILD / PATH_CODE in addition to HERO_SOCKET_DIR / INTEGRATIONTEST so the child's path_var() / path_build() / path_code() fall through to the PATH_ROOT-derived defaults.
read_log_tails tails both stdout.log and stderr.log on spawn failure — the resolved socket path printed by print_startup_banner lives in stdout and is now visible in error messages.
acquire_binary progress lines (using installed …, installed from Forge → …) route to stderr so lab --start --ephemeral --json stdout is exactly one JSON envelope.

Verification

Direct lab service hero_service --start --ephemeral --json returns a one-line JSON envelope on stdout, chatter on stderr, child binds the correct scratch socket.
cargo test --workspace --test greeter_e2e -- --test-threads=1 (hero_service @ fe8c78ad): 5/5 pass.
lab service hero_service --test layer2 exercises the spawn machinery successfully through to nu (only blocker on this dev box is nu not being installed).

Out of scope (filed separately if not already known)

_find E2E suite returns the unfiltered union for every StrFilter / EnumFilter query — pre-existing indexer issue surfaced now that the spawn works.
Four servicecatalog_* handlers reject seeded input (InvalidParams: "service name is required", "Not found:") — pre-existing service-side bug, also unrelated.

## Completed Fixed in PR #301 (squash-merged into `development`). - `crates/lab/src/service/ephemeral.rs::spawn_inner` now clears `PATH_VAR` / `PATH_BUILD` / `PATH_CODE` in addition to `HERO_SOCKET_DIR` / `INTEGRATIONTEST` so the child's `path_var()` / `path_build()` / `path_code()` fall through to the `PATH_ROOT`-derived defaults. - `read_log_tails` tails both `stdout.log` and `stderr.log` on spawn failure — the resolved socket path printed by `print_startup_banner` lives in stdout and is now visible in error messages. - `acquire_binary` progress lines (`using installed …`, `installed from Forge → …`) route to stderr so `lab --start --ephemeral --json` stdout is exactly one JSON envelope. ## Verification - Direct `lab service hero_service --start --ephemeral --json` returns a one-line JSON envelope on stdout, chatter on stderr, child binds the correct scratch socket. - `cargo test --workspace --test greeter_e2e -- --test-threads=1` (hero_service @ `fe8c78ad`): 5/5 pass. - `lab service hero_service --test layer2` exercises the spawn machinery successfully through to `nu` (only blocker on this dev box is `nu` not being installed). ## Out of scope (filed separately if not already known) - `_find` E2E suite returns the unfiltered union for every `StrFilter` / `EnumFilter` query — pre-existing indexer issue surfaced now that the spawn works. - Four `servicecatalog_*` handlers reject seeded input (`InvalidParams: "service name is required"`, `"Not found:"`) — pre-existing service-side bug, also unrelated.

timur closed this issue

2026-05-25 10:33:01 +00:00