Conformance audit: oschema/blueprint gap (home#243) + doc drift #9

New issue

Open

opened 2026-06-08 14:14:58 +00:00 by casper-stevens · 2 comments

casper-stevens commented

2026-06-08 14:14:58 +00:00

Member

Audit of hero_researcher against (1) the canonical Hero service blueprint (lhumina_code/hero_blueprint @ development, blueprints/service/), (2) this repo's own commit history, and (3) the acceptance criteria in lhumina_code/home#243.

lab infocheck passes (2 crates clean, 0 findings) — manifests are valid. The gaps below are structural / scope, not manifest compliance.

1. REQUIRED: oschema + blueprint foundation (home#243)

home#243 mandates the researcher be "made with oschema and blueprint". Current state does not meet this:

hero_researcher_server ships a hand-written src/openrpc.json + hand-written dispatch_rpc() match (crates/hero_researcher_server/src/main.rs:538-608). The blueprint server is generated by herolib_macros::openrpc_server!(spec = "oschema", service_toml = "service.toml").
schema/service.oschema exists but drives no codegen (orphaned).
No _sdk crate (blueprint generates a typed Rust client + docs/openrpc.json from the schema).

→ Migrate _server to openrpc_server! over schema/*.oschema; regenerate SDK + docs. See /hero_server_migration.

2. REQUIRED deliverables still missing (home#243)

docx output via python runner — formatter only emits md/json/html (/md2docx).
light frontend on top of the kimi agent ("= background checks") — current pipeline drives herolib_ai across many models, not a kimi-agent frontend; no wiring to the background_check skill / lab agent.
voice integrated — not present (/hero_voice_widget).
run on main in cloud TFGrid (ZOS VM) — local UDS service only.

3. Crate-shape divergence from blueprint

Canonical: <name> (domain lib) + _server + _admin + _web + _sdk + sdk/{rhai,js,python} + tests/ + docs/.
Researcher: hero_researcher_lib + hero_researcher_server (admin UI folded in as admin.sock) + hero_researcher CLI.
Missing: _sdk, a separate _admin crate, sdk language bindings, an integration tests/ crate, generated docs/openrpc.json.

4. Deliberate decisions confirmed by history (docs-only, or leave)

web.sock → admin.sock was an intentional rename (ee3431b "D-10 sweep"). Code is correct; README + PURPOSE still say web.sock → stale docs, fix docs.
Build is lab-based now (Makefile/scripts removed in bb6c376). README/PURPOSE still reference a Nushell service researcher start script that does not exist in the repo → fix docs.
CLI --start/--stop lifecycle (509b37f) predates the blueprint's "lab owns lifecycle" guidance — candidate to drop during the migration.

5. Toolchain drift (low-risk)

tokio = { features = ["full"] } — blueprint uses a curated feature set and explicitly advises against full.
Root Cargo.toml has no [profile.release] — blueprint defines one (opt-level 2, panic="abort", strip, codegen-units 16).
rust-version = "1.96" was deliberately re-added (b508fa8) but conflicts with workspace policy (1.94–1.95, no pin) and the blueprint (1.93) → confirm intent before changing.

6. README/PURPOSE doc discrepancies (first-pass audit)

GROQ_API_KEY / SAMBANOVA_API_KEY / DEEPINFRA_API_KEY in README are correct (consumed by herolib_ai::AiClient::from_env) — keep.
SCRAPERAPI_API_KEYS / SCRAPFLY_API_KEYS are read in config.rs but never used and undocumented → dead config, wire or remove.
job.rs and log_bridge.rs are missing from the README "Project Structure".
Undocumented server routes: /health, /.well-known/heroservice.json, /api/stats, /api/stats/reset.

What is already done (for reference)

Working multi-model research pipeline (6 search providers, 8 platform scrapers, 3 tiers all verified real), JSON-RPC 2.0 server + HTML admin dashboard over rpc.sock/admin.sock, SQLite job persistence, canonical service.toml + service_base!() (passes lab infocheck), 13 runnable examples.

Suggested sequencing

oschema + blueprint migration of _server (+ generated _sdk) — the home#243 gate.
docx (python runner) + voice + kimi-agent frontend.
TFGrid main deploy (ZOS VM).
Doc + toolchain cleanups (cheap; do alongside).

Refs: lhumina_code/home#243.

Audit of `hero_researcher` against (1) the canonical Hero service blueprint (`lhumina_code/hero_blueprint` @ `development`, `blueprints/service/`), (2) this repo's own commit history, and (3) the acceptance criteria in `lhumina_code/home#243`. `lab infocheck` passes (2 crates clean, 0 findings) — manifests are valid. The gaps below are structural / scope, not manifest compliance. ## 1. REQUIRED: oschema + blueprint foundation (home#243) home#243 mandates the researcher be "made with oschema and blueprint". Current state does not meet this: - `hero_researcher_server` ships a **hand-written** `src/openrpc.json` + hand-written `dispatch_rpc()` match (`crates/hero_researcher_server/src/main.rs:538-608`). The blueprint server is generated by `herolib_macros::openrpc_server!(spec = "oschema", service_toml = "service.toml")`. - `schema/service.oschema` exists but **drives no codegen** (orphaned). - No `_sdk` crate (blueprint generates a typed Rust client + `docs/openrpc.json` from the schema). → Migrate `_server` to `openrpc_server!` over `schema/*.oschema`; regenerate SDK + docs. See `/hero_server_migration`. ## 2. REQUIRED deliverables still missing (home#243) - **docx output via python runner** — formatter only emits md/json/html (`/md2docx`). - **light frontend on top of the kimi agent** ("= background checks") — current pipeline drives `herolib_ai` across many models, not a kimi-agent frontend; no wiring to the `background_check` skill / `lab agent`. - **voice integrated** — not present (`/hero_voice_widget`). - **run on `main` in cloud TFGrid (ZOS VM)** — local UDS service only. ## 3. Crate-shape divergence from blueprint - Canonical: `<name>` (domain lib) + `_server` + `_admin` + `_web` + `_sdk` + `sdk/{rhai,js,python}` + `tests/` + `docs/`. - Researcher: `hero_researcher_lib` + `hero_researcher_server` (admin UI folded in as `admin.sock`) + `hero_researcher` CLI. - Missing: `_sdk`, a separate `_admin` crate, sdk language bindings, an integration `tests/` crate, generated `docs/openrpc.json`. ## 4. Deliberate decisions confirmed by history (docs-only, or leave) - `web.sock` → `admin.sock` was an **intentional** rename (`ee3431b` "D-10 sweep"). Code is correct; README + PURPOSE still say `web.sock` → **stale docs, fix docs**. - Build is `lab`-based now (Makefile/scripts removed in `bb6c376`). README/PURPOSE still reference a Nushell `service researcher start` script that does not exist in the repo → **fix docs**. - CLI `--start/--stop` lifecycle (`509b37f`) predates the blueprint's "lab owns lifecycle" guidance — candidate to drop during the migration. ## 5. Toolchain drift (low-risk) - `tokio = { features = ["full"] }` — blueprint uses a curated feature set and explicitly advises against `full`. - Root `Cargo.toml` has no `[profile.release]` — blueprint defines one (opt-level 2, `panic="abort"`, strip, codegen-units 16). - `rust-version = "1.96"` was **deliberately** re-added (`b508fa8`) but conflicts with workspace policy (1.94–1.95, no pin) and the blueprint (1.93) → **confirm intent before changing**. ## 6. README/PURPOSE doc discrepancies (first-pass audit) - `GROQ_API_KEY` / `SAMBANOVA_API_KEY` / `DEEPINFRA_API_KEY` in README are **correct** (consumed by `herolib_ai::AiClient::from_env`) — keep. - `SCRAPERAPI_API_KEYS` / `SCRAPFLY_API_KEYS` are read in `config.rs` but **never used** and undocumented → dead config, wire or remove. - `job.rs` and `log_bridge.rs` are missing from the README "Project Structure". - Undocumented server routes: `/health`, `/.well-known/heroservice.json`, `/api/stats`, `/api/stats/reset`. ## What is already done (for reference) Working multi-model research pipeline (6 search providers, 8 platform scrapers, 3 tiers all verified real), JSON-RPC 2.0 server + HTML admin dashboard over `rpc.sock`/`admin.sock`, SQLite job persistence, canonical `service.toml` + `service_base!()` (passes `lab infocheck`), 13 runnable examples. ## Suggested sequencing 1. oschema + blueprint migration of `_server` (+ generated `_sdk`) — the home#243 gate. 2. docx (python runner) + voice + kimi-agent frontend. 3. TFGrid `main` deploy (ZOS VM). 4. Doc + toolchain cleanups (cheap; do alongside). Refs: `lhumina_code/home#243`.

casper-stevens commented

2026-06-08 14:55:56 +00:00

Author

Member

Progress + blocker

Done (commit 4112370, doc-conformance slice — §4/§6):

web.sock → admin.sock in README + PURPOSE (the socket was deliberately renamed in ee3431b; docs were stale).
Replaced the non-existent Nushell service researcher flow with the canonical lab build / lab service lifecycle.
Documented the full HTTP endpoint set + the JSON-RPC method surface; added job.rs / log_bridge.rs to the structure.

BLOCKER for the oschema/blueprint migration (§1) — upstream, not in this repo:

The workspace does not currently build. hero_researcher_lib → herolib_ai (feature "hero-proc") → hero_proc_sdk, and hero_proc_sdk at the current development tip (14ee3ea) fails to compile:

error[E0432]: unresolved import `system`            (hero_proc_sdk/src/openrpc_client/mod.rs:40)
error[E0432]: unresolved import `crate::HeroProcClient_`  (hero_proc_sdk/src/sse.rs:32)

This is version skew — the generated client was renamed HeroProcClient_→HeroProcClient and the system domain module dropped, against the current herolib_macros. cargo update locks 0 packages, so this is the live branch tip, not a stale cache.

The oschema migration is compile-driven (the openrpc_server! macro generates the trait/types/spec at build time and the impl is checked against it), so it can't be done safely until the stack builds green.

Unblock options (pick one before the migration):

Fix hero_proc_sdk upstream on hero_proc development (the real fix — it's broken for every consumer right now).
Pin hero_proc to a last-known-good revision in this workspace until upstream is fixed.
Drop the hero-proc feature on the herolib_ai dependency if the lib doesn't need it (it pulls hero_proc_sdk transitively) — needs verification that nothing in _lib relies on it.

Migration readiness: schema/service.oschema already defines the full contract (single coherent API → one main domain, model B). Recon per /hero_server_migration is done; key Step-0 decisions still to confirm: (a) keep the HTML dashboard in _server via serve_domains_with vs. split it into a canonical _admin crate (openrpc_proxy!); (b) wire-name rename research.* → research_* (the admin dashboard JS calls these — must update in lockstep). Ready to execute the moment the build is green.

### Progress + blocker **Done (commit `4112370`, doc-conformance slice — §4/§6):** - `web.sock` → `admin.sock` in README + PURPOSE (the socket was deliberately renamed in `ee3431b`; docs were stale). - Replaced the non-existent Nushell `service researcher` flow with the canonical `lab build` / `lab service` lifecycle. - Documented the full HTTP endpoint set + the JSON-RPC method surface; added `job.rs` / `log_bridge.rs` to the structure. **BLOCKER for the oschema/blueprint migration (§1) — upstream, not in this repo:** The workspace does not currently build. `hero_researcher_lib → herolib_ai (feature "hero-proc") → hero_proc_sdk`, and `hero_proc_sdk` at the current `development` tip (`14ee3ea`) fails to compile: ``` error[E0432]: unresolved import `system` (hero_proc_sdk/src/openrpc_client/mod.rs:40) error[E0432]: unresolved import `crate::HeroProcClient_` (hero_proc_sdk/src/sse.rs:32) ``` This is version skew — the generated client was renamed `HeroProcClient_`→`HeroProcClient` and the `system` domain module dropped, against the current herolib_macros. `cargo update` locks 0 packages, so this is the live branch tip, not a stale cache. The oschema migration is **compile-driven** (the `openrpc_server!` macro generates the trait/types/spec at build time and the impl is checked against it), so it can't be done safely until the stack builds green. **Unblock options (pick one before the migration):** 1. Fix `hero_proc_sdk` upstream on hero_proc `development` (the real fix — it's broken for every consumer right now). 2. Pin `hero_proc` to a last-known-good revision in this workspace until upstream is fixed. 3. Drop the `hero-proc` feature on the `herolib_ai` dependency if the lib doesn't need it (it pulls `hero_proc_sdk` transitively) — needs verification that nothing in `_lib` relies on it. **Migration readiness:** `schema/service.oschema` already defines the full contract (single coherent API → one `main` domain, model B). Recon per `/hero_server_migration` is done; key Step-0 decisions still to confirm: (a) keep the HTML dashboard in `_server` via `serve_domains_with` vs. split it into a canonical `_admin` crate (`openrpc_proxy!`); (b) wire-name rename `research.*` → `research_*` (the admin dashboard JS calls these — must update in lockstep). Ready to execute the moment the build is green.

casper-stevens commented

2026-06-08 18:16:22 +00:00

Author

Member

Blocker now tracked upstream: lhumina_code/hero_proc#146.

Root cause nailed: with the confirmed development→development branch policy, this repo pulls hero_proc development (14ee3ea, = #140) + hero_lib development (277299dc), and hero_proc_sdk's openrpc_client! panics — the hero_proc schema's JobCreate.inputs / RunSpec.inputs are {str: any}, which the current hero_lib macro rejects per openrpc_principles Rule 2 (a recurrence of the closed hero_proc#142/#143). Not a hero_researcher bug; not fixable here.

Supersedes the earlier "pin hero_lib" suggestion — pinning is off the table given development→development. The oschema/blueprint migration (§1) resumes once hero_proc#146 lands and hero_proc development builds against hero_lib development again. (Verified the integration set builds green, so the fix is known; it just needs to reach development.)

Doc slice (§4/§6) remains done at 4112370.

**Blocker now tracked upstream: `lhumina_code/hero_proc#146`.** Root cause nailed: with the confirmed **development→development** branch policy, this repo pulls hero_proc `development` (`14ee3ea`, = #140) + hero_lib `development` (`277299dc`), and `hero_proc_sdk`'s `openrpc_client!` **panics** — the hero_proc schema's `JobCreate.inputs` / `RunSpec.inputs` are `{str: any}`, which the current hero_lib macro rejects per openrpc_principles Rule 2 (a recurrence of the closed hero_proc#142/#143). Not a hero_researcher bug; not fixable here. Supersedes the earlier "pin hero_lib" suggestion — pinning is off the table given development→development. The oschema/blueprint migration (§1) resumes once hero_proc#146 lands and hero_proc `development` builds against hero_lib `development` again. (Verified the `integration` set builds green, so the fix is known; it just needs to reach `development`.) Doc slice (§4/§6) remains done at `4112370`.