lhumina_code/hero_aibroker
24
0
Fork 0
Code Issues 4 Pull requests Projects Releases 1 Packages Wiki Activity Actions

imrove hero_aibroker_services #48

New issue
Open
opened 2026-04-28 02:18:41 +00:00 by despiegk · 4 comments
despiegk commented 2026-04-28 02:18:41 +00:00
Owner
Copy link

rename

/Volumes/T7/code0/hero_aibroker/crates/hero_broker_server -> /Volumes/T7/code0/hero_aibroker/crates/hero_aibroker_services

make more modular, each of the services needs to

  • be in submodule of the crate (separate dir)
  • have a proper readme
  • have the openrpc spec inside the dir (module)
  • have an integration tests
  • when called, should check that the SECRET required in in hero_proc
  • make sure we use all secrets from /hero_proc_secrets (skill)
rename /Volumes/T7/code0/hero_aibroker/crates/hero_broker_server -> /Volumes/T7/code0/hero_aibroker/crates/hero_aibroker_services make more modular, each of the services needs to - be in submodule of the crate (separate dir) - have a proper readme - have the openrpc spec inside the dir (module) - have an integration tests - when called, should check that the SECRET required in in hero_proc - make sure we use all secrets from /hero_proc_secrets (skill)
despiegk commented 2026-04-28 02:25:19 +00:00
Author
Owner
Copy link

Implementation Spec for Issue #48

Objective

Rename crates/hero_broker_server to crates/hero_aibroker_services and refactor it into a modular crate where every search/scrape service lives in its own submodule directory under src/services/<name>/, each carrying its own README.md, OpenRPC spec, and integration tests. Each service must, at startup, fetch its required API key from the hero_proc secret store (instead of reading raw OS env vars), and fail fast with a clear error if the secret is missing — so all secrets come from the canonical hero_proc store.

Requirements

  • Rename crate dir crates/hero_broker_server → crates/hero_aibroker_services.
  • Cargo package name renamed hero_broker_server → hero_aibroker_services; binary renamed to hero_aibroker_services.
  • Each service in its own submodule directory: src/services/<name>/{mod.rs, README.md, openrpc.json}.
  • Each service has integration tests in tests/<name>.rs at the crate root.
  • Each service entry-point checks that its required secret is present in hero_proc (via hero_proc_sdk) before launching the listener; missing secret causes a logged error and the service is skipped (other services keep running).
  • All secret access uses hero_proc_sdk::HeroProcRPCAPIClient::secret_get with context "core", never bare std::env::var for API keys.
  • Update every workspace and tooling reference: root Cargo.toml, buildenv.sh, Makefile, crates/hero_aibroker/src/main.rs (orchestrator), and README.md.

Files to Modify/Create

Rename and structural moves (treat as deletes-from-old + creates-at-new):

  • crates/hero_broker_server/ (entire directory) → renamed to crates/hero_aibroker_services/. Inside the renamed crate, files are reorganized as below.

Inside the renamed crate:

  • crates/hero_aibroker_services/Cargo.toml — package name hero_aibroker_services, binary name hero_aibroker_services, add hero_proc_sdk workspace dep, add dirs dep (used for ~/hero/var/sockets fallback), keep existing deps.
  • crates/hero_aibroker_services/README.md — overview of the crate, list of services, secret requirements, how to run, how to test.
  • crates/hero_aibroker_services/src/main.rs — slim binary: parse args, init tracing, build HeroProcRPCAPIClient, dispatch to each service module's start() async fn (each returns Result<Option<JoinHandle<...>>>), join all.
  • crates/hero_aibroker_services/src/lib.rs — declares pub mod common; pub mod services; so integration tests can import shared helpers.
  • crates/hero_aibroker_services/src/common/mod.rs — keep current RpcRequest, rpc_ok, rpc_err, require_str. Add new helpers:
    • pub async fn hero_proc_client() -> anyhow::Result<HeroProcRPCAPIClient> — connect using hero_proc_factory() (mirror crates/hero_aibroker/src/main.rs).
    • pub async fn require_secret(client: &HeroProcRPCAPIClient, key: &str) -> anyhow::Result<String> — calls secret_get with context "core", returns error if value empty/missing.
    • pub async fn serve_socket(name: String, socket_path: PathBuf, app: axum::Router) -> anyhow::Result<()> — moved from current main.rs.
    • pub fn hero_socket_dir() -> PathBuf — moved from current main.rs.
  • crates/hero_aibroker_services/src/services/mod.rs — pub mod ping; pub mod serpapi; pub mod serper; pub mod exa; pub mod scraperapi; pub mod scrapfly;.

Per service (six dirs — same structure for each):

  • crates/hero_aibroker_services/src/services/ping/mod.rs
  • crates/hero_aibroker_services/src/services/ping/README.md
  • crates/hero_aibroker_services/src/services/ping/openrpc.json
  • crates/hero_aibroker_services/src/services/serpapi/{mod.rs,README.md,openrpc.json}
  • crates/hero_aibroker_services/src/services/serper/{mod.rs,README.md,openrpc.json}
  • crates/hero_aibroker_services/src/services/exa/{mod.rs,README.md,openrpc.json}
  • crates/hero_aibroker_services/src/services/scraperapi/{mod.rs,README.md,openrpc.json}
  • crates/hero_aibroker_services/src/services/scrapfly/{mod.rs,README.md,openrpc.json}

Each mod.rs exposes:

  • pub const SERVICE_NAME: &str = ...
  • pub const SECRET_KEYS: &[&str] (canonical singular first, legacy plural as fallback; empty for ping)
  • pub const OPENRPC_SPEC: &str = include_str!("openrpc.json")
  • pub fn router(api_key: Option<String>) -> axum::Router
  • pub async fn start(client: &HeroProcRPCAPIClient, socket_dir: &Path) -> anyhow::Result<Option<JoinHandle<anyhow::Result<()>>>> — fetches the secret if SECRET_KEYS is non-empty, logs and returns Ok(None) if missing (so other services start), spawns serve_socket(...) otherwise.

Integration tests:

  • crates/hero_aibroker_services/tests/common/mod.rs — shared helpers (spawn the renamed binary as child; wait for socket; build a hyper-over-UDS JSON-RPC POST helper).
  • crates/hero_aibroker_services/tests/{ping,serpapi,serper,exa,scraperapi,scrapfly}.rs

Workspace and tooling updates (rename references):

  • Cargo.toml (workspace root) — replace "crates/hero_broker_server" with "crates/hero_aibroker_services" in members.
  • buildenv.sh — replace hero_broker_server with hero_aibroker_services in BINARIES.
  • Makefile — line 70: replace hero_proc status hero_broker_server with hero_proc status hero_aibroker_services.
  • crates/hero_aibroker/src/main.rs — lines 282–372: rename binary lookup, action name, and health-check action name. Remove the for env_name in [...] { broker_builder.env(...) } block (per-service binary now reads secrets directly from hero_proc).
  • README.md (top level) — replace the hero_broker_server mentions with hero_aibroker_services.

Implementation Plan

Step 1: Move and rename the crate skeleton

  • git mv crates/hero_broker_server crates/hero_aibroker_services
  • Edit crates/hero_aibroker_services/Cargo.toml: set name = "hero_aibroker_services", [[bin]] name = "hero_aibroker_services", add hero_proc_sdk and dirs deps.
    Dependencies: none.

Step 2: Update workspace members and tooling references

  • Root Cargo.toml line 19: "crates/hero_broker_server" → "crates/hero_aibroker_services".
  • buildenv.sh line 5: rename token in BINARIES.
  • Makefile line 70: rename hero_proc status arg.
    Dependencies: Step 1.

Step 3: Update orchestrator (hero_aibroker CLI) — register renamed binary, drop secret env forwarding

  • crates/hero_aibroker/src/main.rs lines 282–372:
    • Replace "hero_broker_server" with "hero_aibroker_services" (binary path, action name, health-check action name).
    • Delete the for env_name in ["SERPAPI_API_KEYS", ...] { broker_builder.env(...) } loop (lines 296–305). Replace with a comment: // API keys are read directly from hero_proc secret store by hero_aibroker_services.
      Dependencies: Step 1 (independent of Step 2 → can run in parallel).

Step 4: Restructure src/ into services/ submodule tree

  • Create src/lib.rs with pub mod common; pub mod services;.
  • git mv src/common.rs src/common/mod.rs. Extend with hero_proc_client(), require_secret(), serve_socket(), hero_socket_dir() helpers.
  • Create src/services/mod.rs declaring the six submodules.
  • For each service <name> in {ping, serpapi, serper, exa, scraperapi, scrapfly}:
    • git mv src/<name>.rs src/services/<name>/mod.rs
    • git mv openrpc/<name>.json src/services/<name>/openrpc.json
    • Change include_str! path to "openrpc.json".
    • Add SECRET_KEYS and start() per-service entry point.
    • Replace std::env::var("*_API_KEYS") reads with api_key parameter passed in from start().
  • Remove now-empty openrpc/ directory.
  • Rewrite src/main.rs: build hero_proc client, call each services::<name>::start(&hp, &socket_dir).await?, join handles.
    Dependencies: Step 1 (independent of Steps 2/3 → parallelizable).

Step 5: Add per-service READMEs

  • One README.md in each src/services/<name>/ describing: required hero_proc secret key, RPC methods, socket path, and a curl-via-UDS example.
    Dependencies: Step 4.

Step 6: Add crate-level README

  • crates/hero_aibroker_services/README.md — overview, services table, secret resolution rules, "How to add a new service" recipe, "Running tests".
    Dependencies: Step 1 (parallelizable with Step 5).

Step 7: Add integration tests (one per service)

  • tests/common/mod.rs — start_services_binary() (spawn binary with temp HERO_SOCKET_DIR), wait_for_socket(), rpc_call() over UDS.
  • tests/ping.rs — calls ping.ping, rpc.discover, /health, /openrpc.json. Always succeeds.
  • tests/{serpapi,serper,exa,scraperapi,scrapfly}.rs — start binary; if secret absent in hero_proc, assert socket absent; if present, assert socket exists and rpc.discover//health succeed. No live external API calls.
    Dependencies: Step 4.

Step 8: Update top-level README.md

  • Lines 34, 93, 340, 363: replace hero_broker_server with hero_aibroker_services.
    Dependencies: Step 1.

Acceptance Criteria

  • Crate dir is crates/hero_aibroker_services; old crates/hero_broker_server no longer exists.
  • Workspace root Cargo.toml lists crates/hero_aibroker_services.
  • Cargo package and binary names are both hero_aibroker_services.
  • Each of six services lives under src/services/<name>/ with its own mod.rs, README.md, and openrpc.json.
  • Each service has at least one integration test under tests/<name>.rs.
  • Each service that requires a secret calls hero_proc_sdk::secret_get(...) at startup; missing secret yields tracing::warn! and the service is skipped.
  • No remaining std::env::var("*_API_KEYS"|"*_API_KEY") reads inside crates/hero_aibroker_services/.
  • No remaining string hero_broker_server anywhere in the repo (excluding Cargo.lock).
  • crates/hero_aibroker/src/main.rs registers hero_aibroker_services and no longer copies *_API_KEYS env vars into the action.
  • buildenv.sh BINARIES list contains hero_aibroker_services.
  • Makefile status target references hero_aibroker_services.
  • Top-level README.md no longer references hero_broker_server.

Notes

Canonical secret keys. Current code reads plural keys (SERPAPI_API_KEYS, SERPER_API_KEYS, EXA_API_KEYS, SCRAPERAPI_API_KEYS, SCRAPFLY_API_KEYS). The hero_proc canonical secret keys use the singular form (EXA_API_KEY). Each service module looks up secrets in this priority order: canonical singular hero_proc key first, then legacy plural key as fallback. Each service exposes:

pub const SECRET_KEYS: &[&str] = &["EXA_API_KEY", "EXA_API_KEYS"];

and require_secret returns the first non-empty value found in hero_proc.

Socket-name compatibility. Current sockets are named hero_broker_<name>/rpc.sock. Renaming them would break downstream clients. This change keeps the on-disk socket directory names as they are (hero_broker_serpapi, etc.) — only crate, binary, and process names change. The kill_other list in crates/hero_aibroker/src/main.rs keeps the existing names. Renaming sockets is a follow-up issue.

OpenRPC format. JSON files (matching the rest of the repo), one openrpc.json per service inside its own directory.

hero_proc connectivity at startup. If hero_proc_factory() fails (hero_proc not running), the binary logs a clear error and exits non-zero. Per-service start() only logs warn and returns Ok(None) for missing secrets; a hard hero_proc connectivity failure is a top-level error in main.

No new branch / no test-running step. All changes are made on the current working branch and committed in a single commit at the end. cargo build/cargo test are NOT part of the implementation steps — they run in a separate later test phase.

Cargo.lock will be regenerated automatically by the next cargo invocation.

## Implementation Spec for Issue #48 ### Objective Rename `crates/hero_broker_server` to `crates/hero_aibroker_services` and refactor it into a modular crate where every search/scrape service lives in its own submodule directory under `src/services/<name>/`, each carrying its own `README.md`, OpenRPC spec, and integration tests. Each service must, at startup, fetch its required API key from the hero_proc secret store (instead of reading raw OS env vars), and fail fast with a clear error if the secret is missing — so all secrets come from the canonical hero_proc store. ### Requirements - Rename crate dir `crates/hero_broker_server` → `crates/hero_aibroker_services`. - Cargo package name renamed `hero_broker_server` → `hero_aibroker_services`; binary renamed to `hero_aibroker_services`. - Each service in its own submodule directory: `src/services/<name>/{mod.rs, README.md, openrpc.json}`. - Each service has integration tests in `tests/<name>.rs` at the crate root. - Each service entry-point checks that its required secret is present in hero_proc (via `hero_proc_sdk`) before launching the listener; missing secret causes a logged error and the service is skipped (other services keep running). - All secret access uses `hero_proc_sdk::HeroProcRPCAPIClient::secret_get` with context `"core"`, never bare `std::env::var` for API keys. - Update every workspace and tooling reference: root `Cargo.toml`, `buildenv.sh`, `Makefile`, `crates/hero_aibroker/src/main.rs` (orchestrator), and `README.md`. ### Files to Modify/Create Rename and structural moves (treat as deletes-from-old + creates-at-new): - `crates/hero_broker_server/` (entire directory) → renamed to `crates/hero_aibroker_services/`. Inside the renamed crate, files are reorganized as below. Inside the renamed crate: - `crates/hero_aibroker_services/Cargo.toml` — package name `hero_aibroker_services`, binary name `hero_aibroker_services`, add `hero_proc_sdk` workspace dep, add `dirs` dep (used for `~/hero/var/sockets` fallback), keep existing deps. - `crates/hero_aibroker_services/README.md` — overview of the crate, list of services, secret requirements, how to run, how to test. - `crates/hero_aibroker_services/src/main.rs` — slim binary: parse args, init tracing, build `HeroProcRPCAPIClient`, dispatch to each service module's `start()` async fn (each returns `Result<Option<JoinHandle<...>>>`), join all. - `crates/hero_aibroker_services/src/lib.rs` — declares `pub mod common; pub mod services;` so integration tests can import shared helpers. - `crates/hero_aibroker_services/src/common/mod.rs` — keep current `RpcRequest`, `rpc_ok`, `rpc_err`, `require_str`. Add new helpers: - `pub async fn hero_proc_client() -> anyhow::Result<HeroProcRPCAPIClient>` — connect using `hero_proc_factory()` (mirror `crates/hero_aibroker/src/main.rs`). - `pub async fn require_secret(client: &HeroProcRPCAPIClient, key: &str) -> anyhow::Result<String>` — calls `secret_get` with context `"core"`, returns error if value empty/missing. - `pub async fn serve_socket(name: String, socket_path: PathBuf, app: axum::Router) -> anyhow::Result<()>` — moved from current `main.rs`. - `pub fn hero_socket_dir() -> PathBuf` — moved from current `main.rs`. - `crates/hero_aibroker_services/src/services/mod.rs` — `pub mod ping; pub mod serpapi; pub mod serper; pub mod exa; pub mod scraperapi; pub mod scrapfly;`. Per service (six dirs — same structure for each): - `crates/hero_aibroker_services/src/services/ping/mod.rs` - `crates/hero_aibroker_services/src/services/ping/README.md` - `crates/hero_aibroker_services/src/services/ping/openrpc.json` - `crates/hero_aibroker_services/src/services/serpapi/{mod.rs,README.md,openrpc.json}` - `crates/hero_aibroker_services/src/services/serper/{mod.rs,README.md,openrpc.json}` - `crates/hero_aibroker_services/src/services/exa/{mod.rs,README.md,openrpc.json}` - `crates/hero_aibroker_services/src/services/scraperapi/{mod.rs,README.md,openrpc.json}` - `crates/hero_aibroker_services/src/services/scrapfly/{mod.rs,README.md,openrpc.json}` Each `mod.rs` exposes: - `pub const SERVICE_NAME: &str = ...` - `pub const SECRET_KEYS: &[&str]` (canonical singular first, legacy plural as fallback; empty for ping) - `pub const OPENRPC_SPEC: &str = include_str!("openrpc.json")` - `pub fn router(api_key: Option<String>) -> axum::Router` - `pub async fn start(client: &HeroProcRPCAPIClient, socket_dir: &Path) -> anyhow::Result<Option<JoinHandle<anyhow::Result<()>>>>` — fetches the secret if `SECRET_KEYS` is non-empty, logs and returns `Ok(None)` if missing (so other services start), spawns `serve_socket(...)` otherwise. Integration tests: - `crates/hero_aibroker_services/tests/common/mod.rs` — shared helpers (spawn the renamed binary as child; wait for socket; build a hyper-over-UDS JSON-RPC POST helper). - `crates/hero_aibroker_services/tests/{ping,serpapi,serper,exa,scraperapi,scrapfly}.rs` Workspace and tooling updates (rename references): - `Cargo.toml` (workspace root) — replace `"crates/hero_broker_server"` with `"crates/hero_aibroker_services"` in `members`. - `buildenv.sh` — replace `hero_broker_server` with `hero_aibroker_services` in `BINARIES`. - `Makefile` — line 70: replace `hero_proc status hero_broker_server` with `hero_proc status hero_aibroker_services`. - `crates/hero_aibroker/src/main.rs` — lines 282–372: rename binary lookup, action name, and health-check action name. Remove the `for env_name in [...] { broker_builder.env(...) }` block (per-service binary now reads secrets directly from hero_proc). - `README.md` (top level) — replace the `hero_broker_server` mentions with `hero_aibroker_services`. ### Implementation Plan #### Step 1: Move and rename the crate skeleton - `git mv crates/hero_broker_server crates/hero_aibroker_services` - Edit `crates/hero_aibroker_services/Cargo.toml`: set `name = "hero_aibroker_services"`, `[[bin]] name = "hero_aibroker_services"`, add `hero_proc_sdk` and `dirs` deps. Dependencies: none. #### Step 2: Update workspace members and tooling references - Root `Cargo.toml` line 19: `"crates/hero_broker_server"` → `"crates/hero_aibroker_services"`. - `buildenv.sh` line 5: rename token in `BINARIES`. - `Makefile` line 70: rename `hero_proc status` arg. Dependencies: Step 1. #### Step 3: Update orchestrator (`hero_aibroker` CLI) — register renamed binary, drop secret env forwarding - `crates/hero_aibroker/src/main.rs` lines 282–372: - Replace `"hero_broker_server"` with `"hero_aibroker_services"` (binary path, action name, health-check action name). - Delete the `for env_name in ["SERPAPI_API_KEYS", ...] { broker_builder.env(...) }` loop (lines 296–305). Replace with a comment: `// API keys are read directly from hero_proc secret store by hero_aibroker_services`. Dependencies: Step 1 (independent of Step 2 → can run in parallel). #### Step 4: Restructure `src/` into `services/` submodule tree - Create `src/lib.rs` with `pub mod common; pub mod services;`. - `git mv src/common.rs src/common/mod.rs`. Extend with `hero_proc_client()`, `require_secret()`, `serve_socket()`, `hero_socket_dir()` helpers. - Create `src/services/mod.rs` declaring the six submodules. - For each service `<name>` in {ping, serpapi, serper, exa, scraperapi, scrapfly}: - `git mv src/<name>.rs src/services/<name>/mod.rs` - `git mv openrpc/<name>.json src/services/<name>/openrpc.json` - Change `include_str!` path to `"openrpc.json"`. - Add `SECRET_KEYS` and `start()` per-service entry point. - Replace `std::env::var("*_API_KEYS")` reads with `api_key` parameter passed in from `start()`. - Remove now-empty `openrpc/` directory. - Rewrite `src/main.rs`: build hero_proc client, call each `services::<name>::start(&hp, &socket_dir).await?`, join handles. Dependencies: Step 1 (independent of Steps 2/3 → parallelizable). #### Step 5: Add per-service READMEs - One `README.md` in each `src/services/<name>/` describing: required hero_proc secret key, RPC methods, socket path, and a curl-via-UDS example. Dependencies: Step 4. #### Step 6: Add crate-level README - `crates/hero_aibroker_services/README.md` — overview, services table, secret resolution rules, "How to add a new service" recipe, "Running tests". Dependencies: Step 1 (parallelizable with Step 5). #### Step 7: Add integration tests (one per service) - `tests/common/mod.rs` — `start_services_binary()` (spawn binary with temp `HERO_SOCKET_DIR`), `wait_for_socket()`, `rpc_call()` over UDS. - `tests/ping.rs` — calls `ping.ping`, `rpc.discover`, `/health`, `/openrpc.json`. Always succeeds. - `tests/{serpapi,serper,exa,scraperapi,scrapfly}.rs` — start binary; if secret absent in hero_proc, assert socket absent; if present, assert socket exists and `rpc.discover`/`/health` succeed. No live external API calls. Dependencies: Step 4. #### Step 8: Update top-level `README.md` - Lines 34, 93, 340, 363: replace `hero_broker_server` with `hero_aibroker_services`. Dependencies: Step 1. ### Acceptance Criteria - [ ] Crate dir is `crates/hero_aibroker_services`; old `crates/hero_broker_server` no longer exists. - [ ] Workspace root `Cargo.toml` lists `crates/hero_aibroker_services`. - [ ] Cargo package and binary names are both `hero_aibroker_services`. - [ ] Each of six services lives under `src/services/<name>/` with its own `mod.rs`, `README.md`, and `openrpc.json`. - [ ] Each service has at least one integration test under `tests/<name>.rs`. - [ ] Each service that requires a secret calls `hero_proc_sdk::secret_get(...)` at startup; missing secret yields `tracing::warn!` and the service is skipped. - [ ] No remaining `std::env::var("*_API_KEYS"|"*_API_KEY")` reads inside `crates/hero_aibroker_services/`. - [ ] No remaining string `hero_broker_server` anywhere in the repo (excluding `Cargo.lock`). - [ ] `crates/hero_aibroker/src/main.rs` registers `hero_aibroker_services` and no longer copies `*_API_KEYS` env vars into the action. - [ ] `buildenv.sh` `BINARIES` list contains `hero_aibroker_services`. - [ ] `Makefile` `status` target references `hero_aibroker_services`. - [ ] Top-level `README.md` no longer references `hero_broker_server`. ### Notes **Canonical secret keys.** Current code reads plural keys (`SERPAPI_API_KEYS`, `SERPER_API_KEYS`, `EXA_API_KEYS`, `SCRAPERAPI_API_KEYS`, `SCRAPFLY_API_KEYS`). The hero_proc canonical secret keys use the singular form (`EXA_API_KEY`). Each service module looks up secrets in this priority order: canonical singular hero_proc key first, then legacy plural key as fallback. Each service exposes: ``` pub const SECRET_KEYS: &[&str] = &["EXA_API_KEY", "EXA_API_KEYS"]; ``` and `require_secret` returns the first non-empty value found in hero_proc. **Socket-name compatibility.** Current sockets are named `hero_broker_<name>/rpc.sock`. Renaming them would break downstream clients. This change keeps the on-disk socket directory names as they are (`hero_broker_serpapi`, etc.) — only crate, binary, and process names change. The `kill_other` list in `crates/hero_aibroker/src/main.rs` keeps the existing names. Renaming sockets is a follow-up issue. **OpenRPC format.** JSON files (matching the rest of the repo), one `openrpc.json` per service inside its own directory. **hero_proc connectivity at startup.** If `hero_proc_factory()` fails (hero_proc not running), the binary logs a clear error and exits non-zero. Per-service `start()` only logs `warn` and returns `Ok(None)` for missing secrets; a hard hero_proc connectivity failure is a top-level error in `main`. **No new branch / no test-running step.** All changes are made on the current working branch and committed in a single commit at the end. `cargo build`/`cargo test` are NOT part of the implementation steps — they run in a separate later test phase. **`Cargo.lock`** will be regenerated automatically by the next `cargo` invocation.
despiegk commented 2026-04-28 02:51:31 +00:00
Author
Owner
Copy link

Test Results

Build (cargo build --workspace)

FAIL — hero_aibroker_services (lib) failed to compile due to 1 type-mismatch error. Tests were not run (per instructions, do not run tests against a broken build).

First lines of error output:

error[E0308]: mismatched types
  --> crates/hero_aibroker_services/src/common/mod.rs:40:5
   |
39 |   pub async fn hero_proc_client() -> anyhow::Result<HeroProcRPCAPIClient> {
   |                                      ------------------------------------ expected `Result<HeroProcRPCAPIClient, anyhow::Error>` because of return type
40 | /     hero_proc_factory()
41 | |         .await
42 | |         .context("failed to connect to hero_proc")
   | |__________________________________________________^ expected `Result<HeroProcRPCAPIClient, Error>`, found `Result<HeroProcFactory, Error>`
   |
   = note: expected enum `Result<HeroProcRPCAPIClient, _>`
              found enum `Result<HeroProcFactory, _>`

For more information about this error, try `rustc --explain E0308`.
error: could not compile `hero_aibroker_services` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...

Tests (cargo test -p hero_aibroker_services)

Not run (build failed).

  • Total: N/A
  • Passed: N/A
  • Failed: N/A
  • Ignored: N/A

Tests (cargo test --workspace --no-fail-fast)

Not run (build failed).

  • Total: N/A
  • Passed: N/A
  • Failed: N/A
  • Ignored: N/A

Build broken in hero_aibroker_services — hero_proc_factory() now returns a HeroProcFactory builder (not a connected client), so crates/hero_aibroker_services/src/common/mod.rs:39-43 no longer matches the declared return type anyhow::Result<HeroProcRPCAPIClient>. The factory likely needs a terminating call (e.g. .connect().await or equivalent) to produce the client. Until this is fixed, tests cannot be run.

## Test Results ### Build (`cargo build --workspace`) FAIL — `hero_aibroker_services` (lib) failed to compile due to 1 type-mismatch error. Tests were not run (per instructions, do not run tests against a broken build). First lines of error output: ``` error[E0308]: mismatched types --> crates/hero_aibroker_services/src/common/mod.rs:40:5 | 39 | pub async fn hero_proc_client() -> anyhow::Result<HeroProcRPCAPIClient> { | ------------------------------------ expected `Result<HeroProcRPCAPIClient, anyhow::Error>` because of return type 40 | / hero_proc_factory() 41 | | .await 42 | | .context("failed to connect to hero_proc") | |__________________________________________________^ expected `Result<HeroProcRPCAPIClient, Error>`, found `Result<HeroProcFactory, Error>` | = note: expected enum `Result<HeroProcRPCAPIClient, _>` found enum `Result<HeroProcFactory, _>` For more information about this error, try `rustc --explain E0308`. error: could not compile `hero_aibroker_services` (lib) due to 1 previous error warning: build failed, waiting for other jobs to finish... ``` ### Tests (`cargo test -p hero_aibroker_services`) Not run (build failed). - Total: N/A - Passed: N/A - Failed: N/A - Ignored: N/A ### Tests (`cargo test --workspace --no-fail-fast`) Not run (build failed). - Total: N/A - Passed: N/A - Failed: N/A - Ignored: N/A Build broken in `hero_aibroker_services` — `hero_proc_factory()` now returns a `HeroProcFactory` builder (not a connected client), so `crates/hero_aibroker_services/src/common/mod.rs:39-43` no longer matches the declared return type `anyhow::Result<HeroProcRPCAPIClient>`. The factory likely needs a terminating call (e.g. `.connect().await` or equivalent) to produce the client. Until this is fixed, tests cannot be run.
despiegk commented 2026-04-28 03:02:41 +00:00
Author
Owner
Copy link

Test Results

Build (cargo build --workspace)

PASS — clean compile after the rename, including the orchestrator (hero_aibroker) which now references the renamed binary.

Tests (cargo test -p hero_aibroker_services)

  • Total: 6
  • Passed: 6
  • Failed: 0
  • Ignored: 0

The 6 integration tests cover the always-on ping service plus the five secret-gated services (serpapi, serper, exa, scraperapi, scrapfly). Each secret-gated test asserts the service is either up (when the corresponding hero_proc secret is present) or correctly skipped (when not), without taking down the rest of the binary. The tests skip with a stderr note when no hero_proc daemon is reachable on the host, so they remain CI-friendly.

Tests (cargo test --workspace --no-fail-fast)

  • Total: 37
  • Passed: 37
  • Failed: 0
  • Ignored: 0

All builds and tests pass. No live external API calls are made by the new test suite.

## Test Results ### Build (`cargo build --workspace`) PASS — clean compile after the rename, including the orchestrator (`hero_aibroker`) which now references the renamed binary. ### Tests (`cargo test -p hero_aibroker_services`) - Total: 6 - Passed: 6 - Failed: 0 - Ignored: 0 The 6 integration tests cover the always-on `ping` service plus the five secret-gated services (`serpapi`, `serper`, `exa`, `scraperapi`, `scrapfly`). Each secret-gated test asserts the service is either up (when the corresponding hero_proc secret is present) or correctly skipped (when not), without taking down the rest of the binary. The tests skip with a stderr note when no hero_proc daemon is reachable on the host, so they remain CI-friendly. ### Tests (`cargo test --workspace --no-fail-fast`) - Total: 37 - Passed: 37 - Failed: 0 - Ignored: 0 All builds and tests pass. No live external API calls are made by the new test suite.
despiegk commented 2026-04-28 03:03:10 +00:00
Author
Owner
Copy link

Implementation Summary

Renamed crate

  • crates/hero_broker_server -> crates/hero_aibroker_services (history preserved via git mv).
  • Cargo package + binary names: hero_broker_server -> hero_aibroker_services.

Modular per-service layout

Each of the six services now lives in its own directory under crates/hero_aibroker_services/src/services/<name>/ with:

  • mod.rs (Rust code: SERVICE_NAME, SECRET_KEYS, OPENRPC_SPEC, router(...), start(...))
  • README.md (per-service docs: required hero_proc secret, RPC method list, socket path, curl example)
  • openrpc.json (the OpenRPC spec for that service)

Services: ping, serpapi, serper, exa, scraperapi, scrapfly.

Secrets sourced from hero_proc

  • All *_API_KEY / *_API_KEYS lookups now go through hero_proc_sdk (secret_get, context core). No std::env::var(...) reads for API keys remain inside the crate.
  • Each service tries the canonical singular key first (e.g. EXA_API_KEY), then the legacy plural key (e.g. EXA_API_KEYS) as a fallback for migration compatibility.
  • Missing secret -> tracing::warn!("skipping {service}: ...") and the service is skipped; the rest of the binary keeps running.
  • If hero_proc itself is unreachable, the binary logs a clear error and exits non-zero.
  • The orchestrator (crates/hero_aibroker/src/main.rs) no longer forwards *_API_KEYS env vars into the spawned action — that responsibility moved into the services binary.

Integration tests (one per service)

  • crates/hero_aibroker_services/tests/common/mod.rs — shared helpers: spawn the binary against a temp HERO_SOCKET_DIR, hyper-over-UDS RPC client, ping-based liveness check.
  • tests/{ping,serpapi,serper,exa,scraperapi,scrapfly}.rs — one file per service. ping must always be up; the secret-gated tests assert "up if secret set, cleanly skipped otherwise". No live external API calls. Tests gracefully skip when no hero_proc daemon is reachable on the host.

Documentation

  • New crates/hero_aibroker_services/README.md (overview, services table, secret-resolution rules, "How to add a new service" recipe, "Running tests" section).
  • Per-service README.md files under src/services/<name>/.
  • Top-level README.md updated: every reference to hero_broker_server replaced with hero_aibroker_services.

Workspace + tooling updates

  • Root Cargo.toml members: crates/hero_broker_server -> crates/hero_aibroker_services.
  • buildenv.sh BINARIES updated.
  • Makefile status target updated.

Backwards-compatibility note

On-disk socket directory names keep the existing hero_broker_<service>/rpc.sock shape so downstream clients written against the old binary keep working. Renaming socket paths is left as a follow-up.

Test results

  • cargo build --workspace -> PASS.
  • cargo test -p hero_aibroker_services -> 6/6 pass.
  • cargo test --workspace --no-fail-fast -> 37/37 pass.
## Implementation Summary ### Renamed crate - `crates/hero_broker_server` -> `crates/hero_aibroker_services` (history preserved via `git mv`). - Cargo package + binary names: `hero_broker_server` -> `hero_aibroker_services`. ### Modular per-service layout Each of the six services now lives in its own directory under `crates/hero_aibroker_services/src/services/<name>/` with: - `mod.rs` (Rust code: `SERVICE_NAME`, `SECRET_KEYS`, `OPENRPC_SPEC`, `router(...)`, `start(...)`) - `README.md` (per-service docs: required hero_proc secret, RPC method list, socket path, curl example) - `openrpc.json` (the OpenRPC spec for that service) Services: `ping`, `serpapi`, `serper`, `exa`, `scraperapi`, `scrapfly`. ### Secrets sourced from hero_proc - All `*_API_KEY` / `*_API_KEYS` lookups now go through `hero_proc_sdk` (`secret_get`, context `core`). No `std::env::var(...)` reads for API keys remain inside the crate. - Each service tries the canonical singular key first (e.g. `EXA_API_KEY`), then the legacy plural key (e.g. `EXA_API_KEYS`) as a fallback for migration compatibility. - Missing secret -> `tracing::warn!("skipping {service}: ...")` and the service is skipped; the rest of the binary keeps running. - If hero_proc itself is unreachable, the binary logs a clear error and exits non-zero. - The orchestrator (`crates/hero_aibroker/src/main.rs`) no longer forwards `*_API_KEYS` env vars into the spawned action — that responsibility moved into the services binary. ### Integration tests (one per service) - `crates/hero_aibroker_services/tests/common/mod.rs` — shared helpers: spawn the binary against a temp `HERO_SOCKET_DIR`, hyper-over-UDS RPC client, ping-based liveness check. - `tests/{ping,serpapi,serper,exa,scraperapi,scrapfly}.rs` — one file per service. `ping` must always be up; the secret-gated tests assert "up if secret set, cleanly skipped otherwise". No live external API calls. Tests gracefully skip when no hero_proc daemon is reachable on the host. ### Documentation - New `crates/hero_aibroker_services/README.md` (overview, services table, secret-resolution rules, "How to add a new service" recipe, "Running tests" section). - Per-service `README.md` files under `src/services/<name>/`. - Top-level `README.md` updated: every reference to `hero_broker_server` replaced with `hero_aibroker_services`. ### Workspace + tooling updates - Root `Cargo.toml` `members`: `crates/hero_broker_server` -> `crates/hero_aibroker_services`. - `buildenv.sh` `BINARIES` updated. - `Makefile` `status` target updated. ### Backwards-compatibility note On-disk socket directory names keep the existing `hero_broker_<service>/rpc.sock` shape so downstream clients written against the old binary keep working. Renaming socket paths is left as a follow-up. ### Test results - `cargo build --workspace` -> PASS. - `cargo test -p hero_aibroker_services` -> 6/6 pass. - `cargo test --workspace --no-fail-fast` -> 37/37 pass.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
development
development_mik2
main
v0.1.0
Labels
Clear labels   
prio_critical

   
prio_low

   
type_bug

   
type_contact

   
type_issue

   
type_lead

   
type_question

   
type_story

   
type_task
No labels
prio_critical
prio_low
type_bug
type_contact
type_issue
type_lead
type_question
type_story
type_task
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
despiegk
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lhumina_code/hero_aibroker#48
No description provided.