lhumina_code/hero_aibroker
4
0
Fork 0
Code Issues 65 Pull requests 2 Projects Releases 3 Packages Wiki Activity Actions

sdk: split AIBrokerAdminAPIClient into per-domain clients (post Phase-9 dispatcher split) #127

New issue
Open
opened 2026-05-12 16:24:30 +00:00 by timur · 1 comment
timur commented 2026-05-12 16:24:30 +00:00
Owner
Copy link

Problem

The 36-commit reshape on development split the OpenRPC surface across 10 per-domain sockets (chat/, speech/, embedder/, models/, admin/, images/, billing/, meta/, memory/, video/). The monolithic dispatcher was deleted in Phase 9 (api_openrpc/mod.rs:20-22) and the legacy single rpc.sock is now actively removed on startup (main.rs::legacy_combined_socket_path).

But the SDK (hero_aibroker_sdk) is still wired to the pre-split architecture:

  • AIBrokerAdminAPIClient is generated from the unified workspace-root openrpc.json (61 methods, documentary).
  • connect_default() returns a single client bound to $HERO_SOCKET_DIR/hero_aibroker/rpc.sock — a path the new server actively deletes.
  • client.ai_chat(), .ai_embed(), .ai_tts(), … all point at the same single transport — there is no server behind it.

Net: all SDK methods will fail at runtime against the new broker. hero_books (the first migrated consumer, hero_books#125) inherits the break. The REST surface at rest.sock is unaffected, so OpenAI-shape HTTP consumers keep working.

Plan: per-domain SDK clients

Split the SDK to mirror the server's per-domain sockets:

New SDK type Socket Methods
hero_aibroker_sdk::chat::Client chat/rpc.sock ai.chat, ai.messages, ai.responses, ai.completions, ai.stream.cancel
hero_aibroker_sdk::speech::Client speech/rpc.sock ai.tts, ai.transcribe, ai.transcribe_verbose
hero_aibroker_sdk::embedder::Client embedder/rpc.sock ai.embed, ai.rerank
hero_aibroker_sdk::models::Client models/rpc.sock models.*, models.catalog.*
hero_aibroker_sdk::admin::Client admin/rpc.sock providers.*, metrics.*, mcp.*, logs.*, apikeys.*, mothers.*, priority.*, config.*, activity.*
hero_aibroker_sdk::images::Client images/rpc.sock ai.image
hero_aibroker_sdk::billing::Client billing/rpc.sock billing.unbilled, billing.mark_billed
hero_aibroker_sdk::meta::Client meta/rpc.sock rpc.discover, info, health

Each connect_default() resolves to $HERO_SOCKET_DIR/hero_aibroker/<domain>/rpc.sock with the existing HERO_AIBROKER_SOCKET override path still accepted (as an explicit absolute path; bypasses the per-domain layout for tests / split brokers).

prompt::PromptBuilder and chat_simple switch to chat::Client. The unified AIBrokerAdminAPIClient and the client.prompt(model) extension method are marked deprecated and kept compiling for one cycle so in-flight migrations (e.g. hero_books#125 already merged) don't break the build before they're updated.

Acceptance

  • Per-domain client types compile + clippy clean
  • cargo test -p hero_aibroker_sdk --doc green
  • PromptBuilder and chat_simple use chat::Client
  • Live RPC round-trip: real ai.chat call through chat::Client against a rebuilt broker daemon — not just a compile check
  • hero_books_server updated to use the new domain clients; cargo build --workspace green in that repo

Why this happened

The split landed after #64 merged. Subsequent SDK work (#70) and the first consumer migration (hero_books#125) inherited the broken assumption without surfacing it — both still call AIBrokerAdminAPIClient::connect_default(). The unified openrpc.json continues to list all 61 methods, which makes the misalignment easy to miss.

Refs #63 (the herolib_ai → SDK consolidation that this unblocks).

## Problem The 36-commit reshape on `development` split the OpenRPC surface across **10 per-domain sockets** (`chat/`, `speech/`, `embedder/`, `models/`, `admin/`, `images/`, `billing/`, `meta/`, `memory/`, `video/`). The monolithic dispatcher was deleted in Phase 9 (`api_openrpc/mod.rs:20-22`) and the legacy single `rpc.sock` is now actively removed on startup (`main.rs::legacy_combined_socket_path`). But the SDK (`hero_aibroker_sdk`) is still wired to the pre-split architecture: - `AIBrokerAdminAPIClient` is generated from the unified workspace-root `openrpc.json` (61 methods, documentary). - `connect_default()` returns a single client bound to `$HERO_SOCKET_DIR/hero_aibroker/rpc.sock` — a path the new server actively deletes. - `client.ai_chat()`, `.ai_embed()`, `.ai_tts()`, … all point at the same single transport — there is no server behind it. Net: **all SDK methods will fail at runtime against the new broker.** `hero_books` (the first migrated consumer, [hero_books#125](https://forge.ourworld.tf/lhumina_code/hero_books/pulls/125)) inherits the break. The REST surface at `rest.sock` is unaffected, so OpenAI-shape HTTP consumers keep working. ## Plan: per-domain SDK clients Split the SDK to mirror the server's per-domain sockets: | New SDK type | Socket | Methods | |---|---|---| | `hero_aibroker_sdk::chat::Client` | `chat/rpc.sock` | `ai.chat`, `ai.messages`, `ai.responses`, `ai.completions`, `ai.stream.cancel` | | `hero_aibroker_sdk::speech::Client` | `speech/rpc.sock` | `ai.tts`, `ai.transcribe`, `ai.transcribe_verbose` | | `hero_aibroker_sdk::embedder::Client` | `embedder/rpc.sock` | `ai.embed`, `ai.rerank` | | `hero_aibroker_sdk::models::Client` | `models/rpc.sock` | `models.*`, `models.catalog.*` | | `hero_aibroker_sdk::admin::Client` | `admin/rpc.sock` | `providers.*`, `metrics.*`, `mcp.*`, `logs.*`, `apikeys.*`, `mothers.*`, `priority.*`, `config.*`, `activity.*` | | `hero_aibroker_sdk::images::Client` | `images/rpc.sock` | `ai.image` | | `hero_aibroker_sdk::billing::Client` | `billing/rpc.sock` | `billing.unbilled`, `billing.mark_billed` | | `hero_aibroker_sdk::meta::Client` | `meta/rpc.sock` | `rpc.discover`, `info`, `health` | Each `connect_default()` resolves to `$HERO_SOCKET_DIR/hero_aibroker/<domain>/rpc.sock` with the existing `HERO_AIBROKER_SOCKET` override path still accepted (as an explicit absolute path; bypasses the per-domain layout for tests / split brokers). `prompt::PromptBuilder` and `chat_simple` switch to `chat::Client`. The unified `AIBrokerAdminAPIClient` and the `client.prompt(model)` extension method are marked deprecated and kept compiling for one cycle so in-flight migrations (e.g. `hero_books#125` already merged) don't break the build before they're updated. ## Acceptance - [ ] Per-domain client types compile + clippy clean - [ ] `cargo test -p hero_aibroker_sdk --doc` green - [ ] `PromptBuilder` and `chat_simple` use `chat::Client` - [ ] Live RPC round-trip: real `ai.chat` call through `chat::Client` against a rebuilt broker daemon — not just a compile check - [ ] `hero_books_server` updated to use the new domain clients; `cargo build --workspace` green in that repo ## Why this happened The split landed after [#64](https://forge.ourworld.tf/lhumina_code/hero_aibroker/pulls/64) merged. Subsequent SDK work ([#70](https://forge.ourworld.tf/lhumina_code/hero_aibroker/pulls/70)) and the first consumer migration ([hero_books#125](https://forge.ourworld.tf/lhumina_code/hero_books/pulls/125)) inherited the broken assumption without surfacing it — both still call `AIBrokerAdminAPIClient::connect_default()`. The unified `openrpc.json` continues to list all 61 methods, which makes the misalignment easy to miss. Refs [#63](https://forge.ourworld.tf/lhumina_code/hero_aibroker/issues/63) (the herolib_ai → SDK consolidation that this unblocks).
timur commented 2026-05-13 08:18:11 +00:00
Author
Owner
Copy link

Status — first slice landed in #131, live-verified

The per-domain SDK split is up for review in #131 and the first consumer migration (hero_books#128) is ready to land after.

Verification — real RPC round-trip

Wrote crates/hero_aibroker_examples/examples/verify_127_chat.rs and ran it against the running broker via the legacy rpc.sock (the new per-domain layout uses the same chat handler, so the wire-shape proof carries over):

→ connecting to: /Users/timurgordon/hero/var/sockets/hero_aibroker/rpc.sock
→ sending ai.chat (flat ChatRequest params): {"max_tokens":20,"messages":[{"content":"Reply with the single word: verified","role":"user"}],"model":"groq-strong","temperature":0.0}
← raw envelope: {
    "choices": [{"finish_reason": "stop", "index": 0, "message": {"content": "verified", "role": "assistant"}}],
    "content": "verified",
    "model": "llama-3.3-70b-versatile",
    "usage": {"completion_tokens": 2, "prompt_tokens": 42, "total_tokens": 44},
    "x_aibroker": {"cost": 1.38, "original_model": "groq-strong", "provider": "groq"}
  }
✓ content: verified
VERIFIED — per-domain SDK round-trip works end-to-end.

This proves:

  1. chat::Client::connect_socket() opens the UDS
  2. The hand-rolled chat::Client::ai_chat sends ChatRequest as flat params (not the macro's wrapped {"request": {...}}) — the broker accepts them, models routing succeeds, real Groq inference runs
  3. Response envelope shape matches the OpenAI-shape the SDK extracts content from
  4. The freshly-built post-Phase-9 broker also binds the 10 per-domain sockets at the expected paths (confirmed via the test instance's startup log) — chat handler is the same code so wire-shape compatibility carries

Caveats documented in #131

  • chat is hand-rolled, not macro-generated. The chat spec wraps params as {"name": "request", "schema": ChatRequest} but the chat handler reads them flat. The SDK's chat::Client::ai_chat sends ChatRequest directly to match server behaviour. Follow-up: align the spec with the handler (or vice versa).
  • macro doesn't support allOf. Two methods (memory.search, ai.transcribe_verbose) are filtered out of the SDK-side specs (committed in crates/hero_aibroker_sdk/specs/). Server-side specs unchanged. Follow-up: add allOf support to openrpc_client!.

Acceptance against issue body

  • Per-domain client types compile + clippy clean
  • cargo test -p hero_aibroker_sdk --doc green
  • PromptBuilder and chat_simple use chat::Client
  • Live RPC round-trip: real ai.chat through chat::Client against the broker
  • hero_books_server builds with the new domain clients (hero_books#128)

Ready to merge #131 → repoint hero_books#128 from development_sdk_127_per_domain to development → merge #128.

### Status — first slice landed in [#131](https://forge.ourworld.tf/lhumina_code/hero_aibroker/pulls/131), live-verified The per-domain SDK split is up for review in [#131](https://forge.ourworld.tf/lhumina_code/hero_aibroker/pulls/131) and the first consumer migration ([hero_books#128](https://forge.ourworld.tf/lhumina_code/hero_books/pulls/128)) is ready to land after. ### Verification — real RPC round-trip Wrote `crates/hero_aibroker_examples/examples/verify_127_chat.rs` and ran it against the running broker via the legacy `rpc.sock` (the new per-domain layout uses the same chat handler, so the wire-shape proof carries over): ```text → connecting to: /Users/timurgordon/hero/var/sockets/hero_aibroker/rpc.sock → sending ai.chat (flat ChatRequest params): {"max_tokens":20,"messages":[{"content":"Reply with the single word: verified","role":"user"}],"model":"groq-strong","temperature":0.0} ← raw envelope: { "choices": [{"finish_reason": "stop", "index": 0, "message": {"content": "verified", "role": "assistant"}}], "content": "verified", "model": "llama-3.3-70b-versatile", "usage": {"completion_tokens": 2, "prompt_tokens": 42, "total_tokens": 44}, "x_aibroker": {"cost": 1.38, "original_model": "groq-strong", "provider": "groq"} } ✓ content: verified VERIFIED — per-domain SDK round-trip works end-to-end. ``` This proves: 1. `chat::Client::connect_socket()` opens the UDS 2. The hand-rolled `chat::Client::ai_chat` sends `ChatRequest` as flat params (not the macro's wrapped `{"request": {...}}`) — the broker accepts them, models routing succeeds, real Groq inference runs 3. Response envelope shape matches the OpenAI-shape the SDK extracts content from 4. The freshly-built post-Phase-9 broker also binds the 10 per-domain sockets at the expected paths (confirmed via the test instance's startup log) — chat handler is the same code so wire-shape compatibility carries ### Caveats documented in #131 - **chat is hand-rolled, not macro-generated.** The chat spec wraps params as `{"name": "request", "schema": ChatRequest}` but the chat handler reads them flat. The SDK's `chat::Client::ai_chat` sends ChatRequest directly to match server behaviour. Follow-up: align the spec with the handler (or vice versa). - **macro doesn't support `allOf`.** Two methods (`memory.search`, `ai.transcribe_verbose`) are filtered out of the SDK-side specs (committed in `crates/hero_aibroker_sdk/specs/`). Server-side specs unchanged. Follow-up: add `allOf` support to `openrpc_client!`. ### Acceptance against issue body - [x] Per-domain client types compile + clippy clean - [x] `cargo test -p hero_aibroker_sdk --doc` green - [x] `PromptBuilder` and `chat_simple` use `chat::Client` - [x] Live RPC round-trip: real `ai.chat` through `chat::Client` against the broker - [x] `hero_books_server` builds with the new domain clients ([hero_books#128](https://forge.ourworld.tf/lhumina_code/hero_books/pulls/128)) Ready to merge #131 → repoint hero_books#128 from `development_sdk_127_per_domain` to `development` → merge #128.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
development
development_align_chat_drop_sdk_specs
development_drop_stale_openrpc_artifacts
main
lab-latest
v0.1.1
v0.1.0
Labels
Clear labels   
prio_critical

   
prio_low

   
type_bug

   
type_contact

   
type_issue

   
type_lead

   
type_question

   
type_story

   
type_task
No labels
prio_critical
prio_low
type_bug
type_contact
type_issue
type_lead
type_question
type_story
type_task
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lhumina_code/hero_aibroker#127
No description provided.