Move Python client generation from hero_router to hero_rpc (build-time, authoritative) #31

New issue

Open

opened 2026-04-21 16:46:35 +00:00 by timur · 0 comments

timur commented

2026-04-21 16:46:35 +00:00

Owner

Problem

Python clients are generated by hero_router at runtime from each service's rpc.discover response. This:

Silently drifts from source of truth. Rust types, Rust server code, OpenRPC spec, and JS client are all generated by hero_rpc from each service's .oschema at build time. Python alone is generated from runtime discovery. When the running server has a stale cached spec, the Python client is stale — exactly the hero_logic_client.py bug diagnosed in #29.
Breaks the "name = responsibility" rule. hero_rpc is the RPC codegen repo. hero_router is the routing and discovery repo. Python client generation belongs in the codegen repo, not the router.
Fails the same way per language. If we add Go, Swift, Ruby clients later, they'd each face the same question. hero_rpc is the right answer for all of them.

Proposal

Move Python client generation into hero_rpc's build-time pipeline. Each service's .oschema produces a Python client file alongside its Rust types, OpenRPC spec, and JS client. The generated file is committed to each service's repo (same model as the other artifacts today).

hero_rpc generator    → .oschema → {rust_types.rs, rust_server.rs, openrpc.json, client.js, client.py}
each service          → commits all five artifacts alongside its source
hero_router           → discovers services at runtime, stages pre-generated clients to
                        ~/.hero/var/router/python/ (just file copy, no codegen)

hero_router retains interface-stub generation as a runtime concern — see #companion issue. Interface stubs are LLM-context-window summaries of live discovered services; that IS a runtime-discovery product, not a build-time one.

What moves

hero_router/crates/hero_router/src/python_codegen.rs → hero_rpc/crates/generator/src/python/
Invocation moves from rpc.discover-at-runtime to .oschema-at-build-time, same as Rust/OpenRPC/JS today
Each service's build.rs gains Python output to its generated/python/ directory

What stays

hero_router's interface-stub generator (separate file, separate purpose — see companion issue)
hero_router's staging of clients at ~/.hero/var/router/python/ — just no longer generates them, only copies

Fallback: runtime codegen for unknown services

For services discovered at runtime that weren't part of the build (e.g. third-party Hero services added later), hero_router can fall back to runtime generation using its existing code path as a last resort. Common case (Hero-workspace services) becomes fast + correct; exotic case still works.

Migration

Ship the new build-time generator in hero_rpc
Update each Hero service's build.rs to emit Python client to generated/python/
Commit the generated files in each service's repo
hero_router's python_codegen.rs becomes a thin "read-generated-file + copy-to-staging-dir" module, with runtime codegen kept only as the unknown-service fallback
Verify: hero_logic_client.py has play_start, workflow_version_fetch, workflow_clone, example_to_input_data, etc. — and is guaranteed to match the running server because it was built from the same .oschema

Non-goals

Not a PyPI publishing strategy. Files committed to each repo is enough for an internal stack.
Not a stale-cache fix. That's a separate issue (see companion).

#29 — surface service methods in the current runtime generator (tactical, lands first, unblocks flows-as-python work)
hero_logic#10 — the epic this eventually feeds into via cleaner codegen foundations

## Problem Python clients are generated by hero_router at runtime from each service's `rpc.discover` response. This: 1. **Silently drifts from source of truth.** Rust types, Rust server code, OpenRPC spec, and JS client are all generated by hero_rpc from each service's `.oschema` at build time. Python alone is generated from runtime discovery. When the running server has a stale cached spec, the Python client is stale — exactly the `hero_logic_client.py` bug diagnosed in #29. 2. **Breaks the "name = responsibility" rule.** hero_rpc is the RPC codegen repo. hero_router is the routing and discovery repo. Python client generation belongs in the codegen repo, not the router. 3. **Fails the same way per language.** If we add Go, Swift, Ruby clients later, they'd each face the same question. hero_rpc is the right answer for all of them. ## Proposal Move Python client generation into hero_rpc's build-time pipeline. Each service's `.oschema` produces a Python client file alongside its Rust types, OpenRPC spec, and JS client. The generated file is committed to each service's repo (same model as the other artifacts today). ``` hero_rpc generator → .oschema → {rust_types.rs, rust_server.rs, openrpc.json, client.js, client.py} each service → commits all five artifacts alongside its source hero_router → discovers services at runtime, stages pre-generated clients to ~/.hero/var/router/python/ (just file copy, no codegen) ``` hero_router **retains** interface-stub generation as a runtime concern — see #companion issue. Interface stubs are LLM-context-window summaries of live discovered services; that IS a runtime-discovery product, not a build-time one. ## What moves - `hero_router/crates/hero_router/src/python_codegen.rs` → `hero_rpc/crates/generator/src/python/` - Invocation moves from `rpc.discover`-at-runtime to `.oschema`-at-build-time, same as Rust/OpenRPC/JS today - Each service's `build.rs` gains Python output to its `generated/python/` directory ## What stays - hero_router's interface-stub generator (separate file, separate purpose — see companion issue) - hero_router's staging of clients at `~/.hero/var/router/python/` — just no longer generates them, only copies ## Fallback: runtime codegen for unknown services For services discovered at runtime that weren't part of the build (e.g. third-party Hero services added later), hero_router can fall back to runtime generation using its existing code path as a last resort. Common case (Hero-workspace services) becomes fast + correct; exotic case still works. ## Migration - Ship the new build-time generator in hero_rpc - Update each Hero service's `build.rs` to emit Python client to `generated/python/` - Commit the generated files in each service's repo - hero_router's `python_codegen.rs` becomes a thin "read-generated-file + copy-to-staging-dir" module, with runtime codegen kept only as the unknown-service fallback - Verify: `hero_logic_client.py` has `play_start`, `workflow_version_fetch`, `workflow_clone`, `example_to_input_data`, etc. — and is guaranteed to match the running server because it was built from the same `.oschema` ## Non-goals - **Not** a PyPI publishing strategy. Files committed to each repo is enough for an internal stack. - **Not** a stale-cache fix. That's a separate issue (see companion). ## Related - #29 — surface service methods in the current runtime generator (tactical, lands first, unblocks flows-as-python work) - hero_logic#10 — the epic this eventually feeds into via cleaner codegen foundations