Migrate OSIS serving to hero_lib serve_rpc_domains and DELETE HeroOsisServer (no adapter, no deprecation) #154
Labels
No labels
prio_critical
prio_low
type_bug
type_contact
type_issue
type_lead
type_question
type_story
type_task
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
lhumina_code/hero_blueprint#154
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Goal
Straight migration, no half-measures. OSIS multi-domain serving must move entirely onto Kristof's
hero_lifecycle::serve_rpc_domains(hero_lib) and the old replica must be deleted —hero_rpc_osis::rpc::bootstrap::HeroOsisServer, thehero_osis_server!()macro path, therpc2_adapter-based serving, and the stopgapHeroOsisServer::serve_domains()(branchfeature/osis-multidomain-serve). hero_osis (and the CRM example) must boot on the new serving.Explicitly NOT wanted: no
RpcModule → axum::Routeradapter/shim, no deprecation window, no back-compat alias. Remove the old code; make OSIS work with the new directly.Why
HeroOsisServerandserve_rpc_domainsare two parallel implementations of the same control-plane + per-domain-socket topology. Kristof builtserve_rpc_domainsin hero_lib; we were replicating it in hero_blueprint over the hero_rpc2 transport. That replica (incl. theserve_domains()I added as a stopgap) is the duplication to delete.Approach (straight migration)
OSIS domain handlers should be served natively by
serve_rpc_domains, which means the OSIS generator emits domain servers the hero_lib way:serve_rpc_domains(Vec<(name, openrpc_json, axum::Router)>)(hero_libcrates/hero_lifecycle/src/manifest.rs:733) wants oneaxum::Router(POST/rpc+ GET/openrpc.json) per domain — produced today byopenrpc_server!→<domain>_rpc::router(impl <Domain>Api).crates/generator/src/rust/rust_osis.rs+ thehero_osis_server!/HeroOsisServermachinery) to emit/feed hero_lib-servable domain routers instead of jsonrpseeRpcModules behind hero_rpc2's own HTTP transport.DBTyped/OSIS-storage CRUD) — only the serving/transport plumbing changes from hero_rpc2 → hero_lib.Delete
HeroOsisServer(crates/osis/src/rpc/bootstrap.rs) incl.serve()and the stopgapserve_domains().hero_osis_server!()macro (crates/osis/src/lib.rs) + the generator's emission of it (crates/generator/src/build/scaffold.rsservermain.rs).rpc2_adapterregister-into-RpcModule+ServerBuilder::serve_httpfor multi-domain) — to the extent it's only used for OSIS serving.Update
crates/hero_osis_server/src/bin/hero_osis.rs+service.toml) to boot viaserve_rpc_domains(control plane + per-domain sockets come for free, incl.discover_domains).examples/crm) servermain.rsto the new path.Done-when
serve_rpc_domains) — noHeroOsisServer, no adapter, no stopgapserve_domains().Contactin identity + business) don't collide;discover_domainsworks.Context: hero_osis re-architecture (Claude). Pairs with #155 (CRUD/openrpc struct shape). Supersedes the stopgap on hero_blueprint
feature/osis-multidomain-serve+ hero_osisfeature/osis-hero-db-storage.Converge OSIS multi-domain serving onto hero_lib serve_rpc_domains (remove HeroOsisServer duplication)to Migrate OSIS serving to hero_lib serve_rpc_domains and DELETE HeroOsisServer (no adapter, no deprecation)Picking this up (together with the paired issue — same OSIS handler regeneration). Working branches: hero_blueprint
feature/osis-multidomain-serve(rebased target) + hero_osisfeature/osis-hero-db-storage.Exploring the exact target first: how a storage-backed OSIS handler plugs into hero_lib
openrpc_server!(<Domain>Apitrait + per-domain router) andserve_rpc_domains. Will post the concrete execution plan here next, then implement in chunks with a comment per chunk so this issue stays a complete record.Execution plan (picked up) — target architecture nailed
Goal restated: delete hero_blueprint's parallel OSIS serving (
HeroOsisServer,hero_osis_server!,rpc2_adapterdispatch,OsisAppRpcHandler, the stopgapserve_domains()) and serve OSIS via hero_lib'sopenrpc_server!+serve_rpc_domains. No adapter, no deprecation.Target shapes (verified against hero_lib
origin/development)herolib_derive::openrpc_server!(spec=<oschema dir>, service_toml=…)generates, per domain:*Input/*Outputwrappers) from the OpenRPC spec.<Domain>Apitrait:async fn <type>_set(&self, ctx:&HeroRequestContext, input:<Type>SetInput) -> Result<<Type>SetOutput, RpcError>etc. CRUD =_get,_set,_delete,_list,_list_full,_exists,_find(no_new;_setis upsert,SetInput{ data:<Type> }, emptysid→create).<domain>_rpc::router(impl) -> axum::Router,<domain>_rpc::OPENRPC_JSON, and free fnsserve_domains(...)/serve_domains_with(...)→manifest().serve_rpc_domains(...)(control plane +rpc_<domain>.sockper domain +discover_domains).Refs: hero_lib
crates/derive/src/openrpc_server.rs:430-760, examplecrates/hero_lifecycle/examples/server/{main.rs,rpc/dom1_impl.rs},crates/oschema/src/oschema/openrpc.rs:520-640.What hero_blueprint's OSIS generator becomes
Emit, per domain:
#[async_trait] impl <Domain>Api for Osis<Domain>whose bodies call the existing DBTyped storage (get/set/delete/list/exists) + indexer write-through, mapping errorsBox<dyn Error>→RpcError._setbody: empty/absentsid⇒ create (mint sid, stamp times) ⇒self.<t>_db.set(&mut obj); else load+update. The servicemain.rsis justserve_domains(OsisDomain::default()/::create(...), …).DELETE (hero_blueprint)
crates/osis/src/rpc/bootstrap.rs(HeroOsisServer + serve/serve_domains),hero_osis_server!macro (crates/osis/src/lib.rs), therpc2_adaptermulti-domain serving +OsisAppRpcHandler/handle_rpcdispatch, and the generator emission of all the above incrates/generator/src/rust/rust_osis.rs(the bespoke CRUD methods +handle_rpc+OsisAppRpcHandlerimpl). Thehero_osis_server!-based servermain.rstemplate incrates/generator/src/build/scaffold.rs.KEEP
DBTyped storage + SID/counter logic, OsisIndexer write-through, the trigger hooks, the rootobject types (modulo the type-ownership decision below).
Step order
impl <Domain>Api(+ run/emitopenrpc_server!for trait/types/router) instead of the bespoke handler/serving.main.rs→serve_domains(...); drop the per-domain-socketservice.tomlhack (serve_rpc_domains owns sockets).⚠️ Gating decision: rootobject type ownership / storage-type impedance
openrpc_server!generates wire types from the spec:User { sid: Option<String>, created_at: Option<String>, … }. OSIS storage (DBTyped<T: OsisObject>) currently usesUser { sid: SmartId, created_at: OTime }andOsisObject::sid()->&SmartId. These don't match. Options:DBTyped/OsisObjectoperate on theopenrpc_server!-generated types (String sid, String/OTimetimes); the generator emitsimpl OsisObject for <Type>(local to the consumer crate, no orphan issue). True convergence, one set of types, butOsisObjectmust accept String sids (small change to the trait + SID minting).impl <Domain>Apiconverts spec-types↔storage-types per call. Preserves typing, but doubles the type set + adds conversion codegen.Recommendation: (A) — it's the actual convergence (single source of truth = the OpenRPC spec) and matches Kristof's "one schema per object" direction (979c1781). Pending confirmation before the generator rewrite, since it shapes everything.
(Cross-refs #155 for the CRUD method/struct shape — same regeneration. Journaling continues here per chunk.)
Decision: (A) one type model — storage adopts the spec/wire types. DBTyped/OsisObject will operate on the
openrpc_server!-generated rootobject types (Stringsid, String/OTimetimestamps); the generator emitsimpl OsisObject for <Type>locally in the consumer crate. Single source of truth = the OpenRPC spec; matches 979c1781 (one schema per object).Implication:
OsisObject(inhero_rpc_osis::db) moves fromsid()->&SmartIdto a String-sid surface, andDBTyped<T>keys on the String sid. This revises the hero_db storage layer (thefeature/osis-hero-db-storagework) — sid minting moves to the create path of_set.Chunk order (each → its own commit + comment here)
OsisObjectString-sid surface +DBTyped<T>keying/minting on String sid; keep redb/OTOML + indexer. (base for everything)#[async_trait] impl <Domain>Api for Osis<Domain>(DBTyped bodies,_setupsert, errors→RpcError) + adoptopenrpc_server!for trait/types/router/spec; delete bespoke CRUD methods +handle_rpc+OsisAppRpcHandleremission.hero_osis_server!+ rpc2_adapter serving + stopgapserve_domains().serve_domains(...)main; build+boot+CRUD+collision check.Starting chunk 1 on hero_blueprint
feature/osis-multidomain-serve.Chunk 1 spec —
OsisObject/DBTypedString-sid foundation (executable)Base branch: hero_blueprint
feature/osis-multidomain-serve(off newest dev). File:crates/osis/src/db/db.rs. (Storage backend stays dev's filesystem-OTOML here; re-landing the hero_db/redb backend fromfeature/osis-hero-db-storageis a separate orthogonal merge — note both branches touch this trait, reconcile during merge.)OsisObject — before → after
Current (
db.rs:~32):After (String-sid, matches
openrpc_server!-generated wire types wheresid: Option<String>):(Generated impls read/write the type's
sid: Option<String>:sid()→self.sid.clone().unwrap_or_default();set_sid(s)→self.sid = Some(s).)DBTyped changes (
db.rs)object_path(sid:&str)→type_dir().join(format!("{sid}.otoml")).get/delete/exists(&self, sid:&str),list(&self)->Result<Vec<String>>(return String sids; filter__osis_meta:-style as today / read dir entries).set(&self, obj:&mut T): ifobj.sid()empty ⇒ mint viaid_new()→obj.set_sid(new_sid_string); else use as-is; write OTOML atobject_path(&obj.sid()).id_new(&self) -> Result<String>: keep the SmartId compute/counter logic internally, returnsid.as_str()(String). (SmartId stays the minting algorithm; the stored/keyed sid is its String form — same as today'sas_str().)NotFound(sid.to_string()).Ripple (chunk 2 handles the generator side)
impl OsisObject for <Type>(currently inrust_struct.rs/ osis_impl) must switch to the String-sid bodies above, against theopenrpc_server!types (sid: Option<String>,created_at/updated_at: Option<String>), and stampcreated_at/updated_atas strings on create.rust_osis.rsCRUD callers: covered by the chunk-2 rewrite toimpl <Domain>Api(the olduser_new/user_set(sid,..)go away;_setupsert readsobj.sid()).rhai.rsDynamicObjectimpl + tests indb/tests.rs: update to String sid.Acceptance
cargo check -p hero_rpc_osis --features rpcgreen after chunk 1 + the matching generated-impl change;db/tests.rs(String sid) pass.Status / compaction boundary: Plan + target shapes + decision (A) + this chunk-1 spec are now fully journaled here — enough to execute chunks 1→5 cleanly in a fresh context. Resuming = start chunk 1 from this comment on
feature/osis-multidomain-serve.Chunk 1 DONE —
OsisObject/DBTypedString-sid foundation ✅Committed
0164e33onfeature/osis-multidomain-serve(pushed).What landed (
crates/osis/src/db/db.rs,db/tests.rs,rhai.rs):OsisObject:sid(&self) -> String/set_sid(&mut self, sid: String). Empty string ⇒ unsaved (create onset). Matches theopenrpc_server!-generated wire types wheresid: Option<String>(generated impls will returnself.sid.clone().unwrap_or_default()).DBTyped<T>:object_path(&str),get/delete/exists(&self, sid: &str),list(&self) -> Result<Vec<String>>(filters to valid-SmartId stems),set()mints on empty sid,id_new() -> Result<String>. SmartId stays the minting algorithm; the stored/keyed sid is its string form (sid.as_str()), so distributed collision-free minting + the flock counter are unchanged.rhai.rsDynamicObject(sid: Option<String>) anddb/tests.rs(SimpleItem/ComplexItemnowOption<String>) updated;gid()test helper decodes a minted sid back to its global id where tests asserted ordering.Verified:
cargo check -p hero_rpc_osisgreen on default /rpc/rhai; 39 db tests pass (1 ignored — needsPATH_ROOT), incl. both concurrency/no-collision tests and counter-persistence.Note for the hero_db merge: this touches the same
OsisObject/DBTypedtrait thatfeature/osis-hero-db-storage(redb backend) revises — reconcile the String-sid surface during that merge. Backend here stays dev's filesystem-OTOML.Next: chunk 2 — generator emits
#[async_trait] impl <Domain>Api for Osis<Domain>(DBTyped bodies,_setupsert, errors→RpcError) and adopts hero_libopenrpc_server!for the types/trait/router/spec; drops the bespoke CRUD +handle_rpc+OsisAppRpcHandleremission (also resolves #155's_new/<Type>Inputremoval in the same pass).Chunk 2 build sheet — generator adopts
openrpc_server!, emitsimpl <Domain>Api(executable)Mapped the full generator call graph + the
openrpc_server!target. Chunk 2 is the keystone; it reshapes three layers (generator emission, scaffold templates, and the core/server crate split). Journaling the concrete plan + the one gating decision it surfaces, so it can be executed in a focused pass without leaving a broken tree.Generator call graph (where the rewrite lands)
build.rs→hero_rpc_osis::build::OschemaBuilder::from_service_toml().generate().OschemaBuilder::generate_domain()(crates/generator/src/build/emit/domain.rs) →Generator::from_dir(dir).server(domain).generate()→crates/generator/src/generate/rust_server.rs→crate::rust::generate_rust_osis(&domains)(rust_osis.rs:319/337).OUT_DIR): core types incrates/<name>/src/<dom>/types_generated.rs; server handler incrates/<name>_server/src/<dom>/server/osis_server_generated.rs; spec atdocs/openrpc.json.crates/generator/src/build/scaffold.rs.⚠️ Gating decision (shapes everything): where is
openrpc_server!invoked?openrpc_server!(spec=<dir>, service_toml=…)generates the wire types itself (the<Type>structs withsid: Option<String>), the<Domain>Apitrait, the<dom>_rpc::router/OPENRPC_JSON, and the top-levelserve_domains(...). So today'srust_struct.rstype emission becomes redundant and must be retired in favour of the macro's types — otherwise two definitions collide.Recommendation — invoke
openrpc_server!in the CORE crate (crates/<name>/), not the server crate. Rationale + resulting layout (minimises ripple, keeps types where SDK/admin/web already import them):crates/<name>): oneopenrpc_server!(spec="schemas", service_toml="…/service.toml")invocation ⇒ emits per-domain module<dom>{ wire types +<Domain>Apitrait +<dom>_rpc(router,OPENRPC_JSON) } + top-levelserve_domains(...). Generator additionally emitsimpl OsisObject for <dom>::<Type>here (same crate as the type ⇒ no orphan issue; String-sid bodies from chunk 1 +indexed_fields_json).crates/<name>_server): generator emits theOsis<Dom>struct (holdsDBTyped<<dom>::<Type>>per rootobject +OsisIndexer+ triggers) and#[async_trait] impl <dom>::<Domain>Api for Osis<Dom>with DBTyped-backed bodies.main.rs=<name>::serve_domains(Osis<Dom1>::create(db,uid)?, Osis<Dom2>::create(...)?, …).await(trait is generic, impls passed in by value — they can live in the server crate whileserve_domainslives in core).This is "true convergence" (decision A): the OpenRPC spec is the single source of the types, generated once by the macro.
Target generated
impl <Domain>Apibody shape (per rootobject, DBTyped-backed)Maps the macro's CRUD trait methods onto chunk-1 storage. No
_new;_setis upsert (resolves #155). ErrorsBox<dyn Error>→RpcError(not_found→-32002,invalid_params→-32602, elseinternal→-32603).Note:
get/delete/existsnow take a&strsid directly (chunk 1) — drop the oldSmartId::parse(sid)step._setno longer constructs from a partial<Type>Input(that struct is gone, #155) — it takes the full object.DELETE (chunk 3, after this compiles)
rust_osis.rsbespoke CRUD (_new, two-arg_set,_rpc_*,handle_rpc,OsisAppRpcHandlerimpl emission);rust_struct.rstype +<Type>Input+From<&Type>emission;schemas/openrpc.rs_new/input-schema emission; the SDKnew/<Type>NewInput. And the runtime infra:crates/osis/src/rpc/bootstrap.rs(HeroOsisServer +serve_domainsstopgap),hero_osis_server!(crates/osis/src/lib.rs),rpc2_adapter+OsisAppRpcHandler(crates/osis/src/rpc/{rpc2_adapter,server}.rs).Scaffold template changes (
build/scaffold.rs)herolib_derive(foropenrpc_server!) +herolib_oschema_server(HeroRequestContext/RpcError) +async-trait.hero_rpc2+jsonrpsee; keephero_rpc_osis(storage/indexer) + addasync-trait,herolib_oschema_server.main.rs: replacehero_osis_server!()...serve()with<name>::serve_domains(...).service.toml: emit control-planerpc.sock+ onerpc_<domain>.sockper domain (serve_rpc_domains Model A), matching the hero_lib example service.toml.Recommended sequencing (de-risk before touching the generator)
examples/crm) to the shape above and get it to build + boot + driver-green — provesopenrpc_server!works against an OSIS oschema (7 rootobjects + service) end-to-end. This is the exact template the generator must emit. (This is the old chunk-4 pulled forward as the proof.)generate_rust_osis+rust_struct+ scaffold emit that template; regenerate CRM and confirm byte-for-byte-equivalent build+boot.feature/osis-hero-db-storage; drop the per-domain-socketservice.tomlhack; build+boot+CRUD+collision+discover_domains.Acceptance (chunk 2)
examples/crmbuilds withopenrpc_server!-generated types/trait/router;Osis<Dom>impls compile against the trait;serve_domainsboots; CRM driver passes all 8 phases over per-domain sockets.Status: chunk 1 landed (
0164e33, verified). Chunk 2 is a focused cross-crate pass best executed start-to-finish in one sitting; this sheet (call graph + layout decision + body shapes + scaffold diffs + sequencing) makes it executable. Resume = step 1 above (hand-prototype CRM target onfeature/osis-multidomain-serve).Decision:
openrpc_server!invoked in the SERVER crate (timur)Confirmed: per Kristof's hero_lib pattern + hero_skills best practices, the macro lives in
crates/<name>_server(not core). Verified againsthero_lib/crates/hero_lifecycle/examples/server/main.rs—openrpc_server!(spec=…, service_toml=…)+serve_domains(...)+#[async_trait] impl <Domain>Apiall live in the server binary; the SDK is independent (openrpc_client!against the spec, its own types), so core needn't own the wire types.Confirmed OSIS oschema format (
Company = { name: str @index … }+service CrmService { … }) is shape-identical to the hero_lib demo oschema, soopenrpc_server!(spec=<dir of domain subdirs>)consumes OSIS schemas as-is.Revised chunk-2 layout (server crate owns the macro):
crates/<name>_server/src/main.rs:openrpc_server!(spec="<schemas dir>", service_toml="service.toml", save_openrpc_dir=…)→ per-domain module {types +<Domain>Api+<dom>_rpc} +serve_domains(...); thenserve_domains(OsisCrm::create(db,uid)?, …).await.impl OsisObject for <dom>::<Type>(String-sid, chunk 1),Osis<Dom>struct (DBTyped + OsisIndexer + triggers),#[async_trait] impl <dom>::<Domain>Api for Osis<Dom>with DBTyped bodies.<Domain>Apimethod names: CRUD<type>_<op>(e.g.company_get), service methods<service_snake>_<method>(e.g.crm_service_open_opportunities).*_get→db.get(&input.sid)then field-by-field into<Type>GetOutput;*_list_full→Vec<<Type>>into…Output{value};*_setupsert (full object, mint on empty sid) + indexer write-through; errors →RpcError.herolib_derive,herolib_oschema_server,async-trait; dropshero_rpc2/jsonrpsee; keepshero_rpc_osis(storage/indexer). service.toml emits control-planerpc.sock+ per-domainrpc_<domain>.sock.Approach: rewriting the generator directly (no separate hand-prototype). The
hero_rpc_generatorlib stays compilable throughout (it only templates strings); the broken stretch is contained to regenerate→build CRM, where I'll iterate to green.Chunk 2 — generator internals mapped (exact edit sites for the execution pass)
Read the full emission. The rewrite touches these exact spots; capturing so the focused pass is pure mechanics:
crates/generator/src/rust/rust_osis.rs(1756 lines) —generate_domain(159) calls, in order:generate_handler_trait,generate_oschema_constant,generate_domain_struct(227 — KEEP, holds*_db: DBTyped<T>+indexer+ service handlers),generate_domain_impl(328 — KEEP ctor/indexer wiring; the CRUD methods it emits viagenerate_crud_methods(518) get rewritten),generate_rpc_handler(818 — DELETE: RpcRequest/Response +handle_rpc+ per-rogenerate_rpc_methods(1067) +generate_service_rpc_methods(1280)),generate_osis_app_rpc_handler(1393 — DELETE, incl. theOsisDomainInitimpl which must be replaced bycreate()kept as a plain fn). New: emit#[async_trait] impl <dom>::<Domain>Api for Osis<Dom>with DBTyped bodies (the_get/_set/_delete/_list/_list_full/_exists/_findmapping from comment 38174) + the service-method trait fns._getdoesSmartId::parse(sid)thendb.get(&smart_id); chunk 1 madedb.get(&str), so drop the parse.obj.sid.as_str().to_string()→obj.sid().crates/generator/src/rust/rust_struct.rs(2316 lines):generate_struct(899) emits<Name>(withpub sid: SmartId,created_at/updated_at: OTime) +<Name>Input+From<&Name>+<Name>FindParams. Under the new model the type + Input come fromopenrpc_server!(server crate) → stop emitting<Name>/<Name>Input/From(resolves #155); keep<Name>FindParams(the macro doesn't generate it) OR confirm the macro emits<Type>FindParams(it referencedTodoFindParams— verify before deleting our emission).generate_osis_impl_block(51) currently emitsimpl OsisObjectwithfn sid(&self)->&SmartId { &self.sid }/set_sid(SmartId)— rewrite to String-sid (self.sid.clone().unwrap_or_default()/self.sid = Some(sid)) against the macro types'sid: Option<String>, and stampcreated_at/updated_atasOption<String>. This is the generator-side of chunk 1; it's the function that survives as the OsisObject bridge.crates/derive— the#[derive(OsisObject)]macro is dead under the new model (can't derive on macro-generated types); generated code uses the explicitimplfromgenerate_osis_impl_block. Remove its use in scaffolds.Build orchestration —
generate/rust_server.rs+build/emit/*+build/scaffold.rs: server crate gets theopenrpc_server!(spec=<schemas>, service_toml=…, save_openrpc_dir=…)invocation inmain.rs,#[path]mods per domain for theimpl <Domain>Api, thenserve_domains(OsisCrm::create(db,uid)?, …).await. service.toml → control-planerpc.sock+ per-domainrpc_<domain>.sock. Cargo.toml: +herolib_derive,+herolib_oschema_server,+async-trait; −hero_rpc2,−jsonrpsee.Latent state after chunk 1 (expected)
generate_osis_impl_block+ theOsisObjectderive still emit&SmartIdbodies, so a consumer regenerated today won't compile against chunk 1's String-sid trait until chunk 2 lands. This is by design (chunk 1 shipped knowing chunk 2 follows) and is contained —hero_rpc_osisitself + its tests are green; no consumer was regenerated.Checkpoint: chunk 1 shipped+verified (
0164e33); chunk 2 fully scoped to exact edit sites + macro-location decided (server crate). Chunk 2 is one atomic cross-crate pass (≈6 files + regenerate/debug CRM to green) with a long consumer-non-compiling window — to be executed as a dedicated focused sitting. Resume = gutrust_osis.rsserving emission + rewritegenerate_osis_impl_block+ wireopenrpc_server!into the server scaffold, then regenerate CRM to green.Chunk 2 —
openrpc_server!identifier rules pinned (no re-derivation needed)Confirmed against hero_lib
crates/derive/src/openrpc_server.rs+crates/oschema/src/oschema/openrpc.rs:to_snake_case(domain.name)→ e.g.crm. Types/trait/<stem>_rpclive atcrate::<mod>::….to_pascal_case(spec.info.title);info.title= firstservice {}block name (openrpc.rs:478-487), else"OSchema API". CRMservice CrmService⇒info.title="CrmService"⇒ traitcrm::CrmServiceApi. (Domains with no service block get a degenerate stem — every OSIS domain should carry a service block.)to_pascal_case(fn_name)+{Input,Output}. CRUD fn =<type>_<op>(company_get→CompanyGetInput/Output); service fn =<service_snake>_<method>(crm_service_open_opportunities→CrmServiceOpenOpportunitiesInput/Output).…GetOutput=full object fields;…SetInput{ #[serde(flatten)] data: <Type> };…SetOutput{value:String};…ListOutput{value:Vec<String>};…ListFullOutput{value:Vec<<Type>>};…ExistsOutput{value:bool};…DeleteOutput{value:bool};…FindInput{params:Option<<Type>FindParams>},…FindOutput{value:Vec<String>}.Starting the emission rewrite now, foundation-first: core type +
OsisObjectimpl →Option<String>sid /Option<String>timestamps (matches the macro wire types + chunk-1 trait), generator lib kept green at each step.Chunk 2a DONE — String-sid wire types + OsisObject impl ✅ (
8ba3eff)Pushed on
feature/osis-multidomain-serve. Generator lib compiles; 151 generator unit tests pass (also fixed 5 pre-existing staleto_openrpc().unwrap()test calls that were blocking the suite).rust_struct.rs generate_struct: root-object fields →sid/created_at/updated_at: Option<String>(wasSmartId/OTime) +skip_serializing_if; dropped theSmartIdimport and the root-object-drivenOTimeimport.rust_struct.rs generate_osis_impl_block:OsisObjectimpl → String-sid surface (sid()->self.sid.clone().unwrap_or_default(),set_sid(Some(..))), matching chunk-1's trait.Refinement to the 2b plan (server emission)
Because the decision is
openrpc_server!in the server crate, the macro generates the wire types there (crate::<dom>::<Type>). Therefore theDBTyped<T>fields andimpl OsisObject for <Type>must be emitted on the macro types in the server crate — so:generate_osis_impl_block(core crate) becomes dead for storage; the OsisObject-impl emission moves intorust_osis.rs(server emission), referencingcrate::<dom>::<Type>and reusing theindexed_fields_jsonclassifier (share it out ofrust_struct.rs).Osis<Dom>’sDBTyped<{ro.name}>fields resolve against the macro module (serveruse crate::<dom>::*rather thanuse super::core::*).2b remaining (server emission rewrite — the large piece)
In
rust_osis.rs: keepgenerate_domain_struct+ ctor/indexer wiring; replacegenerate_crud_methodsoutput with#[async_trait] impl <dom>::<Stem>Api for Osis<Dom>(CRUD bodies from comment 38174,_get/_set/_delete/_list/_list_full/_exists/_find;get/delete/existstake&strdirectly — noSmartId::parse;obj.sid()notobj.sid.as_str()); deletegenerate_rpc_handler/generate_rpc_methods/generate_service_rpc_methods/generate_osis_app_rpc_handler; emitimpl OsisObject for <dom>::<Type>; replaceOsisDomainInit::createwith a plaincreate()fn. Service methods (e.g.crm_service_open_opportunities) bridge the generated trait method to the preservedCrmServiceHandler. Then scaffold (openrpc_server!inmain.rs+#[path]impl mods +serve_domains, per-domain-socketservice.toml, Cargo dep swap) and regenerate CRM to green.Status: chunk 1 (
0164e33) + chunk 2a (8ba3eff) shipped/verified. 2b is the next contiguous large pass (gutrust_osis.rsserving emission + scaffold + CRM regen).Chunk 2b-core DONE — generator emits
impl <Stem>Apioveropenrpc_server!✅ (760ed90)rust_osis.rsrewritten: −1,389 / +389 lines. Generator lib compiles; 151 generator unit tests pass. Per domain it now emits:impl OsisObject for <Type>on the macro wire types (String-sid;indexed_fields_jsonviaserde_json::to_value— uniform numeric/bool/string/enum/Option, no classifier needed).Osis<Domain>struct (per-rootobjectDBTyped+ per-domain indexer + service handlers) withnew()/create()/init_handlers().#[async_trait] impl <Stem>Api for Osis<Domain>— CRUD backed byDBTyped+ indexer write-through (get→GetOutput field-map;setupsert with the full object, mint on empty sid;delete/list/list_full/exists), errors→RpcError; service methods bridge to the contributor<Service>Traithandler.Deleted:
handle_rpc, the_rpc_*dispatchers,RpcRequest/RpcResponse,OsisAppRpcHandlerimpl, the bespoke_new/_set(sid,data)CRUD,OSCHEMA_SOURCEemission (resolves #155's_new/<Type>Inputremoval).Remaining in chunk 2 (scaffold + orchestration + regen — the next pass)
mod <dom>at the crate root, so the generated impl file must live in a module that does NOT clash with asrc/<dom>/dir module — emit it as a distinct module (e.g.#[path] mod osis_<dom>) that doesuse crate::<dom>::*. The current emission’suse crate::<dom>::*;/use super::rpc::*;are provisional until this is fixed inbuild/emit/rust_server.rs+scaffold.rs.main.rs(build/scaffold.rs generate_server_main_rs): replacehero_osis_server!()…serve()withherolib_derive::openrpc_server!(spec="<schemas>", service_toml="service.toml", save_openrpc_dir=…)+#[path]impl mods +serve_domains(OsisCrm::create(db,uid)?, …).await. Update the scaffold tests atscaffold.rs:3511+(they assert the old macro).herolib_derive, +herolib_oschema_server, +async-trait; −hero_rpc2, −jsonrpsee. Ensure schemas reachable at server-crate compile time (the macro readsspecat compile time).rpc.sock+ per-domainrpc_<domain>.sock.rust_struct.rs generate_osis_impl_blockis now redundant (OsisObject moved to the server on macro types) — drop it (and therpc.rswrapper bits that referencedOSCHEMA_SOURCE)._find: currently returns empty (TODO) — lower<Type>FindParams→indexer query (offrust_typestring: String→StrFilter, named→EnumFilter, numeric→NumFilter, bool→BoolFilter).examples/crm/run.sh) → build → boot → driver green; then chunk 5 (hero_osis 15 domains).Shipped so far: chunk 1 (
0164e33), 2a (8ba3eff), 2b-core (760ed90) — all generator-lib/test-verified. Next pass is scaffold + orchestration + CRM regen (verified by the end-to-end run, which needslab+ full-workspace build).Chunk 2 — two design refinements for the orchestration pass (record before bring-up)
Planning the scaffold/orchestration surfaced two decisions that revise 2b-core's service model:
serve_domainstakesimpl <Stem>Apiby value → drop theArc<Handler>+init_handlersmachinery. Implement<Stem>Apidirectly forOsis<Domain>(Send+Sync; passed by value into the router).create()returnsSelf(notArc<Self>). Service-method logic becomes inherentasync fnonOsis<Domain>(contributor-editable inrpc.rs), and the generated trait method just extracts the macro Input fields and callsself.<method>(..). This removes the back-reference problem and is simpler than the old<Service>Trait/<Service>Handlerindirection — but it meansrust_rpc.rsmust emit inherent-method stubs onOsis<Domain>instead of the<Service>Trait+<Service>Handler(a rust_rpc rewrite), andrust_osis.rs's 2b-core service bridge + struct (handler fields/init_handlers) gets revised accordingly.Server-crate module naming:
openrpc_server!expandsmod <domain>at the crate root, which clashes with today'ssrc/<domain>/dir module. Fix: emit the OSIS impl + contributorrpc.rsundersrc/osis_<domain>/(crate::osis_<domain>), leavingcrate::<domain>to the macro.main.rs:openrpc_server!(spec="../<name>/schemas", service_toml="service.toml", save_openrpc_dir="generated/openrpc")+mod osis_<domain>;+serve_domains(osis_<domain>::Osis<D>::create(&format!("{root}/<domain>"),0)?, …)in macro (sorted-subdir) order. (Per-domain#[cfg(feature)]gating drops — the macro'sserve_domainshas fixed arity.)These touch
rust_osis.rs(revise service model),rust_rpc.rs(inherent stubs),build/scaffold.rs(main.rs/Cargo/service.toml templates + tests),build/emit/rust_server.rs+ mod-wiring (relocate toosis_<domain>/), then_findlowering and the CRM regen. The pieces are interdependent and only fully validate when the CRM workspace builds+boots (lab+ server/admin/web/sdk), so this is the contiguous bring-up pass.Shipped/verified to date: chunk 1 (
0164e33), 2a (8ba3eff), 2b-core (760ed90) — generator emission migrated, lib green, 151 tests pass.State of chunk 2 — server-side generator migration DONE & committed
Shipped on
feature/osis-multidomain-serve(all generator-lib + test verified):0164e33chunk 1 — String-sid storage (39 db tests).8ba3eff2a — wire types + OsisObject impl →Option<String>.760ed902b-core —rust_osis.rsemitsimpl <Stem>Api+impl OsisObject+Osis<Domain>; deleted handle_rpc /_rpc_*/ OsisAppRpcHandler /_new(−1389 lines).1fad2b72b-scaffold — servermain.rs→openrpc_server!+serve_domains; Cargo deps swapped; tests updated.Generator lib compiles; 151 generator unit tests pass at each step.
Remaining frontier to reach CRM-green (honest scope)
The generator's server-side emission is migrated. To get
examples/crm/run.shgreen still needs:Generator (server path, generator-lib verifiable):
build/emit/rust_server.rs+ mod-wiring: write the generated OSIS impl + contributorrpc.rsundersrc/osis_<domain>/(matching the newmain.rs), notsrc/<domain>/generated/.rust_rpc.rs: revise the contributorrpc.rsto emit inherentasync fnservice methods onOsis<Domain>(returningResult<T, RpcError>) + the lifecycle triggers — replacing the<Service>Trait/<Service>Handlerindirection (serve_domains takes the impl by value; no Arc back-ref). Reviserust_osis.rs's 2b-core service bridge + struct to callself.<method>(..)and returnSelffromcreate()(drop handler fields/init_handlers).rust_osis.rs:_findquery lowering (currently returns empty) — lower<Type>FindParams→indexer query offrust_type(String→StrFilter, named→EnumFilter, numeric→NumFilter, bool→BoolFilter).rust_struct.rs: drop the now-deadgenerate_osis_impl_block(core); ensure schemas reachable at the server crate (spec="../<name>/schemas").Then the full workspace (validated only by the end-to-end run;
labis available on the dev box):5. SDK emitter — the generated
hero_<name>_sdkmust speak the new wire (Option<String>sid,_setupsert, no_new/<Type>Input). It currently uses jsonrpsee#[rpc]; align to the spec (ideallyopenrpc_client!). The CRM driver imports SDK types (CompanyNewInput, etc.) — those change (#155).6. admin/web crates consume the SDK/types — must compile against the new shape.
7. Regenerate CRM → build (server first, then admin/web/sdk/driver) → boot → driver green. Then chunk 3 (delete
crates/osisHeroOsisServer/rpc2/hero_osis_server!) and chunk 5 (hero_osis 15 domains).Items 1–4 are the next contiguous generator pass (verifiable via generator tests + a server-only build). Items 5–7 are the workspace bring-up. This is a multi-pass effort; the server-emission core (the hard semantic translation) is done.
Chunk 2b — generator-side migration COMPLETE; end-to-end blocked upstream
Pushed
a6f50d7(osis_/ layout + inherent service methods). The server-side generator now fully emits the new model; generator lib + 151 tests green.Validated via a locally-rebuilt
lablabis built fromhero_skills/crates/lab(depends onblueprinter→hero_rpc_generator). Added a temp[patch."…hero_blueprint.git"]→ local working tree inhero_skills/Cargo.toml, rebuiltlab, scaffolded CRM fresh. Confirmed the scaffolder (crates/generator/src/build/scaffold.rsgenerate_server_main_rs— the live path; theblueprints/service/.../main.rstemplate file is overwritten by it) now emits:main.rs=herolib_derive::openrpc_server!(spec="../hero_crm/schemas", service_toml, save_openrpc_dir)+#[path="osis_crm/mod.rs"] mod osis_crm;+serve_domains(osis_crm::OsisCrm::create(&format!("{root}/crm"),0)?).await.Cargo.tomldeps swapped (+herolib_derive/+herolib_oschema_server/+async-trait; −hero_rpc2/−jsonrpsee).End-to-end build blocker (UPSTREAM of OSIS — not this generator)
Building the scaffolded workspace (local
[patch]sobuild.rsruns the local generator) fails before OSIS codegen, in dependency resolution/compile:hero_indexer_sdkwon't compile against currentherolib_derive: itsopenrpc_client!("../hero_indexer_server/openrpc.json")hasdb.create(name, schema)(2 params) — violates the new one-input rule. Pulled in viahero_rpc_osis'srpcfeature (dep:hero_indexer_sdk). hero_indexer needs its own one-input migration (separate repo/issue).hero_theme301s on git fetch (ahero_crm_admindep) — blocks workspace resolution; worked around by temporarily trimming the workspace to core+server.So full CRM bring-up is gated on (1)+(2), independent of the OSIS serving work. I did not hand-edit hero_indexer's generated spec (out of scope, fragile).
Remaining (OSIS scope, after the upstream unblock)
_find: still returns empty — lower<Type>FindParams→indexer query (offrust_type).osis_crm/{osis.rs,rpc.rs,mod.rs}once build.rs can run; fix any emission bugs.openrpc_client!) / admin / web to the new wire; driver type updates (#155).crates/osisHeroOsisServer/rpc2/hero_osis_server!) + chunk 5 (hero_osis 15 domains).Shipped (feature/osis-multidomain-serve)
0164e33ch1 ·8ba3eff2a ·760ed902b-core ·1fad2b72b-scaffold ·a6f50d72b-layout. All generator-lib/test-verified. Temp[patch]left inhero_skills/Cargo.tomlfor the next verification loop (drop when done).⚠️
developmentdiverged — rebase needed (2026-06-03)Pulled all repos.
developmentadvanced +29 commits since this branch forked (merge-basebb95389): a friend's "refactor/hero-lib-base" effort. It did the surrounding hero_lib migration — removedhero_rpc2/jsonrpsee, rewrote admin/web → thin hero_lib shells, rewrote tests →openrpc_server!+ herolib_openrpc client, "workspace fully green on hero_lib" (phase 6) — but left the core OSIS server emission on the OLD model: development'srust_osis.rsstill emitshandle_rpc; scaffoldmain.rsstillhero_osis_server!;OsisObjectstill&SmartId;crates/osis/src/rpc/bootstrap.rsstill present.Conclusion: this branch's 6 commits (the OSIS server-emission migration to
openrpc_server!) are complementary, not redundant — they're the piece the hero-lib-base refactor didn't do. But they sit on the stale base and must be rebased onto the updateddevelopment. Expect conflicts inbuild/scaffold.rs(servermain.rs— keep myopenrpc_server!+serve_domainsversion) and Cargo dep templates (both sides removed hero_rpc2/jsonrpsee — reconcile). The friend's admin/web shell + test rewrites should be KEPT (they cover part of my "SDK/admin/web" remaining frontier).Also: my earlier
hero_theme-301 /hero_indexerone-input blockers were against the stale base —developmentis green on hero_lib, so re-test after rebase; they may already be resolved. (Friend migrated hero_indexer→hero_lib; itsdb.createis still 2-param though.)PR #156 is based on the stale base — refresh it after the rebase.
development HEAD was a BROKEN MERGE — rebased onto coherent base, workspace now builds
Picked this back up. The "rebase onto development" plan from the last comment was wrong in its premise: development HEAD
35d79a6("merge refactor/hero-lib-base … keep development additions") does not compile — for 4 independent reasons, all pre-existing (no OSIS commits involved):Cargo.tomllists deletedcrates/openrpc+crates/hero_rpc2as members → workspace manifest won't load;workspace.package.rust-versiondropped → crates can't inherit it;crates/osis+crates/deriveinherit the deletedhero_rpc2dep;scaffold.rscallsui_emit::{admin,web}_*_htmlfns the refactor removed → 11 generator errors.Root cause: the merge resolved "keep development additions" by resurrecting the pre-refactor manifests/source while keeping the refactor's directory deletions/renames — the two halves are inconsistent. The other merge parent, the refactor tip
a87cac4, is internally coherent (osis onherolib_derive;derive/hero_rpc2/openrpcremoved;rust-versionpresent; builds clean). Everything35d79a6adds overa87cac4is either broken-resurrection or redundant (crates/derive'sopenrpc_client!/openrpc_proxy!are also inherolib_derive).Done
Rebased the OSIS chunks onto
a87cac4(dropping the now-pointlessserve_domainsstopgap — itsbootstrap.rswas already deleted by the refactor). Resolved the conflicts keeping approach A (issue decision 38153/38174: one type model, Stringsid, OSIS owns a local String-sidOsisObjecttrait — not herolib's&SmartIdone). Result: the entire hero_blueprint workspace now builds (cargo build --workspacegreen) and all tests pass — generator 148 unit tests, osis 44 db tests + doctests (default +indexfeature). Branchdevelopment(local) =a87cac4+ 5 OSIS commits.Key finding for the remaining work
a87cac4already contains an approach-B OSIS server emitter (generate/server_main.rs+ scaffoldemit_server_sources): spec-pure wire types + separate*Storetypes carryingsid: SmartId,#[derive(OsisObject)]. Our approach-A chunk-1 String-sid fork breaks approach B (*Storederives the canonical SmartIdOsisObject, butDBTyped<T>now requires the local String one). So the live scaffold's server-source emission must switch B→A —generate_server_cratestill callsemit_server_sources. That switch + the end-to-endlabvalidation (scaffold CRM →cargo buildthe generated crate; approach-Aosis.rscompilability vs the macro wire types was never validated — it was blocked upstream before) is the next focused pass. Then_findlowering + chunk 5 (hero_osis 15 domains).Not pushed yet — remote
developmentis the broken merge; superseding it needs a force-push.Server emitter synced to current macro + compile-validated (pushed)
Followed up comment 38409. Re-synced the OSIS server emitter (
generate/server_main.rs) to the current publishedopenrpc_server!contract (pinned viacargo expand) and pushed todevelopment:3c96ae7— drop<type>_new;_setupsert via{data: <Type>}returning the minted sid;_get/_list_fullOption-wrapsid/created_at/updated_at; service methods use the bare OpenRPC method name (ping) +<Pascal>Input/Output(no<service>prefix); real epoch-secondsnow().39cb90e— fixed the fixture's/Volumes/T7machine-path patch so the compile test is portable.server_emit::emitted_main_compilesnow PASSES — the generated server crate compiles against the real macro for the first time. Generator 149 + osis 44 tests green;cargo build --workspacegreen; reference fixture regenerated (multi-file) and a#[ignore]dregen_reference_fixture_mainkeeps it refreshable. The hero_indexer one-input blocker is resolved (hero_rpc_osis --features indexcompiles).Chunk 5 (hero_osis 15+ domains) — scoped, deliberately NOT rushed
hero_osis_server has 18 feature-gated domains, each carrying hand-written business logic in
src/<domain>/server/rpc.rs, on the OLD build.rs-emission path + the deletedhero_osis_server!macro + deletedhero_rpc2.a87cac4built approach B only for the scaffold path; the build.rs path (generate_server_dir) still emits the old dispatcher model. Migrating hero_osis is therefore a careful per-domain effort (regenerate CRUD on approach B + port each domain's hand-written service logic into the new stubs), not a rebuild — re-scaffolding via lab would clobber the hand-written code. Tracked as the next focused effort; not pushed half-migrated.