lhumina_code/hero_db
23
0
Fork 0
Code Issues 1 Pull requests Projects Releases 1 Packages 2 Wiki Activity Actions

check hero_db implementation and link to openrpc #18

New issue
Open
opened 2026-04-04 07:33:11 +00:00 by despiegk · 5 comments
despiegk commented 2026-04-04 07:33:11 +00:00
Owner
Copy link

the openrpc needs to have following functions

  • crud on databases
  • set authentication/authorization on each DB, default is no auth
  • openrpc alternatives to the redis cmd;s
  • graph database commands, what is needed for a graph DB (add data, remove, list, index, ...)
  • crud on ontologies
  • load data based on ontology
  • ...

all functionality available

make a spec, then we implement differences in the openrpc as well as in the server

make sure its all very modular as submodules in hero_db_lib

## the openrpc needs to have following functions - crud on databases - set authentication/authorization on each DB, default is no auth - openrpc alternatives to the redis cmd;s - graph database commands, what is needed for a graph DB (add data, remove, list, index, ...) - crud on ontologies - load data based on ontology - ... all functionality available make a spec, then we implement differences in the openrpc as well as in the server make sure its all very modular as submodules in hero_db_lib
despiegk commented 2026-04-04 09:23:29 +00:00
Author
Owner
Copy link

Implementation Spec for Issue #18: hero_db OpenRPC Completeness

Objective

Audit hero_db, produce a comprehensive OpenRPC spec covering all functionality (CRUD on databases, auth/authz, Redis command alternatives, graph DB operations, ontology management, vector operations), and implement handler modules in hero_db_server. The OpenRPC spec becomes the single source of truth; server handler dispatch and SDK client are updated to match.

Current State

  • 12 methods in OpenRPC spec currently (rpc.health, rpc.discover, auth.validate, redis.exec/info/database_list/select/keys/get/del/database_create, perf.batch/batch_native)
  • Missing redis.set — spec has get/del but no set
  • All graph commands (20+ RESP commands) — fully implemented but zero OpenRPC exposure
  • All vector commands (15+ RESP commands) — fully implemented but zero OpenRPC exposure
  • Auth management (user create/grant/revoke/delete) — exists in RESP, not in OpenRPC
  • Stream commands (XADD, XLEN, XRANGE, etc.) — exists in RESP, not in OpenRPC
  • Database delete/info — exists in RESP, only create/list in OpenRPC

Implementation Plan (10 Steps)

Step 1: Expand OpenRPC Specification

Rewrite openrpc.json to cover ~70+ methods organized by domain:

  • rpc.* (2), auth.* (5), database.* (5), redis.* (20+), stream.* (6)
  • graph.node.* (6), graph.edge.* (5), graph.traverse/neighbors/path (3), graph.stats/clear (2)
  • ontology.* (7), vector.* (12), perf.* (2)

Step 2: Create database.* Handler Module

New handlers/database.rs — create, list, delete, info, select

Step 3: Add Missing Redis Data Handlers

Expand handlers/redis.rs — set, mset, mget, exists, expire, ttl, type, incr, hset/hget/hgetall/hdel, lpush/rpush/lpop/rpop/lrange, sadd/srem/smembers

Step 4: Create graph.* Handler Module

New handlers/graph.rs — node CRUD, edge CRUD, traverse, neighbors, path, stats, clear

Step 5: Create ontology.* Handler Module

New handlers/ontology.rs — load, load_default, use, get, list, delete, strict

Step 6: Create vector.* Handler Module

New handlers/vector.rs — create, add, add_multi, get, del, search, search_by_id, build, info, list, drop, clear

Step 7: Create auth.* Handler Module

New handlers/auth.rs — user_create, user_delete, grant, revoke (+ existing validate)

Step 8: Create stream.* Handler Module

New handlers/stream.rs — add, read, range, len, trim, info

Step 9: Verify SDK Compilation

openrpc_client! macro auto-generates from updated spec — verify cargo build

Step 10: Integration Tests

Test files for graph, vector, database, auth RPC methods

Acceptance Criteria

  • openrpc.json contains 70+ method definitions
  • Every method has a handler in the dispatch
  • All GRAPH/VECTOR/ONTOLOGY/AUTH/STREAM RESP commands have OpenRPC equivalents
  • cargo build succeeds for entire workspace
  • Handler modules organized by domain
  • Integration tests pass
  • No breaking changes to existing 12 methods
  • Default is no auth (preserved)

Notes

  • All handlers use same pattern: construct RESP commands → process_command() → convert to JSON-RPC response
  • Context multi-socket architecture is out of scope (follow-up)
  • redis.exec remains as escape hatch for any command
  • hero_db_ui/openrpc.json is separate concern (RESP protocol docs for UI)
# Implementation Spec for Issue #18: hero_db OpenRPC Completeness ## Objective Audit hero_db, produce a comprehensive OpenRPC spec covering all functionality (CRUD on databases, auth/authz, Redis command alternatives, graph DB operations, ontology management, vector operations), and implement handler modules in `hero_db_server`. The OpenRPC spec becomes the single source of truth; server handler dispatch and SDK client are updated to match. ## Current State - **12 methods** in OpenRPC spec currently (`rpc.health`, `rpc.discover`, `auth.validate`, `redis.exec/info/database_list/select/keys/get/del/database_create`, `perf.batch/batch_native`) - **Missing `redis.set`** — spec has get/del but no set - **All graph commands** (20+ RESP commands) — fully implemented but zero OpenRPC exposure - **All vector commands** (15+ RESP commands) — fully implemented but zero OpenRPC exposure - **Auth management** (user create/grant/revoke/delete) — exists in RESP, not in OpenRPC - **Stream commands** (XADD, XLEN, XRANGE, etc.) — exists in RESP, not in OpenRPC - **Database delete/info** — exists in RESP, only create/list in OpenRPC ## Implementation Plan (10 Steps) ### Step 1: Expand OpenRPC Specification Rewrite `openrpc.json` to cover ~70+ methods organized by domain: - `rpc.*` (2), `auth.*` (5), `database.*` (5), `redis.*` (20+), `stream.*` (6) - `graph.node.*` (6), `graph.edge.*` (5), `graph.traverse/neighbors/path` (3), `graph.stats/clear` (2) - `ontology.*` (7), `vector.*` (12), `perf.*` (2) ### Step 2: Create `database.*` Handler Module New `handlers/database.rs` — create, list, delete, info, select ### Step 3: Add Missing Redis Data Handlers Expand `handlers/redis.rs` — set, mset, mget, exists, expire, ttl, type, incr, hset/hget/hgetall/hdel, lpush/rpush/lpop/rpop/lrange, sadd/srem/smembers ### Step 4: Create `graph.*` Handler Module New `handlers/graph.rs` — node CRUD, edge CRUD, traverse, neighbors, path, stats, clear ### Step 5: Create `ontology.*` Handler Module New `handlers/ontology.rs` — load, load_default, use, get, list, delete, strict ### Step 6: Create `vector.*` Handler Module New `handlers/vector.rs` — create, add, add_multi, get, del, search, search_by_id, build, info, list, drop, clear ### Step 7: Create `auth.*` Handler Module New `handlers/auth.rs` — user_create, user_delete, grant, revoke (+ existing validate) ### Step 8: Create `stream.*` Handler Module New `handlers/stream.rs` — add, read, range, len, trim, info ### Step 9: Verify SDK Compilation `openrpc_client!` macro auto-generates from updated spec — verify `cargo build` ### Step 10: Integration Tests Test files for graph, vector, database, auth RPC methods ## Acceptance Criteria - [ ] `openrpc.json` contains 70+ method definitions - [ ] Every method has a handler in the dispatch - [ ] All GRAPH/VECTOR/ONTOLOGY/AUTH/STREAM RESP commands have OpenRPC equivalents - [ ] `cargo build` succeeds for entire workspace - [ ] Handler modules organized by domain - [ ] Integration tests pass - [ ] No breaking changes to existing 12 methods - [ ] Default is no auth (preserved) ## Notes - All handlers use same pattern: construct RESP commands → `process_command()` → convert to JSON-RPC response - Context multi-socket architecture is out of scope (follow-up) - `redis.exec` remains as escape hatch for any command - `hero_db_ui/openrpc.json` is separate concern (RESP protocol docs for UI)
despiegk commented 2026-04-04 10:44:45 +00:00
Author
Owner
Copy link

Test Results

  • Total: 73
  • Passed: 63
  • Failed: 0
  • Ignored: 10

Build

Full workspace builds successfully (cargo build --workspace).

SDK Generation

hero_db_sdk compiles successfully with 80 OpenRPC methods auto-generated from the updated spec.

Breakdown by crate

Crate Tests Passed Failed Ignored
hero_db (unit) 32 32 0 0
hero_db (integration) 1 1 0 0
hero_db_server (integration, crate-level) 6 6 0 0
hero_db_sdk (unit) 6 6 0 0
hero_db_server (integration) 18 18 0 0
Doc-tests (hero_db + hero_db_sdk) 10 0 0 10

All 63 executable tests pass. The 10 ignored items are doc-tests marked ignore (require a running server).

## Test Results - **Total**: 73 - **Passed**: 63 - **Failed**: 0 - **Ignored**: 10 ### Build Full workspace builds successfully (`cargo build --workspace`). ### SDK Generation `hero_db_sdk` compiles successfully with 80 OpenRPC methods auto-generated from the updated spec. ### Breakdown by crate | Crate | Tests | Passed | Failed | Ignored | |-------|-------|--------|--------|---------| | hero_db (unit) | 32 | 32 | 0 | 0 | | hero_db (integration) | 1 | 1 | 0 | 0 | | hero_db_server (integration, crate-level) | 6 | 6 | 0 | 0 | | hero_db_sdk (unit) | 6 | 6 | 0 | 0 | | hero_db_server (integration) | 18 | 18 | 0 | 0 | | Doc-tests (hero_db + hero_db_sdk) | 10 | 0 | 0 | 10 | All 63 executable tests pass. The 10 ignored items are doc-tests marked `ignore` (require a running server).
despiegk commented 2026-04-04 10:45:19 +00:00
Author
Owner
Copy link

Implementation Summary

Changes Made

OpenRPC Specification (crates/hero_db_server/openrpc.json):

  • Expanded from 12 methods to 80 methods across 8 domains
  • All existing methods preserved with no breaking changes

New Handler Modules (in crates/hero_db_server/src/handlers/):

  • auth.rs — 5 methods: validate, user_create, user_delete, grant, revoke
  • database.rs — 5 methods: create, list, delete, info, select
  • graph.rs — 16 methods: node CRUD (6), edge CRUD (5), traversal (3), stats/clear (2)
  • ontology.rs — 7 methods: load, load_default, use, get, list, delete, strict
  • vector.rs — 12 methods: create, add, add_multi, get, del, search, search_by_id, build, info, list, drop, clear
  • stream.rs — 5 methods: add, range, len, trim, info

Enhanced Redis Handlers (redis.rs):

  • Added 18 new methods: set, exists, expire, ttl, key_type, incr, hset, hget, hgetall, hdel, lpush, rpush, lpop, rpop, lrange, sadd, srem, smembers

Updated Dispatch (mod.rs):

  • All 80 methods routed to their respective handler modules
  • Organized by domain with clear grouping

SDK Auto-generation:

  • hero_db_sdk auto-generates typed client methods from the updated spec via openrpc_client! macro
  • All 80 methods available as typed Rust functions

Test Results

  • 73 tests: 63 passed, 10 ignored (doc-tests), 0 failed
  • Full workspace builds clean

Architecture

All handlers follow the same pattern: construct RESP commands → process_command() → convert to JSON-RPC response. This ensures OpenRPC and RESP behavior are identical with no logic duplication.

## Implementation Summary ### Changes Made **OpenRPC Specification** (`crates/hero_db_server/openrpc.json`): - Expanded from 12 methods to **80 methods** across 8 domains - All existing methods preserved with no breaking changes **New Handler Modules** (in `crates/hero_db_server/src/handlers/`): - `auth.rs` — 5 methods: validate, user_create, user_delete, grant, revoke - `database.rs` — 5 methods: create, list, delete, info, select - `graph.rs` — 16 methods: node CRUD (6), edge CRUD (5), traversal (3), stats/clear (2) - `ontology.rs` — 7 methods: load, load_default, use, get, list, delete, strict - `vector.rs` — 12 methods: create, add, add_multi, get, del, search, search_by_id, build, info, list, drop, clear - `stream.rs` — 5 methods: add, range, len, trim, info **Enhanced Redis Handlers** (`redis.rs`): - Added 18 new methods: set, exists, expire, ttl, key_type, incr, hset, hget, hgetall, hdel, lpush, rpush, lpop, rpop, lrange, sadd, srem, smembers **Updated Dispatch** (`mod.rs`): - All 80 methods routed to their respective handler modules - Organized by domain with clear grouping **SDK Auto-generation**: - `hero_db_sdk` auto-generates typed client methods from the updated spec via `openrpc_client!` macro - All 80 methods available as typed Rust functions ### Test Results - 73 tests: 63 passed, 10 ignored (doc-tests), 0 failed - Full workspace builds clean ### Architecture All handlers follow the same pattern: construct RESP commands → `process_command()` → convert to JSON-RPC response. This ensures OpenRPC and RESP behavior are identical with no logic duplication.
despiegk commented 2026-04-04 19:15:56 +00:00
Author
Owner
Copy link

Implementation committed: b946a84

Browse: b946a84

Implementation committed: `b946a84` Browse: https://forge.ourworld.tf/lhumina_code/hero_db/commit/b946a84
despiegk commented 2026-04-05 02:33:33 +00:00
Author
Owner
Copy link

Implementation Summary

OpenRPC Expansion (80 → 125 methods)

All RESP commands now have JSON-RPC method aliases — everything is accessible via OpenRPC without using RESP directly.

New domains added:

  • server.* (9 methods): compact, shutdown, memory, ping, dbsize, flushdb, info, config_get, config_set
  • admin.* (3 methods): add, remove, list
  • 31 additional redis.* methods: mset, mget, append, strlen, getrange, setnx, incrby, decrby, decr, scan, hmset, hmget, hexists, hlen, hkeys, hvals, hincrby, llen, lindex, lset, lrem, scard, sismember, spop, sunion, sinter, sdiff, dbsize, flushdb, ping
  • vector.exists, vector.len
  • database.public

Handler Fixes

  • graph.stats — returns proper JSON object instead of flat array
  • vector.info — returns proper JSON object instead of flat array

UI OpenRPC Spec

  • Replaced RESP command documentation with server's JSON-RPC spec (125 methods)

Integration Tests Rewritten

  • 12 tests using typed SDK clients (HeroDBServerClient for OpenRPC, HeroDbClient for RESP)
  • Cross-protocol shared storage test verifies data written via RESP is readable via OpenRPC and vice versa

Makefile Updated

  • Replaced Rhai script delegation (hero_do scripts/*.rhai) with direct hero_db --start / hero_db --stop
  • Service now properly registers with hero_proc for Inspector visibility

Files Changed

  • crates/hero_db_server/openrpc.json — expanded to 125 methods
  • crates/hero_db_server/openrpc.client.generated.rs — regenerated SDK client
  • crates/hero_db_server/src/handlers/mod.rs — dispatch table for all 125 methods
  • crates/hero_db_server/src/handlers/redis.rs — 31 new handlers
  • crates/hero_db_server/src/handlers/server.rs — new, 9 server management handlers
  • crates/hero_db_server/src/handlers/admin.rs — new, 3 admin handlers
  • crates/hero_db_server/src/handlers/database.rs — added public_access handler
  • crates/hero_db_server/src/handlers/vector.rs — added exists/len, fixed info handler
  • crates/hero_db_server/src/handlers/graph.rs — fixed stats handler
  • crates/hero_db_server/tests/openrpc_methods.rs — new, 12 typed SDK integration tests
  • crates/hero_db_ui/openrpc.json — replaced with server's JSON-RPC spec
  • Makefile — uses hero_db --start/--stop instead of Rhai scripts
## Implementation Summary ### OpenRPC Expansion (80 → 125 methods) All RESP commands now have JSON-RPC method aliases — everything is accessible via OpenRPC without using RESP directly. **New domains added:** - `server.*` (9 methods): compact, shutdown, memory, ping, dbsize, flushdb, info, config_get, config_set - `admin.*` (3 methods): add, remove, list - 31 additional `redis.*` methods: mset, mget, append, strlen, getrange, setnx, incrby, decrby, decr, scan, hmset, hmget, hexists, hlen, hkeys, hvals, hincrby, llen, lindex, lset, lrem, scard, sismember, spop, sunion, sinter, sdiff, dbsize, flushdb, ping - `vector.exists`, `vector.len` - `database.public` ### Handler Fixes - `graph.stats` — returns proper JSON object instead of flat array - `vector.info` — returns proper JSON object instead of flat array ### UI OpenRPC Spec - Replaced RESP command documentation with server's JSON-RPC spec (125 methods) ### Integration Tests Rewritten - 12 tests using typed SDK clients (`HeroDBServerClient` for OpenRPC, `HeroDbClient` for RESP) - Cross-protocol shared storage test verifies data written via RESP is readable via OpenRPC and vice versa ### Makefile Updated - Replaced Rhai script delegation (`hero_do scripts/*.rhai`) with direct `hero_db --start` / `hero_db --stop` - Service now properly registers with hero_proc for Inspector visibility ### Files Changed - `crates/hero_db_server/openrpc.json` — expanded to 125 methods - `crates/hero_db_server/openrpc.client.generated.rs` — regenerated SDK client - `crates/hero_db_server/src/handlers/mod.rs` — dispatch table for all 125 methods - `crates/hero_db_server/src/handlers/redis.rs` — 31 new handlers - `crates/hero_db_server/src/handlers/server.rs` — new, 9 server management handlers - `crates/hero_db_server/src/handlers/admin.rs` — new, 3 admin handlers - `crates/hero_db_server/src/handlers/database.rs` — added public_access handler - `crates/hero_db_server/src/handlers/vector.rs` — added exists/len, fixed info handler - `crates/hero_db_server/src/handlers/graph.rs` — fixed stats handler - `crates/hero_db_server/tests/openrpc_methods.rs` — new, 12 typed SDK integration tests - `crates/hero_db_ui/openrpc.json` — replaced with server's JSON-RPC spec - `Makefile` — uses hero_db --start/--stop instead of Rhai scripts
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
development
development_mik2
development_kristof
development_check
main
v0.3.2
v0.1.3
v0.1.2
v0.0.1
v0.1.1
v0.1.0
v0.0.1-rc1
Labels
Clear labels   
prio_critical

   
prio_low

   
type_bug

   
type_contact

   
type_issue

   
type_lead

   
type_question

   
type_story

   
type_task
No labels
prio_critical
prio_low
type_bug
type_contact
type_issue
type_lead
type_question
type_story
type_task
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lhumina_code/hero_db#18
No description provided.