Hero Indexer

Full-text search service for the Hero OS ecosystem, powered by Tantivy.

Hero Indexer provides multi-database full-text search with dynamic schemas, 9 query types, and batch operations — exposed via JSON-RPC over Unix sockets with an admin web UI.

Architecture

hero_router ──HTTP──> hero_indexer/admin.sock (admin dashboard)
 hero_indexer/rpc.sock (HTTP/1.1 JSON-RPC 2.0)

Crate	Type	Purpose
hero_indexer	library	Core types, Tantivy index manager, RPC handlers
hero_indexer_server	binary	Backend — raw JSON-RPC over Unix socket
hero_indexer_sdk	library	Async client SDK (talks to backend socket)
hero_indexer_admin	binary	Admin dashboard (static assets)
hero_indexer_examples	examples	Example programs using the SDK

Quick Start

Build, install, and start all services:

lab build --release --install --start

Stop, restart, or check status:

lab build --stop
lab build --restart
lab build --status

For a full force-rebuild and redeploy:

lab build --restart

Sockets

Service	Socket	Protocol
Server	~/hero/var/sockets/hero_indexer/rpc.sock	JSON-RPC 2.0 over HTTP/1.1
Admin	~/hero/var/sockets/hero_indexer/admin.sock	HTTP (admin dashboard)

Both services bind exclusively to Unix domain sockets. Use hero_router for external TCP access.

JSON-RPC Methods

The backend exposes 18 methods over its Unix socket:

Method	Description
rpc.discover	OpenRPC specification
rpc.health	Health check ({"status":"ok"})
server.ping	Ping / version info
server.stats	Uptime, database count, total docs
server.exit	Graceful shutdown
db.list	List all databases
db.create	Create database with schema
db.delete	Delete a database
db.close	Close database (free memory)
db.select	Select database for operations
db.info	Info about selected database
schema.get	Get schema of selected database
doc.add	Add a single document
doc.add_batch	Add documents in batch
doc.delete	Delete documents by field/value
index.commit	Commit pending changes
index.reload	Reload index reader
search.query	Execute a search query
search.count	Count matching documents

Admin Endpoints

Endpoint	Method	Description
/health	GET	Health check
/*	GET	Admin dashboard (static assets)

Query Types

Type	Description	Example
all	Match all documents	{"type": "all"}
match	Full-text match (tokenized)	{"type": "match", "field": "body", "value": "search terms"}
term	Exact match (keyword)	{"type": "term", "field": "id", "value": "abc123"}
fuzzy	Typo-tolerant match	{"type": "fuzzy", "field": "title", "value": "serch", "distance": 1}
phrase	Exact phrase match	{"type": "phrase", "field": "body", "value": "exact phrase"}
prefix	Prefix / autocomplete	{"type": "prefix", "field": "title", "value": "hel"}
range	Numeric/date range	{"type": "range", "field": "price", "gte": 10, "lt": 100}
regex	Regex pattern	{"type": "regex", "field": "title", "value": "test.*"}
boolean	Combine with must/should/must_not	{"type": "boolean", "must": [...], "must_not": [...]}

Field Types

Type	Description	Example Value
text	Full-text searchable (tokenized)	"Hello World"
str	Exact match keyword (not tokenized)	"user-123"
u64	Unsigned 64-bit integer	42
i64	Signed 64-bit integer	-10
f64	64-bit floating point	3.14
date	DateTime (RFC 3339)	"2024-01-15T10:30:00Z"
bool	Boolean	true
json	JSON object	{"key": "value"}
bytes	Binary data (base64)	"SGVsbG8="
ip	IP address	"192.168.1.1"

Field options: stored (retrieve in results), indexed (searchable), fast (sorting/aggregations), tokenizer (e.g. "en_stem").

SDK

Add hero_indexer_sdk to your project:

[dependencies]
hero_indexer_sdk = { git = "ssh://git@forge.ourworld.tf/lhumina_code/hero_indexer.git" }
tokio = { version = "1", features = ["full"] }
serde_json = "1"

use hero_indexer_sdk::HeroIndexClient;
use serde_json::json;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
 let mut client = HeroIndexClient::connect_default().await?;

 client.db_create("articles", json!({
 "fields": [
 {"name": "title", "type": "text", "stored": true, "indexed": true},
 {"name": "body", "type": "text", "stored": true, "indexed": true}
 ]
 })).await?;

 client.db_select("articles").await?;
 client.doc_add(json!({"title": "Hello", "body": "Rust is awesome"})).await?;
 client.commit().await?;
 client.reload().await?;

 let results = client.search(
 json!({"type": "match", "field": "body", "value": "rust"}),
 10, 0
 ).await?;
 println!("Found {} results", results.total_hits);

 Ok(())
}

Project Structure

hero_indexer/
├── Cargo.toml # Workspace (5 crates)
├── crates/
│ ├── hero_indexer/ # Core library
│ │ └── src/
│ │ ├── lib.rs
│ │ ├── error.rs
│ │ └── modules/
│ │ ├── handlers.rs # 18 RPC method handlers
│ │ ├── index_manager.rs # Tantivy index lifecycle
│ │ ├── schema.rs # Schema types
│ │ ├── query.rs # Query types
│ │ └── rpc.rs # JSON-RPC types + OpenRPC spec
│ ├── hero_indexer_server/ # Server binary
│ │ ├── openrpc.json # OpenRPC specification
│ │ └── src/main.rs # Unix socket server + demo DB
│ ├── hero_indexer_sdk/ # Client SDK
│ │ └── src/
│ │ ├── lib.rs
│ │ ├── client.rs
│ │ ├── types.rs
│ │ └── error.rs
│ ├── hero_indexer_admin/ # Admin dashboard binary
│ │ └── src/main.rs # Unix socket listener + static assets
│ └── hero_indexer_examples/ # Example programs
│ └── examples/
│ ├── health.rs
│ └── basic_usage.rs
├── docs/
│ ├── specs.md # Interface specification
│ └── OPENRPC.md # Full API documentation
└── .forgejo/workflows/ # CI/CD (Linux + macOS)

Requirements

• Rust 1.95 (pinned Hero toolchain)
• Unix-like OS (Linux, macOS)

License

Apache-2.0