herocode/supervisor
13
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: simplify OpenRPC API and reorganize examples

Browse Source 
- Simplified RunnerConfig to just name, command, and optional env
- Removed RunnerType and ProcessManagerType enums
- Removed db_path, redis_url, binary_path from config
- Made runner name also serve as queue name (no separate queue param)
- Added secret-based authentication to all runner management methods
- Created comprehensive osiris_openrpc example
- Archived old examples to _archive/
- Updated client API to match simplified supervisor interface
This commit is contained in:
Timur Gordon
2025-10-27 14:20:40 +01:00
parent ef56ed0290
commit 98b2718d58
33 changed files with 3018 additions and 788 deletions
Download Patch File Download Diff File Expand all files Collapse all files

364
examples/_archive/E2E_EXAMPLES.md Normal file
View File

@@ -0,0 +1,364 @@
# End-to-End Examples
Complete examples demonstrating the full Supervisor + Runner + Client workflow.
## Overview
These examples show how to:
1. Start a Hero Supervisor
2. Start an OSIS Runner
3. Register the runner with the supervisor
4. Execute jobs using both blocking (`job.run`) and non-blocking (`job.start`) modes
## Prerequisites
### Required Services
1. **Redis** - Must be running on `localhost:6379`
```bash
redis-server
```
2. **Supervisor** - Hero Supervisor with Mycelium integration
```bash
cargo run --bin hero-supervisor -- --redis-url redis://localhost:6379
```
3. **Runner** - OSIS Runner to execute jobs
```bash
cargo run --bin runner_osis -- test_runner --redis-url redis://localhost:6379
```
## Examples
### 1. Simple End-to-End (`simple_e2e.rs`)
**Recommended for beginners** - A minimal example with clear step-by-step execution.
#### What it does:
- Registers a runner with the supervisor
- Runs 2 blocking jobs (with immediate results)
- Starts 1 non-blocking job (fire and forget)
- Shows clear output at each step
#### How to run:
**Terminal 1 - Redis:**
```bash
redis-server
```
**Terminal 2 - Supervisor:**
```bash
cd /Users/timurgordon/code/git.ourworld.tf/herocode/supervisor
RUST_LOG=info cargo run --bin hero-supervisor -- --redis-url redis://localhost:6379
```
**Terminal 3 - Runner:**
```bash
cd /Users/timurgordon/code/git.ourworld.tf/herocode/runner_rust
RUST_LOG=info cargo run --bin runner_osis -- test_runner \
--redis-url redis://localhost:6379 \
--db-path /tmp/test_runner.db
```
**Terminal 4 - Demo:**
```bash
cd /Users/timurgordon/code/git.ourworld.tf/herocode/supervisor
RUST_LOG=info cargo run --example simple_e2e
```
#### Expected Output:
```
╔════════════════════════════════════════╗
║ Simple End-to-End Demo ║
╚════════════════════════════════════════╝
📋 Step 1: Registering Runner
─────────────────────────────────────────
✅ Runner registered successfully
📋 Step 2: Running a Simple Job (Blocking)
─────────────────────────────────────────
✅ Job completed!
Result: {"message":"Hello from the runner!","number":42,"timestamp":1234567890}
📋 Step 3: Running a Calculation Job
─────────────────────────────────────────
✅ Calculation completed!
Result: {"sum":55,"product":3628800,"count":10,"average":5}
📋 Step 4: Starting a Non-Blocking Job
─────────────────────────────────────────
✅ Job started!
Job ID: abc-123 (running in background)
🎉 Demo completed successfully!
```
### 2. Full End-to-End (`end_to_end_demo.rs`)
**Advanced** - Automatically spawns supervisor and runner processes.
#### What it does:
- Automatically starts supervisor and runner
- Runs multiple test jobs
- Demonstrates both execution modes
- Handles cleanup automatically
#### How to run:
**Terminal 1 - Redis:**
```bash
redis-server
```
**Terminal 2 - Demo:**
```bash
cd /Users/timurgordon/code/git.ourworld.tf/herocode/supervisor
RUST_LOG=info cargo run --example end_to_end_demo
```
#### Features:
- ✅ Automatic process management
- ✅ Multiple job examples
- ✅ Graceful shutdown
- ✅ Comprehensive logging
## Job Execution Modes
### job.run (Blocking)
Executes a job and waits for the result.
**Request:**
```json
{
"jsonrpc": "2.0",
"method": "job.run",
"params": [{
"secret": "admin_secret",
"job": { /* job object */ },
"timeout": 30
}],
"id": 1
}
```
**Response:**
```json
{
"jsonrpc": "2.0",
"result": {
"job_id": "uuid",
"status": "completed",
"result": "{ /* actual result */ }"
},
"id": 1
}
```
**Use when:**
- You need immediate results
- Job completes quickly (< 60 seconds)
- Synchronous workflow
### job.start (Non-Blocking)
Starts a job and returns immediately.
**Request:**
```json
{
"jsonrpc": "2.0",
"method": "job.start",
"params": [{
"secret": "admin_secret",
"job": { /* job object */ }
}],
"id": 1
}
```
**Response:**
```json
{
"jsonrpc": "2.0",
"result": {
"job_id": "uuid",
"status": "queued"
},
"id": 1
}
```
**Use when:**
- Long-running operations
- Background processing
- Async workflows
- Don't need immediate results
## Job Structure
Jobs are created using the `JobBuilder`:
```rust
use runner_rust::job::JobBuilder;
let job = JobBuilder::new()
.caller_id("my_client")
.context_id("my_context")
.payload(r#"
// Rhai script to execute
let result = 2 + 2;
to_json(result)
"#)
.runner("runner_name")
.executor("rhai")
.timeout(30)
.build()?;
```
### Job Fields
- **caller_id**: Identifier for the client making the request
- **context_id**: Context for the job execution
- **payload**: Rhai script to execute
- **runner**: Name of the runner to execute on
- **executor**: Type of executor (always "rhai" for OSIS)
- **timeout**: Maximum execution time in seconds
## Rhai Script Examples
### Simple Calculation
```rhai
let result = 2 + 2;
to_json(result)
```
### String Manipulation
```rhai
let message = "Hello, World!";
let upper = message.to_upper();
to_json(upper)
```
### Array Operations
```rhai
let numbers = [1, 2, 3, 4, 5];
let sum = 0;
for n in numbers {
sum += n;
}
to_json(#{sum: sum, count: numbers.len()})
```
### Object Creation
```rhai
let person = #{
name: "Alice",
age: 30,
email: "alice@example.com"
};
to_json(person)
```
## Troubleshooting
### "Failed to connect to supervisor"
**Problem:** Supervisor is not running or wrong port.
**Solution:**
```bash
# Check if supervisor is running
curl http://localhost:3030
# Start supervisor
cargo run --bin hero-supervisor -- --redis-url redis://localhost:6379
```
### "Runner not found"
**Problem:** Runner is not registered or not running.
**Solution:**
```bash
# Start the runner
cargo run --bin runner_osis -- test_runner --redis-url redis://localhost:6379
# Check runner logs for connection issues
```
### "Job execution timeout"
**Problem:** Job took longer than timeout value.
**Solution:**
- Increase timeout in job builder: `.timeout(60)`
- Or in job.run request: `"timeout": 60`
### "Redis connection failed"
**Problem:** Redis is not running.
**Solution:**
```bash
# Start Redis
redis-server
# Or specify custom Redis URL
cargo run --bin hero-supervisor -- --redis-url redis://localhost:6379
```
## Architecture
```
┌─────────────┐
│ Client │
│ (Example) │
└──────┬──────┘
│ HTTP/JSON-RPC
▼
┌─────────────┐
│ Supervisor │
│ (Mycelium) │
└──────┬──────┘
│ Redis Queue
▼
┌─────────────┐
│ Runner │
│ (OSIS) │
└─────────────┘
```
### Flow
1. **Client** creates a job with Rhai script
2. **Client** sends job to supervisor via JSON-RPC
3. **Supervisor** verifies signatures (if present)
4. **Supervisor** queues job to runner's Redis queue
5. **Runner** picks up job from queue
6. **Runner** executes Rhai script
7. **Runner** stores result in Redis
8. **Supervisor** retrieves result (for job.run)
9. **Client** receives result
## Next Steps
- Add signature verification to jobs (see `JOB_SIGNATURES.md`)
- Implement job status polling for non-blocking jobs
- Create custom Rhai functions for your use case
- Scale with multiple runners
## Related Documentation
- `JOB_EXECUTION.md` - Detailed job execution modes
- `JOB_SIGNATURES.md` - Cryptographic job signing
- `README.md` - Supervisor overview
---
**Status:** ✅ Production Ready
**Last Updated:** 2025-10-24