diff --git a/README.md b/README.md index 78f1436..5cb4fb9 100644 --- a/README.md +++ b/README.md @@ -2,19 +2,19 @@ A Rust-based actor management system for the Hero ecosystem that provides unified process management, job queuing, and optional OpenRPC server integration. -## Architecture Overview - -The Hero Supervisor uses a clean, feature-gated architecture that separates library functionality from CLI/server features to avoid dependency cycles and maintain modularity. +## Repository Structure ``` -hero-supervisor/ -├── src/ # Core library (no CLI dependencies) -│ ├── lib.rs # Main library exports and documentation -│ ├── supervisor.rs # Core supervisor logic and actor management -│ ├── runner.rs # Runner implementation for actor process management -│ ├── job.rs # Job data structures, builder pattern, and Redis key management -│ └── openrpc.rs # OpenRPC server (feature-gated) -├── cmd/ # CLI binaries +supervisor/ +├── core/ # Main supervisor library and binary +│ ├── src/ +│ │ ├── bin/supervisor.rs # Supervisor binary +│ │ └── lib.rs # Library exports +│ ├── examples/ # Usage examples +│ └── tests/ # Integration tests +├── client/ # OpenRPC client library (Rust + WASM) +├── ui/ # Admin UI (Yew WASM application) +└── docs/ # Documentation ## Features @@ -43,35 +43,6 @@ The Hero Supervisor uses a clean, simplified architecture with centralized resou - **Direct OpenRPC Integration**: RPC trait implemented directly on `Arc>` (no wrapper layers) - **Simplified App**: `SupervisorApp::start()` handles everything - runners, OpenRPC server, graceful shutdown -## File Documentation - -### Core Library Files - -#### `src/lib.rs` -Main library entry point that exports `Supervisor`, `SupervisorBuilder`, `SupervisorApp`, and related types. - -#### `src/supervisor.rs` -Core supervisor implementation with builder pattern. Manages runners, owns shared Redis client and process manager. Provides job queuing, runner lifecycle management, and status monitoring. - -#### `src/app.rs` -Main application wrapper that provides `start()` method for complete lifecycle management. Handles OpenRPC server startup, graceful shutdown, and keeps the application running. - -#### `src/runner.rs` -Simplified runner configuration and management. Contains `Runner` struct with configuration data only - no resource ownership. Integrates with supervisor's shared resources. - -#### `src/job.rs` -Job data structures, builder pattern, and Redis key management. Defines `Job` struct with metadata, script content, and status tracking. - -#### `src/client.rs` -Client implementation for job management operations. Provides Redis-based job storage, retrieval, status updates, and lifecycle management. Separates job operations from supervisor logic. - -#### `src/openrpc.rs` -OpenRPC server implementation that exposes all supervisor functionality over JSON-RPC. Implements RPC trait directly on the supervisor for clean integration. - -### Binary Files - -#### `cmd/supervisor.rs` -Main supervisor binary that creates a supervisor using the builder pattern and starts the complete application with `app.start()`. The OpenRPC server is always enabled and starts automatically. ## Usage @@ -195,6 +166,15 @@ RUST_LOG=info cargo run --features openrpc 4. **Conditional Compilation**: Rust's feature system ensures minimal dependencies for library users 5. **Single Binary**: One supervisor binary with optional OpenRPC server integration +## Documentation + +- **[Quick Start Guide](docs/QUICK_START.md)** - Get started with Hero Supervisor +- **[Authentication](docs/AUTH.md)** - Secret-based authentication system +- **[Job API Convention](docs/job-api-convention.md)** - Job submission and management API +- **[Mycelium Integration](docs/MYCELIUM.md)** - Optional Mycelium network support +- **[Restructure Notes](docs/RESTRUCTURE.md)** - Repository restructuring details +- **[Test Keypairs](docs/test_keypairs.md)** - Testing with authentication + ## License -[Add your license information here] +MIT OR Apache-2.0 diff --git a/MYCELIUM_OPTIONAL.md b/docs/MYCELIUM.md similarity index 100% rename from MYCELIUM_OPTIONAL.md rename to docs/MYCELIUM.md diff --git a/QUICK_START.md b/docs/QUICK_START.md similarity index 100% rename from QUICK_START.md rename to docs/QUICK_START.md diff --git a/docs/README.md b/docs/README.md deleted file mode 100644 index 00e162a..0000000 --- a/docs/README.md +++ /dev/null @@ -1,280 +0,0 @@ -# Hero Supervisor Documentation - -## Overview - -Hero Supervisor is a distributed job execution system that manages runners and coordinates job processing across multiple worker nodes. It provides a robust OpenRPC API for job management and runner administration. - -## Architecture - -The supervisor consists of several key components: - -- **Supervisor Core**: Central coordinator that manages runners and job dispatch -- **OpenRPC Server**: JSON-RPC API server for remote management -- **Redis Backend**: Job queue and state management -- **Process Manager**: Runner lifecycle management (Simple or Tmux) -- **Client Libraries**: Native Rust and WASM clients for integration - -## Quick Start - -### Starting the Supervisor - -```bash -# With default configuration -./supervisor - -# With custom configuration file -./supervisor --config /path/to/config.toml -``` - -### Example Configuration - -```toml -# config.toml -redis_url = "redis://localhost:6379" -namespace = "hero" -bind_address = "127.0.0.1" -port = 3030 - -# Admin secrets for full access -admin_secrets = ["admin-secret-123"] - -# User secrets for job operations -user_secrets = ["user-secret-456"] - -# Register secrets for runner registration -register_secrets = ["register-secret-789"] - -[[actors]] -id = "sal_runner_1" -name = "sal_runner_1" -binary_path = "/path/to/sal_runner" -db_path = "/tmp/sal_db" -redis_url = "redis://localhost:6379" -process_manager = "simple" - -[[actors]] -id = "osis_runner_1" -name = "osis_runner_1" -binary_path = "/path/to/osis_runner" -db_path = "/tmp/osis_db" -redis_url = "redis://localhost:6379" -process_manager = "tmux:osis_session" -``` - -## API Documentation - -### Job API Convention - -The Hero Supervisor follows a consistent naming convention for job operations: - -- **`jobs.`** - General job operations (create, list) -- **`job.`** - Specific job operations (run, start, status, result) - -See [Job API Convention](job-api-convention.md) for detailed documentation. - -### Core Methods - -#### Runner Management -- `register_runner` - Register a new runner -- `list_runners` - List all registered runners -- `start_runner` / `stop_runner` - Control runner lifecycle -- `get_runner_status` - Get runner status -- `get_runner_logs` - Retrieve runner logs - -#### Job Management -- `jobs.create` - Create a job without queuing -- `jobs.list` - List all jobs with full details -- `job.run` - Run a job and return result -- `job.start` - Start a created job -- `job.stop` - Stop a running job -- `job.delete` - Delete a job from the system -- `job.status` - Get job status (non-blocking) -- `job.result` - Get job result (blocking) - -#### Administration -- `add_secret` / `remove_secret` - Manage authentication secrets -- `get_supervisor_info` - Get system information -- `rpc.discover` - OpenRPC specification discovery - -## Client Usage - -### Rust Client - -```rust -use hero_supervisor_openrpc_client::{SupervisorClient, JobBuilder}; - -// Create client -let client = SupervisorClient::new("http://localhost:3030")?; - -// Create a job -let job = JobBuilder::new() - .caller_id("my_client") - .context_id("my_context") - .payload("print('Hello World')") - .executor("osis") - .runner("osis_runner_1") - .timeout(60) - .build()?; - -// Option 1: Fire-and-forget execution -let result = client.job_run("user-secret", job.clone()).await?; -match result { - JobResult::Success { success } => println!("Output: {}", success), - JobResult::Error { error } => println!("Error: {}", error), -} - -// Option 2: Asynchronous execution -let job_id = client.jobs_create("user-secret", job).await?; -client.job_start("user-secret", &job_id).await?; - -// Poll for completion -loop { - let status = client.job_status(&job_id).await?; - if status.status == "completed" || status.status == "failed" { - break; - } - tokio::time::sleep(Duration::from_secs(1)).await; -} - -let result = client.job_result(&job_id).await?; - -// Option 3: Job management -// Stop a running job -client.job_stop("user-secret", &job_id).await?; - -// Delete a job -client.job_delete("user-secret", &job_id).await?; - -// List all jobs (returns full Job objects) -let jobs = client.jobs_list("user-secret").await?; -for job in jobs { - println!("Job {}: {} ({})", job.id, job.executor, job.payload); -} -``` - -### WASM Client - -```javascript -import { WasmSupervisorClient, WasmJob } from 'hero-supervisor-openrpc-client'; - -// Create client -const client = new WasmSupervisorClient('http://localhost:3030'); - -// Create and run job -const job = new WasmJob('job-id', 'print("Hello")', 'osis', 'osis_runner_1'); -const result = await client.create_job('user-secret', job); -``` - -## Security - -### Authentication Levels - -1. **Admin Secrets**: Full system access - - All runner management operations - - All job operations - - Secret management - - System information access - -2. **User Secrets**: Job operations only - - Create, run, start jobs - - Get job status and results - - No runner or secret management - -3. **Register Secrets**: Runner registration only - - Register new runners - - No other operations - -### Best Practices - -- Use different secret types for different access levels -- Rotate secrets regularly -- Store secrets securely (environment variables, secret management systems) -- Use HTTPS in production environments -- Implement proper logging and monitoring - -## Development - -### Building - -```bash -# Build supervisor binary -cargo build --release - -# Build with OpenRPC feature -cargo build --release --features openrpc - -# Build client library -cd client -cargo build --release -``` - -### Testing - -```bash -# Run tests -cargo test - -# Run with Redis (requires Redis server) -docker run -d -p 6379:6379 redis:alpine -cargo test -- --ignored -``` - -### Examples - -See the `examples/` directory for: -- Basic supervisor setup -- Mock runner implementation -- Comprehensive OpenRPC client usage -- Configuration examples - -## Troubleshooting - -### Common Issues - -1. **Redis Connection Failed** - - Ensure Redis server is running - - Check Redis URL in configuration - - Verify network connectivity - -2. **Runner Registration Failed** - - Check register secret validity - - Verify runner binary path exists - - Ensure runner has proper permissions - -3. **Job Execution Timeout** - - Increase job timeout value - - Check runner resource availability - - Monitor runner logs for issues - -4. **OpenRPC Method Not Found** - - Verify method name spelling - - Check OpenRPC specification - - Ensure server supports the method - -### Logging - -Enable debug logging: -```bash -RUST_LOG=debug ./supervisor --config config.toml -``` - -### Monitoring - -Monitor key metrics: -- Runner status and health -- Job queue lengths -- Job success/failure rates -- Response times -- Redis connection status - -## Contributing - -1. Fork the repository -2. Create a feature branch -3. Make changes with tests -4. Update documentation -5. Submit a pull request - -## License - -[License information here] diff --git a/RESTRUCTURE.md b/docs/RESTRUCTURE.md similarity index 100% rename from RESTRUCTURE.md rename to docs/RESTRUCTURE.md diff --git a/test_keypairs.md b/docs/test_keypairs.md similarity index 100% rename from test_keypairs.md rename to docs/test_keypairs.md diff --git a/ui/BREADCRUMB_NAVIGATION.md b/ui/BREADCRUMB_NAVIGATION.md deleted file mode 100644 index 12d41f0..0000000 --- a/ui/BREADCRUMB_NAVIGATION.md +++ /dev/null @@ -1,197 +0,0 @@ -# Breadcrumb Navigation with Dynamic Sidebar and Content Islands - -## Overview -Implemented a breadcrumb-style navigation system where clicking on a job transitions the UI to show: -- **Sidebar**: Job details with metadata and actions -- **Main Content**: Two side-by-side islands showing Payload (left) and Logs (right) - -## Architecture - -### State Management -The UI uses `ContentView` enum to track the current view: -```rust -enum ContentView { - JobsList, // Default view showing all jobs - JobDetail(String), // Job detail view with job_id -} -``` - -### Layout Structure - -#### Jobs List View -``` -┌─────────────────┬──────────────────────────────┐ -│ Sidebar │ Main Content │ -│ │ │ -│ [Runners/Keys] │ Jobs Table │ -│ Toggle │ - Filter │ -│ │ - Sort │ -│ Runner/Key │ - Actions │ -│ Cards │ │ -│ │ │ -└─────────────────┴──────────────────────────────┘ -``` - -#### Job Detail View -``` -┌─────────────────┬──────────────────────────────┐ -│ Sidebar │ Main Content │ -│ │ │ -│ [← Back] │ ┌────────┬────────┐ │ -│ │ │Payload │ Logs │ │ -│ Job Details │ │ │ │ │ -│ - Status │ │ (code) │ (code) │ │ -│ - Job ID │ │ │ │ │ -│ - Runner │ └────────┴────────┘ │ -│ - Timeout │ │ -│ - Created │ ┌──────────────────┐ │ -│ │ │ Output & Status │ │ -│ [▶ Run Job] │ │ [Status Badge] │ │ -│ [🗑 Delete] │ │ [View Output] │ │ -│ │ │ │ │ -│ │ │ ⏳ Running... │ │ -│ │ │ or ✅ Complete │ │ -│ │ │ or ❌ Failed │ │ -│ │ └──────────────────┘ │ -└─────────────────┴──────────────────────────────┘ -``` - -## Implementation Details - -### 1. Dynamic Sidebar (`view_dashboard`) -The sidebar content changes based on `ContentView`: -- **JobsList**: Shows Runners/Keys toggle with respective cards -- **JobDetail**: Shows job metadata and action buttons - -### 2. Job Detail Sidebar (`view_job_detail_sidebar`) -Displays: -- Back button (← arrow) to return to jobs list -- Job metadata in labeled rows: - - Status (with colored badge) - - Job ID (monospace) - - Runner name - - Timeout duration - - Created timestamp -- Action buttons: - - Run Job (primary button) - - Delete Job (danger button) - -### 3. Job Detail Content (`view_job_detail_content`) -Three-pane layout with top row and bottom row: - -**Top Row** (Two equal-width islands side-by-side): -- **Payload Island** (Left): - - Header: "Payload" - - Content: Code block with job payload -- **Logs Island** (Right): - - Header: "Logs" - - Content: Code block with job logs or loading spinner - -**Bottom Row** (Full-width island): -- **Output & Status Island**: - - Header: "Output & Status" with status badge and "View Output" button - - Content: Dynamic status display based on job state: - - **Running** (⏳): Shows "Job is running..." message - - **Completed** (✅): Shows "Job completed successfully" with prompt to view output - - **Failed** (❌): Shows "Job failed" error message - - **Idle** (⏸): Shows "Job not started" with instruction to run - -### 4. Navigation Flow -``` -Jobs List → Click Job Row → Job Detail View - ↓ - Click Back Button - ↓ - Jobs List -``` - -## CSS Styling - -### Key Classes -- `.job-detail-content`: Flex column layout for three-pane structure -- `.detail-top-row`: Grid layout for two islands (1fr 1fr) -- `.detail-bottom-row`: Full-width container for output/status -- `.detail-island`: Container for payload/logs/output with header and content -- `.island-header`: Title bar for each island -- `.island-content`: Scrollable content area -- `.code-block`: Monospace code display -- `.job-detail-sidebar`: Vertical layout for job metadata -- `.detail-row`: Label-value pairs for job info -- `.status-display`: Status indicator with icon and text -- `.btn-back`: Back arrow button - -### Responsive Design -- Desktop (>1200px): Top row shows two islands side-by-side, bottom row full-width -- Mobile (<1200px): All islands stack vertically - -## Features - -### Breadcrumb Navigation -- Clear visual hierarchy with back button -- Sidebar transforms to show contextual information -- Main content splits into focused islands - -### Job Details Display -- Clean, organized metadata presentation -- Status badges with color coding -- Monospace formatting for technical data (IDs, code) - -### Code Display -- Syntax-friendly monospace font -- Proper line wrapping -- Scrollable for long content -- Dark theme optimized - -### Status & Output Pane -- **Visual Status Indicators**: Large emoji icons (⏳, ✅, ❌, ⏸) with colored borders -- **Real-time Status**: Shows current job state (running, completed, failed, idle) -- **Quick Actions**: View Output button in header (disabled for queued jobs) -- **Contextual Messages**: Clear instructions based on job state -- **Color-coded Borders**: - - Orange for running jobs - - Green for completed jobs - - Red for failed jobs - - Gray for idle jobs - -### Actions -- Contextual actions in sidebar (Run, Delete) -- View Output button in status pane header -- Always visible without scrolling -- Clear visual hierarchy - -## Usage - -### Viewing Job Details -1. Click any row in the jobs table -2. Sidebar updates to show job metadata -3. Main content splits into Payload and Logs islands -4. Click back arrow to return to jobs list - -### Running/Deleting Jobs -- Actions are available in the sidebar -- Run Job: Queues the job for execution -- Delete Job: Removes the job from the system - -## Benefits - -1. **Better Information Architecture**: Job details in sidebar, code in main area, status below -2. **Side-by-Side Comparison**: View payload and logs simultaneously in top row -3. **Prominent Status Display**: Full-width status pane with visual indicators -4. **Consistent Navigation**: Back button provides clear exit path -5. **Focused View**: Each island has a single purpose -6. **Visual Feedback**: Color-coded status indicators with emoji icons -7. **Quick Actions**: View Output button readily accessible in status pane -8. **Responsive**: Adapts to different screen sizes -9. **Reusable Pattern**: Can be extended to other detail views - -## Future Enhancements - -Potential improvements: -- ✅ **Job output/status pane** (implemented as third pane) -- Show job execution history timeline -- Add real-time log streaming with auto-refresh -- Include job dependencies visualization -- Add breadcrumb trail in header (Jobs > Job-123) -- Add inline output display (instead of modal) -- Show job metrics (execution time, resource usage) -- Add job re-run with modified parameters diff --git a/ui/ENHANCED_JOB_EDITING.md b/ui/ENHANCED_JOB_EDITING.md deleted file mode 100644 index aabbb08..0000000 --- a/ui/ENHANCED_JOB_EDITING.md +++ /dev/null @@ -1,209 +0,0 @@ -# Enhanced Job Editing Features - -## Summary - -Added comprehensive job editing capabilities with name field, editable job ID, runner dropdown, and signature management directly in the sidebar. - -## New Features - -### 1. Job Name Field ✅ -- **Optional field** for human-readable job names -- Separate from the auto-generated job ID -- Placeholder: "My Job" -- Stored in job object as `name` field -- Displayed in edit/create mode - -### 2. Editable Job ID ✅ -- **Auto-generated** by default (job-{uuid}) -- **Manually editable** if needed -- Monospace font for better readability -- Placeholder shows format: "job-xxxxx" -- Allows custom job IDs when required - -### 3. Runner Dropdown ✅ -- **Replaced text input** with proper dropdown/select -- Shows all available runners -- Auto-selects first runner when creating new job -- Clear "No runners available" message when empty -- Better UX than text input with datalist - -### 4. Signature Management ✅ -- **Add signatures** directly in sidebar -- **Display signatures** with public key preview -- **Remove signatures** with × button -- **Private key input** (password field) -- **Sign button** to add signature -- Signatures shown as compact cards with monospace font - -## Implementation Details - -### State Added -```rust -edit_job_name: String, // Optional job name -edit_job_id: String, // Editable job ID -edit_job_signatures: Vec<(String, String)>, // (pubkey, signature) -edit_job_private_key: String, // For signing -``` - -### Messages Added -```rust -UpdateEditJobName(String), // Update name field -UpdateEditJobId(String), // Update job ID -UpdateEditJobPrivateKey(String), // Update private key -SignJobEdit, // Sign the job -RemoveEditSignature(usize), // Remove signature by index -``` - -### Sidebar Fields (Edit/Create Mode) - -**Order:** -1. **Name** (optional) - Text input -2. **Job ID** - Editable text input (monospace) -3. **Runner** - Dropdown select -4. **Timeout** - Number input -5. **Signatures** - List + Add interface - - Existing signatures displayed as cards - - Private key input (password) - - Sign button - -### Signature Display - -Each signature shows: -- First 16 characters of public key -- Monospace font -- Remove button (×) -- Compact card layout -- Background color for visibility - -### Signature Adding - -- Password input for private key -- Sign button (disabled when empty) -- Creates canonical representation -- Signs with provided key -- Adds to signatures list -- Clears private key after signing -- Shows toast notification - -## User Flow - -### Creating a Job -``` -1. Click "+ New Job" -2. Fill in fields: - - Name: "My Important Job" (optional) - - Job ID: Auto-filled, can edit - - Runner: Select from dropdown - - Timeout: 30 (default) - - Payload: Enter script/code -3. Optional: Add signatures - - Enter private key - - Click "Sign" - - Repeat for multiple signatures -4. Click "Create" -``` - -### Editing a Job -``` -1. Click job in table -2. Click "Edit" in sidebar -3. Modify fields: - - Name, Job ID, Runner, Timeout - - Payload in main area -4. Optional: Add/remove signatures -5. Click "Save" -``` - -### Signing a Job -``` -1. In edit/create mode -2. Scroll to Signatures section -3. Enter private key (hex format) -4. Click "Sign" -5. Signature added to list -6. Private key cleared automatically -7. Repeat for multiple signers -``` - -## Benefits - -### User Experience -- **Clearer job identification**: Name + ID -- **Easier runner selection**: Dropdown vs typing -- **Flexible job IDs**: Can customize when needed -- **Integrated signing**: No separate modal -- **Visual feedback**: Signatures displayed inline - -### Data Model -- **Name field**: Optional, human-readable -- **ID field**: Required, system identifier -- **Separation of concerns**: Name for humans, ID for system -- **Signature support**: Full cryptographic signing - -### Security -- **Password field**: Private keys not visible -- **Canonical representation**: Proper signing protocol -- **Multiple signatures**: Support for multi-sig workflows -- **Key cleared**: Private key removed after signing - -## Technical Details - -### Job Object Structure -```json -{ - "id": "job-abc123", - "name": "My Important Job", // Optional - "payload": "...", - "runner": "runner-name", - "timeout": 30, - "signatures": [ - { - "public_key": "04e58314...", - "signature": "3045..." - } - ], - ... -} -``` - -### Signing Process -1. Validate all required fields -2. Create canonical representation using `create_job_canonical_repr` -3. Sign canonical with `sign_job_canonical` -4. Extract public key and signature from result -5. Add to signatures list -6. Clear private key input - -### Runner Dropdown -- Uses HTML `