280 lines
7.5 KiB
Markdown
280 lines
7.5 KiB
Markdown
# Hero Runner V
|
|
|
|
A V language implementation of the Hero Baobab runner system for distributed job execution. This runner provides a Redis-based job queue system with support for various executors including tmux sessions.
|
|
|
|
## Overview
|
|
|
|
The Hero Runner V is a lightweight, high-performance job execution system built in V lang. It connects to Redis for job queuing and supports multiple execution environments through configurable executors.
|
|
|
|
## Features
|
|
|
|
- **Redis-based Job Queue**: Reliable job queuing and status tracking
|
|
- **Multiple Executors**: Support for tmux sessions, windows, and panes
|
|
- **Job Lifecycle Management**: Complete job status tracking (dispatched → started → finished/error)
|
|
- **Configurable Timeouts**: Per-job timeout configuration
|
|
- **Environment Variables**: Job-specific environment variable support
|
|
- **Namespace Support**: Multi-tenant runner organization
|
|
- **CLI Interface**: Command-line interface with flag parsing
|
|
|
|
## Architecture
|
|
|
|
```text
|
|
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
|
│ Redis Queue │ │ Runner │ │ Executor │
|
|
│ │ │ │ │ │
|
|
│ Job Storage │◄──►│ Job Processor │◄──►│ tmux.session │
|
|
│ Status Updates │ │ Status Manager │ │ tmux.window │
|
|
│ │ │ │ │ tmux.pane │
|
|
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
|
```
|
|
|
|
## Installation
|
|
|
|
### Prerequisites
|
|
|
|
- V language compiler
|
|
- Redis server
|
|
- Herolib dependencies
|
|
|
|
### Quick Install
|
|
|
|
```bash
|
|
# Run the installation script
|
|
./scripts/install.sh
|
|
```
|
|
|
|
This will:
|
|
1. Install V language and Herolib
|
|
2. Set up module dependencies
|
|
3. Link the runner module to vmodules
|
|
|
|
### Manual Installation
|
|
|
|
```bash
|
|
# Install V & Herolib
|
|
curl 'https://raw.githubusercontent.com/freeflowuniverse/herolib/refs/heads/development/install_v.sh' > /tmp/install_v.sh
|
|
bash /tmp/install_v.sh --analyzer --herolib
|
|
|
|
# Install herolib
|
|
cd ${HOME}/code/github/freeflowuniverse/herolib
|
|
bash install_herolib.vsh
|
|
|
|
# Link runner module
|
|
mkdir -p "${HOME}/.vmodules/herocode"
|
|
ln -s "/path/to/runner_v/src" "${HOME}/.vmodules/herocode/runner"
|
|
```
|
|
|
|
## Usage
|
|
|
|
### Starting the Runner
|
|
|
|
```bash
|
|
# Start with default settings
|
|
./scripts/run.sh
|
|
|
|
# Or run directly with custom options
|
|
./cmd/runner.vsh --name my_runner --namespace production --redis_url redis://localhost:6379
|
|
```
|
|
|
|
### Command Line Options
|
|
|
|
```bash
|
|
Usage: runner [flags]
|
|
|
|
Flags:
|
|
-n --name name of the runner (default: test_runner)
|
|
-s --namespace namespace of the runner (default: '')
|
|
-r --redis_url redis url (default: 127.0.0.1:6379)
|
|
-h --help Show help message
|
|
```
|
|
|
|
### Creating and Dispatching Jobs
|
|
|
|
```v
|
|
import herocode.runner
|
|
import freeflowuniverse.herolib.core.redisclient
|
|
import time
|
|
|
|
// Create a job
|
|
job := runner.new_job(runner.Job{
|
|
caller_id: 'my_app'
|
|
context_id: 'task_123'
|
|
runner: 'my_runner'
|
|
executor: 'tmux.session1.window1'
|
|
payload: 'echo "Hello World" && sleep 5'
|
|
timeout: 30
|
|
})!
|
|
|
|
// Connect to Redis and store the job
|
|
mut redis_conn := redisclient.new('127.0.0.1:6379')!
|
|
job.store_in_redis(mut redis_conn, 'production:')!
|
|
|
|
// Add job to runner queue
|
|
runner_queue_key := 'production:runner:my_runner'
|
|
mut queue := redis_conn.queue_get(runner_queue_key)
|
|
queue.add(job.id)!
|
|
```
|
|
|
|
## Job Structure
|
|
|
|
Jobs contain the following fields:
|
|
|
|
```v
|
|
pub struct Job {
|
|
pub mut:
|
|
id string // Unique job identifier
|
|
caller_id string // ID of the calling service
|
|
context_id string // Context/session identifier
|
|
runner string // Target runner name
|
|
executor string // Execution environment (e.g., tmux.session1)
|
|
payload string // Script/command to execute
|
|
status JobStatus // Current job status
|
|
result string // Job execution result
|
|
error string // Error message if failed
|
|
timeout int // Timeout in seconds
|
|
env_vars map[string]string // Environment variables
|
|
created_at time.Time // Job creation timestamp
|
|
updated_at time.Time // Last update timestamp
|
|
}
|
|
```
|
|
|
|
### Job Status Lifecycle
|
|
|
|
```text
|
|
dispatched → started → finished
|
|
↓ ↓ ↑
|
|
└─────── error ──────┘
|
|
```
|
|
|
|
- **dispatched**: Job is queued and waiting for processing
|
|
- **started**: Job execution has begun
|
|
- **finished**: Job completed successfully
|
|
- **error**: Job failed during execution
|
|
|
|
## Executor Types
|
|
|
|
The runner supports various executor types for different execution environments:
|
|
|
|
### Tmux Executors
|
|
|
|
- `tmux.session_name` - Execute in a tmux session
|
|
- `tmux.session_name.window_name` - Execute in a specific window
|
|
- `tmux.session_name.window_name.pane_id` - Execute in a specific pane
|
|
|
|
Example:
|
|
```v
|
|
job := runner.Job{
|
|
// ... other fields
|
|
executor: 'tmux.dev_session.main_window.pane1'
|
|
payload: 'npm run build && npm test'
|
|
}
|
|
```
|
|
|
|
## Examples
|
|
|
|
### Basic Runner
|
|
|
|
```v
|
|
#!/usr/bin/env -S v run
|
|
|
|
import herocode.runner
|
|
import freeflowuniverse.herolib.core.redisclient
|
|
|
|
mut r := &runner.Runner{
|
|
name: 'basic_runner'
|
|
namespace: 'development'
|
|
redis_conn: redisclient.new('127.0.0.1:6379')!
|
|
}
|
|
|
|
// Start the runner
|
|
spawn r.run()
|
|
|
|
// Keep running
|
|
for {}
|
|
```
|
|
|
|
### Job Submission Script
|
|
|
|
See `examples/test_runner_rpc.vsh` for a complete example of submitting multiple jobs to a runner.
|
|
|
|
## Testing
|
|
|
|
Run the test suite:
|
|
|
|
```bash
|
|
# Run all tests
|
|
v test src/
|
|
|
|
# Run specific test files
|
|
v test src/runner_test.v
|
|
v test src/job_test.v
|
|
```
|
|
|
|
## Configuration
|
|
|
|
### Environment Variables
|
|
|
|
- `REDIS_URL`: Redis connection URL (default: 127.0.0.1:6379)
|
|
- `RUNNER_NAME`: Default runner name
|
|
- `RUNNER_NAMESPACE`: Default namespace
|
|
|
|
### Redis Keys
|
|
|
|
The runner uses the following Redis key patterns:
|
|
|
|
- Jobs: `{namespace}job_{id}`
|
|
- Runner Queues: `{namespace}runner:{runner_name}`
|
|
- Status Updates: Stored within job objects
|
|
|
|
## Development
|
|
|
|
### Project Structure
|
|
|
|
```
|
|
runner_v/
|
|
├── cmd/ # Command-line applications
|
|
│ └── runner.vsh # Main runner executable
|
|
├── src/ # Source code
|
|
│ ├── runner.v # Core runner implementation
|
|
│ ├── job.v # Job structure and operations
|
|
│ ├── factory.v # Job creation utilities
|
|
│ └── *_test.v # Test files
|
|
├── examples/ # Usage examples
|
|
├── scripts/ # Build and deployment scripts
|
|
└── README.md # This file
|
|
```
|
|
|
|
### Building
|
|
|
|
```bash
|
|
# Build the runner
|
|
v -prod cmd/runner.vsh -o runner
|
|
|
|
# Run in development mode
|
|
v run cmd/runner.vsh
|
|
```
|
|
|
|
## Contributing
|
|
|
|
1. Fork the repository
|
|
2. Create a feature branch
|
|
3. Make your changes
|
|
4. Add tests for new functionality
|
|
5. Run the test suite
|
|
6. Submit a pull request
|
|
|
|
## License
|
|
|
|
MIT License - see LICENSE file for details.
|
|
|
|
## Related Projects
|
|
|
|
- [Herolib](https://github.com/freeflowuniverse/herolib) - Core utilities and libraries
|
|
- [Hero Baobab](https://github.com/freeflowuniverse/baobab) - Distributed actor system
|
|
|
|
## Support
|
|
|
|
For issues and questions:
|
|
- Open an issue on GitHub
|
|
- Check the examples directory for usage patterns
|
|
- Review the test files for implementation details |