initial commit
This commit is contained in:
Timur Gordon
280
README.md
Normal file
280
README.md
Normal file
@@ -0,0 +1,280 @@
|
||||
# Hero Runner V
|
||||
|
||||
A V language implementation of the Hero Baobab runner system for distributed job execution. This runner provides a Redis-based job queue system with support for various executors including tmux sessions.
|
||||
|
||||
## Overview
|
||||
|
||||
The Hero Runner V is a lightweight, high-performance job execution system built in V lang. It connects to Redis for job queuing and supports multiple execution environments through configurable executors.
|
||||
|
||||
## Features
|
||||
|
||||
- **Redis-based Job Queue**: Reliable job queuing and status tracking
|
||||
- **Multiple Executors**: Support for tmux sessions, windows, and panes
|
||||
- **Job Lifecycle Management**: Complete job status tracking (dispatched → started → finished/error)
|
||||
- **Configurable Timeouts**: Per-job timeout configuration
|
||||
- **Environment Variables**: Job-specific environment variable support
|
||||
- **Namespace Support**: Multi-tenant runner organization
|
||||
- **CLI Interface**: Command-line interface with flag parsing
|
||||
|
||||
## Architecture
|
||||
|
||||
```text
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ Redis Queue │ │ Runner │ │ Executor │
|
||||
│ │ │ │ │ │
|
||||
│ Job Storage │◄──►│ Job Processor │◄──►│ tmux.session │
|
||||
│ Status Updates │ │ Status Manager │ │ tmux.window │
|
||||
│ │ │ │ │ tmux.pane │
|
||||
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||
```
|
||||
|
||||
## Installation
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- V language compiler
|
||||
- Redis server
|
||||
- Herolib dependencies
|
||||
|
||||
### Quick Install
|
||||
|
||||
```bash
|
||||
# Run the installation script
|
||||
./scripts/install.sh
|
||||
```
|
||||
|
||||
This will:
|
||||
1. Install V language and Herolib
|
||||
2. Set up module dependencies
|
||||
3. Link the runner module to vmodules
|
||||
|
||||
### Manual Installation
|
||||
|
||||
```bash
|
||||
# Install V & Herolib
|
||||
curl 'https://raw.githubusercontent.com/freeflowuniverse/herolib/refs/heads/development/install_v.sh' > /tmp/install_v.sh
|
||||
bash /tmp/install_v.sh --analyzer --herolib
|
||||
|
||||
# Install herolib
|
||||
cd ${HOME}/code/github/freeflowuniverse/herolib
|
||||
bash install_herolib.vsh
|
||||
|
||||
# Link runner module
|
||||
mkdir -p "${HOME}/.vmodules/herocode"
|
||||
ln -s "/path/to/runner_v/src" "${HOME}/.vmodules/herocode/runner"
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Starting the Runner
|
||||
|
||||
```bash
|
||||
# Start with default settings
|
||||
./scripts/run.sh
|
||||
|
||||
# Or run directly with custom options
|
||||
./cmd/runner.vsh --name my_runner --namespace production --redis_url redis://localhost:6379
|
||||
```
|
||||
|
||||
### Command Line Options
|
||||
|
||||
```bash
|
||||
Usage: runner [flags]
|
||||
|
||||
Flags:
|
||||
-n --name name of the runner (default: test_runner)
|
||||
-s --namespace namespace of the runner (default: '')
|
||||
-r --redis_url redis url (default: 127.0.0.1:6379)
|
||||
-h --help Show help message
|
||||
```
|
||||
|
||||
### Creating and Dispatching Jobs
|
||||
|
||||
```v
|
||||
import herocode.runner
|
||||
import freeflowuniverse.herolib.core.redisclient
|
||||
import time
|
||||
|
||||
// Create a job
|
||||
job := runner.new_job(runner.Job{
|
||||
caller_id: 'my_app'
|
||||
context_id: 'task_123'
|
||||
runner: 'my_runner'
|
||||
executor: 'tmux.session1.window1'
|
||||
payload: 'echo "Hello World" && sleep 5'
|
||||
timeout: 30
|
||||
})!
|
||||
|
||||
// Connect to Redis and store the job
|
||||
mut redis_conn := redisclient.new('127.0.0.1:6379')!
|
||||
job.store_in_redis(mut redis_conn, 'production:')!
|
||||
|
||||
// Add job to runner queue
|
||||
runner_queue_key := 'production:runner:my_runner'
|
||||
mut queue := redis_conn.queue_get(runner_queue_key)
|
||||
queue.add(job.id)!
|
||||
```
|
||||
|
||||
## Job Structure
|
||||
|
||||
Jobs contain the following fields:
|
||||
|
||||
```v
|
||||
pub struct Job {
|
||||
pub mut:
|
||||
id string // Unique job identifier
|
||||
caller_id string // ID of the calling service
|
||||
context_id string // Context/session identifier
|
||||
runner string // Target runner name
|
||||
executor string // Execution environment (e.g., tmux.session1)
|
||||
payload string // Script/command to execute
|
||||
status JobStatus // Current job status
|
||||
result string // Job execution result
|
||||
error string // Error message if failed
|
||||
timeout int // Timeout in seconds
|
||||
env_vars map[string]string // Environment variables
|
||||
created_at time.Time // Job creation timestamp
|
||||
updated_at time.Time // Last update timestamp
|
||||
}
|
||||
```
|
||||
|
||||
### Job Status Lifecycle
|
||||
|
||||
```text
|
||||
dispatched → started → finished
|
||||
↓ ↓ ↑
|
||||
└─────── error ──────┘
|
||||
```
|
||||
|
||||
- **dispatched**: Job is queued and waiting for processing
|
||||
- **started**: Job execution has begun
|
||||
- **finished**: Job completed successfully
|
||||
- **error**: Job failed during execution
|
||||
|
||||
## Executor Types
|
||||
|
||||
The runner supports various executor types for different execution environments:
|
||||
|
||||
### Tmux Executors
|
||||
|
||||
- `tmux.session_name` - Execute in a tmux session
|
||||
- `tmux.session_name.window_name` - Execute in a specific window
|
||||
- `tmux.session_name.window_name.pane_id` - Execute in a specific pane
|
||||
|
||||
Example:
|
||||
```v
|
||||
job := runner.Job{
|
||||
// ... other fields
|
||||
executor: 'tmux.dev_session.main_window.pane1'
|
||||
payload: 'npm run build && npm test'
|
||||
}
|
||||
```
|
||||
|
||||
## Examples
|
||||
|
||||
### Basic Runner
|
||||
|
||||
```v
|
||||
#!/usr/bin/env -S v run
|
||||
|
||||
import herocode.runner
|
||||
import freeflowuniverse.herolib.core.redisclient
|
||||
|
||||
mut r := &runner.Runner{
|
||||
name: 'basic_runner'
|
||||
namespace: 'development'
|
||||
redis_conn: redisclient.new('127.0.0.1:6379')!
|
||||
}
|
||||
|
||||
// Start the runner
|
||||
spawn r.run()
|
||||
|
||||
// Keep running
|
||||
for {}
|
||||
```
|
||||
|
||||
### Job Submission Script
|
||||
|
||||
See `examples/test_runner_rpc.vsh` for a complete example of submitting multiple jobs to a runner.
|
||||
|
||||
## Testing
|
||||
|
||||
Run the test suite:
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
v test src/
|
||||
|
||||
# Run specific test files
|
||||
v test src/runner_test.v
|
||||
v test src/job_test.v
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
- `REDIS_URL`: Redis connection URL (default: 127.0.0.1:6379)
|
||||
- `RUNNER_NAME`: Default runner name
|
||||
- `RUNNER_NAMESPACE`: Default namespace
|
||||
|
||||
### Redis Keys
|
||||
|
||||
The runner uses the following Redis key patterns:
|
||||
|
||||
- Jobs: `{namespace}job_{id}`
|
||||
- Runner Queues: `{namespace}runner:{runner_name}`
|
||||
- Status Updates: Stored within job objects
|
||||
|
||||
## Development
|
||||
|
||||
### Project Structure
|
||||
|
||||
```
|
||||
runner_v/
|
||||
├── cmd/ # Command-line applications
|
||||
│ └── runner.vsh # Main runner executable
|
||||
├── src/ # Source code
|
||||
│ ├── runner.v # Core runner implementation
|
||||
│ ├── job.v # Job structure and operations
|
||||
│ ├── factory.v # Job creation utilities
|
||||
│ └── *_test.v # Test files
|
||||
├── examples/ # Usage examples
|
||||
├── scripts/ # Build and deployment scripts
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
### Building
|
||||
|
||||
```bash
|
||||
# Build the runner
|
||||
v -prod cmd/runner.vsh -o runner
|
||||
|
||||
# Run in development mode
|
||||
v run cmd/runner.vsh
|
||||
```
|
||||
|
||||
## Contributing
|
||||
|
||||
1. Fork the repository
|
||||
2. Create a feature branch
|
||||
3. Make your changes
|
||||
4. Add tests for new functionality
|
||||
5. Run the test suite
|
||||
6. Submit a pull request
|
||||
|
||||
## License
|
||||
|
||||
MIT License - see LICENSE file for details.
|
||||
|
||||
## Related Projects
|
||||
|
||||
- [Herolib](https://github.com/freeflowuniverse/herolib) - Core utilities and libraries
|
||||
- [Hero Baobab](https://github.com/freeflowuniverse/baobab) - Distributed actor system
|
||||
|
||||
## Support
|
||||
|
||||
For issues and questions:
|
||||
- Open an issue on GitHub
|
||||
- Check the examples directory for usage patterns
|
||||
- Review the test files for implementation details
|
||||
Reference in New Issue
Block a user