initial commit

core/worker/cmd/README.md

@@ -0,0 +1,113 @@
+# Rhai Worker Binary
+
+A command-line worker for executing Rhai scripts from Redis task queues.
+
+## Binary: `worker`
+
+### Installation
+
+Build the binary:
+```bash
+cargo build --bin worker --release
+```
+
+### Usage
+
+```bash
+# Basic usage - requires circle public key
+worker --circle-public-key <CIRCLE_PUBLIC_KEY>
+
+# Custom Redis URL
+worker -c <CIRCLE_PUBLIC_KEY> --redis-url redis://localhost:6379/1
+
+# Custom worker ID and database path
+worker -c <CIRCLE_PUBLIC_KEY> --worker-id my_worker --db-path /tmp/worker_db
+
+# Preserve tasks for debugging/benchmarking
+worker -c <CIRCLE_PUBLIC_KEY> --preserve-tasks
+
+# Remove timestamps from logs
+worker -c <CIRCLE_PUBLIC_KEY> --no-timestamp
+
+# Increase verbosity
+worker -c <CIRCLE_PUBLIC_KEY> -v    # Debug logging
+worker -c <CIRCLE_PUBLIC_KEY> -vv   # Full debug
+worker -c <CIRCLE_PUBLIC_KEY> -vvv  # Trace logging
+```
+
+### Command-Line Options
+
+| Option | Short | Default | Description |
+|--------|-------|---------|-------------|
+| `--circle-public-key` | `-c` | **Required** | Circle public key to listen for tasks |
+| `--redis-url` | `-r` | `redis://localhost:6379` | Redis connection URL |
+| `--worker-id` | `-w` | `worker_1` | Unique worker identifier |
+| `--preserve-tasks` | | `false` | Preserve task details after completion |
+| `--db-path` | | `worker_rhai_temp_db` | Database path for Rhai engine |
+| `--no-timestamp` | | `false` | Remove timestamps from log output |
+| `--verbose` | `-v` | | Increase verbosity (stackable) |
+
+### Features
+
+- **Task Queue Processing**: Listens to Redis queues for Rhai script execution tasks
+- **Performance Optimized**: Configured for maximum Rhai engine performance
+- **Graceful Shutdown**: Supports shutdown signals for clean termination
+- **Flexible Logging**: Configurable verbosity and timestamp control
+- **Database Integration**: Uses heromodels for data persistence
+- **Task Cleanup**: Optional task preservation for debugging/benchmarking
+
+### How It Works
+
+1. **Queue Listening**: Worker listens on Redis queue `rhailib:{circle_public_key}`
+2. **Task Processing**: Receives task IDs, fetches task details from Redis
+3. **Script Execution**: Executes Rhai scripts with configured engine
+4. **Result Handling**: Updates task status and sends results to reply queues
+5. **Cleanup**: Optionally cleans up task details after completion
+
+### Configuration Examples
+
+#### Development Worker
+```bash
+# Simple development worker
+worker -c dev_circle_123
+
+# Development with verbose logging (no timestamps)
+worker -c dev_circle_123 -v --no-timestamp
+```
+
+#### Production Worker
+```bash
+# Production worker with custom configuration
+worker \
+  --circle-public-key prod_circle_456 \
+  --redis-url redis://redis-server:6379/0 \
+  --worker-id prod_worker_1 \
+  --db-path /var/lib/worker/db \
+  --preserve-tasks
+```
+
+#### Benchmarking Worker
+```bash
+# Worker optimized for benchmarking
+worker \
+  --circle-public-key bench_circle_789 \
+  --preserve-tasks \
+  --no-timestamp \
+  -vv
+```
+
+### Error Handling
+
+The worker provides clear error messages for:
+- Missing or invalid circle public key
+- Redis connection failures
+- Script execution errors
+- Database access issues
+
+### Dependencies
+
+- `rhailib_engine`: Rhai engine with heromodels integration
+- `redis`: Redis client for task queue management
+- `rhai`: Script execution engine
+- `clap`: Command-line argument parsing
+- `env_logger`: Logging infrastructure

core/worker/cmd/worker.rs

@@ -0,0 +1,95 @@
+use clap::Parser;
+use rhailib_engine::create_heromodels_engine;
+use rhailib_worker::spawn_rhai_worker;
+use tokio::sync::mpsc;
+
+#[derive(Parser, Debug)]
+#[command(author, version, about, long_about = None)]
+struct Args {
+    /// Worker ID for identification
+    #[arg(short, long)]
+    worker_id: String,
+
+    /// Redis URL
+    #[arg(short, long, default_value = "redis://localhost:6379")]
+    redis_url: String,
+
+    /// Preserve task details after completion (for benchmarking)
+    #[arg(long, default_value = "false")]
+    preserve_tasks: bool,
+
+    /// Root directory for engine database
+    #[arg(long, default_value = "worker_rhai_temp_db")]
+    db_path: String,
+
+    /// Disable timestamps in log output
+    #[arg(long, help = "Remove timestamps from log output")]
+    no_timestamp: bool,
+}
+
+#[tokio::main]
+async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
+    let args = Args::parse();
+
+    // Configure env_logger with or without timestamps
+    if args.no_timestamp {
+        env_logger::Builder::from_default_env()
+            .format_timestamp(None)
+            .init();
+    } else {
+        env_logger::init();
+    }
+
+
+    log::info!("Rhai Worker (binary) starting with performance-optimized engine.");
+    log::info!(
+        "Worker ID: {}, Redis: {}",
+        args.worker_id,
+        args.redis_url
+    );
+
+    let mut engine = create_heromodels_engine();
+
+    // Performance optimizations for benchmarking
+    engine.set_max_operations(0); // Unlimited operations for performance testing
+    engine.set_max_expr_depths(0, 0); // Unlimited expression depth
+    engine.set_max_string_size(0); // Unlimited string size
+    engine.set_max_array_size(0); // Unlimited array size
+    engine.set_max_map_size(0); // Unlimited map size
+
+    // Enable full optimization for maximum performance
+    engine.set_optimization_level(rhai::OptimizationLevel::Full);
+
+    log::info!("Engine configured for maximum performance");
+
+    // Create shutdown channel (for graceful shutdown, though not used in benchmarks)
+    let (_shutdown_tx, shutdown_rx) = mpsc::channel::<()>(1);
+
+    // Spawn the worker
+    let worker_handle = spawn_rhai_worker(
+        args.worker_id,
+        args.db_path,
+        engine,
+        args.redis_url,
+        shutdown_rx,
+        args.preserve_tasks,
+    );
+
+    // Wait for the worker to complete
+    match worker_handle.await {
+        Ok(result) => match result {
+            Ok(_) => {
+                log::info!("Worker completed successfully");
+                Ok(())
+            }
+            Err(e) => {
+                log::error!("Worker failed: {}", e);
+                Err(e)
+            }
+        },
+        Err(e) => {
+            log::error!("Worker task panicked: {}", e);
+            Err(Box::new(e) as Box<dyn std::error::Error + Send + Sync>)
+        }
+    }
+}