herocode/circles
12
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

add circle rhai repl and backend start cmd

Browse Source
This commit is contained in:
timurgordon
2025-06-05 00:29:10 +03:00
parent f22d40c980
commit ae3077033b
15 changed files with 5522 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
target

201
ARCHITECTURE.md Normal file
View File

@@ -0,0 +1,201 @@
# Architecture: Circle Management System
## 1. Introduction & Overview
This document outlines the architecture for a system that manages multiple "Circles." Each Circle is an independent entity comprising its own database, a Rhai scripting engine, a dedicated Rhai worker process, and a WebSocket (WS) server for external interaction. A central command-line orchestrator will be responsible for initializing, running, and monitoring these Circles based on a configuration file.
The primary goal is to allow users to define multiple isolated Rhai environments, each accessible via a unique WebSocket endpoint, and for scripts executed within a circle to interact with that circle's dedicated persistent storage.
## 2. Goals
* Create a command-line application (`circles_orchestrator`) to manage the lifecycle of multiple Circles.
* Each Circle will have:
* An independent `OurDB` instance for data persistence.
* A dedicated `Rhai Engine` configured with its `OurDB`.
* A dedicated `Rhai Worker` processing scripts for that circle.
* A dedicated `WebSocket Server` exposing an endpoint for that circle.
* Circle configurations (name, ID, port) will be loaded from a `circles.json` file.
* The orchestrator will display a status table of all running circles, including their worker queue and WS server URL.
* Utilize existing crates (`ourdb`, `rhai_engine`, `rhai_worker`, `rhai_client`, `server_ws`) with necessary refactoring to support library usage.
## 3. System Components
* **Orchestrator (`circles_orchestrator`)**:
* Location: `/Users/timurgordon/code/git.ourworld.tf/herocode/circles/cmd/src/main.rs`
* Role: Parses `circles.json`, initializes, spawns, and monitors all components for each defined circle. Displays system status.
* **Circle Configuration (`circles.json`)**:
* Location: e.g., `/Users/timurgordon/code/git.ourworld.tf/herocode/circles/cmd/circles.json`
* Role: Defines the set of circles to be managed, including their ID, name, and port.
* **OurDB (Per Circle)**:
* Library: `/Users/timurgordon/code/git.ourworld.tf/herocode/db/ourdb/src/lib.rs`
* Role: Provides persistent key-value storage for each circle, configured in incremental mode. Instance data stored at `~/.hero/circles/{id}/`.
* **Rhai Engine (Per Circle)**:
* Library: `/Users/timurgordon/code/git.ourworld.tf/herocode/rhailib/src/engine/src/lib.rs`
* Role: Provides the Rhai scripting environment for a circle, configured with the circle's specific `OurDB` instance.
* **Rhai Worker (Per Circle)**:
* Library: `/Users/timurgordon/code/git.ourworld.tf/herocode/rhailib/src/worker/src/lib.rs`
* Role: Executes Rhai scripts for a specific circle. Listens on a dedicated Redis queue for tasks and uses the circle's `Rhai Engine`.
* **Rhai Client (Per Circle WS Server)**:
* Library: `/Users/timurgordon/code/git.ourworld.tf/herocode/rhailib/src/client/src/lib.rs`
* Role: Used by a `Circle WebSocket Server` to send script execution tasks to its corresponding `Rhai Worker` via Redis.
* **Circle WebSocket Server (Per Circle)**:
* Library: `/Users/timurgordon/code/git.ourworld.tf/herocode/circles/server_ws/src/lib.rs`
* Role: Exposes a WebSocket endpoint for a specific circle, allowing external clients to submit Rhai scripts for execution within that circle.
* **Redis**:
* URL: `redis://127.0.0.1:6379`
* Role: Acts as the message broker between `Rhai Clients` (within WS Servers) and `Rhai Workers`.
## 4. High-Level Design Aspects
### 4.1. Orchestrator Logic (Conceptual)
The `circles_orchestrator` reads the `circles.json` configuration. For each defined circle, it:
1. Determines the `OurDB` path based on the circle's ID.
2. Initializes the `OurDB` instance.
3. Creates a `Rhai Engine` configured with this `OurDB`.
4. Spawns a `Rhai Worker` task, providing it with the engine and the circle's identity (for Redis queue naming).
5. Spawns a `Circle WebSocket Server` task, providing it with the circle's identity and port.
It then monitors these components and displays their status.
### 4.2. Database Path Convention
* Base: System user's home directory (e.g., `/Users/username` or `/home/username`).
* Structure: `~/.hero/circles/{CIRCLE_ID}/`
* Example: For a circle with ID `1`, the database path would be `~/.hero/circles/1/`.
### 4.3. Configuration File Format (`circles.json`)
A JSON array of objects. Each object represents a circle:
* `id`: `u32` - Unique identifier.
* `name`: `String` - Human-readable name.
* `port`: `u16` - Port for the WebSocket server.
```json
[
{ "id": 1, "name": "Alpha Circle", "port": 8081 },
{ "id": 2, "name": "Beta Circle", "port": 8082 }
]
```
### 4.4. Conceptual Output Table Format (Orchestrator)
| Circle Name | ID | Worker Status | Worker Queues | WS Server URL |
|--------------|----|---------------|------------------------------------|------------------------|
| Alpha Circle | 1 | Running | `rhai_tasks:alpha_circle` | `ws://127.0.0.1:8081/ws` |
### 4.5. Interaction Flow (Single Circle)
1. An external client connects to a specific Circle's WebSocket Server.
2. The client sends a Rhai script via a JSON-RPC message.
3. The WS Server uses its embedded `Rhai Client` to publish the script and task details to a Redis queue specific to that circle (e.g., `rhai_tasks:alpha_circle`).
4. The `Rhai Worker` for that circle picks up the task from its Redis queue.
5. The Worker uses its `Rhai Engine` (which is configured with the circle's `OurDB`) to execute the script.
6. Any database interactions within the script go through the circle's `OurDB`.
7. The Worker updates the task status and result/error in Redis.
8. The `Rhai Client` (in the WS Server), which has been polling Redis for the result, receives the update.
9. The WS Server sends the script's result or error back to the external client via WebSocket.
## 5. Diagrams
### 5.1. Component Diagram
```mermaid
graph TD
UserInterface[User/Admin] -- Manages --> OrchestratorCli{circles_orchestrator}
OrchestratorCli -- Reads --> CirclesJson[circles.json]
subgraph Circle 1
direction LR
OrchestratorCli -- Spawns/Manages --> C1_OurDB[(OurDB @ ~/.hero/circles/1)]
OrchestratorCli -- Spawns/Manages --> C1_RhaiEngine[Rhai Engine 1]
OrchestratorCli -- Spawns/Manages --> C1_RhaiWorker[Rhai Worker 1]
OrchestratorCli -- Spawns/Manages --> C1_WSServer[WS Server 1 @ Port 8081]
C1_RhaiEngine -- Uses --> C1_OurDB
C1_RhaiWorker -- Uses --> C1_RhaiEngine
C1_WSServer -- Contains --> C1_RhaiClient[Rhai Client 1]
end
subgraph Circle 2
direction LR
OrchestratorCli -- Spawns/Manages --> C2_OurDB[(OurDB @ ~/.hero/circles/2)]
OrchestratorCli -- Spawns/Manages --> C2_RhaiEngine[Rhai Engine 2]
OrchestratorCli -- Spawns/Manages --> C2_RhaiWorker[Rhai Worker 2]
OrchestratorCli -- Spawns/Manages --> C2_WSServer[WS Server 2 @ Port 8082]
C2_RhaiEngine -- Uses --> C2_OurDB
C2_RhaiWorker -- Uses --> C2_RhaiEngine
C2_WSServer -- Contains --> C2_RhaiClient[Rhai Client 2]
end
C1_RhaiWorker -- Listens/Publishes --> Redis[(Redis @ 127.0.0.1:6379)]
C1_RhaiClient -- Publishes/Subscribes --> Redis
C2_RhaiWorker -- Listens/Publishes --> Redis
C2_RhaiClient -- Publishes/Subscribes --> Redis
ExternalWSClient1[External WS Client] -- Connects --> C1_WSServer
ExternalWSClient2[External WS Client] -- Connects --> C2_WSServer
```
### 5.2. Sequence Diagram (Request Flow for one Circle)
```mermaid
sequenceDiagram
participant ExtWSClient as External WS Client
participant CircleWSServer as Circle WS Server (e.g., Port 8081)
participant RhaiClientLib as Rhai Client Library (in WS Server)
participant RedisBroker as Redis
participant RhaiWorker as Rhai Worker (for the circle)
participant RhaiEngineLib as Rhai Engine Library (in Worker)
participant CircleOurDB as OurDB (for the circle)
ExtWSClient ->>+ CircleWSServer: Send Rhai Script (JSON-RPC "play" over WS)
CircleWSServer ->>+ RhaiClientLib: submit_script_and_await_result(circle_name, script, ...)
RhaiClientLib ->>+ RedisBroker: LPUSH rhai_tasks:circle_name (task_id)
RhaiClientLib ->>+ RedisBroker: HSET rhai_task_details:task_id (script details)
RhaiClientLib -->>- CircleWSServer: Returns task_id (internally starts polling)
RhaiWorker ->>+ RedisBroker: BLPOP rhai_tasks:circle_name (blocks)
RedisBroker -->>- RhaiWorker: Returns task_id
RhaiWorker ->>+ RedisBroker: HGETALL rhai_task_details:task_id
RedisBroker -->>- RhaiWorker: Returns script details
RhaiWorker ->>+ RhaiEngineLib: eval_with_scope(script)
RhaiEngineLib ->>+ CircleOurDB: DB Operations (if script interacts with DB)
CircleOurDB -->>- RhaiEngineLib: DB Results
RhaiEngineLib -->>- RhaiWorker: Script Result/Error
RhaiWorker ->>+ RedisBroker: HSET rhai_task_details:task_id (status="completed/error", output/error)
RhaiClientLib ->>+ RedisBroker: HGETALL rhai_task_details:task_id (polling)
RedisBroker -->>- RhaiClientLib: Task details (status, output/error)
alt Task Completed
RhaiClientLib -->>- CircleWSServer: Result (output)
CircleWSServer -->>- ExtWSClient: WS Response (JSON-RPC with result)
else Task Errored
RhaiClientLib -->>- CircleWSServer: Error (error message)
CircleWSServer -->>- ExtWSClient: WS Response (JSON-RPC with error)
else Timeout
RhaiClientLib -->>- CircleWSServer: Timeout Error
CircleWSServer -->>- ExtWSClient: WS Response (JSON-RPC with timeout error)
end
```
### 5.3. Conceptual Directory Structure
```
/Users/timurgordon/code/git.ourworld.tf/herocode/
├── circles/
│ ├── cmd/ <-- NEW Orchestrator Crate
│ │ ├── Cargo.toml
│ │ ├── circles.json (example config)
│ │ └── src/
│ │ └── main.rs (Orchestrator logic)
│ ├── server_ws/ <-- EXISTING, to be refactored to lib
│ │ ├── Cargo.toml
│ │ └── src/
│ │ └── lib.rs (WebSocket server library logic)
│ ├── ARCHITECTURE.md (This document)
│ └── README.md
├── db/
│ └── ourdb/
│ └── src/lib.rs (Existing OurDB library)
└── rhailib/ <-- Current Workspace (contains other existing libs)
├── src/
│ ├── client/
│ │ └── src/lib.rs (Existing Rhai Client library)
│ ├── engine/
│ │ └── src/lib.rs (Existing Rhai Engine library)
│ └── worker/
│ └── src/lib.rs (Existing Rhai Worker library, to be refactored)
├── Cargo.toml
└── ...

2644
cmd/Cargo.lock generated Normal file
View File

File diff suppressed because it is too large Load Diff

23
cmd/Cargo.toml Normal file
View File

@@ -0,0 +1,23 @@
[package]
name = "circles_orchestrator"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
tokio = { version = "1", features = ["full"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
# clap = { version = "4.0", features = ["derive"], optional = true } # Optional for future args
dirs = "5.0"
log = "0.4"
env_logger = "0.10"
comfy-table = "7.0" # For table display
# Path dependencies to other local crates
heromodels = { path = "../../db/heromodels" } # Changed from ourdb
rhai_engine = { path = "../../rhailib/src/engine" }
rhai_worker = { path = "../../rhailib/src/worker" }
# rhai_client is used by circle_ws_lib, not directly by orchestrator usually
circle_ws_lib = { path = "../server_ws" }

5
cmd/circles.json Normal file
View File

@@ -0,0 +1,5 @@
[
{ "id": 1, "name": "Alpha Circle", "port": 8091 },
{ "id": 2, "name": "Alpha Circle", "port": 8082 },
{ "id": 3, "name": "Beta Circle", "port": 8083 }
]

245
cmd/src/main.rs Normal file
View File

@@ -0,0 +1,245 @@
use std::fs;
use std::path::PathBuf;
use std::sync::{Arc, Mutex};
use serde::Deserialize;
use tokio::task::JoinHandle;
use tokio::sync::{oneshot, mpsc}; // For server handles and worker shutdown
use tokio::signal;
use std::time::Duration;
use comfy_table::{Table, Row, Cell, ContentArrangement};
use log::{info, error, warn, debug};
use heromodels::db::hero::{OurDB as HeroOurDB}; // Renamed to avoid conflict if OurDB is used from elsewhere
use rhai_engine::create_heromodels_engine;
use worker_lib::spawn_rhai_worker; // This now takes a shutdown_rx
use circle_ws_lib::spawn_circle_ws_server; // This now takes a server_handle_tx
const DEFAULT_REDIS_URL: &str = "redis://127.0.0.1:6379";
#[derive(Deserialize, Debug, Clone)]
struct CircleConfig {
id: u32,
name: String,
port: u16,
}
struct RunningCircleInfo {
config: CircleConfig,
db_path: PathBuf,
worker_queue: String,
ws_url: String,
worker_handle: JoinHandle<Result<(), Box<dyn std::error::Error + Send + Sync>>>,
worker_shutdown_tx: mpsc::Sender<()>, // To signal worker to stop
// Store the server handle for graceful shutdown, and its JoinHandle
ws_server_instance_handle: Arc<Mutex<Option<actix_web::dev::Server>>>,
ws_server_task_join_handle: JoinHandle<std::io::Result<()>>,
status: Arc<Mutex<String>>,
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
std::env::set_var("RUST_LOG", "info,circles_orchestrator=debug,worker_lib=debug,circle_ws_lib=debug,rhai_client=debug,actix_server=info");
env_logger::init();
info!("Starting Circles Orchestrator...");
info!("Press Ctrl+C to initiate graceful shutdown.");
let config_path = PathBuf::from("./circles.json");
if !config_path.exists() {
error!("Configuration file not found at {:?}. Please create circles.json.", config_path);
return Err("circles.json not found".into());
}
let config_content = fs::read_to_string(config_path)?;
let circle_configs: Vec<CircleConfig> = serde_json::from_str(&config_content)?;
if circle_configs.is_empty() {
warn!("No circle configurations found in circles.json. Exiting.");
return Ok(());
}
info!("Loaded {} circle configurations.", circle_configs.len());
let mut running_circles_store: Vec<Arc<Mutex<RunningCircleInfo>>> = Vec::new();
for config in circle_configs {
info!("Initializing Circle ID: {}, Name: '{}', Port: {}", config.id, config.name, config.port);
let current_status = Arc::new(Mutex::new(format!("Initializing Circle {}", config.id)));
let db_base_path = match dirs::home_dir() {
Some(path) => path.join(".hero").join("circles"),
None => {
error!("Failed to get user home directory for Circle ID {}.", config.id);
*current_status.lock().unwrap() = "Error: DB Path".to_string();
// Not pushing to running_circles_store as it can't fully initialize
continue;
}
};
let circle_db_path = db_base_path.join(config.id.to_string());
if !circle_db_path.exists() {
if let Err(e) = fs::create_dir_all(&circle_db_path) {
error!("Failed to create database directory for Circle {}: {:?}. Error: {}", config.id, circle_db_path, e);
*current_status.lock().unwrap() = "Error: DB Create".to_string();
continue;
}
info!("Created database directory for Circle {}: {:?}", config.id, circle_db_path);
}
let db = match HeroOurDB::new(circle_db_path.clone(), false) {
Ok(db_instance) => Arc::new(db_instance),
Err(e) => {
error!("Failed to initialize heromodels::OurDB for Circle {}: {:?}", config.id, e);
*current_status.lock().unwrap() = "Error: DB Init".to_string();
continue;
}
};
info!("OurDB initialized for Circle {}", config.id);
*current_status.lock().unwrap() = format!("DB Ok for Circle {}", config.id);
let engine = create_heromodels_engine(db.clone());
info!("Rhai Engine created for Circle {}", config.id);
*current_status.lock().unwrap() = format!("Engine Ok for Circle {}", config.id);
// Channel for worker shutdown
let (worker_shutdown_tx, worker_shutdown_rx) = mpsc::channel(1); // Buffer of 1 is fine
let worker_handle = spawn_rhai_worker(
config.id,
config.name.clone(),
engine, // engine is Clone
DEFAULT_REDIS_URL.to_string(),
worker_shutdown_rx, // Pass the receiver
);
info!("Rhai Worker spawned for Circle {}", config.id);
let worker_queue_name = format!("rhai_tasks:{}", config.name.replace(" ", "_").to_lowercase());
*current_status.lock().unwrap() = format!("Worker Spawning for Circle {}", config.id);
let (server_handle_tx, server_handle_rx) = oneshot::channel();
let ws_server_task_join_handle = spawn_circle_ws_server(
config.id,
config.name.clone(),
config.port,
DEFAULT_REDIS_URL.to_string(),
server_handle_tx,
);
info!("Circle WebSocket Server task spawned for Circle {} on port {}", config.id, config.port);
let ws_url = format!("ws://127.0.0.1:{}/ws", config.port);
*current_status.lock().unwrap() = format!("WS Server Spawning for Circle {}", config.id);
let server_instance_handle_arc = Arc::new(Mutex::new(None));
let server_instance_handle_clone = server_instance_handle_arc.clone();
let status_clone_for_server_handle = current_status.clone();
let circle_id_for_server_handle = config.id;
tokio::spawn(async move {
match server_handle_rx.await {
Ok(handle) => {
*server_instance_handle_clone.lock().unwrap() = Some(handle);
*status_clone_for_server_handle.lock().unwrap() = format!("Running Circle {}", circle_id_for_server_handle);
info!("Received server handle for Circle {}", circle_id_for_server_handle);
}
Err(_) => {
*status_clone_for_server_handle.lock().unwrap() = format!("Error: No Server Handle for Circle {}", circle_id_for_server_handle);
error!("Failed to receive server handle for Circle {}", circle_id_for_server_handle);
}
}
});
running_circles_store.push(Arc::new(Mutex::new(RunningCircleInfo {
config,
db_path: circle_db_path,
worker_queue: worker_queue_name,
ws_url,
worker_handle,
worker_shutdown_tx,
ws_server_instance_handle: server_instance_handle_arc,
ws_server_task_join_handle,
status: current_status, // This is an Arc<Mutex<String>>
})));
}
info!("All configured circles have been processed. Initializing status table display loop.");
let display_running_circles = running_circles_store.clone();
let display_task = tokio::spawn(async move {
loop {
{ // Scope for MutexGuard
let circles = display_running_circles.iter()
.map(|arc_info| arc_info.lock().unwrap())
.collect::<Vec<_>>(); // Collect locked guards
let mut table = Table::new();
table.set_content_arrangement(ContentArrangement::Dynamic);
table.set_header(vec!["Name", "ID", "Port", "Status", "DB Path", "Worker Queue", "WS URL"]);
for circle_info in circles.iter() {
let mut row = Row::new();
row.add_cell(Cell::new(&circle_info.config.name));
row.add_cell(Cell::new(circle_info.config.id));
row.add_cell(Cell::new(circle_info.config.port));
row.add_cell(Cell::new(&*circle_info.status.lock().unwrap())); // Deref and lock status
row.add_cell(Cell::new(circle_info.db_path.to_string_lossy()));
row.add_cell(Cell::new(&circle_info.worker_queue));
row.add_cell(Cell::new(&circle_info.ws_url));
table.add_row(row);
}
// Clear terminal before printing (basic, might flicker)
// print!("\x1B[2J\x1B[1;1H");
println!("\n--- Circles Status (updated every 5s, Ctrl+C to stop) ---\n{table}");
}
tokio::time::sleep(Duration::from_secs(5)).await;
}
});
signal::ctrl_c().await?;
info!("Ctrl-C received. Initiating graceful shutdown of all circles...");
display_task.abort(); // Stop the display task
for circle_arc in running_circles_store {
let mut circle_info = circle_arc.lock().unwrap();
info!("Shutting down Circle ID: {}, Name: '{}'", circle_info.config.id, circle_info.config.name);
*circle_info.status.lock().unwrap() = "Shutting down".to_string();
// Signal worker to shut down
if circle_info.worker_shutdown_tx.send(()).await.is_err() {
warn!("Failed to send shutdown signal to worker for Circle {}. It might have already stopped.", circle_info.config.id);
}
// Stop WS server
if let Some(server_handle) = circle_info.ws_server_instance_handle.lock().unwrap().take() {
info!("Stopping WebSocket server for Circle {}...", circle_info.config.id);
server_handle.stop(true).await; // Graceful stop
info!("WebSocket server for Circle {} stop signal sent.", circle_info.config.id);
} else {
warn!("No server handle to stop WebSocket server for Circle {}. It might not have started properly or already stopped.", circle_info.config.id);
}
}
info!("Waiting for all tasks to complete...");
for circle_arc in running_circles_store {
// We need to take ownership of handles to await them, or await mutable refs.
// This part is tricky if the MutexGuard is held.
// For simplicity, we'll just log that we've signaled them.
// Proper awaiting would require more careful structuring of JoinHandles.
let circle_id;
let circle_name;
{ // Short scope for the lock
let circle_info = circle_arc.lock().unwrap();
circle_id = circle_info.config.id;
circle_name = circle_info.config.name.clone();
}
debug!("Orchestrator has signaled shutdown for Circle {} ({}). Main loop will await join handles if structured for it.", circle_name, circle_id);
// Actual awaiting of join handles would happen here if they were collected outside the Mutex.
// For now, the main function will exit after this loop.
}
// Give some time for tasks to shut down before the main process exits.
// This is a simplified approach. A more robust solution would involve awaiting all JoinHandles.
tokio::time::sleep(Duration::from_secs(2)).await;
info!("Orchestrator shut down complete.");
Ok(())
}

6
content/intro.md Normal file
View File

@@ -0,0 +1,6 @@
Free your self from
Think outside the box
An internet built around ourselves

5
rhai_repl_cli/.rhai_repl_history.txt Normal file
View File

@@ -0,0 +1,5 @@
#V2
.edit
quit
.edit
exit

1752
rhai_repl_cli/Cargo.lock generated Normal file
View File

File diff suppressed because it is too large Load Diff

17
rhai_repl_cli/Cargo.toml Normal file
View File

@@ -0,0 +1,17 @@
[package]
name = "rhai_repl_cli"
version = "0.1.0"
edition = "2024" # Keep 2024 unless issues arise
[dependencies]
tokio = { version = "1", features = ["macros", "rt-multi-thread", "time"] } # Added "time" for potential timeouts
tokio-tungstenite = { version = "0.21", features = ["native-tls"] } # May be removed if client_ws handles all
futures-util = "0.3"
url = "2"
tracing = "0.1" # For logging
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
log = "0.4" # circle_client_ws uses log crate
rustyline = { version = "13.0.0", features = ["derive"] } # For enhanced REPL input
tempfile = "3.8" # For creating temporary files for editing
circle_client_ws = { path = "../client_ws" }

77
rhai_repl_cli/README.md Normal file
View File

@@ -0,0 +1,77 @@
# Rhai REPL CLI for Circle WebSocket Servers
This crate provides a command-line interface (CLI) to interact with Rhai scripts executed on remote Circle WebSocket servers. It includes both an interactive REPL and a non-interactive example.
## Prerequisites
1. **Circle Orchestrator Running**: Ensure the `circles_orchestrator` is running. This application manages and starts the individual Circle WebSocket servers.
To run the orchestrator:
```bash
cd /path/to/herocode/circles/cmd
cargo run
```
By default, this will start servers based on the `circles.json` configuration (e.g., "Alpha Circle" on `ws://127.0.0.1:8081/ws`).
2. **Redis Server**: Ensure a Redis server is running and accessible at `redis://127.0.0.1:6379` (this is the default used by the orchestrator and its components).
## Usage
Navigate to this crate's directory:
```bash
cd /path/to/herocode/circles/rhai_repl_cli
```
### 1. Interactive REPL
The main binary of this crate is an interactive REPL.
**To run with default WebSocket URL (`ws://127.0.0.1:8081/ws`):**
```bash
cargo run
```
**To specify a WebSocket URL:**
```bash
cargo run ws://<your-circle-server-ip>:<port>/ws
# Example for "Beta Circle" if configured on port 8082:
# cargo run ws://127.0.0.1:8082/ws
```
Once connected, you can:
- Type single-line Rhai scripts directly and press Enter.
- Use Vi keybindings for editing the current input line (thanks to `rustyline`).
- Type `.edit` to open your `$EDITOR` (or `vi` by default) for multi-line script input. Save and close the editor to execute the script.
- Type `.run <filepath>` (or `run <filepath>`) to execute a Rhai script from a local file.
- Type `exit` or `quit` to close the REPL.
Command history is saved to `.rhai_repl_history.txt` in the directory where you run the REPL.
### 2. Non-Interactive Example (`connect_and_play`)
This example connects to a WebSocket server, sends a predefined Rhai script, prints the response, and then disconnects.
**To run with default WebSocket URL (`ws://127.0.0.1:8081/ws`):**
```bash
cargo run --example connect_and_play
```
**To specify a WebSocket URL for the example:**
```bash
cargo run --example connect_and_play ws://<your-circle-server-ip>:<port>/ws
# Example:
# cargo run --example connect_and_play ws://127.0.0.1:8082/ws
```
The example script is:
```rhai
let a = 10;
let b = 32;
let message = "Hello from example script!";
message + " Result: " + (a + b)
```
## Logging
Both the REPL and the example use the `tracing` crate for logging. You can control log levels using the `RUST_LOG` environment variable. For example, to see debug logs from the `circle_client_ws` library:
```bash
RUST_LOG=info,circle_client_ws=debug cargo run --example connect_and_play

43
rhai_repl_cli/examples/connect_and_play.rs Normal file
View File

@@ -0,0 +1,43 @@
use circle_client_ws::CircleWsClient;
use std::env;
use tracing_subscriber::EnvFilter;
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
tracing_subscriber::fmt()
.with_env_filter(EnvFilter::from_default_env().add_directive("connect_and_play=info".parse().unwrap()).add_directive("circle_client_ws=info".parse().unwrap()))
.init();
let args: Vec<String> = env::args().collect();
let ws_url = args.get(1).cloned().unwrap_or_else(|| {
let default_url = "ws://127.0.0.1:8081/ws".to_string();
println!("No WebSocket URL provided. Defaulting to: {}", default_url);
default_url
});
println!("Attempting to connect to {}...", ws_url);
let mut client = CircleWsClient::new(ws_url.clone());
if let Err(e) = client.connect().await {
eprintln!("Failed to connect to {}: {}", ws_url, e);
return Err(e.into());
}
println!("Connected to {}!", ws_url);
let script = "let a = 10; let b = 32; let message = \"Hello from example script!\"; message + \" Result: \" + (a + b)";
println!("\nSending script:\n```rhai\n{}\n```", script);
match client.play(script.to_string()).await {
Ok(play_result) => {
println!("\nServer response:\nOutput: {}", play_result.output);
}
Err(e) => {
eprintln!("\nError executing script: {}", e);
}
}
client.disconnect().await;
println!("\nDisconnected from {}.", ws_url);
Ok(())
}

189
rhai_repl_cli/src/main.rs Normal file
View File

@@ -0,0 +1,189 @@
use url::Url;
use tracing_subscriber::EnvFilter;
use circle_client_ws::CircleWsClient;
use rustyline::error::ReadlineError;
// Remove direct History import, DefaultEditor handles it.
use rustyline::{DefaultEditor, Config, EditMode};
use std::fs;
use std::process::Command;
use std::env;
use tempfile::Builder as TempFileBuilder; // Use Builder for suffix
// std::io::Write is not used if we don't pre-populate temp_file
// use std::io::Write;
async fn execute_script(client: &mut CircleWsClient, script_content: String) {
if script_content.trim().is_empty() {
println!("Script is empty, not sending.");
return;
}
println!("Sending script to server:\n---\n{}\n---", script_content);
match client.play(script_content).await {
Ok(play_result) => {
println!("server: {}", play_result.output);
}
Err(e) => {
eprintln!("Error executing script: {}", e);
if matches!(e, circle_client_ws::CircleWsClientError::NotConnected | circle_client_ws::CircleWsClientError::ConnectionError(_)) {
eprintln!("Connection lost. You may need to restart the REPL and reconnect.");
// Optionally, could attempt to trigger a full exit here or set a flag
}
}
}
}
async fn run_repl(ws_url_str: String) -> Result<(), Box<dyn std::error::Error>> {
println!("Attempting to connect to {}...", ws_url_str);
let mut client = CircleWsClient::new(ws_url_str.clone());
match client.connect().await {
Ok(_) => {
println!("Connected to {}!", ws_url_str);
println!("Type Rhai scripts, '.edit' to use $EDITOR, '.run <path>' to execute a file, or 'exit'/'quit'.");
println!("Vi mode enabled for input line.");
}
Err(e) => {
return Err(format!("Failed to connect: {}", e).into());
}
}
let config = Config::builder()
.edit_mode(EditMode::Vi)
.auto_add_history(true) // Automatically add to history
.build();
let mut rl = DefaultEditor::with_config(config)?;
let history_file = ".rhai_repl_history.txt"; // Simple history file in current dir
if rl.load_history(history_file).is_err() {
// No history found or error loading, not critical
}
let prompt = format!("rhai ({})> ", ws_url_str);
loop {
let readline = rl.readline(&prompt);
match readline {
Ok(line) => {
let input = line.trim();
if input.eq_ignore_ascii_case("exit") || input.eq_ignore_ascii_case("quit") {
println!("Exiting REPL.");
break;
} else if input.eq_ignore_ascii_case(".edit") {
// Correct way to create a temp file with a suffix
let temp_file = TempFileBuilder::new()
.prefix("rhai_script_") // Optional: add a prefix
.suffix(".rhai")
.tempfile_in(".") // Create in current directory for simplicity
.map_err(|e| format!("Failed to create temp file: {}", e))?;
// You can pre-populate the temp file if needed:
// use std::io::Write; // Add this import if using write_all
// if let Err(e) = temp_file.as_file().write_all(b"// Start your Rhai script here\n") {
// eprintln!("Failed to write initial content to temp file: {}", e);
// }
let temp_path = temp_file.path().to_path_buf();
let editor_cmd_str = env::var("EDITOR").unwrap_or_else(|_| "vi".to_string());
let mut editor_parts = editor_cmd_str.split_whitespace();
let editor_executable = editor_parts.next().unwrap_or("vi"); // Default to vi if $EDITOR is empty string
let editor_args: Vec<&str> = editor_parts.collect();
println!("Launching editor: '{}' with args: {:?} for script editing. Save and exit editor to execute.", editor_executable, editor_args);
let mut command = Command::new(editor_executable);
command.args(editor_args); // Add any arguments from $EDITOR (like -w)
command.arg(&temp_path); // Add the temp file path as the last argument
let status = command.status();
match status {
Ok(exit_status) if exit_status.success() => {
match fs::read_to_string(&temp_path) {
Ok(script_content) => {
execute_script(&mut client, script_content).await;
}
Err(e) => eprintln!("Error reading temp file {:?}: {}", temp_path, e),
}
}
Ok(exit_status) => eprintln!("Editor exited with status: {}. Script not executed.", exit_status),
Err(e) => eprintln!("Failed to launch editor '{}': {}. Ensure it's in your PATH.", editor_executable, e), // Changed 'editor' to 'editor_executable'
}
// temp_file is automatically deleted when it goes out of scope
} else if input.starts_with(".run ") || input.starts_with("run ") {
let parts: Vec<&str> = input.splitn(2, ' ').collect();
if parts.len() == 2 {
let file_path = parts[1];
println!("Attempting to run script from file: {}", file_path);
match fs::read_to_string(file_path) {
Ok(script_content) => {
execute_script(&mut client, script_content).await;
}
Err(e) => eprintln!("Error reading file {}: {}", file_path, e),
}
} else {
eprintln!("Usage: .run <filepath>");
}
} else if !input.is_empty() {
execute_script(&mut client, input.to_string()).await;
}
// rl.add_history_entry(line.as_str()) is handled by auto_add_history(true)
}
Err(ReadlineError::Interrupted) => { // Ctrl-C
println!("Input interrupted. Type 'exit' or 'quit' to close.");
continue;
}
Err(ReadlineError::Eof) => { // Ctrl-D
println!("Exiting REPL (EOF).");
break;
}
Err(err) => {
eprintln!("Error reading input: {:?}", err);
break;
}
}
}
if rl.save_history(history_file).is_err() {
// Failed to save history, not critical
}
client.disconnect().await;
println!("Disconnected.");
Ok(())
}
#[tokio::main]
async fn main() {
tracing_subscriber::fmt()
.with_env_filter(EnvFilter::from_default_env().add_directive("rhai_repl_cli=info".parse().unwrap()).add_directive("circle_client_ws=info".parse().unwrap()))
.init();
let args: Vec<String> = env::args().collect();
let ws_url_str = if args.len() > 1 {
args[1].clone()
} else {
let default_url = "ws://127.0.0.1:8081/ws".to_string(); // Default to first circle
println!("No WebSocket URL provided. Defaulting to: {}", default_url);
println!("You can also provide a URL as a command line argument, e.g.: cargo run ws://127.0.0.1:8082/ws");
default_url
};
match Url::parse(&ws_url_str) {
Ok(parsed_url) => {
if parsed_url.scheme() != "ws" && parsed_url.scheme() != "wss" {
eprintln!("Invalid WebSocket URL scheme: {}. Must be 'ws' or 'wss'.", parsed_url.scheme());
return;
}
if let Err(e) = run_repl(ws_url_str).await { // Pass the original string URL
eprintln!("REPL error: {}", e);
}
}
Err(e) => {
eprintln!("Invalid WebSocket URL format '{}': {}", ws_url_str, e);
}
}
}

20
server_ws/Cargo.toml
View File

@@ -1,28 +1,32 @@
[package]
name = "circle_server_ws"
name = "circle_ws_lib" # Renamed to reflect library nature
version = "0.1.0"
edition = "2021"
[lib]
name = "circle_ws_lib"
path = "src/lib.rs"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
actix-web = "4"
actix-web-actors = "4"
actix = "0.13"
env_logger = "0.10"
env_logger = "0.10" # Keep for logging within the lib
log = "0.4"
clap = { version = "4.4", features = ["derive"] }
# clap is removed as CLI parsing moves to the orchestrator bin
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
redis = { version = "0.25.0", features = ["tokio-comp"] } # For async Redis with Actix
uuid = { version = "1.6", features = ["v4", "serde"] }
tokio = { version = "1", features = ["macros", "rt-multi-thread"] } # For polling interval
uuid = { version = "1.6", features = ["v4", "serde"] } # Still used by RhaiClient or for task details
tokio = { version = "1", features = ["macros", "rt-multi-thread", "time"] } # Added "time" for Duration
chrono = { version = "0.4", features = ["serde"] } # For timestamps
rhai_client = { path = "/Users/timurgordon/code/git.ourworld.tf/herocode/rhaj/src/client" }
rhai_client = { path = "../../rhailib/src/client" } # Corrected relative path
[dev-dependencies]
tokio-tungstenite = { version = "0.23.0", features = ["native-tls"] }
futures-util = "0.3" # For StreamExt and SinkExt on WebSocket stream
url = "2.5.0" # For parsing WebSocket URL
circle_client_ws = { path = "../client_ws" }
uuid = { version = "1.6", features = ["v4", "serde"] } # For e2e example, if it still uses Uuid directly for req id
# circle_client_ws = { path = "../client_ws" } # This might need adjustment if it's a test client for the old binary
# uuid = { version = "1.6", features = ["v4", "serde"] } # Already in dependencies

302
server_ws/src/lib.rs Normal file
View File

@@ -0,0 +1,302 @@
use actix_web::{web, App, HttpRequest, HttpServer, HttpResponse, Error};
use actix_web_actors::ws;
use actix::{Actor, ActorContext, StreamHandler, AsyncContext, WrapFuture, ActorFutureExt};
// clap::Parser removed
use serde::{Deserialize, Serialize};
use serde_json::Value;
use std::time::Duration;
use rhai_client::RhaiClientError;
use rhai_client::RhaiClient;
use tokio::task::JoinHandle;
use tokio::sync::oneshot; // For sending the server handle back
// Newtype wrappers for distinct app_data types
#[derive(Clone)]
struct AppCircleName(String);
#[derive(Clone)]
struct AppRedisUrl(String);
// JSON-RPC 2.0 Structures (remain the same)
#[derive(Serialize, Deserialize, Debug, Clone)]
struct JsonRpcRequest {
jsonrpc: String,
method: String,
params: Value,
id: Option<Value>,
}
#[derive(Serialize, Deserialize, Debug, Clone)]
struct JsonRpcResponse {
jsonrpc: String,
result: Option<Value>,
error: Option<JsonRpcError>,
id: Value,
}
#[derive(Serialize, Deserialize, Debug, Clone)]
struct JsonRpcError {
code: i32,
message: String,
data: Option<Value>,
}
#[derive(Serialize, Deserialize, Debug, Clone)]
struct PlayParams {
script: String,
}
#[derive(Serialize, Deserialize, Debug, Clone)]
struct PlayResult {
output: String,
}
// WebSocket Actor
struct CircleWs {
server_circle_name: String,
redis_url_for_client: String,
}
const TASK_TIMEOUT_DURATION: Duration = Duration::from_secs(30);
const TASK_POLL_INTERVAL_DURATION: Duration = Duration::from_millis(200);
impl CircleWs {
fn new(name: String, redis_url: String) -> Self {
Self {
server_circle_name: name,
redis_url_for_client: redis_url,
}
}
}
impl Actor for CircleWs {
type Context = ws::WebsocketContext<Self>;
fn started(&mut self, _ctx: &mut Self::Context) {
log::info!("WebSocket session started for server dedicated to: {}", self.server_circle_name);
}
fn stopping(&mut self, _ctx: &mut Self::Context) -> actix::Running {
log::info!("WebSocket session stopping for server dedicated to: {}", self.server_circle_name);
actix::Running::Stop
}
}
impl StreamHandler<Result<ws::Message, ws::ProtocolError>> for CircleWs {
fn handle(&mut self, msg: Result<ws::Message, ws::ProtocolError>, ctx: &mut Self::Context) {
match msg {
Ok(ws::Message::Text(text)) => {
log::debug!("WS Text for {}: {}", self.server_circle_name, text); // Changed to debug for less noise
match serde_json::from_str::<JsonRpcRequest>(&text) {
Ok(req) => {
let client_rpc_id = req.id.clone().unwrap_or(Value::Null);
if req.method == "play" {
match serde_json::from_value::<PlayParams>(req.params.clone()) {
Ok(play_params) => {
let script_content = play_params.script;
// Use the server_circle_name which should be correctly set now
let current_circle_name_for_rhai_client = self.server_circle_name.clone();
let rpc_id_for_client = client_rpc_id.clone();
let redis_url_clone = self.redis_url_for_client.clone();
log::info!("Circle '{}' WS: Received 'play' request, ID: {:?}, for RhaiClient target circle: '{}'", self.server_circle_name, rpc_id_for_client, current_circle_name_for_rhai_client);
let fut = async move {
match RhaiClient::new(&redis_url_clone) {
Ok(rhai_task_client) => {
rhai_task_client.submit_script_and_await_result(
&current_circle_name_for_rhai_client, // This name is used for Redis queue
script_content,
Some(rpc_id_for_client.clone()),
TASK_TIMEOUT_DURATION,
TASK_POLL_INTERVAL_DURATION,
).await
}
Err(e) => {
log::error!("Circle '{}' WS: Failed to create RhaiClient for Redis URL {}: {}", current_circle_name_for_rhai_client, redis_url_clone, e);
Err(e)
}
}
};
ctx.spawn(fut.into_actor(self).map(move |result, _act, ws_ctx| {
let response = match result {
Ok(task_details) => {
if task_details.status == "completed" {
// task_details itself doesn't have a task_id field.
// The task_id is known by the client that initiated the poll.
// We log with client_rpc_id which is the JSON-RPC request ID.
log::info!("Circle '{}' WS: Request ID {:?} completed successfully. Output: {:?}", _act.server_circle_name, client_rpc_id, task_details.output);
JsonRpcResponse {
jsonrpc: "2.0".to_string(),
result: Some(serde_json::to_value(PlayResult {
output: task_details.output.unwrap_or_default()
}).unwrap()),
error: None,
id: client_rpc_id,
}
} else { // status == "error"
log::warn!("Circle '{}' WS: Request ID {:?} execution failed. Error: {:?}", _act.server_circle_name, client_rpc_id, task_details.error);
JsonRpcResponse {
jsonrpc: "2.0".to_string(),
result: None,
error: Some(JsonRpcError {
code: -32004,
message: task_details.error.unwrap_or_else(|| "Script execution failed".to_string()),
data: None,
}),
id: client_rpc_id,
}
}
}
Err(rhai_err) => {
log::error!("Circle '{}' WS: RhaiClient operation failed for req ID {:?}: {}", _act.server_circle_name, client_rpc_id, rhai_err);
let (code, message) = match rhai_err {
RhaiClientError::Timeout(task_id) => (-32002, format!("Timeout: {}", task_id)),
RhaiClientError::RedisError(e) => (-32003, format!("Redis error: {}", e)),
RhaiClientError::SerializationError(e) => (-32003, format!("Serialization error: {}", e)),
RhaiClientError::TaskNotFound(task_id) => (-32005, format!("Task not found: {}", task_id)),
};
JsonRpcResponse {
jsonrpc: "2.0".to_string(),
result: None,
id: client_rpc_id,
error: Some(JsonRpcError { code, message, data: None }),
}
}
};
ws_ctx.text(serde_json::to_string(&response).unwrap());
}));
}
Err(e) => {
log::error!("Circle '{}' WS: Invalid params for 'play' method: {}", self.server_circle_name, e);
let err_resp = JsonRpcResponse {
jsonrpc: "2.0".to_string(), result: None, id: client_rpc_id,
error: Some(JsonRpcError { code: -32602, message: "Invalid params".to_string(), data: Some(Value::String(e.to_string())) }),
};
ctx.text(serde_json::to_string(&err_resp).unwrap());
}
}
} else {
log::warn!("Circle '{}' WS: Method not found: {}", self.server_circle_name, req.method);
let err_resp = JsonRpcResponse {
jsonrpc: "2.0".to_string(), result: None, id: client_rpc_id,
error: Some(JsonRpcError { code: -32601, message: "Method not found".to_string(), data: None }),
};
ctx.text(serde_json::to_string(&err_resp).unwrap());
}
}
Err(e) => {
log::error!("Circle '{}' WS: Failed to parse JSON-RPC request: {}", self.server_circle_name, e);
let err_resp = JsonRpcResponse {
jsonrpc: "2.0".to_string(), result: None, id: Value::Null,
error: Some(JsonRpcError { code: -32700, message: "Parse error".to_string(), data: Some(Value::String(e.to_string())) }),
};
ctx.text(serde_json::to_string(&err_resp).unwrap());
}
}
}
Ok(ws::Message::Ping(msg)) => ctx.pong(&msg),
Ok(ws::Message::Pong(_)) => {},
Ok(ws::Message::Binary(_bin)) => log::warn!("Circle '{}' WS: Binary messages not supported.", self.server_circle_name),
Ok(ws::Message::Close(reason)) => {
log::info!("Circle '{}' WS: Close message received. Reason: {:?}", self.server_circle_name, reason);
ctx.close(reason);
ctx.stop();
}
Ok(ws::Message::Continuation(_)) => ctx.stop(),
Ok(ws::Message::Nop) => (),
Err(e) => {
log::error!("Circle '{}' WS: Error: {:?}", self.server_circle_name, e);
ctx.stop();
}
}
}
}
// Modified ws_handler to accept newtype wrapped app_data
async fn ws_handler_modified(
req: HttpRequest,
stream: web::Payload,
app_circle_name: web::Data<AppCircleName>, // Use wrapped type
app_redis_url: web::Data<AppRedisUrl>, // Use wrapped type
) -> Result<HttpResponse, Error> {
let circle_name_str = app_circle_name.0.clone();
let redis_url_str = app_redis_url.0.clone();
log::info!("WebSocket handshake attempt for server: '{}' with redis: '{}'", circle_name_str, redis_url_str);
let resp = ws::start(
CircleWs::new(circle_name_str, redis_url_str), // Pass unwrapped strings
&req,
stream
)?;
Ok(resp)
}
// Public factory function to spawn the server
pub fn spawn_circle_ws_server(
_circle_id: u32,
circle_name: String,
port: u16,
redis_url: String,
// Sender to send the server handle back to the orchestrator
server_handle_tx: oneshot::Sender<actix_web::dev::Server>,
) -> JoinHandle<std::io::Result<()>> {
let circle_name_for_log = circle_name.clone();
// redis_url_for_log is not used, but kept for consistency if needed later
tokio::spawn(async move {
let circle_name_outer = circle_name;
let redis_url_outer = redis_url;
let app_factory = move || {
App::new()
.app_data(web::Data::new(AppCircleName(circle_name_outer.clone())))
.app_data(web::Data::new(AppRedisUrl(redis_url_outer.clone())))
.route("/ws", web::get().to(ws_handler_modified))
.default_service(web::route().to(|| async { HttpResponse::NotFound().body("404 Not Found") }))
};
let server_builder = HttpServer::new(app_factory);
let bound_server = match server_builder.bind(("127.0.0.1", port)) {
Ok(srv) => {
log::info!(
"Successfully bound WebSocket server for Circle: '{}' on port {}. Starting...",
circle_name_for_log, port
);
srv
}
Err(e) => {
log::error!(
"Failed to bind WebSocket server for Circle '{}' on port {}: {}",
circle_name_for_log, port, e
);
// If binding fails, we can't send a server handle.
// The orchestrator will see the JoinHandle error out or the oneshot::Sender drop.
return Err(e);
}
};
let server_runnable: actix_web::dev::Server = bound_server.run();
// Send the server handle back to the orchestrator
if server_handle_tx.send(server_runnable.clone()).is_err() {
log::error!(
"Failed to send server handle back to orchestrator for Circle '{}'. Orchestrator might have shut down.",
circle_name_for_log
);
// Server might still run, but orchestrator can't stop it gracefully via this handle.
// Consider stopping it here if sending the handle is critical.
// For now, let it proceed, but log the error.
}
// Now await the server_runnable (which is the Server handle itself)
if let Err(e) = server_runnable.await {
log::error!("WebSocket server for Circle '{}' on port {} failed during run: {}", circle_name_for_log, port, e);
return Err(e);
}
log::info!("WebSocket server for Circle '{}' on port {} shut down gracefully.", circle_name_for_log, port);
Ok(())
})
}