rename worker to actor
This commit is contained in:
BIN
core/actor/.DS_Store
vendored
Normal file
BIN
core/actor/.DS_Store
vendored
Normal file
Binary file not shown.
2
core/actor/.gitignore
vendored
Normal file
2
core/actor/.gitignore
vendored
Normal file
@@ -0,0 +1,2 @@
|
||||
/target
|
||||
actor_rhai_temp_db
|
1423
core/actor/Cargo.lock
generated
Normal file
1423
core/actor/Cargo.lock
generated
Normal file
File diff suppressed because it is too large
Load Diff
40
core/actor/Cargo.toml
Normal file
40
core/actor/Cargo.toml
Normal file
@@ -0,0 +1,40 @@
|
||||
[package]
|
||||
name = "baobab_actor"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
|
||||
[lib]
|
||||
name = "baobab_actor" # Can be different from package name, or same
|
||||
path = "src/lib.rs"
|
||||
|
||||
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
|
||||
|
||||
[dependencies]
|
||||
redis = { version = "0.25.0", features = ["tokio-comp"] }
|
||||
rhai = { version = "1.21.0", features = ["std", "sync", "decimal", "internals"] }
|
||||
serde = { version = "1.0", features = ["derive"] }
|
||||
serde_json = "1.0"
|
||||
tokio = { version = "1", features = ["macros", "rt-multi-thread", "time"] }
|
||||
log = "0.4"
|
||||
env_logger = "0.10"
|
||||
clap = { version = "4.4", features = ["derive"] }
|
||||
uuid = { version = "1.6", features = ["v4", "serde"] } # Though task_id is string, uuid might be useful
|
||||
chrono = { version = "0.4", features = ["serde"] }
|
||||
toml = "0.8"
|
||||
thiserror = "1.0"
|
||||
async-trait = "0.1"
|
||||
hero_supervisor = { path = "../supervisor" }
|
||||
hero_job = { path = "../job" }
|
||||
# heromodels = { path = "../../../db/heromodels", features = ["rhai"] }
|
||||
heromodels = { git = "https://git.ourworld.tf/herocode/db.git" }
|
||||
heromodels_core = { git = "https://git.ourworld.tf/herocode/db.git" }
|
||||
heromodels-derive = { git = "https://git.ourworld.tf/herocode/db.git" }
|
||||
|
||||
[features]
|
||||
default = ["calendar", "finance"]
|
||||
calendar = []
|
||||
finance = []
|
||||
flow = []
|
||||
legal = []
|
||||
projects = []
|
||||
biz = []
|
75
core/actor/README.md
Normal file
75
core/actor/README.md
Normal file
@@ -0,0 +1,75 @@
|
||||
# Actor
|
||||
|
||||
The `actor` crate defines the trait `Actor` is
|
||||
|
||||
and implements a standalone actor service that listens for Rhai script execution tasks from a Redis queue, executes them, and posts results back to Redis. It is designed to be spawned as a separate OS process by an orchestrator like the `launcher` crate.
|
||||
|
||||
## Features
|
||||
|
||||
- **Redis Queue Consumption**: Listens to a specific Redis list (acting as a task queue) for incoming task IDs. The queue is determined by the `--circle-public-key` argument.
|
||||
- **Rhai Script Execution**: Executes Rhai scripts retrieved from Redis based on task IDs.
|
||||
- **Task State Management**: Updates task status (`processing`, `completed`, `error`) and stores results in Redis hashes.
|
||||
- **Script Scope Injection**: Automatically injects two important constants into the Rhai script's scope:
|
||||
- `CONTEXT_ID`: The public key of the actor's own circle.
|
||||
- `CALLER_ID`: The public key of the entity that requested the script execution.
|
||||
- **Asynchronous Operations**: Built with `tokio` for non-blocking Redis communication.
|
||||
- **Graceful Error Handling**: Captures errors during script execution and stores them for the client.
|
||||
|
||||
## Core Components
|
||||
|
||||
- **`actor_lib` (Library Crate)**:
|
||||
- **`Args`**: A struct (using `clap`) for parsing command-line arguments: `--redis-url` and `--circle-public-key`.
|
||||
- **`run_actor_loop(engine: Engine, args: Args)`**: The main asynchronous function that:
|
||||
- Connects to Redis.
|
||||
- Continuously polls the designated Redis queue (`rhai_tasks:<circle_public_key>`) using `BLPOP`.
|
||||
- Upon receiving a `task_id`, it fetches the task details from a Redis hash.
|
||||
- It injects `CALLER_ID` and `CONTEXT_ID` into the script's scope.
|
||||
- It executes the script and updates the task status in Redis with the output or error.
|
||||
|
||||
## How It Works
|
||||
|
||||
1. The actor executable is launched by an external process (e.g., `launcher`), which passes the required command-line arguments.
|
||||
```bash
|
||||
# This is typically done programmatically by a parent process.
|
||||
/path/to/actor --redis-url redis://127.0.0.1/ --circle-public-key 02...abc
|
||||
```
|
||||
2. The `run_actor_loop` connects to Redis and starts listening to its designated task queue (e.g., `rhai_tasks:02...abc`).
|
||||
3. A `rhai_supervisor` submits a task by pushing a `task_id` to this queue and storing the script and other details in a Redis hash.
|
||||
4. The actor's `BLPOP` command picks up the `task_id`.
|
||||
5. The actor retrieves the script from the corresponding `rhai_task_details:<task_id>` hash.
|
||||
6. It updates the task's status to "processing".
|
||||
7. The Rhai script is executed within a scope that contains both `CONTEXT_ID` and `CALLER_ID`.
|
||||
8. After execution, the status is updated to "completed" (with output) or "error" (with an error message).
|
||||
9. The actor then goes back to listening for the next task.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- A running Redis instance accessible by the actor.
|
||||
- An orchestrator process (like `launcher`) to spawn the actor.
|
||||
- A `rhai_supervisor` (or another system) to populate the Redis queues.
|
||||
|
||||
## Building and Running
|
||||
|
||||
The actor is intended to be built as a dependency and run by another program.
|
||||
|
||||
1. **Build the actor:**
|
||||
```bash
|
||||
# From the root of the baobab project
|
||||
cargo build --package actor
|
||||
```
|
||||
The binary will be located at `target/debug/actor`.
|
||||
|
||||
2. **Running the actor:**
|
||||
The actor is not typically run manually. The `launcher` crate is responsible for spawning it with the correct arguments. If you need to run it manually for testing, you must provide the required arguments:
|
||||
```bash
|
||||
./target/debug/actor --redis-url redis://127.0.0.1/ --circle-public-key <a_valid_hex_public_key>
|
||||
```
|
||||
|
||||
## Dependencies
|
||||
|
||||
Key dependencies include:
|
||||
- `redis`: For asynchronous Redis communication.
|
||||
- `rhai`: The Rhai script engine.
|
||||
- `clap`: For command-line argument parsing.
|
||||
- `tokio`: For the asynchronous runtime.
|
||||
- `log`, `env_logger`: For logging.
|
53
core/actor/docs/ARCHITECTURE.md
Normal file
53
core/actor/docs/ARCHITECTURE.md
Normal file
@@ -0,0 +1,53 @@
|
||||
# Architecture of the `baobab_actor` Crate
|
||||
|
||||
The `baobab_actor` crate implements a distributed task execution system for Rhai scripts, providing scalable, reliable script processing through Redis-based task queues. Actors are decoupled from contexts, allowing a single actor to process tasks for multiple contexts (circles).
|
||||
|
||||
## Core Architecture
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
A[Actor Process] --> B[Task Queue Processing]
|
||||
A --> C[Script Execution Engine]
|
||||
A --> D[Result Management]
|
||||
|
||||
B --> B1[Redis Queue Monitoring]
|
||||
B --> B2[Task Deserialization]
|
||||
B --> B3[Priority Handling]
|
||||
|
||||
C --> C1[Rhai Engine Integration]
|
||||
C --> C2[Context Management]
|
||||
C --> C3[Error Handling]
|
||||
|
||||
D --> D1[Result Serialization]
|
||||
D --> D2[Reply Queue Management]
|
||||
D --> D3[Status Updates]
|
||||
```
|
||||
|
||||
## Key Components
|
||||
|
||||
### Task Processing Pipeline
|
||||
- **Queue Monitoring**: Continuous Redis queue polling for new tasks
|
||||
- **Task Execution**: Secure Rhai script execution with proper context
|
||||
- **Result Handling**: Comprehensive result and error management
|
||||
|
||||
### Engine Integration
|
||||
- **baobab Engine**: Full integration with baobab_engine for DSL access
|
||||
- **Context Injection**: Proper authentication and database context setup
|
||||
- **Security**: Isolated execution environment with access controls
|
||||
|
||||
### Scalability Features
|
||||
- **Horizontal Scaling**: Multiple actor instances for load distribution
|
||||
- **Queue-based Architecture**: Reliable task distribution via Redis
|
||||
- **Fault Tolerance**: Robust error handling and recovery mechanisms
|
||||
|
||||
## Dependencies
|
||||
|
||||
- **Redis Integration**: Task queue management and communication
|
||||
- **Rhai Engine**: Script execution with full DSL capabilities
|
||||
- **Client Integration**: Shared data structures with rhai_supervisor
|
||||
- **Heromodels**: Database and business logic integration
|
||||
- **Async Runtime**: Tokio for high-performance concurrent processing
|
||||
|
||||
## Deployment Patterns
|
||||
|
||||
Actors can be deployed as standalone processes, containerized services, or embedded components, providing flexibility for various deployment scenarios from development to production.
|
331
core/actor/src/actor_trait.rs
Normal file
331
core/actor/src/actor_trait.rs
Normal file
@@ -0,0 +1,331 @@
|
||||
//! # Actor Trait Abstraction
|
||||
//!
|
||||
//! This module provides a trait-based abstraction for Rhai actors that eliminates
|
||||
//! code duplication between synchronous and asynchronous actor implementations.
|
||||
//!
|
||||
//! The `Actor` trait defines the common interface and behavior, while specific
|
||||
//! implementations handle job processing differently (sync vs async).
|
||||
//!
|
||||
//! ## Architecture
|
||||
//!
|
||||
//! ```text
|
||||
//! ┌─────────────────┐ ┌─────────────────┐
|
||||
//! │ SyncActor │ │ AsyncActor │
|
||||
//! │ │ │ │
|
||||
//! │ process_job() │ │ process_job() │
|
||||
//! │ (sequential) │ │ (concurrent) │
|
||||
//! └─────────────────┘ └─────────────────┘
|
||||
//! │ │
|
||||
//! └───────┬───────────────┘
|
||||
//! │
|
||||
//! ┌───────▼───────┐
|
||||
//! │ Actor Trait │
|
||||
//! │ │
|
||||
//! │ spawn() │
|
||||
//! │ config │
|
||||
//! │ common loop │
|
||||
//! └───────────────┘
|
||||
//! ```
|
||||
|
||||
use hero_job::Job;
|
||||
use log::{debug, error, info};
|
||||
use redis::AsyncCommands;
|
||||
use rhai::Engine;
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
use tokio::sync::mpsc;
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
use crate::{initialize_redis_connection, NAMESPACE_PREFIX, BLPOP_TIMEOUT_SECONDS};
|
||||
|
||||
/// Configuration for actor instances
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ActorConfig {
|
||||
pub actor_id: String,
|
||||
pub db_path: String,
|
||||
pub redis_url: String,
|
||||
pub preserve_tasks: bool,
|
||||
pub default_timeout: Option<Duration>, // Only used by async actors
|
||||
}
|
||||
|
||||
impl ActorConfig {
|
||||
/// Create a new actor configuration
|
||||
pub fn new(
|
||||
actor_id: String,
|
||||
db_path: String,
|
||||
redis_url: String,
|
||||
preserve_tasks: bool,
|
||||
) -> Self {
|
||||
Self {
|
||||
actor_id,
|
||||
db_path,
|
||||
redis_url,
|
||||
preserve_tasks,
|
||||
default_timeout: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Set default timeout for async actors
|
||||
pub fn with_default_timeout(mut self, timeout: Duration) -> Self {
|
||||
self.default_timeout = Some(timeout);
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
/// Trait defining the common interface for Rhai actors
|
||||
///
|
||||
/// This trait abstracts the common functionality between synchronous and
|
||||
/// asynchronous actors, allowing them to share the same spawn logic and
|
||||
/// Redis polling loop while implementing different job processing strategies.
|
||||
#[async_trait::async_trait]
|
||||
pub trait Actor: Send + Sync + 'static {
|
||||
/// Process a single job
|
||||
///
|
||||
/// This is the core method that differentiates actor implementations:
|
||||
/// - Sync actors process jobs sequentially, one at a time
|
||||
/// - Async actors spawn concurrent tasks for each job
|
||||
///
|
||||
/// # Arguments
|
||||
///
|
||||
/// * `job` - The job to process
|
||||
/// * `redis_conn` - Redis connection for status updates
|
||||
///
|
||||
/// Note: The engine is now owned by the actor implementation as a field
|
||||
async fn process_job(
|
||||
&self,
|
||||
job: Job,
|
||||
redis_conn: &mut redis::aio::MultiplexedConnection,
|
||||
);
|
||||
|
||||
/// Get the actor type name for logging
|
||||
fn actor_type(&self) -> &'static str;
|
||||
|
||||
/// Get actor ID for this actor instance
|
||||
fn actor_id(&self) -> &str;
|
||||
|
||||
/// Get Redis URL for this actor instance
|
||||
fn redis_url(&self) -> &str;
|
||||
|
||||
/// Spawn the actor
|
||||
///
|
||||
/// This method provides the common actor loop implementation that both
|
||||
/// sync and async actors can use. It handles:
|
||||
/// - Redis connection setup
|
||||
/// - Job polling from Redis queue
|
||||
/// - Shutdown signal handling
|
||||
/// - Delegating job processing to the implementation
|
||||
///
|
||||
/// Note: The engine is now owned by the actor implementation as a field
|
||||
fn spawn(
|
||||
self: Arc<Self>,
|
||||
mut shutdown_rx: mpsc::Receiver<()>,
|
||||
) -> JoinHandle<Result<(), Box<dyn std::error::Error + Send + Sync>>> {
|
||||
tokio::spawn(async move {
|
||||
let actor_id = self.actor_id();
|
||||
let redis_url = self.redis_url();
|
||||
let queue_key = format!("{}{}", NAMESPACE_PREFIX, actor_id);
|
||||
info!(
|
||||
"{} Actor '{}' starting. Connecting to Redis at {}. Listening on queue: {}",
|
||||
self.actor_type(),
|
||||
actor_id,
|
||||
redis_url,
|
||||
queue_key
|
||||
);
|
||||
|
||||
let mut redis_conn = initialize_redis_connection(actor_id, redis_url).await?;
|
||||
|
||||
loop {
|
||||
let blpop_keys = vec![queue_key.clone()];
|
||||
tokio::select! {
|
||||
// Listen for shutdown signal
|
||||
_ = shutdown_rx.recv() => {
|
||||
info!("{} Actor '{}': Shutdown signal received. Terminating loop.",
|
||||
self.actor_type(), actor_id);
|
||||
break;
|
||||
}
|
||||
// Listen for tasks from Redis
|
||||
blpop_result = redis_conn.blpop(&blpop_keys, BLPOP_TIMEOUT_SECONDS as f64) => {
|
||||
debug!("{} Actor '{}': Attempting BLPOP on queue: {}",
|
||||
self.actor_type(), actor_id, queue_key);
|
||||
|
||||
let response: Option<(String, String)> = match blpop_result {
|
||||
Ok(resp) => resp,
|
||||
Err(e) => {
|
||||
error!("{} Actor '{}': Redis BLPOP error on queue {}: {}. Actor for this circle might stop.",
|
||||
self.actor_type(), actor_id, queue_key, e);
|
||||
return Err(Box::new(e) as Box<dyn std::error::Error + Send + Sync>);
|
||||
}
|
||||
};
|
||||
|
||||
if let Some((_queue_name_recv, job_id)) = response {
|
||||
info!("{} Actor '{}' received job_id: {} from queue: {}",
|
||||
self.actor_type(), actor_id, job_id, _queue_name_recv);
|
||||
|
||||
// Load the job from Redis
|
||||
match crate::load_job_from_redis(&mut redis_conn, &job_id, actor_id).await {
|
||||
Ok(job) => {
|
||||
// Check for ping job and handle it directly
|
||||
if job.script.trim() == "ping" {
|
||||
info!("{} Actor '{}': Received ping job '{}', responding with pong",
|
||||
self.actor_type(), actor_id, job_id);
|
||||
|
||||
// Update job status to started
|
||||
if let Err(e) = hero_job::Job::update_status(&mut redis_conn, &job_id, hero_job::JobStatus::Started).await {
|
||||
error!("{} Actor '{}': Failed to update ping job '{}' status to Started: {}",
|
||||
self.actor_type(), actor_id, job_id, e);
|
||||
}
|
||||
|
||||
// Set result to "pong" and mark as finished
|
||||
if let Err(e) = hero_job::Job::set_result(&mut redis_conn, &job_id, "pong").await {
|
||||
error!("{} Actor '{}': Failed to set ping job '{}' result: {}",
|
||||
self.actor_type(), actor_id, job_id, e);
|
||||
}
|
||||
|
||||
info!("{} Actor '{}': Successfully responded to ping job '{}' with pong",
|
||||
self.actor_type(), actor_id, job_id);
|
||||
} else {
|
||||
// Delegate job processing to the implementation
|
||||
// The engine is now owned by the actor implementation
|
||||
self.process_job(job, &mut redis_conn).await;
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
error!("{} Actor '{}': Failed to load job '{}': {}",
|
||||
self.actor_type(), actor_id, job_id, e);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
debug!("{} Actor '{}': BLPOP timed out on queue {}. No new tasks.",
|
||||
self.actor_type(), actor_id, queue_key);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
info!("{} Actor '{}' has shut down.", self.actor_type(), actor_id);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Convenience function to spawn a actor with the trait-based interface
|
||||
///
|
||||
/// This function provides a unified interface for spawning any actor implementation
|
||||
/// that implements the Actor trait.
|
||||
///
|
||||
/// # Arguments
|
||||
///
|
||||
/// * `actor` - The actor implementation to spawn
|
||||
/// * `config` - Actor configuration
|
||||
/// * `engine` - Rhai engine for script execution
|
||||
/// * `shutdown_rx` - Channel receiver for shutdown signals
|
||||
///
|
||||
/// # Returns
|
||||
///
|
||||
/// Returns a `JoinHandle` that can be awaited to wait for actor shutdown.
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// use std::sync::Arc;
|
||||
/// use std::time::Duration;
|
||||
///
|
||||
/// let config = ActorConfig::new(
|
||||
/// "actor_1".to_string(),
|
||||
/// "/path/to/db".to_string(),
|
||||
/// "redis://localhost:6379".to_string(),
|
||||
/// false,
|
||||
/// );
|
||||
///
|
||||
/// let actor = Arc::new(SyncActor::new());
|
||||
/// let engine = create_heromodels_engine();
|
||||
/// let (shutdown_tx, shutdown_rx) = mpsc::channel(1);
|
||||
///
|
||||
/// let handle = spawn_actor(actor, config, engine, shutdown_rx);
|
||||
///
|
||||
/// // Later, shutdown the actor
|
||||
/// shutdown_tx.send(()).await.unwrap();
|
||||
/// handle.await.unwrap().unwrap();
|
||||
/// ```
|
||||
pub fn spawn_actor<W: Actor>(
|
||||
actor: Arc<W>,
|
||||
shutdown_rx: mpsc::Receiver<()>,
|
||||
) -> JoinHandle<Result<(), Box<dyn std::error::Error + Send + Sync>>> {
|
||||
actor.spawn(shutdown_rx)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::engine::create_heromodels_engine;
|
||||
|
||||
// Mock actor for testing
|
||||
struct MockActor;
|
||||
|
||||
#[async_trait::async_trait]
|
||||
impl Actor for MockActor {
|
||||
async fn process_job(
|
||||
&self,
|
||||
_job: Job,
|
||||
_redis_conn: &mut redis::aio::MultiplexedConnection,
|
||||
) {
|
||||
// Mock implementation - do nothing
|
||||
// Engine would be owned by the actor implementation as a field
|
||||
}
|
||||
|
||||
fn actor_type(&self) -> &'static str {
|
||||
"Mock"
|
||||
}
|
||||
|
||||
fn actor_id(&self) -> &str {
|
||||
"mock_actor"
|
||||
}
|
||||
|
||||
fn redis_url(&self) -> &str {
|
||||
"redis://localhost:6379"
|
||||
}
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_actor_config_creation() {
|
||||
let config = ActorConfig::new(
|
||||
"test_actor".to_string(),
|
||||
"/tmp".to_string(),
|
||||
"redis://localhost:6379".to_string(),
|
||||
false,
|
||||
);
|
||||
|
||||
assert_eq!(config.actor_id, "test_actor");
|
||||
assert_eq!(config.db_path, "/tmp");
|
||||
assert_eq!(config.redis_url, "redis://localhost:6379");
|
||||
assert!(!config.preserve_tasks);
|
||||
assert!(config.default_timeout.is_none());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_actor_config_with_timeout() {
|
||||
let timeout = Duration::from_secs(300);
|
||||
let config = ActorConfig::new(
|
||||
"test_actor".to_string(),
|
||||
"/tmp".to_string(),
|
||||
"redis://localhost:6379".to_string(),
|
||||
false,
|
||||
).with_default_timeout(timeout);
|
||||
|
||||
assert_eq!(config.default_timeout, Some(timeout));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn test_spawn_actor_function() {
|
||||
let (_shutdown_tx, shutdown_rx) = mpsc::channel(1);
|
||||
let actor = Arc::new(MockActor);
|
||||
|
||||
let handle = spawn_actor(actor, shutdown_rx);
|
||||
|
||||
// The actor should be created successfully
|
||||
assert!(!handle.is_finished());
|
||||
|
||||
// Abort the actor for cleanup
|
||||
handle.abort();
|
||||
}
|
||||
}
|
238
core/actor/src/lib.rs
Normal file
238
core/actor/src/lib.rs
Normal file
@@ -0,0 +1,238 @@
|
||||
use hero_job::{Job, JobStatus};
|
||||
use log::{debug, error, info};
|
||||
use redis::AsyncCommands;
|
||||
use rhai::{Dynamic, Engine};
|
||||
use tokio::sync::mpsc; // For shutdown signal
|
||||
use tokio::task::JoinHandle;
|
||||
|
||||
/// Actor trait abstraction for unified actor interface
|
||||
pub mod actor_trait;
|
||||
|
||||
const NAMESPACE_PREFIX: &str = "hero:job:";
|
||||
const BLPOP_TIMEOUT_SECONDS: usize = 5;
|
||||
|
||||
/// Initialize Redis connection for the actor
|
||||
pub(crate) async fn initialize_redis_connection(
|
||||
actor_id: &str,
|
||||
redis_url: &str,
|
||||
) -> Result<redis::aio::MultiplexedConnection, Box<dyn std::error::Error + Send + Sync>> {
|
||||
let redis_client = redis::Client::open(redis_url)
|
||||
.map_err(|e| {
|
||||
error!("Actor for Actor ID '{}': Failed to open Redis client: {}", actor_id, e);
|
||||
e
|
||||
})?;
|
||||
|
||||
let redis_conn = redis_client.get_multiplexed_async_connection().await
|
||||
.map_err(|e| {
|
||||
error!("Actor for Actor ID '{}': Failed to get Redis connection: {}", actor_id, e);
|
||||
e
|
||||
})?;
|
||||
|
||||
info!("Actor for Actor ID '{}' successfully connected to Redis.", actor_id);
|
||||
Ok(redis_conn)
|
||||
}
|
||||
|
||||
/// Load job from Redis using Job struct
|
||||
pub(crate) async fn load_job_from_redis(
|
||||
redis_conn: &mut redis::aio::MultiplexedConnection,
|
||||
job_id: &str,
|
||||
actor_id: &str,
|
||||
) -> Result<Job, Box<dyn std::error::Error + Send + Sync>> {
|
||||
debug!("Actor '{}', Job {}: Loading job from Redis", actor_id, job_id);
|
||||
|
||||
match Job::load_from_redis(redis_conn, job_id).await {
|
||||
Ok(job) => {
|
||||
debug!("Actor '{}', Job {}: Successfully loaded job", actor_id, job_id);
|
||||
Ok(job)
|
||||
}
|
||||
Err(e) => {
|
||||
error!("Actor '{}', Job {}: Failed to load job from Redis: {}", actor_id, job_id, e);
|
||||
Err(Box::new(e))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Execute the Rhai script and update job status in Redis
|
||||
async fn execute_script_and_update_status(
|
||||
redis_conn: &mut redis::aio::MultiplexedConnection,
|
||||
engine: &mut Engine,
|
||||
job: &Job,
|
||||
db_path: &str,
|
||||
) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
|
||||
let mut db_config = rhai::Map::new();
|
||||
db_config.insert("DB_PATH".into(), db_path.to_string().into());
|
||||
db_config.insert("CALLER_ID".into(), job.caller_id.clone().into());
|
||||
db_config.insert("CONTEXT_ID".into(), job.context_id.clone().into());
|
||||
engine.set_default_tag(Dynamic::from(db_config));
|
||||
|
||||
debug!("Actor for Context ID '{}': Evaluating script with Rhai engine.", job.context_id);
|
||||
|
||||
match engine.eval::<rhai::Dynamic>(&job.script) {
|
||||
Ok(result) => {
|
||||
let output_str = if result.is::<String>() {
|
||||
result.into_string().unwrap()
|
||||
} else {
|
||||
result.to_string()
|
||||
};
|
||||
info!("Actor for Context ID '{}' job {} completed. Output: {}", job.context_id, job.id, output_str);
|
||||
|
||||
// Update job status to finished and set result
|
||||
Job::update_status(redis_conn, &job.id, JobStatus::Finished).await
|
||||
.map_err(|e| {
|
||||
error!("Failed to update job {} status to finished: {}", job.id, e);
|
||||
e
|
||||
})?;
|
||||
|
||||
Job::set_result(redis_conn, &job.id, &output_str).await
|
||||
.map_err(|e| {
|
||||
error!("Failed to set job {} result: {}", job.id, e);
|
||||
e
|
||||
})?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
Err(e) => {
|
||||
let error_str = format!("{:?}", *e);
|
||||
error!("Actor for Context ID '{}' job {} script evaluation failed. Error: {}", job.context_id, job.id, error_str);
|
||||
|
||||
// Update job status to error and set error message
|
||||
Job::update_status(redis_conn, &job.id, JobStatus::Error).await
|
||||
.map_err(|e| {
|
||||
error!("Failed to update job {} status to error: {}", job.id, e);
|
||||
e
|
||||
})?;
|
||||
|
||||
Job::set_error(redis_conn, &job.id, &error_str).await
|
||||
.map_err(|e| {
|
||||
error!("Failed to set job {} error: {}", job.id, e);
|
||||
e
|
||||
})?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Clean up job from Redis if preserve_tasks is false
|
||||
async fn cleanup_job(
|
||||
redis_conn: &mut redis::aio::MultiplexedConnection,
|
||||
job_id: &str,
|
||||
context_id: &str,
|
||||
preserve_tasks: bool,
|
||||
) {
|
||||
if !preserve_tasks {
|
||||
if let Err(e) = Job::delete_from_redis(redis_conn, job_id).await {
|
||||
error!("Actor for Context ID '{}', Job {}: Failed to delete job: {}", context_id, job_id, e);
|
||||
} else {
|
||||
debug!("Actor for Context ID '{}', Job {}: Cleaned up job.", context_id, job_id);
|
||||
}
|
||||
} else {
|
||||
debug!("Actor for Context ID '{}', Job {}: Preserving job (preserve_tasks=true)", context_id, job_id);
|
||||
}
|
||||
}
|
||||
|
||||
/// Process a single job from the queue
|
||||
async fn process_job(
|
||||
redis_conn: &mut redis::aio::MultiplexedConnection,
|
||||
job_id: &str,
|
||||
actor_id: &str,
|
||||
db_path: &str,
|
||||
engine: &mut Engine,
|
||||
preserve_tasks: bool,
|
||||
) {
|
||||
debug!("Actor '{}', Job {}: Processing started.", actor_id, job_id);
|
||||
|
||||
// Load job from Redis
|
||||
match load_job_from_redis(redis_conn, job_id, actor_id).await {
|
||||
Ok(job) => {
|
||||
info!("Actor '{}' processing job_id: {}. Script: {:.50}...", job.context_id, job_id, job.script);
|
||||
|
||||
// Update status to started
|
||||
debug!("Actor for Context ID '{}', Job {}: Attempting to update status to 'started'.", job.context_id, job_id);
|
||||
if let Err(e) = Job::update_status(redis_conn, job_id, JobStatus::Started).await {
|
||||
error!("Actor for Context ID '{}', Job {}: Failed to update status to 'started': {}", job.context_id, job_id, e);
|
||||
} else {
|
||||
debug!("Actor for Context ID '{}', Job {}: Status updated to 'started'.", job.context_id, job_id);
|
||||
}
|
||||
|
||||
// Execute the script and update status
|
||||
if let Err(e) = execute_script_and_update_status(redis_conn, engine, &job, db_path).await {
|
||||
error!("Actor for Context ID '{}', Job {}: Script execution failed: {}", job.context_id, job_id, e);
|
||||
|
||||
// Ensure job status is set to error if execution failed
|
||||
if let Err(status_err) = Job::update_status(redis_conn, job_id, JobStatus::Error).await {
|
||||
error!("Actor for Context ID '{}', Job {}: Failed to update status to error after execution failure: {}", job.context_id, job_id, status_err);
|
||||
}
|
||||
}
|
||||
|
||||
// Clean up job if needed
|
||||
cleanup_job(redis_conn, job_id, &job.context_id, preserve_tasks).await;
|
||||
}
|
||||
Err(e) => {
|
||||
error!("Actor '{}', Job {}: Failed to load job: {}", actor_id, job_id, e);
|
||||
// Clean up invalid job if needed
|
||||
if !preserve_tasks {
|
||||
if let Err(del_err) = Job::delete_from_redis(redis_conn, job_id).await {
|
||||
error!("Actor '{}', Job {}: Failed to delete invalid job: {}", actor_id, job_id, del_err);
|
||||
}
|
||||
} else {
|
||||
debug!("Actor '{}', Job {}: Preserving invalid job (preserve_tasks=true)", actor_id, job_id);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn spawn_rhai_actor(
|
||||
actor_id: String,
|
||||
db_path: String,
|
||||
mut engine: Engine,
|
||||
redis_url: String,
|
||||
mut shutdown_rx: mpsc::Receiver<()>,
|
||||
preserve_tasks: bool,
|
||||
) -> JoinHandle<Result<(), Box<dyn std::error::Error + Send + Sync>>> {
|
||||
tokio::spawn(async move {
|
||||
let queue_key = format!("{}{}", NAMESPACE_PREFIX, actor_id);
|
||||
info!(
|
||||
"Rhai Actor for Actor ID '{}' starting. Connecting to Redis at {}. Listening on queue: {}. Waiting for tasks or shutdown signal.",
|
||||
actor_id, redis_url, queue_key
|
||||
);
|
||||
|
||||
let mut redis_conn = initialize_redis_connection(&actor_id, &redis_url).await?;
|
||||
|
||||
loop {
|
||||
let blpop_keys = vec![queue_key.clone()];
|
||||
tokio::select! {
|
||||
// Listen for shutdown signal
|
||||
_ = shutdown_rx.recv() => {
|
||||
info!("Actor for Actor ID '{}': Shutdown signal received. Terminating loop.", actor_id);
|
||||
break;
|
||||
}
|
||||
// Listen for tasks from Redis
|
||||
blpop_result = redis_conn.blpop(&blpop_keys, BLPOP_TIMEOUT_SECONDS as f64) => {
|
||||
debug!("Actor for Actor ID '{}': Attempting BLPOP on queue: {}", actor_id, queue_key);
|
||||
let response: Option<(String, String)> = match blpop_result {
|
||||
Ok(resp) => resp,
|
||||
Err(e) => {
|
||||
error!("Actor '{}': Redis BLPOP error on queue {}: {}. Actor for this circle might stop.", actor_id, queue_key, e);
|
||||
return Err(Box::new(e) as Box<dyn std::error::Error + Send + Sync>);
|
||||
}
|
||||
};
|
||||
|
||||
if let Some((_queue_name_recv, job_id)) = response {
|
||||
info!("Actor '{}' received job_id: {} from queue: {}", actor_id, job_id, _queue_name_recv);
|
||||
process_job(&mut redis_conn, &job_id, &actor_id, &db_path, &mut engine, preserve_tasks).await;
|
||||
} else {
|
||||
debug!("Actor '{}': BLPOP timed out on queue {}. No new tasks. Checking for shutdown signal again.", actor_id, queue_key);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
info!("Actor '{}' has shut down.", actor_id);
|
||||
Ok(())
|
||||
})
|
||||
}
|
||||
|
||||
// Re-export the main trait-based interface for convenience
|
||||
pub use actor_trait::{Actor, ActorConfig, spawn_actor};
|
||||
|
Reference in New Issue
Block a user