190 lines
6.1 KiB
Markdown
190 lines
6.1 KiB
Markdown
# Architecture of the `rhai_dispatcher` Crate
|
|
|
|
The `rhai_dispatcher` crate provides a Redis-based client library for submitting Rhai scripts to distributed worker services and awaiting their execution results. It implements a request-reply pattern using Redis as the message broker.
|
|
|
|
## Core Architecture
|
|
|
|
The client follows a builder pattern design with clear separation of concerns:
|
|
|
|
```mermaid
|
|
graph TD
|
|
A[RhaiDispatcherBuilder] --> B[RhaiDispatcher]
|
|
B --> C[PlayRequestBuilder]
|
|
C --> D[PlayRequest]
|
|
D --> E[Redis Task Queue]
|
|
E --> F[Worker Service]
|
|
F --> G[Redis Reply Queue]
|
|
G --> H[Client Response]
|
|
|
|
subgraph "Client Components"
|
|
A
|
|
B
|
|
C
|
|
D
|
|
end
|
|
|
|
subgraph "Redis Infrastructure"
|
|
E
|
|
G
|
|
end
|
|
|
|
subgraph "External Services"
|
|
F
|
|
end
|
|
```
|
|
|
|
## Key Components
|
|
|
|
### 1. RhaiDispatcherBuilder
|
|
|
|
A builder pattern implementation for constructing `RhaiDispatcher` instances with proper configuration validation.
|
|
|
|
**Responsibilities:**
|
|
- Configure Redis connection URL
|
|
- Set caller ID for task attribution
|
|
- Validate configuration before building client
|
|
|
|
**Key Methods:**
|
|
- `caller_id(id: &str)` - Sets the caller identifier
|
|
- `redis_url(url: &str)` - Configures Redis connection
|
|
- `build()` - Creates the final `RhaiDispatcher` instance
|
|
|
|
### 2. RhaiDispatcher
|
|
|
|
The main client interface that manages Redis connections and provides factory methods for creating play requests.
|
|
|
|
**Responsibilities:**
|
|
- Maintain Redis connection pool
|
|
- Provide factory methods for request builders
|
|
- Handle low-level Redis operations
|
|
- Manage task status queries
|
|
|
|
**Key Methods:**
|
|
- `new_play_request()` - Creates a new `PlayRequestBuilder`
|
|
- `get_task_status(task_id)` - Queries task status from Redis
|
|
- Internal methods for Redis operations
|
|
|
|
### 3. PlayRequestBuilder
|
|
|
|
A fluent builder for constructing and submitting script execution requests.
|
|
|
|
**Responsibilities:**
|
|
- Configure script execution parameters
|
|
- Handle script loading from files or strings
|
|
- Manage request timeouts
|
|
- Provide submission methods (fire-and-forget vs await-response)
|
|
|
|
**Key Methods:**
|
|
- `worker_id(id: &str)` - Target worker queue (determines which worker processes the task)
|
|
- `context_id(id: &str)` - Target context ID (determines execution context/circle)
|
|
- `script(content: &str)` - Set script content directly
|
|
- `script_path(path: &str)` - Load script from file
|
|
- `timeout(duration: Duration)` - Set execution timeout
|
|
- `submit()` - Fire-and-forget submission
|
|
- `await_response()` - Submit and wait for result
|
|
|
|
**Architecture Note:** The decoupling of `worker_id` and `context_id` allows a single worker to process tasks for multiple contexts (circles), providing greater deployment flexibility.
|
|
|
|
### 4. Data Structures
|
|
|
|
#### RhaiTaskDetails
|
|
Represents the complete state of a task throughout its lifecycle.
|
|
|
|
```rust
|
|
pub struct RhaiTaskDetails {
|
|
pub task_id: String,
|
|
pub script: String,
|
|
pub status: String, // "pending", "processing", "completed", "error"
|
|
pub output: Option<String>,
|
|
pub error: Option<String>,
|
|
pub created_at: DateTime<Utc>,
|
|
pub updated_at: DateTime<Utc>,
|
|
pub caller_id: String,
|
|
}
|
|
```
|
|
|
|
#### RhaiDispatcherError
|
|
Comprehensive error handling for various failure scenarios:
|
|
- `RedisError` - Redis connection/operation failures
|
|
- `SerializationError` - JSON serialization/deserialization issues
|
|
- `Timeout` - Task execution timeouts
|
|
- `TaskNotFound` - Missing tasks after submission
|
|
|
|
## Communication Protocol
|
|
|
|
### Task Submission Flow
|
|
|
|
1. **Task Creation**: Client generates unique UUID for task identification
|
|
2. **Task Storage**: Task details stored in Redis hash: `rhailib:<task_id>`
|
|
3. **Queue Submission**: Task ID pushed to worker queue: `rhailib:<worker_id>`
|
|
4. **Reply Queue Setup**: Client listens on: `rhailib:reply:<task_id>`
|
|
|
|
### Redis Key Patterns
|
|
|
|
- **Task Storage**: `rhailib:<task_id>` (Redis Hash)
|
|
- **Worker Queues**: `rhailib:<worker_id>` (Redis List)
|
|
- **Reply Queues**: `rhailib:reply:<task_id>` (Redis List)
|
|
|
|
### Message Flow Diagram
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant C as Client
|
|
participant R as Redis
|
|
participant W as Worker
|
|
|
|
C->>R: HSET rhailib:task_id (task details)
|
|
C->>R: LPUSH rhailib:worker_id task_id
|
|
C->>R: BLPOP rhailib:reply:task_id (blocking)
|
|
|
|
W->>R: BRPOP rhailib:worker_id (blocking)
|
|
W->>W: Execute Rhai Script
|
|
W->>R: LPUSH rhailib:reply:task_id (result)
|
|
|
|
R->>C: Return result from BLPOP
|
|
C->>R: DEL rhailib:reply:task_id (cleanup)
|
|
```
|
|
|
|
## Concurrency and Async Design
|
|
|
|
The client is built on `tokio` for asynchronous operations:
|
|
|
|
- **Connection Pooling**: Uses Redis multiplexed connections for efficiency
|
|
- **Non-blocking Operations**: All Redis operations are async
|
|
- **Timeout Handling**: Configurable timeouts with proper cleanup
|
|
- **Error Propagation**: Comprehensive error handling with context
|
|
|
|
## Configuration and Deployment
|
|
|
|
### Prerequisites
|
|
- Redis server accessible to both client and workers
|
|
- Proper network connectivity between components
|
|
- Sufficient Redis memory for task storage
|
|
|
|
### Configuration Options
|
|
- **Redis URL**: Connection string for Redis instance
|
|
- **Caller ID**: Unique identifier for client instance
|
|
- **Timeouts**: Per-request timeout configuration
|
|
- **Worker Targeting**: Direct worker queue addressing
|
|
|
|
## Security Considerations
|
|
|
|
- **Task Isolation**: Each task uses unique identifiers
|
|
- **Queue Separation**: Worker-specific queues prevent cross-contamination
|
|
- **Cleanup**: Automatic cleanup of reply queues after completion
|
|
- **Error Handling**: Secure error propagation without sensitive data leakage
|
|
|
|
## Performance Characteristics
|
|
|
|
- **Scalability**: Horizontal scaling through multiple worker instances
|
|
- **Throughput**: Limited by Redis performance and network latency
|
|
- **Memory Usage**: Efficient with connection pooling and cleanup
|
|
- **Latency**: Low latency for local Redis deployments
|
|
|
|
## Integration Points
|
|
|
|
The client integrates with:
|
|
- **Worker Services**: Via Redis queue protocol
|
|
- **Monitoring Systems**: Through structured logging
|
|
- **Application Code**: Via builder pattern API
|
|
- **Configuration Systems**: Through environment variables and builders |