124 lines
7.2 KiB
Markdown
124 lines
7.2 KiB
Markdown
# Rhailib Architecture: Distributed Rhai Scripting
|
|
|
|
## 1. Overview
|
|
|
|
`rhailib` provides a robust infrastructure for executing Rhai scripts in a distributed manner, primarily designed to integrate with and extend the HeroModels ecosystem. It allows for dynamic scripting capabilities, offloading computation, and enabling flexible automation. This document describes the target architecture utilizing dedicated reply queues for efficient result notification.
|
|
|
|
## 2. Core Components
|
|
|
|
The `rhailib` system is composed of the following main components, leveraging Redis for task queuing, state management, and result notification:
|
|
|
|
1. **Rhai Engine (`src/engine`):**
|
|
* The core scripting capability. Provides a Rhai engine pre-configured with various modules.
|
|
* Utilized by the `rhai_worker` to process tasks.
|
|
|
|
2. **Rhai Client (`src/client`):**
|
|
* Offers an interface for applications to submit Rhai scripts as tasks.
|
|
* Submits tasks to named Redis queues ("circles").
|
|
* Waits for results on a dedicated reply queue, avoiding polling.
|
|
|
|
3. **Rhai Worker (`src/worker`):**
|
|
* Listens to Redis task queues ("circles") for incoming task IDs.
|
|
* Fetches task details, executes the script using the `rhai_engine`.
|
|
* Updates task status and results in Redis.
|
|
* Sends a notification/result to the client's dedicated reply queue.
|
|
|
|
4. **Redis:**
|
|
* Acts as the message broker and data store for task queues, detailed task information (including scripts, status, and results), and reply queues for notifications.
|
|
|
|
## 3. Architecture & Workflow (Dedicated Reply Queues)
|
|
|
|
The system employs a "Dedicated Reply Queue" pattern to notify clients of task completion, enhancing efficiency by eliminating client-side polling for results.
|
|
|
|
**Workflow:**
|
|
|
|
1. **Task Submission (Client):**
|
|
a. The client generates a unique `task_id` and a unique `reply_queue_name` (e.g., `rhai_reply:<uuid>`).
|
|
b. Task details, including the script, initial status ("pending"), and the `reply_queue_name`, are stored in a Redis Hash: `rhai_task_details:<task_id>`.
|
|
c. The `task_id` is pushed onto a Redis List acting as a task queue for a specific "circle": `rhai_tasks:<circle_name>`.
|
|
d. The client then performs a blocking pop (`BLPOP`) on its `reply_queue_name`, waiting for the result message.
|
|
|
|
2. **Task Consumption & Processing (Worker):**
|
|
a. A `rhai_worker` instance, listening to one or more `rhai_tasks:<circle_name>` queues, picks up a `task_id` using `BLPOP`.
|
|
b. The worker retrieves the full task details (including the script and `reply_queue_name`) from the `rhai_task_details:<task_id>` hash.
|
|
c. The worker updates the task's status in the hash to "processing".
|
|
d. The Rhai script is executed using an instance of the `rhai_engine`.
|
|
|
|
3. **Result Storage & Notification (Worker):**
|
|
a. Upon completion (or error), the worker updates the `rhai_task_details:<task_id>` hash with the final status ("completed" or "error") and the script's output or error message.
|
|
b. If a `reply_queue_name` was provided in the task details, the worker constructs a result message (e.g., JSON containing `task_id`, final `status`, `output`/`error`).
|
|
c. The worker pushes this result message onto the specified `reply_queue_name` using `LPUSH`.
|
|
|
|
4. **Result Reception (Client):**
|
|
a. The client's `BLPOP` on its `reply_queue_name` receives the result message.
|
|
b. The client processes the result or error.
|
|
c. Optionally, the client may `DEL`ete its temporary reply queue.
|
|
|
|
**Diagram:**
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Client
|
|
participant Redis
|
|
participant Worker
|
|
|
|
Client->>Client: 1. Generate task_id & reply_queue_name
|
|
Client->>Redis: 2. LPUSH task_id to rhai_tasks:<circle>
|
|
Client->>Redis: 3. HSET rhai_task_details:<task_id> (script, status:pending, reply_to:reply_queue_name)
|
|
Client->>Redis: 4. BLPOP from reply_queue_name (waits with timeout)
|
|
|
|
Worker->>Redis: 5. BLPOP from rhai_tasks:<circle>
|
|
Redis-->>Worker: task_id
|
|
Worker->>Redis: 6. HGETALL rhai_task_details:<task_id>
|
|
Redis-->>Worker: task_details (script, reply_to)
|
|
Worker->>Redis: 7. HSET rhai_task_details:<task_id> (status:processing)
|
|
Note over Worker: Executes Rhai script
|
|
|
|
alt Script Success
|
|
Worker->>Redis: 8a. HSET rhai_task_details:<task_id> (status:completed, output)
|
|
Worker->>Redis: 9a. LPUSH to reply_queue_name (message: {task_id, status:completed, output})
|
|
else Script Error
|
|
Worker->>Redis: 8b. HSET rhai_task_details:<task_id> (status:error, error_msg)
|
|
Worker->>Redis: 9b. LPUSH to reply_queue_name (message: {task_id, status:error, error_msg})
|
|
end
|
|
|
|
Redis-->>Client: Result message from reply_queue_name
|
|
Client->>Client: Process result/error
|
|
Client->>Redis: 10. DEL reply_queue_name (optional cleanup)
|
|
```
|
|
|
|
This architecture allows for:
|
|
* Asynchronous and non-blocking script execution for the client.
|
|
* Scalable processing of Rhai scripts by running multiple workers.
|
|
* Efficient, event-driven result notification to clients.
|
|
* Robust task state tracking and observability.
|
|
|
|
## 4. Redis Data Structures
|
|
|
|
* **Task Queues:**
|
|
* Key Pattern: `rhai_tasks:<circle_name>`
|
|
* Type: List
|
|
* Purpose: FIFO queue for `task_id`s waiting to be processed by workers assigned to a specific circle. Workers use `BLPOP`. Clients use `LPUSH`.
|
|
* **Task Details:**
|
|
* Key Pattern: `rhai_task_details:<task_id>`
|
|
* Type: Hash
|
|
* Purpose: Stores all information about a specific task.
|
|
* Key Fields:
|
|
* `script`: The Rhai script content.
|
|
* `status`: Current state of the task (e.g., "pending", "processing", "completed", "error").
|
|
* `client_rpc_id`: Optional client-provided identifier.
|
|
* `output`: The result of a successful script execution.
|
|
* `error`: Error message if script execution failed.
|
|
* `created_at`: Timestamp of task creation.
|
|
* `updated_at`: Timestamp of the last update to the task details.
|
|
* `reply_to_queue`: (New) The name of the dedicated Redis List the client is listening on for the result.
|
|
* **Reply Queues:**
|
|
* Key Pattern: `rhai_reply:<unique_identifier>` (e.g., `rhai_reply:<uuid_generated_by_client>`)
|
|
* Type: List
|
|
* Purpose: A temporary, client-specific queue where the worker pushes the final result/notification for a particular task. The client uses `BLPOP` to wait for a message on this queue.
|
|
|
|
## 5. Key Design Choices
|
|
|
|
* **Hashes for Task Details:** Storing comprehensive task details in a Redis Hash provides a persistent, inspectable, and easily updatable record of each task's lifecycle. This aids in monitoring, debugging, and potential recovery scenarios.
|
|
* **Dedicated Reply Queues:** Using a unique Redis List per client request for result notification offers reliable message delivery (compared to Pub/Sub's fire-and-forget) and allows the client to efficiently block until its specific result is ready, eliminating polling.
|
|
* **Status Tracking in Hash:** Maintaining the `status` field within the task details hash ("pending", "processing", "completed", "error") offers crucial observability into the system's state and the progress of individual tasks, independent of the client-worker notification flow. |