294 lines
11 KiB
Markdown
294 lines
11 KiB
Markdown
# Rhailib Architecture Overview
|
|
|
|
Rhailib is a comprehensive Rust-based ecosystem for executing Rhai scripts in distributed environments with full business domain support, authorization, and scalability features.
|
|
|
|
## System Architecture
|
|
|
|
```mermaid
|
|
graph TB
|
|
subgraph "Client Layer"
|
|
A[rhai_client] --> B[Redis Task Queues]
|
|
UI[rhai_engine_ui] --> B
|
|
REPL[ui_repl] --> B
|
|
end
|
|
|
|
subgraph "Processing Layer"
|
|
B --> C[rhailib_worker]
|
|
C --> D[rhailib_engine]
|
|
D --> E[rhailib_dsl]
|
|
end
|
|
|
|
subgraph "Core Infrastructure"
|
|
E --> F[derive - Procedural Macros]
|
|
E --> G[macros - Authorization]
|
|
D --> H[mock_db - Testing]
|
|
end
|
|
|
|
subgraph "Operations Layer"
|
|
I[monitor] --> B
|
|
I --> C
|
|
end
|
|
|
|
subgraph "Data Layer"
|
|
J[Redis] --> B
|
|
K[Database] --> E
|
|
end
|
|
```
|
|
|
|
## Crate Overview
|
|
|
|
### Core Engine Components
|
|
|
|
#### [`rhailib_engine`](../src/engine/docs/ARCHITECTURE.md)
|
|
The central Rhai scripting engine that orchestrates all business domain modules.
|
|
- **Purpose**: Unified engine creation and script execution
|
|
- **Features**: Mock database, feature-based architecture, performance optimization
|
|
- **Key Functions**: `create_heromodels_engine()`, script compilation and execution
|
|
|
|
#### [`rhailib_dsl`](../src/dsl/docs/ARCHITECTURE.md)
|
|
Comprehensive Domain-Specific Language implementation exposing business models to Rhai.
|
|
- **Purpose**: Business domain integration with Rhai scripting
|
|
- **Domains**: Business operations, finance, content management, workflows, access control
|
|
- **Features**: Fluent APIs, type safety, authorization integration
|
|
|
|
### Code Generation and Utilities
|
|
|
|
#### [`derive`](../src/derive/docs/ARCHITECTURE.md)
|
|
Procedural macros for automatic Rhai integration code generation.
|
|
- **Purpose**: Simplify Rhai integration for custom types
|
|
- **Macros**: `RhaiApi` for DSL generation, `FromVec` for type conversion
|
|
- **Features**: Builder pattern generation, error handling
|
|
|
|
#### [`macros`](../src/macros/docs/ARCHITECTURE.md)
|
|
Authorization macros and utilities for secure database operations.
|
|
- **Purpose**: Declarative security for Rhai functions
|
|
- **Features**: CRUD operation macros, access control, context management
|
|
- **Security**: Multi-level authorization, audit trails
|
|
|
|
### Client and Communication
|
|
|
|
#### [`rhai_client`](../src/client/docs/ARCHITECTURE.md)
|
|
Redis-based client library for distributed script execution.
|
|
- **Purpose**: Submit and manage Rhai script execution requests
|
|
- **Features**: Builder pattern API, timeout handling, request-reply pattern
|
|
- **Architecture**: Async operations, connection pooling, error handling
|
|
|
|
#### [`rhailib_worker`](../src/worker/docs/ARCHITECTURE.md)
|
|
Distributed task execution system for processing Rhai scripts.
|
|
- **Purpose**: Scalable script processing with queue-based architecture
|
|
- **Features**: Multi-context support, horizontal scaling, fault tolerance, context injection
|
|
- **Architecture**: Workers decoupled from contexts, allowing single worker to serve multiple circles
|
|
- **Integration**: Full engine and DSL access, secure execution
|
|
|
|
### User Interfaces
|
|
|
|
#### [`ui_repl`](../src/repl/docs/ARCHITECTURE.md)
|
|
Interactive development environment for Rhai script development.
|
|
- **Purpose**: Real-time script development and testing
|
|
- **Features**: Enhanced CLI, dual execution modes, worker management
|
|
- **Development**: Syntax highlighting, script editing, immediate feedback
|
|
|
|
#### [`rhai_engine_ui`](../src/rhai_engine_ui/docs/ARCHITECTURE.md)
|
|
Web-based interface for Rhai script management and execution.
|
|
- **Purpose**: Browser-based script execution and management
|
|
- **Architecture**: WebAssembly frontend with optional server backend
|
|
- **Features**: Real-time updates, task management, visual interface
|
|
|
|
### Operations and Monitoring
|
|
|
|
#### [`monitor`](../src/monitor/docs/ARCHITECTURE.md)
|
|
Command-line monitoring and management tool for the rhailib ecosystem.
|
|
- **Purpose**: System observability and task management
|
|
- **Features**: Real-time monitoring, performance metrics, queue management
|
|
- **Operations**: Multi-worker support, interactive CLI, visualization
|
|
|
|
## Data Flow Architecture
|
|
|
|
### Script Execution Flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Client as rhai_client
|
|
participant Redis as Redis Queue
|
|
participant Worker as rhailib_worker
|
|
participant Engine as rhailib_engine
|
|
participant DSL as rhailib_dsl
|
|
participant DB as Database
|
|
|
|
Client->>Redis: Submit script task (worker_id + context_id)
|
|
Worker->>Redis: Poll worker queue (worker_id)
|
|
Redis->>Worker: Return task with context_id
|
|
Worker->>Engine: Create configured engine
|
|
Engine->>DSL: Register domain modules
|
|
Worker->>Engine: Execute script with context_id
|
|
Engine->>DSL: Call business functions (context_id)
|
|
DSL->>DB: Perform authorized operations (context_id)
|
|
DB->>DSL: Return results
|
|
DSL->>Engine: Return processed data
|
|
Engine->>Worker: Return execution result
|
|
Worker->>Redis: Publish result to reply queue
|
|
Redis->>Client: Deliver result
|
|
```
|
|
|
|
### Authorization Flow
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant Script as Rhai Script
|
|
participant Macro as Authorization Macro
|
|
participant Context as Execution Context
|
|
participant Access as Access Control
|
|
participant DB as Database
|
|
|
|
Script->>Macro: Call authorized function
|
|
Macro->>Context: Extract caller credentials
|
|
Context->>Access: Validate permissions
|
|
Access->>DB: Check resource access
|
|
DB->>Access: Return authorization result
|
|
Access->>Macro: Grant/deny access
|
|
Macro->>DB: Execute authorized operation
|
|
DB->>Script: Return results
|
|
```
|
|
|
|
## Worker-Context Decoupling Architecture
|
|
|
|
A key architectural feature of rhailib is the decoupling of worker assignment from context management:
|
|
|
|
### Traditional Model (Previous)
|
|
- **One Worker Per Circle**: Each worker was dedicated to a specific circle/context
|
|
- **Queue Per Circle**: Workers listened to circle-specific queues
|
|
- **Tight Coupling**: Worker identity was directly tied to context identity
|
|
|
|
### New Decoupled Model (Current)
|
|
- **Worker ID**: Determines which queue the worker listens to (`rhailib:<worker_id>`)
|
|
- **Context ID**: Provided in task details, determines execution context and database access
|
|
- **Flexible Assignment**: Single worker can process tasks for multiple contexts
|
|
|
|
### Benefits of Decoupling
|
|
|
|
1. **Resource Efficiency**: Better worker utilization across multiple contexts
|
|
2. **Deployment Flexibility**: Easier scaling and resource allocation
|
|
3. **Cost Optimization**: Fewer worker instances needed for multi-context scenarios
|
|
4. **Operational Simplicity**: Centralized worker management with distributed contexts
|
|
|
|
### Implementation Details
|
|
|
|
```mermaid
|
|
graph LR
|
|
subgraph "Client Layer"
|
|
C[Client] --> |worker_id + context_id| Q[Redis Queue]
|
|
end
|
|
|
|
subgraph "Worker Layer"
|
|
W1[Worker 1] --> |listens to| Q1[Queue: worker-1]
|
|
W2[Worker 2] --> |listens to| Q2[Queue: worker-2]
|
|
end
|
|
|
|
subgraph "Context Layer"
|
|
W1 --> |processes| CTX1[Context A]
|
|
W1 --> |processes| CTX2[Context B]
|
|
W2 --> |processes| CTX1
|
|
W2 --> |processes| CTX3[Context C]
|
|
end
|
|
```
|
|
|
|
## Key Design Principles
|
|
|
|
### 1. Security First
|
|
- **Multi-layer Authorization**: Context-based, resource-specific, and operation-level security
|
|
- **Secure Execution**: Isolated script execution with proper context injection
|
|
- **Audit Trails**: Comprehensive logging and monitoring of all operations
|
|
|
|
### 2. Scalability
|
|
- **Horizontal Scaling**: Multiple worker instances for load distribution
|
|
- **Queue-based Architecture**: Reliable task distribution and processing
|
|
- **Async Operations**: Non-blocking I/O throughout the system
|
|
|
|
### 3. Developer Experience
|
|
- **Type Safety**: Comprehensive type checking and conversion utilities
|
|
- **Error Handling**: Detailed error messages and proper error propagation
|
|
- **Interactive Development**: REPL and web interfaces for immediate feedback
|
|
|
|
### 4. Modularity
|
|
- **Feature Flags**: Configurable compilation based on requirements
|
|
- **Crate Separation**: Clear boundaries and responsibilities
|
|
- **Plugin Architecture**: Easy extension and customization
|
|
|
|
## Deployment Patterns
|
|
|
|
### Development Environment
|
|
```
|
|
REPL + Local Engine + Mock Database
|
|
```
|
|
- Interactive development with immediate feedback
|
|
- Full DSL access without external dependencies
|
|
- Integrated testing and debugging
|
|
|
|
### Testing Environment
|
|
```
|
|
Client + Worker + Redis + Mock Database
|
|
```
|
|
- Distributed execution testing
|
|
- Queue-based communication validation
|
|
- Performance and scalability testing
|
|
|
|
### Production Environment
|
|
```
|
|
Multiple Clients + Redis Cluster + Worker Pool + Production Database
|
|
```
|
|
- High availability and fault tolerance
|
|
- Horizontal scaling and load distribution
|
|
- Comprehensive monitoring and observability
|
|
|
|
## Integration Points
|
|
|
|
### External Systems
|
|
- **Redis**: Task queues, result delivery, system coordination
|
|
- **Databases**: Business data persistence and retrieval
|
|
- **Web Browsers**: WebAssembly-based user interfaces
|
|
- **Command Line**: Development and operations tooling
|
|
|
|
### Internal Integration
|
|
- **Macro System**: Code generation and authorization
|
|
- **Type System**: Safe conversions and error handling
|
|
- **Module System**: Domain-specific functionality organization
|
|
- **Context System**: Security and execution environment management
|
|
|
|
## Performance Characteristics
|
|
|
|
### Throughput
|
|
- **Concurrent Execution**: Multiple workers processing tasks simultaneously
|
|
- **Connection Pooling**: Efficient database and Redis connection management
|
|
- **Compiled Scripts**: AST caching for repeated execution optimization
|
|
|
|
### Latency
|
|
- **Local Execution**: Direct engine access for development scenarios
|
|
- **Queue Optimization**: Efficient task distribution and result delivery
|
|
- **Context Caching**: Reduced overhead for authorization and setup
|
|
|
|
### Resource Usage
|
|
- **Memory Management**: Efficient ownership and borrowing patterns
|
|
- **CPU Utilization**: Async operations and non-blocking I/O
|
|
- **Network Efficiency**: Optimized serialization and communication protocols
|
|
|
|
## Future Extensibility
|
|
|
|
### Adding New Domains
|
|
1. Create domain module in `rhailib_dsl`
|
|
2. Implement authorization macros in `macros`
|
|
3. Add feature flags and conditional compilation
|
|
4. Update engine registration and documentation
|
|
|
|
### Custom Authorization
|
|
1. Extend authorization macros with custom logic
|
|
2. Implement domain-specific access control functions
|
|
3. Add audit and logging capabilities
|
|
4. Update security documentation
|
|
|
|
### New Interfaces
|
|
1. Implement client interface following existing patterns
|
|
2. Integrate with Redis communication layer
|
|
3. Add monitoring and observability features
|
|
4. Provide comprehensive documentation
|
|
|
|
This architecture provides a robust, secure, and scalable foundation for distributed Rhai script execution while maintaining excellent developer experience and operational visibility. |