# Rhai Engine Factory Implementation Plan Based on our exploration of the Rhai documentation and your requirements, I'll now outline a detailed plan for implementing the Rhai engine factory module. ## Overview We'll create a Rust module called `rhai_factory` that provides a factory for creating thread-safe Rhai engines with pre-compiled scripts. The factory will: 1. Use the `sync` feature to ensure the engine is `Send + Sync` 2. Compile Rhai modules into a self-contained AST for better performance 3. Provide detailed error handling that shows which module failed to import and why ## Architecture ```mermaid graph TD A[RhaiFactory] --> B[create_engine] A --> C[compile_modules] A --> D[create_engine_with_modules] B --> E[Engine with sync feature] C --> F[Self-contained AST] D --> G[Engine with compiled modules] H[FileModuleResolver] --> C I[Error Handling] --> C J[Module Cache] --> C ``` ## Component Details ### 1. RhaiFactory Module Structure ``` rhai_factory/ ├── Cargo.toml # Dependencies including rhai with sync feature ├── src/ │ ├── lib.rs # Main module exports and unit tests │ ├── factory.rs # Factory implementation and unit tests │ ├── error.rs # Custom error types and unit tests │ └── module_cache.rs # Optional module caching and unit tests └── tests/ ├── common/ # Common test utilities │ └── mod.rs # Test fixtures and helpers ├── integration_tests.rs # Integration tests └── rhai_scripts/ # Test Rhai scripts ├── main.rhai ├── module1.rhai └── module2.rhai ``` ### 2. Factory Implementation The core factory will provide these main functions: 1. **create_engine()** - Creates a basic thread-safe Rhai engine with default configuration 2. **compile_modules(modules_paths, base_path)** - Compiles a list of Rhai modules into a self-contained AST 3. **create_engine_with_modules(modules_paths, base_path)** - Creates an engine with pre-compiled modules ### 3. Error Handling We'll create a custom error type `RhaiFactoryError` that provides detailed information about: - Which module failed to import - The reason for the failure - The path that was attempted - Any nested module import failures ### 4. Module Caching (Optional Enhancement) To improve performance when repeatedly using the same modules: - Implement a module cache that stores compiled ASTs - Use a hash of the module content as the cache key - Provide options to invalidate the cache when modules change ### 5. Thread Safety We'll ensure thread safety by: - Using the `sync` feature of Rhai - Ensuring all our factory methods return thread-safe types - Using appropriate synchronization primitives for any shared state ## Implementation Steps 1. **Setup Project Structure** - Create the directory structure - Set up Cargo.toml with appropriate dependencies - Create initial module files 2. **Implement Basic Factory** - Create the factory struct with configuration options - Implement `create_engine()` method - Add engine configuration options 3. **Implement Module Compilation** - Create the `compile_modules()` method - Implement module resolution logic - Handle recursive module imports 4. **Implement Error Handling** - Create custom error types - Implement detailed error reporting - Add context to error messages 5. **Implement Combined Factory Method** - Create `create_engine_with_modules()` method - Ensure proper error propagation - Add configuration options 6. **Write Tests** - Create test fixtures and helpers - Implement unit tests within each module - Create integration tests - Test thread safety 7. **Documentation** - Add comprehensive documentation - Include examples - Document thread safety guarantees ## Code Outline Here's a sketch of the main components: ### lib.rs ```rust mod factory; mod error; mod module_cache; pub use factory::RhaiFactory; pub use error::RhaiFactoryError; // Re-export commonly used Rhai types pub use rhai::{Engine, AST, Scope, Module}; #[cfg(test)] mod tests { // Unit tests for the library as a whole } ``` ### factory.rs ```rust use rhai::{Engine, AST, Scope, Module}; use std::path::{Path, PathBuf}; use std::sync::Arc; use crate::error::RhaiFactoryError; pub struct RhaiFactory { // Configuration options } impl RhaiFactory { pub fn new() -> Self { // Initialize with default options } pub fn create_engine(&self) -> Engine { // Create a thread-safe engine } pub fn compile_modules>(&self, module_paths: &[P], base_path: Option

) -> Result { // Compile modules into a self-contained AST } pub fn create_engine_with_modules>(&self, module_paths: &[P], base_path: Option

) -> Result<(Engine, AST), RhaiFactoryError> { // Create engine and compile modules } } #[cfg(test)] mod tests { // Unit tests for the factory implementation } ``` ### error.rs ```rust use std::error::Error; use std::fmt; use std::path::PathBuf; #[derive(Debug)] pub struct RhaiFactoryError { module_path: Option, message: String, source: Option>, } impl RhaiFactoryError { pub fn new(message: impl Into) -> Self { // Create a new error } pub fn with_module(mut self, module_path: impl Into) -> Self { // Add module path context } pub fn with_source(mut self, source: impl Error + Send + Sync + 'static) -> Self { // Add source error } } impl fmt::Display for RhaiFactoryError { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { // Format error message with context } } impl Error for RhaiFactoryError { fn source(&self) -> Option<&(dyn Error + 'static)> { // Return source error if any } } ``` ## Testing Strategy ### 1. Unit Tests We'll follow Rust's standard approach of including unit tests in each module: ```rust #[cfg(test)] mod tests { use super::*; // Test fixtures for common setup struct TestFixture { // Common test setup } impl TestFixture { fn new() -> Self { // Initialize test fixture } } #[test] fn test_specific_functionality() { // Test a specific function or behavior } } ``` Key unit test areas: - Factory creation and configuration - Engine creation - Module compilation - Error handling - Module caching - Thread safety ### 2. Integration Tests Integration tests will be placed in the `tests/` directory: ``` tests/ ├── common/ # Common test utilities │ └── mod.rs # Test fixtures and helpers ├── integration_tests.rs # Integration tests └── rhai_scripts/ # Test Rhai scripts ``` The integration tests will focus on: - End-to-end functionality - Module imports and resolution - Thread safety in a real-world context - Error handling with real scripts ### 3. Test Fixtures and Helpers We'll create test fixtures to simplify test setup and reduce code duplication: ```rust // In tests/common/mod.rs pub struct TestFixture { pub factory: RhaiFactory, pub scripts_dir: PathBuf, } impl TestFixture { pub fn new() -> Self { // Initialize test fixture } pub fn script_path(&self, name: &str) -> PathBuf { // Get path to a test script } } ``` ### 4. Example Usage We'll provide example usage in the README.md and documentation: ```rust // Create a factory let factory = RhaiFactory::new(); // Create a thread-safe engine let engine = factory.create_engine(); // Compile modules let ast = factory.compile_modules(&["main.rhai"], Some(Path::new("./scripts")))?; // Use the engine and AST let result: i64 = engine.eval_ast(&ast)?; ``` ## Thread Safety Considerations 1. The Rhai engine with the `sync` feature is `Send + Sync` 2. All factory methods will be thread-safe 3. Any shared state will use appropriate synchronization primitives 4. The compiled AST will be shareable between threads ## Hot Reload Feature ### Overview The hot reload feature will allow the Rhai Factory to automatically detect changes to script files and recompile them without requiring application restart. This is particularly useful for development environments and systems where scripts control behavior that needs to be modified dynamically. As described in the Rhai documentation (_archive/rhai_engine/rhaibook/patterns/hot-reload.md), hot reloading allows scripts to be modified dynamically without re-initializing the host system. The Rhai `Engine` is re-entrant, meaning it's decoupled from scripts, and a new script only needs to be recompiled and the new `AST` replaces the old for new behaviors to be active. ### Architecture Extension ```mermaid graph TD A[RhaiFactory] --> B[HotReloadManager] B --> C[FileWatcher] B --> D[ScriptRegistry] C --> E[File System Events] D --> F[Script Metadata] B --> G[Reload Callbacks] G --> H[Script Consumers] I[ModuleCache] --> B ``` ### Component Details #### 1. Hot Reload Manager The Hot Reload Manager will be responsible for: 1. Tracking which scripts are being watched 2. Detecting changes to script files 3. Recompiling scripts when changes are detected 4. Notifying consumers when scripts have been reloaded This follows the pattern described in the Rhai hot reload documentation, where the system watches for script file changes and recompiles the scripts when changes are detected. #### 2. File Watcher The File Watcher will: 1. Monitor the file system for changes to script files 2. Notify the Hot Reload Manager when changes are detected 3. Support watching individual files or directories This component will implement the "watch for script file change" functionality mentioned in the hot reload documentation. #### 3. Script Registry The Script Registry will: 1. Maintain metadata about watched scripts 2. Track dependencies between scripts 3. Determine which scripts need to be recompiled when a file changes This is important for handling recursive imports and ensuring that all dependent scripts are recompiled when a dependency changes. #### 4. Reload Callbacks The system will provide: 1. A callback mechanism for consumers to be notified when scripts are reloaded 2. Options for synchronous or asynchronous notification This follows the pattern in the hot reload documentation where a callback is provided to handle the reloaded script. ### Implementation Details #### 1. Enhanced RhaiFactory API We'll extend the RhaiFactory with new methods: ```rust impl RhaiFactory { // Existing methods... /// Enable hot reloading for a compiled AST pub fn enable_hot_reload>(&self, ast: Arc>, module_paths: &[P], base_path: Option

, callback: Option> ) -> Result; /// Disable hot reloading for a previously enabled AST pub fn disable_hot_reload(&self, handle: HotReloadHandle); /// Check if any scripts have changed and trigger reloads if necessary pub fn check_for_changes(&self) -> Result; } ``` These methods provide the core functionality needed for hot reloading, following the pattern described in the Rhai documentation. #### 2. Hot Reload Handle We'll create a handle type to manage hot reload sessions: ```rust /// Handle for a hot reload session pub struct HotReloadHandle { id: uuid::Uuid, // Other fields as needed } ``` This handle will be used to identify and manage hot reload sessions. #### 3. Thread-Safe AST Container We'll create a thread-safe container for ASTs that can be updated when scripts change: ```rust /// Thread-safe container for an AST that can be hot reloaded pub struct HotReloadableAST { ast: Arc>, factory: Arc, handle: Option, } impl HotReloadableAST { /// Create a new hot reloadable AST pub fn new(ast: AST, factory: Arc) -> Self; /// Enable hot reloading for this AST pub fn enable_hot_reload>( &mut self, module_paths: &[P], base_path: Option

, callback: Option> ) -> Result<(), RhaiFactoryError>; /// Disable hot reloading for this AST pub fn disable_hot_reload(&mut self); /// Get a reference to the underlying AST pub fn ast(&self) -> &Arc>; } ``` This follows the pattern in the hot reload documentation where the AST is kept with interior mutability using `Rc>`, but adapted for thread safety using `Arc>` as recommended in the documentation for multi-threaded environments. #### 4. Module Cache Extensions We'll extend the ModuleCache to support invalidation when files change: ```rust impl ModuleCache { // Existing methods... /// Invalidate a specific module in the cache pub fn invalidate>(&self, path: P); /// Check if a module in the cache is outdated pub fn is_outdated>(&self, path: P) -> bool; /// Update the cache timestamp for a module pub fn update_timestamp>(&self, path: P); } ``` These methods will help manage the cache when files change, ensuring that the cache is invalidated when necessary. #### 5. File Monitoring We'll implement file monitoring using the `notify` crate: ```rust /// File watcher for hot reloading struct FileWatcher { watcher: notify::RecommendedWatcher, event_receiver: mpsc::Receiver>, watched_paths: HashMap>, } impl FileWatcher { /// Create a new file watcher pub fn new() -> Result; /// Watch a file or directory pub fn watch>(&mut self, path: P, id: uuid::Uuid) -> Result<(), RhaiFactoryError>; /// Stop watching a file or directory pub fn unwatch>(&mut self, path: P, id: uuid::Uuid); /// Check for file changes pub fn check_for_changes(&mut self) -> Vec; } ``` This implements the file watching functionality needed for hot reloading, similar to the "watch for script file change" functionality mentioned in the hot reload documentation. ### Error Handling We'll extend the RhaiFactoryError to include hot reload specific errors: ```rust impl RhaiFactoryError { // Existing methods... /// Create a new hot reload error pub fn hot_reload_error(message: impl Into) -> Self; /// Add file watcher context to the error pub fn with_watcher_context(mut self, context: impl Into) -> Self; } ``` These methods will help provide detailed error information when hot reloading fails. ## Testing Strategy for Hot Reload ### 1. Unit Tests We'll add unit tests for the hot reload functionality: ```rust #[cfg(test)] mod tests { // Existing tests... #[test] fn hot_reload_detects_file_changes() { // Test that file changes are detected } #[test] fn hot_reload_recompiles_changed_scripts() { // Test that scripts are recompiled when changed } #[test] fn hot_reload_updates_ast() { // Test that the AST is updated when scripts change } #[test] fn hot_reload_triggers_callbacks() { // Test that callbacks are triggered when scripts are reloaded } #[test] fn hot_reload_handles_errors() { // Test error handling during hot reload } } ``` These tests will verify that the hot reload functionality works as expected. ### 2. Integration Tests We'll add integration tests for the hot reload functionality: ```rust #[test] fn factory_hot_reloads_scripts() { // Create a temporary script file let temp_dir = tempfile::tempdir().unwrap(); let script_path = temp_dir.path().join("test.rhai"); std::fs::write(&script_path, "40 + 2").unwrap(); // Create a factory and compile the script let factory = Arc::new(RhaiFactory::with_caching()); let ast = factory.compile_modules(&[&script_path], None).unwrap(); let ast = Arc::new(RwLock::new(ast)); // Enable hot reloading let reload_detected = Arc::new(AtomicBool::new(false)); let reload_detected_clone = reload_detected.clone(); let callback = Box::new(move || { reload_detected_clone.store(true, Ordering::SeqCst); }); let handle = factory.enable_hot_reload( ast.clone(), &[&script_path], None, Some(callback) ).unwrap(); // Modify the script std::fs::write(&script_path, "50 + 10").unwrap(); // Wait for the file system to register the change std::thread::sleep(std::time::Duration::from_millis(100)); // Check for changes factory.check_for_changes().unwrap(); // Verify the callback was triggered assert!(reload_detected.load(Ordering::SeqCst)); // Verify the AST was updated let engine = factory.create_engine(); let result: i64 = engine.eval_ast(&ast.read().unwrap()).unwrap(); assert_eq!(result, 60); // Disable hot reloading factory.disable_hot_reload(handle); } ``` This test verifies that the hot reload functionality works end-to-end, following the pattern described in the Rhai documentation. ### 3. Thread Safety Tests We'll add tests to verify thread safety: ```rust #[test] fn hot_reload_is_thread_safe() { // Create a temporary script file let temp_dir = tempfile::tempdir().unwrap(); let script_path = temp_dir.path().join("test.rhai"); std::fs::write(&script_path, "40 + 2").unwrap(); // Create a factory and compile the script let factory = Arc::new(RhaiFactory::with_caching()); let ast = factory.compile_modules(&[&script_path], None).unwrap(); let ast = Arc::new(RwLock::new(ast)); // Enable hot reloading let handle = factory.enable_hot_reload( ast.clone(), &[&script_path], None, None ).unwrap(); // Create threads that read the AST while it's being modified let threads: Vec<_> = (0..10).map(|_| { let factory_clone = factory.clone(); let ast_clone = ast.clone(); std::thread::spawn(move || { for _ in 0..100 { let engine = factory_clone.create_engine(); let _: Result = engine.eval_ast(&ast_clone.read().unwrap()); std::thread::yield_now(); } }) }).collect(); // Modify the script multiple times for i in 0..5 { std::fs::write(&script_path, format!("40 + {}", i)).unwrap(); std::thread::sleep(std::time::Duration::from_millis(10)); factory.check_for_changes().unwrap(); } // Wait for all threads to complete for thread in threads { thread.join().unwrap(); } // Disable hot reloading factory.disable_hot_reload(handle); } ``` This test verifies that the hot reload functionality is thread-safe, following the recommendation in the Rhai documentation to use `Arc`, `RwLock`, and the `sync` feature for multi-threaded environments. ## Usage Examples ### Basic Usage ```rust // Create a factory with caching enabled let factory = Arc::new(RhaiFactory::with_caching()); // Compile the initial script let ast = factory.compile_modules(&["main.rhai"], Some("scripts")).unwrap(); let ast = Arc::new(RwLock::new(ast)); // Enable hot reloading let handle = factory.enable_hot_reload( ast.clone(), &["main.rhai"], Some("scripts"), Some(Box::new(|| println!("Script reloaded!"))) ).unwrap(); // Create an engine and use the AST let engine = factory.create_engine(); // In your application loop loop { // Check for script changes if factory.check_for_changes().unwrap() { println!("Scripts were reloaded"); } // Use the latest version of the script let result: i64 = engine.eval_ast(&ast.read().unwrap()).unwrap(); // Do something with the result std::thread::sleep(std::time::Duration::from_secs(1)); } // When done factory.disable_hot_reload(handle); ``` This example shows how to use the hot reload functionality in a basic application, following the pattern described in the Rhai documentation. ### With HotReloadableAST ```rust // Create a factory with caching enabled let factory = Arc::new(RhaiFactory::with_caching()); // Compile the initial script let ast = factory.compile_modules(&["main.rhai"], Some("scripts")).unwrap(); // Create a hot reloadable AST let mut hot_ast = HotReloadableAST::new(ast, factory.clone()); // Enable hot reloading hot_ast.enable_hot_reload( &["main.rhai"], Some("scripts"), Some(Box::new(|| println!("Script reloaded!"))) ).unwrap(); // Create an engine and use the AST let engine = factory.create_engine(); // In your application loop loop { // Check for script changes factory.check_for_changes().unwrap(); // Use the latest version of the script let result: i64 = engine.eval_ast(&hot_ast.ast().read().unwrap()).unwrap(); // Do something with the result std::thread::sleep(std::time::Duration::from_secs(1)); } // When done hot_ast.disable_hot_reload(); ``` This example shows how to use the `HotReloadableAST` wrapper for a more convenient API. ## Implementation Steps 1. **Add Dependencies** - Add the `notify` crate for file system monitoring - Add the `uuid` crate for unique identifiers 2. **Create Hot Reload Manager** - Implement the `HotReloadManager` struct - Implement the `FileWatcher` struct - Implement the `ScriptRegistry` struct 3. **Extend RhaiFactory** - Add hot reload methods to `RhaiFactory` - Implement the `HotReloadHandle` struct - Implement the `HotReloadableAST` struct 4. **Extend ModuleCache** - Add methods for cache invalidation - Add timestamp tracking for modules 5. **Implement Error Handling** - Extend `RhaiFactoryError` for hot reload errors 6. **Write Tests** - Implement unit tests - Implement integration tests - Implement thread safety tests 7. **Update Documentation** - Add hot reload documentation to README - Add examples and usage guidelines ## Considerations and Trade-offs 1. **Performance Impact** - File system monitoring can have a performance impact - We'll provide options to control the frequency of checks 2. **Memory Usage** - Keeping multiple versions of scripts in memory can increase memory usage - We'll provide options to control caching behavior 3. **Thread Safety** - Hot reloading in a multi-threaded environment requires careful synchronization - We'll use `RwLock` to allow multiple readers but exclusive writers - This follows the recommendation in the Rhai documentation to use `Arc`, `RwLock`, and the `sync` feature for multi-threaded environments 4. **Error Handling** - Script compilation errors during hot reload need to be handled gracefully - We'll provide options to keep the old script or propagate errors 5. **Dependency Tracking** - Changes to imported modules need to trigger recompilation of dependent modules - We'll implement dependency tracking in the `ScriptRegistry` ## References - Rhai Documentation: `_archive/rhai_engine/rhaibook/` - Hot Reload Pattern: `_archive/rhai_engine/rhaibook/patterns/hot-reload.md` - Rhai Engine: `rhai::Engine` - Rhai AST: `rhai::AST` - Rhai Module: `rhai::Module` - Rhai Scope: `rhai::Scope` - Rhai Sync Feature: `sync` ## Conclusion The hot reload feature will enhance the `rhai_factory` module by allowing scripts to be modified dynamically without requiring application restart. This will improve the development experience and enable more flexible runtime behavior. By following the patterns described in the Rhai documentation, we can implement a robust hot reload feature that is thread-safe and provides a good developer experience. ## Tera Engine Factory with Hot Reloadable Rhai Integration ### Overview We'll create a `TeraFactory` module that provides a factory for creating Tera template engines with integrated Rhai scripting support. The factory will: 1. Create Tera engines with specified template directories 2. Integrate with the hot reloadable Rhai AST from the `RhaiFactory` 3. Allow Rhai functions to be called from Tera templates 4. Automatically update available functions when Rhai scripts are hot reloaded ### Architecture Extension ```mermaid graph TD A[TeraFactory] --> B[create_tera_engine] A --> C[create_tera_with_rhai] C --> D[Tera Engine with Rhai Functions] E[RhaiFactory] --> F[HotReloadableAST] F --> C G[RhaiFunctionAdapter] --> D H[Template Directories] --> B ``` ### Component Details #### 1. TeraFactory Module Structure ``` tera_factory/ ├── Cargo.toml # Dependencies including tera and rhai_factory └── src/ ├── lib.rs # Main module exports and unit tests ├── factory.rs # Factory implementation and unit tests ├── error.rs # Custom error types and unit tests └── function_adapter.rs # Rhai function adapter for Tera ``` #### 2. TeraFactory Implementation The core factory will provide these main functions: 1. **create_tera_engine(template_dirs)** - Creates a basic Tera engine with the specified template directories 2. **create_tera_with_rhai(template_dirs, hot_ast)** - Creates a Tera engine with Rhai function integration using a hot reloadable AST #### 3. RhaiFunctionAdapter We'll enhance the existing `RhaiFunctionAdapter` to work with the hot reloadable AST: ```rust /// Thread-safe adapter to use Rhai functions in Tera templates with hot reload support pub struct RhaiFunctionAdapter { fn_name: String, hot_ast: Arc>, } impl TeraFunction for RhaiFunctionAdapter { fn call(&self, args: &HashMap) -> TeraResult { // Convert args from Tera into Rhai's Dynamic let mut scope = Scope::new(); for (key, value) in args { // Convert Tera value to Rhai Dynamic let dynamic = convert_tera_to_rhai(value); scope.push_dynamic(key.clone(), dynamic); } // Create a new engine for each call let engine = Engine::new(); // Get a read lock on the AST let ast = self.hot_ast.read().unwrap(); // Call the function using the latest AST let result = engine .call_fn::(&mut scope, &ast, &self.fn_name, ()) .map_err(|e| tera::Error::msg(format!("Rhai error: {}", e)))?; // Convert Rhai result to Tera value let tera_value = convert_rhai_to_tera(&result); Ok(tera_value) } } ``` #### 4. TeraFactory API ```rust pub struct TeraFactory { // Configuration options } impl TeraFactory { /// Create a new TeraFactory with default settings pub fn new() -> Self { Self {} } /// Create a Tera engine with the specified template directories pub fn create_tera_engine>(&self, template_dirs: &[P]) -> Result { // Create a Tera engine with the specified template directories } /// Create a Tera engine with Rhai function integration pub fn create_tera_with_rhai>( &self, template_dirs: &[P], hot_ast: Arc> ) -> Result { // Create a Tera engine with Rhai function integration } } ``` ### Implementation Details #### 1. Creating a Tera Engine ```rust impl TeraFactory { pub fn create_tera_engine>(&self, template_dirs: &[P]) -> Result { let mut tera = Tera::default(); // Add templates from each directory for template_dir in template_dirs { let pattern = format!("{}/**/*.html", template_dir.as_ref().display()); match Tera::parse(&pattern) { Ok(parsed_tera) => { tera.extend(&parsed_tera).map_err(|e| {