merge branches and document

README.md

@@ -1,73 +1,184 @@
 # SAL (System Abstraction Layer)
 
-A Rust library that provides a unified interface for interacting with operating system features across different platforms. It abstracts away platform-specific details, allowing developers to write cross-platform code with ease.
+**Version: 0.1.0**
 
-## Features
+SAL is a comprehensive Rust library designed to provide a unified and simplified interface for a wide array of system-level operations and interactions. It abstracts platform-specific details, enabling developers to write robust, cross-platform code with greater ease. SAL also includes `herodo`, a powerful command-line tool for executing Rhai scripts that leverage SAL's capabilities for automation and system management tasks.
 
-- **File System Operations**: Simplified file and directory management
-- **Process Management**: Create, monitor, and control processes
-- **System Information**: Access system details and metrics
-- **Git Integration**: Interface with Git repositories
-- **Redis Client**: Robust Redis connection management and command execution
-- **Text Processing**: Utilities for text manipulation and formatting
+## Core Features
 
-## Modules
+SAL offers a broad spectrum of functionalities, including:
 
-### Redis Client
+- **System Operations**: File and directory management, environment variable access, system information retrieval, and OS-specific commands.
+- **Process Management**: Create, monitor, control, and interact with system processes.
+- **Containerization Tools**: 
+    - Integration with **Buildah** for building OCI/Docker-compatible container images.
+    - Integration with **nerdctl** for managing containers (run, stop, list, build, etc.).
+- **Version Control**: Programmatic interaction with Git repositories (clone, commit, push, pull, status, etc.).
+- **Database Clients**:
+    - **Redis**: Robust client for interacting with Redis servers.
+    - **PostgreSQL**: Client for executing queries and managing PostgreSQL databases.
+- **Scripting Engine**: In-built support for the **Rhai** scripting language, allowing SAL functionalities to be scripted and automated, primarily through the `herodo` tool.
+- **Networking & Services**:
+    - **Mycelium**: Tools for Mycelium network peer management and message passing.
+    - **Zinit**: Client for interacting with the Zinit process supervision system.
+    - **RFS (Remote/Virtual Filesystem)**: Mount, manage, pack, and unpack various types of filesystems (local, SSH, S3, WebDAV).
+- **Text Processing**: A suite of utilities for text manipulation, formatting, and regular expressions.
+- **Cryptography (`vault`)**: Functions for common cryptographic operations.
 
-The Redis client module provides a robust wrapper around the Redis client library for Rust, offering:
+## `herodo`: The SAL Scripting Tool
 
-- Automatic connection management and reconnection
-- Support for both Unix socket and TCP connections
-- Database selection via environment variables
-- Thread-safe global client instance
-- Simple command execution interface
+`herodo` is a command-line utility bundled with SAL that executes Rhai scripts. It empowers users to automate tasks and orchestrate complex workflows by leveraging SAL's diverse modules directly from scripts.
 
-[View Redis Client Documentation](src/redisclient/README.md)
+### Usage
 
-### OS Module
+```bash
+herodo -p <path_to_script.rhai>
+# or
+herodo -p <path_to_directory_with_scripts/>
+```
 
-Provides platform-independent interfaces for operating system functionality.
+If a directory is provided, `herodo` will execute all `.rhai` scripts within that directory (and its subdirectories) in alphabetical order.
 
-### Git Module
+### Scriptable SAL Modules via `herodo`
 
-Tools for interacting with Git repositories programmatically.
+The following SAL modules and functionalities are exposed to the Rhai scripting environment through `herodo`:
 
-### Process Module
+- **OS (`os`)**: Comprehensive file system operations, file downloading & installation, and system package management. [Detailed OS Module Documentation](src/os/README.md)
+- **Process (`process`)**: Robust command and script execution, plus process management (listing, finding, killing, checking command existence). [Detailed Process Module Documentation](src/process/README.md)
+- **Buildah (`buildah`)**: OCI/Docker image building functions. [Detailed Buildah Module Documentation](src/virt/buildah/README.md)
+- **nerdctl (`nerdctl`)**: Container lifecycle management (`nerdctl_run`, `nerdctl_stop`, `nerdctl_images`, `nerdctl_image_build`, etc.). [Detailed Nerdctl Module Documentation](src/virt/nerdctl/README.md)
+- **Git (`git`)**: High-level repository management and generic Git command execution with Redis-backed authentication (clone, pull, push, commit, etc.). [Detailed Git Module Documentation](src/git/README.md)
+- **Zinit (`zinit_client`)**: Client for Zinit process supervisor (service management, logs). [Detailed Zinit Client Module Documentation](src/zinit_client/README.md)
+- **Mycelium (`mycelium`)**: Client for Mycelium decentralized networking API (node info, peer management, messaging). [Detailed Mycelium Module Documentation](src/mycelium/README.md)
+- **Text (`text`)**: String manipulation, prefixing, path/name fixing, text replacement, and templating. [Detailed Text Module Documentation](src/text/README.md)
+- **RFS (`rfs`)**: Mount various filesystems (local, SSH, S3, etc.), pack/unpack filesystem layers. [Detailed RFS Module Documentation](src/virt/rfs/README.md)
+- **Cryptography (`crypto` from `vault`)**: Encryption, decryption, hashing, etc.
+- **Redis Client (`redis`)**: Execute Redis commands (`redis_get`, `redis_set`, `redis_execute`, etc.).
+- **PostgreSQL Client (`postgres`)**: Execute SQL queries against PostgreSQL databases.
 
-Utilities for process creation, monitoring, and management.
+### Example `herodo` Rhai Script
 
-### Text Module
+```rhai
+// file: /opt/scripts/example_task.rhai
 
-Text processing utilities for common operations.
+// OS operations
+println("Checking for /tmp/my_app_data...");
+if !exist("/tmp/my_app_data") {
+    mkdir("/tmp/my_app_data");
+    println("Created directory /tmp/my_app_data");
+}
 
-## Usage
+// Redis operations
+println("Setting Redis key 'app_status' to 'running'");
+redis_set("app_status", "running");
+let status = redis_get("app_status");
+println("Current app_status from Redis: " + status);
 
-Add this to your `Cargo.toml`:
+// Process execution
+println("Listing files in /tmp:");
+let output = run("ls -la /tmp");
+println(output.stdout);
+
+println("Script finished.");
+```
+
+Run with: `herodo -p /opt/scripts/example_task.rhai`
+
+For more examples, check the `examples/` and `rhai_tests/` directories in this repository.
+
+## Using SAL as a Rust Library
+
+Add SAL as a dependency to your `Cargo.toml`:
 
 ```toml
 [dependencies]
-sal = "0.1.0"
+sal = "0.1.0" # Or the latest version
 ```
 
-Basic example:
+### Rust Example: Using Redis Client
 
 ```rust
-use sal::redisclient::execute;
-use redis::cmd;
+use sal::redisclient::{get_global_client, execute_cmd_with_args};
+use redis::RedisResult;
+
+async fn example_redis_interaction() -> RedisResult<()> {
+    // Get a connection from the global pool
+    let mut conn = get_global_client().await?.get_async_connection().await?;
+
+    // Set a value
+    execute_cmd_with_args(&mut conn, "SET", vec!["my_key", "my_value"]).await?;
+    println!("Set 'my_key' to 'my_value'");
+
+    // Get a value
+    let value: String = execute_cmd_with_args(&mut conn, "GET", vec!["my_key"]).await?;
+    println!("Retrieved value for 'my_key': {}", value);
 
-fn main() -> redis::RedisResult<()> {
-    // Execute a Redis command
-    let mut cmd = redis::cmd("SET");
-    cmd.arg("example_key").arg("example_value");
-    execute(&mut cmd)?;
-    
-    // Retrieve the value
-    let mut get_cmd = redis::cmd("GET");
-    get_cmd.arg("example_key");
-    let value: String = execute(&mut get_cmd)?;
-    println!("Value: {}", value);
-    
     Ok(())
 }
+
+#[tokio::main]
+asynchronous fn main() {
+    if let Err(e) = example_redis_interaction().await {
+        eprintln!("Redis Error: {}", e);
+    }
+}
 ```
+*(Note: The Redis client API might have evolved; please refer to `src/redisclient/mod.rs` and its documentation for the most current usage.)*
+
+## Modules Overview (Rust Library)
+
+SAL is organized into several modules, each providing specific functionalities:
+
+- **`sal::os`**: Core OS interactions, file system operations, environment access.
+- **`sal::process`**: Process creation, management, and control.
+- **`sal::git`**: Git repository management.
+- **`sal::redisclient`**: Client for Redis database interactions. (See also `src/redisclient/README.md`)
+- **`sal::postgresclient`**: Client for PostgreSQL database interactions.
+- **`sal::rhai`**: Integration layer for the Rhai scripting engine, used by `herodo`.
+- **`sal::text`**: Utilities for text processing and manipulation.
+- **`sal::vault`**: Cryptographic functions.
+- **`sal::virt`**: Virtualization-related utilities, including `rfs` for remote/virtual filesystems.
+- **`sal::mycelium`**: Client for Mycelium network operations.
+- **`sal::zinit_client`**: Client for Zinit process supervisor.
+- **`sal::cmd`**: Implements the command logic for `herodo`.
+- **(Internal integrations for `buildah`, `nerdctl` primarily exposed via Rhai)**
+
+## Building SAL
+
+Build the library and the `herodo` binary using Cargo:
+
+```bash
+cargo build
+```
+
+For a release build:
+
+```bash
+cargo build --release
+```
+
+The `herodo` executable will be located at `target/debug/herodo` or `target/release/herodo`.
+
+The `build_herodo.sh` script is also available for building `herodo`.
+
+## Running Tests
+
+Run Rust unit and integration tests:
+
+```bash
+cargo test
+```
+
+Run Rhai script tests (which exercise `herodo` and SAL's scripted functionalities):
+
+```bash
+./run_rhai_tests.sh
+```
+
+## License
+
+SAL is licensed under the Apache License 2.0. See the [LICENSE](LICENSE) file for details.
+
+## Contributing
+
+Contributions are welcome! Please feel free to submit pull requests or open issues.