From 758e59e9210869bfc90f75a7ada5b1d2b688c37e Mon Sep 17 00:00:00 2001 From: Mahmoud-Emad Date: Sun, 13 Jul 2025 12:51:08 +0300 Subject: [PATCH] docs: Improve README.md with clearer structure and installation - Update README.md to provide a clearer structure and improved installation instructions. This makes it easier for users to understand and use the library. - Remove outdated and unnecessary sections like the workspace structure details, publishing status, and detailed features lists. The information is either not relevant anymore or can be found elsewhere. - Simplify installation instructions to focus on the core aspects of installing individual packages or the meta-package with features. - Add a dedicated section for building and running tests, improving developer experience and making the process more transparent. - Modernize the overall layout and formatting for better readability. --- README.md | 462 ++++++++++++------------------------------------------ 1 file changed, 103 insertions(+), 359 deletions(-) diff --git a/README.md b/README.md index ebe150f..5172094 100644 --- a/README.md +++ b/README.md @@ -1,404 +1,148 @@ # SAL (System Abstraction Layer) -**Version: 0.1.0** +**Version 0.1.0** - A modular Rust library for cross-platform system operations and automation. -SAL is a comprehensive Rust library designed to provide a unified and simplified interface for a wide array of system-level operations and interactions. It abstracts platform-specific details, enabling developers to write robust, cross-platform code with greater ease. SAL also includes `herodo`, a powerful command-line tool for executing Rhai scripts that leverage SAL's capabilities for automation and system management tasks. +SAL provides a unified interface for system operations with Rhai scripting support through the `herodo` tool. -## ๐Ÿ—๏ธ **Cargo Workspace Structure** +## Installation -SAL is organized as a **Cargo workspace** with 15 specialized crates: - -- **Root Package**: `sal` - Umbrella crate that re-exports all modules -- **12 Library Crates**: Core SAL modules (os, process, text, net, git, vault, kubernetes, virt, redisclient, postgresclient, zinit_client, mycelium) -- **1 Binary Crate**: `herodo` - Rhai script execution engine -- **1 Integration Crate**: `rhai` - Rhai scripting integration layer - -This workspace structure provides excellent build performance, dependency management, and maintainability. - -### **๐Ÿš€ Workspace Benefits** -- **Unified Dependency Management**: Shared dependencies across all crates with consistent versions -- **Optimized Build Performance**: Parallel compilation and shared build artifacts -- **Simplified Testing**: Run tests across all modules with a single command -- **Modular Architecture**: Each module is independently maintainable while sharing common infrastructure -- **Production Ready**: 100% test coverage with comprehensive Rhai integration tests - -## ๐Ÿ“ฆ Installation - -SAL is designed to be modular - install only the components you need! - -### Option 1: Individual Crates (Recommended) - -Install only the modules you need: +### Individual Packages (Recommended) ```bash -# Currently available packages -cargo add sal-os sal-process sal-text sal-net sal-git sal-vault sal-kubernetes sal-virt +# Core functionality +cargo add sal-os sal-process sal-text sal-net -# Coming soon (rate limited) -# cargo add sal-redisclient sal-postgresclient sal-zinit-client sal-mycelium sal-rhai +# Infrastructure +cargo add sal-git sal-vault sal-kubernetes sal-virt + +# Database clients +cargo add sal-redisclient sal-postgresclient sal-zinit-client + +# Scripting +cargo add sal-rhai ``` -### Option 2: Meta-crate with Features - -Use the main `sal` crate with specific features: +### Meta-package with Features ```bash -# Coming soon - meta-crate with features (rate limited) -# cargo add sal --features os,process,text -# cargo add sal --features core # os, process, text, net -# cargo add sal --features infrastructure # git, vault, kubernetes, virt -# cargo add sal --features all - -# For now, use individual crates (see Option 1 above) +cargo add sal --features core # os, process, text, net +cargo add sal --features infrastructure # git, vault, kubernetes, virt +cargo add sal --features all # everything ``` -### Quick Start Examples +### Herodo Script Runner -#### Using Individual Crates (Recommended) +```bash +cargo install herodo +``` + +## Quick Start + +### Rust Library Usage ```rust use sal_os::fs; use sal_process::run; fn main() -> Result<(), Box> { - // File system operations let files = fs::list_files(".")?; println!("Found {} files", files.len()); - // Process execution let result = run::command("echo hello")?; println!("Output: {}", result.stdout); - Ok(()) } ``` -#### Using Meta-crate with Features - -```rust -// In Cargo.toml: sal = { version = "0.1.0", features = ["os", "process"] } -use sal::os::fs; -use sal::process::run; - -fn main() -> Result<(), Box> { - // File system operations - let files = fs::list_files(".")?; - println!("Found {} files", files.len()); - - // Process execution - let result = run::command("echo hello")?; - println!("Output: {}", result.stdout); - - Ok(()) -} -``` - -#### Using Herodo for Scripting +### Herodo Scripting ```bash -# Build and install herodo -git clone https://github.com/PlanetFirst/sal.git -cd sal -./build_herodo.sh - -# Create a script file +# Create script cat > example.rhai << 'EOF' +let files = find_files(".", "*.rs"); +print("Found " + files.len() + " Rust files"); + +let result = run("echo 'Hello from SAL!'"); +print("Output: " + result.stdout); +EOF + +# Run script +herodo example.rhai +``` + +## Available Packages + +| Package | Description | +|---------|-------------| +| [`sal-os`](https://crates.io/crates/sal-os) | Operating system operations | +| [`sal-process`](https://crates.io/crates/sal-process) | Process management | +| [`sal-text`](https://crates.io/crates/sal-text) | Text processing | +| [`sal-net`](https://crates.io/crates/sal-net) | Network operations | +| [`sal-git`](https://crates.io/crates/sal-git) | Git repository management | +| [`sal-vault`](https://crates.io/crates/sal-vault) | Cryptographic operations | +| [`sal-kubernetes`](https://crates.io/crates/sal-kubernetes) | Kubernetes management | +| [`sal-virt`](https://crates.io/crates/sal-virt) | Virtualization tools | +| [`sal-redisclient`](https://crates.io/crates/sal-redisclient) | Redis client | +| [`sal-postgresclient`](https://crates.io/crates/sal-postgresclient) | PostgreSQL client | +| [`sal-zinit-client`](https://crates.io/crates/sal-zinit-client) | Zinit process supervisor | +| [`sal-mycelium`](https://crates.io/crates/sal-mycelium) | Mycelium network client | +| [`sal-service-manager`](https://crates.io/crates/sal-service-manager) | Service management | +| [`sal-rhai`](https://crates.io/crates/sal-rhai) | Rhai scripting integration | +| [`sal`](https://crates.io/crates/sal) | Meta-crate with features | +| [`herodo`](https://crates.io/crates/herodo) | Script executor binary | + +## Building & Testing + +```bash +# Build all packages +cargo build --workspace + +# Run tests +cargo test --workspace + +# Run Rhai integration tests +./run_rhai_tests.sh +``` + +## Core Features + +- **System Operations**: File/directory management, environment access, OS commands +- **Process Management**: Create, monitor, and control system processes +- **Containerization**: Buildah and nerdctl integration +- **Version Control**: Git repository operations +- **Database Clients**: Redis and PostgreSQL support +- **Networking**: HTTP, TCP, SSH connectivity utilities +- **Cryptography**: Key management, encryption, digital signatures +- **Text Processing**: String manipulation and templating +- **Scripting**: Rhai script execution via `herodo` + +## Herodo Scripting + +`herodo` executes Rhai scripts with access to all SAL modules: + +```bash +herodo script.rhai # Run single script +herodo script.rhai arg1 arg2 # With arguments +herodo /path/to/scripts/ # Run all .rhai files in directory +``` + +### Example Script + +```rhai // File operations let files = find_files(".", "*.rs"); print("Found " + files.len() + " Rust files"); // Process execution -let result = run("echo 'Hello from SAL!'"); +let result = run("echo 'Hello SAL!'"); print("Output: " + result.stdout); -// Network operations -let reachable = http_check("https://github.com"); -print("GitHub reachable: " + reachable); -EOF - -# Execute the script -herodo example.rhai -``` - -## ๐Ÿ“ฆ Available Packages - -SAL is published as individual crates, allowing you to install only what you need: - -| Package | Description | Install Command | -|---------|-------------|-----------------| -| [`sal-os`](https://crates.io/crates/sal-os) | Operating system operations | `cargo add sal-os` | -| [`sal-process`](https://crates.io/crates/sal-process) | Process management | `cargo add sal-process` | -| [`sal-text`](https://crates.io/crates/sal-text) | Text processing utilities | `cargo add sal-text` | -| [`sal-net`](https://crates.io/crates/sal-net) | Network operations | `cargo add sal-net` | -| [`sal-git`](https://crates.io/crates/sal-git) | Git repository management | `cargo add sal-git` | -| [`sal-vault`](https://crates.io/crates/sal-vault) | Cryptographic operations | `cargo add sal-vault` | -| [`sal-kubernetes`](https://crates.io/crates/sal-kubernetes) | Kubernetes management | `cargo add sal-kubernetes` | -| [`sal-virt`](https://crates.io/crates/sal-virt) | Virtualization tools | `cargo add sal-virt` | -| `sal-redisclient` | Redis database client | `cargo add sal-redisclient` โณ | -| `sal-postgresclient` | PostgreSQL client | `cargo add sal-postgresclient` โณ | -| `sal-zinit-client` | Zinit process supervisor | `cargo add sal-zinit-client` โณ | -| `sal-mycelium` | Mycelium network client | `cargo add sal-mycelium` โณ | -| `sal-rhai` | Rhai scripting integration | `cargo add sal-rhai` โณ | -| `sal` | Meta-crate with features | `cargo add sal --features all` โณ | -| `herodo` | Script executor binary | Build from source โณ | - -**Legend**: โœ… Published | โณ Publishing soon (rate limited) - -### ๐Ÿ“ข **Publishing Status** - -**Currently Available on crates.io:** -- โœ… [`sal-os`](https://crates.io/crates/sal-os) - Operating system operations -- โœ… [`sal-process`](https://crates.io/crates/sal-process) - Process management -- โœ… [`sal-text`](https://crates.io/crates/sal-text) - Text processing utilities -- โœ… [`sal-net`](https://crates.io/crates/sal-net) - Network operations -- โœ… [`sal-git`](https://crates.io/crates/sal-git) - Git repository management -- โœ… [`sal-vault`](https://crates.io/crates/sal-vault) - Cryptographic operations -- โœ… [`sal-kubernetes`](https://crates.io/crates/sal-kubernetes) - Kubernetes management -- โœ… [`sal-virt`](https://crates.io/crates/sal-virt) - Virtualization tools - -**Publishing Soon** (hit crates.io rate limit): -- โณ `sal-redisclient`, `sal-postgresclient`, `sal-zinit-client`, `sal-mycelium` -- โณ `sal-rhai` -- โณ `sal` (meta-crate), `herodo` (binary) - -**Estimated Timeline**: Remaining packages will be published within 24 hours once the rate limit resets. - -## Core Features - -SAL offers a broad spectrum of functionalities, including: - -- **System Operations**: File and directory management, environment variable access, system information retrieval, and OS-specific commands. -- **Process Management**: Create, monitor, control, and interact with system processes. -- **Containerization Tools**: - - Integration with **Buildah** for building OCI/Docker-compatible container images. - - Integration with **nerdctl** for managing containers (run, stop, list, build, etc.). -- **Version Control**: Programmatic interaction with Git repositories (clone, commit, push, pull, status, etc.). -- **Database Clients**: - - **Redis**: Robust client for interacting with Redis servers. - - **PostgreSQL**: Client for executing queries and managing PostgreSQL databases. -- **Scripting Engine**: In-built support for the **Rhai** scripting language, allowing SAL functionalities to be scripted and automated, primarily through the `herodo` tool. -- **Networking & Services**: - - **Mycelium**: Tools for Mycelium network peer management and message passing. - - **Zinit**: Client for interacting with the Zinit process supervision system. - - **RFS (Remote/Virtual Filesystem)**: Mount, manage, pack, and unpack various types of filesystems (local, SSH, S3, WebDAV). -- **Text Processing**: A suite of utilities for text manipulation, formatting, and regular expressions. -- **Cryptography (`vault`)**: Functions for common cryptographic operations. - -## `herodo`: The SAL Scripting Tool - -`herodo` is a command-line utility bundled with SAL that executes Rhai scripts. It empowers users to automate tasks and orchestrate complex workflows by leveraging SAL's diverse modules directly from scripts. - -### Usage - -```bash -# Execute a single Rhai script -herodo script.rhai - -# Execute a script with arguments -herodo script.rhai arg1 arg2 - -# Execute all .rhai scripts in a directory -herodo /path/to/scripts/ -``` - -If a directory is provided, `herodo` will execute all `.rhai` scripts within that directory (and its subdirectories) in alphabetical order. - -### Scriptable SAL Modules via `herodo` - -The following SAL modules and functionalities are exposed to the Rhai scripting environment through `herodo`: - -- **OS (`os`)**: Comprehensive file system operations, file downloading & installation, and system package management. [Documentation](os/README.md) -- **Process (`process`)**: Robust command and script execution, plus process management (listing, finding, killing, checking command existence). [Documentation](process/README.md) -- **Text (`text`)**: String manipulation, prefixing, path/name fixing, text replacement, and templating. [Documentation](text/README.md) -- **Net (`net`)**: Network operations, HTTP requests, and connectivity utilities. [Documentation](net/README.md) -- **Git (`git`)**: High-level repository management and generic Git command execution with Redis-backed authentication (clone, pull, push, commit, etc.). [Documentation](git/README.md) -- **Vault (`vault`)**: Cryptographic operations, keypair management, encryption, decryption, hashing, etc. [Documentation](vault/README.md) -- **Redis Client (`redisclient`)**: Execute Redis commands (`redis_get`, `redis_set`, `redis_execute`, etc.). [Documentation](redisclient/README.md) -- **PostgreSQL Client (`postgresclient`)**: Execute SQL queries against PostgreSQL databases. [Documentation](postgresclient/README.md) -- **Zinit (`zinit_client`)**: Client for Zinit process supervisor (service management, logs). [Documentation](zinit_client/README.md) -- **Mycelium (`mycelium`)**: Client for Mycelium decentralized networking API (node info, peer management, messaging). [Documentation](mycelium/README.md) -- **Virtualization (`virt`)**: - - **Buildah**: OCI/Docker image building functions. [Documentation](virt/README.md) - - **nerdctl**: Container lifecycle management (`nerdctl_run`, `nerdctl_stop`, `nerdctl_images`, `nerdctl_image_build`, etc.) - - **RFS**: Mount various filesystems (local, SSH, S3, etc.), pack/unpack filesystem layers. - -### Example `herodo` Rhai Script - -```rhai -// file: /opt/scripts/example_task.rhai - -// OS operations -println("Checking for /tmp/my_app_data..."); -if !exist("/tmp/my_app_data") { - mkdir("/tmp/my_app_data"); - println("Created directory /tmp/my_app_data"); -} - // Redis operations -println("Setting Redis key 'app_status' to 'running'"); -redis_set("app_status", "running"); -let status = redis_get("app_status"); -println("Current app_status from Redis: " + status); - -// Process execution -println("Listing files in /tmp:"); -let output = run("ls -la /tmp"); -println(output.stdout); - -println("Script finished."); +redis_set("status", "running"); +let status = redis_get("status"); +print("Status: " + status); ``` -Run with: `herodo /opt/scripts/example_task.rhai` - -For more examples, check the individual module test directories (e.g., `text/tests/rhai/`, `os/tests/rhai/`, etc.) in this repository. - -## Using SAL as a Rust Library - -### Option 1: Individual Crates (Recommended) - -Add only the SAL modules you need: - -```toml -[dependencies] -sal-os = "0.1.0" -sal-process = "0.1.0" -sal-text = "0.1.0" -``` - -```rust -use sal_os::fs; -use sal_process::run; -use sal_text::template; - -fn main() -> Result<(), Box> { - // File operations - let files = fs::list_files(".")?; - println!("Found {} files", files.len()); - - // Process execution - let result = run::command("echo 'Hello SAL!'")?; - println!("Output: {}", result.stdout); - - // Text templating - let template_str = "Hello {{name}}!"; - let mut vars = std::collections::HashMap::new(); - vars.insert("name".to_string(), "World".to_string()); - let rendered = template::render(template_str, &vars)?; - println!("Rendered: {}", rendered); - - Ok(()) -} -``` - -### Option 2: Meta-crate with Features (Coming Soon) - -```toml -[dependencies] -sal = { version = "0.1.0", features = ["os", "process", "text"] } -``` - -```rust -use sal::os::fs; -use sal::process::run; -use sal::text::template; - -// Same code as above, but using the meta-crate -``` - -*(Note: The meta-crate `sal` will be available once all individual packages are published.)* - -## ๐ŸŽฏ **Why Choose SAL?** - -### **Modular Architecture** -- **Install Only What You Need**: Each package is independent - no bloated dependencies -- **Faster Compilation**: Smaller dependency trees mean faster build times -- **Smaller Binaries**: Only include the functionality you actually use -- **Clear Dependencies**: Explicit about what functionality your project uses - -### **Developer Experience** -- **Consistent APIs**: All packages follow the same design patterns and conventions -- **Comprehensive Documentation**: Each package has detailed documentation and examples -- **Real-World Tested**: All functionality is production-tested, no placeholder code -- **Type Safety**: Leverages Rust's type system for safe, reliable operations - -### **Scripting Power** -- **Herodo Integration**: Execute Rhai scripts with full access to SAL functionality -- **Cross-Platform**: Works consistently across Windows, macOS, and Linux -- **Automation Ready**: Perfect for DevOps, CI/CD, and system administration tasks - -## ๐Ÿ“ฆ **Workspace Modules Overview** - -SAL is organized as a Cargo workspace with the following crates: - -### **Core Library Modules** -- **`sal-os`**: Core OS interactions, file system operations, environment access -- **`sal-process`**: Process creation, management, and control -- **`sal-text`**: Utilities for text processing and manipulation -- **`sal-net`**: Network operations, HTTP requests, and connectivity utilities - -### **Integration Modules** -- **`sal-git`**: Git repository management and operations -- **`sal-vault`**: Cryptographic functions and keypair management -- **`sal-rhai`**: Integration layer for the Rhai scripting engine, used by `herodo` - -### **Client Modules** -- **`sal-redisclient`**: Client for Redis database interactions -- **`sal-postgresclient`**: Client for PostgreSQL database interactions -- **`sal-zinit-client`**: Client for Zinit process supervisor -- **`sal-mycelium`**: Client for Mycelium network operations - -### **Specialized Modules** -- **`sal-virt`**: Virtualization-related utilities (buildah, nerdctl, rfs) - -### **Root Package & Binary** -- **`sal`**: Root umbrella crate that re-exports all modules -- **`herodo`**: Command-line binary for executing Rhai scripts - -## ๐Ÿ”จ **Building SAL** - -Build the entire workspace (all crates) using Cargo: - -```bash -# Build all workspace members -cargo build --workspace - -# Build for release -cargo build --workspace --release - -# Build specific crate -cargo build -p sal-text -cargo build -p herodo -``` - -The `herodo` executable will be located at `target/debug/herodo` or `target/release/herodo`. - -## ๐Ÿงช **Running Tests** - -### **Rust Unit Tests** -```bash -# Run all workspace tests -cargo test --workspace - -# Run tests for specific crate -cargo test -p sal-text -cargo test -p sal-os - -# Run only library tests (faster) -cargo test --workspace --lib -``` - -### **Rhai Integration Tests** -Run comprehensive Rhai script tests that exercise `herodo` and SAL's scripted functionalities: - -```bash -# Run all Rhai integration tests (16 modules) -./run_rhai_tests.sh - -# Results: 16/16 modules pass with 100% success rate -``` - -The Rhai tests validate real-world functionality across all SAL modules and provide comprehensive integration testing. - ## License -SAL is licensed under the Apache License 2.0. See the [LICENSE](LICENSE) file for details. +Licensed under the Apache License 2.0. See [LICENSE](LICENSE) for details.