herocode/herolib_rust
12
0
Fork 0
Code Issues 5 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: Improve README.md with clearer structure and installation

Browse Source 
- Update README.md to provide a clearer structure and improved
  installation instructions.  This makes it easier for users to
  understand and use the library.
- Remove outdated and unnecessary sections like the workspace
  structure details, publishing status, and detailed features
  lists. The information is either not relevant anymore or can be
  found elsewhere.
- Simplify installation instructions to focus on the core aspects
  of installing individual packages or the meta-package with
  features.
- Add a dedicated section for building and running tests,
  improving developer experience and making the process more
  transparent.
- Modernize the overall layout and formatting for better
  readability.
This commit is contained in:
Mahmoud-Emad
2025-07-13 12:51:08 +03:00
parent f1806eb788
commit 758e59e921
1 changed files with 103 additions and 359 deletions
Download Patch File Download Diff File Expand all files Collapse all files

462
README.md
View File

@@ -1,404 +1,148 @@
# SAL (System Abstraction Layer)
**Version: 0.1.0**
**Version 0.1.0** - A modular Rust library for cross-platform system operations and automation.
SAL is a comprehensive Rust library designed to provide a unified and simplified interface for a wide array of system-level operations and interactions. It abstracts platform-specific details, enabling developers to write robust, cross-platform code with greater ease. SAL also includes `herodo`, a powerful command-line tool for executing Rhai scripts that leverage SAL's capabilities for automation and system management tasks.
SAL provides a unified interface for system operations with Rhai scripting support through the `herodo` tool.
## 🏗️ **Cargo Workspace Structure**
## Installation
SAL is organized as a **Cargo workspace** with 15 specialized crates:
- **Root Package**: `sal` - Umbrella crate that re-exports all modules
- **12 Library Crates**: Core SAL modules (os, process, text, net, git, vault, kubernetes, virt, redisclient, postgresclient, zinit_client, mycelium)
- **1 Binary Crate**: `herodo` - Rhai script execution engine
- **1 Integration Crate**: `rhai` - Rhai scripting integration layer
This workspace structure provides excellent build performance, dependency management, and maintainability.
### **🚀 Workspace Benefits**
- **Unified Dependency Management**: Shared dependencies across all crates with consistent versions
- **Optimized Build Performance**: Parallel compilation and shared build artifacts
- **Simplified Testing**: Run tests across all modules with a single command
- **Modular Architecture**: Each module is independently maintainable while sharing common infrastructure
- **Production Ready**: 100% test coverage with comprehensive Rhai integration tests
## 📦 Installation
SAL is designed to be modular - install only the components you need!
### Option 1: Individual Crates (Recommended)
Install only the modules you need:
### Individual Packages (Recommended)
```bash
# Currently available packages
cargo add sal-os sal-process sal-text sal-net sal-git sal-vault sal-kubernetes sal-virt
# Core functionality
cargo add sal-os sal-process sal-text sal-net
# Coming soon (rate limited)
# cargo add sal-redisclient sal-postgresclient sal-zinit-client sal-mycelium sal-rhai
# Infrastructure
cargo add sal-git sal-vault sal-kubernetes sal-virt
# Database clients
cargo add sal-redisclient sal-postgresclient sal-zinit-client
# Scripting
cargo add sal-rhai
```
### Option 2: Meta-crate with Features
Use the main `sal` crate with specific features:
### Meta-package with Features
```bash
# Coming soon - meta-crate with features (rate limited)
# cargo add sal --features os,process,text
# cargo add sal --features core # os, process, text, net
# cargo add sal --features infrastructure # git, vault, kubernetes, virt
# cargo add sal --features all
# For now, use individual crates (see Option 1 above)
cargo add sal --features core # os, process, text, net
cargo add sal --features infrastructure # git, vault, kubernetes, virt
cargo add sal --features all # everything
```
### Quick Start Examples
### Herodo Script Runner
#### Using Individual Crates (Recommended)
```bash
cargo install herodo
```
## Quick Start
### Rust Library Usage
```rust
use sal_os::fs;
use sal_process::run;
fn main() -> Result<(), Box<dyn std::error::Error>> {
// File system operations
let files = fs::list_files(".")?;
println!("Found {} files", files.len());
// Process execution
let result = run::command("echo hello")?;
println!("Output: {}", result.stdout);
Ok(())
}
```
#### Using Meta-crate with Features
```rust
// In Cargo.toml: sal = { version = "0.1.0", features = ["os", "process"] }
use sal::os::fs;
use sal::process::run;
fn main() -> Result<(), Box<dyn std::error::Error>> {
// File system operations
let files = fs::list_files(".")?;
println!("Found {} files", files.len());
// Process execution
let result = run::command("echo hello")?;
println!("Output: {}", result.stdout);
Ok(())
}
```
#### Using Herodo for Scripting
### Herodo Scripting
```bash
# Build and install herodo
git clone https://github.com/PlanetFirst/sal.git
cd sal
./build_herodo.sh
# Create a script file
# Create script
cat > example.rhai << 'EOF'
let files = find_files(".", "*.rs");
print("Found " + files.len() + " Rust files");
let result = run("echo 'Hello from SAL!'");
print("Output: " + result.stdout);
EOF
# Run script
herodo example.rhai
```
## Available Packages
| Package | Description |
|---------|-------------|
| [`sal-os`](https://crates.io/crates/sal-os) | Operating system operations |
| [`sal-process`](https://crates.io/crates/sal-process) | Process management |
| [`sal-text`](https://crates.io/crates/sal-text) | Text processing |
| [`sal-net`](https://crates.io/crates/sal-net) | Network operations |
| [`sal-git`](https://crates.io/crates/sal-git) | Git repository management |
| [`sal-vault`](https://crates.io/crates/sal-vault) | Cryptographic operations |
| [`sal-kubernetes`](https://crates.io/crates/sal-kubernetes) | Kubernetes management |
| [`sal-virt`](https://crates.io/crates/sal-virt) | Virtualization tools |
| [`sal-redisclient`](https://crates.io/crates/sal-redisclient) | Redis client |
| [`sal-postgresclient`](https://crates.io/crates/sal-postgresclient) | PostgreSQL client |
| [`sal-zinit-client`](https://crates.io/crates/sal-zinit-client) | Zinit process supervisor |
| [`sal-mycelium`](https://crates.io/crates/sal-mycelium) | Mycelium network client |
| [`sal-service-manager`](https://crates.io/crates/sal-service-manager) | Service management |
| [`sal-rhai`](https://crates.io/crates/sal-rhai) | Rhai scripting integration |
| [`sal`](https://crates.io/crates/sal) | Meta-crate with features |
| [`herodo`](https://crates.io/crates/herodo) | Script executor binary |
## Building & Testing
```bash
# Build all packages
cargo build --workspace
# Run tests
cargo test --workspace
# Run Rhai integration tests
./run_rhai_tests.sh
```
## Core Features
- **System Operations**: File/directory management, environment access, OS commands
- **Process Management**: Create, monitor, and control system processes
- **Containerization**: Buildah and nerdctl integration
- **Version Control**: Git repository operations
- **Database Clients**: Redis and PostgreSQL support
- **Networking**: HTTP, TCP, SSH connectivity utilities
- **Cryptography**: Key management, encryption, digital signatures
- **Text Processing**: String manipulation and templating
- **Scripting**: Rhai script execution via `herodo`
## Herodo Scripting
`herodo` executes Rhai scripts with access to all SAL modules:
```bash
herodo script.rhai # Run single script
herodo script.rhai arg1 arg2 # With arguments
herodo /path/to/scripts/ # Run all .rhai files in directory
```
### Example Script
```rhai
// File operations
let files = find_files(".", "*.rs");
print("Found " + files.len() + " Rust files");
// Process execution
let result = run("echo 'Hello from SAL!'");
let result = run("echo 'Hello SAL!'");
print("Output: " + result.stdout);
// Network operations
let reachable = http_check("https://github.com");
print("GitHub reachable: " + reachable);
EOF
# Execute the script
herodo example.rhai
```
## 📦 Available Packages
SAL is published as individual crates, allowing you to install only what you need:
| Package | Description | Install Command |
|---------|-------------|-----------------|
| [`sal-os`](https://crates.io/crates/sal-os) | Operating system operations | `cargo add sal-os` |
| [`sal-process`](https://crates.io/crates/sal-process) | Process management | `cargo add sal-process` |
| [`sal-text`](https://crates.io/crates/sal-text) | Text processing utilities | `cargo add sal-text` |
| [`sal-net`](https://crates.io/crates/sal-net) | Network operations | `cargo add sal-net` |
| [`sal-git`](https://crates.io/crates/sal-git) | Git repository management | `cargo add sal-git` |
| [`sal-vault`](https://crates.io/crates/sal-vault) | Cryptographic operations | `cargo add sal-vault` |
| [`sal-kubernetes`](https://crates.io/crates/sal-kubernetes) | Kubernetes management | `cargo add sal-kubernetes` |
| [`sal-virt`](https://crates.io/crates/sal-virt) | Virtualization tools | `cargo add sal-virt` |
| `sal-redisclient` | Redis database client | `cargo add sal-redisclient` ⏳ |
| `sal-postgresclient` | PostgreSQL client | `cargo add sal-postgresclient` ⏳ |
| `sal-zinit-client` | Zinit process supervisor | `cargo add sal-zinit-client` ⏳ |
| `sal-mycelium` | Mycelium network client | `cargo add sal-mycelium` ⏳ |
| `sal-rhai` | Rhai scripting integration | `cargo add sal-rhai` ⏳ |
| `sal` | Meta-crate with features | `cargo add sal --features all` ⏳ |
| `herodo` | Script executor binary | Build from source ⏳ |
**Legend**: ✅ Published | ⏳ Publishing soon (rate limited)
### 📢 **Publishing Status**
**Currently Available on crates.io:**
- ✅ [`sal-os`](https://crates.io/crates/sal-os) - Operating system operations
- ✅ [`sal-process`](https://crates.io/crates/sal-process) - Process management
- ✅ [`sal-text`](https://crates.io/crates/sal-text) - Text processing utilities
- ✅ [`sal-net`](https://crates.io/crates/sal-net) - Network operations
- ✅ [`sal-git`](https://crates.io/crates/sal-git) - Git repository management
- ✅ [`sal-vault`](https://crates.io/crates/sal-vault) - Cryptographic operations
- ✅ [`sal-kubernetes`](https://crates.io/crates/sal-kubernetes) - Kubernetes management
- ✅ [`sal-virt`](https://crates.io/crates/sal-virt) - Virtualization tools
**Publishing Soon** (hit crates.io rate limit):
- ⏳ `sal-redisclient`, `sal-postgresclient`, `sal-zinit-client`, `sal-mycelium`
- ⏳ `sal-rhai`
- ⏳ `sal` (meta-crate), `herodo` (binary)
**Estimated Timeline**: Remaining packages will be published within 24 hours once the rate limit resets.
## Core Features
SAL offers a broad spectrum of functionalities, including:
- **System Operations**: File and directory management, environment variable access, system information retrieval, and OS-specific commands.
- **Process Management**: Create, monitor, control, and interact with system processes.
- **Containerization Tools**:
- Integration with **Buildah** for building OCI/Docker-compatible container images.
- Integration with **nerdctl** for managing containers (run, stop, list, build, etc.).
- **Version Control**: Programmatic interaction with Git repositories (clone, commit, push, pull, status, etc.).
- **Database Clients**:
- **Redis**: Robust client for interacting with Redis servers.
- **PostgreSQL**: Client for executing queries and managing PostgreSQL databases.
- **Scripting Engine**: In-built support for the **Rhai** scripting language, allowing SAL functionalities to be scripted and automated, primarily through the `herodo` tool.
- **Networking & Services**:
- **Mycelium**: Tools for Mycelium network peer management and message passing.
- **Zinit**: Client for interacting with the Zinit process supervision system.
- **RFS (Remote/Virtual Filesystem)**: Mount, manage, pack, and unpack various types of filesystems (local, SSH, S3, WebDAV).
- **Text Processing**: A suite of utilities for text manipulation, formatting, and regular expressions.
- **Cryptography (`vault`)**: Functions for common cryptographic operations.
## `herodo`: The SAL Scripting Tool
`herodo` is a command-line utility bundled with SAL that executes Rhai scripts. It empowers users to automate tasks and orchestrate complex workflows by leveraging SAL's diverse modules directly from scripts.
### Usage
```bash
# Execute a single Rhai script
herodo script.rhai
# Execute a script with arguments
herodo script.rhai arg1 arg2
# Execute all .rhai scripts in a directory
herodo /path/to/scripts/
```
If a directory is provided, `herodo` will execute all `.rhai` scripts within that directory (and its subdirectories) in alphabetical order.
### Scriptable SAL Modules via `herodo`
The following SAL modules and functionalities are exposed to the Rhai scripting environment through `herodo`:
- **OS (`os`)**: Comprehensive file system operations, file downloading & installation, and system package management. [Documentation](os/README.md)
- **Process (`process`)**: Robust command and script execution, plus process management (listing, finding, killing, checking command existence). [Documentation](process/README.md)
- **Text (`text`)**: String manipulation, prefixing, path/name fixing, text replacement, and templating. [Documentation](text/README.md)
- **Net (`net`)**: Network operations, HTTP requests, and connectivity utilities. [Documentation](net/README.md)
- **Git (`git`)**: High-level repository management and generic Git command execution with Redis-backed authentication (clone, pull, push, commit, etc.). [Documentation](git/README.md)
- **Vault (`vault`)**: Cryptographic operations, keypair management, encryption, decryption, hashing, etc. [Documentation](vault/README.md)
- **Redis Client (`redisclient`)**: Execute Redis commands (`redis_get`, `redis_set`, `redis_execute`, etc.). [Documentation](redisclient/README.md)
- **PostgreSQL Client (`postgresclient`)**: Execute SQL queries against PostgreSQL databases. [Documentation](postgresclient/README.md)
- **Zinit (`zinit_client`)**: Client for Zinit process supervisor (service management, logs). [Documentation](zinit_client/README.md)
- **Mycelium (`mycelium`)**: Client for Mycelium decentralized networking API (node info, peer management, messaging). [Documentation](mycelium/README.md)
- **Virtualization (`virt`)**:
- **Buildah**: OCI/Docker image building functions. [Documentation](virt/README.md)
- **nerdctl**: Container lifecycle management (`nerdctl_run`, `nerdctl_stop`, `nerdctl_images`, `nerdctl_image_build`, etc.)
- **RFS**: Mount various filesystems (local, SSH, S3, etc.), pack/unpack filesystem layers.
### Example `herodo` Rhai Script
```rhai
// file: /opt/scripts/example_task.rhai
// OS operations
println("Checking for /tmp/my_app_data...");
if !exist("/tmp/my_app_data") {
mkdir("/tmp/my_app_data");
println("Created directory /tmp/my_app_data");
}
// Redis operations
println("Setting Redis key 'app_status' to 'running'");
redis_set("app_status", "running");
let status = redis_get("app_status");
println("Current app_status from Redis: " + status);
// Process execution
println("Listing files in /tmp:");
let output = run("ls -la /tmp");
println(output.stdout);
println("Script finished.");
redis_set("status", "running");
let status = redis_get("status");
print("Status: " + status);
```
Run with: `herodo /opt/scripts/example_task.rhai`
For more examples, check the individual module test directories (e.g., `text/tests/rhai/`, `os/tests/rhai/`, etc.) in this repository.
## Using SAL as a Rust Library
### Option 1: Individual Crates (Recommended)
Add only the SAL modules you need:
```toml
[dependencies]
sal-os = "0.1.0"
sal-process = "0.1.0"
sal-text = "0.1.0"
```
```rust
use sal_os::fs;
use sal_process::run;
use sal_text::template;
fn main() -> Result<(), Box<dyn std::error::Error>> {
// File operations
let files = fs::list_files(".")?;
println!("Found {} files", files.len());
// Process execution
let result = run::command("echo 'Hello SAL!'")?;
println!("Output: {}", result.stdout);
// Text templating
let template_str = "Hello {{name}}!";
let mut vars = std::collections::HashMap::new();
vars.insert("name".to_string(), "World".to_string());
let rendered = template::render(template_str, &vars)?;
println!("Rendered: {}", rendered);
Ok(())
}
```
### Option 2: Meta-crate with Features (Coming Soon)
```toml
[dependencies]
sal = { version = "0.1.0", features = ["os", "process", "text"] }
```
```rust
use sal::os::fs;
use sal::process::run;
use sal::text::template;
// Same code as above, but using the meta-crate
```
*(Note: The meta-crate `sal` will be available once all individual packages are published.)*
## 🎯 **Why Choose SAL?**
### **Modular Architecture**
- **Install Only What You Need**: Each package is independent - no bloated dependencies
- **Faster Compilation**: Smaller dependency trees mean faster build times
- **Smaller Binaries**: Only include the functionality you actually use
- **Clear Dependencies**: Explicit about what functionality your project uses
### **Developer Experience**
- **Consistent APIs**: All packages follow the same design patterns and conventions
- **Comprehensive Documentation**: Each package has detailed documentation and examples
- **Real-World Tested**: All functionality is production-tested, no placeholder code
- **Type Safety**: Leverages Rust's type system for safe, reliable operations
### **Scripting Power**
- **Herodo Integration**: Execute Rhai scripts with full access to SAL functionality
- **Cross-Platform**: Works consistently across Windows, macOS, and Linux
- **Automation Ready**: Perfect for DevOps, CI/CD, and system administration tasks
## 📦 **Workspace Modules Overview**
SAL is organized as a Cargo workspace with the following crates:
### **Core Library Modules**
- **`sal-os`**: Core OS interactions, file system operations, environment access
- **`sal-process`**: Process creation, management, and control
- **`sal-text`**: Utilities for text processing and manipulation
- **`sal-net`**: Network operations, HTTP requests, and connectivity utilities
### **Integration Modules**
- **`sal-git`**: Git repository management and operations
- **`sal-vault`**: Cryptographic functions and keypair management
- **`sal-rhai`**: Integration layer for the Rhai scripting engine, used by `herodo`
### **Client Modules**
- **`sal-redisclient`**: Client for Redis database interactions
- **`sal-postgresclient`**: Client for PostgreSQL database interactions
- **`sal-zinit-client`**: Client for Zinit process supervisor
- **`sal-mycelium`**: Client for Mycelium network operations
### **Specialized Modules**
- **`sal-virt`**: Virtualization-related utilities (buildah, nerdctl, rfs)
### **Root Package & Binary**
- **`sal`**: Root umbrella crate that re-exports all modules
- **`herodo`**: Command-line binary for executing Rhai scripts
## 🔨 **Building SAL**
Build the entire workspace (all crates) using Cargo:
```bash
# Build all workspace members
cargo build --workspace
# Build for release
cargo build --workspace --release
# Build specific crate
cargo build -p sal-text
cargo build -p herodo
```
The `herodo` executable will be located at `target/debug/herodo` or `target/release/herodo`.
## 🧪 **Running Tests**
### **Rust Unit Tests**
```bash
# Run all workspace tests
cargo test --workspace
# Run tests for specific crate
cargo test -p sal-text
cargo test -p sal-os
# Run only library tests (faster)
cargo test --workspace --lib
```
### **Rhai Integration Tests**
Run comprehensive Rhai script tests that exercise `herodo` and SAL's scripted functionalities:
```bash
# Run all Rhai integration tests (16 modules)
./run_rhai_tests.sh
# Results: 16/16 modules pass with 100% success rate
```
The Rhai tests validate real-world functionality across all SAL modules and provide comprehensive integration testing.
## License
SAL is licensed under the Apache License 2.0. See the [LICENSE](LICENSE) file for details.
Licensed under the Apache License 2.0. See [LICENSE](LICENSE) for details.