SAL (System Abstraction Layer)

Version: 0.1.0

SAL is a comprehensive Rust library designed to provide a unified and simplified interface for a wide array of system-level operations and interactions. It abstracts platform-specific details, enabling developers to write robust, cross-platform code with greater ease. SAL also includes herodo, a powerful command-line tool for executing Rhai scripts that leverage SAL's capabilities for automation and system management tasks.

🏗️ Cargo Workspace Structure

SAL is organized as a Cargo workspace with 16 specialized crates:

• Root Package: sal - Umbrella crate that re-exports all modules
• 13 Library Crates: Specialized SAL modules (git, text, os, net, etc.)
• 1 Binary Crate: herodo - Rhai script execution engine
• 1 Integration Crate: rhai - Rhai scripting integration layer

This workspace structure provides excellent build performance, dependency management, and maintainability.

🚀 Workspace Benefits

• Unified Dependency Management: Shared dependencies across all crates with consistent versions
• Optimized Build Performance: Parallel compilation and shared build artifacts
• Simplified Testing: Run tests across all modules with a single command
• Modular Architecture: Each module is independently maintainable while sharing common infrastructure
• Production Ready: 100% test coverage with comprehensive Rhai integration tests

📦 Installation

SAL is designed to be modular - install only the components you need!

Option 1: Individual Crates (Recommended)

Install only the modules you need:

# Core system operations
cargo add sal-os sal-process sal-text sal-net

# Database clients
cargo add sal-redisclient sal-postgresclient

# Infrastructure tools
cargo add sal-git sal-vault sal-kubernetes sal-virt

# Service clients
cargo add sal-zinit-client sal-mycelium

# Scripting support
cargo add sal-rhai

Option 2: Meta-crate with Features

Use the main sal crate with specific features:

# Install specific modules
cargo add sal --features os,process,text

# Install feature groups
cargo add sal --features core              # os, process, text, net
cargo add sal --features clients           # redisclient, postgresclient, zinit_client, mycelium
cargo add sal --features infrastructure    # git, vault, kubernetes, virt
cargo add sal --features scripting         # rhai

# Install everything
cargo add sal --features all

Quick Start Examples

Using Individual Crates (Recommended)

use sal_os::fs;
use sal_process::run;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // File system operations
    let files = fs::list_files(".")?;
    println!("Found {} files", files.len());

    // Process execution
    let result = run::command("echo hello")?;
    println!("Output: {}", result.stdout);

    Ok(())
}

Using Meta-crate with Features

// In Cargo.toml: sal = { version = "0.1.0", features = ["os", "process"] }
use sal::os::fs;
use sal::process::run;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // File system operations
    let files = fs::list_files(".")?;
    println!("Found {} files", files.len());

    // Process execution
    let result = run::command("echo hello")?;
    println!("Output: {}", result.stdout);

    Ok(())
}

Using Herodo for Scripting

# Build and install herodo
git clone https://github.com/PlanetFirst/sal.git
cd sal
./build_herodo.sh

# Create a script file
cat > example.rhai << 'EOF'
// File operations
let files = find_files(".", "*.rs");
print("Found " + files.len() + " Rust files");

// Process execution
let result = run("echo 'Hello from SAL!'");
print("Output: " + result.stdout);

// Network operations
let reachable = http_check("https://github.com");
print("GitHub reachable: " + reachable);
EOF

# Execute the script
herodo example.rhai

📦 Available Packages

SAL is published as individual crates, allowing you to install only what you need:

Package	Description	Install Command
sal-os	Operating system operations	cargo add sal-os
sal-process	Process management	cargo add sal-process
sal-text	Text processing utilities	cargo add sal-text
sal-net	Network operations	cargo add sal-net
sal-git	Git repository management	cargo add sal-git
sal-vault	Cryptographic operations	cargo add sal-vault
sal-kubernetes	Kubernetes management	cargo add sal-kubernetes
sal-virt	Virtualization tools	cargo add sal-virt
sal-redisclient	Redis database client	cargo add sal-redisclient ⏳
sal-postgresclient	PostgreSQL client	cargo add sal-postgresclient ⏳
sal-zinit-client	Zinit process supervisor	cargo add sal-zinit-client ⏳
sal-mycelium	Mycelium network client	cargo add sal-mycelium ⏳
sal-rhai	Rhai scripting integration	cargo add sal-rhai ⏳
sal	Meta-crate with features	cargo add sal --features all ⏳
herodo	Script executor binary	Build from source ⏳

Legend: ✅ Published | ⏳ Publishing soon (rate limited)

📢 Publishing Status

Currently Available on crates.io:

• ✅ sal-os - Operating system operations
• ✅ sal-text - Text processing utilities
• ✅ sal-net - Network operations
• ✅ sal-git - Git repository management
• ✅ sal-vault - Cryptographic operations
• ✅ sal-kubernetes - Kubernetes management

Publishing Soon (hit crates.io rate limit):

• ⏳ sal-redisclient, sal-postgresclient, sal-zinit-client, sal-mycelium
• ⏳ sal-process, sal-virt, sal-rhai
• ⏳ sal (meta-crate), herodo (binary)

Estimated Timeline: Remaining packages will be published within 24 hours once the rate limit resets.

Core Features

SAL offers a broad spectrum of functionalities, including:

• System Operations: File and directory management, environment variable access, system information retrieval, and OS-specific commands.
• Process Management: Create, monitor, control, and interact with system processes.
• Containerization Tools:
  • Integration with Buildah for building OCI/Docker-compatible container images.
  • Integration with nerdctl for managing containers (run, stop, list, build, etc.).
• Version Control: Programmatic interaction with Git repositories (clone, commit, push, pull, status, etc.).
• Database Clients:
  • Redis: Robust client for interacting with Redis servers.
  • PostgreSQL: Client for executing queries and managing PostgreSQL databases.
• Scripting Engine: In-built support for the Rhai scripting language, allowing SAL functionalities to be scripted and automated, primarily through the herodo tool.
• Networking & Services:
  • Mycelium: Tools for Mycelium network peer management and message passing.
  • Zinit: Client for interacting with the Zinit process supervision system.
  • RFS (Remote/Virtual Filesystem): Mount, manage, pack, and unpack various types of filesystems (local, SSH, S3, WebDAV).
• Text Processing: A suite of utilities for text manipulation, formatting, and regular expressions.
• Cryptography (vault): Functions for common cryptographic operations.

herodo: The SAL Scripting Tool

herodo is a command-line utility bundled with SAL that executes Rhai scripts. It empowers users to automate tasks and orchestrate complex workflows by leveraging SAL's diverse modules directly from scripts.

Usage

# Execute a single Rhai script
herodo script.rhai

# Execute a script with arguments
herodo script.rhai arg1 arg2

# Execute all .rhai scripts in a directory
herodo /path/to/scripts/

If a directory is provided, herodo will execute all .rhai scripts within that directory (and its subdirectories) in alphabetical order.

Scriptable SAL Modules via herodo

The following SAL modules and functionalities are exposed to the Rhai scripting environment through herodo:

• OS (os): Comprehensive file system operations, file downloading & installation, and system package management. Documentation
• Process (process): Robust command and script execution, plus process management (listing, finding, killing, checking command existence). Documentation
• Text (text): String manipulation, prefixing, path/name fixing, text replacement, and templating. Documentation
• Net (net): Network operations, HTTP requests, and connectivity utilities. Documentation
• Git (git): High-level repository management and generic Git command execution with Redis-backed authentication (clone, pull, push, commit, etc.). Documentation
• Vault (vault): Cryptographic operations, keypair management, encryption, decryption, hashing, etc. Documentation
• Redis Client (redisclient): Execute Redis commands (redis_get, redis_set, redis_execute, etc.). Documentation
• PostgreSQL Client (postgresclient): Execute SQL queries against PostgreSQL databases. Documentation
• Zinit (zinit_client): Client for Zinit process supervisor (service management, logs). Documentation
• Mycelium (mycelium): Client for Mycelium decentralized networking API (node info, peer management, messaging). Documentation
• Virtualization (virt):
  • Buildah: OCI/Docker image building functions. Documentation
  • nerdctl: Container lifecycle management (nerdctl_run, nerdctl_stop, nerdctl_images, nerdctl_image_build, etc.)
  • RFS: Mount various filesystems (local, SSH, S3, etc.), pack/unpack filesystem layers.

Example herodo Rhai Script

// file: /opt/scripts/example_task.rhai

// OS operations
println("Checking for /tmp/my_app_data...");
if !exist("/tmp/my_app_data") {
    mkdir("/tmp/my_app_data");
    println("Created directory /tmp/my_app_data");
}

// Redis operations
println("Setting Redis key 'app_status' to 'running'");
redis_set("app_status", "running");
let status = redis_get("app_status");
println("Current app_status from Redis: " + status);

// Process execution
println("Listing files in /tmp:");
let output = run("ls -la /tmp");
println(output.stdout);

println("Script finished.");

Run with: herodo /opt/scripts/example_task.rhai

For more examples, check the individual module test directories (e.g., text/tests/rhai/, os/tests/rhai/, etc.) in this repository.

Using SAL as a Rust Library

Add SAL as a dependency to your Cargo.toml:

[dependencies]
sal = "0.1.0" # Or the latest version

Rust Example: Using Redis Client

use sal::redisclient::{get_global_client, execute_cmd_with_args};
use redis::RedisResult;

async fn example_redis_interaction() -> RedisResult<()> {
    // Get a connection from the global pool
    let mut conn = get_global_client().await?.get_async_connection().await?;

    // Set a value
    execute_cmd_with_args(&mut conn, "SET", vec!["my_key", "my_value"]).await?;
    println!("Set 'my_key' to 'my_value'");

    // Get a value
    let value: String = execute_cmd_with_args(&mut conn, "GET", vec!["my_key"]).await?;
    println!("Retrieved value for 'my_key': {}", value);

    Ok(())
}

#[tokio::main]
async fn main() {
    if let Err(e) = example_redis_interaction().await {
        eprintln!("Redis Error: {}", e);
    }
}

(Note: The Redis client API might have evolved; please refer to src/redisclient/mod.rs and its documentation for the most current usage.)

🎯 Why Choose SAL?

Modular Architecture

• Install Only What You Need: Each package is independent - no bloated dependencies
• Faster Compilation: Smaller dependency trees mean faster build times
• Smaller Binaries: Only include the functionality you actually use
• Clear Dependencies: Explicit about what functionality your project uses

Developer Experience

• Consistent APIs: All packages follow the same design patterns and conventions
• Comprehensive Documentation: Each package has detailed documentation and examples
• Real-World Tested: All functionality is production-tested, no placeholder code
• Type Safety: Leverages Rust's type system for safe, reliable operations

Scripting Power

• Herodo Integration: Execute Rhai scripts with full access to SAL functionality
• Cross-Platform: Works consistently across Windows, macOS, and Linux
• Automation Ready: Perfect for DevOps, CI/CD, and system administration tasks

📦 Workspace Modules Overview

SAL is organized as a Cargo workspace with the following crates:

Core Library Modules

• sal-os: Core OS interactions, file system operations, environment access
• sal-process: Process creation, management, and control
• sal-text: Utilities for text processing and manipulation
• sal-net: Network operations, HTTP requests, and connectivity utilities

Integration Modules

• sal-git: Git repository management and operations
• sal-vault: Cryptographic functions and keypair management
• sal-rhai: Integration layer for the Rhai scripting engine, used by herodo

Client Modules

• sal-redisclient: Client for Redis database interactions
• sal-postgresclient: Client for PostgreSQL database interactions
• sal-zinit-client: Client for Zinit process supervisor
• sal-mycelium: Client for Mycelium network operations

Specialized Modules

• sal-virt: Virtualization-related utilities (buildah, nerdctl, rfs)

Root Package & Binary

• sal: Root umbrella crate that re-exports all modules
• herodo: Command-line binary for executing Rhai scripts

🔨 Building SAL

Build the entire workspace (all crates) using Cargo:

# Build all workspace members
cargo build --workspace

# Build for release
cargo build --workspace --release

# Build specific crate
cargo build -p sal-text
cargo build -p herodo

The herodo executable will be located at target/debug/herodo or target/release/herodo.

🧪 Running Tests

Rust Unit Tests

# Run all workspace tests
cargo test --workspace

# Run tests for specific crate
cargo test -p sal-text
cargo test -p sal-os

# Run only library tests (faster)
cargo test --workspace --lib

Rhai Integration Tests

Run comprehensive Rhai script tests that exercise herodo and SAL's scripted functionalities:

# Run all Rhai integration tests (16 modules)
./run_rhai_tests.sh

# Results: 16/16 modules pass with 100% success rate

The Rhai tests validate real-world functionality across all SAL modules and provide comprehensive integration testing.

License

SAL is licensed under the Apache License 2.0. See the LICENSE file for details.