heroforge-core

heroforge-core is the core database library for Heroforge, providing a complete API for interacting with Fossil SCM repositories programmatically, without requiring the Fossil CLI to be installed. It supports both reading from existing repositories and creating new ones from scratch.

Quick Start

Option 1: Test Server (Easiest)

The fastest way to get started - creates a test repository, runs tests, and starts a WebDAV server:

./build.sh --test-server

This will:

1. Build with WebDAV support
2. Create a test repository
3. Run automated tests
4. Start server on port 18080

Option 2: Interactive REPL

Start the interactive shell with tab completion, syntax highlighting, and command history:

# Build
cargo build --release

# Start interactive REPL
./target/release/heroforge-core --ui

Option 3: WebDAV Server Mode

Mount a repository as a network drive:

# Build with WebDAV support
cargo build --release --features webdav

# Start WebDAV server (default port: 8080)
./target/release/heroforge-core --server

# Start on custom port
./target/release/heroforge-core --server --port 18080

Connect to WebDAV Server

Command Line:

curl http://localhost:18080/
cadaver http://localhost:18080/

macOS Finder:

1. Finder → Go → Connect to Server (⌘K)
2. Enter: http://localhost:18080/
3. Click Connect

Windows Explorer:

1. Right-click "This PC" → "Map network drive"
2. Enter: http://localhost:18080/

Linux (Nautilus/Files):

1. Other Locations → Connect to Server
2. Enter: dav://localhost:18080/

Supported WebDAV Operations

Operation	Description
GET	Download files
PUT	Upload/update files
DELETE	Remove files
MKCOL	Create directories
COPY	Copy files/directories
MOVE	Move/rename files
PROPFIND	List directories, get file properties
PROPPATCH	Modify file properties
LOCK/UNLOCK	File locking for concurrent editing

Features

Read Operations

• Open and read existing Fossil repositories
• Browse repository history and check-ins
• List and read files at any check-in
• Find files using glob patterns
• Navigate directory structures
• Access branch and tag information

Write Operations

• Create new repositories from scratch
• Commit files with full manifest generation
• Create and manage branches
• Add tags to check-ins
• Manage users and permissions

Filesystem Operations

• Copy, move, rename files and directories
• Delete files and directories
• Change permissions (chmod)
• Create symbolic links
• Advanced find with ignore patterns

WebDAV Server

• Mount repositories as network drives
• Edit files with any application
• Automatic staging and commit
• File locking for concurrent access

Synchronization

• QUIC sync - Modern UDP-based protocol with TLS 1.3 (requires sync-quic feature)

Rhai Scripting

• Interactive REPL with tab completion and syntax highlighting
• Socket-based daemon for fast script execution
• Full API access from Rhai scripts

Installation

Add heroforge-core to your project:

cargo add heroforge-core

Or add to your Cargo.toml:

[dependencies]
heroforge-core = "0.2.2"

Feature Flags

Feature	Description	Dependencies
webdav	WebDAV server for repository access	dav-server, hyper, tokio
rhai	Rhai scripting support (default)	rhai, tokio, rustyline
sync-quic	QUIC protocol sync	quinn, rustls, tokio
git-import	Import git repositories	herolib-os

Example with multiple features:

[dependencies]
heroforge-core = { version = "0.2.2", features = ["webdav", "sync-quic"] }

CLI Usage

The heroforge-core binary provides both an interactive REPL and daemon mode for script execution.

Interactive REPL

# Start interactive shell (default)
heroforge-core

# Or explicitly with --ui flag
heroforge-core --ui

The REPL provides:

• Tab completion for all functions
• Syntax highlighting
• Command history
• Multi-line input support

Running Scripts

# Run a script file
heroforge-core myscript.rhai

# Execute inline script
heroforge-core -e 'let repo = repo_open("project.forge"); print(branches_list(repo));'

# Read script from stdin
echo 'print(version())' | heroforge-core -i

# Run locally without daemon
heroforge-core --local -e 'print("Hello!")'

Daemon Mode

# Start daemon in background
heroforge-core start -bg

# Check daemon status
heroforge-core status

# Stop daemon
heroforge-core stop

REPL Commands

Command	Description
/help	Show help
/functions	List all available functions
/repo	Show repository functions
/fs	Show filesystem functions
/files	Show file functions
/branches	Show branch/tag functions
/modify	Show modify builder functions
/find	Show find builder functions
/utils	Show utility functions
/scope	Show current scope variables
/load <file>	Load and execute a .rhai script
/clear	Clear screen
/quit	Exit REPL

Rust API Quick Start

Reading from an Existing Repository

use heroforge_core::Repository;

fn main() -> heroforge_core::Result<()> {
    // Open a Fossil repository
    let repo = Repository::open("project.forge")?;

    // Get the latest check-in on trunk
    let tip = repo.history().trunk_tip()?;
    println!("Latest: {} by {}", tip.hash, tip.user);
    println!("Comment: {}", tip.comment);

    // List all files on trunk
    let files = repo.files().on_trunk().list()?;
    for file in &files {
        println!("  {}", file.name);
    }

    // Read a specific file from trunk
    let content = repo.files().on_trunk().read("README.md")?;
    println!("README:\n{}", String::from_utf8_lossy(&content));

    Ok(())
}

Creating a New Repository

use heroforge_core::Repository;

fn main() -> heroforge_core::Result<()> {
    // Create a new repository
    let repo = Repository::init("new_project.forge")?;

    // Create initial check-in
    let init_hash = repo.commit_builder()
        .message("initial empty check-in")
        .author("admin")
        .initial()
        .execute()?;

    // Add some files in a new commit
    let commit_hash = repo.commit_builder()
        .message("Initial project structure")
        .author("developer")
        .parent(&init_hash)
        .file("README.md", b"# My Project\n")
        .file("src/main.rs", b"fn main() { println!(\"Hello!\"); }\n")
        .execute()?;

    // Tag the release
    repo.tags()
        .create("v1.0.0")
        .at_commit(&commit_hash)
        .author("developer")
        .execute()?;

    // Create a feature branch
    repo.branches()
        .create("feature-x")
        .from_commit(&commit_hash)
        .author("developer")
        .execute()?;

    Ok(())
}

Using the Rhai API

// Open a repository
let repo = repo_open("project.forge");

// List branches
let branches = branches_list(repo);
for b in branches {
    print("Branch: " + b);
}

// List files
let files = files_list(repo);
for f in files {
    print("  " + f.name() + " (" + f.size() + " bytes)");
}

// Read a file
let content = files_read(repo, "README.md");
print("Content:\n" + content);

// Use the filesystem interface for modifications
let repo_rw = repo_open_rw("project.forge");
let fs = fs_new(repo_rw, "developer@example.com");
fs_write(fs, "newfile.txt", "Hello, World!");
fs_commit(fs);

Finding Files with Glob Patterns

use heroforge_core::Repository;

fn main() -> heroforge_core::Result<()> {
    let repo = Repository::open("project.forge")?;

    // Find all Rust files on trunk
    let rust_files = repo.files().on_trunk().find("**/*.rs")?;
    for file in rust_files {
        println!("Found: {}", file.name);
    }

    // Find files in a specific directory on a branch
    let src_files = repo.files().on_branch("feature-x").find("src/**/*")?;
    
    // Find files at a specific tag
    let tagged_files = repo.files().at_tag("v1.0.0").find("*.md")?;
    
    Ok(())
}

Browsing History

use heroforge_core::Repository;

fn main() -> heroforge_core::Result<()> {
    let repo = Repository::open("project.forge")?;

    // Get recent check-ins
    let history = repo.history().recent(10)?;
    for checkin in history {
        println!("{} | {} | {}", 
            &checkin.hash[..12], 
            checkin.user, 
            checkin.comment
        );
    }

    // List all branches
    let branches = repo.branches().list()?;
    for branch in branches {
        println!("Branch: {}", branch);
    }

    // List all tags
    let tags = repo.tags().list()?;
    for tag in tags {
        println!("Tag: {}", tag);
    }

    // Get the tip of a specific branch
    let feature_tip = repo.history().branch_tip("feature-x")?;
    println!("Feature branch tip: {}", feature_tip.hash);

    Ok(())
}

Filesystem Operations

use heroforge_core::Repository;

fn main() -> heroforge_core::Result<()> {
    let repo = Repository::open_rw("project.forge")?;

    // Copy, move, delete files atomically
    let hash = repo.fs().modify()
        .message("Reorganize project")
        .author("developer")
        .copy_file("README.md", "docs/README.md")
        .move_dir("scripts", "tools")
        .delete_file("old_config.txt")
        .make_executable("tools/build.sh")
        .symlink("build", "tools/build.sh")
        .execute()?;

    // Advanced find with ignore patterns
    let files = repo.fs().find()
        .pattern("**/*.rs")
        .ignore("target/**")
        .ignore_hidden()
        .max_depth(3)
        .paths()?;
    
    // Utility functions
    println!("Exists: {}", repo.fs().exists("README.md")?);
    println!("Is dir: {}", repo.fs().is_dir("src")?);
    println!("Total size: {} bytes", repo.fs().du("**/*.rs")?);

    Ok(())
}

Builder API Overview

The library uses a fluent builder pattern for all operations:

Builder	Entry Point	Purpose
FilesBuilder	repo.files()	Read files, list directories, find with glob patterns
CommitBuilder	repo.commit_builder()	Create new check-ins with files
BranchesBuilder	repo.branches()	List and create branches
TagsBuilder	repo.tags()	List, create, and resolve tags
HistoryBuilder	repo.history()	Browse commits and history
UsersBuilder	repo.users()	Manage repository users
SyncBuilder	repo.sync()	Synchronize with remote repositories (QUIC)
FsOpsBuilder	repo.fs()	Filesystem operations (copy, move, delete, chmod, find, symlinks)

Rhai API Functions

Repository

Function	Return	Description
repo_open(path)	Repository	Open repository read-only
repo_open_rw(path)	Repository	Open repository read-write
repo_init(path)	Repository	Create new repository
repo_project_name(repo)	string	Get project name

Filesystem

Function	Return	Description
fs_new(repo, author)	FsHandle	Create filesystem interface
fs_read(fs, path)	string	Read file as string
fs_write(fs, path, content)	void	Write file
fs_delete(fs, path)	void	Delete file
fs_exists(fs, path)	bool	Check if file exists
fs_list(fs, dir)	array	List directory
fs_commit(fs)	string	Commit changes

Files

Function	Return	Description
files_list(repo)	array	List files on trunk
files_read(repo, path)	string	Read file from trunk
files_find(repo, pattern)	array	Find files by pattern

Branches/Tags

Function	Return	Description
branches_list(repo)	array	List all branches
branches_create(repo, name, parent)	string	Create branch
tags_list(repo)	array	List all tags
tags_create(repo, name, target)	string	Create tag

Utilities

Function	Return	Description
version()	string	Get heroforge-core version
uuid()	string	Generate UUID
timestamp()	i64	Get Unix timestamp
sleep(ms)	void	Sleep for milliseconds
env(name)	string	Get environment variable
cwd()	string	Current directory
home()	string	Home directory

Examples

The repository includes many examples demonstrating various features:

# Basic repository reading
cargo run --example read_repo

# Find files with glob patterns
cargo run --example find_and_read

# Branches and tags demonstration
cargo run --example branch_tag_test

# Comprehensive API demo
cargo run --example comprehensive_test

# Filesystem operations (copy, move, delete, chmod, symlinks)
cargo run --example fs_operations

# Advanced find with ignore patterns
cargo run --example fs_find

# QUIC sync (requires sync-quic feature)
cargo run --example quic_incremental_sync_test --features sync-quic

Documentation

Full API documentation is available at docs.rs/heroforge-core.

Generate local documentation with:

cargo doc --open --all-features

Architecture

A Fossil repository is a SQLite database containing:

• Blobs: Compressed file contents and manifests (zlib)
• Manifests: Check-in metadata with file lists, timestamps, comments
• Tags: Branch names, version tags, and other labels
• Events: Timeline of repository activity
• Delta encoding: Efficient storage of similar content

Acknowledgments

Based on the Fossil SCM project.