DocTree

DocTree is a Rust library for managing collections of markdown documents with powerful include functionality. It provides a robust system for organizing, processing, and retrieving document collections with Redis-backed storage.

Overview

DocTree scans directories for .collection files, which define document collections. Each collection contains markdown documents and other files (like images). The library provides functionality to:

• Scan directories recursively to find collections
• Process includes between documents (allowing one document to include content from another)
• Convert markdown to HTML
• Store document metadata in Redis for efficient retrieval
• Provide a command-line interface for interacting with collections

Key Concepts

Collections

A collection is a group of related documents and files. Collections are defined by a .collection file in a directory. The .collection file can be empty (in which case the directory name is used as the collection name) or it can contain TOML configuration:

name = "my_collection"
# Other configuration options can be added in the future

DocTree

A DocTree is a manager for multiple collections. It provides methods for:

• Adding collections
• Retrieving documents from collections
• Processing includes between documents
• Converting markdown to HTML
• Managing collection metadata in Redis

Includes

One of the most powerful features of DocTree is the ability to include content from one document in another. This is done using the !!include directive:

# My Document

This is my document.

!!include another_collection:some_document.md

More content here...

The include directive supports several formats:

• !!include collection_name:page_name - Include a page from a specific collection
• !!include collection_name:'page name' - Include a page with spaces from a specific collection
• !!include page_name - Include a page from the current collection
• !!include name:'page name' - Include a page with spaces from the current collection

Includes can be nested, allowing for complex document structures.

Storage

DocTree uses Redis as a backend storage system. Document metadata (like paths and names) is stored in Redis, making it efficient to retrieve documents without scanning the filesystem each time.

The Redis keys are structured as:

• collections:{collection_name}:{document_name} - Stores the relative path to a document
• collections:{collection_name}:path - Stores the absolute path to the collection

Command-Line Interface

DocTree comes with a command-line interface (CLI) that provides access to the library's functionality:

DocTree CLI 0.1.0
A tool to manage document collections

USAGE:
    doctreecmd [SUBCOMMAND]

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

SUBCOMMANDS:
    delete    Delete a collection
    get                  Get page content
    html                 Get page content as HTML
    info                 Show detailed information about collections
    list                 List collections
    reset                Delete all collections
    scan                 Scan a directory for .collection files and create collections

Example Commands

Scanning Collections

doctreecmd scan /path/to/documents --doctree my_doctree

This command scans the specified directory for .collection files and creates collections in Redis.

Listing Collections

doctreecmd list --doctree my_doctree

This command lists all collections in the specified doctree.

Getting Document Content

doctreecmd get -c collection_name -p page_name --doctree my_doctree

This command retrieves the content of a document from a collection.

Getting HTML Content

doctreecmd get -c collection_name -p page_name -f html --doctree my_doctree

This command retrieves the HTML content of a document from a collection.

Showing Collection Information

doctreecmd info collection_name --doctree my_doctree

This command shows detailed information about a collection, including its documents and files.

Deleting a Collection

doctreecmd delete collection_name --doctree my_doctree

This command deletes a collection.

Resetting All Collections

doctreecmd reset --doctree my_doctree

This command deletes all collections.

Implementation Details

DocTree is implemented in Rust and uses several key dependencies:

• walkdir for recursively walking directories
• pulldown-cmark for parsing and rendering markdown
• toml for parsing collection configuration files
• redis for interacting with Redis
• clap for the command-line interface

The library is structured into several modules:

• doctree.rs - Core DocTree functionality
• collection.rs - Collection management
• include.rs - Include processing
• storage.rs - Redis storage backend
• utils.rs - Utility functions
• error.rs - Error handling

Use Cases

DocTree is particularly useful for:

1. Documentation Systems: Manage and organize technical documentation with the ability to include common sections across multiple documents.

2. Content Management: Create a flexible content management system where content can be modularized and reused.

3. Knowledge Bases: Build knowledge bases with interconnected documents that can reference each other.

4. Static Site Generation: Generate static websites from markdown documents with the ability to include common elements.

Getting Started

Prerequisites

• Rust (latest stable version)
• Redis server running on localhost:6379 (or configure a different URL)

Building

cargo build --release

Running the CLI

cargo run --bin doctreecmd -- [SUBCOMMAND]

Using the Library

Add doctree to your Cargo.toml:

[dependencies]
doctree = { path = "path/to/doctree" }

Basic usage:

use doctree::{DocTree, RedisStorage, Result, from_directory};
use std::path::Path;

fn main() -> Result<()> {
    // Create a DocTree by scanning a directory
    let doctree = from_directory(Path::new("path/to/documents"), Some("my_doctree"))?;
    
    // List collections
    let collections = doctree.list_collections();
    for collection in collections {
        println!("Collection: {}", collection);
    }
    
    // Get a document with includes processed
    let content = doctree.page_get(Some("collection_name"), "page_name")?;
    println!("{}", content);
    
    // Get a document as HTML
    let html = doctree.page_get_html(Some("collection_name"), "page_name")?;
    println!("{}", html);
    
    Ok(())
}