499 lines
14 KiB
Markdown
499 lines
14 KiB
Markdown
# DocTree Implementation Plan
|
|
|
|
## Overview
|
|
|
|
The DocTree library will be a Rust implementation of the Go reference, maintaining the core functionality while improving the API design to be more idiomatic Rust. We'll use Redis as the storage backend and implement a minimal CLI example to demonstrate usage.
|
|
|
|
## Architecture
|
|
|
|
```mermaid
|
|
classDiagram
|
|
class DocTree {
|
|
+collections: HashMap<String, Collection>
|
|
+default_collection: Option<String>
|
|
+new() DocTreeBuilder
|
|
+add_collection(path, name) Result<&Collection>
|
|
+get_collection(name) Result<&Collection>
|
|
+delete_collection(name) Result<()>
|
|
+list_collections() Vec<String>
|
|
+page_get(collection, page) Result<String>
|
|
+page_get_html(collection, page) Result<String>
|
|
+file_get_url(collection, file) Result<String>
|
|
}
|
|
|
|
class DocTreeBuilder {
|
|
-collections: HashMap<String, Collection>
|
|
-default_collection: Option<String>
|
|
+with_collection(path, name) DocTreeBuilder
|
|
+with_default_collection(name) DocTreeBuilder
|
|
+build() Result<DocTree>
|
|
}
|
|
|
|
class Collection {
|
|
+path: String
|
|
+name: String
|
|
+new(path, name) CollectionBuilder
|
|
+scan() Result<()>
|
|
+page_get(name) Result<String>
|
|
+page_set(name, content) Result<()>
|
|
+page_delete(name) Result<()>
|
|
+page_list() Result<Vec<String>>
|
|
+file_get_url(name) Result<String>
|
|
+file_set(name, content) Result<()>
|
|
+file_delete(name) Result<()>
|
|
+file_list() Result<Vec<String>>
|
|
+page_get_html(name) Result<String>
|
|
}
|
|
|
|
class CollectionBuilder {
|
|
-path: String
|
|
-name: String
|
|
+build() Result<Collection>
|
|
}
|
|
|
|
class RedisStorage {
|
|
+client: redis::Client
|
|
+new(url) Result<RedisStorage>
|
|
+store_collection_entry(collection, key, value) Result<()>
|
|
+get_collection_entry(collection, key) Result<String>
|
|
+delete_collection_entry(collection, key) Result<()>
|
|
+list_collection_entries(collection) Result<Vec<String>>
|
|
+delete_collection(collection) Result<()>
|
|
}
|
|
|
|
class IncludeProcessor {
|
|
+process_includes(content, collection, doctree) Result<String>
|
|
}
|
|
|
|
DocTree --> DocTreeBuilder : creates
|
|
DocTree --> "0..*" Collection : contains
|
|
Collection --> CollectionBuilder : creates
|
|
DocTree --> RedisStorage : uses
|
|
Collection --> RedisStorage : uses
|
|
DocTree --> IncludeProcessor : uses
|
|
```
|
|
|
|
## Implementation Steps
|
|
|
|
### 1. Project Setup and Dependencies
|
|
|
|
1. Update the Cargo.toml files with necessary dependencies:
|
|
- redis (for Redis client)
|
|
- walkdir (for directory traversal)
|
|
- pulldown-cmark (for Markdown to HTML conversion)
|
|
- thiserror (for error handling)
|
|
- clap (for CLI argument parsing in doctreecmd)
|
|
|
|
### 2. Core Library Structure
|
|
|
|
1. **Error Module**
|
|
- Create a custom error type using thiserror
|
|
- Define specific error variants for different failure scenarios
|
|
|
|
2. **Storage Module**
|
|
- Implement the RedisStorage struct to handle Redis operations
|
|
- Provide methods for storing, retrieving, and deleting collection entries
|
|
- Implement connection pooling for efficient Redis access
|
|
|
|
3. **Utils Module**
|
|
- Implement utility functions like name_fix (equivalent to tools.NameFix in Go)
|
|
- Implement markdown to HTML conversion using pulldown-cmark
|
|
|
|
### 3. Collection Implementation
|
|
|
|
1. **Collection Module**
|
|
- Implement the Collection struct to represent a collection of documents
|
|
- Implement the CollectionBuilder for creating Collection instances
|
|
- Implement methods for scanning directories, managing pages and files
|
|
|
|
2. **Collection Builder Pattern**
|
|
- Create a builder pattern for Collection creation
|
|
- Allow configuration of Collection properties before building
|
|
|
|
### 4. DocTree Implementation
|
|
|
|
1. **DocTree Module**
|
|
- Implement the DocTree struct to manage multiple collections
|
|
- Implement the DocTreeBuilder for creating DocTree instances
|
|
- Implement methods for managing collections and accessing documents
|
|
|
|
2. **DocTree Builder Pattern**
|
|
- Create a builder pattern for DocTree creation
|
|
- Allow adding collections and setting default collection before building
|
|
|
|
### 5. Include Processor Implementation
|
|
|
|
1. **Include Module**
|
|
- Implement the IncludeProcessor to handle include directives
|
|
- Implement parsing of include directives
|
|
- Implement recursive processing of includes
|
|
|
|
### 6. CLI Example
|
|
|
|
1. **Update doctreecmd**
|
|
- Implement a minimal CLI interface using clap
|
|
- Provide commands for basic operations:
|
|
- Scanning a directory
|
|
- Listing collections
|
|
- Getting page content
|
|
- Getting HTML content
|
|
|
|
## Detailed Module Breakdown
|
|
|
|
### Error Module (src/error.rs)
|
|
|
|
```rust
|
|
use thiserror::Error;
|
|
|
|
#[derive(Error, Debug)]
|
|
pub enum DocTreeError {
|
|
#[error("IO error: {0}")]
|
|
IoError(#[from] std::io::Error),
|
|
|
|
#[error("Redis error: {0}")]
|
|
RedisError(#[from] redis::RedisError),
|
|
|
|
#[error("Collection not found: {0}")]
|
|
CollectionNotFound(String),
|
|
|
|
#[error("Page not found: {0}")]
|
|
PageNotFound(String),
|
|
|
|
#[error("File not found: {0}")]
|
|
FileNotFound(String),
|
|
|
|
#[error("Invalid include directive: {0}")]
|
|
InvalidIncludeDirective(String),
|
|
|
|
#[error("No default collection set")]
|
|
NoDefaultCollection,
|
|
|
|
#[error("Invalid number of arguments")]
|
|
InvalidArgumentCount,
|
|
}
|
|
|
|
pub type Result<T> = std::result::Result<T, DocTreeError>;
|
|
```
|
|
|
|
### Storage Module (src/storage.rs)
|
|
|
|
```rust
|
|
use redis::{Client, Commands, Connection};
|
|
use crate::error::{DocTreeError, Result};
|
|
|
|
pub struct RedisStorage {
|
|
client: Client,
|
|
}
|
|
|
|
impl RedisStorage {
|
|
pub fn new(url: &str) -> Result<Self> {
|
|
let client = Client::open(url)?;
|
|
Ok(Self { client })
|
|
}
|
|
|
|
pub fn get_connection(&self) -> Result<Connection> {
|
|
Ok(self.client.get_connection()?)
|
|
}
|
|
|
|
pub fn store_collection_entry(&self, collection: &str, key: &str, value: &str) -> Result<()> {
|
|
let mut conn = self.get_connection()?;
|
|
let collection_key = format!("collections:{}", collection);
|
|
conn.hset(collection_key, key, value)?;
|
|
Ok(())
|
|
}
|
|
|
|
pub fn get_collection_entry(&self, collection: &str, key: &str) -> Result<String> {
|
|
let mut conn = self.get_connection()?;
|
|
let collection_key = format!("collections:{}", collection);
|
|
let value: String = conn.hget(collection_key, key)?;
|
|
Ok(value)
|
|
}
|
|
|
|
// Additional methods for Redis operations
|
|
}
|
|
```
|
|
|
|
### Utils Module (src/utils.rs)
|
|
|
|
```rust
|
|
use pulldown_cmark::{Parser, Options, html};
|
|
|
|
pub fn name_fix(name: &str) -> String {
|
|
// Implementation of name_fix similar to tools.NameFix in Go
|
|
// Normalize the name by converting to lowercase, replacing spaces with hyphens, etc.
|
|
}
|
|
|
|
pub fn markdown_to_html(markdown: &str) -> String {
|
|
let mut options = Options::empty();
|
|
options.insert(Options::ENABLE_TABLES);
|
|
options.insert(Options::ENABLE_FOOTNOTES);
|
|
options.insert(Options::ENABLE_STRIKETHROUGH);
|
|
|
|
let parser = Parser::new_ext(markdown, options);
|
|
let mut html_output = String::new();
|
|
html::push_html(&mut html_output, parser);
|
|
|
|
html_output
|
|
}
|
|
```
|
|
|
|
### Collection Module (src/collection.rs)
|
|
|
|
```rust
|
|
use std::path::{Path, PathBuf};
|
|
use walkdir::WalkDir;
|
|
use crate::error::Result;
|
|
use crate::storage::RedisStorage;
|
|
use crate::utils::name_fix;
|
|
|
|
pub struct Collection {
|
|
pub path: PathBuf,
|
|
pub name: String,
|
|
storage: RedisStorage,
|
|
}
|
|
|
|
pub struct CollectionBuilder {
|
|
path: PathBuf,
|
|
name: String,
|
|
storage: Option<RedisStorage>,
|
|
}
|
|
|
|
impl Collection {
|
|
pub fn builder<P: AsRef<Path>>(path: P, name: &str) -> CollectionBuilder {
|
|
CollectionBuilder {
|
|
path: path.as_ref().to_path_buf(),
|
|
name: name_fix(name),
|
|
storage: None,
|
|
}
|
|
}
|
|
|
|
pub fn scan(&self) -> Result<()> {
|
|
// Implementation of scanning directory and storing in Redis
|
|
}
|
|
|
|
pub fn page_get(&self, page_name: &str) -> Result<String> {
|
|
// Implementation of getting page content
|
|
}
|
|
|
|
// Additional methods for Collection
|
|
}
|
|
|
|
impl CollectionBuilder {
|
|
pub fn with_storage(mut self, storage: RedisStorage) -> Self {
|
|
self.storage = Some(storage);
|
|
self
|
|
}
|
|
|
|
pub fn build(self) -> Result<Collection> {
|
|
let storage = self.storage.ok_or_else(|| {
|
|
std::io::Error::new(std::io::ErrorKind::Other, "Storage not provided")
|
|
})?;
|
|
|
|
let collection = Collection {
|
|
path: self.path,
|
|
name: self.name,
|
|
storage,
|
|
};
|
|
|
|
Ok(collection)
|
|
}
|
|
}
|
|
```
|
|
|
|
### DocTree Module (src/doctree.rs)
|
|
|
|
```rust
|
|
use std::collections::HashMap;
|
|
use std::path::Path;
|
|
use crate::collection::{Collection, CollectionBuilder};
|
|
use crate::error::{DocTreeError, Result};
|
|
use crate::storage::RedisStorage;
|
|
|
|
pub struct DocTree {
|
|
collections: HashMap<String, Collection>,
|
|
default_collection: Option<String>,
|
|
storage: RedisStorage,
|
|
}
|
|
|
|
pub struct DocTreeBuilder {
|
|
collections: HashMap<String, Collection>,
|
|
default_collection: Option<String>,
|
|
storage: Option<RedisStorage>,
|
|
}
|
|
|
|
impl DocTree {
|
|
pub fn builder() -> DocTreeBuilder {
|
|
DocTreeBuilder {
|
|
collections: HashMap::new(),
|
|
default_collection: None,
|
|
storage: None,
|
|
}
|
|
}
|
|
|
|
pub fn add_collection<P: AsRef<Path>>(&mut self, path: P, name: &str) -> Result<&Collection> {
|
|
// Implementation of adding a collection
|
|
}
|
|
|
|
// Additional methods for DocTree
|
|
}
|
|
|
|
impl DocTreeBuilder {
|
|
pub fn with_storage(mut self, storage: RedisStorage) -> Self {
|
|
self.storage = Some(storage);
|
|
self
|
|
}
|
|
|
|
pub fn with_collection<P: AsRef<Path>>(mut self, path: P, name: &str) -> Result<Self> {
|
|
// Implementation of adding a collection during building
|
|
Ok(self)
|
|
}
|
|
|
|
pub fn with_default_collection(mut self, name: &str) -> Self {
|
|
self.default_collection = Some(name.to_string());
|
|
self
|
|
}
|
|
|
|
pub fn build(self) -> Result<DocTree> {
|
|
let storage = self.storage.ok_or_else(|| {
|
|
std::io::Error::new(std::io::ErrorKind::Other, "Storage not provided")
|
|
})?;
|
|
|
|
let doctree = DocTree {
|
|
collections: self.collections,
|
|
default_collection: self.default_collection,
|
|
storage,
|
|
};
|
|
|
|
Ok(doctree)
|
|
}
|
|
}
|
|
```
|
|
|
|
### Include Module (src/include.rs)
|
|
|
|
```rust
|
|
use crate::doctree::DocTree;
|
|
use crate::error::Result;
|
|
|
|
pub fn process_includes(content: &str, collection_name: &str, doctree: &DocTree) -> Result<String> {
|
|
// Implementation of processing include directives
|
|
}
|
|
|
|
fn parse_include_line(line: &str) -> Result<(Option<String>, Option<String>)> {
|
|
// Implementation of parsing include directives
|
|
}
|
|
|
|
fn handle_include(page_name: &str, collection_name: &str, doctree: &DocTree) -> Result<String> {
|
|
// Implementation of handling include directives
|
|
}
|
|
```
|
|
|
|
### Main Library File (src/lib.rs)
|
|
|
|
```rust
|
|
mod error;
|
|
mod storage;
|
|
mod utils;
|
|
mod collection;
|
|
mod doctree;
|
|
mod include;
|
|
|
|
pub use error::{DocTreeError, Result};
|
|
pub use storage::RedisStorage;
|
|
pub use collection::{Collection, CollectionBuilder};
|
|
pub use doctree::{DocTree, DocTreeBuilder};
|
|
pub use include::process_includes;
|
|
```
|
|
|
|
### CLI Example (doctreecmd/src/main.rs)
|
|
|
|
```rust
|
|
use clap::{App, Arg, SubCommand};
|
|
use doctree::{DocTree, RedisStorage};
|
|
|
|
fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|
let matches = App::new("DocTree CLI")
|
|
.version("0.1.0")
|
|
.author("Your Name")
|
|
.about("A tool to manage document collections")
|
|
.subcommand(
|
|
SubCommand::with_name("scan")
|
|
.about("Scan a directory and create a collection")
|
|
.arg(Arg::with_name("path").required(true))
|
|
.arg(Arg::with_name("name").required(true)),
|
|
)
|
|
.subcommand(
|
|
SubCommand::with_name("list")
|
|
.about("List collections"),
|
|
)
|
|
.subcommand(
|
|
SubCommand::with_name("get")
|
|
.about("Get page content")
|
|
.arg(Arg::with_name("collection").required(true))
|
|
.arg(Arg::with_name("page").required(true)),
|
|
)
|
|
.get_matches();
|
|
|
|
// Implementation of CLI commands
|
|
|
|
Ok(())
|
|
}
|
|
```
|
|
|
|
## Example Usage
|
|
|
|
Here's how the library would be used with the builder pattern:
|
|
|
|
```rust
|
|
use doctree::{DocTree, RedisStorage};
|
|
|
|
fn main() -> Result<(), Box<dyn std::error::Error>> {
|
|
// Create a Redis storage instance
|
|
let storage = RedisStorage::new("redis://localhost:6379")?;
|
|
|
|
// Create a DocTree instance using the builder pattern
|
|
let mut doctree = DocTree::builder()
|
|
.with_storage(storage.clone())
|
|
.with_collection("path/to/collection", "my-collection")?
|
|
.with_default_collection("my-collection")
|
|
.build()?;
|
|
|
|
// Get page content
|
|
let content = doctree.page_get("my-collection", "page-name")?;
|
|
println!("Page content: {}", content);
|
|
|
|
// Get HTML content
|
|
let html = doctree.page_get_html("my-collection", "page-name")?;
|
|
println!("HTML content: {}", html);
|
|
|
|
Ok(())
|
|
}
|
|
```
|
|
|
|
## Testing Strategy
|
|
|
|
1. **Unit Tests**
|
|
- Test individual components in isolation
|
|
- Mock Redis for testing storage operations
|
|
- Test utility functions
|
|
|
|
2. **Integration Tests**
|
|
- Test the interaction between components
|
|
- Test the builder pattern
|
|
- Test include processing
|
|
|
|
3. **End-to-End Tests**
|
|
- Test the complete workflow with real files
|
|
- Test the CLI interface
|
|
|
|
## Timeline
|
|
|
|
1. **Project Setup and Dependencies**: 1 day
|
|
2. **Core Library Structure**: 2 days
|
|
3. **Collection Implementation**: 2 days
|
|
4. **DocTree Implementation**: 2 days
|
|
5. **Include Processor Implementation**: 1 day
|
|
6. **CLI Example**: 1 day
|
|
7. **Testing and Documentation**: 2 days
|
|
|
|
Total estimated time: 11 days |