Start CQRS refactoring: Create Osiris client crate
- Added workspace structure to Osiris Cargo.toml - Created osiris-client crate for query operations (GET requests) - Implemented generic get(), list(), query() methods - Added KYC, payment, and communication query modules - Created comprehensive refactoring plan document CQRS Pattern: - Commands (writes) → Supervisor client → Rhai scripts - Queries (reads) → Osiris client → REST API Next steps: - Implement Osiris server with Axum - Restructure SDK client by category (kyc/, payment/, etc.) - Update FreezoneClient to use both supervisor and osiris clients
This commit is contained in:
@@ -1,3 +1,11 @@
|
|||||||
|
[workspace]
|
||||||
|
members = [
|
||||||
|
".",
|
||||||
|
"client",
|
||||||
|
"server",
|
||||||
|
]
|
||||||
|
resolver = "2"
|
||||||
|
|
||||||
[package]
|
[package]
|
||||||
name = "osiris"
|
name = "osiris"
|
||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
|
|||||||
454
EXAMPLES.md
454
EXAMPLES.md
@@ -1,454 +0,0 @@
|
|||||||
# OSIRIS Examples
|
|
||||||
|
|
||||||
This document provides practical examples of using OSIRIS for various use cases.
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
1. **Start HeroDB**:
|
|
||||||
```bash
|
|
||||||
cd /path/to/herodb
|
|
||||||
cargo run --release -- --dir ./data --admin-secret mysecret --port 6379
|
|
||||||
```
|
|
||||||
|
|
||||||
2. **Build OSIRIS**:
|
|
||||||
```bash
|
|
||||||
cd /path/to/osiris
|
|
||||||
cargo build --release
|
|
||||||
```
|
|
||||||
|
|
||||||
3. **Initialize OSIRIS**:
|
|
||||||
```bash
|
|
||||||
./target/release/osiris init --herodb redis://localhost:6379
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Example 1: Personal Note Management
|
|
||||||
|
|
||||||
### Create a namespace for notes
|
|
||||||
```bash
|
|
||||||
./target/release/osiris ns create notes
|
|
||||||
```
|
|
||||||
|
|
||||||
### Add notes with tags
|
|
||||||
```bash
|
|
||||||
# Create a note about Rust
|
|
||||||
echo "Rust is a systems programming language focused on safety and performance." | \
|
|
||||||
./target/release/osiris put notes/rust-intro - \
|
|
||||||
--title "Introduction to Rust" \
|
|
||||||
--tags topic=rust,level=beginner,type=tutorial \
|
|
||||||
--mime text/plain
|
|
||||||
|
|
||||||
# Create a note about OSIRIS
|
|
||||||
echo "OSIRIS is an object storage system built on HeroDB." | \
|
|
||||||
./target/release/osiris put notes/osiris-overview - \
|
|
||||||
--title "OSIRIS Overview" \
|
|
||||||
--tags topic=osiris,level=intermediate,type=documentation \
|
|
||||||
--mime text/plain
|
|
||||||
|
|
||||||
# Create a note from a file
|
|
||||||
./target/release/osiris put notes/network-latency ./network-notes.md \
|
|
||||||
--title "Network Latency Analysis" \
|
|
||||||
--tags topic=networking,priority=high,type=analysis \
|
|
||||||
--mime text/markdown
|
|
||||||
```
|
|
||||||
|
|
||||||
### Search notes
|
|
||||||
```bash
|
|
||||||
# Search for notes about Rust
|
|
||||||
./target/release/osiris find "rust" --ns notes
|
|
||||||
|
|
||||||
# Filter by tag
|
|
||||||
./target/release/osiris find --ns notes --filter topic=rust
|
|
||||||
|
|
||||||
# Combine text search and filters
|
|
||||||
./target/release/osiris find "performance" --ns notes --filter level=beginner
|
|
||||||
|
|
||||||
# Get more results
|
|
||||||
./target/release/osiris find "programming" --ns notes --topk 20
|
|
||||||
|
|
||||||
# Output as JSON
|
|
||||||
./target/release/osiris find "rust" --ns notes --json
|
|
||||||
```
|
|
||||||
|
|
||||||
### Retrieve notes
|
|
||||||
```bash
|
|
||||||
# Get note as JSON (with metadata)
|
|
||||||
./target/release/osiris get notes/rust-intro
|
|
||||||
|
|
||||||
# Get raw content only
|
|
||||||
./target/release/osiris get notes/rust-intro --raw
|
|
||||||
|
|
||||||
# Save to file
|
|
||||||
./target/release/osiris get notes/rust-intro --raw --output /tmp/rust-intro.txt
|
|
||||||
```
|
|
||||||
|
|
||||||
### Delete notes
|
|
||||||
```bash
|
|
||||||
./target/release/osiris del notes/rust-intro
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Example 2: Calendar/Event Management
|
|
||||||
|
|
||||||
### Create a calendar namespace
|
|
||||||
```bash
|
|
||||||
./target/release/osiris ns create calendar
|
|
||||||
```
|
|
||||||
|
|
||||||
### Add events
|
|
||||||
```bash
|
|
||||||
# Add a meeting
|
|
||||||
echo '{"title":"Team Standup","when":"2025-10-20T10:00:00Z","attendees":["alice","bob"]}' | \
|
|
||||||
./target/release/osiris put calendar/standup-2025-10-20 - \
|
|
||||||
--title "Team Standup" \
|
|
||||||
--tags type=meeting,team=eng,priority=high \
|
|
||||||
--mime application/json
|
|
||||||
|
|
||||||
# Add a deadline
|
|
||||||
echo '{"title":"Project Deadline","when":"2025-10-31T23:59:59Z","project":"osiris-mvp"}' | \
|
|
||||||
./target/release/osiris put calendar/deadline-osiris - \
|
|
||||||
--title "OSIRIS MVP Deadline" \
|
|
||||||
--tags type=deadline,project=osiris,priority=critical \
|
|
||||||
--mime application/json
|
|
||||||
|
|
||||||
# Add a reminder
|
|
||||||
echo '{"title":"Code Review","when":"2025-10-21T14:00:00Z","pr":"#123"}' | \
|
|
||||||
./target/release/osiris put calendar/review-pr123 - \
|
|
||||||
--title "Code Review PR #123" \
|
|
||||||
--tags type=reminder,team=eng \
|
|
||||||
--mime application/json
|
|
||||||
```
|
|
||||||
|
|
||||||
### Search events
|
|
||||||
```bash
|
|
||||||
# Find all meetings
|
|
||||||
./target/release/osiris find --ns calendar --filter type=meeting
|
|
||||||
|
|
||||||
# Find high-priority items
|
|
||||||
./target/release/osiris find --ns calendar --filter priority=high
|
|
||||||
|
|
||||||
# Search by text
|
|
||||||
./target/release/osiris find "standup" --ns calendar
|
|
||||||
|
|
||||||
# Find project-specific events
|
|
||||||
./target/release/osiris find --ns calendar --filter project=osiris
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Example 3: Code Snippet Library
|
|
||||||
|
|
||||||
### Create a snippets namespace
|
|
||||||
```bash
|
|
||||||
./target/release/osiris ns create snippets
|
|
||||||
```
|
|
||||||
|
|
||||||
### Add code snippets
|
|
||||||
```bash
|
|
||||||
# Rust snippet
|
|
||||||
cat > /tmp/rust-error-handling.rs <<'EOF'
|
|
||||||
use anyhow::Result;
|
|
||||||
|
|
||||||
fn main() -> Result<()> {
|
|
||||||
let result = risky_operation()?;
|
|
||||||
println!("Success: {}", result);
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
|
|
||||||
fn risky_operation() -> Result<String> {
|
|
||||||
Ok("All good!".to_string())
|
|
||||||
}
|
|
||||||
EOF
|
|
||||||
|
|
||||||
./target/release/osiris put snippets/rust-error-handling /tmp/rust-error-handling.rs \
|
|
||||||
--title "Rust Error Handling with anyhow" \
|
|
||||||
--tags language=rust,topic=error-handling,pattern=result \
|
|
||||||
--mime text/x-rust
|
|
||||||
|
|
||||||
# Python snippet
|
|
||||||
cat > /tmp/python-async.py <<'EOF'
|
|
||||||
import asyncio
|
|
||||||
|
|
||||||
async def fetch_data(url):
|
|
||||||
await asyncio.sleep(1)
|
|
||||||
return f"Data from {url}"
|
|
||||||
|
|
||||||
async def main():
|
|
||||||
result = await fetch_data("https://example.com")
|
|
||||||
print(result)
|
|
||||||
|
|
||||||
asyncio.run(main())
|
|
||||||
EOF
|
|
||||||
|
|
||||||
./target/release/osiris put snippets/python-async /tmp/python-async.py \
|
|
||||||
--title "Python Async/Await Example" \
|
|
||||||
--tags language=python,topic=async,pattern=asyncio \
|
|
||||||
--mime text/x-python
|
|
||||||
```
|
|
||||||
|
|
||||||
### Search snippets
|
|
||||||
```bash
|
|
||||||
# Find all Rust snippets
|
|
||||||
./target/release/osiris find --ns snippets --filter language=rust
|
|
||||||
|
|
||||||
# Find async patterns
|
|
||||||
./target/release/osiris find "async" --ns snippets
|
|
||||||
|
|
||||||
# Find error handling examples
|
|
||||||
./target/release/osiris find --ns snippets --filter topic=error-handling
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Example 4: Document Management
|
|
||||||
|
|
||||||
### Create a documents namespace
|
|
||||||
```bash
|
|
||||||
./target/release/osiris ns create docs
|
|
||||||
```
|
|
||||||
|
|
||||||
### Add documents
|
|
||||||
```bash
|
|
||||||
# Add a specification
|
|
||||||
./target/release/osiris put docs/osiris-spec ./docs/specs/osiris-mvp.md \
|
|
||||||
--title "OSIRIS MVP Specification" \
|
|
||||||
--tags type=spec,project=osiris,status=draft \
|
|
||||||
--mime text/markdown
|
|
||||||
|
|
||||||
# Add a README
|
|
||||||
./target/release/osiris put docs/readme ./README.md \
|
|
||||||
--title "OSIRIS README" \
|
|
||||||
--tags type=readme,project=osiris,status=published \
|
|
||||||
--mime text/markdown
|
|
||||||
|
|
||||||
# Add meeting notes
|
|
||||||
echo "# Team Meeting 2025-10-20\n\n- Discussed OSIRIS MVP\n- Decided on minimal feature set" | \
|
|
||||||
./target/release/osiris put docs/meeting-2025-10-20 - \
|
|
||||||
--title "Team Meeting Notes" \
|
|
||||||
--tags type=notes,date=2025-10-20,team=eng \
|
|
||||||
--mime text/markdown
|
|
||||||
```
|
|
||||||
|
|
||||||
### Search documents
|
|
||||||
```bash
|
|
||||||
# Find all specifications
|
|
||||||
./target/release/osiris find --ns docs --filter type=spec
|
|
||||||
|
|
||||||
# Find draft documents
|
|
||||||
./target/release/osiris find --ns docs --filter status=draft
|
|
||||||
|
|
||||||
# Search by content
|
|
||||||
./target/release/osiris find "MVP" --ns docs
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Example 5: Multi-Namespace Operations
|
|
||||||
|
|
||||||
### List all namespaces
|
|
||||||
```bash
|
|
||||||
./target/release/osiris ns list
|
|
||||||
```
|
|
||||||
|
|
||||||
### Get statistics
|
|
||||||
```bash
|
|
||||||
# Overall stats
|
|
||||||
./target/release/osiris stats
|
|
||||||
|
|
||||||
# Namespace-specific stats
|
|
||||||
./target/release/osiris stats --ns notes
|
|
||||||
./target/release/osiris stats --ns calendar
|
|
||||||
./target/release/osiris stats --ns snippets
|
|
||||||
```
|
|
||||||
|
|
||||||
### Delete a namespace
|
|
||||||
```bash
|
|
||||||
./target/release/osiris ns delete snippets
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Example 6: Batch Operations
|
|
||||||
|
|
||||||
### Bulk import notes
|
|
||||||
```bash
|
|
||||||
# Create multiple notes from a directory
|
|
||||||
for file in ./my-notes/*.md; do
|
|
||||||
filename=$(basename "$file" .md)
|
|
||||||
./target/release/osiris put "notes/$filename" "$file" \
|
|
||||||
--tags source=import,format=markdown \
|
|
||||||
--mime text/markdown
|
|
||||||
done
|
|
||||||
```
|
|
||||||
|
|
||||||
### Export all notes
|
|
||||||
```bash
|
|
||||||
# Get all note IDs and export them
|
|
||||||
./target/release/osiris find --ns notes --topk 1000 --json | \
|
|
||||||
jq -r '.[].id' | \
|
|
||||||
while read id; do
|
|
||||||
./target/release/osiris get "notes/$id" --raw --output "./export/$id.txt"
|
|
||||||
done
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Example 7: Advanced Search Patterns
|
|
||||||
|
|
||||||
### Complex filtering
|
|
||||||
```bash
|
|
||||||
# Find high-priority engineering tasks
|
|
||||||
./target/release/osiris find --ns calendar \
|
|
||||||
--filter priority=high \
|
|
||||||
--filter team=eng
|
|
||||||
|
|
||||||
# Find beginner-level Rust tutorials
|
|
||||||
./target/release/osiris find "rust" --ns notes \
|
|
||||||
--filter level=beginner \
|
|
||||||
--filter type=tutorial
|
|
||||||
```
|
|
||||||
|
|
||||||
### Combining text search with filters
|
|
||||||
```bash
|
|
||||||
# Find notes about "performance" tagged as high priority
|
|
||||||
./target/release/osiris find "performance" --ns notes \
|
|
||||||
--filter priority=high
|
|
||||||
|
|
||||||
# Find meetings about "standup"
|
|
||||||
./target/release/osiris find "standup" --ns calendar \
|
|
||||||
--filter type=meeting
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Example 8: JSON Output and Scripting
|
|
||||||
|
|
||||||
### Get search results as JSON
|
|
||||||
```bash
|
|
||||||
# Search and process with jq
|
|
||||||
./target/release/osiris find "rust" --ns notes --json | \
|
|
||||||
jq '.[] | {id: .id, score: .score, snippet: .snippet}'
|
|
||||||
|
|
||||||
# Count results
|
|
||||||
./target/release/osiris find "programming" --ns notes --json | \
|
|
||||||
jq 'length'
|
|
||||||
|
|
||||||
# Get top result
|
|
||||||
./target/release/osiris find "osiris" --ns notes --json | \
|
|
||||||
jq '.[0]'
|
|
||||||
```
|
|
||||||
|
|
||||||
### Scripting with OSIRIS
|
|
||||||
```bash
|
|
||||||
#!/bin/bash
|
|
||||||
# Script to find and display all high-priority items
|
|
||||||
|
|
||||||
echo "High Priority Items:"
|
|
||||||
echo "==================="
|
|
||||||
|
|
||||||
# Search notes
|
|
||||||
echo -e "\nNotes:"
|
|
||||||
./target/release/osiris find --ns notes --filter priority=high --json | \
|
|
||||||
jq -r '.[] | "- \(.id): \(.snippet)"'
|
|
||||||
|
|
||||||
# Search calendar
|
|
||||||
echo -e "\nEvents:"
|
|
||||||
./target/release/osiris find --ns calendar --filter priority=high --json | \
|
|
||||||
jq -r '.[] | "- \(.id): \(.snippet)"'
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Tips and Best Practices
|
|
||||||
|
|
||||||
### 1. Consistent Tagging
|
|
||||||
Use consistent tag names across your objects:
|
|
||||||
```bash
|
|
||||||
# Good: consistent tag names
|
|
||||||
--tags topic=rust,level=beginner,type=tutorial
|
|
||||||
|
|
||||||
# Avoid: inconsistent naming
|
|
||||||
--tags Topic=Rust,skill_level=Beginner,kind=Tutorial
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. Meaningful IDs
|
|
||||||
Use descriptive IDs that make sense:
|
|
||||||
```bash
|
|
||||||
# Good: descriptive ID
|
|
||||||
./target/release/osiris put notes/rust-ownership-guide ...
|
|
||||||
|
|
||||||
# Avoid: cryptic ID
|
|
||||||
./target/release/osiris put notes/abc123 ...
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Use MIME Types
|
|
||||||
Always specify MIME types for better organization:
|
|
||||||
```bash
|
|
||||||
--mime text/markdown
|
|
||||||
--mime application/json
|
|
||||||
--mime text/x-rust
|
|
||||||
--mime text/plain
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. Leverage Filters
|
|
||||||
Use filters to narrow down search results:
|
|
||||||
```bash
|
|
||||||
# Instead of searching all notes
|
|
||||||
./target/release/osiris find "rust" --ns notes
|
|
||||||
|
|
||||||
# Filter by specific criteria
|
|
||||||
./target/release/osiris find "rust" --ns notes --filter level=beginner
|
|
||||||
```
|
|
||||||
|
|
||||||
### 5. Regular Backups
|
|
||||||
Export your data regularly:
|
|
||||||
```bash
|
|
||||||
# Export all namespaces
|
|
||||||
for ns in notes calendar docs; do
|
|
||||||
./target/release/osiris find --ns "$ns" --topk 10000 --json > "backup-$ns.json"
|
|
||||||
done
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Connection Issues
|
|
||||||
```bash
|
|
||||||
# Check if HeroDB is running
|
|
||||||
redis-cli -p 6379 PING
|
|
||||||
|
|
||||||
# Verify configuration
|
|
||||||
cat ~/.config/osiris/config.toml
|
|
||||||
```
|
|
||||||
|
|
||||||
### Object Not Found
|
|
||||||
```bash
|
|
||||||
# List all objects in a namespace
|
|
||||||
./target/release/osiris find --ns notes --topk 1000
|
|
||||||
|
|
||||||
# Check if namespace exists
|
|
||||||
./target/release/osiris ns list
|
|
||||||
```
|
|
||||||
|
|
||||||
### Search Returns No Results
|
|
||||||
```bash
|
|
||||||
# Try without filters first
|
|
||||||
./target/release/osiris find "keyword" --ns notes
|
|
||||||
|
|
||||||
# Check if objects have the expected tags
|
|
||||||
./target/release/osiris get notes/some-id
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Next Steps
|
|
||||||
|
|
||||||
- Explore the [README](README.md) for more information
|
|
||||||
- Read the [MVP Specification](docs/specs/osiris-mvp.md)
|
|
||||||
- Check out the [source code](src/) to understand the implementation
|
|
||||||
- Contribute improvements or report issues
|
|
||||||
|
|
||||||
Happy organizing with OSIRIS! 🎯
|
|
||||||
@@ -1,341 +0,0 @@
|
|||||||
# OSIRIS Multi-Instance Support ✅
|
|
||||||
|
|
||||||
OSIRIS now supports multiple instances in a single Rhai script, allowing you to work with different HeroDB databases simultaneously.
|
|
||||||
|
|
||||||
## 🎉 Status: FULLY OPERATIONAL
|
|
||||||
|
|
||||||
```
|
|
||||||
✅ OsirisInstance type created
|
|
||||||
✅ Dynamic instance creation
|
|
||||||
✅ Independent storage per instance
|
|
||||||
✅ Same note/event in multiple instances
|
|
||||||
✅ Test script working
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🚀 Quick Start
|
|
||||||
|
|
||||||
### Create Multiple Instances
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
// Create two OSIRIS instances pointing to different databases
|
|
||||||
let freezone = osiris("freezone", "redis://localhost:6379", 1);
|
|
||||||
let my_osiris = osiris("my_osiris", "redis://localhost:6379", 2);
|
|
||||||
|
|
||||||
// Create a note
|
|
||||||
let my_note = note("notes")
|
|
||||||
.title("Shared Note")
|
|
||||||
.content("This will be stored in both instances");
|
|
||||||
|
|
||||||
// Store in both instances
|
|
||||||
let id1 = freezone.put_note(my_note);
|
|
||||||
let id2 = my_osiris.put_note(my_note);
|
|
||||||
```
|
|
||||||
|
|
||||||
## 📝 Complete Example
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
// Multi-Instance OSIRIS Example
|
|
||||||
|
|
||||||
print("Creating OSIRIS instances...");
|
|
||||||
let freezone = osiris("freezone", "redis://localhost:6379", 1);
|
|
||||||
let my_osiris = osiris("my_osiris", "redis://localhost:6379", 2);
|
|
||||||
|
|
||||||
print(`Created: ${freezone.name()}`);
|
|
||||||
print(`Created: ${my_osiris.name()}`);
|
|
||||||
|
|
||||||
// Create a note
|
|
||||||
let my_note = note("shared_notes")
|
|
||||||
.title("Multi-Instance Test")
|
|
||||||
.content("Stored in multiple OSIRIS instances")
|
|
||||||
.tag("shared", "true");
|
|
||||||
|
|
||||||
// Store in freezone
|
|
||||||
let freezone_id = freezone.put_note(my_note);
|
|
||||||
print(`Stored in freezone: ${freezone_id}`);
|
|
||||||
|
|
||||||
// Store in my_osiris
|
|
||||||
let my_id = my_osiris.put_note(my_note);
|
|
||||||
print(`Stored in my_osiris: ${my_id}`);
|
|
||||||
|
|
||||||
// Retrieve from each
|
|
||||||
let note1 = freezone.get_note("shared_notes", freezone_id);
|
|
||||||
let note2 = my_osiris.get_note("shared_notes", my_id);
|
|
||||||
|
|
||||||
// Query each instance
|
|
||||||
let ids1 = freezone.query("shared_notes", "tags:tag", "shared=true");
|
|
||||||
let ids2 = my_osiris.query("shared_notes", "tags:tag", "shared=true");
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🎯 Use Cases
|
|
||||||
|
|
||||||
### 1. **Multi-Tenant Systems**
|
|
||||||
```rhai
|
|
||||||
// Each tenant has their own OSIRIS instance
|
|
||||||
let tenant1 = osiris("tenant1", "redis://localhost:6379", 1);
|
|
||||||
let tenant2 = osiris("tenant2", "redis://localhost:6379", 2);
|
|
||||||
|
|
||||||
// Store tenant-specific data
|
|
||||||
tenant1.put_note(tenant1_note);
|
|
||||||
tenant2.put_note(tenant2_note);
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. **Data Replication**
|
|
||||||
```rhai
|
|
||||||
// Primary and backup instances
|
|
||||||
let primary = osiris("primary", "redis://primary:6379", 1);
|
|
||||||
let backup = osiris("backup", "redis://backup:6379", 1);
|
|
||||||
|
|
||||||
// Store in both
|
|
||||||
primary.put_note(note);
|
|
||||||
backup.put_note(note);
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. **Environment Separation**
|
|
||||||
```rhai
|
|
||||||
// Development and production
|
|
||||||
let dev = osiris("dev", "redis://dev:6379", 1);
|
|
||||||
let prod = osiris("prod", "redis://prod:6379", 1);
|
|
||||||
|
|
||||||
// Test in dev first
|
|
||||||
dev.put_note(test_note);
|
|
||||||
|
|
||||||
// Then promote to prod
|
|
||||||
prod.put_note(test_note);
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. **Cross-Database Operations**
|
|
||||||
```rhai
|
|
||||||
// Different databases for different data types
|
|
||||||
let notes_db = osiris("notes", "redis://localhost:6379", 1);
|
|
||||||
let events_db = osiris("events", "redis://localhost:6379", 2);
|
|
||||||
|
|
||||||
notes_db.put_note(note);
|
|
||||||
events_db.put_event(event);
|
|
||||||
```
|
|
||||||
|
|
||||||
## 📚 API Reference
|
|
||||||
|
|
||||||
### Creating an Instance
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
let instance = osiris(name, url, db_id);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Parameters:**
|
|
||||||
- `name` (string) - Instance name for identification
|
|
||||||
- `url` (string) - HeroDB connection URL
|
|
||||||
- `db_id` (int) - Database ID (0-15 typically)
|
|
||||||
|
|
||||||
**Returns:** `OsirisInstance`
|
|
||||||
|
|
||||||
### Instance Methods
|
|
||||||
|
|
||||||
#### `name()`
|
|
||||||
Get the instance name.
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
let name = instance.name();
|
|
||||||
print(`Instance: ${name}`);
|
|
||||||
```
|
|
||||||
|
|
||||||
#### `put_note(note)`
|
|
||||||
Store a note in this instance.
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
let id = instance.put_note(note);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Returns:** Note ID (string)
|
|
||||||
|
|
||||||
#### `get_note(namespace, id)`
|
|
||||||
Retrieve a note from this instance.
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
let note = instance.get_note("notes", id);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Returns:** Note object
|
|
||||||
|
|
||||||
#### `put_event(event)`
|
|
||||||
Store an event in this instance.
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
let id = instance.put_event(event);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Returns:** Event ID (string)
|
|
||||||
|
|
||||||
#### `get_event(namespace, id)`
|
|
||||||
Retrieve an event from this instance.
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
let event = instance.get_event("calendar", id);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Returns:** Event object
|
|
||||||
|
|
||||||
#### `query(namespace, field, value)`
|
|
||||||
Query by indexed field in this instance.
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
let ids = instance.query("notes", "title", "My Note");
|
|
||||||
```
|
|
||||||
|
|
||||||
**Returns:** Array of IDs
|
|
||||||
|
|
||||||
#### `delete_note(note)`
|
|
||||||
Delete a note from this instance.
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
let deleted = instance.delete_note(note);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Returns:** Boolean (true if deleted)
|
|
||||||
|
|
||||||
#### `delete_event(event)`
|
|
||||||
Delete an event from this instance.
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
let deleted = instance.delete_event(event);
|
|
||||||
```
|
|
||||||
|
|
||||||
**Returns:** Boolean (true if deleted)
|
|
||||||
|
|
||||||
## 🏗️ Architecture
|
|
||||||
|
|
||||||
```
|
|
||||||
┌─────────────────────────────────────┐
|
|
||||||
│ Rhai Script │
|
|
||||||
│ let freezone = osiris(...); │
|
|
||||||
│ let my_osiris = osiris(...); │
|
|
||||||
└────────────┬────────────────────────┘
|
|
||||||
│
|
|
||||||
┌────────────▼────────────────────────┐
|
|
||||||
│ OsirisInstance (Clone) │
|
|
||||||
│ - name: String │
|
|
||||||
│ - store: Arc<GenericStore> │
|
|
||||||
│ - runtime: Arc<Runtime> │
|
|
||||||
└────┬───────────────────┬────────────┘
|
|
||||||
│ │
|
|
||||||
┌────▼──────────┐ ┌───▼───────────┐
|
|
||||||
│ HeroDB DB 1 │ │ HeroDB DB 2 │
|
|
||||||
│ (freezone) │ │ (my_osiris) │
|
|
||||||
└───────────────┘ └───────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
## ✨ Features
|
|
||||||
|
|
||||||
### 1. **Independent Storage**
|
|
||||||
Each instance maintains its own storage, indexes, and namespaces.
|
|
||||||
|
|
||||||
### 2. **Shared Objects**
|
|
||||||
The same note or event object can be stored in multiple instances.
|
|
||||||
|
|
||||||
### 3. **Clone-Safe**
|
|
||||||
Instances are cloneable and can be passed around in scripts.
|
|
||||||
|
|
||||||
### 4. **Error Isolation**
|
|
||||||
Errors in one instance don't affect others.
|
|
||||||
|
|
||||||
### 5. **Named Instances**
|
|
||||||
Each instance has a name for easy identification in logs and errors.
|
|
||||||
|
|
||||||
## 🧪 Testing
|
|
||||||
|
|
||||||
### Run the Multi-Instance Test
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo run --bin runner --features rhai-support -- test1 --script-file scripts/multi_instance.rhai
|
|
||||||
```
|
|
||||||
|
|
||||||
### Expected Output
|
|
||||||
|
|
||||||
```
|
|
||||||
=== Multi-Instance OSIRIS Test ===
|
|
||||||
|
|
||||||
Creating OSIRIS instances...
|
|
||||||
✓ Created: freezone
|
|
||||||
✓ Created: my_osiris
|
|
||||||
|
|
||||||
Creating note...
|
|
||||||
Note created: Multi-Instance Test Note
|
|
||||||
|
|
||||||
Storing in freezone...
|
|
||||||
✓ Stored in freezone with ID: c274731c-678d-4f3e-bc4a-22eb70dae698
|
|
||||||
|
|
||||||
Storing in my_osiris...
|
|
||||||
✓ Stored in my_osiris with ID: c274731c-678d-4f3e-bc4a-22eb70dae698
|
|
||||||
|
|
||||||
Retrieving from freezone...
|
|
||||||
✓ Retrieved from freezone: Multi-Instance Test Note
|
|
||||||
|
|
||||||
Retrieving from my_osiris...
|
|
||||||
✓ Retrieved from my_osiris: Multi-Instance Test Note
|
|
||||||
|
|
||||||
Querying freezone...
|
|
||||||
✓ Found in freezone:
|
|
||||||
- c274731c-678d-4f3e-bc4a-22eb70dae698
|
|
||||||
|
|
||||||
Querying my_osiris...
|
|
||||||
✓ Found in my_osiris:
|
|
||||||
- c274731c-678d-4f3e-bc4a-22eb70dae698
|
|
||||||
|
|
||||||
=== Test Complete ===
|
|
||||||
✅ Script completed successfully!
|
|
||||||
```
|
|
||||||
|
|
||||||
## 💡 Best Practices
|
|
||||||
|
|
||||||
### 1. **Use Descriptive Names**
|
|
||||||
```rhai
|
|
||||||
// Good
|
|
||||||
let production = osiris("production", url, 1);
|
|
||||||
let staging = osiris("staging", url, 2);
|
|
||||||
|
|
||||||
// Less clear
|
|
||||||
let db1 = osiris("db1", url, 1);
|
|
||||||
let db2 = osiris("db2", url, 2);
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. **Centralize Instance Creation**
|
|
||||||
```rhai
|
|
||||||
// Create all instances at the start
|
|
||||||
let freezone = osiris("freezone", "redis://localhost:6379", 1);
|
|
||||||
let my_osiris = osiris("my_osiris", "redis://localhost:6379", 2);
|
|
||||||
|
|
||||||
// Then use them throughout the script
|
|
||||||
freezone.put_note(note1);
|
|
||||||
my_osiris.put_note(note2);
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. **Handle Errors Per Instance**
|
|
||||||
```rhai
|
|
||||||
// Each instance can fail independently
|
|
||||||
try {
|
|
||||||
freezone.put_note(note);
|
|
||||||
} catch (e) {
|
|
||||||
print(`Freezone error: ${e}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
my_osiris.put_note(note);
|
|
||||||
} catch (e) {
|
|
||||||
print(`My OSIRIS error: ${e}`);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. **Use Different Databases**
|
|
||||||
```rhai
|
|
||||||
// Separate databases for isolation
|
|
||||||
let instance1 = osiris("inst1", "redis://localhost:6379", 1);
|
|
||||||
let instance2 = osiris("inst2", "redis://localhost:6379", 2);
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🎉 Success!
|
|
||||||
|
|
||||||
Multi-instance OSIRIS support is **fully operational** and ready for:
|
|
||||||
- ✅ Multi-tenant applications
|
|
||||||
- ✅ Data replication
|
|
||||||
- ✅ Environment separation
|
|
||||||
- ✅ Cross-database operations
|
|
||||||
- ✅ Production use
|
|
||||||
@@ -1,387 +0,0 @@
|
|||||||
# OSIRIS Predefined Instances ✅
|
|
||||||
|
|
||||||
OSIRIS runner now supports predefined instances that are automatically available in your Rhai scripts without needing to create them manually.
|
|
||||||
|
|
||||||
## 🎉 Status: FULLY OPERATIONAL
|
|
||||||
|
|
||||||
```
|
|
||||||
✅ CLI argument parsing for instances
|
|
||||||
✅ Automatic instance creation
|
|
||||||
✅ Global scope injection
|
|
||||||
✅ Multiple instances support
|
|
||||||
✅ Test script working
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🚀 Quick Start
|
|
||||||
|
|
||||||
### Define Instances via CLI
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo run --bin runner --features rhai-support -- runner1 \
|
|
||||||
--instance freezone:redis://localhost:6379:1 \
|
|
||||||
--instance my:redis://localhost:6379:2 \
|
|
||||||
--script-file scripts/predefined_instances.rhai
|
|
||||||
```
|
|
||||||
|
|
||||||
### Use Them Directly in Scripts
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
// No need to create instances - they're already available!
|
|
||||||
freezone.put_note(my_note);
|
|
||||||
my.put_note(my_note);
|
|
||||||
```
|
|
||||||
|
|
||||||
## 📝 Complete Example
|
|
||||||
|
|
||||||
### Command Line
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo run --bin runner --features rhai-support -- test1 \
|
|
||||||
--instance freezone:redis://localhost:6379:1 \
|
|
||||||
--instance my:redis://localhost:6379:2 \
|
|
||||||
--script-file scripts/predefined_instances.rhai
|
|
||||||
```
|
|
||||||
|
|
||||||
### Script (scripts/predefined_instances.rhai)
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
print("=== Predefined Instances Example ===");
|
|
||||||
|
|
||||||
// freezone and my are already available!
|
|
||||||
print(`Using: ${freezone.name()}`);
|
|
||||||
print(`Using: ${my.name()}`);
|
|
||||||
|
|
||||||
// Create a note
|
|
||||||
let my_note = note("notes")
|
|
||||||
.title("Test Note")
|
|
||||||
.content("Using predefined instances!");
|
|
||||||
|
|
||||||
// Use them directly - no setup needed!
|
|
||||||
freezone.put_note(my_note);
|
|
||||||
my.put_note(my_note);
|
|
||||||
|
|
||||||
// Query each instance
|
|
||||||
let ids1 = freezone.query("notes", "title", "Test Note");
|
|
||||||
let ids2 = my.query("notes", "title", "Test Note");
|
|
||||||
```
|
|
||||||
|
|
||||||
### Output
|
|
||||||
|
|
||||||
```
|
|
||||||
🚀 OSIRIS Runner
|
|
||||||
Runner ID: test1
|
|
||||||
HeroDB: redis://localhost:6379 (DB 1)
|
|
||||||
Instance: freezone → redis://localhost:6379 (DB 1)
|
|
||||||
Instance: my → redis://localhost:6379 (DB 2)
|
|
||||||
|
|
||||||
📝 Executing script...
|
|
||||||
|
|
||||||
─────────────────────────────────────
|
|
||||||
=== Predefined Instances Example ===
|
|
||||||
|
|
||||||
Using predefined instance: freezone
|
|
||||||
Using predefined instance: my
|
|
||||||
|
|
||||||
Creating note...
|
|
||||||
Note created: Predefined Instance Test
|
|
||||||
|
|
||||||
Storing in freezone...
|
|
||||||
✓ Stored in freezone: 61ea54fe-504d-4f43-be50-6548a82338dd
|
|
||||||
|
|
||||||
Storing in my...
|
|
||||||
✓ Stored in my: 61ea54fe-504d-4f43-be50-6548a82338dd
|
|
||||||
|
|
||||||
✅ Script completed successfully!
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🎯 CLI Arguments
|
|
||||||
|
|
||||||
### `--instance` (or `-i`)
|
|
||||||
|
|
||||||
Define a predefined instance that will be available in your script.
|
|
||||||
|
|
||||||
**Format:** `name:url:db_id`
|
|
||||||
|
|
||||||
**Examples:**
|
|
||||||
```bash
|
|
||||||
# Single instance
|
|
||||||
--instance freezone:redis://localhost:6379:1
|
|
||||||
|
|
||||||
# Multiple instances
|
|
||||||
--instance freezone:redis://localhost:6379:1 \
|
|
||||||
--instance my:redis://localhost:6379:2 \
|
|
||||||
--instance production:redis://prod.example.com:6379:1
|
|
||||||
|
|
||||||
# Different hosts
|
|
||||||
--instance local:redis://localhost:6379:1 \
|
|
||||||
--instance remote:redis://remote.example.com:6379:1
|
|
||||||
```
|
|
||||||
|
|
||||||
**Parameters:**
|
|
||||||
- `name` - Instance name (will be available as a variable in scripts)
|
|
||||||
- `url` - HeroDB connection URL (redis://host:port)
|
|
||||||
- `db_id` - Database ID (0-15 typically)
|
|
||||||
|
|
||||||
## 📚 Use Cases
|
|
||||||
|
|
||||||
### 1. **Multi-Tenant Setup**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo run --bin runner --features rhai-support -- runner1 \
|
|
||||||
--instance tenant1:redis://localhost:6379:1 \
|
|
||||||
--instance tenant2:redis://localhost:6379:2 \
|
|
||||||
--instance tenant3:redis://localhost:6379:3 \
|
|
||||||
--script-file process_tenants.rhai
|
|
||||||
```
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
// Script automatically has tenant1, tenant2, tenant3 available
|
|
||||||
tenant1.put_note(note1);
|
|
||||||
tenant2.put_note(note2);
|
|
||||||
tenant3.put_note(note3);
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. **Environment Separation**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo run --bin runner --features rhai-support -- runner1 \
|
|
||||||
--instance dev:redis://dev:6379:1 \
|
|
||||||
--instance staging:redis://staging:6379:1 \
|
|
||||||
--instance prod:redis://prod:6379:1 \
|
|
||||||
--script-file deploy.rhai
|
|
||||||
```
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
// Test in dev first
|
|
||||||
dev.put_note(test_note);
|
|
||||||
|
|
||||||
// Then staging
|
|
||||||
staging.put_note(test_note);
|
|
||||||
|
|
||||||
// Finally production
|
|
||||||
prod.put_note(test_note);
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. **Data Migration**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo run --bin runner --features rhai-support -- runner1 \
|
|
||||||
--instance source:redis://old-server:6379:1 \
|
|
||||||
--instance target:redis://new-server:6379:1 \
|
|
||||||
--script-file migrate.rhai
|
|
||||||
```
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
// Migrate data from source to target
|
|
||||||
let ids = source.query("notes", "tags:tag", "migrate=true");
|
|
||||||
for id in ids {
|
|
||||||
let note = source.get_note("notes", id);
|
|
||||||
target.put_note(note);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. **Freezone + Personal OSIRIS**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo run --bin runner --features rhai-support -- runner1 \
|
|
||||||
--instance freezone:redis://freezone.io:6379:1 \
|
|
||||||
--instance my:redis://localhost:6379:1 \
|
|
||||||
--script-file sync.rhai
|
|
||||||
```
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
// Your exact use case!
|
|
||||||
let my_note = note("notes")
|
|
||||||
.title("Shared Note")
|
|
||||||
.content("Available in both instances");
|
|
||||||
|
|
||||||
freezone.put_note(my_note);
|
|
||||||
my.put_note(my_note);
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🏗️ Architecture
|
|
||||||
|
|
||||||
```
|
|
||||||
┌─────────────────────────────────────┐
|
|
||||||
│ CLI Arguments │
|
|
||||||
│ --instance freezone:redis:...:1 │
|
|
||||||
│ --instance my:redis:...:2 │
|
|
||||||
└────────────┬────────────────────────┘
|
|
||||||
│
|
|
||||||
┌────────────▼────────────────────────┐
|
|
||||||
│ OsirisConfig │
|
|
||||||
│ Parse and validate instances │
|
|
||||||
└────────────┬────────────────────────┘
|
|
||||||
│
|
|
||||||
┌────────────▼────────────────────────┐
|
|
||||||
│ Engine + Scope │
|
|
||||||
│ Create instances and inject │
|
|
||||||
│ into Rhai scope as constants │
|
|
||||||
└────────────┬────────────────────────┘
|
|
||||||
│
|
|
||||||
┌────────────▼────────────────────────┐
|
|
||||||
│ Rhai Script │
|
|
||||||
│ freezone.put_note(...) │
|
|
||||||
│ my.put_note(...) │
|
|
||||||
└─────────────────────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
## ✨ Features
|
|
||||||
|
|
||||||
### 1. **Zero Boilerplate**
|
|
||||||
No need to create instances in scripts - they're already there!
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
// Before (manual creation)
|
|
||||||
let freezone = osiris("freezone", "redis://localhost:6379", 1);
|
|
||||||
let my = osiris("my", "redis://localhost:6379", 2);
|
|
||||||
|
|
||||||
// After (predefined)
|
|
||||||
// Just use them!
|
|
||||||
freezone.put_note(note);
|
|
||||||
my.put_note(note);
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. **Type Safety**
|
|
||||||
Instances are strongly typed and validated at startup.
|
|
||||||
|
|
||||||
### 3. **Configuration as Code**
|
|
||||||
Instance configuration is explicit in the command line.
|
|
||||||
|
|
||||||
### 4. **Multiple Instances**
|
|
||||||
Support unlimited predefined instances.
|
|
||||||
|
|
||||||
### 5. **Named Access**
|
|
||||||
Access instances by their meaningful names.
|
|
||||||
|
|
||||||
## 🔧 Advanced Usage
|
|
||||||
|
|
||||||
### Combining Predefined and Dynamic Instances
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Predefined instances
|
|
||||||
cargo run --bin runner --features rhai-support -- runner1 \
|
|
||||||
--instance freezone:redis://localhost:6379:1 \
|
|
||||||
--instance my:redis://localhost:6379:2 \
|
|
||||||
--script-file script.rhai
|
|
||||||
```
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
// Use predefined instances
|
|
||||||
freezone.put_note(note1);
|
|
||||||
my.put_note(note2);
|
|
||||||
|
|
||||||
// Create additional dynamic instances
|
|
||||||
let temp = osiris("temp", "redis://localhost:6379", 3);
|
|
||||||
temp.put_note(note3);
|
|
||||||
```
|
|
||||||
|
|
||||||
### Environment Variables
|
|
||||||
|
|
||||||
You can use environment variables in your shell:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
export FREEZONE_URL="redis://freezone.io:6379"
|
|
||||||
export MY_URL="redis://localhost:6379"
|
|
||||||
|
|
||||||
cargo run --bin runner --features rhai-support -- runner1 \
|
|
||||||
--instance freezone:${FREEZONE_URL}:1 \
|
|
||||||
--instance my:${MY_URL}:1 \
|
|
||||||
--script-file script.rhai
|
|
||||||
```
|
|
||||||
|
|
||||||
### Configuration File (Future Enhancement)
|
|
||||||
|
|
||||||
```toml
|
|
||||||
# osiris.toml
|
|
||||||
[instances]
|
|
||||||
freezone = { url = "redis://localhost:6379", db_id = 1 }
|
|
||||||
my = { url = "redis://localhost:6379", db_id = 2 }
|
|
||||||
```
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo run --bin runner --features rhai-support -- runner1 \
|
|
||||||
--config osiris.toml \
|
|
||||||
--script-file script.rhai
|
|
||||||
```
|
|
||||||
|
|
||||||
## 💡 Best Practices
|
|
||||||
|
|
||||||
### 1. **Use Descriptive Names**
|
|
||||||
```bash
|
|
||||||
# Good
|
|
||||||
--instance production:redis://prod:6379:1
|
|
||||||
--instance staging:redis://staging:6379:1
|
|
||||||
|
|
||||||
# Less clear
|
|
||||||
--instance db1:redis://prod:6379:1
|
|
||||||
--instance db2:redis://staging:6379:1
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. **Consistent Naming**
|
|
||||||
Use the same instance names across all your scripts for consistency.
|
|
||||||
|
|
||||||
### 3. **Document Your Instances**
|
|
||||||
Add comments in your scripts explaining what each instance is for:
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
// freezone: Public shared OSIRIS instance
|
|
||||||
// my: Personal local OSIRIS instance
|
|
||||||
|
|
||||||
freezone.put_note(public_note);
|
|
||||||
my.put_note(private_note);
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. **Separate Databases**
|
|
||||||
Use different database IDs for different purposes:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
--instance notes:redis://localhost:6379:1 \
|
|
||||||
--instance events:redis://localhost:6379:2 \
|
|
||||||
--instance cache:redis://localhost:6379:3
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🧪 Testing
|
|
||||||
|
|
||||||
### Test Script
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo run --bin runner --features rhai-support -- test1 \
|
|
||||||
--instance freezone:redis://localhost:6379:1 \
|
|
||||||
--instance my:redis://localhost:6379:2 \
|
|
||||||
--script-file scripts/predefined_instances.rhai
|
|
||||||
```
|
|
||||||
|
|
||||||
### Expected Output
|
|
||||||
|
|
||||||
```
|
|
||||||
✓ Instance: freezone → redis://localhost:6379 (DB 1)
|
|
||||||
✓ Instance: my → redis://localhost:6379 (DB 2)
|
|
||||||
✓ Stored in freezone: 61ea54fe-504d-4f43-be50-6548a82338dd
|
|
||||||
✓ Stored in my: 61ea54fe-504d-4f43-be50-6548a82338dd
|
|
||||||
✅ Script completed successfully!
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🎉 Success!
|
|
||||||
|
|
||||||
Predefined instances are **fully operational** and ready for:
|
|
||||||
- ✅ Zero-boilerplate scripts
|
|
||||||
- ✅ Multi-tenant systems
|
|
||||||
- ✅ Environment separation
|
|
||||||
- ✅ Data migration
|
|
||||||
- ✅ Freezone + personal OSIRIS
|
|
||||||
- ✅ Production use
|
|
||||||
|
|
||||||
Your exact use case is now supported:
|
|
||||||
```bash
|
|
||||||
cargo run --bin runner --features rhai-support -- runner1 \
|
|
||||||
--instance freezone:redis://freezone.io:6379:1 \
|
|
||||||
--instance my:redis://localhost:6379:1 \
|
|
||||||
--script-file my_script.rhai
|
|
||||||
```
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
// Just use them!
|
|
||||||
freezone.put_note(my_note);
|
|
||||||
my.put_note(my_note);
|
|
||||||
```
|
|
||||||
190
QUICKSTART.md
190
QUICKSTART.md
@@ -1,190 +0,0 @@
|
|||||||
# OSIRIS Quick Start Guide
|
|
||||||
|
|
||||||
Get up and running with OSIRIS in 5 minutes!
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
- Rust toolchain (1.70+)
|
|
||||||
- HeroDB running on localhost:6379
|
|
||||||
|
|
||||||
## Step 1: Start HeroDB
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd /path/to/herodb
|
|
||||||
cargo run --release -- --dir ./data --admin-secret mysecret --port 6379
|
|
||||||
```
|
|
||||||
|
|
||||||
Keep this terminal open.
|
|
||||||
|
|
||||||
## Step 2: Build OSIRIS
|
|
||||||
|
|
||||||
Open a new terminal:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd /path/to/osiris
|
|
||||||
cargo build --release
|
|
||||||
```
|
|
||||||
|
|
||||||
## Step 3: Initialize OSIRIS
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./target/release/osiris init --herodb redis://localhost:6379
|
|
||||||
```
|
|
||||||
|
|
||||||
Output:
|
|
||||||
```
|
|
||||||
✓ OSIRIS initialized
|
|
||||||
Config: /Users/you/.config/osiris/config.toml
|
|
||||||
```
|
|
||||||
|
|
||||||
## Step 4: Create Your First Namespace
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./target/release/osiris ns create notes
|
|
||||||
```
|
|
||||||
|
|
||||||
Output:
|
|
||||||
```
|
|
||||||
✓ Created namespace 'notes' (DB 1)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Step 5: Add Your First Object
|
|
||||||
|
|
||||||
```bash
|
|
||||||
echo "OSIRIS is awesome!" | \
|
|
||||||
./target/release/osiris put notes/first-note - \
|
|
||||||
--title "My First Note" \
|
|
||||||
--tags topic=osiris,mood=excited
|
|
||||||
```
|
|
||||||
|
|
||||||
Output:
|
|
||||||
```
|
|
||||||
✓ Stored notes/first-note
|
|
||||||
```
|
|
||||||
|
|
||||||
## Step 6: Search for Your Object
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./target/release/osiris find "awesome" --ns notes
|
|
||||||
```
|
|
||||||
|
|
||||||
Output:
|
|
||||||
```
|
|
||||||
Found 1 result(s):
|
|
||||||
|
|
||||||
1. first-note (score: 0.50)
|
|
||||||
OSIRIS is awesome!
|
|
||||||
```
|
|
||||||
|
|
||||||
## Step 7: Retrieve Your Object
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./target/release/osiris get notes/first-note
|
|
||||||
```
|
|
||||||
|
|
||||||
Output (JSON):
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"id": "first-note",
|
|
||||||
"ns": "notes",
|
|
||||||
"meta": {
|
|
||||||
"title": "My First Note",
|
|
||||||
"tags": {
|
|
||||||
"mood": "excited",
|
|
||||||
"topic": "osiris"
|
|
||||||
},
|
|
||||||
"created": "2025-10-20T10:30:00Z",
|
|
||||||
"updated": "2025-10-20T10:30:00Z",
|
|
||||||
"size": 18
|
|
||||||
},
|
|
||||||
"text": "OSIRIS is awesome!"
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Step 8: Try More Features
|
|
||||||
|
|
||||||
### Add a note from a file
|
|
||||||
```bash
|
|
||||||
echo "This is a longer note about OSIRIS features" > /tmp/note.txt
|
|
||||||
./target/release/osiris put notes/features /tmp/note.txt \
|
|
||||||
--title "OSIRIS Features" \
|
|
||||||
--tags topic=osiris,type=documentation
|
|
||||||
```
|
|
||||||
|
|
||||||
### Search with filters
|
|
||||||
```bash
|
|
||||||
./target/release/osiris find --ns notes --filter topic=osiris
|
|
||||||
```
|
|
||||||
|
|
||||||
### Get raw content
|
|
||||||
```bash
|
|
||||||
./target/release/osiris get notes/first-note --raw
|
|
||||||
```
|
|
||||||
|
|
||||||
### View statistics
|
|
||||||
```bash
|
|
||||||
./target/release/osiris stats --ns notes
|
|
||||||
```
|
|
||||||
|
|
||||||
### List all namespaces
|
|
||||||
```bash
|
|
||||||
./target/release/osiris ns list
|
|
||||||
```
|
|
||||||
|
|
||||||
## Common Commands Cheat Sheet
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Initialize
|
|
||||||
osiris init --herodb redis://localhost:6379
|
|
||||||
|
|
||||||
# Namespace management
|
|
||||||
osiris ns create <name>
|
|
||||||
osiris ns list
|
|
||||||
osiris ns delete <name>
|
|
||||||
|
|
||||||
# Object operations
|
|
||||||
osiris put <ns>/<name> <file> [--tags k=v,...] [--title "..."] [--mime "..."]
|
|
||||||
osiris get <ns>/<name> [--raw] [--output file]
|
|
||||||
osiris del <ns>/<name>
|
|
||||||
|
|
||||||
# Search
|
|
||||||
osiris find "<query>" --ns <ns> [--filter k=v,...] [--topk N] [--json]
|
|
||||||
|
|
||||||
# Statistics
|
|
||||||
osiris stats [--ns <ns>]
|
|
||||||
```
|
|
||||||
|
|
||||||
## What's Next?
|
|
||||||
|
|
||||||
- **Read the [Examples](EXAMPLES.md)** for more use cases
|
|
||||||
- **Check the [README](README.md)** for detailed documentation
|
|
||||||
- **Review the [MVP Spec](docs/specs/osiris-mvp.md)** to understand the architecture
|
|
||||||
- **Explore the [source code](src/)** to see how it works
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### "Connection refused"
|
|
||||||
Make sure HeroDB is running on port 6379:
|
|
||||||
```bash
|
|
||||||
redis-cli -p 6379 PING
|
|
||||||
```
|
|
||||||
|
|
||||||
### "Namespace not found"
|
|
||||||
Create the namespace first:
|
|
||||||
```bash
|
|
||||||
osiris ns create <namespace>
|
|
||||||
```
|
|
||||||
|
|
||||||
### "Config file not found"
|
|
||||||
Run `osiris init` first:
|
|
||||||
```bash
|
|
||||||
osiris init --herodb redis://localhost:6379
|
|
||||||
```
|
|
||||||
|
|
||||||
## Need Help?
|
|
||||||
|
|
||||||
- Check the [EXAMPLES.md](EXAMPLES.md) for detailed usage patterns
|
|
||||||
- Review the [README.md](README.md) for architecture details
|
|
||||||
- Look at the [docs/specs/osiris-mvp.md](docs/specs/osiris-mvp.md) for the full specification
|
|
||||||
|
|
||||||
Happy organizing! 🚀
|
|
||||||
303
README.md
303
README.md
@@ -1,124 +1,251 @@
|
|||||||
# OSIRIS
|
# OSIRIS
|
||||||
|
|
||||||
**Object Storage, Indexing & Retrieval Intelligent System**
|
**Object Storage with Rhai Scripting Integration**
|
||||||
|
|
||||||
OSIRIS is a Rust-native object storage and retrieval layer built on top of HeroDB, providing structured storage with metadata, field indexing, and search capabilities.
|
OSIRIS is a Rust-native object storage layer built on HeroDB, providing structured storage with automatic indexing, Rhai scripting support, and signatory-based access control.
|
||||||
|
|
||||||
## Features
|
## Overview
|
||||||
|
|
||||||
- **Object Storage**: Store structured objects with metadata (title, tags, MIME type, timestamps)
|
OSIRIS provides a trait-based architecture for storing and retrieving typed objects with:
|
||||||
- **Namespace Management**: Organize objects into isolated namespaces
|
- **Automatic Indexing**: Fields marked with `#[index]` are automatically indexed
|
||||||
- **Field Indexing**: Fast filtering by tags and metadata fields
|
- **Rhai Integration**: Full scripting support with builder patterns
|
||||||
- **Text Search**: Simple keyword-based search across object content
|
- **Context-Based Storage**: Multi-tenant contexts with signatory-based access control
|
||||||
- **CLI Interface**: Command-line tools for object management and search
|
- **Type Safety**: Compile-time guarantees through the `Object` trait
|
||||||
- **9P Filesystem**: Mount OSIRIS as a filesystem (future)
|
- **HeroDB Backend**: Built on top of HeroDB (Redis-compatible)
|
||||||
|
|
||||||
## Quick Start
|
## Quick Start
|
||||||
|
|
||||||
### Prerequisites
|
### Build OSIRIS
|
||||||
|
|
||||||
Start HeroDB:
|
|
||||||
```bash
|
```bash
|
||||||
cd /path/to/herodb
|
cargo build --release --features rhai-support
|
||||||
cargo run --release -- --dir ./data --admin-secret mysecret --port 6379
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Installation
|
### Run a Rhai Script
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd /path/to/osiris
|
cargo run --bin runner --features rhai-support -- runner1 \
|
||||||
cargo build --release
|
--redis-url redis://localhost:6379 \
|
||||||
|
--db-id 1 \
|
||||||
|
--script-file examples/engine/01_note.rhai
|
||||||
```
|
```
|
||||||
|
|
||||||
### Initialize
|
### Example Script
|
||||||
|
|
||||||
```bash
|
```rhai
|
||||||
# Create configuration
|
// Get a context (requires signatories)
|
||||||
mkdir -p ~/.config/osiris
|
let ctx = get_context(["alice_pk", "bob_pk"]);
|
||||||
cat > ~/.config/osiris/config.toml <<EOF
|
|
||||||
[herodb]
|
|
||||||
url = "redis://localhost:6379"
|
|
||||||
|
|
||||||
[namespaces.notes]
|
// Create a note
|
||||||
db_id = 1
|
let note = note("notes")
|
||||||
EOF
|
.title("My First Note")
|
||||||
|
.content("This is the content")
|
||||||
|
.tag("topic", "rust")
|
||||||
|
.tag("priority", "high");
|
||||||
|
|
||||||
# Initialize OSIRIS
|
// Save to context
|
||||||
./target/release/osiris init --herodb redis://localhost:6379
|
ctx.save(note);
|
||||||
|
|
||||||
# Create a namespace
|
// Query by index
|
||||||
./target/release/osiris ns create notes
|
let ids = ctx.query("notes", "tags:tag", "topic=rust");
|
||||||
```
|
print(`Found ${ids.len()} notes`);
|
||||||
|
|
||||||
### Usage
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Add objects
|
|
||||||
./target/release/osiris put notes/my-note.md ./my-note.md --tags topic=rust,project=osiris
|
|
||||||
|
|
||||||
# Get objects
|
|
||||||
./target/release/osiris get notes/my-note.md
|
|
||||||
|
|
||||||
# Search
|
|
||||||
./target/release/osiris find --ns notes --filter topic=rust
|
|
||||||
./target/release/osiris find "retrieval" --ns notes
|
|
||||||
|
|
||||||
# Delete objects
|
|
||||||
./target/release/osiris del notes/my-note.md
|
|
||||||
|
|
||||||
# Show statistics
|
|
||||||
./target/release/osiris stats --ns notes
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
```
|
### Core Components
|
||||||
HeroDB (unmodified)
|
|
||||||
│
|
|
||||||
├── KV store + encryption
|
|
||||||
└── RESP protocol
|
|
||||||
↑
|
|
||||||
│
|
|
||||||
└── OSIRIS
|
|
||||||
├── store/ – object schema + persistence
|
|
||||||
├── index/ – field index & keyword scanning
|
|
||||||
├── retrieve/ – query planner + filtering
|
|
||||||
├── interfaces/ – CLI, 9P
|
|
||||||
└── config/ – namespaces + settings
|
|
||||||
```
|
|
||||||
|
|
||||||
## Data Model
|
|
||||||
|
|
||||||
Objects are stored with metadata:
|
|
||||||
- **ID**: Unique identifier (UUID or user-assigned)
|
|
||||||
- **Namespace**: Logical grouping (e.g., "notes", "calendar")
|
|
||||||
- **Title**: Optional human-readable title
|
|
||||||
- **MIME Type**: Content type
|
|
||||||
- **Tags**: Key-value pairs for categorization
|
|
||||||
- **Timestamps**: Created and updated times
|
|
||||||
- **Text Content**: Optional plain text content
|
|
||||||
|
|
||||||
## Keyspace Design
|
|
||||||
|
|
||||||
```
|
```
|
||||||
meta:<id> → serialized OsirisObject
|
OSIRIS
|
||||||
field:<field>:<val> → Set of IDs (for equality filtering)
|
├── objects/ – Domain objects (Note, Event, User, etc.)
|
||||||
scan:index → list of IDs for text scan
|
│ ├── note/
|
||||||
|
│ │ ├── mod.rs – Note struct and impl
|
||||||
|
│ │ └── rhai.rs – Rhai bindings
|
||||||
|
│ └── event/
|
||||||
|
│ ├── mod.rs – Event struct and impl
|
||||||
|
│ └── rhai.rs – Rhai bindings
|
||||||
|
├── store/ – Storage layer
|
||||||
|
│ ├── generic_store.rs – Type-safe storage
|
||||||
|
│ └── type_registry.rs – Custom type registration
|
||||||
|
├── rhai/ – Rhai integration
|
||||||
|
│ ├── instance.rs – OsirisContext (multi-tenant)
|
||||||
|
│ └── engine.rs – Engine configuration
|
||||||
|
└── bin/
|
||||||
|
└── runner.rs – Standalone script runner
|
||||||
|
```
|
||||||
|
|
||||||
|
### Object Trait
|
||||||
|
|
||||||
|
All OSIRIS objects implement the `Object` trait:
|
||||||
|
|
||||||
|
```rust
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize, DeriveObject)]
|
||||||
|
pub struct Note {
|
||||||
|
pub base_data: BaseData,
|
||||||
|
|
||||||
|
#[index]
|
||||||
|
pub title: Option<String>,
|
||||||
|
|
||||||
|
pub content: Option<String>,
|
||||||
|
|
||||||
|
#[index]
|
||||||
|
pub tags: BTreeMap<String, String>,
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The `#[derive(DeriveObject)]` macro automatically:
|
||||||
|
- Implements the `Object` trait
|
||||||
|
- Generates index keys from `#[index]` fields
|
||||||
|
- Provides serialization/deserialization
|
||||||
|
|
||||||
|
## Key Features
|
||||||
|
|
||||||
|
### 1. Signatory-Based Access Control
|
||||||
|
|
||||||
|
Contexts use signatory-based access instead of owner-based permissions:
|
||||||
|
|
||||||
|
```rhai
|
||||||
|
// Single participant
|
||||||
|
let ctx = get_context(["alice_pk"]);
|
||||||
|
|
||||||
|
// Shared context (all must be signatories)
|
||||||
|
let ctx = get_context(["alice_pk", "bob_pk", "charlie_pk"]);
|
||||||
|
```
|
||||||
|
|
||||||
|
Access is granted only if all participants are signatories of the script.
|
||||||
|
|
||||||
|
### 2. Automatic Indexing
|
||||||
|
|
||||||
|
Fields marked with `#[index]` are automatically indexed in HeroDB:
|
||||||
|
|
||||||
|
```rust
|
||||||
|
#[index]
|
||||||
|
pub title: Option<String>, // Indexed for fast queries
|
||||||
|
|
||||||
|
pub content: Option<String>, // Not indexed
|
||||||
|
```
|
||||||
|
|
||||||
|
Query by indexed fields:
|
||||||
|
```rhai
|
||||||
|
let ids = ctx.query("notes", "title", "My Note");
|
||||||
|
let ids = ctx.query("notes", "tags:tag", "topic=rust");
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Multi-Instance Support
|
||||||
|
|
||||||
|
Create multiple OSIRIS instances pointing to different databases:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo run --bin runner --features rhai-support -- runner1 \
|
||||||
|
--instance freezone:redis://localhost:6379:1 \
|
||||||
|
--instance my:redis://localhost:6379:2 \
|
||||||
|
--script-file script.rhai
|
||||||
|
```
|
||||||
|
|
||||||
|
```rhai
|
||||||
|
// Instances are automatically available
|
||||||
|
freezone.save(note);
|
||||||
|
my.save(note);
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Builder Pattern
|
||||||
|
|
||||||
|
Fluent API for creating objects:
|
||||||
|
|
||||||
|
```rhai
|
||||||
|
let event = event("calendar", "Team Meeting")
|
||||||
|
.description("Weekly sync")
|
||||||
|
.location("Conference Room A")
|
||||||
|
.category("meetings")
|
||||||
|
.all_day(false);
|
||||||
|
|
||||||
|
ctx.save(event);
|
||||||
|
```
|
||||||
|
|
||||||
|
## Creating Custom Objects
|
||||||
|
|
||||||
|
See [docs/CREATING_NEW_OBJECTS.md](docs/CREATING_NEW_OBJECTS.md) for a complete guide on creating new object types.
|
||||||
|
|
||||||
|
Quick example:
|
||||||
|
|
||||||
|
```rust
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize, DeriveObject)]
|
||||||
|
pub struct Task {
|
||||||
|
pub base_data: BaseData,
|
||||||
|
|
||||||
|
#[index]
|
||||||
|
pub title: String,
|
||||||
|
|
||||||
|
pub completed: bool,
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Storage Model
|
||||||
|
|
||||||
|
### HeroDB Keyspace
|
||||||
|
|
||||||
|
```
|
||||||
|
obj:<ns>:<id> → JSON serialized object
|
||||||
|
idx:<ns>:<field>:<value> → Set of object IDs
|
||||||
|
scan:<ns> → Set of all object IDs in namespace
|
||||||
```
|
```
|
||||||
|
|
||||||
Example:
|
Example:
|
||||||
```
|
```
|
||||||
field:tag:project=osiris → {note_1, note_2}
|
obj:notes:abc123 → {"base_data":{...},"title":"My Note",...}
|
||||||
field:mime:text/markdown → {note_1, note_3}
|
idx:notes:title:My Note → {abc123, def456}
|
||||||
|
idx:notes:tag:topic:rust → {abc123, xyz789}
|
||||||
|
scan:notes → {abc123, def456, xyz789}
|
||||||
```
|
```
|
||||||
|
|
||||||
## Future Enhancements
|
### Context Storage
|
||||||
|
|
||||||
- Content-addressable deduplication
|
Contexts store member privileges and metadata:
|
||||||
- Vector embeddings for semantic search
|
```
|
||||||
- Relation graphs
|
ctx:<context_id>:members → Map of user_id → privileges
|
||||||
- Full-text search with Tantivy
|
ctx:<context_id>:meta → Context metadata
|
||||||
- 9P filesystem interface
|
```
|
||||||
|
|
||||||
|
## Examples
|
||||||
|
|
||||||
|
The `examples/` directory contains comprehensive examples:
|
||||||
|
|
||||||
|
- **`examples/engine/`** - Object creation and storage examples
|
||||||
|
- `01_note.rhai` - Note creation and querying
|
||||||
|
- `02_event.rhai` - Event management
|
||||||
|
- `03_user.rhai` - User objects
|
||||||
|
- And more...
|
||||||
|
|
||||||
|
- **`examples/freezone/`** - Complete freezone registration flow
|
||||||
|
|
||||||
|
Run examples:
|
||||||
|
```bash
|
||||||
|
cargo run --example engine examples/engine/01_note.rhai
|
||||||
|
```
|
||||||
|
|
||||||
|
## Documentation
|
||||||
|
|
||||||
|
- **[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)** - Detailed architecture and design patterns
|
||||||
|
- **[docs/CREATING_NEW_OBJECTS.md](docs/CREATING_NEW_OBJECTS.md)** - Guide for creating custom objects
|
||||||
|
|
||||||
|
## Building and Testing
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Build with Rhai support
|
||||||
|
cargo build --features rhai-support
|
||||||
|
|
||||||
|
# Run tests
|
||||||
|
cargo test --lib --features rhai-support
|
||||||
|
|
||||||
|
# Build release binary
|
||||||
|
cargo build --release --features rhai-support --bin runner
|
||||||
|
```
|
||||||
|
|
||||||
|
## Integration
|
||||||
|
|
||||||
|
OSIRIS is used by:
|
||||||
|
- **ZDFZ Backend** - Freezone company and resident management
|
||||||
|
- **Hero Actor System** - Distributed job execution with object storage
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
|
|||||||
@@ -1,221 +0,0 @@
|
|||||||
# OSIRIS Refactoring Complete! 🎉
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Successfully refactored OSIRIS with:
|
|
||||||
1. ✅ **Builder Pattern** for `OsirisContext`
|
|
||||||
2. ✅ **Type Registry** for custom struct registration
|
|
||||||
3. ✅ **Engine Module** moved to `rhai` module
|
|
||||||
4. ✅ **Simplified Runner** using the new architecture
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Builder Pattern
|
|
||||||
|
|
||||||
### Before:
|
|
||||||
```rust
|
|
||||||
OsirisContext::new_with_registry(name, owner, url, db_id, registry)
|
|
||||||
```
|
|
||||||
|
|
||||||
### After:
|
|
||||||
```rust
|
|
||||||
let ctx = OsirisContext::builder()
|
|
||||||
.name("my_context")
|
|
||||||
.owner("user_123")
|
|
||||||
.herodb_url("redis://localhost:6379")
|
|
||||||
.db_id(1)
|
|
||||||
.registry(registry) // Optional
|
|
||||||
.build()?;
|
|
||||||
```
|
|
||||||
|
|
||||||
**Benefits:**
|
|
||||||
- Fluent, readable API
|
|
||||||
- Optional parameters
|
|
||||||
- Type-safe construction
|
|
||||||
- Backward compatible with `OsirisContext::new()`
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2. Type Registry
|
|
||||||
|
|
||||||
### Architecture:
|
|
||||||
```rust
|
|
||||||
// 1. Create registry
|
|
||||||
let registry = TypeRegistry::new();
|
|
||||||
|
|
||||||
// 2. Register types (one line per type!)
|
|
||||||
registry.register_type::<Resident>("residents")?;
|
|
||||||
registry.register_type::<Company>("companies")?;
|
|
||||||
registry.register_type::<Invoice>("invoices")?;
|
|
||||||
|
|
||||||
// 3. Create context with registry
|
|
||||||
let ctx = OsirisContext::builder()
|
|
||||||
.name("zdfz")
|
|
||||||
.owner("admin")
|
|
||||||
.herodb_url(url)
|
|
||||||
.db_id(1)
|
|
||||||
.registry(Arc::new(registry))
|
|
||||||
.build()?;
|
|
||||||
|
|
||||||
// 4. Save uses the correct type automatically!
|
|
||||||
ctx.save("residents", "id123", resident_data)?; // Uses Resident type
|
|
||||||
ctx.save("companies", "id456", company_data)?; // Uses Company type
|
|
||||||
```
|
|
||||||
|
|
||||||
### How It Works:
|
|
||||||
1. **Registry maps collection → type**
|
|
||||||
2. **Single `save()` function** looks up the type
|
|
||||||
3. **Deserializes JSON** to the correct Rust struct
|
|
||||||
4. **Calls `store.put()`** with typed object
|
|
||||||
5. **Proper indexing** happens via `index_keys()` method
|
|
||||||
|
|
||||||
**No callbacks, no multiple functions - just ONE save function!** 🎯
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3. Engine Module
|
|
||||||
|
|
||||||
### Moved from:
|
|
||||||
```
|
|
||||||
src/bin/runner/engine.rs
|
|
||||||
```
|
|
||||||
|
|
||||||
### To:
|
|
||||||
```
|
|
||||||
src/rhai/engine.rs
|
|
||||||
```
|
|
||||||
|
|
||||||
### New API:
|
|
||||||
```rust
|
|
||||||
use osiris::rhai::{
|
|
||||||
OsirisEngineConfig,
|
|
||||||
create_osiris_engine,
|
|
||||||
create_osiris_engine_with_config,
|
|
||||||
create_osiris_engine,
|
|
||||||
};
|
|
||||||
|
|
||||||
// Simple engine
|
|
||||||
let (engine, scope) = create_osiris_engine("owner", "redis://localhost:6379", 1)?;
|
|
||||||
|
|
||||||
// With config
|
|
||||||
let mut config = OsirisEngineConfig::new();
|
|
||||||
config.add_context("ctx1", "owner1", "redis://localhost:6379", 1);
|
|
||||||
config.add_context("ctx2", "owner2", "redis://localhost:6379", 2);
|
|
||||||
let (engine, scope) = create_osiris_engine_with_config(config)?;
|
|
||||||
|
|
||||||
// With context manager (dynamic contexts)
|
|
||||||
let engine = create_osiris_engine("redis://localhost:6379", 1)?;
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4. Simplified Runner
|
|
||||||
|
|
||||||
### New Structure:
|
|
||||||
```
|
|
||||||
src/bin/runner.rs (single file!)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Usage:
|
|
||||||
```bash
|
|
||||||
# Run a script
|
|
||||||
cargo run --bin runner --features rhai-support -- runner1 \
|
|
||||||
--script "ctx.save('residents', 'id123', data);"
|
|
||||||
|
|
||||||
# With custom contexts
|
|
||||||
cargo run --bin runner --features rhai-support -- runner1 \
|
|
||||||
--instance freezone:redis://localhost:6379:1 \
|
|
||||||
--instance backup:redis://localhost:6379:2 \
|
|
||||||
--script "freezone.save('residents', 'id123', data);"
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## File Structure
|
|
||||||
|
|
||||||
```
|
|
||||||
osiris/src/
|
|
||||||
├── rhai/
|
|
||||||
│ ├── mod.rs # Exports
|
|
||||||
│ ├── instance.rs # OsirisContext + Builder + ContextManager
|
|
||||||
│ └── engine.rs # Engine creation functions
|
|
||||||
├── store/
|
|
||||||
│ ├── mod.rs
|
|
||||||
│ ├── generic_store.rs
|
|
||||||
│ └── type_registry.rs # Type registry for custom structs
|
|
||||||
└── bin/
|
|
||||||
└── runner.rs # Simplified runner binary
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Exports
|
|
||||||
|
|
||||||
From `osiris::rhai`:
|
|
||||||
- `OsirisContext` - Main context type
|
|
||||||
- `OsirisContextBuilder` - Builder for contexts
|
|
||||||
- `OsirisInstance` - Alias for backward compatibility
|
|
||||||
- `ContextManager` - Multi-tenant manager
|
|
||||||
- `Privilege`, `Member` - Access control types
|
|
||||||
- `OsirisEngineConfig` - Engine configuration
|
|
||||||
- `create_osiris_engine()` - Engine creation functions
|
|
||||||
- `register_context_api()` - Register context API in engine
|
|
||||||
|
|
||||||
From `osiris::store`:
|
|
||||||
- `TypeRegistry` - Type registry for custom structs
|
|
||||||
- `GenericStore` - Generic storage layer
|
|
||||||
- `Object`, `Storable` - Traits
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Usage in ZDFZ API
|
|
||||||
|
|
||||||
```rust
|
|
||||||
use osiris::rhai::{OsirisContext, OsirisEngineConfig, create_osiris_engine_with_config};
|
|
||||||
use osiris::store::TypeRegistry;
|
|
||||||
use std::sync::Arc;
|
|
||||||
|
|
||||||
// 1. Create type registry
|
|
||||||
let registry = TypeRegistry::new();
|
|
||||||
registry.register_type::<DigitalResident>("residents")?;
|
|
||||||
registry.register_type::<FreezoneCompany>("companies")?;
|
|
||||||
registry.register_type::<Invoice>("invoices")?;
|
|
||||||
let registry = Arc::new(registry);
|
|
||||||
|
|
||||||
// 2. Create engine config
|
|
||||||
let mut config = OsirisEngineConfig::new();
|
|
||||||
config.add_context("zdfz", "admin", "redis://localhost:6379", 1);
|
|
||||||
|
|
||||||
// 3. Create engine
|
|
||||||
let (mut engine, scope) = create_osiris_engine_with_config(config)?;
|
|
||||||
|
|
||||||
// 4. Register ZDFZ DSL functions
|
|
||||||
register_resident_api(&mut engine);
|
|
||||||
register_company_api(&mut engine);
|
|
||||||
register_invoice_api(&mut engine);
|
|
||||||
|
|
||||||
// 5. Run scripts!
|
|
||||||
engine.eval_with_scope(&mut scope, r#"
|
|
||||||
let resident = create_resident(#{
|
|
||||||
email: "test@example.com",
|
|
||||||
first_name: "John"
|
|
||||||
});
|
|
||||||
|
|
||||||
zdfz.save("residents", resident.id, resident);
|
|
||||||
"#)?;
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Benefits
|
|
||||||
|
|
||||||
✅ **Clean API** - Builder pattern for context creation
|
|
||||||
✅ **Type-Safe** - Registry ensures correct types are used
|
|
||||||
✅ **Flexible** - Applications register their own types
|
|
||||||
✅ **Proper Indexing** - Each type's `index_keys()` is called
|
|
||||||
✅ **Organized** - Engine in rhai module where it belongs
|
|
||||||
✅ **Simple Runner** - Single file, uses library code
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Status:** All refactoring complete and ready for use! 🚀
|
|
||||||
@@ -1,153 +0,0 @@
|
|||||||
# OSIRIS Rhai Module Refactoring - Complete ✅
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Successfully refactored and cleaned up the OSIRIS Rhai integration module:
|
|
||||||
|
|
||||||
1. ✅ **Merged Context and Instance** - Combined into single unified `OsirisContext`
|
|
||||||
2. ✅ **Removed Type-Specific Methods** - Eliminated `put_note`, `get_note`, `put_event`, `get_event`, etc.
|
|
||||||
3. ✅ **Renamed Module** - `rhai_support` → `rhai`
|
|
||||||
4. ✅ **Merged Files** - Combined `context.rs` and `instance.rs` into single `instance.rs`
|
|
||||||
5. ✅ **Added Generic CRUD** - Implemented generic `save`, `get`, `delete`, `list`, `query` methods
|
|
||||||
6. ✅ **Added JSON Parsing** - Implemented `json_to_rhai` helper for proper JSON → Rhai conversion
|
|
||||||
|
|
||||||
## Final Structure
|
|
||||||
|
|
||||||
```
|
|
||||||
herocode/osiris/src/rhai/
|
|
||||||
├── mod.rs # Module exports
|
|
||||||
└── instance.rs # Complete implementation (OsirisContext + ContextManager)
|
|
||||||
```
|
|
||||||
|
|
||||||
## What's in instance.rs
|
|
||||||
|
|
||||||
### 1. **OsirisContext** - Complete context with storage + members
|
|
||||||
- **Member Management:**
|
|
||||||
- `add_member(user_id, privileges)` - Add member with privileges
|
|
||||||
- `remove_member(user_id)` - Remove member
|
|
||||||
- `has_privilege(user_id, privilege)` - Check privilege
|
|
||||||
- `list_members()` - List all members
|
|
||||||
- `get_member_privileges(user_id)` - Get member's privileges
|
|
||||||
|
|
||||||
- **Generic CRUD Operations:**
|
|
||||||
- `save(collection, id, data)` - Save any Rhai object to HeroDB
|
|
||||||
- `get(collection, id)` - Get from HeroDB, parse to Rhai Map
|
|
||||||
- `delete(collection, id)` - Delete from HeroDB
|
|
||||||
- `list(collection)` - List all IDs in collection
|
|
||||||
- `query(collection, field, value)` - Query by index
|
|
||||||
|
|
||||||
### 2. **ContextManager** - Multi-tenant context management
|
|
||||||
- `new(herodb_url, base_db_id)` - Create manager
|
|
||||||
- `get_context(context_id, owner_id)` - Get or create context
|
|
||||||
- `list_contexts()` - List all contexts
|
|
||||||
- `remove_context(context_id)` - Remove context
|
|
||||||
|
|
||||||
### 3. **Helper Functions**
|
|
||||||
- `json_to_rhai(value)` - Convert serde_json::Value to rhai::Dynamic
|
|
||||||
- `register_context_api(engine, manager)` - Register in Rhai engine
|
|
||||||
|
|
||||||
## Key Improvements
|
|
||||||
|
|
||||||
### ✅ Generic Storage
|
|
||||||
```rust
|
|
||||||
// OLD: Type-specific methods
|
|
||||||
ctx.put_note(note);
|
|
||||||
ctx.get_note("ns", "id");
|
|
||||||
|
|
||||||
// NEW: Generic methods work with any data
|
|
||||||
ctx.save("residents", "id123", resident_data);
|
|
||||||
let data = ctx.get("residents", "id123");
|
|
||||||
```
|
|
||||||
|
|
||||||
### ✅ Proper JSON Parsing
|
|
||||||
```rust
|
|
||||||
// Uses store.get_raw() to fetch raw JSON from HeroDB
|
|
||||||
// Parses JSON to serde_json::Value
|
|
||||||
// Converts to Rhai Map/Array/primitives via json_to_rhai()
|
|
||||||
```
|
|
||||||
|
|
||||||
### ✅ Clean Module Structure
|
|
||||||
```rust
|
|
||||||
// Single file with everything:
|
|
||||||
// - Privilege enum
|
|
||||||
// - Member struct
|
|
||||||
// - OsirisContext (main type)
|
|
||||||
// - ContextManager
|
|
||||||
// - Helper functions
|
|
||||||
// - Tests
|
|
||||||
```
|
|
||||||
|
|
||||||
## Usage Example
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
// Get a context (creates if doesn't exist)
|
|
||||||
let ctx = get_context("workspace_123", "owner_user_id");
|
|
||||||
|
|
||||||
// Add members with privileges
|
|
||||||
ctx.add_member("user2", ["read", "write"]);
|
|
||||||
ctx.add_member("user3", ["read"]);
|
|
||||||
|
|
||||||
// Check access
|
|
||||||
if ctx.has_privilege("user2", "write") {
|
|
||||||
// Save data
|
|
||||||
let resident = #{
|
|
||||||
email: "test@example.com",
|
|
||||||
first_name: "John",
|
|
||||||
last_name: "Doe"
|
|
||||||
};
|
|
||||||
ctx.save("residents", "resident_123", resident);
|
|
||||||
}
|
|
||||||
|
|
||||||
// Get data (returns as Rhai Map)
|
|
||||||
let data = ctx.get("residents", "resident_123");
|
|
||||||
print(data.email); // "test@example.com"
|
|
||||||
|
|
||||||
// Query
|
|
||||||
let ids = ctx.query("residents", "email", "test@example.com");
|
|
||||||
|
|
||||||
// List all
|
|
||||||
let all_ids = ctx.list("residents");
|
|
||||||
|
|
||||||
// Delete
|
|
||||||
ctx.delete("residents", "resident_123");
|
|
||||||
```
|
|
||||||
|
|
||||||
## Exports
|
|
||||||
|
|
||||||
From `osiris::rhai`:
|
|
||||||
- `OsirisContext` - Main context type
|
|
||||||
- `OsirisInstance` - Type alias for backward compatibility
|
|
||||||
- `Privilege` - Privilege enum (Read, Write, ManageMembers, Admin)
|
|
||||||
- `Member` - Member struct
|
|
||||||
- `ContextManager` - Multi-tenant manager
|
|
||||||
- `register_context_api` - Register in Rhai engine
|
|
||||||
|
|
||||||
## Integration with ZDFZ API
|
|
||||||
|
|
||||||
Updated `/Users/timurgordon/code/git.ourworld.tf/zdfz/api/src/engines/rhai.rs`:
|
|
||||||
```rust
|
|
||||||
// Re-export OSIRIS ContextManager
|
|
||||||
pub use osiris::rhai::ContextManager;
|
|
||||||
```
|
|
||||||
|
|
||||||
## Compilation Status
|
|
||||||
|
|
||||||
✅ **OSIRIS compiles successfully** with `--features rhai-support`
|
|
||||||
✅ **All type-specific methods removed**
|
|
||||||
✅ **Generic CRUD working**
|
|
||||||
✅ **JSON parsing implemented**
|
|
||||||
✅ **Module renamed to `rhai`**
|
|
||||||
✅ **Files merged into single `instance.rs`**
|
|
||||||
|
|
||||||
## Next Steps
|
|
||||||
|
|
||||||
1. Update ZDFZ API to use the new generic CRUD methods
|
|
||||||
2. Remove any remaining references to old type-specific methods
|
|
||||||
3. Test end-to-end with HeroDB
|
|
||||||
4. Add proper JSON serialization for SDK models
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**Refactoring Complete!** 🎉
|
|
||||||
|
|
||||||
The OSIRIS Rhai module is now clean, generic, and ready for production use.
|
|
||||||
328
RUNNER.md
328
RUNNER.md
@@ -1,328 +0,0 @@
|
|||||||
# OSIRIS Runner - Standalone Binary ✅
|
|
||||||
|
|
||||||
The OSIRIS runner is a standalone binary that executes Rhai scripts with full OSIRIS object support.
|
|
||||||
|
|
||||||
## 🎉 Status: FULLY OPERATIONAL
|
|
||||||
|
|
||||||
```
|
|
||||||
✅ Binary created at src/bin/runner/
|
|
||||||
✅ Rhai engine with OSIRIS objects
|
|
||||||
✅ Note and Event support
|
|
||||||
✅ Automatic indexing
|
|
||||||
✅ Query functionality
|
|
||||||
✅ Test scripts working
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🚀 Quick Start
|
|
||||||
|
|
||||||
### Run a Script File
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo run --bin runner --features rhai-support -- runner1 --script-file scripts/test_note.rhai
|
|
||||||
```
|
|
||||||
|
|
||||||
### Run Inline Script
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo run --bin runner --features rhai-support -- runner1 \
|
|
||||||
--script 'let note = note("test").title("Hi"); print(note.get_title());'
|
|
||||||
```
|
|
||||||
|
|
||||||
## 📝 Example Output
|
|
||||||
|
|
||||||
```
|
|
||||||
🚀 OSIRIS Runner
|
|
||||||
Runner ID: test1
|
|
||||||
HeroDB: redis://localhost:6379 (DB 1)
|
|
||||||
|
|
||||||
📝 Executing script...
|
|
||||||
|
|
||||||
─────────────────────────────────────
|
|
||||||
=== OSIRIS Note Test ===
|
|
||||||
|
|
||||||
Creating note...
|
|
||||||
Note created: Test from OSIRIS Runner
|
|
||||||
Storing note...
|
|
||||||
✓ Note stored with ID: 46a064c7-9062-4858-a390-c11f0d5877a7
|
|
||||||
|
|
||||||
Retrieving note...
|
|
||||||
✓ Retrieved: Test from OSIRIS Runner
|
|
||||||
Content: This note was created using the OSIRIS standalone runner!
|
|
||||||
|
|
||||||
Querying notes by tag...
|
|
||||||
✓ Found notes:
|
|
||||||
- 2c54e1ec-bed9-41ea-851c-f7313abbd7cd
|
|
||||||
- 392f3f7f-47e7-444f-ba11-db38d74b12af
|
|
||||||
- 46a064c7-9062-4858-a390-c11f0d5877a7
|
|
||||||
=== Test Complete ===
|
|
||||||
─────────────────────────────────────
|
|
||||||
|
|
||||||
✅ Script completed successfully!
|
|
||||||
```
|
|
||||||
|
|
||||||
## 📖 Usage
|
|
||||||
|
|
||||||
### Command Line Options
|
|
||||||
|
|
||||||
```
|
|
||||||
OSIRIS Rhai Script Runner
|
|
||||||
|
|
||||||
Usage: runner [OPTIONS] <RUNNER_ID>
|
|
||||||
|
|
||||||
Arguments:
|
|
||||||
<RUNNER_ID> Runner ID
|
|
||||||
|
|
||||||
Options:
|
|
||||||
-r, --redis-url <REDIS_URL>
|
|
||||||
HeroDB URL [default: redis://localhost:6379]
|
|
||||||
-d, --db-id <DB_ID>
|
|
||||||
HeroDB database ID [default: 1]
|
|
||||||
-s, --script <SCRIPT>
|
|
||||||
Script to execute in single-job mode (optional)
|
|
||||||
-f, --script-file <SCRIPT_FILE>
|
|
||||||
Script file to execute
|
|
||||||
-h, --help
|
|
||||||
Print help
|
|
||||||
-V, --version
|
|
||||||
Print version
|
|
||||||
```
|
|
||||||
|
|
||||||
### Script Examples
|
|
||||||
|
|
||||||
#### Create and Store a Note
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
// scripts/test_note.rhai
|
|
||||||
let note = note("notes")
|
|
||||||
.title("My Note")
|
|
||||||
.content("This is the content")
|
|
||||||
.tag("project", "osiris")
|
|
||||||
.tag("priority", "high");
|
|
||||||
|
|
||||||
let id = put_note(note);
|
|
||||||
print(`Stored: ${id}`);
|
|
||||||
|
|
||||||
let retrieved = get_note("notes", id);
|
|
||||||
print(`Title: ${retrieved.get_title()}`);
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Create and Store an Event
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
// scripts/test_event.rhai
|
|
||||||
let event = event("calendar", "Team Meeting")
|
|
||||||
.description("Weekly sync")
|
|
||||||
.location("Conference Room A")
|
|
||||||
.category("meetings");
|
|
||||||
|
|
||||||
let id = put_event(event);
|
|
||||||
print(`Event stored: ${id}`);
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Query by Index
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
let ids = query("notes", "tags:tag", "project=osiris");
|
|
||||||
print("Found notes:");
|
|
||||||
for id in ids {
|
|
||||||
let note = get_note("notes", id);
|
|
||||||
print(` - ${note.get_title()}`);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🏗️ Architecture
|
|
||||||
|
|
||||||
```
|
|
||||||
┌─────────────────────────────────────┐
|
|
||||||
│ OSIRIS Runner Binary │
|
|
||||||
│ (osiris/src/bin/runner/) │
|
|
||||||
├─────────────────────────────────────┤
|
|
||||||
│ ├── main.rs │
|
|
||||||
│ │ - CLI argument parsing │
|
|
||||||
│ │ - Script loading │
|
|
||||||
│ │ - Execution orchestration │
|
|
||||||
│ └── engine.rs │
|
|
||||||
│ - Engine factory │
|
|
||||||
│ - OSIRIS integration │
|
|
||||||
└────────────┬────────────────────────┘
|
|
||||||
│
|
|
||||||
┌────────────▼────────────────────────┐
|
|
||||||
│ OSIRIS Rhai Support │
|
|
||||||
│ (osiris/src/rhai_support/) │
|
|
||||||
├─────────────────────────────────────┤
|
|
||||||
│ ├── note_rhai.rs │
|
|
||||||
│ │ - Note CustomType │
|
|
||||||
│ │ - Builder methods │
|
|
||||||
│ ├── event_rhai.rs │
|
|
||||||
│ │ - Event CustomType │
|
|
||||||
│ │ - Builder methods │
|
|
||||||
│ └── engine.rs │
|
|
||||||
│ - OsirisRhaiEngine │
|
|
||||||
│ - Async → Sync bridge │
|
|
||||||
└────────────┬────────────────────────┘
|
|
||||||
│
|
|
||||||
┌────────────▼────────────────────────┐
|
|
||||||
│ OSIRIS Core │
|
|
||||||
│ (osiris/src/) │
|
|
||||||
├─────────────────────────────────────┤
|
|
||||||
│ ├── objects/ │
|
|
||||||
│ │ - Note, Event │
|
|
||||||
│ │ - #[derive(DeriveObject)] │
|
|
||||||
│ ├── store/ │
|
|
||||||
│ │ - GenericStore │
|
|
||||||
│ │ - Automatic indexing │
|
|
||||||
│ └── index/ │
|
|
||||||
│ - FieldIndex │
|
|
||||||
└────────────┬────────────────────────┘
|
|
||||||
│
|
|
||||||
┌────────────▼────────────────────────┐
|
|
||||||
│ HeroDB │
|
|
||||||
│ (Redis-compatible storage) │
|
|
||||||
└─────────────────────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🎯 Features
|
|
||||||
|
|
||||||
### 1. **Fluent Builder Pattern**
|
|
||||||
```rhai
|
|
||||||
let note = note("ns")
|
|
||||||
.title("Title")
|
|
||||||
.content("Content")
|
|
||||||
.tag("key", "value");
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. **Automatic Indexing**
|
|
||||||
Fields marked with `#[index]` are automatically indexed:
|
|
||||||
```rust
|
|
||||||
#[derive(DeriveObject)]
|
|
||||||
pub struct Note {
|
|
||||||
pub base_data: BaseData,
|
|
||||||
|
|
||||||
#[index]
|
|
||||||
pub title: Option<String>, // Indexed!
|
|
||||||
|
|
||||||
pub content: Option<String>, // Not indexed
|
|
||||||
|
|
||||||
#[index]
|
|
||||||
pub tags: BTreeMap<String, String>, // Indexed!
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. **Type-Safe Operations**
|
|
||||||
- Compile-time type checking
|
|
||||||
- Runtime validation
|
|
||||||
- Clear error messages
|
|
||||||
|
|
||||||
### 4. **Query Support**
|
|
||||||
```rhai
|
|
||||||
// Query by any indexed field
|
|
||||||
let ids = query("namespace", "field_name", "value");
|
|
||||||
|
|
||||||
// Query by tag
|
|
||||||
let ids = query("notes", "tags:tag", "project=osiris");
|
|
||||||
|
|
||||||
// Query by title
|
|
||||||
let ids = query("notes", "title", "My Note");
|
|
||||||
```
|
|
||||||
|
|
||||||
## 📚 Available Functions
|
|
||||||
|
|
||||||
### Note API
|
|
||||||
|
|
||||||
| Function | Description |
|
|
||||||
|----------|-------------|
|
|
||||||
| `note(ns)` | Create new note |
|
|
||||||
| `.title(s)` | Set title (chainable) |
|
|
||||||
| `.content(s)` | Set content (chainable) |
|
|
||||||
| `.tag(k, v)` | Add tag (chainable) |
|
|
||||||
| `.mime(s)` | Set MIME type (chainable) |
|
|
||||||
| `put_note(note)` | Store note, returns ID |
|
|
||||||
| `get_note(ns, id)` | Retrieve note by ID |
|
|
||||||
| `.get_id()` | Get note ID |
|
|
||||||
| `.get_title()` | Get note title |
|
|
||||||
| `.get_content()` | Get note content |
|
|
||||||
| `.to_json()` | Serialize to JSON |
|
|
||||||
|
|
||||||
### Event API
|
|
||||||
|
|
||||||
| Function | Description |
|
|
||||||
|----------|-------------|
|
|
||||||
| `event(ns, title)` | Create new event |
|
|
||||||
| `.description(s)` | Set description (chainable) |
|
|
||||||
| `.location(s)` | Set location (chainable) |
|
|
||||||
| `.category(s)` | Set category (chainable) |
|
|
||||||
| `.all_day(b)` | Set all-day flag (chainable) |
|
|
||||||
| `put_event(event)` | Store event, returns ID |
|
|
||||||
| `get_event(ns, id)` | Retrieve event by ID |
|
|
||||||
| `.get_id()` | Get event ID |
|
|
||||||
| `.get_title()` | Get event title |
|
|
||||||
| `.to_json()` | Serialize to JSON |
|
|
||||||
|
|
||||||
### Query API
|
|
||||||
|
|
||||||
| Function | Description |
|
|
||||||
|----------|-------------|
|
|
||||||
| `query(ns, field, value)` | Query by indexed field, returns array of IDs |
|
|
||||||
|
|
||||||
## 🧪 Testing
|
|
||||||
|
|
||||||
### Test Scripts Included
|
|
||||||
|
|
||||||
1. **`scripts/test_note.rhai`** - Complete note workflow
|
|
||||||
2. **`scripts/test_event.rhai`** - Complete event workflow
|
|
||||||
|
|
||||||
### Run Tests
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Test notes
|
|
||||||
cargo run --bin runner --features rhai-support -- test1 --script-file scripts/test_note.rhai
|
|
||||||
|
|
||||||
# Test events
|
|
||||||
cargo run --bin runner --features rhai-support -- test1 --script-file scripts/test_event.rhai
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🔧 Building
|
|
||||||
|
|
||||||
### Development Build
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo build --bin runner --features rhai-support
|
|
||||||
```
|
|
||||||
|
|
||||||
### Release Build
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo build --bin runner --features rhai-support --release
|
|
||||||
```
|
|
||||||
|
|
||||||
### Run Without Cargo
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# After building
|
|
||||||
./target/release/runner runner1 --script-file scripts/test_note.rhai
|
|
||||||
```
|
|
||||||
|
|
||||||
## 📦 Integration with runner_rust
|
|
||||||
|
|
||||||
The OSIRIS runner can also be integrated into the runner_rust infrastructure for distributed task execution. See `runner_rust/src/engine/osiris.rs` for the integration.
|
|
||||||
|
|
||||||
## ✅ Verification
|
|
||||||
|
|
||||||
Run this to verify everything works:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo run --bin runner --features rhai-support -- test1 --script-file scripts/test_note.rhai
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected output:
|
|
||||||
```
|
|
||||||
✅ Script completed successfully!
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🎉 Success!
|
|
||||||
|
|
||||||
The OSIRIS runner is **fully operational** and ready for:
|
|
||||||
- ✅ Standalone script execution
|
|
||||||
- ✅ Integration with runner_rust
|
|
||||||
- ✅ Production use
|
|
||||||
- ✅ Custom object types (via derive macro)
|
|
||||||
@@ -1,176 +0,0 @@
|
|||||||
# Signatory-Based Access Control for OSIRIS Contexts
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
OSIRIS contexts now use **signatory-based access control** instead of owner-based permissions. This means that access to a context is granted only if all participants are signatories of the Rhai script.
|
|
||||||
|
|
||||||
## Key Changes
|
|
||||||
|
|
||||||
### 1. **Removed Owner Field**
|
|
||||||
- `OsirisContext` no longer has an `owner_id` field
|
|
||||||
- Replaced with `participants: Vec<String>` - a list of public keys
|
|
||||||
|
|
||||||
### 2. **Context Identification**
|
|
||||||
- Context ID is now a **sorted, comma-separated list** of participant public keys
|
|
||||||
- Example: `"pk1,pk2,pk3"` for a context with three participants
|
|
||||||
- Sorting ensures consistent IDs regardless of input order
|
|
||||||
|
|
||||||
### 3. **Access Control via SIGNATORIES**
|
|
||||||
- `get_context()` function checks the `SIGNATORIES` tag in the Rhai execution context
|
|
||||||
- All requested participants must be present in the SIGNATORIES list
|
|
||||||
- If any participant is not a signatory, access is denied
|
|
||||||
|
|
||||||
## API Changes
|
|
||||||
|
|
||||||
### Rhai Script Usage
|
|
||||||
|
|
||||||
**Old way (deprecated):**
|
|
||||||
```rhai
|
|
||||||
let ctx = osiris("context_name", "owner_id", "redis://localhost:6379", 1);
|
|
||||||
```
|
|
||||||
|
|
||||||
**New way:**
|
|
||||||
```rhai
|
|
||||||
// Single participant
|
|
||||||
let ctx = get_context(["pk1"]);
|
|
||||||
|
|
||||||
// Multiple participants (shared context)
|
|
||||||
let ctx = get_context(["pk1", "pk2", "pk3"]);
|
|
||||||
```
|
|
||||||
|
|
||||||
### Setting Up SIGNATORIES
|
|
||||||
|
|
||||||
When creating a Rhai engine, you must set up the SIGNATORIES tag:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
use rhai::{Engine, Dynamic, Map, Array};
|
|
||||||
|
|
||||||
let mut engine = create_osiris_engine("redis://localhost:6379", 1)?;
|
|
||||||
|
|
||||||
// Create context tags
|
|
||||||
let mut tag_map = Map::new();
|
|
||||||
|
|
||||||
// SIGNATORIES must be a Rhai array of strings
|
|
||||||
let signatories: Array = vec![
|
|
||||||
Dynamic::from("pk1".to_string()),
|
|
||||||
Dynamic::from("pk2".to_string()),
|
|
||||||
Dynamic::from("pk3".to_string()),
|
|
||||||
];
|
|
||||||
|
|
||||||
tag_map.insert("SIGNATORIES".into(), Dynamic::from(signatories));
|
|
||||||
tag_map.insert("DB_PATH".into(), "/path/to/db".to_string().into());
|
|
||||||
tag_map.insert("CONTEXT_ID".into(), "script_context".to_string().into());
|
|
||||||
|
|
||||||
engine.set_default_tag(Dynamic::from(tag_map));
|
|
||||||
```
|
|
||||||
|
|
||||||
## Access Control Flow
|
|
||||||
|
|
||||||
1. **Script Execution**: Rhai script calls `get_context(["pk1", "pk2"])`
|
|
||||||
2. **Extract SIGNATORIES**: Function reads `SIGNATORIES` from context tag
|
|
||||||
3. **Verify Participants**: Checks that all participants (`pk1`, `pk2`) are in SIGNATORIES
|
|
||||||
4. **Grant/Deny Access**:
|
|
||||||
- ✅ If all participants are signatories → create/return context
|
|
||||||
- ❌ If any participant is missing → return error
|
|
||||||
|
|
||||||
## Example: Shared Context
|
|
||||||
|
|
||||||
```rhai
|
|
||||||
// Three people want to collaborate
|
|
||||||
// All three must have signed the script
|
|
||||||
let shared_ctx = get_context(["alice_pk", "bob_pk", "charlie_pk"]);
|
|
||||||
|
|
||||||
// Now all three can access the same data
|
|
||||||
shared_ctx.save("notes", "note1", #{
|
|
||||||
title: "Meeting Notes",
|
|
||||||
content: "Discussed project timeline"
|
|
||||||
});
|
|
||||||
|
|
||||||
// Context ID will be: "alice_pk,bob_pk,charlie_pk" (sorted)
|
|
||||||
print(shared_ctx.context_id());
|
|
||||||
```
|
|
||||||
|
|
||||||
## Builder API Changes
|
|
||||||
|
|
||||||
### Old API (deprecated):
|
|
||||||
```rust
|
|
||||||
OsirisContext::builder()
|
|
||||||
.name("context_name")
|
|
||||||
.owner("owner_id") // Deprecated
|
|
||||||
.herodb_url("redis://localhost:6379")
|
|
||||||
.db_id(1)
|
|
||||||
.build()?
|
|
||||||
```
|
|
||||||
|
|
||||||
### New API:
|
|
||||||
```rust
|
|
||||||
OsirisContext::builder()
|
|
||||||
.participants(vec!["pk1".to_string(), "pk2".to_string()])
|
|
||||||
.herodb_url("redis://localhost:6379")
|
|
||||||
.db_id(1)
|
|
||||||
.build()?
|
|
||||||
```
|
|
||||||
|
|
||||||
## Member Management
|
|
||||||
|
|
||||||
All participants automatically get **admin privileges** in the context:
|
|
||||||
- `Privilege::Admin`
|
|
||||||
- `Privilege::Read`
|
|
||||||
- `Privilege::Write`
|
|
||||||
- `Privilege::ManageMembers`
|
|
||||||
|
|
||||||
Additional members can still be added with custom privileges:
|
|
||||||
```rhai
|
|
||||||
let ctx = get_context(["pk1", "pk2"]);
|
|
||||||
ctx.add_member("pk3", ["read", "write"]);
|
|
||||||
```
|
|
||||||
|
|
||||||
## Security Benefits
|
|
||||||
|
|
||||||
1. **Multi-signature Support**: Contexts can be shared between multiple parties
|
|
||||||
2. **Script-level Authorization**: Access control is enforced at script execution time
|
|
||||||
3. **No Hardcoded Owners**: Flexible participant model
|
|
||||||
4. **Transparent Access**: All participants have equal rights by default
|
|
||||||
|
|
||||||
## Testing
|
|
||||||
|
|
||||||
Tests verify:
|
|
||||||
- ✅ Valid signatories can create contexts
|
|
||||||
- ✅ Invalid signatories are denied access
|
|
||||||
- ✅ Context IDs are properly sorted
|
|
||||||
- ✅ Multiple participants work correctly
|
|
||||||
|
|
||||||
Run tests:
|
|
||||||
```bash
|
|
||||||
cargo test --lib --features rhai-support
|
|
||||||
```
|
|
||||||
|
|
||||||
## Migration Guide
|
|
||||||
|
|
||||||
If you have existing code using the old API:
|
|
||||||
|
|
||||||
1. **Replace `owner()` with `participants()`**:
|
|
||||||
```rust
|
|
||||||
// Old
|
|
||||||
.owner("user_id")
|
|
||||||
|
|
||||||
// New
|
|
||||||
.participants(vec!["user_id".to_string()])
|
|
||||||
```
|
|
||||||
|
|
||||||
2. **Update Rhai scripts**:
|
|
||||||
```rhai
|
|
||||||
// Old
|
|
||||||
let ctx = osiris("name", "owner", "url", 1);
|
|
||||||
|
|
||||||
// New
|
|
||||||
let ctx = get_context(["owner"]);
|
|
||||||
```
|
|
||||||
|
|
||||||
3. **Set up SIGNATORIES tag** in your engine configuration
|
|
||||||
|
|
||||||
4. **Update tests** to use the new API
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**All core functionality tested and verified!** 🎉
|
|
||||||
317
STRUCTURE.md
317
STRUCTURE.md
@@ -1,317 +0,0 @@
|
|||||||
# OSIRIS Code Structure
|
|
||||||
|
|
||||||
## 📁 Directory Organization
|
|
||||||
|
|
||||||
### Objects (`src/objects/`)
|
|
||||||
|
|
||||||
Each OSIRIS object has its own directory with:
|
|
||||||
- `mod.rs` - Object definition and core logic
|
|
||||||
- `rhai.rs` - Rhai integration (CustomType, builder API)
|
|
||||||
|
|
||||||
```
|
|
||||||
src/objects/
|
|
||||||
├── mod.rs # Module exports
|
|
||||||
├── note/
|
|
||||||
│ ├── mod.rs # Note object definition
|
|
||||||
│ └── rhai.rs # Note Rhai integration
|
|
||||||
└── event/
|
|
||||||
├── mod.rs # Event object definition
|
|
||||||
└── rhai.rs # Event Rhai integration
|
|
||||||
```
|
|
||||||
|
|
||||||
### Rhai Support (`src/rhai_support/`)
|
|
||||||
|
|
||||||
Core Rhai infrastructure (not object-specific):
|
|
||||||
- `instance.rs` - OsirisInstance type for multi-instance support
|
|
||||||
- `mod.rs` - Module exports and re-exports
|
|
||||||
|
|
||||||
```
|
|
||||||
src/rhai_support/
|
|
||||||
├── mod.rs # Re-exports from object modules
|
|
||||||
└── instance.rs # OsirisInstance implementation
|
|
||||||
```
|
|
||||||
|
|
||||||
### Runner (`src/bin/runner/`)
|
|
||||||
|
|
||||||
Standalone binary for running OSIRIS scripts:
|
|
||||||
- `main.rs` - CLI and script execution
|
|
||||||
- `engine.rs` - Engine factory with instance configuration
|
|
||||||
|
|
||||||
```
|
|
||||||
src/bin/runner/
|
|
||||||
├── main.rs # CLI interface
|
|
||||||
└── engine.rs # Engine configuration
|
|
||||||
```
|
|
||||||
|
|
||||||
### Core (`src/`)
|
|
||||||
|
|
||||||
Core OSIRIS functionality:
|
|
||||||
- `store/` - GenericStore, HeroDbClient, BaseData
|
|
||||||
- `index/` - Field indexing
|
|
||||||
- `error/` - Error types
|
|
||||||
- `config/` - Configuration
|
|
||||||
- `interfaces/` - Traits and interfaces
|
|
||||||
- `retrieve/` - Retrieval logic
|
|
||||||
|
|
||||||
## 🎯 Design Principles
|
|
||||||
|
|
||||||
### 1. **Co-location**
|
|
||||||
Object-specific code lives with the object:
|
|
||||||
```
|
|
||||||
objects/note/
|
|
||||||
├── mod.rs # Note struct, impl
|
|
||||||
└── rhai.rs # Note Rhai support
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. **Separation of Concerns**
|
|
||||||
- **Objects** - Domain models and business logic
|
|
||||||
- **Rhai Support** - Scripting integration
|
|
||||||
- **Store** - Persistence layer
|
|
||||||
- **Runner** - Execution environment
|
|
||||||
|
|
||||||
### 3. **Feature Gating**
|
|
||||||
Rhai support is optional via `rhai-support` feature:
|
|
||||||
```rust
|
|
||||||
#[cfg(feature = "rhai-support")]
|
|
||||||
pub mod rhai;
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. **Clean Exports**
|
|
||||||
Clear module boundaries with re-exports:
|
|
||||||
```rust
|
|
||||||
// src/objects/mod.rs
|
|
||||||
pub mod note;
|
|
||||||
pub use note::Note;
|
|
||||||
|
|
||||||
// src/rhai_support/mod.rs
|
|
||||||
pub use crate::objects::note::rhai::register_note_api;
|
|
||||||
```
|
|
||||||
|
|
||||||
## 📝 Adding a New Object
|
|
||||||
|
|
||||||
### 1. Create Object Directory
|
|
||||||
```bash
|
|
||||||
mkdir -p src/objects/task
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. Create Object Definition (`mod.rs`)
|
|
||||||
```rust
|
|
||||||
use crate::store::BaseData;
|
|
||||||
use serde::{Deserialize, Serialize};
|
|
||||||
|
|
||||||
#[cfg(feature = "rhai-support")]
|
|
||||||
pub mod rhai;
|
|
||||||
|
|
||||||
#[derive(Debug, Clone, Serialize, Deserialize, crate::DeriveObject)]
|
|
||||||
pub struct Task {
|
|
||||||
pub base_data: BaseData,
|
|
||||||
|
|
||||||
#[index]
|
|
||||||
pub title: String,
|
|
||||||
|
|
||||||
pub completed: bool,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl Task {
|
|
||||||
pub fn new(ns: String) -> Self {
|
|
||||||
Self {
|
|
||||||
base_data: BaseData::new(ns),
|
|
||||||
title: String::new(),
|
|
||||||
completed: false,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Create Rhai Integration (`rhai.rs`)
|
|
||||||
```rust
|
|
||||||
use crate::objects::Task;
|
|
||||||
use rhai::{CustomType, Engine, TypeBuilder};
|
|
||||||
|
|
||||||
impl CustomType for Task {
|
|
||||||
fn build(mut builder: TypeBuilder<Self>) {
|
|
||||||
builder
|
|
||||||
.with_name("Task")
|
|
||||||
.with_fn("new", |ns: String| Task::new(ns))
|
|
||||||
.with_fn("set_title", |task: &mut Task, title: String| {
|
|
||||||
task.title = title;
|
|
||||||
task.base_data.update_modified();
|
|
||||||
})
|
|
||||||
.with_fn("set_completed", |task: &mut Task, completed: bool| {
|
|
||||||
task.completed = completed;
|
|
||||||
task.base_data.update_modified();
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
pub fn register_task_api(engine: &mut Engine) {
|
|
||||||
engine.build_type::<Task>();
|
|
||||||
|
|
||||||
// Builder pattern
|
|
||||||
engine.register_fn("task", |ns: String| Task::new(ns));
|
|
||||||
engine.register_fn("title", |mut task: Task, title: String| {
|
|
||||||
task.title = title;
|
|
||||||
task.base_data.update_modified();
|
|
||||||
task
|
|
||||||
});
|
|
||||||
engine.register_fn("completed", |mut task: Task, completed: bool| {
|
|
||||||
task.completed = completed;
|
|
||||||
task.base_data.update_modified();
|
|
||||||
task
|
|
||||||
});
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. Update Module Exports
|
|
||||||
```rust
|
|
||||||
// src/objects/mod.rs
|
|
||||||
pub mod task;
|
|
||||||
pub use task::Task;
|
|
||||||
|
|
||||||
// src/rhai_support/mod.rs
|
|
||||||
pub use crate::objects::task::rhai::register_task_api;
|
|
||||||
```
|
|
||||||
|
|
||||||
### 5. Register in Engine
|
|
||||||
```rust
|
|
||||||
// src/bin/runner/engine.rs
|
|
||||||
use osiris::rhai_support::register_task_api;
|
|
||||||
|
|
||||||
register_task_api(&mut engine);
|
|
||||||
```
|
|
||||||
|
|
||||||
### 6. Use in Scripts
|
|
||||||
```rhai
|
|
||||||
let my_task = task("tasks")
|
|
||||||
.title("Complete OSIRIS integration")
|
|
||||||
.completed(true);
|
|
||||||
|
|
||||||
my_instance.put_task(my_task);
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🔍 File Responsibilities
|
|
||||||
|
|
||||||
### Object `mod.rs`
|
|
||||||
- Struct definition with `#[derive(DeriveObject)]`
|
|
||||||
- Index fields marked with `#[index]`
|
|
||||||
- Constructor methods (`new`, `with_id`)
|
|
||||||
- Business logic methods
|
|
||||||
- Builder pattern methods (optional)
|
|
||||||
|
|
||||||
### Object `rhai.rs`
|
|
||||||
- `CustomType` implementation
|
|
||||||
- `register_*_api` function
|
|
||||||
- Rhai-specific builder methods
|
|
||||||
- Type conversions for Rhai
|
|
||||||
|
|
||||||
### `rhai_support/instance.rs`
|
|
||||||
- `OsirisInstance` struct
|
|
||||||
- Multi-instance support
|
|
||||||
- CRUD operations per instance
|
|
||||||
- Async → Sync bridge
|
|
||||||
|
|
||||||
### `bin/runner/engine.rs`
|
|
||||||
- `OsirisConfig` for instance configuration
|
|
||||||
- `create_osiris_engine_with_config` function
|
|
||||||
- Engine setup and registration
|
|
||||||
|
|
||||||
### `bin/runner/main.rs`
|
|
||||||
- CLI argument parsing
|
|
||||||
- Instance configuration from CLI
|
|
||||||
- Script loading and execution
|
|
||||||
|
|
||||||
## 🎨 Benefits of This Structure
|
|
||||||
|
|
||||||
### 1. **Discoverability**
|
|
||||||
All code for an object is in one place:
|
|
||||||
```
|
|
||||||
objects/note/
|
|
||||||
├── mod.rs # "What is a Note?"
|
|
||||||
└── rhai.rs # "How do I use Note in Rhai?"
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. **Maintainability**
|
|
||||||
Changes to an object are localized:
|
|
||||||
- Add a field → Update `mod.rs` and `rhai.rs`
|
|
||||||
- No hunting through multiple directories
|
|
||||||
|
|
||||||
### 3. **Scalability**
|
|
||||||
Easy to add new objects:
|
|
||||||
- Create directory
|
|
||||||
- Add two files
|
|
||||||
- Update exports
|
|
||||||
- Done!
|
|
||||||
|
|
||||||
### 4. **Testability**
|
|
||||||
Each object can have its own tests:
|
|
||||||
```
|
|
||||||
objects/note/
|
|
||||||
├── mod.rs
|
|
||||||
├── rhai.rs
|
|
||||||
└── tests.rs
|
|
||||||
```
|
|
||||||
|
|
||||||
### 5. **Clear Dependencies**
|
|
||||||
```
|
|
||||||
Objects (domain) → Independent
|
|
||||||
↓
|
|
||||||
Rhai Support → Depends on Objects
|
|
||||||
↓
|
|
||||||
Runner → Depends on Rhai Support
|
|
||||||
```
|
|
||||||
|
|
||||||
## 📊 Module Graph
|
|
||||||
|
|
||||||
```
|
|
||||||
osiris (lib)
|
|
||||||
├── objects/
|
|
||||||
│ ├── note/
|
|
||||||
│ │ ├── mod.rs (Note struct)
|
|
||||||
│ │ └── rhai.rs (Note Rhai)
|
|
||||||
│ └── event/
|
|
||||||
│ ├── mod.rs (Event struct)
|
|
||||||
│ └── rhai.rs (Event Rhai)
|
|
||||||
├── rhai_support/
|
|
||||||
│ ├── instance.rs (OsirisInstance)
|
|
||||||
│ └── mod.rs (re-exports)
|
|
||||||
├── store/ (GenericStore, BaseData)
|
|
||||||
├── index/ (FieldIndex)
|
|
||||||
└── error/ (Error types)
|
|
||||||
|
|
||||||
runner (bin)
|
|
||||||
├── main.rs (CLI)
|
|
||||||
└── engine.rs (Engine factory)
|
|
||||||
```
|
|
||||||
|
|
||||||
## ✅ Summary
|
|
||||||
|
|
||||||
**Old Structure:**
|
|
||||||
```
|
|
||||||
src/rhai_support/
|
|
||||||
├── note_rhai.rs
|
|
||||||
├── event_rhai.rs
|
|
||||||
├── engine.rs
|
|
||||||
└── instance.rs
|
|
||||||
```
|
|
||||||
|
|
||||||
**New Structure:**
|
|
||||||
```
|
|
||||||
src/objects/
|
|
||||||
├── note/
|
|
||||||
│ ├── mod.rs
|
|
||||||
│ └── rhai.rs
|
|
||||||
└── event/
|
|
||||||
├── mod.rs
|
|
||||||
└── rhai.rs
|
|
||||||
|
|
||||||
src/rhai_support/
|
|
||||||
├── instance.rs
|
|
||||||
└── mod.rs (re-exports only)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Benefits:**
|
|
||||||
- ✅ Better organization
|
|
||||||
- ✅ Co-located code
|
|
||||||
- ✅ Easier to find things
|
|
||||||
- ✅ Cleaner separation
|
|
||||||
- ✅ Scalable structure
|
|
||||||
@@ -1,154 +0,0 @@
|
|||||||
# OSIRIS Tests Complete! ✅
|
|
||||||
|
|
||||||
## Test Coverage Summary
|
|
||||||
|
|
||||||
Successfully added comprehensive tests for all OSIRIS rhai module files:
|
|
||||||
|
|
||||||
### **Builder Tests** (`src/rhai/builder.rs`)
|
|
||||||
- ✅ Basic builder creation
|
|
||||||
- ✅ Custom owner configuration
|
|
||||||
- ✅ Registry integration
|
|
||||||
- ✅ Missing required fields validation
|
|
||||||
- ✅ Fluent API chaining
|
|
||||||
|
|
||||||
**7 tests total**
|
|
||||||
|
|
||||||
### **Instance Tests** (`src/rhai/instance.rs`)
|
|
||||||
- ✅ Context creation
|
|
||||||
- ✅ Member management (add/remove/list)
|
|
||||||
- ✅ Privilege checking
|
|
||||||
- ✅ Save and get operations
|
|
||||||
- ✅ Delete operations
|
|
||||||
- ✅ Context manager (single and multiple contexts)
|
|
||||||
- ✅ Privilege enum behavior
|
|
||||||
|
|
||||||
**11 tests total**
|
|
||||||
|
|
||||||
### **Type Registry Tests** (`src/store/type_registry.rs`)
|
|
||||||
- ✅ Registry creation
|
|
||||||
- ✅ Type registration
|
|
||||||
- ✅ Multiple type registration
|
|
||||||
- ✅ Save with registry
|
|
||||||
- ✅ Unregistered collection handling
|
|
||||||
- ✅ List collections
|
|
||||||
- ✅ Has type checking
|
|
||||||
|
|
||||||
**7 tests total**
|
|
||||||
|
|
||||||
### **Engine Tests** (`src/rhai/engine.rs`)
|
|
||||||
- ✅ Engine config creation
|
|
||||||
- ✅ Add context to config
|
|
||||||
- ✅ Single context config
|
|
||||||
- ✅ Create OSIRIS engine
|
|
||||||
- ✅ Create engine with config
|
|
||||||
- ✅ Create engine with manager
|
|
||||||
- ✅ Dynamic context creation
|
|
||||||
- ✅ Context operations
|
|
||||||
|
|
||||||
**8 tests total**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Test Results
|
|
||||||
|
|
||||||
```
|
|
||||||
running 38 tests
|
|
||||||
test result: ok. 34 passed; 0 failed; 4 ignored; 0 measured; 0 filtered out
|
|
||||||
```
|
|
||||||
|
|
||||||
**Status:** ✅ All tests passing!
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## What the Tests Verify
|
|
||||||
|
|
||||||
### **Builder Pattern**
|
|
||||||
- Required fields are enforced
|
|
||||||
- Optional fields work correctly
|
|
||||||
- Fluent API chains properly
|
|
||||||
- Builder creates valid contexts
|
|
||||||
|
|
||||||
### **Context Functionality**
|
|
||||||
- Contexts are created with correct owner
|
|
||||||
- Members can be added/removed
|
|
||||||
- Privileges are checked correctly
|
|
||||||
- CRUD operations work as expected
|
|
||||||
- Multiple contexts can coexist
|
|
||||||
|
|
||||||
### **Type Registry**
|
|
||||||
- Types can be registered for collections
|
|
||||||
- Multiple collections supported
|
|
||||||
- Unregistered collections are detected
|
|
||||||
- Registry tracks all registered types
|
|
||||||
|
|
||||||
### **Engine Creation**
|
|
||||||
- Engines can be created with different configurations
|
|
||||||
- Single context mode works
|
|
||||||
- Multi-context mode works
|
|
||||||
- Context manager mode works
|
|
||||||
- Dynamic context creation works
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Running the Tests
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Run all OSIRIS tests
|
|
||||||
cargo test --lib --features rhai-support
|
|
||||||
|
|
||||||
# Run specific module tests
|
|
||||||
cargo test --lib --features rhai-support rhai::builder
|
|
||||||
cargo test --lib --features rhai-support rhai::instance
|
|
||||||
cargo test --lib --features rhai-support rhai::engine
|
|
||||||
cargo test --lib --features rhai-support store::type_registry
|
|
||||||
|
|
||||||
# Run with output
|
|
||||||
cargo test --lib --features rhai-support -- --nocapture
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Test Design Notes
|
|
||||||
|
|
||||||
### **No Redis Required**
|
|
||||||
Tests are designed to work without a running Redis instance:
|
|
||||||
- Builder tests only verify construction
|
|
||||||
- Instance tests verify in-memory state
|
|
||||||
- Registry tests verify type registration
|
|
||||||
- Engine tests verify configuration
|
|
||||||
|
|
||||||
### **Isolated Tests**
|
|
||||||
Each test is independent and doesn't affect others:
|
|
||||||
- No shared state between tests
|
|
||||||
- Each test creates its own contexts
|
|
||||||
- Clean setup and teardown
|
|
||||||
|
|
||||||
### **Comprehensive Coverage**
|
|
||||||
Tests cover:
|
|
||||||
- ✅ Happy paths (normal usage)
|
|
||||||
- ✅ Error paths (missing fields, invalid data)
|
|
||||||
- ✅ Edge cases (multiple contexts, unregistered types)
|
|
||||||
- ✅ Integration (builder → context → operations)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Future Test Improvements
|
|
||||||
|
|
||||||
1. **Integration Tests with Redis**
|
|
||||||
- Add optional integration tests that require Redis
|
|
||||||
- Test actual save/load/delete operations
|
|
||||||
- Test concurrent access
|
|
||||||
|
|
||||||
2. **Rhai Script Tests**
|
|
||||||
- Test actual Rhai scripts using the engine
|
|
||||||
- Test error handling in scripts
|
|
||||||
- Test complex workflows
|
|
||||||
|
|
||||||
3. **Performance Tests**
|
|
||||||
- Benchmark context creation
|
|
||||||
- Benchmark CRUD operations
|
|
||||||
- Test with large datasets
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
**All core functionality is now tested and verified!** 🎉
|
|
||||||
14
client/Cargo.toml
Normal file
14
client/Cargo.toml
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
[package]
|
||||||
|
name = "osiris-client"
|
||||||
|
version = "0.1.0"
|
||||||
|
edition = "2021"
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
serde = { version = "1.0", features = ["derive"] }
|
||||||
|
serde_json = "1.0"
|
||||||
|
reqwest = { version = "0.11", features = ["json"] }
|
||||||
|
anyhow = "1.0"
|
||||||
|
thiserror = "1.0"
|
||||||
|
|
||||||
|
[dev-dependencies]
|
||||||
|
tokio = { version = "1.23", features = ["full", "macros"] }
|
||||||
37
client/src/communication.rs
Normal file
37
client/src/communication.rs
Normal file
@@ -0,0 +1,37 @@
|
|||||||
|
//! Communication query methods (email verification, etc.)
|
||||||
|
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use crate::{OsirisClient, OsirisClientError};
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
pub struct Verification {
|
||||||
|
pub id: String,
|
||||||
|
pub email: String,
|
||||||
|
pub code: String,
|
||||||
|
pub transport: String,
|
||||||
|
pub status: VerificationStatus,
|
||||||
|
pub created_at: i64,
|
||||||
|
pub expires_at: i64,
|
||||||
|
pub verified_at: Option<i64>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "lowercase")]
|
||||||
|
pub enum VerificationStatus {
|
||||||
|
Pending,
|
||||||
|
Verified,
|
||||||
|
Expired,
|
||||||
|
Failed,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl OsirisClient {
|
||||||
|
/// Get verification by ID
|
||||||
|
pub async fn get_verification(&self, verification_id: &str) -> Result<Verification, OsirisClientError> {
|
||||||
|
self.get("verification", verification_id).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Get verification by email
|
||||||
|
pub async fn get_verification_by_email(&self, email: &str) -> Result<Vec<Verification>, OsirisClientError> {
|
||||||
|
self.query("verification", &format!("email={}", email)).await
|
||||||
|
}
|
||||||
|
}
|
||||||
38
client/src/kyc.rs
Normal file
38
client/src/kyc.rs
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
//! KYC query methods
|
||||||
|
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use crate::{OsirisClient, OsirisClientError};
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
pub struct KycSession {
|
||||||
|
pub id: String,
|
||||||
|
pub resident_id: String,
|
||||||
|
pub status: KycSessionStatus,
|
||||||
|
pub kyc_url: Option<String>,
|
||||||
|
pub created_at: i64,
|
||||||
|
pub updated_at: i64,
|
||||||
|
pub expires_at: i64,
|
||||||
|
pub verified_at: Option<i64>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "lowercase")]
|
||||||
|
pub enum KycSessionStatus {
|
||||||
|
Pending,
|
||||||
|
InProgress,
|
||||||
|
Completed,
|
||||||
|
Failed,
|
||||||
|
Expired,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl OsirisClient {
|
||||||
|
/// Get KYC session by ID
|
||||||
|
pub async fn get_kyc_session(&self, session_id: &str) -> Result<KycSession, OsirisClientError> {
|
||||||
|
self.get("kyc_session", session_id).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// List all KYC sessions for a resident
|
||||||
|
pub async fn list_kyc_sessions_by_resident(&self, resident_id: &str) -> Result<Vec<KycSession>, OsirisClientError> {
|
||||||
|
self.query("kyc_session", &format!("resident_id={}", resident_id)).await
|
||||||
|
}
|
||||||
|
}
|
||||||
119
client/src/lib.rs
Normal file
119
client/src/lib.rs
Normal file
@@ -0,0 +1,119 @@
|
|||||||
|
//! Osiris Client - Query API for Osiris data structures
|
||||||
|
//!
|
||||||
|
//! This client provides read-only access to Osiris data via REST API.
|
||||||
|
//! Follows CQRS pattern: queries go through this client, commands go through Rhai scripts.
|
||||||
|
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use thiserror::Error;
|
||||||
|
|
||||||
|
pub mod kyc;
|
||||||
|
pub mod payment;
|
||||||
|
pub mod communication;
|
||||||
|
|
||||||
|
pub use kyc::*;
|
||||||
|
pub use payment::*;
|
||||||
|
pub use communication::*;
|
||||||
|
|
||||||
|
#[derive(Debug, Error)]
|
||||||
|
pub enum OsirisClientError {
|
||||||
|
#[error("HTTP request failed: {0}")]
|
||||||
|
RequestFailed(#[from] reqwest::Error),
|
||||||
|
|
||||||
|
#[error("Resource not found: {0}")]
|
||||||
|
NotFound(String),
|
||||||
|
|
||||||
|
#[error("Deserialization failed: {0}")]
|
||||||
|
DeserializationFailed(String),
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Osiris client for querying data
|
||||||
|
#[derive(Clone, Debug)]
|
||||||
|
pub struct OsirisClient {
|
||||||
|
base_url: String,
|
||||||
|
client: reqwest::Client,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl OsirisClient {
|
||||||
|
/// Create a new Osiris client
|
||||||
|
pub fn new(base_url: impl Into<String>) -> Self {
|
||||||
|
Self {
|
||||||
|
base_url: base_url.into(),
|
||||||
|
client: reqwest::Client::new(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Generic GET request for any struct by ID
|
||||||
|
pub async fn get<T>(&self, struct_name: &str, id: &str) -> Result<T, OsirisClientError>
|
||||||
|
where
|
||||||
|
T: for<'de> Deserialize<'de>,
|
||||||
|
{
|
||||||
|
let url = format!("{}/api/{}/{}", self.base_url, struct_name, id);
|
||||||
|
|
||||||
|
let response = self.client
|
||||||
|
.get(&url)
|
||||||
|
.send()
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
if response.status() == 404 {
|
||||||
|
return Err(OsirisClientError::NotFound(format!("{}/{}", struct_name, id)));
|
||||||
|
}
|
||||||
|
|
||||||
|
let data = response
|
||||||
|
.json::<T>()
|
||||||
|
.await
|
||||||
|
.map_err(|e| OsirisClientError::DeserializationFailed(e.to_string()))?;
|
||||||
|
|
||||||
|
Ok(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Generic LIST request for all instances of a struct
|
||||||
|
pub async fn list<T>(&self, struct_name: &str) -> Result<Vec<T>, OsirisClientError>
|
||||||
|
where
|
||||||
|
T: for<'de> Deserialize<'de>,
|
||||||
|
{
|
||||||
|
let url = format!("{}/api/{}", self.base_url, struct_name);
|
||||||
|
|
||||||
|
let response = self.client
|
||||||
|
.get(&url)
|
||||||
|
.send()
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
let data = response
|
||||||
|
.json::<Vec<T>>()
|
||||||
|
.await
|
||||||
|
.map_err(|e| OsirisClientError::DeserializationFailed(e.to_string()))?;
|
||||||
|
|
||||||
|
Ok(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Generic QUERY request with filters
|
||||||
|
pub async fn query<T>(&self, struct_name: &str, query: &str) -> Result<Vec<T>, OsirisClientError>
|
||||||
|
where
|
||||||
|
T: for<'de> Deserialize<'de>,
|
||||||
|
{
|
||||||
|
let url = format!("{}/api/{}?{}", self.base_url, struct_name, query);
|
||||||
|
|
||||||
|
let response = self.client
|
||||||
|
.get(&url)
|
||||||
|
.send()
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
let data = response
|
||||||
|
.json::<Vec<T>>()
|
||||||
|
.await
|
||||||
|
.map_err(|e| OsirisClientError::DeserializationFailed(e.to_string()))?;
|
||||||
|
|
||||||
|
Ok(data)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn test_client_creation() {
|
||||||
|
let client = OsirisClient::new("http://localhost:8080");
|
||||||
|
assert_eq!(client.base_url, "http://localhost:8080");
|
||||||
|
}
|
||||||
|
}
|
||||||
39
client/src/payment.rs
Normal file
39
client/src/payment.rs
Normal file
@@ -0,0 +1,39 @@
|
|||||||
|
//! Payment query methods
|
||||||
|
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use crate::{OsirisClient, OsirisClientError};
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
pub struct Payment {
|
||||||
|
pub id: String,
|
||||||
|
pub amount: f64,
|
||||||
|
pub currency: String,
|
||||||
|
pub status: PaymentStatus,
|
||||||
|
pub description: String,
|
||||||
|
pub payment_url: Option<String>,
|
||||||
|
pub created_at: i64,
|
||||||
|
pub updated_at: i64,
|
||||||
|
pub completed_at: Option<i64>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||||
|
#[serde(rename_all = "lowercase")]
|
||||||
|
pub enum PaymentStatus {
|
||||||
|
Pending,
|
||||||
|
Processing,
|
||||||
|
Completed,
|
||||||
|
Failed,
|
||||||
|
Cancelled,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl OsirisClient {
|
||||||
|
/// Get payment by ID
|
||||||
|
pub async fn get_payment(&self, payment_id: &str) -> Result<Payment, OsirisClientError> {
|
||||||
|
self.get("payment", payment_id).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// List all payments
|
||||||
|
pub async fn list_payments(&self) -> Result<Vec<Payment>, OsirisClientError> {
|
||||||
|
self.list("payment").await
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,8 +1,8 @@
|
|||||||
# OSIRIS Architecture - Trait-Based Generic Objects
|
# OSIRIS Architecture
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
OSIRIS has been refactored to use a trait-based architecture similar to heromodels, allowing any object implementing the `Object` trait to be stored and indexed automatically based on field attributes.
|
OSIRIS uses a trait-based architecture for storing and retrieving typed objects with automatic indexing, Rhai scripting support, and signatory-based access control.
|
||||||
|
|
||||||
## Core Concepts
|
## Core Concepts
|
||||||
|
|
||||||
@@ -311,100 +311,59 @@ When an object is deleted:
|
|||||||
3. **Remove** from all indexes
|
3. **Remove** from all indexes
|
||||||
4. **Delete** the object
|
4. **Delete** the object
|
||||||
|
|
||||||
## Comparison with heromodels
|
## Rhai Integration
|
||||||
|
|
||||||
| Feature | heromodels | OSIRIS |
|
OSIRIS provides full Rhai scripting support through the `rhai` module:
|
||||||
|---------|-----------|--------|
|
|
||||||
| Base struct | `BaseModelData` | `BaseData` |
|
|
||||||
| Core trait | `Model` | `Object` |
|
|
||||||
| ID type | `u32` (auto-increment) | `String` (UUID) |
|
|
||||||
| Timestamps | `i64` (Unix) | `OffsetDateTime` |
|
|
||||||
| Index macro | `#[index]` (derive) | Manual `index_keys()` |
|
|
||||||
| Storage | OurDB/Postgres | HeroDB (Redis) |
|
|
||||||
| Serialization | CBOR/JSON | JSON |
|
|
||||||
|
|
||||||
## Future Enhancements
|
### OsirisContext
|
||||||
|
|
||||||
### 1. Derive Macro for #[index]
|
Multi-tenant context with signatory-based access control:
|
||||||
|
|
||||||
Create a proc macro to automatically generate `index_keys()` from field attributes:
|
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
#[derive(Object)]
|
pub struct OsirisContext {
|
||||||
pub struct Note {
|
context_id: String,
|
||||||
pub base_data: BaseData,
|
participants: Vec<String>, // Public keys
|
||||||
|
members: HashMap<String, Vec<Privilege>>,
|
||||||
#[index]
|
store: Arc<GenericStore>,
|
||||||
pub title: Option<String>,
|
|
||||||
|
|
||||||
pub content: Option<String>,
|
|
||||||
|
|
||||||
#[index]
|
|
||||||
pub tags: BTreeMap<String, String>,
|
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
### 2. Query Builder
|
**Key Features:**
|
||||||
|
- **Signatory-based access**: All participants must be signatories
|
||||||
|
- **Member management**: Add/remove members with privileges
|
||||||
|
- **Generic CRUD**: `save()`, `get()`, `delete()`, `list()`, `query()`
|
||||||
|
|
||||||
Type-safe query builder for indexed fields:
|
### Rhai API
|
||||||
|
|
||||||
|
Each object type provides Rhai bindings in its `rhai.rs` file:
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
let results = store
|
#[export_module]
|
||||||
.query::<Note>("notes")
|
mod rhai_note_module {
|
||||||
.filter("tag:topic", "rust")
|
#[rhai_fn(name = "note", return_raw)]
|
||||||
.filter("tag:priority", "high")
|
pub fn new_note(ns: String) -> Result<Note, Box<EvalAltResult>> {
|
||||||
.limit(10)
|
Ok(Note::new(ns))
|
||||||
.execute()
|
|
||||||
.await?;
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Relations
|
|
||||||
|
|
||||||
Support for typed relations between objects:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
pub struct Note {
|
|
||||||
pub base_data: BaseData,
|
|
||||||
pub title: String,
|
|
||||||
|
|
||||||
#[relation(target = "Note", label = "references")]
|
|
||||||
pub references: Vec<String>,
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. Validation
|
|
||||||
|
|
||||||
Trait-based validation:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
pub trait Validate {
|
|
||||||
fn validate(&self) -> Result<()>;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Validate for Note {
|
#[rhai_fn(name = "title", return_raw)]
|
||||||
fn validate(&self) -> Result<()> {
|
pub fn set_title(note: Note, title: String) -> Result<Note, Box<EvalAltResult>> {
|
||||||
if self.title.is_none() {
|
Ok(note.title(title))
|
||||||
return Err(Error::InvalidInput("Title required".into()));
|
|
||||||
}
|
|
||||||
Ok(())
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
## Migration from Old API
|
### Engine Configuration
|
||||||
|
|
||||||
The old `OsirisObject` API is still available for backwards compatibility:
|
The runner binary supports multiple configurations:
|
||||||
|
|
||||||
```rust
|
```bash
|
||||||
// Old API (still works)
|
# Single instance
|
||||||
use osiris::store::OsirisObject;
|
cargo run --bin runner -- runner1 --redis-url redis://localhost:6379 --db-id 1
|
||||||
let obj = OsirisObject::new("notes".to_string(), Some("text".to_string()));
|
|
||||||
|
|
||||||
// New API (recommended)
|
# Multiple predefined instances
|
||||||
use osiris::objects::Note;
|
cargo run --bin runner -- runner1 \
|
||||||
let note = Note::new("notes".to_string())
|
--instance freezone:redis://localhost:6379:1 \
|
||||||
.set_title("Title")
|
--instance my:redis://localhost:6379:2
|
||||||
.set_content("text");
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Benefits of Trait-Based Architecture
|
## Benefits of Trait-Based Architecture
|
||||||
@@ -418,9 +377,12 @@ let note = Note::new("notes".to_string())
|
|||||||
|
|
||||||
## Summary
|
## Summary
|
||||||
|
|
||||||
The trait-based architecture makes OSIRIS:
|
OSIRIS provides:
|
||||||
- **More flexible**: Any type can be stored by implementing `Object`
|
- **Type-safe storage**: Any type implementing `Object` can be stored
|
||||||
- **More consistent**: Follows heromodels patterns
|
- **Automatic indexing**: Fields marked with `#[index]` are automatically indexed
|
||||||
- **More powerful**: Automatic indexing based on object structure
|
- **Rhai scripting**: Full scripting support with builder patterns
|
||||||
- **More maintainable**: Clear separation of concerns
|
- **Multi-tenant contexts**: Signatory-based access control
|
||||||
- **More extensible**: Easy to add new object types and features
|
- **HeroDB backend**: Redis-compatible storage with encryption
|
||||||
|
- **Extensibility**: Easy to add new object types and features
|
||||||
|
|
||||||
|
See [CREATING_NEW_OBJECTS.md](CREATING_NEW_OBJECTS.md) for a guide on creating custom objects.
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# Guide: Creating New OSIRIS Objects
|
# Creating New OSIRIS Objects
|
||||||
|
|
||||||
This guide explains how to properly create new object types in OSIRIS that integrate with the Rhai scripting engine and context storage.
|
This guide explains how to create new object types in OSIRIS with automatic indexing and Rhai scripting support.
|
||||||
|
|
||||||
## Step-by-Step Process
|
## Step-by-Step Process
|
||||||
|
|
||||||
|
|||||||
@@ -1,195 +0,0 @@
|
|||||||
# OSIRIS Derive Macro
|
|
||||||
|
|
||||||
The `#[derive(DeriveObject)]` macro automatically implements the `Object` trait for your structs, generating index keys based on fields marked with `#[index]`.
|
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
```rust
|
|
||||||
use osiris::{BaseData, DeriveObject};
|
|
||||||
use serde::{Deserialize, Serialize};
|
|
||||||
use std::collections::BTreeMap;
|
|
||||||
|
|
||||||
#[derive(Debug, Clone, Serialize, Deserialize, DeriveObject)]
|
|
||||||
pub struct Note {
|
|
||||||
pub base_data: BaseData,
|
|
||||||
|
|
||||||
#[index]
|
|
||||||
pub title: Option<String>,
|
|
||||||
|
|
||||||
pub content: Option<String>,
|
|
||||||
|
|
||||||
#[index]
|
|
||||||
pub tags: BTreeMap<String, String>,
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## What Gets Generated
|
|
||||||
|
|
||||||
The derive macro automatically implements:
|
|
||||||
|
|
||||||
1. **`object_type()`** - Returns the struct name as a string
|
|
||||||
2. **`base_data()`** - Returns a reference to `base_data`
|
|
||||||
3. **`base_data_mut()`** - Returns a mutable reference to `base_data`
|
|
||||||
4. **`index_keys()`** - Generates index keys for all `#[index]` fields
|
|
||||||
5. **`indexed_fields()`** - Returns a list of indexed field names
|
|
||||||
|
|
||||||
## Supported Field Types
|
|
||||||
|
|
||||||
### Option<T>
|
|
||||||
```rust
|
|
||||||
#[index]
|
|
||||||
pub title: Option<String>,
|
|
||||||
```
|
|
||||||
Generates: `IndexKey { name: "title", value: <string_value> }` (only if Some)
|
|
||||||
|
|
||||||
### BTreeMap<String, String>
|
|
||||||
```rust
|
|
||||||
#[index]
|
|
||||||
pub tags: BTreeMap<String, String>,
|
|
||||||
```
|
|
||||||
Generates: `IndexKey { name: "tags:tag", value: "key=value" }` for each entry
|
|
||||||
|
|
||||||
### Vec<T>
|
|
||||||
```rust
|
|
||||||
#[index]
|
|
||||||
pub items: Vec<String>,
|
|
||||||
```
|
|
||||||
Generates: `IndexKey { name: "items:item", value: "0:value" }` for each item
|
|
||||||
|
|
||||||
### OffsetDateTime
|
|
||||||
```rust
|
|
||||||
#[index]
|
|
||||||
pub start_time: OffsetDateTime,
|
|
||||||
```
|
|
||||||
Generates: `IndexKey { name: "start_time", value: "2025-10-20" }` (date only)
|
|
||||||
|
|
||||||
### Enums and Other Types
|
|
||||||
```rust
|
|
||||||
#[index]
|
|
||||||
pub status: EventStatus,
|
|
||||||
```
|
|
||||||
Generates: `IndexKey { name: "status", value: "Debug(status)" }` (using Debug format)
|
|
||||||
|
|
||||||
## Complete Example
|
|
||||||
|
|
||||||
```rust
|
|
||||||
use osiris::{BaseData, DeriveObject};
|
|
||||||
use serde::{Deserialize, Serialize};
|
|
||||||
use time::OffsetDateTime;
|
|
||||||
|
|
||||||
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
|
|
||||||
pub enum EventStatus {
|
|
||||||
Draft,
|
|
||||||
Published,
|
|
||||||
Cancelled,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Debug, Clone, Serialize, Deserialize, DeriveObject)]
|
|
||||||
pub struct Event {
|
|
||||||
pub base_data: BaseData,
|
|
||||||
|
|
||||||
#[index]
|
|
||||||
pub title: String,
|
|
||||||
|
|
||||||
pub description: Option<String>,
|
|
||||||
|
|
||||||
#[index]
|
|
||||||
#[serde(with = "time::serde::timestamp")]
|
|
||||||
pub start_time: OffsetDateTime,
|
|
||||||
|
|
||||||
#[index]
|
|
||||||
pub location: Option<String>,
|
|
||||||
|
|
||||||
#[index]
|
|
||||||
pub status: EventStatus,
|
|
||||||
|
|
||||||
pub all_day: bool,
|
|
||||||
|
|
||||||
#[index]
|
|
||||||
pub category: Option<String>,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl Event {
|
|
||||||
pub fn new(ns: String, title: impl ToString) -> Self {
|
|
||||||
let now = OffsetDateTime::now_utc();
|
|
||||||
Self {
|
|
||||||
base_data: BaseData::new(ns),
|
|
||||||
title: title.to_string(),
|
|
||||||
description: None,
|
|
||||||
start_time: now,
|
|
||||||
location: None,
|
|
||||||
status: EventStatus::Draft,
|
|
||||||
all_day: false,
|
|
||||||
category: None,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Generated Index Keys
|
|
||||||
|
|
||||||
For the Event example above with:
|
|
||||||
- `title = "Team Meeting"`
|
|
||||||
- `start_time = 2025-10-20T10:00:00Z`
|
|
||||||
- `location = Some("Room 101")`
|
|
||||||
- `status = EventStatus::Published`
|
|
||||||
- `category = Some("work")`
|
|
||||||
|
|
||||||
The generated index keys would be:
|
|
||||||
```rust
|
|
||||||
vec![
|
|
||||||
IndexKey { name: "mime", value: "application/json" }, // from base_data
|
|
||||||
IndexKey { name: "title", value: "Team Meeting" },
|
|
||||||
IndexKey { name: "start_time", value: "2025-10-20" },
|
|
||||||
IndexKey { name: "location", value: "Room 101" },
|
|
||||||
IndexKey { name: "status", value: "Published" },
|
|
||||||
IndexKey { name: "category", value: "work" },
|
|
||||||
]
|
|
||||||
```
|
|
||||||
|
|
||||||
## HeroDB Storage
|
|
||||||
|
|
||||||
These index keys are stored in HeroDB as:
|
|
||||||
```
|
|
||||||
idx:events:title:Team Meeting → {event_id}
|
|
||||||
idx:events:start_time:2025-10-20 → {event_id}
|
|
||||||
idx:events:location:Room 101 → {event_id}
|
|
||||||
idx:events:status:Published → {event_id}
|
|
||||||
idx:events:category:work → {event_id}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Querying by Index
|
|
||||||
|
|
||||||
```rust
|
|
||||||
use osiris::store::GenericStore;
|
|
||||||
|
|
||||||
let store = GenericStore::new(client);
|
|
||||||
|
|
||||||
// Get all events on a specific date
|
|
||||||
let ids = store.get_ids_by_index("events", "start_time", "2025-10-20").await?;
|
|
||||||
|
|
||||||
// Get all published events
|
|
||||||
let ids = store.get_ids_by_index("events", "status", "Published").await?;
|
|
||||||
|
|
||||||
// Get all events in a category
|
|
||||||
let ids = store.get_ids_by_index("events", "category", "work").await?;
|
|
||||||
```
|
|
||||||
|
|
||||||
## Requirements
|
|
||||||
|
|
||||||
1. **Must have `base_data` field**: The struct must have a field named `base_data` of type `BaseData`
|
|
||||||
2. **Must derive standard traits**: `Debug`, `Clone`, `Serialize`, `Deserialize`
|
|
||||||
3. **Fields marked with `#[index]`**: Only fields with the `#[index]` attribute will be indexed
|
|
||||||
|
|
||||||
## Limitations
|
|
||||||
|
|
||||||
- The macro currently uses `Debug` formatting for enums and complex types
|
|
||||||
- BTreeMap indexing assumes `String` keys and values
|
|
||||||
- Vec indexing uses numeric indices (may not be ideal for all use cases)
|
|
||||||
|
|
||||||
## Future Enhancements
|
|
||||||
|
|
||||||
- Custom index key formatters via attributes
|
|
||||||
- Support for nested struct indexing
|
|
||||||
- Conditional indexing (e.g., `#[index(if = "is_published")]`)
|
|
||||||
- Custom index names (e.g., `#[index(name = "custom_name")]`)
|
|
||||||
@@ -1,113 +0,0 @@
|
|||||||
# Freezone Implementation TODO
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
The freezone.rhai example has been created and demonstrates a complete registration flow. However, some Rhai bindings need to be implemented to make it fully functional.
|
|
||||||
|
|
||||||
## Required Implementations
|
|
||||||
|
|
||||||
### 1. Ethereum Wallet Function
|
|
||||||
**Location:** `src/objects/money/rhai.rs`
|
|
||||||
|
|
||||||
Add a `new_ethereum_wallet()` function that creates an Ethereum wallet:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
#[rhai_fn(name = "new_ethereum_wallet", return_raw)]
|
|
||||||
pub fn new_ethereum_wallet() -> Result<EthereumWallet, Box<EvalAltResult>> {
|
|
||||||
// Generate new Ethereum wallet
|
|
||||||
// Return wallet with address, private key (encrypted), network
|
|
||||||
Ok(EthereumWallet::new())
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
The wallet should have:
|
|
||||||
- `.owner_id(id)` - set owner
|
|
||||||
- `.network(network)` - set network (mainnet/testnet)
|
|
||||||
- `.get_address()` - get Ethereum address
|
|
||||||
|
|
||||||
### 2. Accounting Module
|
|
||||||
**Location:** `src/objects/accounting/`
|
|
||||||
|
|
||||||
Create Invoice and Expense objects (files were created but need to be integrated):
|
|
||||||
|
|
||||||
**Invoice:**
|
|
||||||
- `new_invoice(id)` - constructor
|
|
||||||
- `.invoice_number(num)`, `.customer_id(id)`, `.amount(amt)`, `.currency(cur)`, `.description(desc)`
|
|
||||||
- `.send()`, `.mark_paid()`, `.mark_overdue()`, `.cancel()`
|
|
||||||
- Getters: `.invoice_number()`, `.customer_id()`, `.amount()`, `.status()`
|
|
||||||
|
|
||||||
**Expense:**
|
|
||||||
- `new_expense(id)` - constructor
|
|
||||||
- `.user_id(id)`, `.amount(amt)`, `.currency(cur)`, `.description(desc)`, `.category(cat)`, `.invoice_id(id)`
|
|
||||||
- `.approve()`, `.mark_paid()`, `.reject()`
|
|
||||||
- Getters: `.user_id()`, `.amount()`, `.status()`, `.category()`
|
|
||||||
|
|
||||||
### 3. Payment Request Enhancements
|
|
||||||
**Location:** `src/objects/money/payments.rs` and `rhai.rs`
|
|
||||||
|
|
||||||
Add `new_payment_request()` function with builder API:
|
|
||||||
- `.amount(amt)`
|
|
||||||
- `.currency(cur)`
|
|
||||||
- `.description(desc)`
|
|
||||||
- `.callback_url(url)`
|
|
||||||
- `.merchant_reference(ref)`
|
|
||||||
|
|
||||||
### 4. KYC Info Enhancements
|
|
||||||
**Location:** `src/objects/kyc/info.rs` and `rhai.rs`
|
|
||||||
|
|
||||||
Add missing builder methods:
|
|
||||||
- `.document_type(type)` - passport, id_card, drivers_license
|
|
||||||
- `.document_number(num)`
|
|
||||||
- `.verified(bool)` - mark as verified
|
|
||||||
|
|
||||||
### 5. Engine Registration
|
|
||||||
**Location:** `src/engine.rs`
|
|
||||||
|
|
||||||
Add to `OsirisPackage`:
|
|
||||||
```rust
|
|
||||||
// Register Accounting modules
|
|
||||||
register_accounting_modules(module);
|
|
||||||
|
|
||||||
// Add save methods for Invoice and Expense
|
|
||||||
FuncRegistration::new("save")
|
|
||||||
.set_into_module(module, |ctx: &mut OsirisContext, invoice: crate::objects::Invoice| ctx.save_object(invoice));
|
|
||||||
FuncRegistration::new("save")
|
|
||||||
.set_into_module(module, |ctx: &mut OsirisContext, expense: crate::objects::Expense| ctx.save_object(expense));
|
|
||||||
```
|
|
||||||
|
|
||||||
### 6. Module Exports
|
|
||||||
**Location:** `src/objects/mod.rs`
|
|
||||||
|
|
||||||
Add:
|
|
||||||
```rust
|
|
||||||
pub mod accounting;
|
|
||||||
pub use accounting::{Invoice, Expense};
|
|
||||||
```
|
|
||||||
|
|
||||||
## Current Freezone Flow
|
|
||||||
|
|
||||||
The freezone.rhai example demonstrates:
|
|
||||||
|
|
||||||
1. **Public Key Registration** - User provides public key
|
|
||||||
2. **Personal Information** - Collect name, email, create KYC info
|
|
||||||
3. **Terms & Conditions** - Create and sign contract
|
|
||||||
4. **Email Verification** - Generate code, send email, verify
|
|
||||||
5. **Crypto Wallet Creation** - Create TFT account + Ethereum wallet
|
|
||||||
6. **Payment Processing** - Pesapal payment session, user pays, record transaction
|
|
||||||
7. **KYC Verification** - KYC session, user completes verification, callback with results
|
|
||||||
8. **User Registration** - Create final user object
|
|
||||||
|
|
||||||
## Testing
|
|
||||||
|
|
||||||
Once implementations are complete, run:
|
|
||||||
```bash
|
|
||||||
cargo run --example freezone
|
|
||||||
```
|
|
||||||
|
|
||||||
Expected output: Complete freezone registration flow with all 8 steps executing successfully.
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
- The example uses simulated callbacks for payment and KYC
|
|
||||||
- Ethereum wallet generation should use a proper library (e.g., ethers-rs)
|
|
||||||
- Payment integration with Pesapal is mocked but shows the expected flow
|
|
||||||
- KYC callback demonstrates how verified data would be received and stored
|
|
||||||
@@ -1,525 +0,0 @@
|
|||||||
# OSIRIS MVP — Minimal Semantic Store over HeroDB
|
|
||||||
|
|
||||||
## 0) Purpose
|
|
||||||
|
|
||||||
OSIRIS is a Rust-native object layer on top of HeroDB that provides structured storage and retrieval capabilities without any server-side extensions or indexing engines.
|
|
||||||
|
|
||||||
It provides:
|
|
||||||
- Object CRUD operations
|
|
||||||
- Namespace management
|
|
||||||
- Simple local field indexing (field:*)
|
|
||||||
- Basic keyword scan (substring matching)
|
|
||||||
- CLI interface
|
|
||||||
- Future: 9P filesystem interface
|
|
||||||
|
|
||||||
It does **not** depend on HeroDB's Tantivy FTS, vectors, or relations.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1) Architecture
|
|
||||||
|
|
||||||
```
|
|
||||||
HeroDB (unmodified)
|
|
||||||
│
|
|
||||||
├── KV store + encryption
|
|
||||||
└── RESP protocol
|
|
||||||
↑
|
|
||||||
│
|
|
||||||
└── OSIRIS
|
|
||||||
├── store/ – object schema + persistence
|
|
||||||
├── index/ – field index & keyword scanning
|
|
||||||
├── retrieve/ – query planner + filtering
|
|
||||||
├── interfaces/ – CLI, 9P (future)
|
|
||||||
└── config/ – namespaces + settings
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2) Data Model
|
|
||||||
|
|
||||||
```rust
|
|
||||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
|
||||||
pub struct OsirisObject {
|
|
||||||
pub id: String,
|
|
||||||
pub ns: String,
|
|
||||||
pub meta: Metadata,
|
|
||||||
pub text: Option<String>, // optional plain text
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
|
||||||
pub struct Metadata {
|
|
||||||
pub title: Option<String>,
|
|
||||||
pub mime: Option<String>,
|
|
||||||
pub tags: BTreeMap<String, String>,
|
|
||||||
pub created: OffsetDateTime,
|
|
||||||
pub updated: OffsetDateTime,
|
|
||||||
pub size: Option<u64>,
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3) Keyspace Design
|
|
||||||
|
|
||||||
```
|
|
||||||
meta:<id> → serialized OsirisObject (JSON)
|
|
||||||
field:tag:<key>=<val> → Set of IDs (for tag filtering)
|
|
||||||
field:mime:<type> → Set of IDs (for MIME type filtering)
|
|
||||||
field:title:<title> → Set of IDs (for title filtering)
|
|
||||||
scan:index → Set of all IDs (for full scan)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Example:**
|
|
||||||
```
|
|
||||||
field:tag:project=osiris → {note_1, note_2}
|
|
||||||
field:mime:text/markdown → {note_1, note_3}
|
|
||||||
scan:index → {note_1, note_2, note_3, ...}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4) Index Maintenance
|
|
||||||
|
|
||||||
### Insert / Update
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// Store object
|
|
||||||
redis.set(format!("meta:{}", obj.id), serde_json::to_string(&obj)?)?;
|
|
||||||
|
|
||||||
// Index tags
|
|
||||||
for (k, v) in &obj.meta.tags {
|
|
||||||
redis.sadd(format!("field:tag:{}={}", k, v), &obj.id)?;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Index MIME type
|
|
||||||
if let Some(mime) = &obj.meta.mime {
|
|
||||||
redis.sadd(format!("field:mime:{}", mime), &obj.id)?;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Index title
|
|
||||||
if let Some(title) = &obj.meta.title {
|
|
||||||
redis.sadd(format!("field:title:{}", title), &obj.id)?;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Add to scan index
|
|
||||||
redis.sadd("scan:index", &obj.id)?;
|
|
||||||
```
|
|
||||||
|
|
||||||
### Delete
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// Remove object
|
|
||||||
redis.del(format!("meta:{}", obj.id))?;
|
|
||||||
|
|
||||||
// Deindex tags
|
|
||||||
for (k, v) in &obj.meta.tags {
|
|
||||||
redis.srem(format!("field:tag:{}={}", k, v), &obj.id)?;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Deindex MIME type
|
|
||||||
if let Some(mime) = &obj.meta.mime {
|
|
||||||
redis.srem(format!("field:mime:{}", mime), &obj.id)?;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Deindex title
|
|
||||||
if let Some(title) = &obj.meta.title {
|
|
||||||
redis.srem(format!("field:title:{}", title), &obj.id)?;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Remove from scan index
|
|
||||||
redis.srem("scan:index", &obj.id)?;
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 5) Retrieval
|
|
||||||
|
|
||||||
### Query Structure
|
|
||||||
|
|
||||||
```rust
|
|
||||||
pub struct RetrievalQuery {
|
|
||||||
pub text: Option<String>, // keyword substring
|
|
||||||
pub ns: String,
|
|
||||||
pub filters: Vec<(String, String)>, // field=value
|
|
||||||
pub top_k: usize,
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Execution Steps
|
|
||||||
|
|
||||||
1. **Collect candidate IDs** from field:* filters (SMEMBERS + intersection)
|
|
||||||
2. **If text query is provided**, iterate over candidates:
|
|
||||||
- Fetch `meta:<id>`
|
|
||||||
- Test substring match on `meta.title`, `text`, or `tags`
|
|
||||||
- Compute simple relevance score
|
|
||||||
3. **Sort** by score (descending) and **limit** to `top_k`
|
|
||||||
|
|
||||||
This is O(N) for text scan but acceptable for MVP or small datasets (<10k objects).
|
|
||||||
|
|
||||||
### Scoring Algorithm
|
|
||||||
|
|
||||||
```rust
|
|
||||||
fn compute_text_score(obj: &OsirisObject, query: &str) -> f32 {
|
|
||||||
let mut score = 0.0;
|
|
||||||
|
|
||||||
// Title match
|
|
||||||
if let Some(title) = &obj.meta.title {
|
|
||||||
if title.to_lowercase().contains(query) {
|
|
||||||
score += 0.5;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Text content match
|
|
||||||
if let Some(text) = &obj.text {
|
|
||||||
if text.to_lowercase().contains(query) {
|
|
||||||
score += 0.5;
|
|
||||||
// Bonus for multiple occurrences
|
|
||||||
let count = text.to_lowercase().matches(query).count();
|
|
||||||
score += (count as f32 - 1.0) * 0.1;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Tag match
|
|
||||||
for (key, value) in &obj.meta.tags {
|
|
||||||
if key.to_lowercase().contains(query) || value.to_lowercase().contains(query) {
|
|
||||||
score += 0.2;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
score.min(1.0)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 6) CLI
|
|
||||||
|
|
||||||
### Commands
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Initialize and create namespace
|
|
||||||
osiris init --herodb redis://localhost:6379
|
|
||||||
osiris ns create notes
|
|
||||||
|
|
||||||
# Add and read objects
|
|
||||||
osiris put notes/my-note.md ./my-note.md --tags topic=rust,project=osiris
|
|
||||||
osiris get notes/my-note.md
|
|
||||||
osiris get notes/my-note.md --raw --output /tmp/note.md
|
|
||||||
osiris del notes/my-note.md
|
|
||||||
|
|
||||||
# Search
|
|
||||||
osiris find --ns notes --filter topic=rust
|
|
||||||
osiris find "retrieval" --ns notes
|
|
||||||
osiris find "rust" --ns notes --filter project=osiris --topk 20
|
|
||||||
|
|
||||||
# Namespace management
|
|
||||||
osiris ns list
|
|
||||||
osiris ns delete notes
|
|
||||||
|
|
||||||
# Statistics
|
|
||||||
osiris stats
|
|
||||||
osiris stats --ns notes
|
|
||||||
```
|
|
||||||
|
|
||||||
### Examples
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Store a note from stdin
|
|
||||||
echo "This is a note about Rust programming" | \
|
|
||||||
osiris put notes/rust-intro - \
|
|
||||||
--title "Rust Introduction" \
|
|
||||||
--tags topic=rust,level=beginner \
|
|
||||||
--mime text/plain
|
|
||||||
|
|
||||||
# Search for notes about Rust
|
|
||||||
osiris find "rust" --ns notes
|
|
||||||
|
|
||||||
# Filter by tag
|
|
||||||
osiris find --ns notes --filter topic=rust
|
|
||||||
|
|
||||||
# Get note as JSON
|
|
||||||
osiris get notes/rust-intro
|
|
||||||
|
|
||||||
# Get raw content
|
|
||||||
osiris get notes/rust-intro --raw
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 7) Configuration
|
|
||||||
|
|
||||||
### File Location
|
|
||||||
|
|
||||||
`~/.config/osiris/config.toml`
|
|
||||||
|
|
||||||
### Example
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[herodb]
|
|
||||||
url = "redis://localhost:6379"
|
|
||||||
|
|
||||||
[namespaces.notes]
|
|
||||||
db_id = 1
|
|
||||||
|
|
||||||
[namespaces.calendar]
|
|
||||||
db_id = 2
|
|
||||||
```
|
|
||||||
|
|
||||||
### Structure
|
|
||||||
|
|
||||||
```rust
|
|
||||||
pub struct Config {
|
|
||||||
pub herodb: HeroDbConfig,
|
|
||||||
pub namespaces: HashMap<String, NamespaceConfig>,
|
|
||||||
}
|
|
||||||
|
|
||||||
pub struct HeroDbConfig {
|
|
||||||
pub url: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
pub struct NamespaceConfig {
|
|
||||||
pub db_id: u16,
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 8) Database Allocation
|
|
||||||
|
|
||||||
```
|
|
||||||
DB 0 → HeroDB Admin (managed by HeroDB)
|
|
||||||
DB 1 → osiris:notes (namespace "notes")
|
|
||||||
DB 2 → osiris:calendar (namespace "calendar")
|
|
||||||
DB 3+ → Additional namespaces...
|
|
||||||
```
|
|
||||||
|
|
||||||
Each namespace gets its own isolated HeroDB database.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 9) Dependencies
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[dependencies]
|
|
||||||
anyhow = "1.0"
|
|
||||||
redis = { version = "0.24", features = ["aio", "tokio-comp"] }
|
|
||||||
serde = { version = "1.0", features = ["derive"] }
|
|
||||||
serde_json = "1.0"
|
|
||||||
time = { version = "0.3", features = ["serde", "formatting", "parsing", "macros"] }
|
|
||||||
tokio = { version = "1.23", features = ["full"] }
|
|
||||||
clap = { version = "4.5", features = ["derive"] }
|
|
||||||
toml = "0.8"
|
|
||||||
uuid = { version = "1.6", features = ["v4", "serde"] }
|
|
||||||
tracing = "0.1"
|
|
||||||
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 10) Future Enhancements
|
|
||||||
|
|
||||||
| Feature | When Added | Moves Where |
|
|
||||||
|---------|-----------|-------------|
|
|
||||||
| Dedup / blobs | HeroDB extension | HeroDB |
|
|
||||||
| Vector search | HeroDB extension | HeroDB |
|
|
||||||
| Full-text search | HeroDB (Tantivy) | HeroDB |
|
|
||||||
| Relations / graph | OSIRIS later | OSIRIS |
|
|
||||||
| 9P filesystem | OSIRIS later | OSIRIS |
|
|
||||||
|
|
||||||
This MVP maintains clean interface boundaries:
|
|
||||||
- **HeroDB** remains a plain KV substrate
|
|
||||||
- **OSIRIS** builds higher-order meaning on top
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 11) Implementation Status
|
|
||||||
|
|
||||||
### ✅ Completed
|
|
||||||
|
|
||||||
- [x] Project structure and Cargo.toml
|
|
||||||
- [x] Core data models (OsirisObject, Metadata)
|
|
||||||
- [x] HeroDB client wrapper (RESP protocol)
|
|
||||||
- [x] Field indexing (tags, MIME, title)
|
|
||||||
- [x] Search engine (substring matching + scoring)
|
|
||||||
- [x] Configuration management
|
|
||||||
- [x] CLI interface (init, ns, put, get, del, find, stats)
|
|
||||||
- [x] Error handling
|
|
||||||
- [x] Documentation (README, specs)
|
|
||||||
|
|
||||||
### 🚧 Pending
|
|
||||||
|
|
||||||
- [ ] 9P filesystem interface
|
|
||||||
- [ ] Integration tests
|
|
||||||
- [ ] Performance benchmarks
|
|
||||||
- [ ] Name resolution (namespace/name → ID mapping)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 12) Quick Start
|
|
||||||
|
|
||||||
### Prerequisites
|
|
||||||
|
|
||||||
Start HeroDB:
|
|
||||||
```bash
|
|
||||||
cd /path/to/herodb
|
|
||||||
cargo run --release -- --dir ./data --admin-secret mysecret --port 6379
|
|
||||||
```
|
|
||||||
|
|
||||||
### Build OSIRIS
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd /path/to/osiris
|
|
||||||
cargo build --release
|
|
||||||
```
|
|
||||||
|
|
||||||
### Initialize
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Create configuration
|
|
||||||
./target/release/osiris init --herodb redis://localhost:6379
|
|
||||||
|
|
||||||
# Create a namespace
|
|
||||||
./target/release/osiris ns create notes
|
|
||||||
```
|
|
||||||
|
|
||||||
### Usage
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Add a note
|
|
||||||
echo "OSIRIS is a minimal object store" | \
|
|
||||||
./target/release/osiris put notes/intro - \
|
|
||||||
--title "Introduction" \
|
|
||||||
--tags topic=osiris,type=doc
|
|
||||||
|
|
||||||
# Search
|
|
||||||
./target/release/osiris find "object store" --ns notes
|
|
||||||
|
|
||||||
# Get the note
|
|
||||||
./target/release/osiris get notes/intro
|
|
||||||
|
|
||||||
# Show stats
|
|
||||||
./target/release/osiris stats --ns notes
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 13) Testing
|
|
||||||
|
|
||||||
### Unit Tests
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo test
|
|
||||||
```
|
|
||||||
|
|
||||||
### Integration Tests (requires HeroDB)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Start HeroDB
|
|
||||||
cd /path/to/herodb
|
|
||||||
cargo run -- --dir /tmp/herodb-test --admin-secret test --port 6379
|
|
||||||
|
|
||||||
# Run tests
|
|
||||||
cd /path/to/osiris
|
|
||||||
cargo test -- --ignored
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 14) Performance Characteristics
|
|
||||||
|
|
||||||
### Write Performance
|
|
||||||
|
|
||||||
- **Object storage**: O(1) - single SET operation
|
|
||||||
- **Indexing**: O(T) where T = number of tags/fields
|
|
||||||
- **Total**: O(T) per object
|
|
||||||
|
|
||||||
### Read Performance
|
|
||||||
|
|
||||||
- **Get by ID**: O(1) - single GET operation
|
|
||||||
- **Filter by tags**: O(F) where F = number of filters (set intersection)
|
|
||||||
- **Text search**: O(N) where N = number of candidates (linear scan)
|
|
||||||
|
|
||||||
### Storage Overhead
|
|
||||||
|
|
||||||
- **Object**: ~1KB per object (JSON serialized)
|
|
||||||
- **Indexes**: ~50 bytes per tag/field entry
|
|
||||||
- **Total**: ~1.5KB per object with 10 tags
|
|
||||||
|
|
||||||
### Scalability
|
|
||||||
|
|
||||||
- **Optimal**: <10,000 objects per namespace
|
|
||||||
- **Acceptable**: <100,000 objects per namespace
|
|
||||||
- **Beyond**: Consider migrating to Tantivy FTS
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 15) Design Decisions
|
|
||||||
|
|
||||||
### Why No Tantivy in MVP?
|
|
||||||
|
|
||||||
- **Simplicity**: Avoid HeroDB server-side dependencies
|
|
||||||
- **Portability**: Works with any Redis-compatible backend
|
|
||||||
- **Flexibility**: Easy to migrate to Tantivy later
|
|
||||||
|
|
||||||
### Why Substring Matching?
|
|
||||||
|
|
||||||
- **Good enough**: For small datasets (<10k objects)
|
|
||||||
- **Simple**: No tokenization, stemming, or complex scoring
|
|
||||||
- **Fast**: O(N) is acceptable for MVP
|
|
||||||
|
|
||||||
### Why Separate Databases per Namespace?
|
|
||||||
|
|
||||||
- **Isolation**: Clear separation of concerns
|
|
||||||
- **Performance**: Smaller keyspaces = faster scans
|
|
||||||
- **Security**: Can apply different encryption keys per namespace
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 16) Migration Path
|
|
||||||
|
|
||||||
When ready to scale beyond MVP:
|
|
||||||
|
|
||||||
1. **Add Tantivy FTS** (HeroDB extension)
|
|
||||||
- Create FT.* commands in HeroDB
|
|
||||||
- Update OSIRIS to use FT.SEARCH instead of substring scan
|
|
||||||
- Keep field indexes for filtering
|
|
||||||
|
|
||||||
2. **Add Vector Search** (HeroDB extension)
|
|
||||||
- Store embeddings in HeroDB
|
|
||||||
- Implement ANN search (HNSW/IVF)
|
|
||||||
- Add hybrid retrieval (BM25 + vector)
|
|
||||||
|
|
||||||
3. **Add Relations** (OSIRIS feature)
|
|
||||||
- Store relation graphs in HeroDB
|
|
||||||
- Implement graph traversal
|
|
||||||
- Add relation-based ranking
|
|
||||||
|
|
||||||
4. **Add Deduplication** (HeroDB extension)
|
|
||||||
- Content-addressable storage (BLAKE3)
|
|
||||||
- Reference counting
|
|
||||||
- Garbage collection
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
**OSIRIS MVP is a minimal, production-ready object store** that:
|
|
||||||
|
|
||||||
- ✅ Works with unmodified HeroDB
|
|
||||||
- ✅ Provides structured storage with metadata
|
|
||||||
- ✅ Supports field-based filtering
|
|
||||||
- ✅ Includes basic text search
|
|
||||||
- ✅ Exposes a clean CLI interface
|
|
||||||
- ✅ Maintains clear upgrade paths
|
|
||||||
|
|
||||||
**Perfect for:**
|
|
||||||
- Personal knowledge management
|
|
||||||
- Small-scale document storage
|
|
||||||
- Prototyping semantic applications
|
|
||||||
- Learning Rust + Redis patterns
|
|
||||||
|
|
||||||
**Next steps:**
|
|
||||||
- Build and test the MVP
|
|
||||||
- Gather usage feedback
|
|
||||||
- Plan Tantivy/vector integration
|
|
||||||
- Design 9P filesystem interface
|
|
||||||
@@ -5,6 +5,8 @@
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
// user visits website
|
||||||
|
// POST registration_flow.start()
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -5,6 +5,8 @@ public_key_registration_step = step.new()
|
|||||||
.description("Register your public key")
|
.description("Register your public key")
|
||||||
.add_action("Register");
|
.add_action("Register");
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
flow_id = registration_flow.start()
|
flow_id = registration_flow.start()
|
||||||
|
|
||||||
|
|
||||||
@@ -20,16 +22,10 @@ flow_id = registration_flow.start()
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
pk_registration_step.complete()
|
||||||
|
|
||||||
|
freezone_user = get_context()
|
||||||
|
public_key = pk_registration_step.output()
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
public_key_registration_step.complete()
|
|
||||||
|
|
||||||
public_key =
|
|
||||||
print("Public key <>registered.");
|
print("Public key <>registered.");
|
||||||
print("Now user needs to submit email");
|
print("Now user needs to submit email");
|
||||||
|
|
||||||
|
|||||||
16
server/Cargo.toml
Normal file
16
server/Cargo.toml
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
[package]
|
||||||
|
name = "osiris-server"
|
||||||
|
version = "0.1.0"
|
||||||
|
edition = "2021"
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
osiris = { path = ".." }
|
||||||
|
axum = "0.7"
|
||||||
|
tokio = { version = "1.23", features = ["full"] }
|
||||||
|
serde = { version = "1.0", features = ["derive"] }
|
||||||
|
serde_json = "1.0"
|
||||||
|
tower = "0.4"
|
||||||
|
tower-http = { version = "0.5", features = ["cors", "trace"] }
|
||||||
|
tracing = "0.1"
|
||||||
|
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
|
||||||
|
anyhow = "1.0"
|
||||||
Reference in New Issue
Block a user