init projectmycelium
This commit is contained in:
mik-tf
241
docs/dev/design/archive/SLICE_ARCHITECTURE.md
Normal file
241
docs/dev/design/archive/SLICE_ARCHITECTURE.md
Normal file
@@ -0,0 +1,241 @@
|
||||
# 🍰 Project Mycelium Slice Management System - Complete Implementation Recap
|
||||
|
||||
## 📋 Overview
|
||||
|
||||
The Project Mycelium now features a **comprehensive slice management system** that automatically converts physical node capacity into rentable compute slices. This system replaces manual slice configuration with intelligent, automatic slice calculation based on real node data from the ThreeFold Grid.
|
||||
|
||||
## 🎯 Core Concept: "Slices"
|
||||
|
||||
**What is a Slice?**
|
||||
- A **slice** is a standardized unit of compute resources: **1 vCPU + 4GB RAM + 200GB storage**
|
||||
- Slices can be **combined** to create larger configurations (2x, 4x, 8x, etc.)
|
||||
- Each slice **inherits characteristics** from its parent node (location, uptime, certification, bandwidth)
|
||||
|
||||
**Why Slices?**
|
||||
- **Standardization**: Consistent resource units across all nodes
|
||||
- **Flexibility**: Users can rent exactly what they need
|
||||
- **Scalability**: Easy to combine slices for larger workloads
|
||||
- **Inheritance**: Slices maintain node quality characteristics
|
||||
|
||||
## 🏗️ System Architecture
|
||||
|
||||
### 1. **Automatic Slice Calculation** [`SliceCalculatorService`](projectmycelium/src/services/slice_calculator.rs:139)
|
||||
```rust
|
||||
// Base slice unit: 1 vCPU, 4GB RAM, 200GB storage
|
||||
let base_slice = SliceUnit {
|
||||
cpu_cores: 1,
|
||||
memory_gb: 4,
|
||||
storage_gb: 200,
|
||||
};
|
||||
|
||||
// Automatically calculates how many base slices a node can provide
|
||||
let total_base_slices = calculate_max_base_slices(&node.capacity);
|
||||
|
||||
// Generates combinations: 1x, 2x, 4x, 8x slices
|
||||
let combinations = generate_slice_combinations(total_base_slices);
|
||||
```
|
||||
|
||||
### 2. **Node-Slice Inheritance** [`SliceCombination`](projectmycelium/src/services/slice_calculator.rs:24)
|
||||
```rust
|
||||
pub struct SliceCombination {
|
||||
// Slice specifications
|
||||
pub cpu_cores: i32,
|
||||
pub memory_gb: i32,
|
||||
pub storage_gb: i32,
|
||||
pub multiplier: u32,
|
||||
|
||||
// INHERITED from parent node
|
||||
pub node_uptime_percentage: f32,
|
||||
pub node_bandwidth_mbps: i32,
|
||||
pub node_location: String,
|
||||
pub node_certification_type: String,
|
||||
pub node_id: String,
|
||||
|
||||
// Availability & pricing
|
||||
pub quantity_available: u32,
|
||||
pub price_per_hour: Decimal,
|
||||
}
|
||||
```
|
||||
|
||||
### 3. **Atomic Slice Rental** [`SliceRentalService`](projectmycelium/src/services/slice_rental.rs:42)
|
||||
```rust
|
||||
// File locking prevents conflicts during rental
|
||||
let rental = slice_rental_service.rent_slice_combination(
|
||||
&user_email,
|
||||
&farmer_email,
|
||||
&node_id,
|
||||
&combination_id,
|
||||
quantity,
|
||||
rental_duration_hours,
|
||||
)?;
|
||||
```
|
||||
|
||||
### 4. **Grid Integration** [`FarmerService`](projectmycelium/src/services/farmer.rs:237)
|
||||
```rust
|
||||
// Automatically fetch real node data from ThreeFold Grid
|
||||
let grid_data = self.grid_service.fetch_node_data(grid_node_id).await?;
|
||||
|
||||
// Calculate slice capacity from real grid data
|
||||
let total_base_slices = self.slice_calculator
|
||||
.calculate_max_base_slices(&grid_data.total_resources);
|
||||
|
||||
// Auto-generate slice combinations with node inheritance
|
||||
node.available_combinations = self.slice_calculator
|
||||
.generate_slice_combinations(total_base_slices, allocated_slices, &node, user_email);
|
||||
```
|
||||
|
||||
## 🔄 Data Flow
|
||||
|
||||
### **1. Node Registration**
|
||||
```
|
||||
Farmer adds node → Grid data fetched → Slice capacity calculated → Combinations generated → Marketplace products created
|
||||
```
|
||||
|
||||
### **2. Slice Rental Process**
|
||||
```
|
||||
User browses slices → Selects combination → Rental request → Atomic allocation → Capacity updated → Confirmation sent
|
||||
```
|
||||
|
||||
### **3. Capacity Management**
|
||||
```
|
||||
Real-time sync → Grid data updated → Slice availability recalculated → Marketplace refreshed
|
||||
```
|
||||
|
||||
## 🛠️ Key Components
|
||||
|
||||
### **Services**
|
||||
- [`SliceCalculatorService`](projectmycelium/src/services/slice_calculator.rs) - Automatic slice calculation and combination generation
|
||||
- [`SliceRentalService`](projectmycelium/src/services/slice_rental.rs) - Atomic rental operations with conflict resolution
|
||||
- [`NodeMarketplaceService`](projectmycelium/src/services/node_marketplace.rs) - Converts slices to marketplace products
|
||||
- [`FarmerService`](projectmycelium/src/services/farmer.rs) - Enhanced with grid integration methods
|
||||
|
||||
### **Models**
|
||||
- [`SliceCombination`](projectmycelium/src/services/slice_calculator.rs:24) - Represents a rentable slice configuration
|
||||
- [`SliceAllocation`](projectmycelium/src/services/slice_calculator.rs:49) - Tracks allocated slices
|
||||
- [`SliceRental`](projectmycelium/src/services/slice_calculator.rs:65) - Records rental transactions
|
||||
- [`FarmNode`](projectmycelium/src/models/user.rs) - Enhanced with slice management fields
|
||||
|
||||
### **API Endpoints**
|
||||
- `POST /marketplace/rent-slice` - Rent slice combinations
|
||||
- `GET /api/dashboard/slice-products` - Get user's slice products
|
||||
- `POST /api/dashboard/slice-products` - Create custom slice products
|
||||
- `DELETE /api/dashboard/slice-products/{id}` - Delete slice products
|
||||
|
||||
## 📊 Enhanced Data Models
|
||||
|
||||
### **FarmNode Enhancement**
|
||||
```rust
|
||||
pub struct FarmNode {
|
||||
// ... existing fields ...
|
||||
|
||||
// NEW: Automatic slice management
|
||||
pub total_base_slices: u32, // Total slices this node can provide
|
||||
pub allocated_base_slices: u32, // Currently allocated slices
|
||||
pub slice_allocations: Vec<SliceAllocation>, // Active allocations
|
||||
pub available_combinations: Vec<SliceCombination>, // Available slice combinations
|
||||
pub slice_pricing: SlicePricing, // Pricing configuration
|
||||
pub slice_last_calculated: Option<DateTime<Utc>>, // Last calculation timestamp
|
||||
}
|
||||
```
|
||||
|
||||
### **Marketplace Integration**
|
||||
```rust
|
||||
// Slices appear as marketplace products with inherited characteristics
|
||||
attributes.insert("node_characteristics", ProductAttribute {
|
||||
value: json!({
|
||||
"uptime_percentage": combination.node_uptime_percentage,
|
||||
"bandwidth_mbps": combination.node_bandwidth_mbps,
|
||||
"location": combination.node_location,
|
||||
"certification_type": combination.node_certification_type,
|
||||
"node_id": combination.node_id
|
||||
}),
|
||||
attribute_type: AttributeType::Custom("node_inheritance".to_string()),
|
||||
});
|
||||
```
|
||||
|
||||
## 🎮 User Experience
|
||||
|
||||
### **For Farmers**
|
||||
1. **Add nodes** from ThreeFold Grid (automatic capacity detection)
|
||||
2. **View slice statistics** in dashboard (total slices, allocated, earnings)
|
||||
3. **Sync with grid** for real-time capacity updates
|
||||
4. **Automatic marketplace** product generation
|
||||
|
||||
### **For Users**
|
||||
1. **Browse slice marketplace** with inherited node characteristics
|
||||
2. **Filter by location, certification, uptime** (inherited from nodes)
|
||||
3. **Rent exact combinations** needed (1x, 2x, 4x, 8x slices)
|
||||
4. **Real-time availability** and pricing
|
||||
|
||||
## 🔧 Technical Benefits
|
||||
|
||||
### **Automatic Management**
|
||||
- No manual slice configuration required
|
||||
- Real-time capacity calculation from grid data
|
||||
- Automatic marketplace product generation
|
||||
|
||||
### **Data Consistency**
|
||||
- Atomic operations with file locking
|
||||
- Conflict resolution during concurrent rentals
|
||||
- Real-time availability updates
|
||||
|
||||
### **Scalability**
|
||||
- Standardized slice units enable easy scaling
|
||||
- Efficient combination generation algorithms
|
||||
- Optimized marketplace filtering and search
|
||||
|
||||
### **Quality Inheritance**
|
||||
- Slices inherit node uptime, location, certification
|
||||
- Users can make informed decisions based on node quality
|
||||
- Transparent pricing based on node characteristics
|
||||
|
||||
## 📁 File Structure
|
||||
|
||||
```
|
||||
projectmycelium/src/
|
||||
├── services/
|
||||
│ ├── slice_calculator.rs # Core slice calculation logic
|
||||
│ ├── slice_rental.rs # Atomic rental operations
|
||||
│ ├── node_marketplace.rs # Marketplace integration
|
||||
│ ├── farmer.rs # Enhanced with grid integration
|
||||
│ └── grid.rs # ThreeFold Grid API integration
|
||||
├── controllers/
|
||||
│ ├── marketplace.rs # Slice rental API endpoint
|
||||
│ └── dashboard.rs # Slice management UI
|
||||
└── models/
|
||||
├── user.rs # Enhanced FarmNode with slice fields
|
||||
└── product.rs # Slice product representations
|
||||
```
|
||||
|
||||
## 🚀 Future AI Developer Notes
|
||||
|
||||
### **Key Concepts to Remember**
|
||||
1. **Base Slice Unit**: Always 1 vCPU + 4GB RAM + 200GB storage
|
||||
2. **Inheritance**: Slices ALWAYS inherit parent node characteristics
|
||||
3. **Atomic Operations**: Use file locking for rental operations
|
||||
4. **Real-time Sync**: Grid integration keeps data current
|
||||
5. **Builder Pattern**: All services follow consistent builder pattern
|
||||
|
||||
### **Common Operations**
|
||||
```rust
|
||||
// Calculate slice capacity
|
||||
let total_slices = slice_calculator.calculate_max_base_slices(&node_capacity);
|
||||
|
||||
// Generate combinations
|
||||
let combinations = slice_calculator.generate_slice_combinations(total_slices, allocated, &node, farmer_email);
|
||||
|
||||
// Rent a slice
|
||||
let rental = slice_rental_service.rent_slice_combination(user, farmer, node, combination, quantity, duration)?;
|
||||
|
||||
// Sync with grid
|
||||
farmer_service.sync_node_with_grid(user_email, node_id).await?;
|
||||
```
|
||||
|
||||
### **Data Persistence**
|
||||
- All slice data stored in `./user_data/{email}.json`
|
||||
- Real-time updates to marketplace products
|
||||
- Automatic cleanup of expired rentals
|
||||
|
||||
## ✅ Implementation Status: 100% COMPLETE
|
||||
|
||||
The slice management system provides a **complete, automated, and scalable solution** for converting ThreeFold Grid nodes into rentable compute resources while maintaining quality inheritance and ensuring data consistency. All core functionality is implemented and operational.
|
||||
Reference in New Issue
Block a user