# Project Mycelium Slice Management Implementation Plan ## Overview This document outlines the implementation plan for the new slice management system in the Project Mycelium. The system replaces the current manual slice configuration with an automatic slice calculation system based on node capacity fetched from gridproxy. ## Core Requirements ### Slice Definition - **Base Slice Unit**: 1 vCPU, 4GB RAM, 200GB storage - **Automatic Calculation**: System calculates maximum slices from node capacity - **Dynamic Combinations**: All possible slice combinations are available in marketplace - **Real-time Availability**: Slice availability updates when rented/released ### Key Principles - **No Manual Slice Configuration**: Farmers cannot create custom slices - **No Full Node Rental**: Only slice-based rentals are supported - **Grid-based Resource Data**: All capacity data comes from gridproxy API - **Slice-only Marketplace**: Focus exclusively on the new slice system - **Node-Slice Inheritance**: Slices inherit node characteristics (uptime, bandwidth, location) while having their own resource specs (CPU, RAM, storage) ## Architecture Overview ```mermaid graph TD A[Farmer Adds Node ID] --> B[GridService Fetches Real Data] B --> C[SliceCalculatorService] C --> D[Calculate Base Slices from Capacity] D --> E[Generate All Slice Combinations] E --> F[Create Marketplace Products] F --> G[Update User Data] H[User Rents Slice] --> I[Update Node Allocation] I --> J[Recalculate Available Combinations] J --> K[Update Marketplace Availability] L[Marketplace Browse] --> M[Real-time Availability Check] M --> N[Display Current Slice Options] ``` ## Slice Calculation Logic We present an example of the logic on how to set up a node with slices. ### Base Calculation ``` Node Capacity: 4 vCPU, 16GB RAM, 800GB Storage Base Slice: 1 vCPU, 4GB RAM, 200GB Storage Maximum Base Slices: min(4/1, 16/4, 800/200) = min(4, 4, 4) = 4 slices ``` ### Available Combinations From 4 base slices, the marketplace shows: - **1x Large**: 4 vCPU, 16GB RAM, 800GB Storage (uses 4 base slices) - **2x Medium**: 2 vCPU, 8GB RAM, 400GB Storage (uses 2 base slices each) - **4x Small**: 1 vCPU, 4GB RAM, 200GB Storage (uses 1 base slice each) **All slices inherit node characteristics:** - **Uptime**: 99.8% (from parent node) - **Bandwidth**: 1000 Mbps (shared from parent node) - **Location**: Belgium (from parent node) - **Certification**: Certified (from parent node) ### Dynamic Availability After user rents 1x Medium (2 vCPU, 8GB RAM, 400GB Storage): - **Remaining**: 2 base slices available - **New Options**: - 1x Medium: 2 vCPU, 8GB RAM, 400GB Storage (still inherits 99.8% uptime, 1000 Mbps bandwidth) - 2x Small: 1 vCPU, 4GB RAM, 200GB Storage (still inherits 99.8% uptime, 1000 Mbps bandwidth) ## Implementation Components ### 1. SliceCalculatorService ```rust /// Core service for slice calculations following builder pattern pub struct SliceCalculatorService { base_slice: SliceUnit, pricing_calculator: PricingCalculator, } /// Base slice unit definition pub struct SliceUnit { pub cpu_cores: u32, // 1 pub memory_gb: u32, // 4 pub storage_gb: u32, // 200 } /// Calculated slice combination pub struct SliceCombination { pub id: String, pub multiplier: u32, // How many base slices this uses pub cpu_cores: u32, // Slice-specific resource pub memory_gb: u32, // Slice-specific resource pub storage_gb: u32, // Slice-specific resource pub quantity_available: u32, // How many of this combination available pub price_per_hour: Decimal, pub base_slices_required: u32, // Inherited from parent node pub node_uptime_percentage: f64, pub node_bandwidth_mbps: u32, pub node_location: String, pub node_certification_type: String, } impl SliceCalculatorService { /// Calculate maximum base slices from node capacity pub fn calculate_max_base_slices(&self, capacity: &NodeCapacity) -> u32; /// Generate all possible slice combinations pub fn generate_slice_combinations(&self, max_base_slices: u32, allocated_slices: u32) -> Vec; /// Update availability after rental pub fn update_availability_after_rental(&self, node: &mut FarmNode, rented_combination: &SliceCombination) -> Result<(), String>; } ``` ### 2. Enhanced Data Models ```rust /// Enhanced FarmNode with slice allocation tracking pub struct FarmNode { // Existing fields... pub id: String, pub name: String, pub location: String, pub status: NodeStatus, pub capacity: NodeCapacity, pub grid_node_id: Option, pub grid_data: Option, // Node characteristics that slices inherit pub uptime_percentage: f64, pub bandwidth_mbps: u32, pub certification_type: String, pub farming_policy_id: u32, // NEW: Slice management fields pub total_base_slices: u32, pub allocated_base_slices: u32, pub slice_allocations: Vec, pub available_combinations: Vec, pub slice_pricing: SlicePricing, } /// Track individual slice rentals pub struct SliceAllocation { pub allocation_id: String, pub slice_combination_id: String, pub renter_email: String, pub base_slices_used: u32, pub rental_start: DateTime, pub rental_end: Option>, pub status: AllocationStatus, } /// Pricing configuration for node slices pub struct SlicePricing { pub base_price_per_hour: Decimal, // Price for 1 base slice per hour pub currency: String, pub pricing_multiplier: Decimal, // Farmer can adjust pricing (0.5x - 2.0x) } pub enum AllocationStatus { Active, Expired, Cancelled, } ``` ### 3. Updated UserPersistentData ```rust /// Remove old slice system, focus on new slice allocations pub struct UserPersistentData { // Existing fields... pub user_email: String, pub nodes: Vec, // REMOVED: slice_products field (old system) // NEW: Slice rental tracking pub slice_rentals: Vec, } pub struct SliceRental { pub rental_id: String, pub node_id: String, pub farmer_email: String, pub slice_allocation: SliceAllocation, pub total_cost: Decimal, pub payment_status: PaymentStatus, } ``` ### 4. Grid Integration Enhancement ```rust impl FarmerService { /// Enhanced node addition with automatic slice calculation pub async fn add_node_from_grid(&self, user_email: &str, grid_node_id: u32) -> Result { // 1. Fetch real node data from gridproxy let grid_data = self.grid_service.fetch_node_data(grid_node_id).await?; // 2. Calculate slice capacity let slice_calculator = SliceCalculatorService::builder().build()?; let max_base_slices = slice_calculator.calculate_max_base_slices(&grid_data.capacity); // 3. Generate initial slice combinations let available_combinations = slice_calculator.generate_slice_combinations(max_base_slices, 0); // 4. Create node with slice data let node = FarmNode { // ... basic node data from grid_data ... total_base_slices: max_base_slices, allocated_base_slices: 0, slice_allocations: Vec::new(), available_combinations, slice_pricing: SlicePricing::default(), }; // 5. Save and create marketplace products self.save_node_and_create_products(user_email, node).await } } ``` ### 5. Marketplace Integration ```rust impl NodeMarketplaceService { /// Get all available slice products from all farmer nodes pub fn get_available_slice_products(&self) -> Vec { let mut slice_products = Vec::new(); // Iterate through all farmer nodes for user_data in self.get_all_user_data() { for node in &user_data.nodes { // Convert each available slice combination to marketplace product for combination in &node.available_combinations { if combination.quantity_available > 0 { let product = self.create_slice_product(node, combination, &user_data.user_email); slice_products.push(product); } } } } slice_products } /// Create marketplace product from slice combination fn create_slice_product(&self, node: &FarmNode, combination: &SliceCombination, farmer_email: &str) -> Product { let mut attributes = HashMap::new(); // Slice-specific attributes attributes.insert("cpu_cores".to_string(), ProductAttribute { key: "cpu_cores".to_string(), value: serde_json::Value::Number(combination.cpu_cores.into()), attribute_type: AttributeType::Number, is_searchable: true, is_filterable: true, display_order: Some(1), }); attributes.insert("memory_gb".to_string(), ProductAttribute { key: "memory_gb".to_string(), value: serde_json::Value::Number(combination.memory_gb.into()), attribute_type: AttributeType::Number, is_searchable: true, is_filterable: true, display_order: Some(2), }); attributes.insert("storage_gb".to_string(), ProductAttribute { key: "storage_gb".to_string(), value: serde_json::Value::Number(combination.storage_gb.into()), attribute_type: AttributeType::Number, is_searchable: true, is_filterable: true, display_order: Some(3), }); // Inherited node characteristics attributes.insert("uptime_percentage".to_string(), ProductAttribute { key: "uptime_percentage".to_string(), value: serde_json::Value::Number(serde_json::Number::from_f64(node.uptime_percentage).unwrap()), attribute_type: AttributeType::Number, is_searchable: true, is_filterable: true, display_order: Some(4), }); attributes.insert("bandwidth_mbps".to_string(), ProductAttribute { key: "bandwidth_mbps".to_string(), value: serde_json::Value::Number(node.bandwidth_mbps.into()), attribute_type: AttributeType::Number, is_searchable: true, is_filterable: true, display_order: Some(5), }); attributes.insert("location".to_string(), ProductAttribute { key: "location".to_string(), value: serde_json::Value::String(node.location.clone()), attribute_type: AttributeType::Text, is_searchable: true, is_filterable: true, display_order: Some(6), }); attributes.insert("certification_type".to_string(), ProductAttribute { key: "certification_type".to_string(), value: serde_json::Value::String(node.certification_type.clone()), attribute_type: AttributeType::Text, is_searchable: true, is_filterable: true, display_order: Some(7), }); Product { id: format!("slice_{}_{}", node.id, combination.id), name: format!("{} vCPU, {}GB RAM, {}GB Storage - {}% Uptime", combination.cpu_cores, combination.memory_gb, combination.storage_gb, node.uptime_percentage), category_id: "compute".to_string(), description: format!("Slice from node {} in {} - {}% uptime, {} Mbps bandwidth", node.name, node.location, node.uptime_percentage, node.bandwidth_mbps), base_price: combination.price_per_hour, base_currency: "USD".to_string(), attributes, provider_id: farmer_email.to_string(), provider_name: farmer_email.to_string(), availability: if combination.quantity_available > 0 { ProductAvailability::Available } else { ProductAvailability::Unavailable }, metadata: ProductMetadata { tags: vec![ "compute".to_string(), "slice".to_string(), node.location.clone(), format!("{}vcpu", combination.cpu_cores), format!("{}gb-ram", combination.memory_gb), ], location: Some(node.location.clone()), rating: None, review_count: 0, custom_fields: HashMap::new(), }, created_at: Utc::now(), updated_at: Utc::now(), } } } ``` ## Current Marketplace Integration Analysis ### Existing Product Attributes System The current marketplace uses a generic `ProductAttribute` system that needs to be enhanced for slice inheritance: ```rust // Current system supports these attribute types: pub enum AttributeType { Text, Number, SliceConfiguration, // Already exists! Boolean, Select(Vec), // ... others } ``` ### Required Marketplace Refactoring #### 1. Product Search and Filtering **Current**: Users can filter by category, location, price range **Enhanced**: Add slice-specific filters with inherited characteristics ```rust // Enhanced search criteria for slices pub struct SliceSearchCriteria { // Slice-specific filters pub min_cpu_cores: Option, pub max_cpu_cores: Option, pub min_memory_gb: Option, pub max_memory_gb: Option, pub min_storage_gb: Option, pub max_storage_gb: Option, // Inherited node characteristics filters pub min_uptime_percentage: Option, pub min_bandwidth_mbps: Option, pub locations: Option>, pub certification_types: Option>, // Existing filters pub price_range: Option<(Decimal, Decimal)>, pub availability: Option, } ``` #### 2. Marketplace Controller Updates **Current**: `MarketplaceController::compute_resources()` shows generic compute products **Enhanced**: Show slice combinations with inherited characteristics ```rust impl MarketplaceController { pub async fn compute_resources(/* ... */) -> Result { // Get slice products instead of generic compute products let node_marketplace_service = NodeMarketplaceService::builder().build()?; let slice_products = node_marketplace_service.get_available_slice_products(); // Apply slice-specific filtering let filtered_slices = self.apply_slice_filters(&slice_products, &query); // Group slices by node characteristics for better UX let grouped_slices = self.group_slices_by_node_characteristics(&filtered_slices); ctx.insert("slice_products", &grouped_slices); ctx.insert("slice_filters", &self.get_slice_filter_options()); } } ``` #### 3. Template Updates **Current**: Generic product display templates **Enhanced**: Slice-specific templates showing inheritance ```html

{{cpu_cores}} vCPU, {{memory_gb}}GB RAM, {{storage_gb}}GB Storage

{{cpu_cores}} vCPU {{memory_gb}}GB RAM {{storage_gb}}GB Storage
{{uptime_percentage}}% Uptime {{bandwidth_mbps}} Mbps {{location}} {{certification_type}}
{{formatted_price}}/hour
``` #### 4. Enhanced Phase 4 Implementation **Updated Phase 4 tasks** to include inheritance-aware marketplace integration: - [ ] Update `NodeMarketplaceService` for slice products with inheritance - [ ] Modify marketplace controllers to show slice + node characteristics - [ ] Update compute resources page templates with inheritance display - [ ] Add slice-specific filtering (CPU, RAM, storage) + inherited filters (uptime, bandwidth, location) - [ ] Real-time availability updates with inheritance preservation - [ ] Group slices by node characteristics for better user experience ## Implementation Phases ### Phase 1: Core Slice Calculator (Week 1) - [ ] Create `SliceCalculatorService` with builder pattern - [ ] Implement base slice calculation algorithm - [ ] Implement slice combination generation - [ ] Add comprehensive unit tests - [ ] Integration with existing `GridService` ### Phase 2: Enhanced Data Models (Week 2) - [ ] Update `FarmNode` structure with slice fields - [ ] Remove old `slice_products` from `UserPersistentData` - [ ] Add new slice allocation tracking - [ ] Update JSON serialization/deserialization - [ ] Database migration scripts ### Phase 3: Grid Integration (Week 3) - [ ] Enhance `FarmerService::add_node()` to use gridproxy - [ ] Integrate `SliceCalculatorService` into node addition - [ ] Remove manual slice configuration options - [ ] Update farmer dashboard to show calculated slices - [ ] Add node validation and error handling ### Phase 4: Marketplace Integration (Week 4) - [ ] Update `NodeMarketplaceService` for slice products - [ ] Modify marketplace controllers for slice display - [ ] Update compute resources page templates - [ ] Add slice filtering and search capabilities - [ ] Real-time availability updates ### Phase 5: Rental System (Week 5) - [ ] Implement slice rental functionality - [ ] Add slice allocation tracking - [ ] Real-time availability recalculation - [ ] Rental management and tracking - [ ] Payment integration for slice rentals ### Phase 6: Testing & Optimization (Week 6) - [ ] End-to-end testing of slice system - [ ] Performance optimization for large node counts - [ ] User interface testing and refinement - [ ] Documentation and farmer onboarding - [ ] Production deployment preparation ## Technical Specifications ### Slice Calculation Algorithm ```rust fn calculate_max_base_slices(capacity: &NodeCapacity) -> u32 { let cpu_slices = capacity.cpu_cores / BASE_SLICE_CPU; let memory_slices = capacity.memory_gb / BASE_SLICE_MEMORY; let storage_slices = capacity.storage_gb / BASE_SLICE_STORAGE; // Return the limiting factor std::cmp::min(std::cmp::min(cpu_slices, memory_slices), storage_slices) } fn generate_combinations(max_base_slices: u32) -> Vec { let mut combinations = Vec::new(); // Generate all divisors of max_base_slices for multiplier in 1..=max_base_slices { if max_base_slices % multiplier == 0 { let quantity = max_base_slices / multiplier; combinations.push(SliceCombination { multiplier, cpu_cores: BASE_SLICE_CPU * multiplier, memory_gb: BASE_SLICE_MEMORY * multiplier, storage_gb: BASE_SLICE_STORAGE * multiplier, quantity_available: quantity, base_slices_required: multiplier, // ... other fields }); } } combinations } ``` ### Real-time Availability Updates ```rust impl SliceCalculatorService { pub fn update_availability_after_rental(&self, node: &mut FarmNode, rented_slices: u32) -> Result<(), String> { // Update allocated count node.allocated_base_slices += rented_slices; // Recalculate available combinations let available_base_slices = node.total_base_slices - node.allocated_base_slices; node.available_combinations = self.generate_slice_combinations(available_base_slices, 0); Ok(()) } } ``` ### Pricing Strategy - **Base Price**: Farmer sets price per base slice per hour - **Linear Scaling**: Larger combinations cost proportionally more - **Market Dynamics**: Farmers can adjust pricing multiplier (0.5x - 2.0x) - **Location Premium**: Optional location-based pricing adjustments ## Migration Strategy ### Complete System Replacement - **Remove old slice system entirely** - no backward compatibility needed - **Convert existing nodes** to use new slice calculation - **Migrate active rentals** to new slice allocation system - **Update all farmer dashboards** to show new slice system only ### Data Migration Steps 1. **Backup existing data** before migration 2. **Calculate slices for existing nodes** using gridproxy data 3. **Convert active slice rentals** to new allocation format 4. **Remove old slice_products fields** from user data 5. **Update marketplace products** to use new slice system ## Success Metrics ### Technical Metrics - **Slice Calculation Performance**: < 100ms per node - **Marketplace Load Time**: < 2s for slice product listing - **Real-time Updates**: < 500ms availability refresh - **System Reliability**: 99.9% uptime for slice calculations ### Business Metrics - **Farmer Adoption**: 90% of farmers successfully add nodes - **User Experience**: Intuitive slice selection interface - **Resource Utilization**: Improved node capacity utilization - **Marketplace Activity**: Increased slice rental transactions ## Risk Mitigation ### Technical Risks - **GridProxy Dependency**: Implement fallback mechanisms for API failures - **Performance Scaling**: Optimize for large numbers of nodes and combinations - **Data Consistency**: Ensure slice allocation accuracy across concurrent rentals ### Business Risks - **Farmer Resistance**: Clear communication about benefits of new system - **User Confusion**: Intuitive UI design for slice selection - **Migration Issues**: Thorough testing and gradual rollout ## Conclusion This implementation plan provides a comprehensive roadmap for replacing the current manual slice system with an automatic, grid-based slice calculation system. The new architecture ensures: - **Automatic slice calculation** from real node capacity - **Dynamic marketplace products** with real-time availability - **Simplified farmer experience** with no manual configuration - **Improved resource utilization** through standardized slicing - **Scalable architecture** following existing patterns The phased implementation approach minimizes risk while delivering a robust, user-friendly slice management system that aligns with ThreeFold's vision of decentralized compute resources.