tfgrid/projectmycelium

Fork 0

Files

mik-tf b41efb0e99 init projectmycelium

2025-09-01 21:37:01 -04:00

21 KiB

Raw Blame History

Project Mycelium - Comprehensive Slice Management Plan

Complete UX Vision & Implementation Strategy

🌟 Vision & End Goals

The Big Picture

Transform the Project Mycelium from a node-centric to a slice-centric compute marketplace, where users can rent standardized compute units instead of entire nodes. This democratizes access to ThreeFold's decentralized infrastructure by making it more affordable and accessible.

Core Value Proposition

For Users: Rent exactly what you need (1 vCPU, 4GB RAM, 200GB storage) instead of entire nodes
For Farmers: Maximize node utilization by selling multiple slices per node
For ThreeFold: Increase marketplace liquidity and accessibility

Key Success Metrics

Increased marketplace activity - More rentals due to lower entry barriers
Higher node utilization - Farmers earn more from partially-used nodes
Better user experience - Clear, standardized compute units
Reduced complexity - No more guessing about resource allocation

🎯 Complete User Experience Design

🚜 Farmer Journey: From Node to Slices

Current State (Node-based)

Farmer adds 32 vCPU, 128GB RAM, 2TB storage node
→ Sets price for entire node ($200/month)
→ Node sits empty until someone needs ALL resources
→ Low utilization, missed revenue

Target State (Slice-based)

Farmer adds 32 vCPU, 128GB RAM, 2TB storage node
→ System automatically calculates: 32 slices available
   (min(32/1, 128/4, 2000/200) = min(32, 32, 10) = 10 slices)
→ Farmer sets price per slice ($25/month per slice)
→ 10 different users can rent individual slices
→ Maximum utilization, predictable revenue

Detailed Farmer UX Flow

Step 1: Node Addition (Enhanced)

Farmer navigates to "Add Node" in dashboard
Enters ThreeFold Grid node ID
System fetches real-time capacity from gridproxy API
NEW: System automatically shows "This node can provide X slices"
NEW: Farmer sees potential monthly revenue: "Up to $X/month if all slices rented"

Step 2: Slice Configuration (New)

Farmer sets price per slice (within platform min/max limits)
NEW: Farmer sets node-level SLA commitments:
- Minimum uptime guarantee (e.g., 99.5%)
- Minimum bandwidth guarantee (e.g., 100 Mbps)
NEW: System shows competitive analysis: "Similar nodes in your region charge $X-Y"
Farmer confirms and publishes slices to marketplace

Step 3: Slice Management Dashboard (Enhanced)

NEW: Real-time slice allocation view:

Node: farmer-node-001 (Belgium)
├── Slice 1: 🟢 Rented by user@example.com (expires: 2025-02-15)
├── Slice 2: 🟢 Rented by dev@startup.com (expires: 2025-03-01)
├── Slice 3: 🟡 Available
└── Slice 4: 🟡 Available

NEW: Revenue tracking: "This month: $150 earned, $100 pending"
NEW: Performance monitoring: "Uptime: 99.8% (above SLA)"

Step 4: Ongoing Management (Enhanced)

Farmer receives notifications when slices are rented/released
NEW: Automatic price suggestions based on demand
NEW: SLA monitoring alerts if uptime/bandwidth drops below commitments

👤 User Journey: From Complex to Simple

Current State (Node-based)

User needs 1 vCPU for small app
→ Must rent entire 32 vCPU node ($200/month)
→ Wastes 31 vCPUs, overpays by 3100%
→ Complex resource planning required

Target State (Slice-based)

User needs 1 vCPU for small app
→ Rents 1 slice with exactly 1 vCPU, 4GB RAM, 200GB storage
→ Pays fair price ($25/month)
→ Simple, predictable resources

Detailed User UX Flow

Step 1: Marketplace Discovery (Enhanced)

User visits /marketplace/compute

NEW: Clear slice-based interface:

🍰 Available Compute Slices

Each slice provides:
• 1 vCPU core
• 4GB RAM
• 200GB SSD storage

[Filter by Location] [Filter by Price] [Filter by SLA]

Step 2: Slice Selection (New)

NEW: Standardized slice cards showing:

📍 Belgium • Node: farmer-node-001
💰 $25/month • 🔄 99.8% uptime • 🌐 150 Mbps

Resources (guaranteed):
🖥️ 1 vCPU  💾 4GB RAM  💿 200GB Storage

[Rent This Slice]

NEW: No complex resource calculations needed
NEW: Clear SLA commitments visible upfront

Step 3: Rental Process (New)

User clicks "Rent This Slice"

NEW: Simple rental modal:

Rent Compute Slice

Duration: [1 month ▼] [3 months] [6 months] [12 months]
Total cost: $25/month

You'll get:
• Dedicated 1 vCPU, 4GB RAM, 200GB storage
• Access credentials within 5 minutes
• 99.8% uptime SLA guarantee

[Confirm Rental - $25]

Step 4: Instant Access (Enhanced)

NEW: Atomic allocation prevents double-booking
NEW: Immediate access credentials provided
NEW: Clear resource boundaries (no sharing confusion)

Step 5: Management Dashboard (Enhanced)

NEW: Simple slice tracking:

My Compute Slices

slice-001 (Belgium)
├── Status: 🟢 Active
├── Resources: 1 vCPU, 4GB RAM, 200GB storage
├── Expires: 2025-02-15
├── Cost: $25/month
└── [Extend Rental] [Access Console]

🏗️ Technical Architecture & Implementation

System Design Principles

Leverage Existing Infrastructure: Build on robust GridService, FarmNode, NodeMarketplaceService
Atomic Operations: Prevent slice double-booking with file locking
Clear Separation of Concerns:
- Node-level: SLA characteristics (uptime, bandwidth, location)
- Slice-level: Compute resources (1 vCPU, 4GB RAM, 200GB storage)
Backward Compatibility: Existing node rentals continue working

Core Algorithm: Slice Calculation

// The heart of the system: How many slices can a node provide?
fn calculate_max_slices(node: &FarmNode) -> i32 {
    let available_cpu = node.capacity.cpu_cores - node.used_capacity.cpu_cores;
    let available_ram = node.capacity.memory_gb - node.used_capacity.memory_gb;
    let available_storage = node.capacity.storage_gb - node.used_capacity.storage_gb;
    
    // Each slice needs: 1 vCPU, 4GB RAM, 200GB storage
    let max_by_cpu = available_cpu / 1;
    let max_by_ram = available_ram / 4;
    let max_by_storage = available_storage / 200;
    
    // Bottleneck determines maximum slices
    std::cmp::min(std::cmp::min(max_by_cpu, max_by_ram), max_by_storage)
}

Example Calculations:

High-CPU node: 32 vCPU, 64GB RAM, 1TB storage → min(32, 16, 5) = 5 slices
Balanced node: 16 vCPU, 64GB RAM, 4TB storage → min(16, 16, 20) = 16 slices
Storage-heavy node: 8 vCPU, 32GB RAM, 10TB storage → min(8, 8, 50) = 8 slices

Data Model Enhancement

Enhanced FarmNode Structure

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct FarmNode {
    // ... ALL existing fields remain unchanged ...
    
    // NEW: Slice management
    pub calculated_slices: Vec<CalculatedSlice>,
    pub allocated_slices: Vec<AllocatedSlice>,
    pub slice_price_per_hour: Decimal,
    
    // NEW: Node-level SLA (inherited by all slices)
    pub min_uptime_sla: f32,        // e.g., 99.5%
    pub min_bandwidth_mbps: i32,    // e.g., 100 Mbps
    pub price_locked: bool,         // Prevent farmer price manipulation
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct CalculatedSlice {
    pub id: String,                 // e.g., "node123_slice_1"
    pub cpu_cores: i32,            // Always 1
    pub memory_gb: i32,            // Always 4
    pub storage_gb: i32,           // Always 200
    pub status: SliceStatus,       // Available/Reserved/Allocated
}

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AllocatedSlice {
    pub slice_id: String,
    pub renter_email: String,
    pub rental_start: DateTime<Utc>,
    pub rental_end: Option<DateTime<Utc>>,
    pub monthly_cost: Decimal,
}

Service Architecture

1. SliceCalculatorService (New)

Purpose: Calculate available slices from node capacity
Key Method: calculate_available_slices(node: &FarmNode) -> Vec<CalculatedSlice>
Algorithm: Uses min(CPU/1, RAM/4, Storage/200) formula

2. SliceAllocationService (New)

Purpose: Atomic slice rental with conflict prevention
Key Method: rent_slice_atomic(node_id, slice_id, user_email, duration)
Features: File locking, double-booking prevention, automatic billing

3. Enhanced GridService

New Method: fetch_node_with_slices(node_id) -> FarmNode
Enhancement: Automatically calculates slices when fetching from gridproxy

4. Enhanced NodeMarketplaceService

New Method: get_all_available_slices() -> Vec<SliceMarketplaceInfo>
Enhancement: Scans all farmer data, returns available slices instead of nodes

Marketplace Integration

Enhanced `/marketplace/compute` Endpoint

// Transform from node-centric to slice-centric
pub async fn compute_resources() -> Result<impl Responder> {
    // Get all available slices from all farmers
    let available_slices = node_marketplace_service.get_all_available_slices();
    
    // Apply filters (location, price, SLA requirements)
    let filtered_slices = apply_slice_filters(&available_slices, &query);
    
    // Paginate and format for display
    let slice_products = format_slices_for_display(&filtered_slices);
    
    ctx.insert("slices", &slice_products);
    render_template(&tmpl, "marketplace/compute.html", &ctx)
}

New `/api/marketplace/rent-slice` Endpoint

pub async fn rent_slice(request: web::Json<RentSliceRequest>) -> Result<impl Responder> {
    match SliceAllocationService::rent_slice_atomic(
        &request.node_id,
        &request.slice_id,
        &user_email,
        request.duration_months,
    ) {
        Ok(_) => Ok(HttpResponse::Ok().json({
            "success": true,
            "message": "Slice rented successfully",
            "access_info": "Credentials will be provided within 5 minutes"
        })),
        Err(e) => Ok(HttpResponse::BadRequest().json({
            "success": false,
            "message": e
        }))
    }
}

🎨 Frontend User Interface Design

Marketplace Display (Enhanced)

Before: Node-centric Display

Available Compute Nodes

Node: farmer-node-001 (Belgium)
Resources: 32 vCPU, 128GB RAM, 2TB storage
Price: $200/month
[Rent Entire Node]

After: Slice-centric Display

🍰 Available Compute Slices

<div class="slice-marketplace">
    <div class="slice-info-banner">
        <h4>What's a slice?</h4>
        <p>Each slice provides exactly <strong>1 vCPU, 4GB RAM, 200GB storage</strong> - perfect for small to medium applications.</p>
    </div>
    
    <div class="table-responsive">
        <table class="table table-striped">
            <thead>
                <tr>
                    <th>Location</th>
                    <th>Node</th>
                    <th>Resources</th>
                    <th>SLA Guarantee</th>
                    <th>Price</th>
                    <th>Action</th>
                </tr>
            </thead>
            <tbody>
                {{#each slices}}
                <tr>
                    <td>
                        <span class="location-badge">📍 {{node_info.location}}</span>
                    </td>
                    <td>{{node_info.name}}</td>
                    <td>
                        <div class="resource-badges">
                            <span class="badge badge-primary">🖥️ 1 vCPU</span>
                            <span class="badge badge-primary">💾 4GB RAM</span>
                            <span class="badge badge-primary">💿 200GB SSD</span>
                        </div>
                    </td>
                    <td>
                        <div class="sla-info">
                            <div>🔄 {{node_info.uptime_sla}}% uptime</div>
                            <div>🌐 {{node_info.bandwidth_sla}} Mbps</div>
                        </div>
                    </td>
                    <td>
                        <div class="price-info">
                            <strong>{{formatted_price}}</strong>
                            <small>/month</small>
                        </div>
                    </td>
                    <td>
                        <button class="btn btn-success btn-sm rent-slice-btn" 
                                data-slice-id="{{slice.id}}" 
                                data-node-id="{{node_info.id}}"
                                data-price="{{node_info.price_per_hour}}">
                            Rent Slice
                        </button>
                    </td>
                </tr>
                {{/each}}
            </tbody>
        </table>
    </div>
</div>

<div class="modal fade" id="rentSliceModal">
    <div class="modal-dialog">
        <div class="modal-content">
            <div class="modal-header">
                <h5>🍰 Rent Compute Slice</h5>
            </div>
            <div class="modal-body">
                <div class="slice-summary">
                    <h6>You're renting:</h6>
                    <ul>
                        <li>🖥️ 1 dedicated vCPU core</li>
                        <li>💾 4GB dedicated RAM</li>
                        <li>💿 200GB SSD storage</li>
                        <li>📍 Located in <span id="slice-location"></span></li>
                        <li>🔄 <span id="slice-uptime"></span>% uptime guarantee</li>
                    </ul>
                </div>
                
                <div class="rental-options">
                    <label>Rental Duration:</label>
                    <select id="rental-duration" class="form-control">
                        <option value="1">1 month - $<span class="monthly-price"></span></option>
                        <option value="3">3 months - $<span class="quarterly-price"></span> (5% discount)</option>
                        <option value="6">6 months - $<span class="biannual-price"></span> (10% discount)</option>
                        <option value="12">12 months - $<span class="annual-price"></span> (15% discount)</option>
                    </select>
                </div>
                
                <div class="access-info">
                    <h6>Access Information:</h6>
                    <p>After rental confirmation, you'll receive:</p>
                    <ul>
                        <li>SSH access credentials</li>
                        <li>IP address and connection details</li>
                        <li>Setup instructions</li>
                    </ul>
                    <p><small>⏱️ Access typically provided within 5 minutes</small></p>
                </div>
            </div>
            <div class="modal-footer">
                <button type="button" class="btn btn-secondary" data-dismiss="modal">Cancel</button>
                <button type="button" class="btn btn-success" id="confirm-rental">
                    Confirm Rental - $<span id="total-cost"></span>
                </button>
            </div>
        </div>
    </div>
</div>

Dashboard Enhancements

Farmer Dashboard: Slice Management

<div class="farmer-slice-dashboard">
    <h3>🚜 My Nodes & Slices</h3>
    
    {{#each nodes}}
    <div class="node-card">
        <div class="node-header">
            <h5>{{name}} ({{location}})</h5>
            <span class="utilization-badge">
                {{allocated_slices.length}}/{{calculated_slices.length}} slices rented
            </span>
        </div>
        
        <div class="slice-grid">
            {{#each calculated_slices}}
            <div class="slice-item {{#if (eq status 'Allocated')}}allocated{{else}}available{{/if}}">
                <div class="slice-id">Slice {{@index}}</div>
                <div class="slice-status">
                    {{#if (eq status 'Allocated')}}
                        🟢 Rented
                        <small>by {{renter_email}}</small>
                    {{else}}
                        🟡 Available
                    {{/if}}
                </div>
            </div>
            {{/each}}
        </div>
        
        <div class="node-stats">
            <div class="stat">
                <strong>${{monthly_revenue}}</strong>
                <small>Monthly Revenue</small>
            </div>
            <div class="stat">
                <strong>{{utilization_percentage}}%</strong>
                <small>Utilization</small>
            </div>
            <div class="stat">
                <strong>{{uptime_percentage}}%</strong>
                <small>Uptime</small>
            </div>
        </div>
    </div>
    {{/each}}
</div>

User Dashboard: My Slices

<div class="user-slice-dashboard">
    <h3>👤 My Compute Slices</h3>
    
    {{#each rented_slices}}
    <div class="slice-rental-card">
        <div class="slice-header">
            <h5>{{slice_id}}</h5>
            <span class="status-badge status-{{status}}">{{status}}</span>
        </div>
        
        <div class="slice-details">
            <div class="detail-row">
                <span>📍 Location:</span>
                <span>{{node_location}}</span>
            </div>
            <div class="detail-row">
                <span>🖥️ Resources:</span>
                <span>1 vCPU, 4GB RAM, 200GB storage</span>
            </div>
            <div class="detail-row">
                <span>💰 Cost:</span>
                <span>${{monthly_cost}}/month</span>
            </div>
            <div class="detail-row">
                <span>📅 Expires:</span>
                <span>{{rental_end}}</span>
            </div>
        </div>
        
        <div class="slice-actions">
            <button class="btn btn-primary btn-sm" onclick="accessSlice('{{slice_id}}')">
                Access Console
            </button>
            <button class="btn btn-success btn-sm" onclick="extendRental('{{slice_id}}')">
                Extend Rental
            </button>
        </div>
    </div>
    {{/each}}
</div>

🔄 Complete Implementation Workflow

Phase 1: Foundation (Day 1)

Enhance FarmNode model - Add slice fields to existing FarmNode
Create SliceCalculatorService - Implement slice calculation algorithm
Test slice calculation - Verify algorithm works with existing node data

Phase 2: Core Services (Day 2)

Enhance GridService - Add slice calculation to node fetching
Create SliceAllocationService - Implement atomic rental with file locking
Test allocation logic - Verify no double-booking possible

Phase 3: Marketplace Integration (Day 3)

Enhance NodeMarketplaceService - Add slice marketplace methods
Modify MarketplaceController - Update compute endpoint for slices
Add rental endpoint - Implement /api/marketplace/rent-slice

Phase 4: Frontend Enhancement (Day 4)

Update marketplace template - Replace node display with slice display
Add rental modal - Implement slice rental interface
Enhance dashboards - Add slice management for farmers and users

Phase 5: Testing & Polish (Day 5)

End-to-end testing - Full farmer and user journeys
Performance optimization - Ensure slice calculations are fast
Documentation - Update user guides and API docs

🎯 Success Criteria & Validation

Technical Validation

Slice calculation algorithm works correctly for all node types
Atomic allocation prevents double-booking under concurrent load
Marketplace displays slices instead of nodes
Rental process completes in under 30 seconds
Dashboard shows real-time slice allocation status

User Experience Validation

New users can understand slice concept without explanation
Farmers can see potential revenue increase from slice model
Rental process requires no technical knowledge
Users receive access credentials within 5 minutes
Both farmers and users prefer slice model over node model

Business Impact Validation

Increased number of rentals (lower barrier to entry)
Higher average node utilization (multiple slices per node)
Reduced support tickets (clearer resource allocation)
Positive farmer feedback (increased revenue potential)

🚀 Long-term Vision

Phase 2 Enhancements (Future)

Auto-scaling slices - Automatically adjust slice resources based on usage
Slice networking - Allow users to connect multiple slices
Slice templates - Pre-configured slices for specific use cases (web hosting, databases, etc.)
Advanced SLA monitoring - Real-time uptime and performance tracking
Slice marketplace analytics - Demand forecasting and pricing optimization

Integration Opportunities

ThreeFold Grid integration - Direct deployment to rented slices
Kubernetes support - Deploy containerized applications to slices
Monitoring integration - Built-in monitoring and alerting for slices
Backup services - Automated backup and restore for slice data

This comprehensive plan transforms the Project Mycelium into a modern, accessible, slice-based compute platform while leveraging all existing infrastructure and maintaining backward compatibility.

21 KiB Raw Blame History

Project Mycelium - Comprehensive Slice Management Plan

Complete UX Vision & Implementation Strategy

🌟 Vision & End Goals

The Big Picture

Core Value Proposition

Key Success Metrics

🎯 Complete User Experience Design

🚜 Farmer Journey: From Node to Slices

Current State (Node-based)

Target State (Slice-based)

Detailed Farmer UX Flow

👤 User Journey: From Complex to Simple

Current State (Node-based)

Target State (Slice-based)

Detailed User UX Flow

🏗️ Technical Architecture & Implementation

System Design Principles

Core Algorithm: Slice Calculation

Data Model Enhancement

Enhanced FarmNode Structure

Service Architecture

1. SliceCalculatorService (New)

2. SliceAllocationService (New)

3. Enhanced GridService

4. Enhanced NodeMarketplaceService

Marketplace Integration

Enhanced /marketplace/compute Endpoint

New /api/marketplace/rent-slice Endpoint

🎨 Frontend User Interface Design

Marketplace Display (Enhanced)

Before: Node-centric Display

After: Slice-centric Display

Rental Modal (New)

Dashboard Enhancements

Farmer Dashboard: Slice Management

User Dashboard: My Slices

🔄 Complete Implementation Workflow

Phase 1: Foundation (Day 1)

Phase 2: Core Services (Day 2)

Phase 3: Marketplace Integration (Day 3)

Phase 4: Frontend Enhancement (Day 4)

Phase 5: Testing & Polish (Day 5)

🎯 Success Criteria & Validation

Technical Validation

User Experience Validation

Business Impact Validation

🚀 Long-term Vision

Phase 2 Enhancements (Future)

Integration Opportunities

21 KiB

Raw Blame History

Enhanced `/marketplace/compute` Endpoint

New `/api/marketplace/rent-slice` Endpoint