init projectmycelium

2025-09-01 21:37:01 -04:00
commit b41efb0e99
319 changed files with 128160 additions and 0 deletions
--- a/docs/dev/design/archive/plan_refactor_final.md
+++ b/docs/dev/design/archive/plan_refactor_final.md
@@ -0,0 +1,665 @@
+# Project Mycelium Credits System - Complete Refactor Plan
+
+## Overview
+
+This document outlines the complete transformation of the Project Mycelium from a TFP-based system to a simplified USD credits system inspired by OpenRouter's user experience. This is a major refactor that will become the new standard for the marketplace.
+
+## Current State vs Desired State
+
+### Current State (TFP System)
+- **Currency**: TFP (ThreeFold Points) with 1 TFP = 0.1 USD
+- **Complexity**: Multi-currency display (USD, EUR, CAD, TFP)
+- **Trading**: Complex Pools page with TFP/TFT/PEAQ trading
+- **UX**: Crypto-like terminology and fractional pricing
+- **Purchase Flow**: Add to cart → checkout (Buy Now exists but broken)
+- **Top-up**: Manual wallet top-up with currency conversion
+- **Pricing**: Products cost 50 TFP for $5 item (confusing)
+
+### Desired State (USD Credits System)
+- **Currency**: USD Credits with 1 Credit = $1 USD
+- **Simplicity**: Pure USD pricing throughout
+- **No Trading**: Hide Pools page (keep code for future)
+- **UX**: Clean "credits" terminology like mainstream SaaS
+- **Purchase Flow**: Seamless Buy Now + Add to Cart options
+- **Auto Top-up**: OpenRouter-style automatic credit purchasing
+- **Pricing**: Products cost $5 for $5 item (intuitive)
+
+## Detailed Transformation Plan
+
+### 1. Currency System Transformation
+
+#### Current TFP System
+```rust
+// Current: 1 TFP = 0.1 USD
+let tfp_amount = usd_amount * 10; // $5 = 50 TFP
+```
+
+#### New USD Credits System
+```rust
+// New: 1 Credit = 1 USD
+let credit_amount = usd_amount; // $5 = 5 Credits
+```
+
+#### Implementation Changes
+- **Currency Service**: Update exchange rate from 0.1 to 1.0
+- **Display Logic**: Remove TFP terminology, use "Credits" and "$" symbols
+- **Database**: Store amounts as USD credits (multiply existing TFP by 10)
+- **API Responses**: Return credit amounts instead of TFP amounts
+
+### 2. User Interface Transformation
+
+#### Navigation & Terminology
+- **Backend/Routes**: Keep "Wallet" terminology (`/dashboard/wallet`)
+- **UI Labels**: Use "Credits" terminology in content
+- **Example**: Sidebar shows "Wallet", page content shows "Buy Credits"
+
+#### Wallet Page Changes
+**Before:**
+```
+Wallet Balance: 150.5 TFP
+≈ $15.05 USD
+
+Quick Top-up:
+- Add 100 TFP ($10)
+- Add 250 TFP ($25)
+```
+
+**After:**
+```
+Credit Balance: $25.00
+
+Buy Credits:
+- Add $10 Credits
+- Add $25 Credits
+- Add $50 Credits
+- Custom Amount: $____
+
+Auto Top-up:
+☐ Enable Auto Top-up
+When credits are below: $10
+Purchase this amount: $25
+```
+
+#### Marketplace Pricing
+**Before:**
+```
+VM Instance: 50.0 TFP/month
+≈ $5.00 USD/month
+```
+
+**After:**
+```
+VM Instance: $5.00/month
+```
+
+### 3. Auto Top-up Implementation
+
+#### OpenRouter-Style Configuration
+```
+Auto Top-up Settings:
+☐ Enable auto top-up
+
+When credits are below:
+$ [10] 
+
+Purchase this amount:
+$ [25]
+
+Payment Method: [Credit Card ****1234] [Change]
+
+[Save Settings]
+```
+
+#### Technical Implementation
+```rust
+#[derive(Debug, Serialize, Deserialize)]
+pub struct AutoTopUpSettings {
+    pub enabled: bool,
+    pub threshold_amount: Decimal,  // USD amount
+    pub topup_amount: Decimal,      // USD amount
+    pub payment_method_id: String,
+    pub daily_limit: Option<Decimal>,
+    pub monthly_limit: Option<Decimal>,
+}
+```
+
+#### Auto Top-up Flow
+1. **Trigger Check**: Before any purchase, check if balance < threshold
+2. **Automatic Purchase**: If enabled, automatically buy credits
+3. **Fallback**: If auto top-up fails, prompt user for manual top-up
+4. **Limits**: Respect daily/monthly spending limits for security
+
+### 4. Buy Now Functionality Fix
+
+#### Current Issue
+- Buy Now buttons exist in marketplace templates
+- JavaScript event handlers are missing
+- Backend instant purchase API exists but not connected
+
+#### Solution Implementation
+```javascript
+// Fix: Add proper event handlers
+document.addEventListener('DOMContentLoaded', function() {
+    document.querySelectorAll('.buy-now-btn').forEach(button => {
+        button.addEventListener('click', async function() {
+            const productId = this.dataset.productId;
+            const productName = this.dataset.productName;
+            const unitPrice = parseFloat(this.dataset.unitPrice);
+            
+            await buyNow(productId, productName, 'compute', 1, unitPrice);
+        });
+    });
+});
+
+// Enhanced buyNow function with auto top-up integration
+window.buyNow = async function(productId, productName, category, quantity, unitPrice) {
+    try {
+        // Check balance and trigger auto top-up if needed
+        const affordabilityCheck = await fetch(`/api/wallet/check-affordability?amount=${unitPrice * quantity}`);
+        const result = await affordabilityCheck.json();
+        
+        if (!result.can_afford) {
+            const autoTopUpResult = await handleInsufficientBalance(result.shortfall_info);
+            if (!autoTopUpResult.success) return;
+        }
+        
+        // Proceed with instant purchase
+        const purchaseResponse = await fetch('/api/wallet/instant-purchase', {
+            method: 'POST',
+            headers: { 'Content-Type': 'application/json' },
+            body: JSON.stringify({
+                product_id: productId,
+                product_name: productName,
+                quantity: quantity,
+                unit_price_usd: unitPrice
+            })
+        });
+        
+        const purchaseResult = await purchaseResponse.json();
+        handlePurchaseResult(purchaseResult);
+        
+    } catch (error) {
+        console.error('Purchase failed:', error);
+        showErrorMessage('Purchase failed. Please try again.');
+    }
+};
+```
+
+### 5. Burn Rate Tracking
+
+#### User Dashboard Enhancement
+```
+Credit Usage Overview:
+
+Current Balance: $47.50
+
+Monthly Burn Rate: $23.00/month
+├── VM-Production: $15.00/month
+├── Gateway-EU: $5.00/month
+└── Storage-Backup: $3.00/month
+
+Projected Balance in 30 days: $24.50
+⚠️ Consider enabling auto top-up
+
+Auto Top-up Status: ✅ Enabled
+Next top-up triggers at: $10.00
+```
+
+#### Implementation
+```rust
+#[derive(Debug, Serialize)]
+pub struct BurnRateAnalysis {
+    pub monthly_burn_rate: Decimal,
+    pub daily_burn_rate: Decimal,
+    pub deployments: Vec<DeploymentBurnRate>,
+    pub projected_balance_30_days: Decimal,
+    pub days_until_empty: Option<u32>,
+}
+
+#[derive(Debug, Serialize)]
+pub struct DeploymentBurnRate {
+    pub deployment_id: String,
+    pub deployment_name: String,
+    pub monthly_cost: Decimal,
+    pub daily_cost: Decimal,
+    pub status: String,
+}
+```
+
+### 6. Pools Page Strategy
+
+#### Current Pools Functionality
+- TFP ↔ USD/EUR/CAD trading
+- TFP ↔ TFT/PEAQ crypto trading
+- Staking with marketplace benefits
+- Complex liquidity pool management
+
+#### New Strategy: Hide but Preserve
+- **Hide**: Remove Pools from main navigation
+- **Preserve**: Keep all code and functionality intact
+- **Access**: Add hidden route for future use
+- **Rationale**: Credits system eliminates need for trading, but keep for potential future features
+
+#### Implementation
+```rust
+// Hide pools route from main navigation
+// Comment out in routes/mod.rs:
+// .route("/dashboard/pools", web::get().to(DashboardController::pools))
+
+// Keep as hidden route for admin/future use:
+// .route("/dashboard/pools-advanced", web::get().to(DashboardController::pools))
+```
+
+### 7. Database Schema Updates
+
+#### UserPersistentData Changes
+```rust
+pub struct UserPersistentData {
+    // Change from TFP to USD credits
+    pub wallet_balance: Decimal,  // Now in USD (multiply existing TFP by 10)
+    
+    // Remove TFP-specific fields
+    // pub display_currency: Option<String>, // Remove - always USD now
+    
+    // Add auto top-up settings
+    pub auto_topup_settings: Option<AutoTopUpSettings>,
+    
+    // Update transaction amounts to USD
+    pub transactions: Vec<Transaction>,
+    
+    // Keep existing fields
+    pub user_email: String,
+    pub services: Vec<Service>,
+    // ... other fields unchanged
+}
+```
+
+#### Transaction Model Updates
+```rust
+pub enum TransactionType {
+    // Update existing types to use USD
+    Purchase { product_id: String, amount_usd: Decimal },
+    TopUp { amount_usd: Decimal, payment_method: String },
+    
+    // Add new auto top-up type
+    AutoTopUp { 
+        triggered_by_purchase: Option<String>,
+        threshold_amount: Decimal,
+        amount_usd: Decimal,
+        payment_method: String 
+    },
+    
+    // Keep existing types but update amounts to USD
+    Transfer { to_user: String, amount_usd: Decimal },
+    Earning { source: String, amount_usd: Decimal },
+}
+```
+
+### 8. API Endpoint Updates
+
+#### New Auto Top-up Endpoints
+```rust
+// Auto top-up configuration
+POST /api/wallet/auto-topup/configure
+GET  /api/wallet/auto-topup/status
+POST /api/wallet/auto-topup/trigger
+
+// Enhanced purchase with auto top-up
+POST /api/wallet/purchase-with-topup
+GET  /api/wallet/burn-rate-analysis
+```
+
+#### Updated Existing Endpoints
+```rust
+// Wallet info now returns USD credits
+GET /api/wallet/info
+// Response:
+{
+    "balance": 25.00,
+    "currency": "USD",
+    "auto_topup_enabled": true,
+    "auto_topup_threshold": 10.00
+}
+
+// Navbar data with credits
+GET /api/navbar/dropdown-data
+// Response:
+{
+    "wallet_balance_formatted": "$25.00",
+    "display_currency": "USD",
+    "auto_topup_enabled": true,
+    "auto_topup_status": "Active"
+}
+```
+
+### 9. Migration Strategy
+
+#### Data Migration
+```rust
+// Convert existing TFP balances to USD credits
+fn migrate_tfp_to_usd_credits(user_data: &mut UserPersistentData) {
+    // 1 TFP = 0.1 USD, so multiply by 10 to get USD credits
+    user_data.wallet_balance = user_data.wallet_balance * dec!(10);
+    
+    // Update all transaction amounts
+    for transaction in &mut user_data.transactions {
+        match &mut transaction.transaction_type {
+            TransactionType::Purchase { amount, .. } => *amount *= dec!(10),
+            TransactionType::TopUp { amount, .. } => *amount *= dec!(10),
+            // ... update other transaction types
+        }
+    }
+}
+```
+
+#### Deployment Strategy
+1. **Phase 1**: Deploy new code with migration logic
+2. **Phase 2**: Run migration script on user data
+3. **Phase 3**: Update frontend templates and JavaScript
+4. **Phase 4**: Hide pools page and update navigation
+
+### 10. User Experience Flow
+
+#### New User Journey
+1. **Registration**: User creates account
+2. **Wallet Setup**: Sees "$0.00" credit balance
+3. **Browse Marketplace**: Products priced in clear USD amounts
+4. **First Purchase**: 
+   - Clicks "Buy Now" on $5 VM
+   - Prompted to "Buy Credits" (insufficient balance)
+   - Buys $25 credits via credit card
+   - Purchase completes automatically
+5. **Auto Top-up Setup**: 
+   - Configures auto top-up: "When below $10, buy $25"
+   - Enables for seamless future purchases
+6. **Ongoing Usage**: 
+   - Monitors burn rate dashboard
+   - Auto top-up handles low balances
+   - Focuses on using services, not managing credits
+
+#### Purchase Flow Comparison
+
+**Before (TFP):**
+```
+1. See "VM: 50 TFP/month (≈$5 USD)"
+2. Check wallet: "15.5 TFP (≈$1.55)"
+3. Calculate: Need 34.5 more TFP
+4. Go to wallet, buy TFP in preferred currency
+5. Convert currency to TFP
+6. Return to marketplace
+7. Add to cart and checkout
+```
+
+**After (USD Credits):**
+```
+1. See "VM: $5.00/month"
+2. Click "Buy Now"
+3. Auto top-up triggers: Buys $25 credits
+4. Purchase completes: VM deployed
+5. Balance: $20.00 remaining
+```
+
+### 11. Implementation Timeline
+
+#### Week 1: Core System Changes
+- Update currency service (TFP → USD credits)
+- Modify database schema and migration scripts
+- Update wallet controller and API endpoints
+- Test data migration with mock data
+
+#### Week 2: Auto Top-up Implementation
+- Implement auto top-up service and settings
+- Add auto top-up configuration UI
+- Integrate auto top-up with purchase flow
+- Add burn rate calculation and display
+
+#### Week 3: UI/UX Updates
+- Update all templates (wallet, marketplace, navbar)
+- Fix Buy Now JavaScript event handlers
+- Hide pools page and update navigation
+- Add burn rate dashboard
+
+#### Week 4: Testing and Polish
+- End-to-end testing of purchase flows
+- Auto top-up testing with various scenarios
+- Performance testing and optimization
+- Documentation and deployment
+
+### 12. Success Metrics
+
+#### User Experience Metrics
+- **Purchase Completion Rate**: Target 95%+ (vs current cart abandonment)
+- **Time to First Purchase**: Target <2 minutes from registration
+- **Auto Top-up Adoption**: Target 40% of active users
+- **Support Tickets**: Reduce currency/balance related tickets by 80%
+
+#### Technical Metrics
+- **Buy Now Success Rate**: Target 99%+ (currently 0% due to broken JS)
+- **Auto Top-up Success Rate**: Target 98%+
+- **Page Load Performance**: Maintain <500ms for wallet/marketplace pages
+- **API Response Times**: <200ms for balance checks and purchases
+
+#### Business Metrics
+- **Average Purchase Size**: Expect increase due to easier purchasing
+- **Purchase Frequency**: Expect increase due to auto top-up
+- **User Retention**: Expect improvement due to better UX
+- **Revenue per User**: Expect increase due to reduced friction
+
+### 13. Risk Mitigation
+
+#### Technical Risks
+- **Data Migration**: Comprehensive testing with backup/rollback plan
+- **Payment Integration**: Multiple payment method support and fallbacks
+- **Auto Top-up Failures**: Graceful degradation and user notification
+- **Performance Impact**: Load testing and database optimization
+
+#### User Experience Risks
+- **Change Management**: Clear communication about improvements
+- **Learning Curve**: Intuitive design minimizes confusion
+- **Trust Issues**: Transparent pricing and clear auto top-up controls
+- **Edge Cases**: Comprehensive error handling and user guidance
+
+#### Business Risks
+- **Revenue Disruption**: Phased rollout with monitoring
+- **Regulatory Compliance**: Ensure auto top-up meets financial regulations
+- **Competitive Position**: Align with industry standards (OpenRouter model)
+- **Technical Debt**: Clean implementation to avoid future issues
+
+### 14. Testing Strategy
+
+#### Unit Tests
+- Currency conversion accuracy (TFP → USD credits)
+- Auto top-up trigger logic and limits
+- Purchase flow with insufficient balance scenarios
+- Burn rate calculation accuracy
+
+#### Integration Tests
+- End-to-end purchase flow (Browse → Buy Now → Auto Top-up → Success)
+- Auto top-up configuration and execution
+- Wallet balance updates across all interfaces
+- Cross-browser JavaScript functionality
+
+#### User Acceptance Tests
+- New user onboarding flow
+- Existing user experience with migrated data
+- Auto top-up configuration and usage
+- Mobile responsiveness and accessibility
+
+### 15. Monitoring and Analytics
+
+#### Key Performance Indicators
+- **Credit Purchase Volume**: Daily/monthly credit sales
+- **Auto Top-up Usage**: Frequency and amounts
+- **Purchase Success Rates**: Buy Now vs Add to Cart
+- **User Engagement**: Time spent in marketplace
+- **Error Rates**: Failed purchases, auto top-ups, payments
+
+#### Alerting System
+- **High Error Rates**: >5% purchase failures
+- **Auto Top-up Issues**: >10% auto top-up failures
+- **Performance Degradation**: >1s response times
+- **Payment Problems**: Payment processor issues
+
+#### Analytics Dashboard
+- **Real-time Metrics**: Active users, current purchases, credit balances
+- **Trend Analysis**: Purchase patterns, auto top-up adoption, user growth
+- **Financial Tracking**: Revenue, credit sales, burn rates
+- **User Behavior**: Most popular products, purchase paths, drop-off points
+
+## Conclusion
+
+This refactor transforms the Project Mycelium from a complex crypto-like TFP system to a clean, intuitive USD credits system that matches modern SaaS expectations. The OpenRouter-inspired approach eliminates currency confusion, enables seamless purchasing, and provides automatic balance management through auto top-up.
+
+Key benefits:
+- **Simplified UX**: Clear USD pricing eliminates confusion
+- **Reduced Friction**: Buy Now + Auto Top-up enables instant purchases
+- **Better Retention**: Easier purchasing leads to higher engagement
+- **Scalable Foundation**: Clean architecture supports future enhancements
+
+The implementation maintains backward compatibility where possible while providing a clear migration path to the new system. By hiding rather than removing the Pools functionality, we preserve advanced features for potential future use while focusing on the core marketplace experience.
+
+This refactor positions Project Mycelium as a modern, user-friendly platform that competes effectively with established SaaS providers while maintaining the unique value proposition of the ThreeFold ecosystem.
+
+## Documentation Updates
+
+### Current Documentation State
+The marketplace includes comprehensive documentation at `/docs` covering:
+- **TFP System**: Detailed explanation of ThreeFold Points (1 TFP = 0.1 USD)
+- **Getting Started**: Wallet setup with TFP terminology
+- **Marketplace Categories**: All sections reference TFP-based transactions
+- **Core Concepts**: TFP as the fundamental value exchange mechanism
+
+### Required Documentation Changes
+
+#### 1. TFP → USD Credits Terminology Update
+**Files to Update:**
+- `/docs/tfp.html` → Rename to `/docs/credits.html`
+- `/docs/index.html` → Update all TFP references
+- `/docs/getting_started.html` → Update wallet setup instructions
+- All marketplace category docs (compute, 3nodes, gateways, applications, services)
+
+#### 2. Key Content Updates
+
+**TFP Documentation Page (`/docs/tfp.html` → `/docs/credits.html`):**
+```html
+<!-- Before -->
+<h1>ThreeFold Points (TFP)</h1>
+<p>TFP maintains a consistent value of 0.1 USD</p>
+
+<!-- After -->
+<h1>USD Credits</h1>
+<p>Credits maintain a 1:1 value with USD ($1 = 1 Credit)</p>
+```
+
+**Getting Started Page:**
+```html
+<!-- Before -->
+<h2>Step 2: Setup Your TFP Wallet</h2>
+<p>You'll need a ThreeFold Points (TFP) wallet</p>
+<li>Add TFP to your wallet through one of the available methods</li>
+
+<!-- After -->
+<h2>Step 2: Setup Your Wallet</h2>
+<p>You'll need to add USD credits to your wallet</p>
+<li>Buy credits through the simple top-up interface</li>
+```
+
+**Main Documentation Index:**
+```html
+<!-- Before -->
+<td>TFP transferred based on resource utilization</td>
+<td>TFP transferred based on hardware value</td>
+
+<!-- After -->
+<td>USD credits charged based on resource utilization</td>
+<td>USD credits charged based on hardware value</td>
+```
+
+#### 3. Navigation Updates
+**Documentation Sidebar (`/docs/layout.html`):**
+```html
+<!-- Before -->
+<a class="nav-link" href="/docs/tfp">
+    <i class="bi bi-currency-exchange me-1"></i>
+    ThreeFold Points (TFP)
+</a>
+
+<!-- After -->
+<a class="nav-link" href="/docs/credits">
+    <i class="bi bi-currency-dollar me-1"></i>
+    USD Credits
+</a>
+```
+
+#### 4. New Documentation Sections
+
+**Auto Top-Up Documentation:**
+```html
+<!-- New section in /docs/credits.html -->
+<div class="doc-section">
+    <h2>Auto Top-Up</h2>
+    <p>Automatically purchase credits when your balance falls below a threshold:</p>
+    <ul>
+        <li>Set your preferred threshold (e.g., $10)</li>
+        <li>Configure top-up amount (e.g., $25)</li>
+        <li>Choose payment method</li>
+        <li>Enable with a simple toggle</li>
+    </ul>
+</div>
+```
+
+**Buy Now Documentation:**
+```html
+<!-- New section in marketplace category docs -->
+<div class="doc-section">
+    <h2>Instant Purchase</h2>
+    <p>Skip the cart and buy instantly:</p>
+    <ol>
+        <li>Click "Buy Now" on any product</li>
+        <li>Auto top-up triggers if balance is low</li>
+        <li>Purchase completes immediately</li>
+        <li>Service deploys automatically</li>
+    </ol>
+</div>
+```
+
+#### 5. Implementation Timeline for Documentation
+
+**Week 3: Documentation Updates (alongside UI/UX updates)**
+- Update all TFP references to USD credits
+- Add auto top-up and buy now documentation
+- Update getting started guide
+- Refresh all code examples and screenshots
+- Update navigation and cross-references
+
+**Week 4: Documentation Testing**
+- Review all documentation for consistency
+- Test all internal links and references
+- Ensure screenshots match new UI
+- Validate code examples work with new system
+
+#### 6. Documentation Migration Strategy
+
+**Content Audit:**
+```bash
+# Search for all TFP references in documentation
+grep -r "TFP\|ThreeFold Points" projectmycelium/src/views/docs/
+
+# Update patterns:
+# "TFP" → "USD credits" or "credits"
+# "ThreeFold Points" → "USD credits"
+# "0.1 USD" → "1 USD"
+# "50 TFP" → "$5.00"
+```
+
+**URL Redirects:**
+```rust
+// Add redirect for old TFP documentation
+.route("/docs/tfp", web::get().to(|| async {
+    HttpResponse::MovedPermanently()
+        .append_header(("Location", "/docs/credits"))
+        .finish()
+}))
+```
+
+This ensures that the documentation accurately reflects the new USD credits system and provides users with up-to-date information about the simplified marketplace experience.