projectmycelium/docs/dev/design/archive/threefold-marketplace-roadmap.md

# Project Mycelium - Complete Vision & Roadmap

**Last Updated:** 2025-08-20 21:25:00 (EST)

**Purpose:** Complete architectural vision, current state, and actionable roadmap for finalizing the Project Mycelium.

## 🎯 Project Status: Ready for Feature Development

**Compilation Status:** ✅ Zero errors achieved - codebase compiles successfully  
**Data Migration:** ✅ Transitioned from mock to persistent data architecture  
**Next Phase:** Runtime testing, feature completion, and production readiness

---

## 1. Architecture Vision (Target State)

### Core Design Principles

#### **Builder Pattern as Single Source of Truth**
- [`SessionDataBuilder`](src/services/session_data.rs), [`ConfigurationBuilder`](src/config/builder.rs), [`ResponseBuilder`](src/utils/response_builder.rs), [`ServiceFactory`](src/services/factory.rs) centralize construction and lifecycles
- All HTTP endpoints return via [`ResponseBuilder`](src/utils/response_builder.rs) with consistent JSON envelopes

#### **ResponseBuilder Envelope & Frontend Contract**
```json
{
  "success": true|false,
  "data": { ... },
  "error"?: { ... }
}
```
- Frontend must always unwrap: `const data = result.data || result;` before accessing fields

#### **CSP-Compliant Frontend: External JS + JSON Hydration**
- **Zero inline scripts/handlers** - code lives in [`src/static/js/*.js`](src/static/js/) and is served under `/static/js/*`
- **Data hydration** via `<script type="application/json" id="...">` blocks
- **Individual field encoding**: Each field is JSON-encoded individually (server-side serializer)
- **Template structure**: Templates expose `{% block scripts %}` and `{% block head %}` at top level only
- **Static mapping**: Actix serves `/static/*` from `./src/static` (see `src/main.rs`)

#### **Persistent-Data-Only Runtime**
- **No mock data** in production code
- **Canonical user data model** persisted per user under [`./user_data/{email_encoded}.json`](user_data/) via [`UserPersistence`](src/services/user_persistence.rs)
- **All operations** (orders, services, wallet, products) read/written through persistence services

#### **Product Model: User-Owned SOT + Derived Catalog**
- **Products owned by provider users** in their persistent files
- **[`ProductService`](src/services/product.rs)** aggregates derived marketplace catalog
- **Category ID normalization** with optional dev TTL cache (disabled by default in prod)

#### **Currency System**
- **USD as base currency** with display currencies: USD/TFC/EUR/CAD (extensible)
- **TFC credits settle** at 1 TFC = 1 USD
- **Server-formatted amounts**: Server returns formatted display amounts and currency code
- **Frontend renders** without recomputing

#### **Unified Insufficient-Balance Contract**
- **Target**: HTTP 402 Payment Required for all insufficient funds cases
- **Canonical error payload** with `error.details` containing currency-aware amounts and deficit

---

## 2. Current Implementation State

### ✅ Completed Foundation
- **CSP externalization** complete across marketplace and dashboard
- **ResponseBuilder integration** applied across all controllers (100% coverage)
- **Orders & invoices** persisted under user data with HTML invoice view
- **Currency system** working with multi-currency display
- **Insufficient-funds responses** unified to HTTP 402 with canonical error envelope
- **Mock data elimination** completed - persistent-only architecture established

### ✅ Core Services Architecture
- **Authentication**: GitEa OAuth integration with user creation
- **Data Persistence**: [`UserPersistence`](src/services/user_persistence.rs) as single source of truth
- **Product Management**: [`ProductService`](src/services/product.rs) with category normalization
- **Order Processing**: Complete order lifecycle with invoice generation
- **Wallet Operations**: Multi-currency support with transaction history

### 🔧 Architecture Patterns Established

#### **Data Flow Pattern**
```rust
// Persistent data access pattern
let persistent_data = UserPersistence::load_user_data(&user_email)?;
// Direct usage of persistent data
persistent_data.wallet_balance_usd
```

#### **Transaction Pattern**
```rust
Transaction {
    id: transaction_id,
    user_id: user_email,
    transaction_type: TransactionType::Purchase { product_id },
    amount: amount_to_add,
    currency: Some("USD".to_string()),
    timestamp: chrono::Utc::now(),
    status: TransactionStatus::Completed,
}
```

#### **ResponseBuilder Pattern**
```rust
ResponseBuilder::success()
    .data(json!({ "products": products }))
    .build()
```

---

## 3. Immediate Development Priorities

### **A. Runtime Testing & Validation (Week 1)**
1. **Critical User Flows**
   - User registration and authentication via GitEa OAuth
   - Product browsing, search, and filtering functionality
   - Wallet operations (top-up, balance checks, transactions)
   - Order placement and payment processing
   - Provider product creation and management

2. **Integration Testing**
   - End-to-end purchase workflows
   - Multi-currency display and calculations
   - Insufficient funds handling (HTTP 402 responses)
   - Invoice generation and viewing

3. **Data Persistence Validation**
   - User data creation, updates, and retrieval
   - Transaction history accuracy
   - Product catalog aggregation performance

### **B. Feature Completion (Week 2)**
1. **Payment Integration**
   - Complete TFC payment flow implementation
   - Payment method management
   - Auto-topup functionality
   - Exchange rate handling

2. **Provider Features**
   - Service provider dashboard completion
   - App provider deployment tools
   - Farmer node management interface
   - Revenue tracking and analytics

3. **Marketplace Features**
   - Advanced search and filtering
   - Product recommendations
   - Category browsing optimization
   - Featured products management

### **C. Security & Production Readiness (Week 3+)**
1. **Security Hardening**
   - CSRF protection implementation
   - Rate limiting configuration
   - Session security hardening
   - Input validation and sanitization

2. **Performance Optimization**
   - Catalog aggregation caching
   - Database query optimization
   - Asset optimization and CDN integration
   - Response time monitoring

3. **Monitoring & Observability**
   - Health check endpoints
   - Metrics collection and alerting
   - Error tracking and logging
   - Performance monitoring

---

## 4. Development Guidelines for AI Coders

### **Code Organization**
- **Controllers**: [`src/controllers/`](src/controllers/) - HTTP request handling with ResponseBuilder
- **Services**: [`src/services/`](src/services/) - Business logic and data operations
- **Models**: [`src/models/`](src/models/) - Data structures and builders
- **Utils**: [`src/utils/`](src/utils/) - Shared utilities and helpers

### **Key Patterns to Follow**
1. **Always use [`ResponseBuilder`](src/utils/response_builder.rs)** for HTTP responses
2. **Persistent data only** - no mock data in production code
3. **CSP compliance** - external JS files only, no inline scripts
4. **Builder patterns** for complex object construction
5. **Error handling** with proper HTTP status codes (especially 402 for insufficient funds)

### **Testing Strategy**
- **Unit tests** for service layer logic
- **Integration tests** for controller endpoints
- **End-to-end tests** for critical user workflows
- **Performance tests** for catalog aggregation and search

### **Configuration Management**
- **Environment-based** configuration via [`ConfigurationBuilder`](src/config/builder.rs)
- **Feature flags** for dev/prod differences
- **Database connection** management
- **External service** integration settings

---

## 5. Success Criteria

### **Functional Requirements**
- [ ] Complete user registration and authentication flow
- [ ] Full product browsing and purchase workflow
- [ ] Working payment processing with TFC integration
- [ ] Provider dashboards for all user types (service, app, farmer)
- [ ] Real-time wallet and transaction management

### **Technical Requirements**
- [ ] Zero compilation errors (✅ Achieved)
- [ ] All tests passing with >90% coverage
- [ ] Performance benchmarks met (sub-second page loads)
- [ ] Security audit passed
- [ ] Production deployment ready

### **User Experience Goals**
- [ ] Intuitive navigation and product discovery
- [ ] Clear pricing and payment flow
- [ ] Responsive design across devices
- [ ] Comprehensive provider management tools
- [ ] Real-time updates and notifications

---

## 6. Architecture Decision Records

### **Data Architecture**
- **Decision**: Persistent file-based user data storage
- **Rationale**: Simplicity, portability, and direct user ownership
- **Implementation**: [`UserPersistence`](src/services/user_persistence.rs) service layer

### **Frontend Architecture**
- **Decision**: CSP-compliant external JS with JSON hydration
- **Rationale**: Security, maintainability, and separation of concerns
- **Implementation**: [`src/static/js/`](src/static/js/) modules with data hydration

### **API Design**
- **Decision**: Consistent JSON envelope via [`ResponseBuilder`](src/utils/response_builder.rs)
- **Rationale**: Predictable frontend integration and error handling
- **Implementation**: All controllers use ResponseBuilder pattern

---

This roadmap provides the complete vision and current state for an AI coder to continue development. The foundation is solid with zero compilation errors, established architectural patterns, and clear next steps for feature completion and production readiness.