hero/interfaces/websocket/server/docs/webhooks.md

# Webhook Integration Architecture

## Overview

This document outlines the architecture for adding webhook handling capabilities to the Circle WebSocket Server. The integration adds HTTP webhook endpoints alongside the existing WebSocket functionality without disrupting the current system.

## Architecture Diagram

```mermaid
graph TB
    subgraph "External Services"
        A[Stripe Webhooks]
        B[iDenfy Webhooks]
    end
    
    subgraph "Circle Server"
        C[HTTP Router]
        D[WebSocket Handler]
        E[Webhook Handler]
        F[Stripe Verifier]
        G[iDenfy Verifier]
        H[Script Dispatcher]
        I[RhaiDispatcherBuilder]
    end
    
    subgraph "Configuration"
        J[.env File]
        K[Environment Variables]
    end
    
    subgraph "Backend"
        L[Redis]
        M[Rhai Worker]
    end
    
    A --> |POST /webhooks/stripe/{circle_pk}| E
    B --> |POST /webhooks/idenfy/{circle_pk}| E
    
    C --> D
    C --> E
    
    E --> F
    E --> G
    F --> H
    G --> H
    H --> I
    I --> L
    L --> M
    
    J --> K
    K --> F
    K --> G
    
    D --> I
```

## URL Structure

### Webhook Endpoints
- **Stripe**: `POST /webhooks/stripe/{circle_pk}`
- **iDenfy**: `POST /webhooks/idenfy/{circle_pk}`

### Existing WebSocket Endpoints (Unchanged)
- **WebSocket**: `GET /{circle_pk}` (upgrades to WebSocket)

## Configuration

### Environment Variables (.env file)
```bash
# Webhook secrets for signature verification
STRIPE_WEBHOOK_SECRET=whsec_...
IDENFY_WEBHOOK_SECRET=your_idenfy_secret

# Existing configuration
REDIS_URL=redis://127.0.0.1/
```

### Server Configuration Updates
```rust
pub struct ServerConfig {
    // ... existing fields
    pub stripe_webhook_secret: Option<String>,
    pub idenfy_webhook_secret: Option<String>,
}
```

## Webhook Processing Flow

### 1. Request Reception
```mermaid
sequenceDiagram
    participant WS as Webhook Service
    participant CS as Circle Server
    participant WV as Webhook Verifier
    participant SD as Script Dispatcher
    participant RC as RhaiDispatcher
    participant RW as Rhai Worker

    WS->>CS: POST /webhooks/stripe/{circle_pk}
    CS->>CS: Extract circle_pk from URL
    CS->>CS: Read request body and headers
    CS->>WV: Verify webhook signature
    
    alt Stripe Webhook
        WV->>WV: Verify Stripe signature using STRIPE_WEBHOOK_SECRET
        WV->>WV: Deserialize to Stripe webhook payload
    else iDenfy Webhook
        WV->>WV: Verify iDenfy signature using IDENFY_WEBHOOK_SECRET
        WV->>WV: Deserialize to iDenfy webhook payload
    end
    
    WV->>CS: Return verification result + parsed payload
    
    alt Verification Success
        CS->>SD: Dispatch appropriate script
        SD->>RC: Create RhaiDispatcherBuilder
        RC->>RC: Set caller_id="stripe" or "idenfy"
        RC->>RC: Set recipient_id=circle_pk
        RC->>RC: Set script="stripe_webhook_received" or "idenfy_webhook_received"
        RC->>RW: Execute via Redis
        RW->>RC: Return result
        RC->>CS: Script execution result
        CS->>WS: HTTP 200 OK
    else Verification Failed
        CS->>WS: HTTP 401 Unauthorized
    end
```

### 2. Signature Verification

#### Stripe Verification
- Uses `Stripe-Signature` header
- HMAC-SHA256 verification with `STRIPE_WEBHOOK_SECRET`
- Follows Stripe's webhook signature verification protocol

#### iDenfy Verification
- Uses appropriate iDenfy signature header
- HMAC verification with `IDENFY_WEBHOOK_SECRET`
- Follows iDenfy's webhook signature verification protocol

### 3. Payload Deserialization

#### Type Definitions in Heromodels Library

Webhook payload types are now defined in the `heromodels` library for better code organization and reusability:

- **Stripe Types**: Located in `heromodels::models::payment::stripe`
- **iDenfy Types**: Located in `heromodels::models::identity::kyc`

#### Stripe Payload Structure
```rust
// From heromodels::models::payment::StripeWebhookEvent
#[derive(Debug, Clone, Deserialize, Serialize)]
pub struct StripeWebhookEvent {
    pub id: String,
    pub object: String,
    pub api_version: Option<String>,
    pub created: i64,
    pub data: StripeEventData,
    pub livemode: bool,
    pub pending_webhooks: i32,
    pub request: Option<StripeEventRequest>,
    #[serde(rename = "type")]
    pub event_type: String,
}
```

#### iDenfy Payload Structure
```rust
// From heromodels::models::identity::IdenfyWebhookEvent
#[derive(Debug, Clone, Deserialize, Serialize)]
pub struct IdenfyWebhookEvent {
    #[serde(rename = "clientId")]
    pub client_id: String,
    #[serde(rename = "scanRef")]
    pub scan_ref: String,
    pub status: String,
    pub platform: String,
    #[serde(rename = "startedAt")]
    pub started_at: String,
    #[serde(rename = "finishedAt")]
    pub finished_at: Option<String>,
    pub data: Option<IdenfyVerificationData>,
    // ... additional fields
}
```

### 4. Script Execution

#### Script Names
- **Stripe**: `stripe_webhook_received`
- **iDenfy**: `idenfy_webhook_received`

#### Script Context
The Rhai scripts will receive structured data:

```javascript
// For Stripe webhooks
let webhook_data = {
    "caller_id": "stripe",
    "circle_id": "circle_public_key",
    "event_type": "payment_intent.succeeded",
    "event_id": "evt_...",
    "created": 1234567890,
    "livemode": false,
    "data": { /* Stripe event data */ }
};

// For iDenfy webhooks  
let webhook_data = {
    "caller_id": "idenfy",
    "circle_id": "circle_public_key", 
    "final_decision": "APPROVED",
    "platform": "PC",
    "status": { /* iDenfy status data */ },
    "data": { /* iDenfy verification data */ }
};
```

## Implementation Structure

### Current File Structure
```
src/server/src/
├── webhook/
│   ├── mod.rs              # Main webhook module with route configuration
│   ├── handlers/
│   │   ├── mod.rs         # Handler module exports
│   │   ├── common.rs      # Common utilities and app state
│   │   ├── stripe.rs      # Stripe webhook handler
│   │   └── idenfy.rs      # iDenfy webhook handler
│   ├── verifiers.rs       # Signature verification for all providers
│   └── types.rs           # Local webhook types (config, errors, etc.)
└── .env                   # Environment configuration
```

### Heromodels Library Structure
```
heromodels/src/models/
├── payment/
│   ├── mod.rs             # Payment module exports
│   └── stripe.rs          # Stripe webhook event types
└── identity/
    ├── mod.rs             # Identity module exports
    └── kyc.rs             # iDenfy KYC webhook event types
```

### Key Architectural Changes
- **Type Organization**: Webhook payload types moved to `heromodels` library for reusability
- **Modular Handlers**: Separate handler files for each webhook provider
- **Simplified Architecture**: Removed unnecessary dispatcher complexity
- **Direct Script Execution**: Handlers directly use `RhaiDispatcher` for script execution

### Modified Files
- `src/lib.rs` - Add webhook routes and module imports
- `Cargo.toml` - Add heromodels dependency and webhook-related dependencies
- `cmd/main.rs` - Load .env file and configure webhook secrets

### Dependencies
```toml
[dependencies]
# Existing dependencies...

# Heromodels library for shared types
heromodels = { path = "../../../db/heromodels" }

# For webhook signature verification
hmac = "0.12"
sha2 = "0.10"
hex = { workspace = true }

# For environment variable loading
dotenv = "0.15"

# For HTTP request handling
bytes = "1.0"
thiserror = { workspace = true }
```

## Security Considerations

### Signature Verification
- **Mandatory**: All webhook requests must have valid signatures
- **Timing Attack Protection**: Use constant-time comparison for signatures
- **Secret Management**: Webhook secrets loaded from environment variables only

### Error Handling
- **No Information Leakage**: Generic error responses for invalid webhooks
- **Logging**: Detailed logging for debugging (same as existing WebSocket errors)
- **Graceful Degradation**: Webhook failures don't affect WebSocket functionality

### Request Validation
- **Content-Type**: Verify appropriate content types
- **Payload Size**: No explicit limits initially (as requested)
- **Rate Limiting**: Consider future implementation

## Backward Compatibility

### WebSocket Functionality
- **Zero Impact**: Existing WebSocket routes and functionality unchanged
- **Authentication**: WebSocket authentication system remains independent
- **Performance**: No performance impact on WebSocket connections

### Configuration
- **Optional**: Webhook functionality only enabled when secrets are configured
- **Graceful Fallback**: Server starts normally even without webhook configuration

## Testing Strategy

### Unit Tests
- Webhook signature verification for both providers
- Payload deserialization
- Error handling scenarios

### Integration Tests
- End-to-end webhook processing
- Script dispatch verification
- Configuration loading

### Mock Testing
- Simulated Stripe webhook calls
- Simulated iDenfy webhook calls
- Invalid signature scenarios

## Deployment Considerations

### Environment Setup
```bash
# .env file in src/server/
STRIPE_WEBHOOK_SECRET=whsec_1234567890abcdef...
IDENFY_WEBHOOK_SECRET=your_idenfy_webhook_secret
REDIS_URL=redis://127.0.0.1/
```

### Server Startup
- Load .env file before server initialization
- Validate webhook secrets if webhook endpoints are to be enabled
- Log webhook endpoint availability

### Monitoring
- Log webhook reception and processing
- Track script execution success/failure rates
- Monitor webhook signature verification failures

## Future Enhancements

### Potential Additions
- Additional webhook providers
- Webhook retry mechanisms
- Webhook event filtering
- Rate limiting implementation
- Webhook event queuing for high-volume scenarios

### Scalability Considerations
- Webhook processing can be made asynchronous if needed
- Multiple server instances can handle webhooks independently
- Redis-based script execution provides natural load distribution