diff --git a/heromodels/src/models/grid4/specs/README.md b/heromodels/src/models/grid4/specs/README.md new file mode 100644 index 0000000..5aa351e --- /dev/null +++ b/heromodels/src/models/grid4/specs/README.md @@ -0,0 +1,194 @@ + +# Grid4 Data Model + +This module defines data models for nodes, groups, and slices in a cloud/grid infrastructure. Each root object is marked with `@[heap]` and can be indexed for efficient querying. + +## Root Objects Overview + +| Object | Description | Index Fields | +| ----------- | --------------------------------------------- | ------------------------------ | +| `Node` | Represents a single node in the grid | `id`, `nodegroupid`, `country` | +| `NodeGroup` | Represents a group of nodes owned by a farmer | `id`, `farmerid` | + +--- + +## Node + +Represents a single node in the grid with slices, devices, and capacity. + +| Field | Type | Description | Indexed | +| --------------- | ---------------- | -------------------------------------------- | ------- | +| `id` | `int` | Unique node ID | ✅ | +| `nodegroupid` | `int` | ID of the owning node group | ✅ | +| `uptime` | `int` | Uptime percentage (0-100) | ✅ | +| `computeslices` | `[]ComputeSlice` | List of compute slices | ❌ | +| `storageslices` | `[]StorageSlice` | List of storage slices | ❌ | +| `devices` | `DeviceInfo` | Hardware device info (storage, memory, etc.) | ❌ | +| `country` | `string` | 2-letter country code | ✅ | +| `capacity` | `NodeCapacity` | Aggregated hardware capacity | ❌ | +| `provisiontime` | `u32` | Provisioning time (simple/compatible format) | ✅ | + +--- + +## NodeGroup + +Represents a group of nodes owned by a farmer, with policies. + +| Field | Type | Description | Indexed | +| ------------------------------------- | --------------- | ---------------------------------------------- | ------- | +| `id` | `u32` | Unique group ID | ✅ | +| `farmerid` | `u32` | Farmer/user ID | ✅ | +| `secret` | `string` | Encrypted secret for booting nodes | ❌ | +| `description` | `string` | Group description | ❌ | +| `slapolicy` | `SLAPolicy` | SLA policy details | ❌ | +| `pricingpolicy` | `PricingPolicy` | Pricing policy details | ❌ | +| `compute_slice_normalized_pricing_cc` | `f64` | Pricing per 2GB compute slice in cloud credits | ❌ | +| `storage_slice_normalized_pricing_cc` | `f64` | Pricing per 1GB storage slice in cloud credits | ❌ | +| `reputation` | `int` | Reputation (0-100) | ✅ | +| `uptime` | `int` | Uptime (0-100) | ✅ | + +--- + +## ComputeSlice + +Represents a compute slice (e.g., 1GB memory unit). + +| Field | Type | Description | +| -------------------------- | --------------- | -------------------------------- | +| `nodeid` | `u32` | Owning node ID | +| `id` | `int` | Slice ID in node | +| `mem_gb` | `f64` | Memory in GB | +| `storage_gb` | `f64` | Storage in GB | +| `passmark` | `int` | Passmark score | +| `vcores` | `int` | Virtual cores | +| `cpu_oversubscription` | `int` | CPU oversubscription ratio | +| `storage_oversubscription` | `int` | Storage oversubscription ratio | +| `price_range` | `[]f64` | Price range [min, max] | +| `gpus` | `u8` | Number of GPUs | +| `price_cc` | `f64` | Price per slice in cloud credits | +| `pricing_policy` | `PricingPolicy` | Pricing policy | +| `sla_policy` | `SLAPolicy` | SLA policy | + +--- + +## StorageSlice + +Represents a 1GB storage slice. + +| Field | Type | Description | +| ---------------- | --------------- | -------------------------------- | +| `nodeid` | `u32` | Owning node ID | +| `id` | `int` | Slice ID in node | +| `price_cc` | `f64` | Price per slice in cloud credits | +| `pricing_policy` | `PricingPolicy` | Pricing policy | +| `sla_policy` | `SLAPolicy` | SLA policy | + +--- + +## DeviceInfo + +Hardware device information for a node. + +| Field | Type | Description | +| --------- | ----------------- | ----------------------- | +| `vendor` | `string` | Vendor of the node | +| `storage` | `[]StorageDevice` | List of storage devices | +| `memory` | `[]MemoryDevice` | List of memory devices | +| `cpu` | `[]CPUDevice` | List of CPU devices | +| `gpu` | `[]GPUDevice` | List of GPU devices | +| `network` | `[]NetworkDevice` | List of network devices | + +--- + +## StorageDevice + +| Field | Type | Description | +| ------------- | -------- | --------------------- | +| `id` | `string` | Unique ID for device | +| `size_gb` | `f64` | Size in GB | +| `description` | `string` | Description of device | + +--- + +## MemoryDevice + +| Field | Type | Description | +| ------------- | -------- | --------------------- | +| `id` | `string` | Unique ID for device | +| `size_gb` | `f64` | Size in GB | +| `description` | `string` | Description of device | + +--- + +## CPUDevice + +| Field | Type | Description | +| ------------- | -------- | ------------------------ | +| `id` | `string` | Unique ID for device | +| `cores` | `int` | Number of CPU cores | +| `passmark` | `int` | Passmark benchmark score | +| `description` | `string` | Description of device | +| `cpu_brand` | `string` | Brand of the CPU | +| `cpu_version` | `string` | Version of the CPU | + +--- + +## GPUDevice + +| Field | Type | Description | +| ------------- | -------- | --------------------- | +| `id` | `string` | Unique ID for device | +| `cores` | `int` | Number of GPU cores | +| `memory_gb` | `f64` | GPU memory in GB | +| `description` | `string` | Description of device | +| `gpu_brand` | `string` | Brand of the GPU | +| `gpu_version` | `string` | Version of the GPU | + +--- + +## NetworkDevice + +| Field | Type | Description | +| ------------- | -------- | --------------------- | +| `id` | `string` | Unique ID for device | +| `speed_mbps` | `int` | Network speed in Mbps | +| `description` | `string` | Description of device | + +--- + +## NodeCapacity + +Aggregated hardware capacity for a node. + +| Field | Type | Description | +| ------------ | ----- | ---------------------- | +| `storage_gb` | `f64` | Total storage in GB | +| `mem_gb` | `f64` | Total memory in GB | +| `mem_gb_gpu` | `f64` | Total GPU memory in GB | +| `passmark` | `int` | Total passmark score | +| `vcores` | `int` | Total virtual cores | + +--- + +## SLAPolicy + +Service Level Agreement policy for slices or node groups. + +| Field | Type | Description | +| -------------------- | ----- | --------------------------------------- | +| `sla_uptime` | `int` | Required uptime % (e.g., 90) | +| `sla_bandwidth_mbit` | `int` | Guaranteed bandwidth in Mbps (0 = none) | +| `sla_penalty` | `int` | Penalty % if SLA is breached (0-100) | + +--- + +## PricingPolicy + +Pricing policy for slices or node groups. + +| Field | Type | Description | +| ---------------------------- | ------- | --------------------------------------------------------- | +| `marketplace_year_discounts` | `[]int` | Discounts for 1Y, 2Y, 3Y prepaid usage (e.g. [30,40,50]) | +| `volume_discounts` | `[]int` | Volume discounts based on purchase size (e.g. [10,20,30]) | + + diff --git a/heromodels/src/models/grid4/specs/model_bid.v b/heromodels/src/models/grid4/specs/model_bid.v new file mode 100644 index 0000000..0ca7b3f --- /dev/null +++ b/heromodels/src/models/grid4/specs/model_bid.v @@ -0,0 +1,37 @@ +module datamodel + +// I can bid for infra, and optionally get accepted +@[heap] +pub struct Bid { +pub mut: + id u32 + customer_id u32 // links back to customer for this capacity (user on ledger) + compute_slices_nr int // nr of slices I need in 1 machine + compute_slice_price f64 // price per 1 GB slice I want to accept + storage_slices_nr int + storage_slice_price f64 // price per 1 GB storage slice I want to accept + storage_slices_nr int + status BidStatus + obligation bool // if obligation then will be charged and money needs to be in escrow, otherwise its an intent + start_date u32 // epoch + end_date u32 + signature_user string // signature as done by a user/consumer to validate their identity and intent + billing_period BillingPeriod +} + +pub enum BidStatus { + pending + confirmed + assigned + cancelled + done +} + + +pub enum BillingPeriod { + hourly + monthly + yearly + biannually + triannually +} diff --git a/heromodels/src/models/grid4/specs/model_contract.v b/heromodels/src/models/grid4/specs/model_contract.v new file mode 100644 index 0000000..f9fc26b --- /dev/null +++ b/heromodels/src/models/grid4/specs/model_contract.v @@ -0,0 +1,52 @@ +module datamodel + +// I can bid for infra, and optionally get accepted +@[heap] +pub struct Contract { +pub mut: + id u32 + customer_id u32 // links back to customer for this capacity (user on ledger) + compute_slices []ComputeSliceProvisioned + storage_slices []StorageSliceProvisioned + compute_slice_price f64 // price per 1 GB agreed upon + storage_slice_price f64 // price per 1 GB agreed upon + network_slice_price f64 // price per 1 GB agreed upon (transfer) + status ContractStatus + start_date u32 // epoch + end_date u32 + signature_user string // signature as done by a user/consumer to validate their identity and intent + signature_hoster string // signature as done by the hoster + billing_period BillingPeriod +} + +pub enum ConctractStatus { + active + cancelled + error + paused +} + + +// typically 1GB of memory, but can be adjusted based based on size of machine +pub struct ComputeSliceProvisioned { +pub mut: + node_id u32 + id u16 // the id of the slice in the node + mem_gb f64 + storage_gb f64 + passmark int + vcores int + cpu_oversubscription int + tags string +} + +// 1GB of storage +pub struct StorageSliceProvisioned { +pub mut: + node_id u32 + id u16 // the id of the slice in the node, are tracked in the node itself + storage_size_gb int + tags string +} + + diff --git a/heromodels/src/models/grid4/specs/model_node.v b/heromodels/src/models/grid4/specs/model_node.v new file mode 100644 index 0000000..b451fa6 --- /dev/null +++ b/heromodels/src/models/grid4/specs/model_node.v @@ -0,0 +1,104 @@ +module datamodel + +//ACCESS ONLY TF + +@[heap] +pub struct Node { +pub mut: + id int + nodegroupid int + uptime int // 0..100 + computeslices []ComputeSlice + storageslices []StorageSlice + devices DeviceInfo + country string // 2 letter code as specified in lib/data/countries/data/countryInfo.txt, use that library for validation + capacity NodeCapacity // Hardware capacity details + birthtime u32 // first time node was active + pubkey string + signature_node string // signature done on node to validate pubkey with privkey + signature_farmer string // signature as done by farmers to validate their identity +} + +pub struct DeviceInfo { +pub mut: + vendor string + storage []StorageDevice + memory []MemoryDevice + cpu []CPUDevice + gpu []GPUDevice + network []NetworkDevice +} + +pub struct StorageDevice { +pub mut: + id string // can be used in node + size_gb f64 // Size of the storage device in gigabytes + description string // Description of the storage device +} + +pub struct MemoryDevice { +pub mut: + id string // can be used in node + size_gb f64 // Size of the memory device in gigabytes + description string // Description of the memory device +} + +pub struct CPUDevice { +pub mut: + id string // can be used in node + cores int // Number of CPU cores + passmark int + description string // Description of the CPU + cpu_brand string // Brand of the CPU + cpu_version string // Version of the CPU +} + +pub struct GPUDevice { +pub mut: + id string // can be used in node + cores int // Number of GPU cores + memory_gb f64 // Size of the GPU memory in gigabytes + description string // Description of the GPU + gpu_brand string + gpu_version string +} + +pub struct NetworkDevice { +pub mut: + id string // can be used in node + speed_mbps int // Network speed in Mbps + description string // Description of the network device +} + +// NodeCapacity represents the hardware capacity details of a node. +pub struct NodeCapacity { +pub mut: + storage_gb f64 // Total storage in gigabytes + mem_gb f64 // Total memory in gigabytes + mem_gb_gpu f64 // Total GPU memory in gigabytes + passmark int // Passmark score for the node + vcores int // Total virtual cores +} + +// typically 1GB of memory, but can be adjusted based based on size of machine +pub struct ComputeSlice { +pub mut: + u16 int // the id of the slice in the node + mem_gb f64 + storage_gb f64 + passmark int + vcores int + cpu_oversubscription int + storage_oversubscription int + gpus u8 // nr of GPU's see node to know what GPU's are +} + +// 1GB of storage +pub struct StorageSlice { +pub mut: + u16 int // the id of the slice in the node, are tracked in the node itself +} + +fn (mut n Node) check() ! { + // todo calculate NodeCapacity out of the devices on the Node +} diff --git a/heromodels/src/models/grid4/specs/model_nodegroup.v b/heromodels/src/models/grid4/specs/model_nodegroup.v new file mode 100644 index 0000000..ae4858b --- /dev/null +++ b/heromodels/src/models/grid4/specs/model_nodegroup.v @@ -0,0 +1,33 @@ +module datamodel + +// is a root object, is the only obj farmer needs to configure in the UI, this defines how slices will be created +@[heap] +pub struct NodeGroup { +pub mut: + id u32 + farmerid u32 // link back to farmer who owns the nodegroup, is a user? + secret string // only visible by farmer, in future encrypted, used to boot a node + description string + slapolicy SLAPolicy + pricingpolicy PricingPolicy + compute_slice_normalized_pricing_cc f64 // pricing in CC - cloud credit, per 2GB node slice + storage_slice_normalized_pricing_cc f64 // pricing in CC - cloud credit, per 1GB storage slice + signature_farmer string // signature as done by farmers to validate that they created this group +} + +pub struct SLAPolicy { +pub mut: + sla_uptime int // should +90 + sla_bandwidth_mbit int // minimal mbits we can expect avg over 1h per node, 0 means we don't guarantee + sla_penalty int // 0-100, percent of money given back in relation to month if sla breached, e.g. 200 means we return 2 months worth of rev if sla missed +} + +pub struct PricingPolicy { +pub mut: + marketplace_year_discounts []int = [30, 40, 50] // e.g. 30,40,50 means if user has more CC in wallet than 1 year utilization on all his purchaes then this provider gives 30%, 2Y 40%, ... + // volume_discounts []int = [10, 20, 30] // e.g. 10,20,30 +} + + + + diff --git a/heromodels/src/models/grid4/specs/model_reputation.v b/heromodels/src/models/grid4/specs/model_reputation.v new file mode 100644 index 0000000..0d65749 --- /dev/null +++ b/heromodels/src/models/grid4/specs/model_reputation.v @@ -0,0 +1,19 @@ + +@[heap] +pub struct NodeGroupReputation { +pub mut: + nodegroup_id u32 + reputation int = 50 // between 0 and 100, earned over time + uptime int // between 0 and 100, set by system, farmer has no ability to set this + nodes []NodeReputation +} + +pub struct NodeReputation { +pub mut: + node_id u32 + reputation int = 50 // between 0 and 100, earned over time + uptime int // between 0 and 100, set by system, farmer has no ability to set this +} + + + diff --git a/specs/billingmanager_research/billingmanager.md b/specs/billingmanager_research/billingmanager.md new file mode 100644 index 0000000..5e81d97 --- /dev/null +++ b/specs/billingmanager_research/billingmanager.md @@ -0,0 +1,345 @@ + +### 2.1 Accounts + +* **id**: `BIGINT` identity (non-negative), unique account id +* **pubkey**: `BYTEA` unique public key for signing/encryption +* **display\_name**: `TEXT` (optional) +* **created\_at**: `TIMESTAMPTZ` + +### 2.2 Currencies + +* **asset\_code**: `TEXT` PK (e.g., `USDC-ETH`, `EUR`, `LND`) +* **name**: `TEXT` +* **symbol**: `TEXT` +* **decimals**: `INT` (default 2) + +--- + +## 3) Services & Groups + +### 3.1 Services + +* **id**: `BIGINT` identity +* **name**: `TEXT` unique +* **description**: `TEXT` +* **default\_billing\_mode**: `ENUM('per_second','per_request')` +* **default\_price**: `NUMERIC(38,18)` (≥0) +* **default\_currency**: FK → `currencies(asset_code)` +* **max\_request\_seconds**: `INT` (>0 or `NULL`) +* **schema\_heroscript**: `TEXT` +* **schema\_json**: `JSONB` +* **created\_at**: `TIMESTAMPTZ` + +#### Accepted Currencies (per service) + +* **service\_id**: FK → `services(id)` +* **asset\_code**: FK → `currencies(asset_code)` +* **price\_override**: `NUMERIC(38,18)` (optional) +* **billing\_mode\_override**: `ENUM` (optional) + Primary key: `(service_id, asset_code)` + +### 3.2 Service Groups + +* **id**: `BIGINT` identity +* **name**: `TEXT` unique +* **description**: `TEXT` +* **created\_at**: `TIMESTAMPTZ` + +#### Group Memberships + +* **group\_id**: FK → `service_groups(id)` +* **service\_id**: FK → `services(id)` + Primary key: `(group_id, service_id)` + +--- + +## 4) Providers & Runners + +### 4.1 Service Providers + +* **id**: `BIGINT` identity +* **account\_id**: FK → `accounts(id)` (the owning account) +* **name**: `TEXT` unique +* **description**: `TEXT` +* **created\_at**: `TIMESTAMPTZ` + +#### Providers Offer Groups + +* **provider\_id**: FK → `service_providers(id)` +* **group\_id**: FK → `service_groups(id)` + Primary key: `(provider_id, group_id)` + +#### Provider Pricing Overrides (optional) + +* **provider\_id**: FK → `service_providers(id)` +* **service\_id**: FK → `services(id)` +* **asset\_code**: FK → `currencies(asset_code)` (nullable for currency-agnostic override) +* **price\_override**: `NUMERIC(38,18)` (optional) +* **billing\_mode\_override**: `ENUM` (optional) +* **max\_request\_seconds\_override**: `INT` (optional) + Primary key: `(provider_id, service_id, asset_code)` + +### 4.2 Runners + +* **id**: `BIGINT` identity +* **address**: `INET` (must be IPv6) +* **name**: `TEXT` +* **description**: `TEXT` +* **pubkey**: `BYTEA` (optional) +* **created\_at**: `TIMESTAMPTZ` + +#### Runner Ownership (many-to-many) + +* **runner\_id**: FK → `runners(id)` +* **provider\_id**: FK → `service_providers(id)` + Primary key: `(runner_id, provider_id)` + +#### Routing (provider → service/service\_group → runners) + +* **provider\_service\_runners**: `(provider_id, service_id, runner_id)` PK +* **provider\_service\_group\_runners**: `(provider_id, group_id, runner_id)` PK + +--- + +## 5) Subscriptions & Spend Control + +A subscription authorizes an **account** to use either a **service** **or** a **service group**, with optional spend limits and allowed providers. + +* **id**: `BIGINT` identity +* **account\_id**: FK → `accounts(id)` +* **service\_id** *xor* **group\_id**: FK (exactly one must be set) +* **secret**: `BYTEA` (random, provided by subscriber; recommend storing a hash) +* **subscription\_data**: `JSONB` (free-form) +* **limit\_amount**: `NUMERIC(38,18)` (optional) +* **limit\_currency**: FK → `currencies(asset_code)` (optional) +* **limit\_period**: `ENUM('hour','day','month')` (optional) +* **active**: `BOOLEAN` default `TRUE` +* **created\_at**: `TIMESTAMPTZ` + +#### Allowed Providers per Subscription + +* **subscription\_id**: FK → `subscriptions(id)` +* **provider\_id**: FK → `service_providers(id)` + Primary key: `(subscription_id, provider_id)` + +**Intended Use:** + +* Subscribers bound spending by amount/currency/period. +* Merchant (provider) can claim charges for requests fulfilled under an active subscription, within limits, and only if listed in `subscription_providers`. + +--- + +## 6) Requests & Billing + +### 6.1 Request Lifecycle + +* **id**: `BIGINT` identity +* **account\_id**: FK → `accounts(id)` +* **subscription\_id**: FK → `subscriptions(id)` +* **provider\_id**: FK → `service_providers(id)` +* **service\_id**: FK → `services(id)` +* **runner\_id**: FK → `runners(id)` (nullable) +* **request\_schema**: `JSONB` (payload matching `schema_json`/`schema_heroscript`) +* **started\_at**, **ended\_at**: `TIMESTAMPTZ` +* **status**: `ENUM('pending','running','succeeded','failed','canceled')` +* **created\_at**: `TIMESTAMPTZ` + +### 6.2 Billing Ledger (append-only) + +* **id**: `BIGINT` identity +* **account\_id**: FK → `accounts(id)` +* **provider\_id**: FK → `service_providers(id)` (nullable) +* **service\_id**: FK → `services(id)` (nullable) +* **request\_id**: FK → `requests(id)` (nullable) +* **amount**: `NUMERIC(38,18)` (debit = positive, credit/refund = negative) +* **asset\_code**: FK → `currencies(asset_code)` +* **entry\_type**: `ENUM('debit','credit','adjustment')` +* **description**: `TEXT` +* **created\_at**: `TIMESTAMPTZ` + +**Balances View (example):** + +* `account_balances(account_id, asset_code, balance)` as a view over `billing_ledger`. + +--- + +## 7) Pricing Precedence + +When computing the **effective** pricing, billing mode, and max duration for a `(provider, service, currency)`: + +1. **Provider override for (service, asset\_code)** — if present, use it. +2. **Service accepted currency override** — if present, use it. +3. **Service defaults** — fallback. + +If `billing_mode` or `max_request_seconds` are not overridden at steps (1) or (2), inherit from the next step down. + +--- + +## 8) Key Constraints & Validations + +* All identity ids are non-negative (`CHECK (id >= 0)`). +* Runner IPv6 enforcement: `CHECK (family(address) = 6)`. +* Subscriptions must point to **exactly one** of `service_id` or `group_id`. +* Prices and limits must be non-negative if set. +* Unique natural keys where appropriate: service names, provider names, currency asset codes, account pubkeys. + +--- + +## 9) Mermaid Diagrams + +### 9.1 Entity–Relationship Overview + +```mermaid +erDiagram + ACCOUNTS ||--o{ SERVICE_PROVIDERS : "owns via account_id" + ACCOUNTS ||--o{ SUBSCRIPTIONS : has + CURRENCIES ||--o{ SERVICES : "default_currency" + CURRENCIES ||--o{ SERVICE_ACCEPTED_CURRENCIES : "asset_code" + CURRENCIES ||--o{ PROVIDER_SERVICE_OVERRIDES : "asset_code" + CURRENCIES ||--o{ BILLING_LEDGER : "asset_code" + + SERVICES ||--o{ SERVICE_ACCEPTED_CURRENCIES : has + SERVICES ||--o{ SERVICE_GROUP_MEMBERS : member_of + SERVICE_GROUPS ||--o{ SERVICE_GROUP_MEMBERS : contains + + SERVICE_PROVIDERS ||--o{ PROVIDER_SERVICE_GROUPS : offers + SERVICE_PROVIDERS ||--o{ PROVIDER_SERVICE_OVERRIDES : sets + SERVICE_PROVIDERS ||--o{ RUNNER_OWNERS : owns + SERVICE_PROVIDERS ||--o{ PROVIDER_SERVICE_RUNNERS : routes + SERVICE_PROVIDERS ||--o{ PROVIDER_SERVICE_GROUP_RUNNERS : routes + + RUNNERS ||--o{ RUNNER_OWNERS : owned_by + RUNNERS ||--o{ PROVIDER_SERVICE_RUNNERS : executes + RUNNERS ||--o{ PROVIDER_SERVICE_GROUP_RUNNERS : executes + + SUBSCRIPTIONS ||--o{ SUBSCRIPTION_PROVIDERS : allow + SERVICE_PROVIDERS ||--o{ SUBSCRIPTION_PROVIDERS : allowed + + REQUESTS }o--|| ACCOUNTS : by + REQUESTS }o--|| SUBSCRIPTIONS : under + REQUESTS }o--|| SERVICE_PROVIDERS : via + REQUESTS }o--|| SERVICES : for + REQUESTS }o--o{ RUNNERS : executed_by + + BILLING_LEDGER }o--|| ACCOUNTS : charges + BILLING_LEDGER }o--o{ SERVICES : reference + BILLING_LEDGER }o--o{ SERVICE_PROVIDERS : reference + BILLING_LEDGER }o--o{ REQUESTS : reference +``` + +### 9.2 Request Flow (Happy Path) + +```mermaid +sequenceDiagram + autonumber + participant AC as Account + participant API as Broker/API + participant PR as Provider + participant RU as Runner + participant DB as PostgreSQL + + AC->>API: Submit request (subscription_id, service_id, payload, secret) + API->>DB: Validate subscription (active, provider allowed, spend limits) + DB-->>API: OK + effective pricing (resolve precedence) + API->>PR: Dispatch request (service, payload) + PR->>DB: Select runner (provider_service_runners / group runners) + PR->>RU: Start job (payload) + RU-->>PR: Job started (started_at) + PR->>DB: Update REQUESTS (status=running, started_at) + RU-->>PR: Job finished (duration, result) + PR->>DB: Update REQUESTS (status=succeeded, ended_at) + API->>DB: Insert BILLING_LEDGER (debit per effective price) + DB-->>API: Ledger entry id + API-->>AC: Return result + charge info +``` + +### 9.3 Pricing Resolution + +```mermaid +flowchart TD + A[Input: provider_id, service_id, asset_code] --> B{Provider override exists for (service, asset_code)?} + B -- Yes --> P1[Use provider price/mode/max] + B -- No --> C{Service accepted currency override exists?} + C -- Yes --> P2[Use service currency price/mode] + C -- No --> P3[Use service defaults] + P1 --> OUT[Effective pricing] + P2 --> OUT + P3 --> OUT +``` + +--- + +## 10) Operational Notes + +* **Secrets:** store a hash (e.g., `digest(secret,'sha256')`) rather than raw `secret`. Keep the original only client-side. +* **Limits enforcement:** before insert of a debit ledger entry, compute period window (hour/day/month UTC or tenant TZ) and enforce `SUM(amount) + new_amount ≤ limit_amount`. +* **Durations:** enforce `max_request_seconds` (effective) at orchestration and/or via DB trigger on `REQUESTS` when transitioning to `running/succeeded`. +* **Routing:** prefer `provider_service_runners` when a request targets a service directly; otherwise use the union of runners from `provider_service_group_runners` for the group. +* **Balances:** serve balance queries via the `account_balances` view or a materialized cache updated by triggers/jobs. + +--- + +## 11) Example Effective Pricing Query (sketch) + +```sql +-- Inputs: :provider_id, :service_id, :asset_code +WITH p AS ( + SELECT price_override, billing_mode_override, max_request_seconds_override + FROM provider_service_overrides + WHERE provider_id = :provider_id + AND service_id = :service_id + AND (asset_code = :asset_code) +), +sac AS ( + SELECT price_override, billing_mode_override + FROM service_accepted_currencies + WHERE service_id = :service_id AND asset_code = :asset_code +), +svc AS ( + SELECT default_price AS price, default_billing_mode AS mode, max_request_seconds + FROM services WHERE id = :service_id +) +SELECT + COALESCE(p.price_override, sac.price_override, svc.price) AS effective_price, + COALESCE(p.billing_mode_override, sac.billing_mode_override, svc.mode) AS effective_mode, + COALESCE(p.max_request_seconds_override, svc.max_request_seconds) AS effective_max_seconds; +``` + +--- + +## 12) Indices (non-exhaustive) + +* `services(default_currency)` +* `service_accepted_currencies(service_id)` +* `provider_service_overrides(service_id, provider_id)` +* `requests(account_id)`, `requests(provider_id)`, `requests(service_id)` +* `billing_ledger(account_id, asset_code)` +* `subscriptions(account_id) WHERE active` + +--- + +## 13) Migration & Compatibility + +* Prefer additive migrations (new columns/tables) to avoid downtime. +* Use `ENUM` via `CREATE TYPE`; when extending, plan for `ALTER TYPE ... ADD VALUE`. +* For high-write ledgers, consider partitioning `billing_ledger` by `created_at` (monthly) and indexing partitions. + +--- + +## 14) Non-Goals + +* Wallet custody and on-chain settlement are out of scope. +* SLA tracking and detailed observability (metrics/log schema) are not part of this spec. + +--- + +## 15) Acceptance Criteria + +* Can represent services, groups, and providers with currency-specific pricing. +* Can route requests to runners by service or group. +* Can authorize usage via subscriptions, enforce spend limits, and record charges. +* Can reconstruct balances and audit via append-only ledger. + +--- + +**End of Spec** diff --git a/specs/billingmanager_research/conceptnote.md b/specs/billingmanager_research/conceptnote.md new file mode 100644 index 0000000..9743f3b --- /dev/null +++ b/specs/billingmanager_research/conceptnote.md @@ -0,0 +1,225 @@ + +# Concept Note: Generic Billing & Tracking Framework + +## 1) Purpose + +The model is designed to support a **flexible, generic, and auditable** billing environment that can be applied across diverse services and providers — from compute time billing to per-request API usage, across multiple currencies, with dynamic provider-specific overrides. + +It is **not tied to a single business domain** — the same framework can be used for: + +* Cloud compute time (per second) +* API transactions (per request) +* Data transfer charges +* Managed service subscriptions +* Brokered third-party service reselling + +--- + +## 2) Key Concepts + +### 2.1 Accounts + +An **account** represents an economic actor in the system — typically a customer or a service provider. + +* Identified by a **public key** (for authentication & cryptographic signing). +* Every billing action traces back to an account. + +--- + +### 2.2 Currencies & Asset Codes + +The system supports **multiple currencies** (crypto or fiat) via **asset codes**. + +* Asset codes identify the unit of billing (e.g. `USDC-ETH`, `EUR`, `LND`). +* Currencies are **decoupled from services** so you can add or remove supported assets at any time. + +--- + +### 2.3 Services & Groups + +* **Service** = a billable offering (e.g., "Speech-to-Text", "VM Hosting"). + + * Has a **billing mode** (`per_second` or `per_request`). + * Has a **default price** and **default currency**. + * Supports **multiple accepted currencies** with optional per-currency pricing overrides. + * Has execution constraints (e.g. `max_request_seconds`). + * Includes structured schemas for request payloads. + +* **Service Group** = a logical grouping of services. + + * Groups make it easy to **bundle related services** and manage them together. + * Providers can offer entire groups rather than individual services. + +--- + +### 2.4 Service Providers + +A **service provider** is an **account** that offers services or service groups. +They can: + +* Override **pricing** for their offered services (per currency). +* Route requests to their own **runners** (execution agents). +* Manage multiple **service groups** under one provider identity. + +--- + +### 2.5 Runners + +A **runner** is an execution agent — a node, VM, or service endpoint that can fulfill requests. + +* Identified by an **IPv6 address** (supports Mycelium or other overlay networks). +* Can be owned by one or multiple providers. +* Providers map **services/groups → runners** to define routing. + +--- + +### 2.6 Subscriptions + +A **subscription** is **the authorization mechanism** for usage and spending control: + +* Links an **account** to a **service** or **service group**. +* Defines **spending limits** (amount, currency, period: hour/day/month). +* Restricts which **providers** are allowed to serve the subscription. +* Uses a **secret** chosen by the subscriber — providers use this to claim charges. + +--- + +### 2.7 Requests + +A **request** represents a single execution under a subscription: + +* Tied to **account**, **subscription**, **provider**, **service**, and optionally **runner**. +* Has **status** (`pending`, `running`, `succeeded`, `failed`, `canceled`). +* Records start/end times for duration-based billing. + +--- + +### 2.8 Billing Ledger + +The **ledger** is **append-only** — the source of truth for all charges and credits. + +* Each entry records: + + * `amount` (positive = debit, negative = credit/refund) + * `asset_code` + * Links to `account`, `provider`, `service`, and/or `request` +* From the ledger, **balances** can be reconstructed at any time. + +--- + +## 3) How Billing Works — Step by Step + +### 3.1 Setup + +1. **Define services** with default pricing & schemas. +2. **Define currencies** and accepted currencies for services. +3. **Group services** into service groups. +4. **Onboard providers** (accounts) and associate them with service groups. +5. **Assign runners** to services or groups for execution routing. + +--- + +### 3.2 Subscription Creation + +1. Customer **creates a subscription**: + + * Chooses service or service group. + * Sets **spending limit** (amount, currency, period). + * Chooses **secret**. + * Selects **allowed providers**. +2. Subscription is stored in DB. + +--- + +### 3.3 Request Execution + +1. Customer sends a request to broker/API with: + + * `subscription_id` + * Target `service_id` + * Payload + signature using account pubkey. +2. Broker: + + * Validates **subscription active**. + * Validates **provider allowed**. + * Checks **spend limit** hasn’t been exceeded for current period. + * Resolves **effective price** via: + + 1. Provider override (currency-specific) + 2. Service accepted currency override + 3. Service default +3. Broker selects **runner** from provider’s routing tables. +4. Runner executes request and returns result. + +--- + +### 3.4 Billing Entry + +1. When the request completes: + + * If `per_second` mode → calculate `duration × rate`. + * If `per_request` mode → apply flat rate. +2. Broker **inserts ledger entry**: + + * Debit from customer account. + * Credit to provider account (can be separate entries or aggregated). +3. Ledger is append-only — historical billing cannot be altered. + +--- + +### 3.5 Balance & Tracking + +* **Current balances** are a sum of all ledger entries per account+currency. +* Spend limits are enforced by **querying the ledger** for the current period before each charge. +* Audit trails are guaranteed via immutable ledger entries. + +--- + +## 4) Why This is Generic & Reusable + +This design **decouples**: + +* **Service definition** from **provider pricing** → multiple providers can sell the same service at different rates. +* **Execution agents** (runners) from **service definitions** → easy scaling or outsourcing of execution. +* **Billing rules** (per-second vs per-request) from **subscription limits** → same service can be sold in different billing modes. +* **Currencies** from the service → enabling multi-asset billing without changing the service definition. + +Because of these separations, you can: + +* Reuse the model for **compute**, **APIs**, **storage**, **SaaS features**, etc. +* Plug in different **payment backends** (on-chain, centralized payment processor, prepaid balance). +* Use the same model for **internal cost allocation** or **external customer billing**. + +--- + +## 5) Potential Extensions + +* **Prepaid model**: enforce that ledger debits can’t exceed balance. +* **On-chain settlement**: periodically export ledger entries to blockchain transactions. +* **Discount models**: percentage or fixed-amount discounts per subscription. +* **Usage analytics**: aggregate requests/billing by time period, provider, or service. +* **SLAs**: link billing adjustments to performance metrics in requests. + +--- + +## 6) Conceptual Diagram — Billing Flow + +```mermaid +sequenceDiagram + participant C as Customer Account + participant B as Broker/API + participant P as Provider + participant R as Runner + participant DB as Ledger DB + + C->>B: Request(service, subscription, payload, secret) + B->>DB: Validate subscription & spend limit + DB-->>B: OK + effective pricing + B->>P: Forward request + P->>R: Execute request + R-->>P: Result + execution time + P->>B: Return result + B->>DB: Insert debit (customer) + credit (provider) + DB-->>B: Ledger updated + B-->>C: Return result + charge info +``` diff --git a/specs/billingmanager_research/schema.sql b/specs/billingmanager_research/schema.sql new file mode 100644 index 0000000..6573553 --- /dev/null +++ b/specs/billingmanager_research/schema.sql @@ -0,0 +1,234 @@ +-- Enable useful extensions (optional) +CREATE EXTENSION IF NOT EXISTS pgcrypto; -- for digests/hashes if you want +CREATE EXTENSION IF NOT EXISTS btree_gist; -- for exclusion/partial indexes + +-- ========================= +-- Core: Accounts & Currency +-- ========================= + +CREATE TABLE accounts ( + id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY, + pubkey BYTEA NOT NULL UNIQUE, + display_name TEXT, + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + CHECK (id >= 0) +); + +CREATE TABLE currencies ( + asset_code TEXT PRIMARY KEY, -- e.g. "USDC-ETH", "EUR", "LND" + name TEXT NOT NULL, + symbol TEXT, -- e.g. "$", "€" + decimals INT NOT NULL DEFAULT 2, -- how many decimal places + UNIQUE (name) +); + +-- ========================= +-- Services & Groups +-- ========================= + +CREATE TYPE billing_mode AS ENUM ('per_second', 'per_request'); + +CREATE TABLE services ( + id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY, + name TEXT NOT NULL UNIQUE, + description TEXT, + default_billing_mode billing_mode NOT NULL, + default_price NUMERIC(38, 18) NOT NULL, -- default price in "unit currency" (see accepted currencies) + default_currency TEXT NOT NULL REFERENCES currencies(asset_code) ON UPDATE CASCADE, + max_request_seconds INTEGER, -- nullable means no cap + schema_heroscript TEXT, + schema_json JSONB, + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + CHECK (id >= 0), + CHECK (default_price >= 0), + CHECK (max_request_seconds IS NULL OR max_request_seconds > 0) +); + +-- Accepted currencies for a service (subset + optional specific price per currency) +CREATE TABLE service_accepted_currencies ( + service_id BIGINT NOT NULL REFERENCES services(id) ON DELETE CASCADE, + asset_code TEXT NOT NULL REFERENCES currencies(asset_code) ON UPDATE CASCADE, + price_override NUMERIC(38, 18), -- if set, overrides default_price for this currency + billing_mode_override billing_mode, -- if set, overrides default_billing_mode + PRIMARY KEY (service_id, asset_code), + CHECK (price_override IS NULL OR price_override >= 0) +); + +CREATE TABLE service_groups ( + id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY, + name TEXT NOT NULL UNIQUE, + description TEXT, + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + CHECK (id >= 0) +); + +CREATE TABLE service_group_members ( + group_id BIGINT NOT NULL REFERENCES service_groups(id) ON DELETE CASCADE, + service_id BIGINT NOT NULL REFERENCES services(id) ON DELETE RESTRICT, + PRIMARY KEY (group_id, service_id) +); + +-- ========================= +-- Providers, Runners, Routing +-- ========================= + +CREATE TABLE service_providers ( + id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY, + account_id BIGINT NOT NULL REFERENCES accounts(id) ON DELETE CASCADE, -- provider is an account + name TEXT NOT NULL, + description TEXT, + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + UNIQUE (name), + CHECK (id >= 0) +); + +-- Providers can offer groups (which imply their services) +CREATE TABLE provider_service_groups ( + provider_id BIGINT NOT NULL REFERENCES service_providers(id) ON DELETE CASCADE, + group_id BIGINT NOT NULL REFERENCES service_groups(id) ON DELETE CASCADE, + PRIMARY KEY (provider_id, group_id) +); + +-- Providers may set per-service overrides (price/mode/max seconds) (optionally per currency) +CREATE TABLE provider_service_overrides ( + provider_id BIGINT NOT NULL REFERENCES service_providers(id) ON DELETE CASCADE, + service_id BIGINT NOT NULL REFERENCES services(id) ON DELETE CASCADE, + asset_code TEXT REFERENCES currencies(asset_code) ON UPDATE CASCADE, + price_override NUMERIC(38, 18), + billing_mode_override billing_mode, + max_request_seconds_override INTEGER, + PRIMARY KEY (provider_id, service_id, asset_code), + CHECK (price_override IS NULL OR price_override >= 0), + CHECK (max_request_seconds_override IS NULL OR max_request_seconds_override > 0) +); + +-- Runners +CREATE TABLE runners ( + id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY, + address INET NOT NULL, -- IPv6 (INET supports both IPv4/IPv6; require v6 via CHECK below if you like) + name TEXT NOT NULL, + description TEXT, + pubkey BYTEA, -- optional + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + UNIQUE (address), + CHECK (id >= 0), + CHECK (family(address) = 6) -- ensure IPv6 +); + +-- Runner ownership: a runner can be owned by multiple providers +CREATE TABLE runner_owners ( + runner_id BIGINT NOT NULL REFERENCES runners(id) ON DELETE CASCADE, + provider_id BIGINT NOT NULL REFERENCES service_providers(id) ON DELETE CASCADE, + PRIMARY KEY (runner_id, provider_id) +); + +-- Routing: link providers' services to specific runners +CREATE TABLE provider_service_runners ( + provider_id BIGINT NOT NULL REFERENCES service_providers(id) ON DELETE CASCADE, + service_id BIGINT NOT NULL REFERENCES services(id) ON DELETE CASCADE, + runner_id BIGINT NOT NULL REFERENCES runners(id) ON DELETE CASCADE, + PRIMARY KEY (provider_id, service_id, runner_id) +); + +-- Routing: link providers' service groups to runners +CREATE TABLE provider_service_group_runners ( + provider_id BIGINT NOT NULL REFERENCES service_providers(id) ON DELETE CASCADE, + group_id BIGINT NOT NULL REFERENCES service_groups(id) ON DELETE CASCADE, + runner_id BIGINT NOT NULL REFERENCES runners(id) ON DELETE CASCADE, + PRIMARY KEY (provider_id, group_id, runner_id) +); + +-- ========================= +-- Subscriptions & Spend Control +-- ========================= + +CREATE TYPE spend_period AS ENUM ('hour', 'day', 'month'); + +-- A subscription ties an account to a specific service OR a service group, with spend limits and allowed providers +CREATE TABLE subscriptions ( + id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY, + account_id BIGINT NOT NULL REFERENCES accounts(id) ON DELETE CASCADE, + service_id BIGINT REFERENCES services(id) ON DELETE CASCADE, + group_id BIGINT REFERENCES service_groups(id) ON DELETE CASCADE, + secret BYTEA NOT NULL, -- caller-chosen secret (consider storing a hash instead) + subscription_data JSONB, -- arbitrary client-supplied info + limit_amount NUMERIC(38, 18), -- allowed spend in the selected currency per period + limit_currency TEXT REFERENCES currencies(asset_code) ON UPDATE CASCADE, + limit_period spend_period, -- period for the limit + active BOOLEAN NOT NULL DEFAULT TRUE, + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + -- Ensure exactly one of service_id or group_id + CHECK ( (service_id IS NOT NULL) <> (group_id IS NOT NULL) ), + CHECK (limit_amount IS NULL OR limit_amount >= 0), + CHECK (id >= 0) +); + +-- Providers that are allowed to serve under a subscription +CREATE TABLE subscription_providers ( + subscription_id BIGINT NOT NULL REFERENCES subscriptions(id) ON DELETE CASCADE, + provider_id BIGINT NOT NULL REFERENCES service_providers(id) ON DELETE CASCADE, + PRIMARY KEY (subscription_id, provider_id) +); + +-- ========================= +-- Usage, Requests & Billing +-- ========================= + +-- A request lifecycle record (optional but useful for auditing and max duration enforcement) +CREATE TYPE request_status AS ENUM ('pending', 'running', 'succeeded', 'failed', 'canceled'); + +CREATE TABLE requests ( + id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY, + account_id BIGINT NOT NULL REFERENCES accounts(id) ON DELETE CASCADE, + subscription_id BIGINT NOT NULL REFERENCES subscriptions(id) ON DELETE RESTRICT, + provider_id BIGINT NOT NULL REFERENCES service_providers(id) ON DELETE RESTRICT, + service_id BIGINT NOT NULL REFERENCES services(id) ON DELETE RESTRICT, + runner_id BIGINT REFERENCES runners(id) ON DELETE SET NULL, + request_schema JSONB, -- concrete task payload (conforms to schema_json/heroscript) + started_at TIMESTAMPTZ, + ended_at TIMESTAMPTZ, + status request_status NOT NULL DEFAULT 'pending', + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + CHECK (id >= 0), + CHECK (ended_at IS NULL OR started_at IS NULL OR ended_at >= started_at) +); + +-- Billing ledger (debits/credits). Positive amount = debit to account (charge). Negative = credit/refund. +CREATE TYPE ledger_entry_type AS ENUM ('debit', 'credit', 'adjustment'); + +CREATE TABLE billing_ledger ( + id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY, + account_id BIGINT NOT NULL REFERENCES accounts(id) ON DELETE CASCADE, + provider_id BIGINT REFERENCES service_providers(id) ON DELETE SET NULL, + service_id BIGINT REFERENCES services(id) ON DELETE SET NULL, + request_id BIGINT REFERENCES requests(id) ON DELETE SET NULL, + amount NUMERIC(38, 18) NOT NULL, -- positive for debit, negative for credit + asset_code TEXT NOT NULL REFERENCES currencies(asset_code) ON UPDATE CASCADE, + entry_type ledger_entry_type NOT NULL, + description TEXT, + created_at TIMESTAMPTZ NOT NULL DEFAULT now(), + CHECK (id >= 0) +); + +-- Optional: running balances per account/currency (materialized view or real-time view) +-- This is a plain view; for performance, you might maintain a cached table. +CREATE VIEW account_balances AS +SELECT + account_id, + asset_code, + SUM(amount) AS balance +FROM billing_ledger +GROUP BY account_id, asset_code; + +-- ========================= +-- Helpful Indexes +-- ========================= + +CREATE INDEX idx_services_default_currency ON services(default_currency); +CREATE INDEX idx_service_accepted_currencies_service ON service_accepted_currencies(service_id); +CREATE INDEX idx_provider_overrides_service ON provider_service_overrides(service_id); +CREATE INDEX idx_requests_account ON requests(account_id); +CREATE INDEX idx_requests_provider ON requests(provider_id); +CREATE INDEX idx_requests_service ON requests(service_id); +CREATE INDEX idx_billing_account_currency ON billing_ledger(account_id, asset_code); +CREATE INDEX idx_subscriptions_account_active ON subscriptions(account_id) WHERE active; diff --git a/specs/billingmanager_research/summary.md b/specs/billingmanager_research/summary.md new file mode 100644 index 0000000..fd78def --- /dev/null +++ b/specs/billingmanager_research/summary.md @@ -0,0 +1,266 @@ +# Billing Logic — Whiteboard Version (for Devs) + +## 1) Inputs You Always Need + +* `account_id`, `subscription_id` +* `service_id` (or group → resolved to a service at dispatch) +* `provider_id`, `asset_code` +* `payload` (validated against service schema) +* (Optional) `runner_id` +* Idempotency key for the request (client-provided) + +--- + +## 2) Gatekeeping (Hard Checks) + +1. **Subscription** + +* Must be `active`. +* Must target **exactly one** of {service, group}. +* If group: ensure `service_id` is a member. + +2. **Provider Allowlist** + +* If `subscription_providers` exists → `provider_id` must be listed. + +3. **Spend Limit** (if set) + +* Compute window by `limit_period` (`hour`/`day`/`month`, UTC unless tenant TZ). +* Current period spend = `SUM(ledger.amount WHERE account & currency & period)`. +* `current_spend + estimated_charge ≤ limit_amount`. + +4. **Max Duration** (effective; see §3): + +* If billing mode is `per_second`, reject if requested/max exceeds effective cap. + +--- + +## 3) Effective Pricing (Single Resolution Function) + +Inputs: `provider_id`, `service_id`, `asset_code` + +Precedence: + +1. `provider_service_overrides` for `(service_id, asset_code)` +2. `service_accepted_currencies` for `(service_id, asset_code)` +3. `services` defaults + +Outputs: + +* `effective_billing_mode ∈ {per_request, per_second}` +* `effective_price` (NUMERIC) +* `effective_max_request_seconds` (nullable) + +--- + +## 4) Request Lifecycle (States) + +* `pending` → `running` → (`succeeded` | `failed` | `canceled`) +* Timestamps: set `started_at` on `running`, `ended_at` on terminal states. +* Enforce `ended_at ≥ started_at` and `duration ≤ effective_max_request_seconds` (if set). + +--- + +## 5) Charging Rules + +### A) Per Request + +``` +charge = effective_price +``` + +### B) Per Second + +``` +duration_seconds = ceil(extract(epoch from (ended_at - started_at))) +charge = duration_seconds * effective_price +``` + +* Cap with `effective_max_request_seconds` if present. +* If ended early/failed before `started_at`: charge = 0. + +--- + +## 6) Idempotency & Atomicity + +* **Idempotency key** per `(account_id, subscription_id, provider_id, service_id, request_external_id)`; store on `requests` and enforce unique index. +* **Single transaction** to: + + 1. finalize `REQUESTS` status + timestamps, + 2. insert **one** debit entry into `billing_ledger`. +* Never mutate ledger entries; use compensating **credit** entries for adjustments/refunds. + +--- + +## 7) Spend-Limit Enforcement (Before Charging) + +Pseudocode (SQL-ish): + +```sql +WITH window AS ( + SELECT tsrange(period_start(:limit_period), period_end(:limit_period)) AS w +), +spent AS ( + SELECT COALESCE(SUM(amount), 0) AS total + FROM billing_ledger, window + WHERE account_id = :account_id + AND asset_code = :asset_code + AND created_at <@ (SELECT w FROM window) +), +check AS ( + SELECT (spent.total + :estimated_charge) <= :limit_amount AS ok FROM spent +) +SELECT ok FROM check; +``` + +* If not ok → reject before dispatch, or allow but **set hard cap** on max seconds and auto-stop at limit. + +--- + +## 8) Suggested DB Operations (Happy Path) + +1. **Create request** + +```sql +INSERT INTO requests (...) +VALUES (...) +ON CONFLICT (idempotency_key) DO NOTHING +RETURNING id; +``` + +2. **Start execution** + +```sql +UPDATE requests +SET status='running', started_at=now() +WHERE id=:id AND status='pending'; +``` + +3. **Finish & bill** (single transaction) + +```sql +BEGIN; + +-- lock for update to avoid double-billing +UPDATE requests +SET status=:final_status, ended_at=now() +WHERE id=:id AND status='running' +RETURNING started_at, ended_at; + +-- compute charge in app (see §5), re-check spend window here + +INSERT INTO billing_ledger ( + account_id, provider_id, service_id, request_id, + amount, asset_code, entry_type, description +) VALUES ( + :account_id, :provider_id, :service_id, :id, + :charge, :asset_code, 'debit', :desc +); + +COMMIT; +``` + +--- + +## 9) Balances & Reporting + +* **Current balance** = `SUM(billing_ledger.amount) GROUP BY account_id, asset_code`. +* Keep a **view** or **materialized view**; refresh asynchronously if needed. +* Never rely on cached balance for hard checks — re-check within the billing transaction if **prepaid** semantics are required. + +--- + +## 10) Error & Edge Rules + +* If runner fails before `running` → no charge. +* If runner starts, then fails: + + * **per\_second**: bill actual seconds (can be 0). + * **per\_request**: default is **no charge** unless policy says otherwise; if charging partials, document it. +* Partial refunds/adjustments → insert **negative** ledger entries (type `credit`/`adjustment`) tied to the original `request_id`. + +--- + +## 11) Minimal Pricing Resolver (Sketch) + +```sql +WITH p AS ( + SELECT price_override AS price, + billing_mode_override AS mode, + max_request_seconds_override AS maxsec + FROM provider_service_overrides + WHERE provider_id = :pid AND service_id = :sid AND asset_code = :asset + LIMIT 1 +), +sac AS ( + SELECT price_override AS price, + billing_mode_override AS mode + FROM service_accepted_currencies + WHERE service_id = :sid AND asset_code = :asset + LIMIT 1 +), +svc AS ( + SELECT default_price AS price, + default_billing_mode AS mode, + max_request_seconds AS maxsec + FROM services WHERE id = :sid +) +SELECT + COALESCE(p.price, sac.price, svc.price) AS price, + COALESCE(p.mode, sac.mode, svc.mode) AS mode, + COALESCE(p.maxsec, svc.maxsec) AS max_seconds; +``` + +--- + +## 12) Mermaid — Decision Trees + +### Pricing & Duration + +```mermaid +flowchart TD + A[provider_id, service_id, asset_code] --> B{Provider override exists?} + B -- yes --> P[Use provider price/mode/max] + B -- no --> C{Service currency override?} + C -- yes --> S[Use service currency price/mode] + C -- no --> D[Use service defaults] + P --> OUT[effective price/mode/max] + S --> OUT + D --> OUT +``` + +### Spend Check & Charge + +```mermaid +flowchart TD + S[Has subscription limit?] -->|No| D1[Dispatch] + S -->|Yes| C{current_spend + est_charge <= limit?} + C -->|No| REJ[Reject or cap duration] + C -->|Yes| D1[Dispatch] + D1 --> RUN[Run request] + RUN --> DONE[Finalize + insert ledger] +``` + +--- + +## 13) Security Posture + +* Store **hash of subscription secret**; compare hash on use. +* Sign client requests with **account pubkey**; verify before dispatch. +* Limit **request schema** to validated fields; reject unknowns. +* Enforce **IPv6** for runners where required. + +--- + +## 14) What To Implement First + +1. Pricing resolver (single function). +2. Spend-window checker (single query). +3. Request lifecycle + idempotency. +4. Ledger write (append-only) + balances view. + +Everything else layers on top. + +--- + +If you want, I can turn this into a small **README.md** with code blocks you can paste into the repo (plus a couple of SQL functions and example tests). diff --git a/specs/models_marketplace/main/currency.v b/specs/models_marketplace/main/currency.v index 1c54e89..f285bbd 100644 --- a/specs/models_marketplace/main/currency.v +++ b/specs/models_marketplace/main/currency.v @@ -24,16 +24,6 @@ pub enum CurrencyType { custom } -pub struct Price { -pub mut: - base_amount f64 // Using f64 for Decimal - base_currency string - display_currency string - display_amount f64 // Using f64 for Decimal - formatted_display string - conversion_rate f64 // Using f64 for Decimal - conversion_timestamp u64 // Unix timestamp -} pub struct MarketplaceCurrencyConfig { pub mut: diff --git a/specs/models/biz/company.v b/specs/models_old/biz/company.v similarity index 100% rename from specs/models/biz/company.v rename to specs/models_old/biz/company.v diff --git a/specs/models/biz/payment.v b/specs/models_old/biz/payment.v similarity index 100% rename from specs/models/biz/payment.v rename to specs/models_old/biz/payment.v diff --git a/specs/models/biz/product.v b/specs/models_old/biz/product.v similarity index 100% rename from specs/models/biz/product.v rename to specs/models_old/biz/product.v diff --git a/specs/models/biz/sale.v b/specs/models_old/biz/sale.v similarity index 100% rename from specs/models/biz/sale.v rename to specs/models_old/biz/sale.v diff --git a/specs/models/biz/shareholder.v b/specs/models_old/biz/shareholder.v similarity index 100% rename from specs/models/biz/shareholder.v rename to specs/models_old/biz/shareholder.v diff --git a/specs/models/calendar/calendar.v b/specs/models_old/calendar/calendar.v similarity index 100% rename from specs/models/calendar/calendar.v rename to specs/models_old/calendar/calendar.v diff --git a/specs/models/calendar/contacts.v b/specs/models_old/calendar/contacts.v similarity index 100% rename from specs/models/calendar/contacts.v rename to specs/models_old/calendar/contacts.v diff --git a/specs/models/calendar/event.v b/specs/models_old/calendar/event.v similarity index 100% rename from specs/models/calendar/event.v rename to specs/models_old/calendar/event.v diff --git a/specs/models/calendar/message.v b/specs/models_old/calendar/message.v similarity index 100% rename from specs/models/calendar/message.v rename to specs/models_old/calendar/message.v diff --git a/specs/models/circle/circle.v b/specs/models_old/circle/circle.v similarity index 100% rename from specs/models/circle/circle.v rename to specs/models_old/circle/circle.v diff --git a/specs/models/circle/member.v b/specs/models_old/circle/member.v similarity index 100% rename from specs/models/circle/member.v rename to specs/models_old/circle/member.v diff --git a/specs/models/circle/name.v b/specs/models_old/circle/name.v similarity index 100% rename from specs/models/circle/name.v rename to specs/models_old/circle/name.v diff --git a/specs/models/circle/wallet.v b/specs/models_old/circle/wallet.v similarity index 100% rename from specs/models/circle/wallet.v rename to specs/models_old/circle/wallet.v diff --git a/specs/models/core/base.v b/specs/models_old/core/base.v similarity index 100% rename from specs/models/core/base.v rename to specs/models_old/core/base.v diff --git a/specs/models/core/comment.v b/specs/models_old/core/comment.v similarity index 100% rename from specs/models/core/comment.v rename to specs/models_old/core/comment.v diff --git a/specs/models/finance/account.v b/specs/models_old/finance/account.v similarity index 100% rename from specs/models/finance/account.v rename to specs/models_old/finance/account.v diff --git a/specs/models/finance/asset.v b/specs/models_old/finance/asset.v similarity index 100% rename from specs/models/finance/asset.v rename to specs/models_old/finance/asset.v diff --git a/specs/models/finance/marketplace.v b/specs/models_old/finance/marketplace.v similarity index 100% rename from specs/models/finance/marketplace.v rename to specs/models_old/finance/marketplace.v diff --git a/specs/models/gov/committee.v b/specs/models_old/gov/committee.v similarity index 100% rename from specs/models/gov/committee.v rename to specs/models_old/gov/committee.v diff --git a/specs/models/gov/company.v b/specs/models_old/gov/company.v similarity index 100% rename from specs/models/gov/company.v rename to specs/models_old/gov/company.v diff --git a/specs/models/gov/meeting.v b/specs/models_old/gov/meeting.v similarity index 100% rename from specs/models/gov/meeting.v rename to specs/models_old/gov/meeting.v diff --git a/specs/models/gov/resolution.v b/specs/models_old/gov/resolution.v similarity index 100% rename from specs/models/gov/resolution.v rename to specs/models_old/gov/resolution.v diff --git a/specs/models/gov/shareholder.v b/specs/models_old/gov/shareholder.v similarity index 100% rename from specs/models/gov/shareholder.v rename to specs/models_old/gov/shareholder.v diff --git a/specs/models/gov/types.v b/specs/models_old/gov/types.v similarity index 100% rename from specs/models/gov/types.v rename to specs/models_old/gov/types.v diff --git a/specs/models/gov/user.v b/specs/models_old/gov/user.v similarity index 100% rename from specs/models/gov/user.v rename to specs/models_old/gov/user.v diff --git a/specs/models/gov/vote.v b/specs/models_old/gov/vote.v similarity index 100% rename from specs/models/gov/vote.v rename to specs/models_old/gov/vote.v diff --git a/specs/models/governance/activity.v b/specs/models_old/governance/activity.v similarity index 100% rename from specs/models/governance/activity.v rename to specs/models_old/governance/activity.v diff --git a/specs/models/governance/attached_file.v b/specs/models_old/governance/attached_file.v similarity index 100% rename from specs/models/governance/attached_file.v rename to specs/models_old/governance/attached_file.v diff --git a/specs/models/governance/committee.v b/specs/models_old/governance/committee.v similarity index 100% rename from specs/models/governance/committee.v rename to specs/models_old/governance/committee.v diff --git a/specs/models/governance/company.v b/specs/models_old/governance/company.v similarity index 100% rename from specs/models/governance/company.v rename to specs/models_old/governance/company.v diff --git a/specs/models/governance/meeting.v b/specs/models_old/governance/meeting.v similarity index 100% rename from specs/models/governance/meeting.v rename to specs/models_old/governance/meeting.v diff --git a/specs/models/governance/proposal.v b/specs/models_old/governance/proposal.v similarity index 100% rename from specs/models/governance/proposal.v rename to specs/models_old/governance/proposal.v diff --git a/specs/models/governance/resolution.v b/specs/models_old/governance/resolution.v similarity index 100% rename from specs/models/governance/resolution.v rename to specs/models_old/governance/resolution.v diff --git a/specs/models/governance/user.v b/specs/models_old/governance/user.v similarity index 100% rename from specs/models/governance/user.v rename to specs/models_old/governance/user.v diff --git a/specs/models/governance/vote.v b/specs/models_old/governance/vote.v similarity index 100% rename from specs/models/governance/vote.v rename to specs/models_old/governance/vote.v diff --git a/specs/models/legal/contract.v b/specs/models_old/legal/contract.v similarity index 100% rename from specs/models/legal/contract.v rename to specs/models_old/legal/contract.v diff --git a/specs/models/library/book.v b/specs/models_old/library/book.v similarity index 100% rename from specs/models/library/book.v rename to specs/models_old/library/book.v diff --git a/specs/models/library/collection.v b/specs/models_old/library/collection.v similarity index 100% rename from specs/models/library/collection.v rename to specs/models_old/library/collection.v diff --git a/specs/models/library/image.v b/specs/models_old/library/image.v similarity index 100% rename from specs/models/library/image.v rename to specs/models_old/library/image.v diff --git a/specs/models/library/markdown.v b/specs/models_old/library/markdown.v similarity index 100% rename from specs/models/library/markdown.v rename to specs/models_old/library/markdown.v diff --git a/specs/models/library/pdf.v b/specs/models_old/library/pdf.v similarity index 100% rename from specs/models/library/pdf.v rename to specs/models_old/library/pdf.v diff --git a/specs/models/library/slideshow.v b/specs/models_old/library/slideshow.v similarity index 100% rename from specs/models/library/slideshow.v rename to specs/models_old/library/slideshow.v diff --git a/specs/models/projects/base.v b/specs/models_old/projects/base.v similarity index 100% rename from specs/models/projects/base.v rename to specs/models_old/projects/base.v diff --git a/specs/models/projects/epic.v b/specs/models_old/projects/epic.v similarity index 100% rename from specs/models/projects/epic.v rename to specs/models_old/projects/epic.v diff --git a/specs/models/projects/sprint.v b/specs/models_old/projects/sprint.v similarity index 100% rename from specs/models/projects/sprint.v rename to specs/models_old/projects/sprint.v diff --git a/specs/models/projects/task.v b/specs/models_old/projects/task.v similarity index 100% rename from specs/models/projects/task.v rename to specs/models_old/projects/task.v diff --git a/specs/modelsold/base/base.v b/specs/models_older/base/base.v similarity index 100% rename from specs/modelsold/base/base.v rename to specs/models_older/base/base.v diff --git a/specs/modelsold/biz/company.v b/specs/models_older/biz/company.v similarity index 100% rename from specs/modelsold/biz/company.v rename to specs/models_older/biz/company.v diff --git a/specs/modelsold/biz/product.v b/specs/models_older/biz/product.v similarity index 100% rename from specs/modelsold/biz/product.v rename to specs/models_older/biz/product.v diff --git a/specs/modelsold/biz/sale.v b/specs/models_older/biz/sale.v similarity index 100% rename from specs/modelsold/biz/sale.v rename to specs/models_older/biz/sale.v diff --git a/specs/modelsold/biz/shareholder.v b/specs/models_older/biz/shareholder.v similarity index 100% rename from specs/modelsold/biz/shareholder.v rename to specs/models_older/biz/shareholder.v diff --git a/specs/modelsold/biz/user.v b/specs/models_older/biz/user.v similarity index 100% rename from specs/modelsold/biz/user.v rename to specs/models_older/biz/user.v diff --git a/specs/modelsold/circle/attachment.v b/specs/models_older/circle/attachment.v similarity index 100% rename from specs/modelsold/circle/attachment.v rename to specs/models_older/circle/attachment.v diff --git a/specs/modelsold/circle/config.v b/specs/models_older/circle/config.v similarity index 100% rename from specs/modelsold/circle/config.v rename to specs/models_older/circle/config.v diff --git a/specs/modelsold/circle/domainnames.v b/specs/models_older/circle/domainnames.v similarity index 100% rename from specs/modelsold/circle/domainnames.v rename to specs/models_older/circle/domainnames.v diff --git a/specs/modelsold/circle/group.v b/specs/models_older/circle/group.v similarity index 100% rename from specs/modelsold/circle/group.v rename to specs/models_older/circle/group.v diff --git a/specs/modelsold/circle/user.v b/specs/models_older/circle/user.v similarity index 100% rename from specs/modelsold/circle/user.v rename to specs/models_older/circle/user.v diff --git a/specs/modelsold/crm/account.v b/specs/models_older/crm/account.v similarity index 100% rename from specs/modelsold/crm/account.v rename to specs/models_older/crm/account.v diff --git a/specs/modelsold/crm/call.v b/specs/models_older/crm/call.v similarity index 100% rename from specs/modelsold/crm/call.v rename to specs/models_older/crm/call.v diff --git a/specs/modelsold/crm/campaign.v b/specs/models_older/crm/campaign.v similarity index 100% rename from specs/modelsold/crm/campaign.v rename to specs/models_older/crm/campaign.v diff --git a/specs/modelsold/crm/case.v b/specs/models_older/crm/case.v similarity index 100% rename from specs/modelsold/crm/case.v rename to specs/models_older/crm/case.v diff --git a/specs/modelsold/crm/contact.v b/specs/models_older/crm/contact.v similarity index 100% rename from specs/modelsold/crm/contact.v rename to specs/models_older/crm/contact.v diff --git a/specs/modelsold/crm/lead.v b/specs/models_older/crm/lead.v similarity index 100% rename from specs/modelsold/crm/lead.v rename to specs/models_older/crm/lead.v diff --git a/specs/modelsold/crm/opportunity.v b/specs/models_older/crm/opportunity.v similarity index 100% rename from specs/modelsold/crm/opportunity.v rename to specs/models_older/crm/opportunity.v diff --git a/specs/modelsold/crm/task.v b/specs/models_older/crm/task.v similarity index 100% rename from specs/modelsold/crm/task.v rename to specs/models_older/crm/task.v diff --git a/specs/modelsold/finance/account.v b/specs/models_older/finance/account.v similarity index 100% rename from specs/modelsold/finance/account.v rename to specs/models_older/finance/account.v diff --git a/specs/modelsold/finance/asset.v b/specs/models_older/finance/asset.v similarity index 100% rename from specs/modelsold/finance/asset.v rename to specs/models_older/finance/asset.v diff --git a/specs/modelsold/finance/marketplace.v b/specs/models_older/finance/marketplace.v similarity index 100% rename from specs/modelsold/finance/marketplace.v rename to specs/models_older/finance/marketplace.v diff --git a/specs/modelsold/flow/flow.v b/specs/models_older/flow/flow.v similarity index 100% rename from specs/modelsold/flow/flow.v rename to specs/models_older/flow/flow.v diff --git a/specs/modelsold/governance/proposal.v b/specs/models_older/governance/proposal.v similarity index 100% rename from specs/modelsold/governance/proposal.v rename to specs/models_older/governance/proposal.v diff --git a/specs/modelsold/legal/contract.v b/specs/models_older/legal/contract.v similarity index 100% rename from specs/modelsold/legal/contract.v rename to specs/models_older/legal/contract.v diff --git a/specs/modelsold/mcc/README.md b/specs/models_older/mcc/README.md similarity index 100% rename from specs/modelsold/mcc/README.md rename to specs/models_older/mcc/README.md diff --git a/specs/modelsold/mcc/calendar.v b/specs/models_older/mcc/calendar.v similarity index 100% rename from specs/modelsold/mcc/calendar.v rename to specs/models_older/mcc/calendar.v diff --git a/specs/modelsold/mcc/contacts.v b/specs/models_older/mcc/contacts.v similarity index 100% rename from specs/modelsold/mcc/contacts.v rename to specs/models_older/mcc/contacts.v diff --git a/specs/modelsold/mcc/message.v b/specs/models_older/mcc/message.v similarity index 100% rename from specs/modelsold/mcc/message.v rename to specs/models_older/mcc/message.v diff --git a/specs/modelsold/projects/base.v b/specs/models_older/projects/base.v similarity index 100% rename from specs/modelsold/projects/base.v rename to specs/models_older/projects/base.v diff --git a/specs/modelsold/projects/epic.v b/specs/models_older/projects/epic.v similarity index 100% rename from specs/modelsold/projects/epic.v rename to specs/models_older/projects/epic.v diff --git a/specs/modelsold/projects/issue.v b/specs/models_older/projects/issue.v similarity index 100% rename from specs/modelsold/projects/issue.v rename to specs/models_older/projects/issue.v diff --git a/specs/modelsold/projects/kanban.v b/specs/models_older/projects/kanban.v similarity index 100% rename from specs/modelsold/projects/kanban.v rename to specs/models_older/projects/kanban.v diff --git a/specs/modelsold/projects/sprint.v b/specs/models_older/projects/sprint.v similarity index 100% rename from specs/modelsold/projects/sprint.v rename to specs/models_older/projects/sprint.v diff --git a/specs/modelsold/projects/story.v b/specs/models_older/projects/story.v similarity index 100% rename from specs/modelsold/projects/story.v rename to specs/models_older/projects/story.v diff --git a/specs/modelsold/ticket/ticket.v b/specs/models_older/ticket/ticket.v similarity index 100% rename from specs/modelsold/ticket/ticket.v rename to specs/models_older/ticket/ticket.v diff --git a/specs/modelsold/ticket/ticket_comment.v b/specs/models_older/ticket/ticket_comment.v similarity index 100% rename from specs/modelsold/ticket/ticket_comment.v rename to specs/models_older/ticket/ticket_comment.v diff --git a/specs/modelsold/ticket/ticket_enums.v b/specs/models_older/ticket/ticket_enums.v similarity index 100% rename from specs/modelsold/ticket/ticket_enums.v rename to specs/models_older/ticket/ticket_enums.v diff --git a/specs/modelsold/user.v b/specs/models_older/user.v similarity index 100% rename from specs/modelsold/user.v rename to specs/models_older/user.v diff --git a/specs/models_threefold/aiinstruct.md b/specs/models_threefold_old/aiinstruct.md similarity index 100% rename from specs/models_threefold/aiinstruct.md rename to specs/models_threefold_old/aiinstruct.md diff --git a/specs/models_threefold/core b/specs/models_threefold_old/core similarity index 100% rename from specs/models_threefold/core rename to specs/models_threefold_old/core diff --git a/specs/models_threefold/main/secretbox.v b/specs/models_threefold_old/main/secretbox.v similarity index 100% rename from specs/models_threefold/main/secretbox.v rename to specs/models_threefold_old/main/secretbox.v diff --git a/specs/models_threefold/main/signature.v b/specs/models_threefold_old/main/signature.v similarity index 100% rename from specs/models_threefold/main/signature.v rename to specs/models_threefold_old/main/signature.v diff --git a/specs/models_threefold/main/user.v b/specs/models_threefold_old/main/user.v similarity index 100% rename from specs/models_threefold/main/user.v rename to specs/models_threefold_old/main/user.v diff --git a/specs/models_threefold/main/user_kvs.v b/specs/models_threefold_old/main/user_kvs.v similarity index 100% rename from specs/models_threefold/main/user_kvs.v rename to specs/models_threefold_old/main/user_kvs.v