No description
  • Go 97.4%
  • Makefile 1.4%
  • Python 0.9%
  • Dockerfile 0.3%
Find a file
thabeta 8db9ef4694
All checks were successful
Test / test (push) Successful in 1m52s
Build / build (push) Successful in 10m16s
Lint / lint (push) Successful in 10m35s
Merge pull request 'add e2e testing' (#28) from master_e2e into main
Reviewed-on: #28
2026-02-24 09:37:51 +00:00
.forgejo/workflows refactor: fix lint sqlite3 type error 2026-02-24 10:40:04 +02:00
caddy refactor: remove unneeded matcher and var 2026-02-16 09:36:50 +02:00
cmds refactor: refactor E2E testing setup to use go instead of bash script 2026-02-24 10:28:55 +02:00
dev refactor: split backend services into separate TLS and no-TLS instances with isolated mycelium networks 2026-02-24 10:28:55 +02:00
docs update module name 2026-01-28 16:08:00 +02:00
e2e docs: remove /etc/hosts requirement and update test command in E2E README 2026-02-24 10:28:55 +02:00
grafana refactor: simplify docker networking to be bridged 2026-02-12 14:54:14 +02:00
internal Merge pull request 'use sqllite error ErrConstraintUnique' (#26) from fix-error-handling into master 2026-02-23 13:55:23 +00:00
pkg/crypto refactor: refactor E2E testing setup to use go instead of bash script 2026-02-24 10:28:55 +02:00
prometheus refactor: replace grafana-dashboard service with preconfigured image and use host networking 2026-02-11 13:05:01 +02:00
step support local development 2026-01-27 16:44:03 +02:00
.gitignore refactor: replace grafana-dashboard service with preconfigured image and use host networking 2026-02-11 13:05:01 +02:00
.golangci.yml add build, test and lint workflows (#13) 2026-02-10 13:17:49 +00:00
caddy.dev.json refactor use ready caddy middleware and implement matcher 2026-02-15 16:33:17 +02:00
caddy.json refactor: replace grafana-dashboard service with preconfigured image and use host networking 2026-02-11 13:05:01 +02:00
docker-compose.dev.yml refactor: simplify docker networking to be bridged 2026-02-12 14:54:14 +02:00
docker-compose.yml refactor: simplify docker networking to be bridged 2026-02-12 14:54:14 +02:00
Dockerfile feat: add docker compose setup with prometheus and grafana monitoring 2026-02-11 08:35:47 +02:00
go.mod refactor: add build tags for e2e tests and implement HTTP retry logic with certificate bundling 2026-02-24 10:28:55 +02:00
go.sum refactor: add build tags for e2e tests and implement HTTP retry logic with certificate bundling 2026-02-24 10:28:55 +02:00
Makefile refactor: add build tags for e2e tests and implement HTTP retry logic with certificate bundling 2026-02-24 10:28:55 +02:00
README.md remove outdated docs 2026-02-15 13:00:36 +02:00

webgateway

Secure Layer 7 reverse proxy for bridging public internet traffic to private Mycelium backends.

Quick Start

Prerequisites

  • Go 1.21+
  • Mycelium daemon running (see Mycelium repo)
  • Domain name with DNS access
  • Public IP address

DNS Setup

Before running the gateway, you must configure 3 DNS records for your domain:

  1. Domain A Record: Point your domain to the gateway's public IP

    Domain    A    <public_ip>
    
  2. Wildcard CNAME Record: Enable subdomain routing

    *.Domain    CNAME    Domain
    
  3. ACME Challenge NS Record: Allow automatic SSL certificate generation

    _acme-challenge.Domain    NS    Domain
    

Example (for domain gateway.example.com with IP 1.2.3.4):

gateway.example.com          A     1.2.3.4
*.gateway.example.com        CNAME gateway.example.com
_acme-challenge.gateway.example.com  NS  gateway.example.com

Configuration

Create or edit caddy.json with the following required fields:

{
  "admin": {
    "config": {
      "persist": true,
      "load": {
        "module": "mycelium",
        "base_domain": "gateway.example.com",
        "public_ip": "1.2.3.4",
        "admin_id": "admin",
        "services_storage": {
          "load": {
            "module": "sqlite",
            "db_path": "gateway.db"
          }
        },
        "pub_key_store": {
          "load": {
            "module": "directory",
            "path": "./test_keys"
          }
        }
      }
    }
  }
}

Configuration Fields:

  • base_domain: Your domain name (e.g., gateway.example.com)
  • public_ip: The public IP address of your gateway server
  • admin_id: Administrator identifier for gateway operations (default: admin.pub)
  • services_storage: Storage module configuration for route persistence (required)
    • module: Storage module type (e.g., sqlite)
    • db_path: Path to the SQLite database file for route storage
  • pub_key_store: Public key store module configuration for HTTP Signature verification (required)
    • module: Key store module type (e.g., directory)
    • path: Path to the directory containing public keys

Optional Configuration Fields:

  • http_port: HTTP port for internal routing (default: 80)
  • https_port: HTTPS port for Layer4 listener (default: 443)
  • load_delay: Delay before loading config (default: 60s)
  • log_level: Logging level (default: DEBUG)
  • local_acme: URL of the local ACME CA directory endpoint (optional)
    • When set, uses a local ACME server (e.g., step-ca) instead of Let's Encrypt
    • Example: https://localhost:9000/acme/acme/directory
  • local_acme_root: Path to the local ACME CA root certificate file (optional)
    • When set, adds the root certificate to trusted roots for the local CA
    • Recommended when using local_acme to ensure certificates are trusted
    • Example: ./step/certs/root_ca.crt

Running the Gateway

# Start Mycelium daemon (sidecar)
mycelium --peers tcp://188.40.132.242:9651 quic://185.69.166.8:9651

# Build gateway
make build

# Run gateway
./bin/webgateway run --config caddy.json

The gateway will:

  • Listen on ports 80 (HTTP), 443 (HTTPS), and 53 (DNS)
  • Automatically generate SSL certificates:
    • Production mode (default): Via ACME with DNS-01 challenge (Let's Encrypt)
    • Local dev mode: When local_acme is set, uses a local ACME server (e.g., step-ca) with DNS-01 challenge
  • Dynamically generate Caddy configuration based on routes stored in services_storage
  • Route traffic to Mycelium backends based on configured routes
  • Handle TLS passthrough for services that require it
  • Filter out routes from suspended users automatically

Note: The gateway uses a ConfigLoader module that dynamically generates the Caddy configuration from routes stored in the database. Configuration is regenerated when routes are added, updated, or deleted, and suspended users' routes are automatically excluded.

Docker Deployment

Prerequisites

  • Docker Engine 20.10+
  • Docker Compose V2 (or docker-compose V1)

Production Deployment

The production setup includes:

  • webgateway - The main gateway service
  • prometheus - Metrics collection
  • grafana - Dashboard and monitoring
# Build and start all services
make docker-up

# View logs
docker compose logs -f webgateway

# Stop and remove volumes (data loss!)
make docker-down

Production Configuration:

  • caddy.json - Main configuration
  • data/ - Persistent SQLite database (auto-created)
  • prometheus/prometheus.yml - Prometheus config
  • grafana/ - Dashboard and datasource provisioning

Accessing Services

Local Development

See dev/README.md for additional local development setup instructions.

Route Configuration

Routes are managed using the gwctl CLI tool. All mutation operations require HTTP Signature authentication using your private key.

Routing Types

The gateway supports two types of routing:

  1. Regular HTTPS Routes (default): The gateway terminates TLS and proxies HTTP requests to backends

    • TLS termination at the gateway
    • Backends receive plain HTTP traffic
    • Supports load balancing (round_robin, random)
  2. TLS Passthrough Routes: The gateway proxies TLS traffic directly to backends without termination

    • Backend terminates TLS
    • Requires FQDN (not subdomain)
    • Useful for services that need end-to-end TLS or custom certificates

Installing gwctl

# Build gwctl
cd cmds/gwctl
go build -o gwctl
# Or install globally
go install

Creating a Route

# Create a route with FQDN and single backend
gwctl routes create \
  -i user1.pub \
  -k test_keys/user1.priv \
  -f myapp.gateway.example.com \
  -b [54f:b680:ba6e:7ced:355f:346f:d97b:eecb]:8080

# Create a route with subdomain (alternative to FQDN)
gwctl routes create \
  -i user1.pub \
  -k test_keys/user1.priv \
  -s myapp \
  -b [54f:b680:ba6e:7ced:355f:346f:d97b:eecb]:8080

# Create a route with multiple backends (load balancing)
gwctl routes create \
  -i user1.pub \
  -k test_keys/user1.priv \
  -f api.gateway.example.com \
  -b [54f:b680:ba6e:7ced:355f:346f:d97b:eecb]:8080 \
  -b [449:abcd:0123:defa::1]:8080 \
  -m round_robin

# Create a route with random load balancing
gwctl routes create \
  -i user1.pub \
  -k test_keys/user1.priv \
  -f secure.example.com \
  -b [54f:b680:ba6e:7ced:355f:346f:d97b:eecb]:443 \
  -m random \
  -t  # Enable TLS passthrough (requires FQDN)

Managing Routes

# List all your routes
gwctl routes list -i user1.pub -k test_keys/user1.priv

# Get route details by ID
gwctl routes get -i user1.pub -k test_keys/user1.priv -r <route-id>

# Update a route (backends, method, TLS passthrough)
gwctl routes update \
  -i user1.pub \
  -k test_keys/user1.priv \
  -r <route-id> \
  -b [54f:b680:ba6e:7ced:355f:346f:d97b:eecb]:9000 \
  -m round_robin

# Update a route with multiple backends
gwctl routes update \
  -i user1.pub \
  -k test_keys/user1.priv \
  -r <route-id> \
  -b [54f:b680:ba6e:7ced:355f:346f:d97b:eecb]:8080 \
  -b [449:abcd:0123:defa::1]:8080 \
  -m random

# Delete a route
gwctl routes delete -i user1.pub -k test_keys/user1.priv -r <route-id>

Backend Address Format

Backends must be Mycelium IPv6 addresses in the 400::/7 range:

  • Example: 54f:b680:ba6e:7ced:355f:346f:d97b:eecb
  • Full address with port: [54f:b680:ba6e:7ced:355f:346f:d97b:eecb]:8080

To find your Mycelium IP:

mycelium inspect --json
# Returns: {"publicKey": "...", "address": "54f:b680:ba6e:7ced:355f:346f:d97b:eecb"}

Using gwctl

The gwctl CLI tool provides a convenient interface to manage routes and perform administrative operations.

Route Management Commands

gwctl routes list

List all routes for the authenticated user.

gwctl routes list -i user1.pub -k test_keys/user1.priv

gwctl routes create

Create a new route. Requires either --fqdn or --subdomain, and at least one --backend.

Flags:

  • -f, --fqdn - Fully qualified domain name (e.g., example.com)
  • -s, --subdomain - Subdomain (alternative to FQDN, e.g., myapp becomes myapp.<base-domain>)
  • -b, --backend - Backend address (host:port), can be repeated for multiple backends
  • -m, --method - Routing method: round_robin (default) or random
  • -t, --tls-pass - Enable TLS passthrough (requires FQDN, not subdomain)

Examples:

# Create with FQDN
gwctl routes create -i user1.pub -k test_keys/user1.priv \
  -f example.com -b 127.0.0.1:8080

# Create with subdomain
gwctl routes create -i user1.pub -k test_keys/user1.priv \
  -s myapp -b 127.0.0.1:8080

# Create with multiple backends and load balancing
gwctl routes create -i user1.pub -k test_keys/user1.priv \
  -f api.example.com -b 127.0.0.1:8080 -b 127.0.0.1:8081 -m random

# Create with TLS passthrough
gwctl routes create -i user1.pub -k test_keys/user1.priv \
  -f secure.example.com -b [400::1]:443 -t

gwctl routes get

Get details of a specific route by its ID.

gwctl routes get -i user1.pub -k test_keys/user1.priv -r <route-id>

Flags:

  • -r, --route - Route ID (required)

gwctl routes update

Update an existing route's configuration (backends, routing method, TLS passthrough).

Flags:

  • -r, --route - Route ID (required)
  • -b, --backend - Backend address, can be repeated (at least one required)
  • -m, --method - Routing method: round_robin or random
  • -t, --tls-pass - Enable TLS passthrough

Example:

gwctl routes update -i user1.pub -k test_keys/user1.priv \
  -r <route-id> -b 127.0.0.1:9000 -m round_robin

gwctl routes delete

Delete a route by its ID.

gwctl routes delete -i user1.pub -k test_keys/user1.priv -r <route-id>

Flags:

  • -r, --route - Route ID (required)

Admin Commands

gwctl admin suspend

Suspend a user, preventing them from accessing their routes. Requires admin authentication.

gwctl admin suspend -i admin.pub -k test_keys/admin.priv --user user1.pub

Flags:

  • --user - User public key ID to suspend (required)

gwctl admin unsuspend

Unsuspend a user, restoring their access to routes. Requires admin authentication.

gwctl admin unsuspend -i admin.pub -k test_keys/admin.priv --user user1.pub

Flags:

  • --user - User public key ID to unsuspend (required)

Utility Commands

gwctl health

Check gateway health status. No authentication required.

gwctl health

gwctl version

Display the version of gwctl.

gwctl version

Global Flags

All commands support these global flags:

  • -u, --url - Gateway RPC URL (default: http://localhost:8080/rpc)
  • -i, --id - Public key ID for authentication
  • -k, --key - Private key file path
  • -v, --verbose - Enable verbose output (shows request/response details)

Note: The gateway also exposes an OpenRPC API at the base domain for programmatic access. See the OpenRPC specification for detailed endpoint documentation.

Security

HTTP Signature (httpsig)

All mutation endpoints require valid HTTP Signature (RFC 9421):

  • Algorithm: ED25519 (recommended) or RSA
  • Signed Headers: (request-target), date, content-digest
  • Key ID: Identifier to fetch public key from configured store
  • Public Key Validation: Fetched from configured URI, cached with TTL

Mycelium Security

  • E2E Encryption: All overlay traffic encrypted (x25519)
  • Identity Binding: IPv6 address cryptographically bound to key pair
  • Route Signing: Route updates signed by source nodes
  • Private Networks: Optional PSK for isolated deployments

Architecture

Request Flow

[Public Client]
     |
     | HTTPS (port 443)
     ▼
┌────────────────────────────────┐
│   Layer4 (Port 443)            │
│   - TLS Passthrough Routes     │
│     → Direct proxy to backend  │
│   - Regular Routes             │
│     → Terminate TLS            │
│     → Proxy to HTTP app (80)   │
└─────────────┬──────────────────┘
              │
              │ HTTP (port 80, 127.0.0.1)
              ▼
┌────────────────────────────────┐
│   HTTP App (Port 80)           │
│   - External requests          │
│     → Redirect to HTTPS (443)  │
│   - Internal requests          │
│     → Base domain: OpenRPC     │
│     → Service routes: Reverse  │
│        proxy to backends       │
│     → Catch-all: 503 response  │
└─────────────┬──────────────────┘
              │
              │ Mycelium IPv6 Overlay
              ▼
┌─────────────────────────────────┐
│   Mycelium Daemon (Rust)        │
│   - IPv6 Overlay (400::/7)      │
│   - HTTP API (port 8989)        │
│   - JSON-RPC (port 8990)        │
│   - E2E Encryption (x25519)     │
└─────────────┬───────────────────┘
              │
              │ IPv6 Overlay Network
              │ (TUN interface)
              ▼
     [Mycelium Network]
              │
              ▼
┌────────────────────────────────┐
│   Backend Services             │
│   - Mycelium IPv6 addresses    │
│   - Web apps, APIs, etc.       │
└────────────────────────────────┘

Development

Building the Project

# Build gateway
make build

# Build gwctl CLI tool
cd cmds/gwctl && go build -o gwctl

# Production
make release

# Tests
make test

License

Apache-2.0