feat: heartbeat identity signing + mycelium IP preservation + dioxus native island #91

Open
mik-tf wants to merge 8 commits from development_mik02_heartbeat_preserve_ip into development
Owner

Summary

Two independent feature tracks developed during sessions 17-18:

1. Heartbeat Identity and Signing

  • Persistent ed25519 keypair — node generates and stores identity at startup (NEAR implicit account)
  • Signed heartbeat payloads with cryptographic verification
  • Explorer verifies signatures with legacy grace period for unsigned nodes
  • Fixes mycelium_ip being overwritten with empty string on heartbeat

2. Dioxus Native Island (additive, non-breaking)

  • hero_cloud_app crate — native dioxus-bootstrap-css admin island for hero_compute
  • Part of broader migration (lhumina_code/home#104)
  • Purely additive — existing hero_compute_ui (Askama iframe) is unchanged
  • Currently disabled in hero_os — re-enable after this PR merges

3. Regenerated OSIS bindings

  • Cloud types and RPC bindings regenerated to match current OSchema

After merge

  • Re-enable hero_cloud_app dependency in hero_os/crates/hero_os_app/Cargo.toml (line 309, currently commented out)
  • Add hero_cloud_app back to island-compute-native feature

Test plan

  • Verify heartbeat signing works (node generates keypair, signs payload)
  • Verify explorer accepts signed heartbeats
  • Verify mycelium_ip preserved when incoming value is empty
  • cargo build --release succeeds
  • After merge: re-enable hero_cloud_app in hero_os, verify WASM builds
## Summary Two independent feature tracks developed during sessions 17-18: ### 1. Heartbeat Identity and Signing - Persistent ed25519 keypair — node generates and stores identity at startup (NEAR implicit account) - Signed heartbeat payloads with cryptographic verification - Explorer verifies signatures with legacy grace period for unsigned nodes - Fixes mycelium_ip being overwritten with empty string on heartbeat ### 2. Dioxus Native Island (additive, non-breaking) - hero_cloud_app crate — native dioxus-bootstrap-css admin island for hero_compute - Part of broader migration (https://forge.ourworld.tf/lhumina_code/home/issues/104) - Purely additive — existing hero_compute_ui (Askama iframe) is unchanged - Currently disabled in hero_os — re-enable after this PR merges ### 3. Regenerated OSIS bindings - Cloud types and RPC bindings regenerated to match current OSchema ## After merge - Re-enable `hero_cloud_app` dependency in `hero_os/crates/hero_os_app/Cargo.toml` (line 309, currently commented out) - Add `hero_cloud_app` back to `island-compute-native` feature ## Test plan - [ ] Verify heartbeat signing works (node generates keypair, signs payload) - [ ] Verify explorer accepts signed heartbeats - [ ] Verify mycelium_ip preserved when incoming value is empty - [ ] cargo build --release succeeds - [ ] After merge: re-enable hero_cloud_app in hero_os, verify WASM builds
fix: preserve mycelium_ip on heartbeat when incoming value is empty
All checks were successful
Test / test (push) Successful in 1m25s
Test / test (pull_request) Successful in 1m30s
ef17fe81f2
The explorer's ExplorerService.node_heartbeat handler unconditionally
wrote the incoming mycelium_ip onto the existing node record. When the
server's MYCELIUM_IP env var was unset, the heartbeat sender sent an
empty string, which then clobbered the previously-stored address.

Any downstream consumer that filters nodes by mycelium_ip (e.g. the
marketplace backend's ComputeClient::lookup_by_mycelium_ip) silently
stopped finding the node, even though it was still online and
heartbeating.

Fix on both sides:
- hero_compute_explorer (rpc.rs): only overwrite node.mycelium_ip when
  the incoming value is non-empty. Preserves whatever was stored when
  the server sends "".
- hero_compute_server (main.rs): log a WARN at startup when the
  heartbeat sender is configured but MYCELIUM_IP env var is unset, so
  operators notice the misconfiguration instead of having it silently
  zero out the field.

Test: new inline test in rpc.rs heartbeat_preserve_tests module. Runs
two heartbeats (first with a real IP, second with ""), asserts the
first IP survives. Verified red→green — reverting the rpc.rs fix makes
the test fail with the expected message.

Discovered while debugging the dev VM's hero_compute node showing
offline in the marketplace, even after the heartbeat sender was
correctly configured.
Phase 3 step 1 of the scaling architecture initiative.

Adds a NodeIdentity module to hero_compute_server that manages a
persistent ed25519 keypair, stored as a 32-byte seed in
$data_dir/identity.bin with chmod 600. The node's NEAR implicit account
ID is hex(pubkey), matching the hero_ledger gateway's
`implicit_account_id` convention so node identities are directly
predictable from their pubkeys.

Capabilities:
- load_or_create(data_dir): idempotent — loads existing seed or
  generates a fresh one from OsRng
- implicit_account(): 64-char lowercase hex for on-chain predecessor_account_id
- public_key_bytes() / public_key_near_format() for RPC payloads
- sign_hashed(payload) / verify_hashed(acct, payload, sig) pairs with
  the sha256-then-ed25519 pattern already used by the hero_ledger
  gateway auth layer (consistent signing shape across the ecosystem)
- sign_raw / verify_raw for callers that need to choose their own
  canonicalization

Storage decision — plain file, not sled:
The server's OSIS data layer is schema-first and typed, not suited to
raw untyped secret bytes. A chmod-600 file matches the scaling
architecture spec's direct suggestion and avoids pulling sled as a
direct dep.

Deps added:
- ed25519-dalek 2 (with rand_core feature) — gen/sign/verify
- rand 0.8 — OsRng for keygen
- sha2 0.10 — payload hashing (pairs with gateway auth convention)
- hex 0.4 — implicit account encoding
- bs58 0.5 — NEAR-format public key
- tempfile 3 (dev-only) — test isolation

Tests: 13 unit tests covering keygen, persistence, chmod enforcement,
signature roundtrip, cross-key rejection, payload tampering rejection,
corrupted file handling, and convention pinning against the
hero_ledger gateway's implicit_account_id shape.

No wiring into main.rs startup or heartbeat_sender yet — that lands
in the next commits.
Phase 3 step 2 of the scaling architecture initiative.

Wires NodeIdentity::load_or_create into hero_compute_server startup
right after config validation. First-boot generates a fresh ed25519
keypair at $data_dir/identity.bin; subsequent boots load the same
seed. Startup logs include:

- Implicit account (64-char lowercase hex) — this is the on-chain
  predecessor_account_id used for hosting.farm_create / node_register
- Pubkey in NEAR ed25519:<base58> wire format

Operators can copy the logged implicit account directly into the
hero_ledger gateway to query the node's on-chain state, match it to
hosting.node_get_by_identity results, or pair with a farmer.

Stored as Arc<NodeIdentity> so it can be cloned into the heartbeat
sender (next commit) and the first-boot registration hook without
reloading the file.

No behavior change beyond logging — heartbeats are not yet signed.
Phase 3 step 3 of the scaling architecture initiative.

The heartbeat sender now signs each heartbeat with the node's
persistent ed25519 identity (introduced in the previous commit).
Explorers can verify the signature against the node's implicit
account (hex(pubkey)) so only the real node can update its own
record — no more trusting hostname/IP blindly.

Protocol
--------

Canonical signed payload is a sorted-key JSON blob (BTreeMap → stable
byte order) over exactly:

  { available_slices, hostname, mycelium_ip, node_account,
    slice_count, timestamp }

The sender computes sha256 of those bytes, signs with ed25519, and
adds three new fields to the node_heartbeat RPC params:

  node_account  — 64-char hex implicit account (hex(pubkey))
  timestamp     — unix secs, included in the signed hash for replay
                  protection
  signature     — base64 ed25519 sig over sha256(canonical_payload)

The signing shape (sha256-then-ed25519) matches the hero_ledger
gateway auth layer so the Hero ecosystem has exactly one
canonicalization convention.

Why sorted-key JSON:
- Language-independent — any verifier with serde_json or BTreeMap or
  Python's json.dumps(sort_keys=True) reconstructs the same bytes.
- Human-inspectable in logs if needed.
- Byte-stable regardless of source ordering of struct fields.

Backward compat
---------------

The three new fields are additive — older explorers without
signature-verification logic simply ignore them and process the
heartbeat exactly as before. The explorer-side verification lands
in the next commit, feature-flagged behind ALLOW_LEGACY_HEARTBEATS
with a grace period default (accepts unsigned).

Deps added
----------
- base64 = "0.22" for signature encoding

Tests
-----
New `canonical_payload_tests` module with 5 tests:
- canonical_payload_keys_are_sorted — lexicographic order pinned
- canonical_payload_is_byte_stable — repeat calls return identical bytes
- signed_payload_roundtrip_verifies — full sign→verify happy path
- tampered_timestamp_breaks_verification — replay protection works
- tampered_capacity_breaks_verification — field integrity works

Full hero_compute_server test count: 29 (13 identity + 5 canonical
payload + 11 pre-existing cloud CRUD). All green. Explorer test
(node_heartbeat_preserves_mycelium_ip_when_incoming_is_empty) from
the earlier commit in this PR also still green.
feat(explorer): verify heartbeat signatures with legacy grace period
Some checks failed
Test / test (pull_request) Failing after 1m23s
Test / test (push) Failing after 1m25s
38b1c8e503
Phase 3 step 4 of the scaling architecture initiative.

Adds ed25519 signature verification to ExplorerService.node_heartbeat.
The explorer can now authenticate which node sent each heartbeat and
reject spoofing attempts that used to be possible by just matching the
hostname.

Schema changes
--------------

schemas/explorer/explorer.oschema:
- node_heartbeat gets three new trailing params:
    node_account: str      // 64-char hex = hex(pubkey) = NEAR implicit account
    timestamp:    u64      // unix secs, signed to prevent simple replay
    signature:    str      // base64 ed25519 sig over sha256(canonical payload)
- ExplorerNode gets a new `node_account: str @index` field so the
  explorer can look nodes up by on-chain identity, not just hostname.

Generated files regenerate from build.rs (rpc_generated.rs,
types_generated.rs, osis_server_generated.rs, openrpc.json, mod.rs).

Handler logic (rpc.rs)
----------------------

A heartbeat is considered "signed" iff `signature` is non-empty.

- Signed heartbeat: verify_heartbeat_signature is called. It rebuilds the
  canonical sorted-key JSON byte-identically to the server's
  canonical_signed_payload, sha256-hashes it, decodes the base64 sig and
  hex public key, and runs ed25519 verify. Also enforces a 120-second
  timestamp skew window against the explorer's clock as basic replay
  protection.

- Legacy unsigned heartbeat: accepted iff env var
  ALLOW_LEGACY_HEARTBEATS is unset or set to "true"/"1"/"yes" (default
  true during the Phase 3 transition). Set it to "false" once all nodes
  have upgraded to fail closed on unsigned traffic.

- Identity-rotation guard: if a hostname already has a recorded
  node_account and a new signed heartbeat claims a different one,
  reject. Prevents silent hostname takeover.

- node_account is only persisted on signed heartbeats, so a stray
  legacy heartbeat cannot clobber a previously-recorded on-chain
  identity.

Tests
-----

New signature_tests module with 9 tests:
- valid_signature_is_accepted (happy path)
- tampered_capacity_is_rejected
- tampered_hostname_is_rejected
- signature_from_different_key_is_rejected
- stale_timestamp_is_rejected (outside -120s window)
- future_timestamp_is_rejected (outside +120s window)
- invalid_base64_signature_is_rejected
- non_hex_node_account_is_rejected
- wrong_length_signature_is_rejected

These call verify_heartbeat_signature directly rather than going through
the RPC handler because ALLOW_LEGACY_HEARTBEATS is a process-wide env
var and mutating it from parallel tests is fragile. The handler-level
legacy path stays exercised by the existing
heartbeat_preserve_tests::node_heartbeat_preserves_mycelium_ip_when_incoming_is_empty
test, which now passes String::new() / 0 / String::new() for the three
new params to exercise the grace-period path.

Deps added (crate-level)
------------------------
- ed25519-dalek = "2" (verification side, no rand_core feature)
- sha2 = "0.10"
- hex = "0.4"
- base64 = "0.22"
- dev: rand 0.8 + ed25519-dalek with rand_core feature for test key gen

Full hero_compute test count after this commit: 29 server + 15 explorer
= 44 unit tests, all green.
mik-tf changed title from fix: preserve mycelium_ip on heartbeat when incoming value is empty to fix: heartbeat mycelium_ip preserve + Phase 3 node crypto identity 2026-04-11 01:56:47 +00:00
style: cargo fmt Phase 3 additions
Some checks failed
Test / test (push) Failing after 1m29s
Test / test (pull_request) Failing after 1m30s
fcf82c1ec7
CI rustfmt --check wanted tighter line breaks in heartbeat_sender.rs
and identity.rs after the Phase 3 commits. No semantic changes,
cargo test still green (29 server + 15 explorer = 44 unit tests).
style: allow dead_code on Phase 3.5 identity API
All checks were successful
Test / test (pull_request) Successful in 1m28s
Test / test (push) Successful in 1m30s
54eb0ac6ff
Clippy under `-D warnings` fails on unused methods that will be used by
the Phase 3.5 first-boot registration hook (separate follow-up PR):

- NodeIdentity::public_key_bytes — used by NEAR tx construction later
- NodeIdentity::sign_raw — used to sign raw borsh-serialized NEAR txs
- NodeIdentity::verify_raw / verify_hashed — server-side helpers whose
  runtime callers live in hero_compute_explorer::explorer::rpc (which
  has its own inline implementation — cannot cross-import because
  identity.rs is a binary module, not a library one)
- verifying_key_from_implicit — only called by the verify_* helpers

Targeted `#[allow(dead_code)]` on each with a doc comment explaining
why it is intentional, rather than a blanket module-level allow.
Unit tests still exercise these paths (tests are cfg(test) so clippy
on the main binary doesn't count them).

No semantic change. 29 server + 15 explorer tests still green.
Author
Owner

@mahmoud — review ping. This PR combines the small defensive heartbeat fix (preserve mycelium_ip on empty-string input) with Phase 3 Steps 1-4 of the marketplace scaling architecture initiative (mycelium_code/home#72): persistent ed25519 node identity, signed heartbeats, explorer-side signature verification, and an ALLOW_LEGACY_HEARTBEATS grace period.

7 commits, 44 unit tests green, CI (Test + Format + Clippy) all passing on 54eb0ac. Phase 3 Step 5 (first-boot registration hook) is intentionally deferred to a stacked follow-up PR because it drags in near-jsonrpc-client / near-primitives and cannot be E2E-verified until the hosting contract is deployed on devnet.

Full scope breakdown in the PR description. Non-urgent — we have marketplace-side Phase 5 work lined up that does not block on this merge.

@mahmoud — review ping. This PR combines the small defensive heartbeat fix (preserve mycelium_ip on empty-string input) with Phase 3 Steps 1-4 of the marketplace scaling architecture initiative ([mycelium_code/home#72](https://forge.ourworld.tf/mycelium_code/home/issues/72)): persistent ed25519 node identity, signed heartbeats, explorer-side signature verification, and an `ALLOW_LEGACY_HEARTBEATS` grace period. 7 commits, 44 unit tests green, CI (Test + Format + Clippy) all passing on `54eb0ac`. Phase 3 Step 5 (first-boot registration hook) is intentionally deferred to a stacked follow-up PR because it drags in `near-jsonrpc-client` / `near-primitives` and cannot be E2E-verified until the hosting contract is deployed on devnet. Full scope breakdown in the PR description. Non-urgent — we have marketplace-side Phase 5 work lined up that does not block on this merge.
chore: regenerated OSIS cloud types and RPC bindings
All checks were successful
Test / test (push) Successful in 1m47s
Test / test (pull_request) Successful in 3m43s
7565124210
Signed-off-by: mik-tf
mik-tf changed title from fix: heartbeat mycelium_ip preserve + Phase 3 node crypto identity to feat: heartbeat identity signing + mycelium IP preservation + dioxus native island 2026-04-12 14:06:54 +00:00
All checks were successful
Test / test (push) Successful in 1m47s
Test / test (pull_request) Successful in 3m43s
This pull request can be merged automatically.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin development_mik02_heartbeat_preserve_ip:development_mik02_heartbeat_preserve_ip
git switch development_mik02_heartbeat_preserve_ip

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch development
git merge --no-ff development_mik02_heartbeat_preserve_ip
git switch development_mik02_heartbeat_preserve_ip
git rebase development
git switch development
git merge --ff-only development_mik02_heartbeat_preserve_ip
git switch development_mik02_heartbeat_preserve_ip
git rebase development
git switch development
git merge --no-ff development_mik02_heartbeat_preserve_ip
git switch development
git merge --squash development_mik02_heartbeat_preserve_ip
git switch development
git merge --ff-only development_mik02_heartbeat_preserve_ip
git switch development
git merge development_mik02_heartbeat_preserve_ip
git push origin development
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lhumina_code/hero_compute!91
No description provided.