lhumina_code/hero_shrimp

Fork 0

migrate text id's to int #15

New issue

Open

opened 2026-04-30 05:46:35 +00:00 by despiegk · 2 comments

despiegk commented

2026-04-30 05:46:35 +00:00

Owner

in line to other services, we try to use autoincrement int's (u32, but in sqlite i64)
we also found in this codebase there are some mix up's in the DB, this will fix this too

in line to other services, we try to use autoincrement int's (u32, but in sqlite i64) we also found in this codebase there are some mix up's in the DB, this will fix this too

despiegk commented

2026-04-30 05:59:39 +00:00

Author

Owner

Migration spec: text IDs → `i64` autoincrement integers

Branch: development_id
Issue: #15
Strategy: wipe-and-replace, no data migration. After this lands, on first service_shrimp start --clear everyone gets a fresh integer-keyed DB.

1. Goals

Every row PK that today is TEXT becomes INTEGER PRIMARY KEY AUTOINCREMENT (SQLite INTEGER = i64).
Every Rust newtype wrapping a row PK (SessionId, RunId, conversation/run/phase/etc. on the wire) becomes a wrapper around i64, not String.
Every OpenRPC method that today takes a string ID changes its JSON Schema to integer.
The two duplicate SessionId types are unified.
Database schema mix-ups (mixed PK types in tables that should be uniform) are cleaned up.

2. Non-goals

No data migration. Old DBs are wiped via service_shrimp start --clear. There is no ALTER TABLE / move-data-into-new-schema path.
No backwards compatibility on the wire. Old SDK clients break — that's accepted because nothing outside this repo consumes the API in production yet.
ClientId stays String. It identifies an external client install, not a row PK.
PhaseId stays String. It's a label (e.g. "runtime", "reflect"), not a row primary key. There is no phases table.
External-system identifiers stay String — external_id (idempotency keys), dedup_key, user_id in user_profiles, cache_key, briefings.date. Same for the user-facing crypto.uuid_v4 Rhai tool — that's a tool output for users, not a hero_shrimp row identifier.
Idempotency-scope strings stay strings. e.g. format!("run:{run_id}:phase:runtime:attempt:1"). After the migration, run_id inside that format is an i64, but the resulting string is still a string.

3. Wire-format change (OpenRPC)

Every field below in crates/hero_shrimp_types/src/openrpc.json flips from "type": "string" to "type": "integer" (with "format": "int64").

Method	Field
`session.create`	`params.conversation_id`, `result.session_id`, `result.conversation_id`
`session.close`	`params.session_id`
`session.list`	`result.sessions[].id`, `result.sessions[].conversation_id`
`session.subscribe`	`params.session_id`, `result.subscription_id`
`session.cancel`	`params.session_id`
`message.send`	`params.session_id`
`message.list`	`params.conversation_id`
`tool.call`	`params.session_id`
`audit.list`	`params.session_id`
`run.cancel`, `run.files`, `run.file`, `run.execution_control`, `run.rollback_file`, `run.commit`, `run.bundle`, `run.workspace_bundle`	`params.run_id`
`doai.start`	`params.session_id`
`doai.follow_up`	`params.run_id`, `params.session_id`
`doai.status`	`params.run_id`

Already integer (no change): memory.update_status, memory.drilldown, episode.drilldown, playbook.export.

Result schemas typed as "object"/array-of-object inherit the change automatically when the Rust struct fields flip to i64 and serde re-serializes — but the OpenRPC schema should also be tightened to declare the integer type explicitly. Out of scope for the first PR if it bloats the diff; flag as follow-up.

4. Schema change (SQLite)

crates/hero_shrimp_store/src/db/. Before-and-after, tables only:

Tables whose PK changes from `TEXT` → `INTEGER PRIMARY KEY AUTOINCREMENT`

File	Table
`schema.rs`	`conversations`, `subagent_jobs`, `detached_tasks`, `task_flows`
`sessions.rs`	`sessions`
`autonomy_runs.rs`	`autonomy_runs`
`council.rs`	`council_sessions`
`dreams.rs`	`dream_cycles`
`projects.rs`	`projects`
`scheduled_jobs.rs`	`scheduled_jobs`

Tables whose PK is the FK of another table (1:1 satellite tables) — PK becomes `INTEGER` matching the parent

File	Table	Parent
`conversation_briefs.rs`	`conversation_briefs` (`conversation_id` PK)	`conversations`
`run_briefs.rs`	`run_briefs` (`run_id` PK)	`autonomy_runs`
`run_state.rs`	`run_state_snapshots` (`run_id` PK)	`autonomy_runs`

Foreign-key columns that change from `TEXT NOT NULL` → `INTEGER NOT NULL`

File	Table	Column → references
`schema.rs`	`audit_log`	`run_id`, `session_id` (`phase_id` stays TEXT — it's a label)
`schema.rs`	`memories`	`source_run_id` (and `source_phase_id` stays TEXT)
`schema.rs`	`memory_outbox`	`conversation_id`, `run_id`
`schema.rs`	`subagent_jobs`	`parent_id` (self), `related_run_id`
`schema.rs`	`detached_tasks`	`flow_id`, `related_run_id`, `parent_task_id`
`schema.rs`	`task_flows`	`related_run_id`, `root_task_id`
`council.rs`	`council_positions`	`session_id` (`councilor_id` stays TEXT — string label)
`autonomy_runs.rs`	`autonomy_runs`	`parent_run_id` (self)
`dreams.rs`	`dream_insights`, `dream_open_questions`	`cycle_id`
`artifacts.rs`	`phase_artifacts`	`run_id`
`migrations.rs`	`conversation_compactions`	`conversation_id`
`conversation_briefs.rs`	`conversation_briefs`	`project_id`

Tables NOT changing (already integer, or correctly using TEXT for natural keys)

✅ Already integer: messages, audit_log PK, usage_log, skill_usage_log, memories, memory_recalls, memory_outbox, council_positions, dream_insights, dream_open_questions, autonomy_lessons, memory_episodes, memory_playbooks, phase_artifacts, todos, conversation_compactions, user_facts.

✅ Correctly TEXT (natural / external keys): briefings.date, tool_cache.cache_key, inbound_dedup.dedup_key, tool_invocations.key, secrets.key, state.key, user_profiles.user_id.

5. Newtype changes (`hero_shrimp_types`)

Unify `SessionId`

Today there are two: proto::ids::SessionId(String) and domain::ids::SessionId(String). Keep one in proto::ids, delete domain::ids::SessionId, re-export from domain::ids so existing imports keep working.

Newtype inner type changes

Type	Before	After
`proto::ids::SessionId`	`String`	`i64`
`domain::ids::RunId`	`String`	`i64`
`domain::ids::PhaseId`	`String`	stays `String` (label, not PK)
`proto::ids::ClientId`	`String`	stays `String` (external identity)

Other ID-bearing fields in the types crate (drop directly to `i64`, no newtype)

File:line	Field	Before	After
`proto/rpc.rs:74`	`SessionInit.conversation_id`	`Option<String>`	`Option<i64>`
`proto/rpc.rs:92`	`SessionSummary.conversation_id`	`Option<String>`	`Option<i64>`
`domain/state_store.rs:5`	`RunStateSnapshot.run_id`	`String`	`i64`
`domain/run.rs:64,103,219,220,233,249`	`run_id`/`left_run_id`/`right_run_id`/`job_id`	`String`	`i64` (each)
`domain/run.rs` (same file)	`phase_id` instances	`String`	stays `String`
`domain/contracts.rs:32`	`requirement_id`	`String`	`i64`
`domain/contracts.rs:35`	`phase_id`	`Option<String>`	stays `Option<String>`

6. ID-generation strategy

Today there are ~15 places that mint or derive IDs by format!. The migration introduces a small number of canonical mint sites in the store crate, and replaces every former format! site that produced a row PK with one of:

Database::lookup_or_create_session(channel: &str, user_key: &str) -> i64 — replaces format!("session:cli:{n}"), format!("session:admin:{n}"), format!("session:whatsapp:chat:{chat_id}"), format!("session:telegram:chat:{chat_id}"), format!("session:schedule:{schedule_name}"). Inserts on miss using INSERT ... RETURNING id. The (channel, user_key) pair is stored on the row and reads use it directly instead of parsing the id string.
Database::next_autonomy_run_id() -> i64 — replaces wherever requested_autonomy_run_id was injected as a string. Implementation: INSERT INTO autonomy_runs(...) RETURNING id.
Database::next_conversation_id() -> i64 — replaces format!("run:{run_id}") insertions into conversations. Same INSERT ... RETURNING id pattern.

Idempotency-scope strings (e.g. format!("run:{run_id}:phase:runtime:attempt:1")) keep using format!; the substituted run_id is now i64. The resulting string is still used as a string key in tool_invocations etc.

Graph-edge labels (db/work_history.rs:50, db/conversation_graph.rs:93,102 doing format!("run:{}", run_id)) keep being strings — they're not row PKs, they're typed edge tags in a graph store. Those tags now embed an integer instead of a UUID. The string "run:42" is just as good as the string "run:UUID-..." for graph-edge purposes.

7. RPC handlers (`hero_shrimp_server`)

crates/hero_shrimp_server/src/rpc/jsonrpc.rs — single 5600-line file.

Add helper optional_i64(params: &Value, key: &str) -> Option<i64> next to existing optional_string. Same for require_i64.
Replace every optional_string(params, "session_id" | "run_id" | "conversation_id" | "subscription_id") with the integer variant.
Methods touched (full list): method_session_create, method_session_close, method_session_list, method_session_subscribe, method_session_cancel, method_message_send, method_message_list, method_audit_list, method_tool_call, method_run_status, method_run_files, method_run_file, method_run_execution_control, method_run_rollback_file, method_run_commit, method_run_bundle, method_run_workspace_bundle, method_doai_start, method_doai_follow_up, method_doai_status.
Result-building serde_json::json!({...}) macros need no change beyond passing the integer values — serde_json serializes i64 as a JSON number automatically.
Tests in this file that hardcode "session:cli:1" etc. need updating.

8. Engine (`hero_shrimp_engine`)

Per-file change list (these are the intent edits — the rest is mechanical):

src/engine_client/in_process.rs:56 — replace SessionId::new(format!(...)) with a call to db.lookup_or_create_session(channel, user_key) -> SessionId(i64).
src/pipeline.rs:29,63,67,296,300 — every format!("session:...") and format!("run:...") site: drop the format if it produced a row PK; keep it if it produced an idempotency-scope string with the inner ID now i64.
src/autonomy.rs:177-212 — requested_autonomy_run_id(&ToolContext) -> Option<i64> (was Option<String>); parse the integer instead of the string.
src/autonomy/persistence.rs:120,222, src/autonomy/promote.rs:183,188, src/subagents.rs:306,326,356, src/agent/mod.rs:166, src/execution_control.rs:1175,1179,1684,1726 — all idempotency/graph-label format strings; just propagate the type change of the embedded ID.
src/gateways/whatsapp.rs:314, src/gateways/telegram.rs:490, src/schedules.rs:158 — channel-specific session creation: replace the format! mint with db.lookup_or_create_session(channel, user_key).
src/gateways/admin/filters.rs:129 — from == format!("run:{run_id}") graph-walk match: run_id is now i64, the format produces "run:42", comparison still works structurally — but audit this site: it relies on the string-id format being predictable.
src/gateways/admin/routes/chat.rs:139 — same.
Every EngineClient trait method whose signature mentions SessionId / RunId — signature change is automatic once the newtype's inner changes; affects in-process and any out-of-process implementations.

9. CLI (`hero_shrimp` binary — manager + client)

src/main.rs:151 — Cancel { session_id: String } → i64. Change clap to value_parser = clap::value_parser!(i64).
src/main.rs:205,272-294,317,326 — session_id plumbing through run_send/run_cancel. Type signatures change; print sites use Display (already implemented on the newtype).
src/tui.rs:117,140,169,213,223,310 — AppState.session_id: String → i64; format strings using session_id need .to_string() or rely on Display.

10. UI (`hero_shrimp_ui`)

src/routes.rs:252,287 — RunFileDownloadQuery { run_id: String } and RunDownloadQuery { run_id: String } → i64. Axum's Query<> + serde will deserialize numeric query strings as i64 if the target field is i64.
src/routes.rs:269,300,325 — JSON forwarding to backend; serde handles the type switch.
static/js/dashboard.js:850 — breaking change: conversation_id: \ui-${kind}-${nonce}`is a client-supplied string. After migration, the UI must callsession.create*without* aconversation_id` and use the integer the server returns.
static/js/dashboard.js other sites — IDs flow through as JS numbers (JSON numbers parse to JS numbers); call sites pass them back unchanged. Audit any === comparisons that today assume string equality (none found in survey, but worth a grep before merging).
Templates (templates/*.html) — none interpolate IDs directly. No change.

11. SDK (`hero_shrimp_sdk`)

Auto-generated from openrpc.json via openrpc_client! macro. Once openrpc.json is updated, the SDK regenerates automatically on next cargo build. No manual edits. Consumers of the SDK in crates/hero_shrimp_examples/ will fail to compile until their call sites are fixed; treat as part of the migration.

12. Test plan

Schema unit tests — every db/*.rs module has mod tests (~20 fixture sites use hardcoded string IDs like "run:test-1"). Update each to use i64 literals from freshly-minted IDs (INSERT ... RETURNING) or fixed integer literals.
Wire round-trip tests — crates/hero_shrimp_examples/tests/integration.rs::test_openrpc_spec_valid already verifies the spec parses. Add: walk every method, assert that ID-typed params/results are "type": "integer".
End-to-end — once a daemon-startup integration test lands, that test exercises session.create → message.send → session.close with real integer IDs over HTTP/UDS.
CLI smoke — hero_shrimp send "hi" must complete; hero_shrimp cancel <int> must accept an integer.
UI smoke — start the service, open the dashboard, click through to memory and runs panes; verify nothing displays "[object Object]" or "NaN".

13. Rollout (wipe-and-replace)

Strict ordering:

PR opened on development_id with all changes squashed.
Reviewer runs locally: service_shrimp start --reset --clear. The --clear wipes the existing default.db whose schema is incompatible.
After review, merge development_id → development.
Communicate to anyone running hero_shrimp: first start after pulling requires --clear. Any saved sessions / runs / memories from before are gone.

The wipe is mandatory because the schema migration goes from TEXT PRIMARY KEY to INTEGER PRIMARY KEY AUTOINCREMENT — SQLite has no in-place ALTER for that. Adding a migration path was considered and rejected per Issue #15: data lifetime in this DB is short, value of preserving old rows is low.

14. Sanity checks called out for the reviewer

Channel encoding leakage. format!("session:cli:{n}") today encodes the channel name in the ID string. The sessions.channel column already stores it separately, so the migration preserves the information — but every code site that today does if id.starts_with("session:cli:") (or grep equivalents) must be replaced by reading .channel from the row. Survey lists call sites; PR must address every one.
Two SessionId types. Don't leave both in the tree post-migration. If domain layer needs richer invariants, that's a separate refactor — the migration's job is to unify on the integer wire type.
requirement_id in domain::contracts. Migrating to i64 along with everything else (decided 2026-04-30 in this thread). If requirement_id is currently a runtime-only field with no SQLite table backing it, the change is purely type-level — no schema work. If a requirements table is later added, give it INTEGER PRIMARY KEY AUTOINCREMENT from the start.
phase_id stays String. No phases table exists. If the team later adds one with an INTEGER PRIMARY KEY, do that change in a separate PR — it's larger than it looks because phase_id is referenced from many FK columns.
Out-of-process EngineClient impls. Survey only inspected InProcessEngineClient. If any out-of-process implementation exists outside this repo, it needs updating in lockstep.
Tests that hardcode string IDs. ~20 sites. Use rg -n '"(run|session|flow):[a-z0-9_:-]*"' crates/ before declaring done.

15. Estimated PR size

hero_shrimp_types: ~5 files, ~50 LOC delta + openrpc.json changes
hero_shrimp_store: ~30 files, ~600 LOC delta (mostly mechanical query/RowMapper updates)
hero_shrimp_engine: ~15 files, ~80 LOC delta (most call sites are leaf format!s that stay)
hero_shrimp_server: 1 file (rpc/jsonrpc.rs), ~150 LOC delta
hero_shrimp (CLI): 2 files, ~20 LOC
hero_shrimp_ui: 2 files (routes.rs, dashboard.js), ~10 LOC
Tests: ~20 fixture updates across the workspace
SDK: 0 manual edits (regenerated)

Total: ~900 LOC, single squashed PR.

# Migration spec: text IDs → `i64` autoincrement integers Branch: `development_id` Issue: #15 Strategy: **wipe-and-replace**, no data migration. After this lands, on first `service_shrimp start --clear` everyone gets a fresh integer-keyed DB. --- ## 1. Goals 1. Every row PK that today is `TEXT` becomes `INTEGER PRIMARY KEY AUTOINCREMENT` (SQLite `INTEGER` = `i64`). 2. Every Rust newtype wrapping a row PK (`SessionId`, `RunId`, conversation/run/phase/etc. on the wire) becomes a wrapper around `i64`, not `String`. 3. Every OpenRPC method that today takes a string ID changes its JSON Schema to `integer`. 4. The two duplicate `SessionId` types are unified. 5. Database schema mix-ups (mixed PK types in tables that should be uniform) are cleaned up. ## 2. Non-goals - **No data migration.** Old DBs are wiped via `service_shrimp start --clear`. There is no ALTER TABLE / move-data-into-new-schema path. - **No backwards compatibility on the wire.** Old SDK clients break — that's accepted because nothing outside this repo consumes the API in production yet. - **`ClientId` stays `String`.** It identifies an external client install, not a row PK. - **`PhaseId` stays `String`.** It's a label (e.g. `"runtime"`, `"reflect"`), not a row primary key. There is no `phases` table. - **External-system identifiers stay `String`** — `external_id` (idempotency keys), `dedup_key`, `user_id` in `user_profiles`, `cache_key`, `briefings.date`. Same for the user-facing `crypto.uuid_v4` Rhai tool — that's a tool output for users, not a hero_shrimp row identifier. - **Idempotency-scope strings stay strings.** e.g. `format!("run:{run_id}:phase:runtime:attempt:1")`. After the migration, `run_id` inside that format is an `i64`, but the resulting *string* is still a string. ## 3. Wire-format change (OpenRPC) Every field below in `crates/hero_shrimp_types/src/openrpc.json` flips from `"type": "string"` to `"type": "integer"` (with `"format": "int64"`). | Method | Field | |---|---| | `session.create` | `params.conversation_id`, `result.session_id`, `result.conversation_id` | | `session.close` | `params.session_id` | | `session.list` | `result.sessions[].id`, `result.sessions[].conversation_id` | | `session.subscribe` | `params.session_id`, `result.subscription_id` | | `session.cancel` | `params.session_id` | | `message.send` | `params.session_id` | | `message.list` | `params.conversation_id` | | `tool.call` | `params.session_id` | | `audit.list` | `params.session_id` | | `run.cancel`, `run.files`, `run.file`, `run.execution_control`, `run.rollback_file`, `run.commit`, `run.bundle`, `run.workspace_bundle` | `params.run_id` | | `doai.start` | `params.session_id` | | `doai.follow_up` | `params.run_id`, `params.session_id` | | `doai.status` | `params.run_id` | Already integer (no change): `memory.update_status`, `memory.drilldown`, `episode.drilldown`, `playbook.export`. Result schemas typed as `"object"`/array-of-object inherit the change automatically when the Rust struct fields flip to `i64` and serde re-serializes — but the OpenRPC schema should also be tightened to declare the integer type explicitly. Out of scope for the first PR if it bloats the diff; flag as follow-up. ## 4. Schema change (SQLite) `crates/hero_shrimp_store/src/db/`. Before-and-after, tables only: ### Tables whose PK changes from `TEXT` → `INTEGER PRIMARY KEY AUTOINCREMENT` | File | Table | |---|---| | `schema.rs` | `conversations`, `subagent_jobs`, `detached_tasks`, `task_flows` | | `sessions.rs` | `sessions` | | `autonomy_runs.rs` | `autonomy_runs` | | `council.rs` | `council_sessions` | | `dreams.rs` | `dream_cycles` | | `projects.rs` | `projects` | | `scheduled_jobs.rs` | `scheduled_jobs` | ### Tables whose PK is the FK of another table (1:1 satellite tables) — PK becomes `INTEGER` matching the parent | File | Table | Parent | |---|---|---| | `conversation_briefs.rs` | `conversation_briefs` (`conversation_id` PK) | `conversations` | | `run_briefs.rs` | `run_briefs` (`run_id` PK) | `autonomy_runs` | | `run_state.rs` | `run_state_snapshots` (`run_id` PK) | `autonomy_runs` | ### Foreign-key columns that change from `TEXT NOT NULL` → `INTEGER NOT NULL` | File | Table | Column → references | |---|---|---| | `schema.rs` | `audit_log` | `run_id`, `session_id` (`phase_id` stays TEXT — it's a label) | | `schema.rs` | `memories` | `source_run_id` (and `source_phase_id` stays TEXT) | | `schema.rs` | `memory_outbox` | `conversation_id`, `run_id` | | `schema.rs` | `subagent_jobs` | `parent_id` (self), `related_run_id` | | `schema.rs` | `detached_tasks` | `flow_id`, `related_run_id`, `parent_task_id` | | `schema.rs` | `task_flows` | `related_run_id`, `root_task_id` | | `council.rs` | `council_positions` | `session_id` (`councilor_id` stays TEXT — string label) | | `autonomy_runs.rs` | `autonomy_runs` | `parent_run_id` (self) | | `dreams.rs` | `dream_insights`, `dream_open_questions` | `cycle_id` | | `artifacts.rs` | `phase_artifacts` | `run_id` | | `migrations.rs` | `conversation_compactions` | `conversation_id` | | `conversation_briefs.rs` | `conversation_briefs` | `project_id` | ### Tables NOT changing (already integer, or correctly using TEXT for natural keys) ✅ Already integer: `messages`, `audit_log` PK, `usage_log`, `skill_usage_log`, `memories`, `memory_recalls`, `memory_outbox`, `council_positions`, `dream_insights`, `dream_open_questions`, `autonomy_lessons`, `memory_episodes`, `memory_playbooks`, `phase_artifacts`, `todos`, `conversation_compactions`, `user_facts`. ✅ Correctly TEXT (natural / external keys): `briefings.date`, `tool_cache.cache_key`, `inbound_dedup.dedup_key`, `tool_invocations.key`, `secrets.key`, `state.key`, `user_profiles.user_id`. ## 5. Newtype changes (`hero_shrimp_types`) ### Unify `SessionId` Today there are two: `proto::ids::SessionId(String)` and `domain::ids::SessionId(String)`. **Keep one in `proto::ids`**, delete `domain::ids::SessionId`, re-export from `domain::ids` so existing imports keep working. ### Newtype inner type changes | Type | Before | After | |---|---|---| | `proto::ids::SessionId` | `String` | `i64` | | `domain::ids::RunId` | `String` | `i64` | | `domain::ids::PhaseId` | `String` | **stays `String`** (label, not PK) | | `proto::ids::ClientId` | `String` | **stays `String`** (external identity) | ### Other ID-bearing fields in the types crate (drop directly to `i64`, no newtype) | File:line | Field | Before | After | |---|---|---|---| | `proto/rpc.rs:74` | `SessionInit.conversation_id` | `Option<String>` | `Option<i64>` | | `proto/rpc.rs:92` | `SessionSummary.conversation_id` | `Option<String>` | `Option<i64>` | | `domain/state_store.rs:5` | `RunStateSnapshot.run_id` | `String` | `i64` | | `domain/run.rs:64,103,219,220,233,249` | `run_id`/`left_run_id`/`right_run_id`/`job_id` | `String` | `i64` (each) | | `domain/run.rs` (same file) | `phase_id` instances | `String` | **stays `String`** | | `domain/contracts.rs:32` | `requirement_id` | `String` | `i64` | | `domain/contracts.rs:35` | `phase_id` | `Option<String>` | **stays `Option<String>`** | ## 6. ID-generation strategy Today there are ~15 places that mint or derive IDs by `format!`. The migration introduces a small number of canonical mint sites in the store crate, and replaces every former `format!` site that produced a row PK with one of: - **`Database::lookup_or_create_session(channel: &str, user_key: &str) -> i64`** — replaces `format!("session:cli:{n}")`, `format!("session:admin:{n}")`, `format!("session:whatsapp:chat:{chat_id}")`, `format!("session:telegram:chat:{chat_id}")`, `format!("session:schedule:{schedule_name}")`. Inserts on miss using `INSERT ... RETURNING id`. The `(channel, user_key)` pair is stored on the row and reads use it directly instead of parsing the id string. - **`Database::next_autonomy_run_id() -> i64`** — replaces wherever `requested_autonomy_run_id` was injected as a string. Implementation: `INSERT INTO autonomy_runs(...) RETURNING id`. - **`Database::next_conversation_id() -> i64`** — replaces `format!("run:{run_id}")` insertions into `conversations`. Same `INSERT ... RETURNING id` pattern. Idempotency-scope strings (e.g. `format!("run:{run_id}:phase:runtime:attempt:1")`) keep using `format!`; the substituted `run_id` is now `i64`. The resulting string is still used as a string key in `tool_invocations` etc. Graph-edge labels (`db/work_history.rs:50`, `db/conversation_graph.rs:93,102` doing `format!("run:{}", run_id)`) keep being strings — they're not row PKs, they're typed edge tags in a graph store. Those tags now embed an integer instead of a UUID. The string `"run:42"` is just as good as the string `"run:UUID-..."` for graph-edge purposes. ## 7. RPC handlers (`hero_shrimp_server`) `crates/hero_shrimp_server/src/rpc/jsonrpc.rs` — single 5600-line file. - Add helper `optional_i64(params: &Value, key: &str) -> Option<i64>` next to existing `optional_string`. Same for `require_i64`. - Replace every `optional_string(params, "session_id" | "run_id" | "conversation_id" | "subscription_id")` with the integer variant. - Methods touched (full list): `method_session_create`, `method_session_close`, `method_session_list`, `method_session_subscribe`, `method_session_cancel`, `method_message_send`, `method_message_list`, `method_audit_list`, `method_tool_call`, `method_run_status`, `method_run_files`, `method_run_file`, `method_run_execution_control`, `method_run_rollback_file`, `method_run_commit`, `method_run_bundle`, `method_run_workspace_bundle`, `method_doai_start`, `method_doai_follow_up`, `method_doai_status`. - Result-building `serde_json::json!({...})` macros need no change beyond passing the integer values — `serde_json` serializes `i64` as a JSON number automatically. - Tests in this file that hardcode `"session:cli:1"` etc. need updating. ## 8. Engine (`hero_shrimp_engine`) Per-file change list (these are the *intent* edits — the rest is mechanical): - `src/engine_client/in_process.rs:56` — replace `SessionId::new(format!(...))` with a call to `db.lookup_or_create_session(channel, user_key) -> SessionId(i64)`. - `src/pipeline.rs:29,63,67,296,300` — every `format!("session:...")` and `format!("run:...")` site: drop the format if it produced a row PK; keep it if it produced an idempotency-scope string with the inner ID now `i64`. - `src/autonomy.rs:177-212` — `requested_autonomy_run_id(&ToolContext) -> Option<i64>` (was `Option<String>`); parse the integer instead of the string. - `src/autonomy/persistence.rs:120,222`, `src/autonomy/promote.rs:183,188`, `src/subagents.rs:306,326,356`, `src/agent/mod.rs:166`, `src/execution_control.rs:1175,1179,1684,1726` — all idempotency/graph-label format strings; just propagate the type change of the embedded ID. - `src/gateways/whatsapp.rs:314`, `src/gateways/telegram.rs:490`, `src/schedules.rs:158` — channel-specific session creation: replace the `format!` mint with `db.lookup_or_create_session(channel, user_key)`. - `src/gateways/admin/filters.rs:129` — `from == format!("run:{run_id}")` graph-walk match: `run_id` is now `i64`, the format produces `"run:42"`, comparison still works structurally — but **audit this site**: it relies on the string-id format being predictable. - `src/gateways/admin/routes/chat.rs:139` — same. - Every `EngineClient` trait method whose signature mentions `SessionId` / `RunId` — signature change is automatic once the newtype's inner changes; affects in-process and any out-of-process implementations. ## 9. CLI (`hero_shrimp` binary — manager + client) - `src/main.rs:151` — `Cancel { session_id: String }` → `i64`. Change clap to `value_parser = clap::value_parser!(i64)`. - `src/main.rs:205,272-294,317,326` — `session_id` plumbing through `run_send`/`run_cancel`. Type signatures change; print sites use `Display` (already implemented on the newtype). - `src/tui.rs:117,140,169,213,223,310` — `AppState.session_id: String → i64`; format strings using `session_id` need `.to_string()` or rely on `Display`. ## 10. UI (`hero_shrimp_ui`) - `src/routes.rs:252,287` — `RunFileDownloadQuery { run_id: String }` and `RunDownloadQuery { run_id: String }` → `i64`. Axum's `Query<>` + serde will deserialize numeric query strings as `i64` if the target field is `i64`. - `src/routes.rs:269,300,325` — JSON forwarding to backend; serde handles the type switch. - `static/js/dashboard.js:850` — **breaking change**: `conversation_id: \`ui-${kind}-${nonce}\`` is a client-supplied string. After migration, the UI must call `session.create` *without* a `conversation_id` and use the integer the server returns. - `static/js/dashboard.js` other sites — IDs flow through as JS numbers (JSON numbers parse to JS numbers); call sites pass them back unchanged. Audit any `===` comparisons that today assume string equality (none found in survey, but worth a grep before merging). - Templates (`templates/*.html`) — none interpolate IDs directly. No change. ## 11. SDK (`hero_shrimp_sdk`) Auto-generated from `openrpc.json` via `openrpc_client!` macro. Once `openrpc.json` is updated, the SDK regenerates automatically on next `cargo build`. **No manual edits.** Consumers of the SDK in `crates/hero_shrimp_examples/` will fail to compile until their call sites are fixed; treat as part of the migration. ## 12. Test plan 1. **Schema unit tests** — every `db/*.rs` module has `mod tests` (~20 fixture sites use hardcoded string IDs like `"run:test-1"`). Update each to use `i64` literals from freshly-minted IDs (`INSERT ... RETURNING`) or fixed integer literals. 2. **Wire round-trip tests** — `crates/hero_shrimp_examples/tests/integration.rs::test_openrpc_spec_valid` already verifies the spec parses. Add: walk every method, assert that ID-typed params/results are `"type": "integer"`. 3. **End-to-end** — once a daemon-startup integration test lands, that test exercises `session.create → message.send → session.close` with real integer IDs over HTTP/UDS. 4. **CLI smoke** — `hero_shrimp send "hi"` must complete; `hero_shrimp cancel <int>` must accept an integer. 5. **UI smoke** — start the service, open the dashboard, click through to memory and runs panes; verify nothing displays "[object Object]" or "NaN". ## 13. Rollout (wipe-and-replace) Strict ordering: 1. **PR opened on `development_id`** with all changes squashed. 2. **Reviewer runs locally**: `service_shrimp start --reset --clear`. The `--clear` wipes the existing `default.db` whose schema is incompatible. 3. After review, merge `development_id` → `development`. 4. Communicate to anyone running hero_shrimp: **first start after pulling requires `--clear`**. Any saved sessions / runs / memories from before are gone. The wipe is mandatory because the schema migration goes from `TEXT PRIMARY KEY` to `INTEGER PRIMARY KEY AUTOINCREMENT` — SQLite has no in-place ALTER for that. Adding a migration path was considered and rejected per Issue #15: data lifetime in this DB is short, value of preserving old rows is low. ## 14. Sanity checks called out for the reviewer 1. **Channel encoding leakage.** `format!("session:cli:{n}")` today encodes the channel name in the ID string. The `sessions.channel` column already stores it separately, so the migration preserves the information — but every code site that today does `if id.starts_with("session:cli:")` (or grep equivalents) must be replaced by reading `.channel` from the row. Survey lists call sites; PR must address every one. 2. **Two `SessionId` types.** Don't leave both in the tree post-migration. If domain layer needs richer invariants, that's a separate refactor — the migration's job is to unify on the integer wire type. 3. **`requirement_id` in `domain::contracts`.** Migrating to `i64` along with everything else (decided 2026-04-30 in this thread). If `requirement_id` is currently a runtime-only field with no SQLite table backing it, the change is purely type-level — no schema work. If a `requirements` table is later added, give it `INTEGER PRIMARY KEY AUTOINCREMENT` from the start. 4. **`phase_id` stays `String`.** No `phases` table exists. If the team later adds one with an `INTEGER PRIMARY KEY`, do that change in a separate PR — it's larger than it looks because `phase_id` is referenced from many FK columns. 5. **Out-of-process `EngineClient` impls.** Survey only inspected `InProcessEngineClient`. If any out-of-process implementation exists outside this repo, it needs updating in lockstep. 6. **Tests that hardcode string IDs.** ~20 sites. Use `rg -n '"(run|session|flow):[a-z0-9_:-]*"' crates/` before declaring done. ## 15. Estimated PR size - `hero_shrimp_types`: ~5 files, ~50 LOC delta + openrpc.json changes - `hero_shrimp_store`: ~30 files, ~600 LOC delta (mostly mechanical query/RowMapper updates) - `hero_shrimp_engine`: ~15 files, ~80 LOC delta (most call sites are leaf format!s that stay) - `hero_shrimp_server`: 1 file (`rpc/jsonrpc.rs`), ~150 LOC delta - `hero_shrimp` (CLI): 2 files, ~20 LOC - `hero_shrimp_ui`: 2 files (`routes.rs`, `dashboard.js`), ~10 LOC - Tests: ~20 fixture updates across the workspace - SDK: 0 manual edits (regenerated) Total: ~900 LOC, single squashed PR.

thabeta commented

2026-04-30 09:01:03 +00:00

Owner

Implementation Spec for Issue #15

Objective

Migrate internal database and API identifiers for sessions and runs from text IDs to autoincrementing integer IDs, using a wipe-and-replace rollout with no data migration. Preserve string IDs for natural, external, or scoped identifiers such as ClientId, PhaseId, external IDs, cache keys, dedup keys, conversation IDs, user IDs, and channel names.

Requirements

SQLite primary keys for sessions, autonomy_runs, run_briefs, and run_state_snapshots must become INTEGER PRIMARY KEY AUTOINCREMENT.
SessionId and RunId must serialize as JSON integers and map cleanly to SQLite i64.
PhaseId must remain a string.
Duplicate SessionId definitions in domain::ids and proto::ids must be unified.
OpenRPC session_id, run_id, and related result fields must use integer with format: int64.
Existing natural string identifiers must stay string typed, including ClientId, PhaseId, conversation_id, external_id, idempotency_key, cache_key, dedup_key, from_user, user_id, and channel names.
RPC handlers must parse integer IDs with as_i64() instead of as_str() for session/run IDs.
CLI, TUI, UI JavaScript, and SDK examples must treat session/run IDs as numbers where the API defines them as numeric.
Rollout must support an explicit wipe path because old DB schemas are incompatible.
Tests must be updated to pin numeric wire format and integer DB schemas.
Current local uncommitted changes exist in crates/hero_shrimp/src/main.rs, crates/hero_shrimp/src/tui.rs, crates/hero_shrimp_server/src/rpc/jsonrpc.rs, and scripts/service_shrimp.nu; implementation should avoid overwriting unrelated edits in those files.

Files to Modify/Create

crates/hero_shrimp_types/src/domain/ids.rs
crates/hero_shrimp_types/src/proto/ids.rs
crates/hero_shrimp_types/src/proto/rpc.rs
crates/hero_shrimp_types/src/proto/events.rs
crates/hero_shrimp_types/src/domain/state_store.rs
crates/hero_shrimp_types/src/domain/run.rs
crates/hero_shrimp_types/src/openrpc.json
crates/hero_shrimp_types/src/openrpc.client.generated.rs
crates/hero_shrimp_store/src/db/schema.rs
crates/hero_shrimp_store/src/db/sessions.rs
crates/hero_shrimp_store/src/db/autonomy_runs.rs
crates/hero_shrimp_store/src/db/run_briefs.rs
crates/hero_shrimp_store/src/db/run_state.rs
crates/hero_shrimp_store/src/db/artifacts.rs
crates/hero_shrimp_store/src/db/types.rs
crates/hero_shrimp_store/src/db/work_history.rs
crates/hero_shrimp_store/src/runtime_artifacts.rs
crates/hero_shrimp_store/src/runtime_context.rs
crates/hero_shrimp_store/src/events.rs
crates/hero_shrimp_store/src/inbound.rs
crates/hero_shrimp_engine/src/engine_client/mod.rs
crates/hero_shrimp_engine/src/engine_client/in_process.rs
crates/hero_shrimp_engine/src/engine_client/rpc_client.rs
crates/hero_shrimp_engine/src/runtime_state.rs
crates/hero_shrimp_engine/src/types.rs
crates/hero_shrimp_engine/src/pipeline.rs
crates/hero_shrimp_engine/src/autonomy.rs
crates/hero_shrimp_engine/src/agent/context.rs
crates/hero_shrimp_engine/src/agent/persistence.rs
crates/hero_shrimp_server/src/lib.rs
crates/hero_shrimp_server/src/main.rs
crates/hero_shrimp_server/src/rpc/jsonrpc.rs
crates/hero_shrimp/src/main.rs
crates/hero_shrimp/src/tui.rs
crates/hero_shrimp/src/service.rs
crates/hero_shrimp_ui/static/js/dashboard.js
crates/hero_shrimp_sdk/src/lib.rs
crates/hero_shrimp_examples/examples/basic_usage.rs
crates/hero_shrimp_examples/tests/integration.rs
crates/hero_shrimp_types/src/proto/tests.rs
crates/hero_shrimp_store/src/db/tests/core.rs
Relevant crates/**/tests/*.rs files that currently assert string session/run IDs

Implementation Plan

Step 1: Unify ID Types and Numeric Serialization

Files: crates/hero_shrimp_types/src/domain/ids.rs, crates/hero_shrimp_types/src/proto/ids.rs, crates/hero_shrimp_types/src/proto/rpc.rs, crates/hero_shrimp_types/src/proto/events.rs, crates/hero_shrimp_types/src/domain/state_store.rs, crates/hero_shrimp_types/src/domain/run.rs

Replace SessionId(pub String) and RunId(pub String) with integer-backed newtypes, preferably i64 to match SQLite and OpenRPC int64.
Keep PhaseId(pub String) unchanged.
Remove the duplicate proto-level SessionId implementation by re-exporting or importing the domain SessionId.
Update constructors, display helpers, conversions, serde behavior, and hash/map usage for integer IDs.
Replace as_str() callers for numeric IDs with value(), as_i64(), or to_string() only at file/path/logging boundaries.
Keep ClientId as string in proto::ids.
Dependencies: none

Step 2: Change Core SQLite Schemas to Autoincrement Integers

Files: crates/hero_shrimp_store/src/db/sessions.rs, crates/hero_shrimp_store/src/db/autonomy_runs.rs, crates/hero_shrimp_store/src/db/run_briefs.rs, crates/hero_shrimp_store/src/db/run_state.rs, crates/hero_shrimp_store/src/db/artifacts.rs, crates/hero_shrimp_store/src/db/types.rs, crates/hero_shrimp_store/src/db/schema.rs

Change sessions.id from TEXT PRIMARY KEY to INTEGER PRIMARY KEY AUTOINCREMENT.
Change autonomy_runs.id from TEXT PRIMARY KEY to INTEGER PRIMARY KEY AUTOINCREMENT.
Change run_briefs.run_id and run_state_snapshots.run_id to integer references keyed by run ID.
Change phase_artifacts.run_id to integer while keeping phase_artifacts.phase_id TEXT.
Update row structs so DB-owned session/run IDs are i64.
Keep string columns for parent_run_id only if it is truly an external/natural link; otherwise convert it to nullable integer consistently with autonomy_runs.id.
Preserve string columns for phase_id, source_phase_id, conversation_id, related_run_id if they are external/natural or workspace-facing identifiers.
Dependencies: Step 1

Step 3: Add Explicit Wipe-and-Replace Startup Path

Files: crates/hero_shrimp_server/src/lib.rs, crates/hero_shrimp_server/src/main.rs, crates/hero_shrimp/src/main.rs, crates/hero_shrimp/src/service.rs

Add explicit --clear handling for startup rollout.
Ensure hero_shrimp --start --clear removes the configured SQLite database before registering/restarting services.
Ensure direct hero_shrimp_server --clear daemon startup also clears the DB before Database::open.
Do not add schema migrations for old text-ID databases.
Make accidental wipes impossible without the explicit flag.
Dependencies: Step 2

Step 4: Allocate Session IDs from SQLite

Files: crates/hero_shrimp_store/src/db/sessions.rs, crates/hero_shrimp_engine/src/engine_client/in_process.rs, crates/hero_shrimp_engine/src/engine_client/mod.rs, crates/hero_shrimp_server/src/rpc/jsonrpc.rs

Replace timestamp/channel-derived session IDs like session:cli:<nonce> with SQLite-created integer IDs.
Add a DB method that inserts a new session row and returns its autoincrement ID.
Update InProcessEngineClient::create_session to allocate through the DB instead of generating a string.
Keep channel, user_key, and metadata separate so channel values do not leak into encoded session IDs.
Update session lookup, close, list, reaper, and in-memory maps to use integer SessionId.
Dependencies: Step 2

Step 5: Allocate Run IDs from SQLite and Preserve Artifact Path Compatibility

Files: crates/hero_shrimp_store/src/db/autonomy_runs.rs, crates/hero_shrimp_store/src/db/run_briefs.rs, crates/hero_shrimp_store/src/db/run_state.rs, crates/hero_shrimp_store/src/runtime_artifacts.rs, crates/hero_shrimp_engine/src/autonomy.rs, crates/hero_shrimp_server/src/rpc/jsonrpc.rs

Replace string-generated persisted run IDs with SQLite autoincrement IDs.
Update AutonomyRunRow, RunBriefRow, and RunStateSnapshotRow to use integer run IDs.
Convert numeric run IDs to strings only when deriving workspace or artifact directory names.
Keep artifact_run_id as a separate string only where it refers to an existing filesystem/live-run directory identifier.
Ensure live run reconciliation can match persisted numeric IDs to live workspace metadata without confusing DB IDs with artifact directory names.
Dependencies: Step 2

Step 6: Update RPC Parsing and Responses

Files: crates/hero_shrimp_server/src/rpc/jsonrpc.rs

Replace string extraction for numeric fields with integer extraction for session_id and run_id.
Update session.create, session.close, session.cancel, session.subscribe, message.send, audit.list, tool.call, doai.start, doai.follow_up, doai.status, run.cancel, run.files, run.file, run.execution_control, run.rollback_file, run.commit, run.bundle, and run.workspace_bundle.
Return numeric session_id, run_id, and row id fields where the ID is DB-owned.
Keep phase_id, conversation_id, artifact_run_id, cache keys, dedup keys, and external IDs as strings.
Audit helper functions such as optional_string and add numeric equivalents where needed.
Dependencies: Step 4, Step 5

Step 7: Update OpenRPC and Generated SDK Surface

Files: crates/hero_shrimp_types/src/openrpc.json, crates/hero_shrimp_types/src/openrpc.client.generated.rs, crates/hero_shrimp_sdk/src/lib.rs, crates/hero_shrimp_examples/examples/basic_usage.rs, crates/hero_shrimp_examples/tests/integration.rs

Change OpenRPC schemas for session_id, run_id, DB row id, and subscription IDs from string to { "type": "integer", "format": "int64" }.
Keep phase_id, conversation_id, workspace_path, artifact_run_id, and natural/external identifiers as strings.
Regenerate or refresh the generated client inspection file so SDK input/output structs use integer types for numeric IDs.
Update SDK examples to store and pass session_id as an integer.
Add or update SDK/OpenRPC tests to assert numeric schema types.
Dependencies: Step 6

Step 8: Update CLI and TUI Clients

Files: crates/hero_shrimp/src/main.rs, crates/hero_shrimp/src/tui.rs

Change cancel <session> parsing from string to integer.
Extract session_id from session.create responses with numeric JSON access.
Send numeric session_id values in message.send, session.close, and session.cancel.
Update user-visible printing to format integer IDs cleanly.
Preserve existing Unix socket JSON-RPC behavior and avoid unrelated transport changes.
Dependencies: Step 7

Step 9: Update Admin UI JavaScript

Files: crates/hero_shrimp_ui/static/js/dashboard.js

Treat session_id and run_id as numbers in RPC params.
Convert IDs to strings only for display, compaction, DOM attributes, URL query params, and file path labels.
Audit helpers such as runId, runSessionId, runArtifactRunId, selectRun, console session handling, cancel/follow-up/run file actions, and diagnostics.
Keep phase_id command handling as string.
Dependencies: Step 7

Step 10: Update Engine and Store ID Plumbing

Files: crates/hero_shrimp_engine/src/runtime_state.rs, crates/hero_shrimp_engine/src/types.rs, crates/hero_shrimp_engine/src/pipeline.rs, crates/hero_shrimp_engine/src/agent/context.rs, crates/hero_shrimp_engine/src/agent/persistence.rs, crates/hero_shrimp_store/src/runtime_context.rs, crates/hero_shrimp_store/src/events.rs, crates/hero_shrimp_store/src/inbound.rs, crates/hero_shrimp_store/src/db/work_history.rs

Update cancellation maps, tool context, runtime identity, audit options, inbound dedup scope, and pipeline request fields to carry numeric session/run IDs where DB-owned.
Preserve external_id and idempotency keys as strings.
Keep dedup keys stable by explicitly stringifying numeric session IDs only when constructing dedup key material.
Ensure audit filters use numeric session_id and run_id columns where converted.
Dependencies: Step 6

Step 11: Update Tests and Fixtures

Files: crates/hero_shrimp_types/src/proto/tests.rs, crates/hero_shrimp_store/src/db/tests/core.rs, relevant crates/**/tests/*.rs

Update protocol round-trip tests to assert numeric session JSON such as "session":1.
Update DB tests to assert integer IDs and autoincrement behavior.
Update work history and run brief tests to use numeric run IDs.
Add regression coverage that PhaseId stays string.
Add regression coverage that OpenRPC numeric IDs generate integer SDK fields.
Add regression coverage that --clear wipes old incompatible schemas only when explicitly requested.
Dependencies: Step 10

Step 12: Verification

Files: Cargo.toml, all modified crates

Run formatting.
Run cargo check --workspace.
Run targeted tests for hero_shrimp_types, hero_shrimp_store, hero_shrimp_engine, hero_shrimp_server, hero_shrimp, SDK examples, and UI-adjacent RPC behavior.
Manually verify hero_shrimp --start --clear, hero_shrimp send, hero_shrimp cancel <numeric-session-id>, session.create, message.send, doai.start, doai.status, run.status, and UI run selection.
Dependencies: Step 11

Acceptance Criteria

New databases create sessions.id and autonomy_runs.id as INTEGER PRIMARY KEY AUTOINCREMENT.
No migration code attempts to preserve or transform old text-ID data.
hero_shrimp --start --clear and direct server clear startup paths wipe incompatible old DBs explicitly.
SessionId exists as one shared type and serializes as a JSON integer.
RunId serializes as a JSON integer.
PhaseId remains string typed in Rust, SQLite, OpenRPC, RPC handlers, and UI code.
OpenRPC and generated SDK structs use integer fields for session_id and run_id.
RPC handlers reject string session_id and run_id values for endpoints whose schema is now integer.
CLI, TUI, and UI can create sessions, send messages, cancel sessions, start DoAI runs, inspect run status, and read run files with numeric IDs.
Audit filtering, tool contexts, cancellation, and dedup logic do not encode channel names into session IDs.
Tests pass across affected crates.

Notes

Current local uncommitted changes are present in crates/hero_shrimp/src/main.rs, crates/hero_shrimp/src/tui.rs, crates/hero_shrimp_server/src/rpc/jsonrpc.rs, and scripts/service_shrimp.nu.
The most important risk is confusing numeric DB run IDs with string filesystem artifact IDs. Keep artifact_run_id explicit where filesystem paths or live workspace IDs still require strings.
The second major risk is accidentally converting natural identifiers. phase_id, conversation_id, ClientId, external_id, cache_key, dedup_key, from_user, and user_id should remain strings.
The existing code currently generates session IDs with channel text embedded in them; the implementation should remove that pattern and keep channel as its own field.

## Implementation Spec for Issue #15 ### Objective Migrate internal database and API identifiers for sessions and runs from text IDs to autoincrementing integer IDs, using a wipe-and-replace rollout with no data migration. Preserve string IDs for natural, external, or scoped identifiers such as `ClientId`, `PhaseId`, external IDs, cache keys, dedup keys, conversation IDs, user IDs, and channel names. ### Requirements - SQLite primary keys for `sessions`, `autonomy_runs`, `run_briefs`, and `run_state_snapshots` must become `INTEGER PRIMARY KEY AUTOINCREMENT`. - `SessionId` and `RunId` must serialize as JSON integers and map cleanly to SQLite `i64`. - `PhaseId` must remain a string. - Duplicate `SessionId` definitions in `domain::ids` and `proto::ids` must be unified. - OpenRPC `session_id`, `run_id`, and related result fields must use `integer` with `format: int64`. - Existing natural string identifiers must stay string typed, including `ClientId`, `PhaseId`, `conversation_id`, `external_id`, `idempotency_key`, `cache_key`, `dedup_key`, `from_user`, `user_id`, and channel names. - RPC handlers must parse integer IDs with `as_i64()` instead of `as_str()` for session/run IDs. - CLI, TUI, UI JavaScript, and SDK examples must treat session/run IDs as numbers where the API defines them as numeric. - Rollout must support an explicit wipe path because old DB schemas are incompatible. - Tests must be updated to pin numeric wire format and integer DB schemas. - Current local uncommitted changes exist in `crates/hero_shrimp/src/main.rs`, `crates/hero_shrimp/src/tui.rs`, `crates/hero_shrimp_server/src/rpc/jsonrpc.rs`, and `scripts/service_shrimp.nu`; implementation should avoid overwriting unrelated edits in those files. ### Files to Modify/Create - `crates/hero_shrimp_types/src/domain/ids.rs` - `crates/hero_shrimp_types/src/proto/ids.rs` - `crates/hero_shrimp_types/src/proto/rpc.rs` - `crates/hero_shrimp_types/src/proto/events.rs` - `crates/hero_shrimp_types/src/domain/state_store.rs` - `crates/hero_shrimp_types/src/domain/run.rs` - `crates/hero_shrimp_types/src/openrpc.json` - `crates/hero_shrimp_types/src/openrpc.client.generated.rs` - `crates/hero_shrimp_store/src/db/schema.rs` - `crates/hero_shrimp_store/src/db/sessions.rs` - `crates/hero_shrimp_store/src/db/autonomy_runs.rs` - `crates/hero_shrimp_store/src/db/run_briefs.rs` - `crates/hero_shrimp_store/src/db/run_state.rs` - `crates/hero_shrimp_store/src/db/artifacts.rs` - `crates/hero_shrimp_store/src/db/types.rs` - `crates/hero_shrimp_store/src/db/work_history.rs` - `crates/hero_shrimp_store/src/runtime_artifacts.rs` - `crates/hero_shrimp_store/src/runtime_context.rs` - `crates/hero_shrimp_store/src/events.rs` - `crates/hero_shrimp_store/src/inbound.rs` - `crates/hero_shrimp_engine/src/engine_client/mod.rs` - `crates/hero_shrimp_engine/src/engine_client/in_process.rs` - `crates/hero_shrimp_engine/src/engine_client/rpc_client.rs` - `crates/hero_shrimp_engine/src/runtime_state.rs` - `crates/hero_shrimp_engine/src/types.rs` - `crates/hero_shrimp_engine/src/pipeline.rs` - `crates/hero_shrimp_engine/src/autonomy.rs` - `crates/hero_shrimp_engine/src/agent/context.rs` - `crates/hero_shrimp_engine/src/agent/persistence.rs` - `crates/hero_shrimp_server/src/lib.rs` - `crates/hero_shrimp_server/src/main.rs` - `crates/hero_shrimp_server/src/rpc/jsonrpc.rs` - `crates/hero_shrimp/src/main.rs` - `crates/hero_shrimp/src/tui.rs` - `crates/hero_shrimp/src/service.rs` - `crates/hero_shrimp_ui/static/js/dashboard.js` - `crates/hero_shrimp_sdk/src/lib.rs` - `crates/hero_shrimp_examples/examples/basic_usage.rs` - `crates/hero_shrimp_examples/tests/integration.rs` - `crates/hero_shrimp_types/src/proto/tests.rs` - `crates/hero_shrimp_store/src/db/tests/core.rs` - Relevant `crates/**/tests/*.rs` files that currently assert string session/run IDs ### Implementation Plan #### Step 1: Unify ID Types and Numeric Serialization Files: `crates/hero_shrimp_types/src/domain/ids.rs`, `crates/hero_shrimp_types/src/proto/ids.rs`, `crates/hero_shrimp_types/src/proto/rpc.rs`, `crates/hero_shrimp_types/src/proto/events.rs`, `crates/hero_shrimp_types/src/domain/state_store.rs`, `crates/hero_shrimp_types/src/domain/run.rs` - Replace `SessionId(pub String)` and `RunId(pub String)` with integer-backed newtypes, preferably `i64` to match SQLite and OpenRPC `int64`. - Keep `PhaseId(pub String)` unchanged. - Remove the duplicate proto-level `SessionId` implementation by re-exporting or importing the domain `SessionId`. - Update constructors, display helpers, conversions, serde behavior, and hash/map usage for integer IDs. - Replace `as_str()` callers for numeric IDs with `value()`, `as_i64()`, or `to_string()` only at file/path/logging boundaries. - Keep `ClientId` as string in `proto::ids`. Dependencies: none #### Step 2: Change Core SQLite Schemas to Autoincrement Integers Files: `crates/hero_shrimp_store/src/db/sessions.rs`, `crates/hero_shrimp_store/src/db/autonomy_runs.rs`, `crates/hero_shrimp_store/src/db/run_briefs.rs`, `crates/hero_shrimp_store/src/db/run_state.rs`, `crates/hero_shrimp_store/src/db/artifacts.rs`, `crates/hero_shrimp_store/src/db/types.rs`, `crates/hero_shrimp_store/src/db/schema.rs` - Change `sessions.id` from `TEXT PRIMARY KEY` to `INTEGER PRIMARY KEY AUTOINCREMENT`. - Change `autonomy_runs.id` from `TEXT PRIMARY KEY` to `INTEGER PRIMARY KEY AUTOINCREMENT`. - Change `run_briefs.run_id` and `run_state_snapshots.run_id` to integer references keyed by run ID. - Change `phase_artifacts.run_id` to integer while keeping `phase_artifacts.phase_id TEXT`. - Update row structs so DB-owned session/run IDs are `i64`. - Keep string columns for `parent_run_id` only if it is truly an external/natural link; otherwise convert it to nullable integer consistently with `autonomy_runs.id`. - Preserve string columns for `phase_id`, `source_phase_id`, `conversation_id`, `related_run_id` if they are external/natural or workspace-facing identifiers. Dependencies: Step 1 #### Step 3: Add Explicit Wipe-and-Replace Startup Path Files: `crates/hero_shrimp_server/src/lib.rs`, `crates/hero_shrimp_server/src/main.rs`, `crates/hero_shrimp/src/main.rs`, `crates/hero_shrimp/src/service.rs` - Add explicit `--clear` handling for startup rollout. - Ensure `hero_shrimp --start --clear` removes the configured SQLite database before registering/restarting services. - Ensure direct `hero_shrimp_server --clear` daemon startup also clears the DB before `Database::open`. - Do not add schema migrations for old text-ID databases. - Make accidental wipes impossible without the explicit flag. Dependencies: Step 2 #### Step 4: Allocate Session IDs from SQLite Files: `crates/hero_shrimp_store/src/db/sessions.rs`, `crates/hero_shrimp_engine/src/engine_client/in_process.rs`, `crates/hero_shrimp_engine/src/engine_client/mod.rs`, `crates/hero_shrimp_server/src/rpc/jsonrpc.rs` - Replace timestamp/channel-derived session IDs like `session:cli:<nonce>` with SQLite-created integer IDs. - Add a DB method that inserts a new session row and returns its autoincrement ID. - Update `InProcessEngineClient::create_session` to allocate through the DB instead of generating a string. - Keep `channel`, `user_key`, and metadata separate so channel values do not leak into encoded session IDs. - Update session lookup, close, list, reaper, and in-memory maps to use integer `SessionId`. Dependencies: Step 2 #### Step 5: Allocate Run IDs from SQLite and Preserve Artifact Path Compatibility Files: `crates/hero_shrimp_store/src/db/autonomy_runs.rs`, `crates/hero_shrimp_store/src/db/run_briefs.rs`, `crates/hero_shrimp_store/src/db/run_state.rs`, `crates/hero_shrimp_store/src/runtime_artifacts.rs`, `crates/hero_shrimp_engine/src/autonomy.rs`, `crates/hero_shrimp_server/src/rpc/jsonrpc.rs` - Replace string-generated persisted run IDs with SQLite autoincrement IDs. - Update `AutonomyRunRow`, `RunBriefRow`, and `RunStateSnapshotRow` to use integer run IDs. - Convert numeric run IDs to strings only when deriving workspace or artifact directory names. - Keep `artifact_run_id` as a separate string only where it refers to an existing filesystem/live-run directory identifier. - Ensure live run reconciliation can match persisted numeric IDs to live workspace metadata without confusing DB IDs with artifact directory names. Dependencies: Step 2 #### Step 6: Update RPC Parsing and Responses Files: `crates/hero_shrimp_server/src/rpc/jsonrpc.rs` - Replace string extraction for numeric fields with integer extraction for `session_id` and `run_id`. - Update `session.create`, `session.close`, `session.cancel`, `session.subscribe`, `message.send`, `audit.list`, `tool.call`, `doai.start`, `doai.follow_up`, `doai.status`, `run.cancel`, `run.files`, `run.file`, `run.execution_control`, `run.rollback_file`, `run.commit`, `run.bundle`, and `run.workspace_bundle`. - Return numeric `session_id`, `run_id`, and row `id` fields where the ID is DB-owned. - Keep `phase_id`, `conversation_id`, `artifact_run_id`, cache keys, dedup keys, and external IDs as strings. - Audit helper functions such as `optional_string` and add numeric equivalents where needed. Dependencies: Step 4, Step 5 #### Step 7: Update OpenRPC and Generated SDK Surface Files: `crates/hero_shrimp_types/src/openrpc.json`, `crates/hero_shrimp_types/src/openrpc.client.generated.rs`, `crates/hero_shrimp_sdk/src/lib.rs`, `crates/hero_shrimp_examples/examples/basic_usage.rs`, `crates/hero_shrimp_examples/tests/integration.rs` - Change OpenRPC schemas for `session_id`, `run_id`, DB row `id`, and subscription IDs from `string` to `{ "type": "integer", "format": "int64" }`. - Keep `phase_id`, `conversation_id`, `workspace_path`, `artifact_run_id`, and natural/external identifiers as strings. - Regenerate or refresh the generated client inspection file so SDK input/output structs use integer types for numeric IDs. - Update SDK examples to store and pass `session_id` as an integer. - Add or update SDK/OpenRPC tests to assert numeric schema types. Dependencies: Step 6 #### Step 8: Update CLI and TUI Clients Files: `crates/hero_shrimp/src/main.rs`, `crates/hero_shrimp/src/tui.rs` - Change `cancel <session>` parsing from string to integer. - Extract `session_id` from `session.create` responses with numeric JSON access. - Send numeric `session_id` values in `message.send`, `session.close`, and `session.cancel`. - Update user-visible printing to format integer IDs cleanly. - Preserve existing Unix socket JSON-RPC behavior and avoid unrelated transport changes. Dependencies: Step 7 #### Step 9: Update Admin UI JavaScript Files: `crates/hero_shrimp_ui/static/js/dashboard.js` - Treat `session_id` and `run_id` as numbers in RPC params. - Convert IDs to strings only for display, compaction, DOM attributes, URL query params, and file path labels. - Audit helpers such as `runId`, `runSessionId`, `runArtifactRunId`, `selectRun`, console session handling, cancel/follow-up/run file actions, and diagnostics. - Keep `phase_id` command handling as string. Dependencies: Step 7 #### Step 10: Update Engine and Store ID Plumbing Files: `crates/hero_shrimp_engine/src/runtime_state.rs`, `crates/hero_shrimp_engine/src/types.rs`, `crates/hero_shrimp_engine/src/pipeline.rs`, `crates/hero_shrimp_engine/src/agent/context.rs`, `crates/hero_shrimp_engine/src/agent/persistence.rs`, `crates/hero_shrimp_store/src/runtime_context.rs`, `crates/hero_shrimp_store/src/events.rs`, `crates/hero_shrimp_store/src/inbound.rs`, `crates/hero_shrimp_store/src/db/work_history.rs` - Update cancellation maps, tool context, runtime identity, audit options, inbound dedup scope, and pipeline request fields to carry numeric session/run IDs where DB-owned. - Preserve `external_id` and idempotency keys as strings. - Keep dedup keys stable by explicitly stringifying numeric session IDs only when constructing dedup key material. - Ensure audit filters use numeric `session_id` and `run_id` columns where converted. Dependencies: Step 6 #### Step 11: Update Tests and Fixtures Files: `crates/hero_shrimp_types/src/proto/tests.rs`, `crates/hero_shrimp_store/src/db/tests/core.rs`, relevant `crates/**/tests/*.rs` - Update protocol round-trip tests to assert numeric session JSON such as `"session":1`. - Update DB tests to assert integer IDs and autoincrement behavior. - Update work history and run brief tests to use numeric run IDs. - Add regression coverage that `PhaseId` stays string. - Add regression coverage that OpenRPC numeric IDs generate integer SDK fields. - Add regression coverage that `--clear` wipes old incompatible schemas only when explicitly requested. Dependencies: Step 10 #### Step 12: Verification Files: `Cargo.toml`, all modified crates - Run formatting. - Run `cargo check --workspace`. - Run targeted tests for `hero_shrimp_types`, `hero_shrimp_store`, `hero_shrimp_engine`, `hero_shrimp_server`, `hero_shrimp`, SDK examples, and UI-adjacent RPC behavior. - Manually verify `hero_shrimp --start --clear`, `hero_shrimp send`, `hero_shrimp cancel <numeric-session-id>`, `session.create`, `message.send`, `doai.start`, `doai.status`, `run.status`, and UI run selection. Dependencies: Step 11 ### Acceptance Criteria - New databases create `sessions.id` and `autonomy_runs.id` as `INTEGER PRIMARY KEY AUTOINCREMENT`. - No migration code attempts to preserve or transform old text-ID data. - `hero_shrimp --start --clear` and direct server clear startup paths wipe incompatible old DBs explicitly. - `SessionId` exists as one shared type and serializes as a JSON integer. - `RunId` serializes as a JSON integer. - `PhaseId` remains string typed in Rust, SQLite, OpenRPC, RPC handlers, and UI code. - OpenRPC and generated SDK structs use integer fields for `session_id` and `run_id`. - RPC handlers reject string `session_id` and `run_id` values for endpoints whose schema is now integer. - CLI, TUI, and UI can create sessions, send messages, cancel sessions, start DoAI runs, inspect run status, and read run files with numeric IDs. - Audit filtering, tool contexts, cancellation, and dedup logic do not encode channel names into session IDs. - Tests pass across affected crates. ### Notes - Current local uncommitted changes are present in `crates/hero_shrimp/src/main.rs`, `crates/hero_shrimp/src/tui.rs`, `crates/hero_shrimp_server/src/rpc/jsonrpc.rs`, and `scripts/service_shrimp.nu`. - The most important risk is confusing numeric DB run IDs with string filesystem artifact IDs. Keep `artifact_run_id` explicit where filesystem paths or live workspace IDs still require strings. - The second major risk is accidentally converting natural identifiers. `phase_id`, `conversation_id`, `ClientId`, `external_id`, `cache_key`, `dedup_key`, `from_user`, and `user_id` should remain strings. - The existing code currently generates session IDs with channel text embedded in them; the implementation should remove that pattern and keep channel as its own field.