lhumina_code/hero_slides
22
0
Fork 0
Code Issues 6 Pull requests Projects Releases Packages Wiki Activity Actions

default scan directory #12

New issue
Open
opened 2026-04-14 04:23:48 +00:00 by despiegk · 4 comments
despiegk commented 2026-04-14 04:23:48 +00:00
Owner
Copy link

image

look if $CODEROOT/ourworld_slides exist if yes auto scan this one, if nothing scanned yet
CODEROOT comes from the secrets

use skill /hero_proc_secrets to get the CODEROOT

remember the last scan in our hero_db
see skill /hero_db

so when we start again we check, what was last scan, if dir exists, load that one
if that dir does not exist, remove the last scan and check $CODEROOT/ourworld_slides exist exists

![image](/attachments/57a3db4c-9dfb-4d7c-9c6d-ea2685b832f3) look if $CODEROOT/ourworld_slides exist if yes auto scan this one, if nothing scanned yet CODEROOT comes from the secrets use skill /hero_proc_secrets to get the CODEROOT remember the last scan in our hero_db see skill /hero_db so when we start again we check, what was last scan, if dir exists, load that one if that dir does not exist, remove the last scan and check $CODEROOT/ourworld_slides exist exists
image.png
198 KiB
despiegk commented 2026-04-14 06:01:56 +00:00
Author
Owner
Copy link

Implementation Spec — Issue #12: Default Scan Directory

Objective

On server startup, hero_slides_server should automatically resolve a default scan directory and perform an initial deck.scan without any user interaction. The logic uses two persistence mechanisms:

  1. hero_proc secrets — to read CODEROOT (the code workspace root).
  2. hero_db (Redis) — to store and recall the last successfully scanned directory path.

The fallback directory is $CODEROOT/ourworld_slides. On each startup the server checks the remembered path first; if it still exists the scan proceeds immediately; if it is gone the stored entry is cleared and the fallback path is tried instead.

Requirements

  • On startup, hero_slides_server must attempt an automatic scan before accepting user-initiated requests.
  • The server connects to hero_proc via its Unix socket and calls secret.get with key coderoot (context "core") to obtain CODEROOT.
  • After obtaining CODEROOT, the fallback candidate is $CODEROOT/ourworld_slides.
  • The server reads key hero_slides:last_scan_path from hero_db (Redis) to recall the previously successful scan root.
  • Decision logic:
    • If last_scan_path exists in Redis AND that directory is present on disk → scan that path.
    • If last_scan_path exists in Redis BUT that directory is missing on disk → delete the Redis key, then try the CODEROOT fallback.
    • If there is no last_scan_path in Redis → try the CODEROOT fallback.
    • CODEROOT fallback: if $CODEROOT/ourworld_slides is a directory → scan it and save the path to Redis.
    • If neither path is available → log a warning and start with an empty deck list (no error, server starts normally).
  • A successful scan must update hero_slides:last_scan_path in Redis.
  • The auto-scan must not block server startup — it runs as a background task after the Axum socket is bound.
  • Any failure to reach hero_proc or hero_db must be logged at warn level and silently skipped; the server must not fail to start.

Files to Modify / Create

  • crates/hero_slides_server/src/startup.rs — Create: Contains auto_scan_on_startup() and all helpers for secret fetching, Redis persistence, and path resolution.
  • crates/hero_slides_server/src/main.rs — Modify: Add mod startup, make socket_base_dir() pub(crate), spawn background auto-scan task after binding socket.
  • crates/hero_slides_server/Cargo.toml — Modify: Add redis = { version = "0.27", features = ["tokio-comp", "aio"] }.

Implementation Plan

Step 1 — Add Redis dependency to Cargo.toml

Files: crates/hero_slides_server/Cargo.toml
Dependencies: none

Add redis = { version = "0.27", features = ["tokio-comp", "aio"] } to [dependencies].

Step 2 — Create startup.rs

Files: crates/hero_slides_server/src/startup.rs
Dependencies: Step 1

Create module with:

  • fetch_coderoot() -> Option<String> — connects to hero_proc, calls secret.get with key "coderoot", context "core".
  • redis_socket_path() -> PathBuf — returns $HERO_DB_SOCKET or socket_base_dir()/hero_db/redis.sock.
  • read_last_scan_path() -> Option<String> — GET hero_slides:last_scan_path from Redis.
  • save_last_scan_path(path: &str) — SET hero_slides:last_scan_path in Redis.
  • delete_last_scan_path() — DEL hero_slides:last_scan_path from Redis.
  • do_scan(root: &Path, state: &Arc<ServerState>) -> bool — calls hero_slides_lib::scan_decks(), upserts results into state.
  • pub async fn auto_scan_on_startup(state: Arc<ServerState>) — orchestrates the full decision tree.

Step 3 — Wire into main.rs

Files: crates/hero_slides_server/src/main.rs
Dependencies: Step 2

  • Add mod startup;
  • Change fn socket_base_dir() to pub(crate) fn socket_base_dir()
  • Spawn tokio::spawn(async move { startup::auto_scan_on_startup(state_clone).await; }) after axum serve setup.

Acceptance Criteria

  • On startup with CODEROOT secret set and $CODEROOT/ourworld_slides existing, decks appear in deck.list without user interaction.
  • On subsequent restarts, the server re-scans the last remembered path from Redis.
  • If the remembered path is missing, the Redis key is deleted and the CODEROOT fallback is tried.
  • If both paths are unavailable, server starts normally with an empty deck list.
  • If hero_proc is unreachable, server starts normally (logs warn, skips auto-scan).
  • If hero_db is unreachable, server starts normally (logs warn, tries CODEROOT fallback without persistence).
  • Manual deck.scan via UI/RPC still works after startup.
  • cargo build succeeds with no new warnings.

Notes

  • hero_db connection string: redis+unix:///absolute/path/to/socket?db=0
  • hero_proc secret key: coderoot (lowercase), context "core"
  • Race-free startup: The auto-scan is a detached tokio::spawn task. Requests that arrive before scan completes see an empty deck list — acceptable.
  • No new RPC endpoints are added; all logic is internal to the server startup.
## Implementation Spec — Issue #12: Default Scan Directory ### Objective On server startup, `hero_slides_server` should automatically resolve a default scan directory and perform an initial `deck.scan` without any user interaction. The logic uses two persistence mechanisms: 1. **hero_proc secrets** — to read `CODEROOT` (the code workspace root). 2. **hero_db (Redis)** — to store and recall the last successfully scanned directory path. The fallback directory is `$CODEROOT/ourworld_slides`. On each startup the server checks the remembered path first; if it still exists the scan proceeds immediately; if it is gone the stored entry is cleared and the fallback path is tried instead. --- ### Requirements - On startup, `hero_slides_server` must attempt an automatic scan before accepting user-initiated requests. - The server connects to hero_proc via its Unix socket and calls `secret.get` with key `coderoot` (context `"core"`) to obtain `CODEROOT`. - After obtaining `CODEROOT`, the fallback candidate is `$CODEROOT/ourworld_slides`. - The server reads key `hero_slides:last_scan_path` from hero_db (Redis) to recall the previously successful scan root. - Decision logic: - If `last_scan_path` exists in Redis AND that directory is present on disk → scan that path. - If `last_scan_path` exists in Redis BUT that directory is missing on disk → delete the Redis key, then try the CODEROOT fallback. - If there is no `last_scan_path` in Redis → try the CODEROOT fallback. - CODEROOT fallback: if `$CODEROOT/ourworld_slides` is a directory → scan it and save the path to Redis. - If neither path is available → log a warning and start with an empty deck list (no error, server starts normally). - A successful scan must update `hero_slides:last_scan_path` in Redis. - The auto-scan must not block server startup — it runs as a background task after the Axum socket is bound. - Any failure to reach hero_proc or hero_db must be logged at `warn` level and silently skipped; the server must not fail to start. --- ### Files to Modify / Create - `crates/hero_slides_server/src/startup.rs` — **Create**: Contains `auto_scan_on_startup()` and all helpers for secret fetching, Redis persistence, and path resolution. - `crates/hero_slides_server/src/main.rs` — **Modify**: Add `mod startup`, make `socket_base_dir()` pub(crate), spawn background auto-scan task after binding socket. - `crates/hero_slides_server/Cargo.toml` — **Modify**: Add `redis = { version = "0.27", features = ["tokio-comp", "aio"] }`. --- ### Implementation Plan #### Step 1 — Add Redis dependency to Cargo.toml Files: `crates/hero_slides_server/Cargo.toml` Dependencies: none Add `redis = { version = "0.27", features = ["tokio-comp", "aio"] }` to `[dependencies]`. #### Step 2 — Create `startup.rs` Files: `crates/hero_slides_server/src/startup.rs` Dependencies: Step 1 Create module with: - `fetch_coderoot() -> Option<String>` — connects to hero_proc, calls `secret.get` with key `"coderoot"`, context `"core"`. - `redis_socket_path() -> PathBuf` — returns `$HERO_DB_SOCKET` or `socket_base_dir()/hero_db/redis.sock`. - `read_last_scan_path() -> Option<String>` — GET `hero_slides:last_scan_path` from Redis. - `save_last_scan_path(path: &str)` — SET `hero_slides:last_scan_path` in Redis. - `delete_last_scan_path()` — DEL `hero_slides:last_scan_path` from Redis. - `do_scan(root: &Path, state: &Arc<ServerState>) -> bool` — calls `hero_slides_lib::scan_decks()`, upserts results into state. - `pub async fn auto_scan_on_startup(state: Arc<ServerState>)` — orchestrates the full decision tree. #### Step 3 — Wire into `main.rs` Files: `crates/hero_slides_server/src/main.rs` Dependencies: Step 2 - Add `mod startup;` - Change `fn socket_base_dir()` to `pub(crate) fn socket_base_dir()` - Spawn `tokio::spawn(async move { startup::auto_scan_on_startup(state_clone).await; })` after axum serve setup. --- ### Acceptance Criteria - [ ] On startup with CODEROOT secret set and `$CODEROOT/ourworld_slides` existing, decks appear in `deck.list` without user interaction. - [ ] On subsequent restarts, the server re-scans the last remembered path from Redis. - [ ] If the remembered path is missing, the Redis key is deleted and the CODEROOT fallback is tried. - [ ] If both paths are unavailable, server starts normally with an empty deck list. - [ ] If hero_proc is unreachable, server starts normally (logs warn, skips auto-scan). - [ ] If hero_db is unreachable, server starts normally (logs warn, tries CODEROOT fallback without persistence). - [ ] Manual `deck.scan` via UI/RPC still works after startup. - [ ] `cargo build` succeeds with no new warnings. --- ### Notes - **hero_db connection string:** `redis+unix:///absolute/path/to/socket?db=0` - **hero_proc secret key:** `coderoot` (lowercase), context `"core"` - **Race-free startup:** The auto-scan is a detached `tokio::spawn` task. Requests that arrive before scan completes see an empty deck list — acceptable. - **No new RPC endpoints** are added; all logic is internal to the server startup.
casper-stevens commented 2026-04-21 10:11:57 +00:00
Member
Copy link

Implementation Spec for Issue #12

Objective

At hero_slides_server startup, if the in-memory deck list is empty, auto-populate it by scanning a deck root discovered through this cascade: (1) last successfully scanned directory remembered in hero_db, if it still exists on disk; otherwise (2) $CODEROOT/ourworld_slides, where CODEROOT is read from hero_proc's secret store. Persist the scanned path back into hero_db so the next startup picks the same root.

Requirements

  • Read CODEROOT from hero_proc secrets (context core, key coderoot). Do not read $CODEROOT from the process environment.
  • Use hero_db (not raw redis crate) as the persistence layer for last_scan_path.
  • Reuse the existing scan flow in handle_deck_scan (hero_slides_lib::scan_decks + upsert into ServerState::decks) — do not duplicate logic.
  • Run the auto-scan as a non-blocking background task so the HTTP server binds on the socket even if hero_proc or hero_db are unreachable.
  • If the remembered last_scan_path no longer exists on disk, delete the key from hero_db and fall back to $CODEROOT/ourworld_slides.
  • If $CODEROOT/ourworld_slides also does not exist (or CODEROOT cannot be read), log a warning and leave the deck list empty — the user can still trigger deck.scan manually.
  • Any failure to reach hero_proc or hero_db must log a warning and continue; it must never crash the server.

Files to Modify/Create

  • crates/hero_slides_server/Cargo.toml — add hero_db_sdk dependency (git, branch development), matching existing hero_proc_sdk style. No raw redis crate.
  • crates/hero_slides_server/src/main.rs — declare new mod startup;, make socket_base_dir() visible to the module (pub(crate) or move into startup), spawn the auto-scan task after the Unix socket is bound (so startup is not blocked by network I/O against hero_proc/hero_db).
  • crates/hero_slides_server/src/startup.rs — NEW — all auto-scan logic: fetch_coderoot, hero_db_socket_path, read_last_scan_path, save_last_scan_path, delete_last_scan_path, do_scan, pub async fn auto_scan_on_startup(state: Arc<ServerState>).

Implementation Plan

Step 1: Add hero_db_sdk dependency to hero_slides_server

Files: crates/hero_slides_server/Cargo.toml

  • Add, below the existing hero_proc_sdk line:
    hero_db_sdk = { git = "https://forge.ourworld.tf/lhumina_code/hero_db.git", branch = "development" }
  • Do NOT add a bare redis dependency. hero_db_sdk re-exports a RESP2 HeroDbClient (hero_db_sdk::hero_db_client::HeroDbClient) built on redis internally, which is the supported pattern.

Dependencies: none.

Step 2: Create crates/hero_slides_server/src/startup.rs

Files: crates/hero_slides_server/src/startup.rs (new)

Module contents:

  1. pub async fn auto_scan_on_startup(state: Arc<ServerState>) — orchestrator, spawned from main. Steps:

    • If state.decks.read().await.len() > 0, return early (idempotent).
    • Try read_last_scan_path(). If Some(path) and PathBuf::from(&path).is_dir(), call do_scan(path, &state).await and return. If the directory doesn't exist, call delete_last_scan_path().await and continue to fallback.
    • Fallback: call fetch_coderoot().await; if it returns a non-empty value, compute candidate = PathBuf::from(coderoot).join("ourworld_slides") and, if candidate.is_dir(), call do_scan(candidate.to_string_lossy().to_string(), &state).await.
    • Any step that fails must tracing::warn! with context and return.

  2. async fn fetch_coderoot() -> Option<String>

    • Resolve the hero_proc socket via hero_proc_sdk::socket::default_path() (existing pattern in agent.rs and generate_job.rs).
    • Connect with HeroProcRPCAPIClient::connect_socket(&socket_path.to_string_lossy()).await. On error, warn! and return None.
    • Call client.secret_get(SecretGetInput { key: "coderoot".into(), context: Some("core".into()) }).await. On Ok, return Some(output.value) (unless empty). On Err, warn + None.

  3. fn hero_db_socket_path() -> PathBuf

    • Cascade: $HERO_DB_SOCKET → $HERO_SOCKET_DIR/hero_db/resp.sock → $HOME/hero/var/sockets/hero_db/resp.sock.
    • Use resp.sock (RESP2 data API), NOT rpc.sock.

  4. async fn open_hero_db_client() -> Option<HeroDbClient>

    • Build path via hero_db_socket_path(). If missing, warn and return None.
    • Wrap HeroDbClient::new_unix(path, "") in tokio::task::spawn_blocking (blocking under the hood).

  5. async fn read_last_scan_path() -> Option<String>

    • Key: hero_slides:last_scan_path.
    • client.get(key) inside spawn_blocking.

  6. async fn save_last_scan_path(path: &str)

    • client.set(key, path) inside spawn_blocking, fire-and-log.

  7. async fn delete_last_scan_path()

    • client.del(key) inside spawn_blocking.

  8. async fn do_scan(root: String, state: &ServerState) -> bool

    • Validate PathBuf::from(&root).is_dir(); if not, warn + return false.
    • Call hero_slides_lib::scan_decks(&root_pb) inside tokio::task::spawn_blocking (mirroring rpc.rs handle_deck_scan).
    • If empty, warn and return false — do NOT save to hero_db.
    • Otherwise upsert each DeckInfo into state.decks using the shared helper from Step 3 (or duplicate the loop from handle_deck_scan).
    • On success, call save_last_scan_path(&root).await, log info!, return true.

Dependencies: Step 1.

Step 3: Extract a shared deck-upsert helper (optional, recommended)

Files: crates/hero_slides_server/src/rpc.rs

  • Add pub(crate) async fn upsert_decks(state: &ServerState, found: Vec<hero_slides_lib::DeckInfo>) -> Vec<DeckInfo> near handle_deck_scan.
  • Refactor handle_deck_scan to call it.
  • If skipped, Step 2's do_scan duplicates the upsert loop (acceptable).

Dependencies: none.

Step 4: Wire auto-scan into main.rs

Files: crates/hero_slides_server/src/main.rs

  • Add mod startup; next to the existing module declarations.
  • Change fn socket_base_dir() to pub(crate) fn socket_base_dir().
  • After UnixListener::bind and the permissions block, BEFORE axum::serve(...), spawn: 
    let auto_scan_state = state.clone();
tokio::spawn(async move {
    startup::auto_scan_on_startup(auto_scan_state).await;
});
    This way the socket is reachable before the (possibly slow) scan completes.

Dependencies: Step 2.

Step 5: Manual verification

  • cargo build -p hero_slides_server.
  • With hero_proc running and coderoot secret set, and hero_db running at $HOME/hero/var/sockets/hero_db/resp.sock: start the server, confirm info! logs "auto-scan: loaded N decks from ..." and deck.list returns populated decks.
  • Confirm hero_slides:last_scan_path is set: redis-cli -s ~/hero/var/sockets/hero_db/resp.sock GET hero_slides:last_scan_path.
  • Restart the server, confirm the saved path is reused.
  • Remove the remembered directory, restart — confirm key deletion and fallback.
  • Kill hero_proc and/or hero_db, restart — confirm the server still binds and serves.

Dependencies: Steps 1–4.

Acceptance Criteria

  • crates/hero_slides_server/Cargo.toml depends on hero_db_sdk (no bare redis crate).
  • New module crates/hero_slides_server/src/startup.rs exposes pub async fn auto_scan_on_startup(state: Arc<ServerState>).
  • main.rs declares mod startup;, spawns the task after bind and before serve, and socket_base_dir is pub(crate).
  • Auto-scan reads CODEROOT via hero_proc_sdk::HeroProcRPCAPIClient::secret_get(SecretGetInput { key: "coderoot", context: Some("core") }) — not from std::env.
  • Auto-scan persists via hero_db_sdk::hero_db_client::HeroDbClient at socket $HERO_SOCKET_DIR/hero_db/resp.sock, key hero_slides:last_scan_path.
  • Remembered path is validated with is_dir(); missing directory → key deleted, fallback attempted.
  • $CODEROOT/ourworld_slides fallback only runs when no valid remembered path exists.
  • Auto-scan runs only when state.decks is empty (idempotent across accidental re-runs).
  • Any failure to reach hero_proc or hero_db logs a warning and is non-fatal — server startup never blocks or aborts.
  • deck.scan RPC behaviour is unchanged for callers.
  • cargo build -p hero_slides_server succeeds.

Notes

Divergences from the earlier proposal that this spec corrects:

  • Earlier: "Add Redis dep to Cargo.toml." Corrected: workspace convention is hero_db_sdk, not the bare redis crate. hero_db_sdk::hero_db_client::HeroDbClient wraps redis with the correct new_unix(path, secret) and matches how other Hero services talk to hero_db.
  • Earlier: redis+unix:///absolute/path/to/socket?db=0. Corrected: HeroDbClient::new_unix builds the URL internally; callers pass the socket path and secret directly.
  • Earlier: redis_socket_path. Corrected: canonical name is hero_db_socket_path and it points at resp.sock (RESP2 data API), NOT rpc.sock (which is the JSON-RPC management API).
  • Earlier: secret.get with context: "core". Corrected: generated SecretGetInput is { key: String, context: Option<String> } — pass Some("core".into()).
  • Earlier: scan_decks returns Result<...>. Corrected: hero_slides_lib::scan_decks(root: &Path) -> Vec<hero_slides_lib::DeckInfo> returns an empty Vec on IO errors internally.

Other notes:

  • ServerState.decks is a tokio::sync::RwLock<Vec<rpc::DeckInfo>> (not lib::DeckInfo) — do_scan must convert via the existing impl From<hero_slides_lib::DeckInfo> for rpc::DeckInfo in rpc.rs.
  • hero_db RESP2 auth: construct the client with secret = "". If a future change requires auth, fetch it the same way CODEROOT is fetched.
  • hero_db_sdk uses the blocking redis crate; all HeroDbClient calls (including construction) MUST be wrapped in tokio::task::spawn_blocking.
  • The background task captures Arc<ServerState> by clone — no extra synchronization needed.
## Implementation Spec for Issue #12 ### Objective At `hero_slides_server` startup, if the in-memory deck list is empty, auto-populate it by scanning a deck root discovered through this cascade: (1) last successfully scanned directory remembered in `hero_db`, if it still exists on disk; otherwise (2) `$CODEROOT/ourworld_slides`, where `CODEROOT` is read from `hero_proc`'s secret store. Persist the scanned path back into `hero_db` so the next startup picks the same root. ### Requirements - Read `CODEROOT` from `hero_proc` secrets (context `core`, key `coderoot`). Do not read `$CODEROOT` from the process environment. - Use `hero_db` (not raw `redis` crate) as the persistence layer for `last_scan_path`. - Reuse the existing scan flow in `handle_deck_scan` (`hero_slides_lib::scan_decks` + upsert into `ServerState::decks`) — do not duplicate logic. - Run the auto-scan as a non-blocking background task so the HTTP server binds on the socket even if `hero_proc` or `hero_db` are unreachable. - If the remembered `last_scan_path` no longer exists on disk, delete the key from `hero_db` and fall back to `$CODEROOT/ourworld_slides`. - If `$CODEROOT/ourworld_slides` also does not exist (or `CODEROOT` cannot be read), log a warning and leave the deck list empty — the user can still trigger `deck.scan` manually. - Any failure to reach `hero_proc` or `hero_db` must log a warning and continue; it must never crash the server. ### Files to Modify/Create - `crates/hero_slides_server/Cargo.toml` — add `hero_db_sdk` dependency (git, branch `development`), matching existing `hero_proc_sdk` style. No raw `redis` crate. - `crates/hero_slides_server/src/main.rs` — declare new `mod startup;`, make `socket_base_dir()` visible to the module (`pub(crate)` or move into `startup`), spawn the auto-scan task after the Unix socket is bound (so startup is not blocked by network I/O against hero_proc/hero_db). - `crates/hero_slides_server/src/startup.rs` — NEW — all auto-scan logic: `fetch_coderoot`, `hero_db_socket_path`, `read_last_scan_path`, `save_last_scan_path`, `delete_last_scan_path`, `do_scan`, `pub async fn auto_scan_on_startup(state: Arc<ServerState>)`. ### Implementation Plan #### Step 1: Add `hero_db_sdk` dependency to `hero_slides_server` Files: `crates/hero_slides_server/Cargo.toml` - Add, below the existing `hero_proc_sdk` line: `hero_db_sdk = { git = "https://forge.ourworld.tf/lhumina_code/hero_db.git", branch = "development" }` - Do NOT add a bare `redis` dependency. `hero_db_sdk` re-exports a RESP2 `HeroDbClient` (`hero_db_sdk::hero_db_client::HeroDbClient`) built on `redis` internally, which is the supported pattern. Dependencies: none. #### Step 2: Create `crates/hero_slides_server/src/startup.rs` Files: `crates/hero_slides_server/src/startup.rs` (new) Module contents: 1. `pub async fn auto_scan_on_startup(state: Arc<ServerState>)` — orchestrator, spawned from `main`. Steps: - If `state.decks.read().await.len() > 0`, return early (idempotent). - Try `read_last_scan_path()`. If `Some(path)` and `PathBuf::from(&path).is_dir()`, call `do_scan(path, &state).await` and return. If the directory doesn't exist, call `delete_last_scan_path().await` and continue to fallback. - Fallback: call `fetch_coderoot().await`; if it returns a non-empty value, compute `candidate = PathBuf::from(coderoot).join("ourworld_slides")` and, if `candidate.is_dir()`, call `do_scan(candidate.to_string_lossy().to_string(), &state).await`. - Any step that fails must `tracing::warn!` with context and return. 2. `async fn fetch_coderoot() -> Option<String>` - Resolve the hero_proc socket via `hero_proc_sdk::socket::default_path()` (existing pattern in `agent.rs` and `generate_job.rs`). - Connect with `HeroProcRPCAPIClient::connect_socket(&socket_path.to_string_lossy()).await`. On error, `warn!` and return `None`. - Call `client.secret_get(SecretGetInput { key: "coderoot".into(), context: Some("core".into()) }).await`. On `Ok`, return `Some(output.value)` (unless empty). On `Err`, warn + `None`. 3. `fn hero_db_socket_path() -> PathBuf` - Cascade: `$HERO_DB_SOCKET` → `$HERO_SOCKET_DIR/hero_db/resp.sock` → `$HOME/hero/var/sockets/hero_db/resp.sock`. - Use `resp.sock` (RESP2 data API), NOT `rpc.sock`. 4. `async fn open_hero_db_client() -> Option<HeroDbClient>` - Build path via `hero_db_socket_path()`. If missing, warn and return `None`. - Wrap `HeroDbClient::new_unix(path, "")` in `tokio::task::spawn_blocking` (blocking under the hood). 5. `async fn read_last_scan_path() -> Option<String>` - Key: `hero_slides:last_scan_path`. - `client.get(key)` inside `spawn_blocking`. 6. `async fn save_last_scan_path(path: &str)` - `client.set(key, path)` inside `spawn_blocking`, fire-and-log. 7. `async fn delete_last_scan_path()` - `client.del(key)` inside `spawn_blocking`. 8. `async fn do_scan(root: String, state: &ServerState) -> bool` - Validate `PathBuf::from(&root).is_dir()`; if not, warn + return `false`. - Call `hero_slides_lib::scan_decks(&root_pb)` inside `tokio::task::spawn_blocking` (mirroring `rpc.rs` `handle_deck_scan`). - If empty, warn and return `false` — do NOT save to hero_db. - Otherwise upsert each `DeckInfo` into `state.decks` using the shared helper from Step 3 (or duplicate the loop from `handle_deck_scan`). - On success, call `save_last_scan_path(&root).await`, log `info!`, return `true`. Dependencies: Step 1. #### Step 3: Extract a shared deck-upsert helper (optional, recommended) Files: `crates/hero_slides_server/src/rpc.rs` - Add `pub(crate) async fn upsert_decks(state: &ServerState, found: Vec<hero_slides_lib::DeckInfo>) -> Vec<DeckInfo>` near `handle_deck_scan`. - Refactor `handle_deck_scan` to call it. - If skipped, Step 2's `do_scan` duplicates the upsert loop (acceptable). Dependencies: none. #### Step 4: Wire auto-scan into `main.rs` Files: `crates/hero_slides_server/src/main.rs` - Add `mod startup;` next to the existing module declarations. - Change `fn socket_base_dir()` to `pub(crate) fn socket_base_dir()`. - After `UnixListener::bind` and the permissions block, BEFORE `axum::serve(...)`, spawn: ```rust let auto_scan_state = state.clone(); tokio::spawn(async move { startup::auto_scan_on_startup(auto_scan_state).await; }); ``` This way the socket is reachable before the (possibly slow) scan completes. Dependencies: Step 2. #### Step 5: Manual verification - `cargo build -p hero_slides_server`. - With `hero_proc` running and `coderoot` secret set, and `hero_db` running at `$HOME/hero/var/sockets/hero_db/resp.sock`: start the server, confirm `info!` logs `"auto-scan: loaded N decks from ..."` and `deck.list` returns populated decks. - Confirm `hero_slides:last_scan_path` is set: `redis-cli -s ~/hero/var/sockets/hero_db/resp.sock GET hero_slides:last_scan_path`. - Restart the server, confirm the saved path is reused. - Remove the remembered directory, restart — confirm key deletion and fallback. - Kill `hero_proc` and/or `hero_db`, restart — confirm the server still binds and serves. Dependencies: Steps 1–4. ### Acceptance Criteria - [ ] `crates/hero_slides_server/Cargo.toml` depends on `hero_db_sdk` (no bare `redis` crate). - [ ] New module `crates/hero_slides_server/src/startup.rs` exposes `pub async fn auto_scan_on_startup(state: Arc<ServerState>)`. - [ ] `main.rs` declares `mod startup;`, spawns the task after bind and before serve, and `socket_base_dir` is `pub(crate)`. - [ ] Auto-scan reads `CODEROOT` via `hero_proc_sdk::HeroProcRPCAPIClient::secret_get(SecretGetInput { key: "coderoot", context: Some("core") })` — not from `std::env`. - [ ] Auto-scan persists via `hero_db_sdk::hero_db_client::HeroDbClient` at socket `$HERO_SOCKET_DIR/hero_db/resp.sock`, key `hero_slides:last_scan_path`. - [ ] Remembered path is validated with `is_dir()`; missing directory → key deleted, fallback attempted. - [ ] `$CODEROOT/ourworld_slides` fallback only runs when no valid remembered path exists. - [ ] Auto-scan runs only when `state.decks` is empty (idempotent across accidental re-runs). - [ ] Any failure to reach `hero_proc` or `hero_db` logs a warning and is non-fatal — server startup never blocks or aborts. - [ ] `deck.scan` RPC behaviour is unchanged for callers. - [ ] `cargo build -p hero_slides_server` succeeds. ### Notes Divergences from the earlier proposal that this spec corrects: - Earlier: "Add Redis dep to Cargo.toml." Corrected: workspace convention is `hero_db_sdk`, not the bare `redis` crate. `hero_db_sdk::hero_db_client::HeroDbClient` wraps `redis` with the correct `new_unix(path, secret)` and matches how other Hero services talk to hero_db. - Earlier: `redis+unix:///absolute/path/to/socket?db=0`. Corrected: `HeroDbClient::new_unix` builds the URL internally; callers pass the socket path and secret directly. - Earlier: `redis_socket_path`. Corrected: canonical name is `hero_db_socket_path` and it points at `resp.sock` (RESP2 data API), NOT `rpc.sock` (which is the JSON-RPC management API). - Earlier: `secret.get` with `context: "core"`. Corrected: generated `SecretGetInput` is `{ key: String, context: Option<String> }` — pass `Some("core".into())`. - Earlier: `scan_decks` returns `Result<...>`. Corrected: `hero_slides_lib::scan_decks(root: &Path) -> Vec<hero_slides_lib::DeckInfo>` returns an empty Vec on IO errors internally. Other notes: - `ServerState.decks` is a `tokio::sync::RwLock<Vec<rpc::DeckInfo>>` (not `lib::DeckInfo`) — `do_scan` must convert via the existing `impl From<hero_slides_lib::DeckInfo> for rpc::DeckInfo` in `rpc.rs`. - hero_db RESP2 auth: construct the client with `secret = ""`. If a future change requires auth, fetch it the same way `CODEROOT` is fetched. - `hero_db_sdk` uses the blocking `redis` crate; all `HeroDbClient` calls (including construction) MUST be wrapped in `tokio::task::spawn_blocking`. - The background task captures `Arc<ServerState>` by clone — no extra synchronization needed.
casper-stevens commented 2026-04-21 10:32:34 +00:00
Member
Copy link

Build & Test Results

cargo build -p hero_slides_server

PASS (exit 0). No warnings or errors.

cargo test -p hero_slides_server

  • Total: 0
  • Passed: 0
  • Failed: 0
  • Ignored: 0

No unit tests are defined in the hero_slides_server crate itself (it is a thin main.rs binary). All relevant logic lives in hero_slides_lib and is exercised by the workspace run below.

cargo test (workspace)

  • Total: 85 (68 lib unit + 14 integration + 3 doctests)
  • Passed: 84
  • Failed: 0
  • Ignored: 1 (test_generate_single_slide_ai — requires live AI provider)

Breakdown:

  • hero_slides_lib unit tests: 68 passed, 0 failed, 0 ignored
  • hero_slides_lib integration tests: 13 passed, 0 failed, 1 ignored
  • hero_slides_lib doctests: 2 passed
  • hero_slides_rhai doctests: 1 passed
  • hero_slides, hero_slides_server, hero_slides_ui, hero_slides_sdk, hero_slides_rhai: 0 tests (binary / stub crates)

Notes

No new unit tests were added for the auto_scan_on_startup path — the logic requires hero_proc + hero_db running and a populated HERO_SLIDES_ROOT, so it is best verified via the manual integration steps listed in the spec. The existing discovery::scan_decks behaviour it delegates to is already covered by discovery::tests::test_scan_decks_dirs and the integration-level test_deck_scan_finds_created_decks / test_deck_scan_empty_dir cases, all green.

## Build & Test Results ### cargo build -p hero_slides_server PASS (exit 0). No warnings or errors. ### cargo test -p hero_slides_server - Total: 0 - Passed: 0 - Failed: 0 - Ignored: 0 No unit tests are defined in the `hero_slides_server` crate itself (it is a thin `main.rs` binary). All relevant logic lives in `hero_slides_lib` and is exercised by the workspace run below. ### cargo test (workspace) - Total: 85 (68 lib unit + 14 integration + 3 doctests) - Passed: 84 - Failed: 0 - Ignored: 1 (`test_generate_single_slide_ai` — requires live AI provider) Breakdown: - `hero_slides_lib` unit tests: 68 passed, 0 failed, 0 ignored - `hero_slides_lib` integration tests: 13 passed, 0 failed, 1 ignored - `hero_slides_lib` doctests: 2 passed - `hero_slides_rhai` doctests: 1 passed - `hero_slides`, `hero_slides_server`, `hero_slides_ui`, `hero_slides_sdk`, `hero_slides_rhai`: 0 tests (binary / stub crates) ### Notes No new unit tests were added for the `auto_scan_on_startup` path — the logic requires `hero_proc` + `hero_db` running and a populated `HERO_SLIDES_ROOT`, so it is best verified via the manual integration steps listed in the spec. The existing `discovery::scan_decks` behaviour it delegates to is already covered by `discovery::tests::test_scan_decks_dirs` and the integration-level `test_deck_scan_finds_created_decks` / `test_deck_scan_empty_dir` cases, all green.
casper-stevens commented 2026-04-21 10:34:23 +00:00
Member
Copy link

Implementation Summary

Auto-scan on server startup is now wired into hero_slides_server. If the in-memory deck list is empty at boot, the server resolves a deck root via hero_db (remembered last scan) with a fallback to $CODEROOT/ourworld_slides (CODEROOT from hero_proc secrets), runs the existing scan pipeline, and persists the scanned root back to hero_db for next time.

Files changed

  • crates/hero_slides_server/Cargo.toml — added hero_db_sdk dependency (git, branch development), matching the existing hero_proc_sdk style. No bare redis crate.
  • crates/hero_slides_server/src/rpc.rs — extracted a shared pub(crate) async fn upsert_decks(state: &ServerState, found: Vec<hero_slides_lib::DeckInfo>) -> Vec<DeckInfo> helper; handle_deck_scan now delegates upsertion to it. External RPC behaviour unchanged.
  • crates/hero_slides_server/src/startup.rs — NEW. Contains all auto-scan logic behind a single public entry point pub async fn auto_scan_on_startup(state: Arc<ServerState>). Private helpers handle CODEROOT lookup via hero_proc_sdk::HeroProcRPCAPIClient::secret_get, hero_db connectivity via hero_db_sdk::hero_db_client::HeroDbClient (all blocking calls wrapped in tokio::task::spawn_blocking), and the full decision cascade. Redis key used: hero_slides:last_scan_path.
  • crates/hero_slides_server/src/main.rs — declared mod startup;, promoted socket_base_dir() to pub(crate) (reused by startup::hero_db_socket_path), and spawned startup::auto_scan_on_startup(state.clone()) as a detached task after the Unix socket binds but before axum::serve, so startup never blocks on hero_proc / hero_db.

Key design choices

  • Uses hero_db_sdk instead of the raw redis crate — workspace convention.
  • hero_db socket resolved via $HERO_DB_SOCKET → $HERO_SOCKET_DIR/hero_db/resp.sock → $HOME/hero/var/sockets/hero_db/resp.sock (RESP2 data socket, NOT rpc.sock).
  • Auto-scan runs only when state.decks is empty, so manual deck.scan calls always take precedence and the task is idempotent across restarts.
  • Any failure to reach hero_proc or hero_db logs at warn level and exits cleanly — the server never fails to start because of optional auto-scan dependencies.
  • HeroDbClient methods are &mut self, so each helper constructs its own client per operation (cheap; single Unix socket).
  • hero_slides_lib::scan_decks is synchronous and returns Vec<DeckInfo> (empty on errors) — all calls wrapped in spawn_blocking.

Build & Test

  • cargo build -p hero_slides_server: PASS (no warnings)
  • cargo test -p hero_slides_server: PASS (0 tests — binary crate)
  • cargo test (workspace): 84 passed, 0 failed, 1 ignored (test_generate_single_slide_ai — pre-existing; requires live AI provider)

No new unit tests were added. The auto-scan logic depends on a running hero_proc and hero_db, so it is best validated via the manual verification steps listed in the spec (running the server with CODEROOT set, restarting to confirm the remembered path is reused, removing the remembered directory to confirm fallback, killing hero_proc / hero_db to confirm graceful degradation).

Notes

  • ServerState.decks is tokio::sync::RwLock<Vec<rpc::DeckInfo>>; the scan helper converts via the existing impl From<hero_slides_lib::DeckInfo> for rpc::DeckInfo in rpc.rs.
  • hero_db client is opened with secret = "". If hero_db auth is added later, the secret can be fetched the same way CODEROOT is.
  • SecretGetInput passes context: Some("core".into()) explicitly, matching the issue brief, though the hero_proc server defaults to "core" when None.
## Implementation Summary Auto-scan on server startup is now wired into `hero_slides_server`. If the in-memory deck list is empty at boot, the server resolves a deck root via `hero_db` (remembered last scan) with a fallback to `$CODEROOT/ourworld_slides` (CODEROOT from `hero_proc` secrets), runs the existing scan pipeline, and persists the scanned root back to `hero_db` for next time. ### Files changed - `crates/hero_slides_server/Cargo.toml` — added `hero_db_sdk` dependency (git, branch `development`), matching the existing `hero_proc_sdk` style. No bare `redis` crate. - `crates/hero_slides_server/src/rpc.rs` — extracted a shared `pub(crate) async fn upsert_decks(state: &ServerState, found: Vec<hero_slides_lib::DeckInfo>) -> Vec<DeckInfo>` helper; `handle_deck_scan` now delegates upsertion to it. External RPC behaviour unchanged. - `crates/hero_slides_server/src/startup.rs` — NEW. Contains all auto-scan logic behind a single public entry point `pub async fn auto_scan_on_startup(state: Arc<ServerState>)`. Private helpers handle CODEROOT lookup via `hero_proc_sdk::HeroProcRPCAPIClient::secret_get`, `hero_db` connectivity via `hero_db_sdk::hero_db_client::HeroDbClient` (all blocking calls wrapped in `tokio::task::spawn_blocking`), and the full decision cascade. Redis key used: `hero_slides:last_scan_path`. - `crates/hero_slides_server/src/main.rs` — declared `mod startup;`, promoted `socket_base_dir()` to `pub(crate)` (reused by `startup::hero_db_socket_path`), and spawned `startup::auto_scan_on_startup(state.clone())` as a detached task after the Unix socket binds but before `axum::serve`, so startup never blocks on hero_proc / hero_db. ### Key design choices - Uses `hero_db_sdk` instead of the raw `redis` crate — workspace convention. - hero_db socket resolved via `$HERO_DB_SOCKET` → `$HERO_SOCKET_DIR/hero_db/resp.sock` → `$HOME/hero/var/sockets/hero_db/resp.sock` (RESP2 data socket, NOT `rpc.sock`). - Auto-scan runs only when `state.decks` is empty, so manual `deck.scan` calls always take precedence and the task is idempotent across restarts. - Any failure to reach hero_proc or hero_db logs at `warn` level and exits cleanly — the server never fails to start because of optional auto-scan dependencies. - `HeroDbClient` methods are `&mut self`, so each helper constructs its own client per operation (cheap; single Unix socket). - `hero_slides_lib::scan_decks` is synchronous and returns `Vec<DeckInfo>` (empty on errors) — all calls wrapped in `spawn_blocking`. ### Build & Test - `cargo build -p hero_slides_server`: PASS (no warnings) - `cargo test -p hero_slides_server`: PASS (0 tests — binary crate) - `cargo test` (workspace): 84 passed, 0 failed, 1 ignored (`test_generate_single_slide_ai` — pre-existing; requires live AI provider) No new unit tests were added. The auto-scan logic depends on a running `hero_proc` and `hero_db`, so it is best validated via the manual verification steps listed in the spec (running the server with CODEROOT set, restarting to confirm the remembered path is reused, removing the remembered directory to confirm fallback, killing hero_proc / hero_db to confirm graceful degradation). ### Notes - `ServerState.decks` is `tokio::sync::RwLock<Vec<rpc::DeckInfo>>`; the scan helper converts via the existing `impl From<hero_slides_lib::DeckInfo> for rpc::DeckInfo` in `rpc.rs`. - `hero_db` client is opened with `secret = ""`. If hero_db auth is added later, the secret can be fetched the same way `CODEROOT` is. - `SecretGetInput` passes `context: Some("core".into())` explicitly, matching the issue brief, though the hero_proc server defaults to `"core"` when `None`.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
development
development_casper
development_default_scan_directory
main
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lhumina_code/hero_slides#12
No description provided.