lhumina_code/hero_codescalers
27
0
Fork 0
Code Issues 2 Pull requests Projects Releases Packages Wiki Activity Actions

Tag jobs and actions for bulk cleanup in hero_proc #10

New issue
Closed
opened 2026-04-26 12:18:41 +00:00 by mahmoud · 4 comments
mahmoud commented 2026-04-26 12:18:41 +00:00
Owner
Copy link

Parent: #8

Problem

Jobs and Actions triggered by hero_codescalers accumulate in hero_proc indefinitely. There is no mechanism to clean them up in bulk.

Solution

  • Tag all hero_codescalers-triggered Actions and Jobs with a consistent prefix/label (e.g. codescalers_<feature>_<input_hash>)
  • Implement or expose a clear function that deletes all tagged jobs and actions via hero_proc's JSON-RPC API (92 methods available over $HERO_SOCKET_DIR/hero_proc/rpc.sock)
  • This mirrors the pattern used in hero_books_server's submit_or_dedup_docs_job with action key docs_<verb>_<hash>

Relevant paths

  • hero_proc/crates/hero_proc_server/ — where action/job management lives
  • hero_proc/docs/hero_proc_openrpc/ — check existing clear/delete methods
  • hero_codescalers/crates/hero_codescalers_server/ — where jobs are triggered

Acceptance Criteria

  • All codescalers-triggered jobs/actions carry a consistent tag
  • Clear function removes all tagged jobs + actions cleanly
  • hero_proc does not grow unbounded after repeated feature executions
Parent: #8 ## Problem Jobs and Actions triggered by hero_codescalers accumulate in hero_proc indefinitely. There is no mechanism to clean them up in bulk. ## Solution - Tag all hero_codescalers-triggered Actions and Jobs with a consistent prefix/label (e.g. `codescalers_<feature>_<input_hash>`) - Implement or expose a clear function that deletes all tagged jobs and actions via hero_proc's JSON-RPC API (92 methods available over `$HERO_SOCKET_DIR/hero_proc/rpc.sock`) - This mirrors the pattern used in hero_books_server's `submit_or_dedup_docs_job` with action key `docs_<verb>_<hash>` ## Relevant paths - `hero_proc/crates/hero_proc_server/` — where action/job management lives - `hero_proc/docs/hero_proc_openrpc/` — check existing clear/delete methods - `hero_codescalers/crates/hero_codescalers_server/` — where jobs are triggered ## Acceptance Criteria - [ ] All codescalers-triggered jobs/actions carry a consistent tag - [ ] Clear function removes all tagged jobs + actions cleanly - [ ] hero_proc does not grow unbounded after repeated feature executions
mahmoud commented 2026-04-27 06:27:08 +00:00
Author
Owner
Copy link

Implementation Spec for Issue #10 — jobs.cleanup RPC + CLI

Objective

Provide a single, admin-gated mechanism for bulk-removing every hero_proc job and action that hero_codescalers has ever enqueued, so hero_proc does not grow unbounded across repeated feature runs. Exposed as jobs.cleanup over JSON-RPC and as a hero_codescalers jobs-cleanup CLI subcommand. Tagging itself is already in place (build_tags() in jobs.rs); the missing piece is the cleanup operation that consumes those tags.

Requirements

  • New backend function jobs::cleanup(state, predicate) in hero_codescalers_server.
  • Always scope to the canonical codescaler tag — never let a caller use this method to nuke unrelated hero_proc state.
  • Optional narrowing: kind, target, actor, older_than_ms.
  • Two-step deletion: delete jobs first (using force=true), then delete the unique action specs they originated from. Action deletion is best-effort because the same action_id may be referenced by other jobs we do not own.
  • Return a structured summary: { jobs_deleted, actions_deleted, errors[] }.
  • dry_run: bool flag — list what would be deleted, change nothing.
  • Admin-gated like jobs.delete, jobs.bulk, jobs.cancel.
  • Register in openrpc.json. The SDK regenerates automatically at build time via the openrpc_client! macro.
  • CLI subcommand: hero_codescalers jobs-cleanup [--kind <k>] [--target <t>] [--actor <a>] [--older-than <duration>] [--dry-run].

Files to Modify/Create

  • crates/hero_codescalers_server/src/jobs.rs — add CleanupPredicate struct, CleanupSummary struct, and cleanup() async function.
  • crates/hero_codescalers_server/src/main.rs — add jobs.cleanup arm to dispatch, admin-gated.
  • crates/hero_codescalers_server/openrpc.json — register jobs.cleanup method with params/result schema; hero_codescalers_sdk regenerates automatically on next cargo build because it consumes this file via the openrpc_client! macro.
  • crates/hero_codescalers/src/main.rs — add JobsCleanup variant to Commands and its match arm in main().

Implementation Plan

Step 1: jobs::cleanup function

Files: crates/hero_codescalers_server/src/jobs.rs

  1. New input/output types:

    #[derive(Debug, Default, Clone)]
pub struct CleanupPredicate {
    pub kind: Option<String>,        // "user" | "template" | "service" | "install"
    pub target: Option<String>,
    pub actor: Option<String>,
    pub older_than_ms: Option<i64>,  // wall-clock now() - this many ms; only delete jobs created before that cutoff
    pub dry_run: bool,
}

#[derive(Debug, Default, Serialize)]
pub struct CleanupSummary {
    pub jobs_deleted: u32,
    pub actions_deleted: u32,
    pub job_ids: Vec<i64>,         // matched jobs (for visibility / dry-run)
    pub action_ids: Vec<String>,   // matched actions
    pub errors: Vec<String>,
    pub dry_run: bool,
}

  2. Function signature:

    pub async fn cleanup(state: &AppState, pred: CleanupPredicate) -> Result<Value>

  3. Body, in order:

    1. Connect to hero_proc via hero_proc_sdk::hero_proc_factory() (same pattern as list()).
    2. Pick the most-specific tag for the server-side filter:
      • if pred.actor → format!("codescaler_{actor}")
      • else if pred.kind → format!("codescaler_kind_{kind}")
      • else if pred.target → format!("codescaler_target_{target}")
      • else → "codescaler".
    3. Call hp.job_list(JobListInput { filter: Some(JobFilter { tag: Some(filter_tag), limit: Some(10_000), ..Default::default() }) }).
    4. For each JobSummary, post-filter in Rust:
      • Must have codescaler in tags (defense-in-depth).
      • If pred.kind set: tags must contain codescaler_kind_<kind>.
      • If pred.target set: tags must contain codescaler_target_<target>.
      • If pred.actor set: tags must contain codescaler_<actor>.
      • If pred.older_than_ms set: j.created_at_ms.unwrap_or(i64::MAX) <= chrono::Utc::now().timestamp_millis() - older_than_ms.
    5. Collect (id, action_id) pairs from matched jobs. Build summary.job_ids and a HashSet<String> of unique action_id values.
    6. If pred.dry_run: set summary.dry_run = true, leave deletion counts at 0, return.
    7. Otherwise:
      • For each matched job id: hp.job_delete(JobDeleteInput { id, force: Some(true) }). On Ok bump jobs_deleted. On Err push into summary.errors.
      • For each unique action_id: hp.action_delete(ActionDeleteInput { name: action_id, context: None }). Best-effort; failures go into errors.
    8. Return Ok(json!(summary)).

Dependencies: none.

Step 2: Wire jobs.cleanup arm in dispatch

Files: crates/hero_codescalers_server/src/main.rs

Insert after "jobs.bulk" arm (~line 583):

"jobs.cleanup" => {
    require_admin(state, caller_ip).await?;
    let pred = jobs::CleanupPredicate {
        kind:           params["kind"].as_str().map(str::to_string),
        target:         params["target"].as_str().map(str::to_string),
        actor:          params["actor"].as_str().map(str::to_string),
        older_than_ms:  params["older_than_ms"].as_i64(),
        dry_run:        params["dry_run"].as_bool().unwrap_or(false),
    };
    jobs::cleanup(state, pred).await
}

Dependencies: Step 1.

Step 3: Register jobs.cleanup in openrpc.json

Files: crates/hero_codescalers_server/openrpc.json

Insert after jobs.bulk (line 383), before actor.info (line 384):

{
  "name": "jobs.cleanup",
  "summary": "Bulk-delete codescaler jobs and their unique actions, optionally filtered by actor/kind/target/age. Dry-run lists matches without deleting.",
  "params": [
    { "name": "kind",          "schema": { "type": "string" },  "required": false },
    { "name": "target",        "schema": { "type": "string" },  "required": false },
    { "name": "actor",         "schema": { "type": "string" },  "required": false },
    { "name": "older_than_ms", "schema": { "type": "integer" }, "required": false },
    { "name": "dry_run",       "schema": { "type": "boolean" }, "required": false }
  ],
  "result": {
    "name": "JobsCleanupResult",
    "schema": {
      "type": "object",
      "properties": {
        "jobs_deleted":    { "type": "integer" },
        "actions_deleted": { "type": "integer" },
        "job_ids":         { "type": "array", "items": { "type": "integer" } },
        "action_ids":      { "type": "array", "items": { "type": "string"  } },
        "errors":          { "type": "array", "items": { "type": "string"  } },
        "dry_run":         { "type": "boolean" }
      }
    }
  }
}

The hero_codescalers_sdk regenerates JobsCleanupInput/JobsCleanupOutput and the jobs_cleanup() method automatically on next cargo build via the openrpc_client! macro. No manual SDK changes.

Dependencies: Step 2.

Step 4: CLI subcommand jobs-cleanup

Files: crates/hero_codescalers/src/main.rs

Add Commands::JobsCleanup variant with flags --kind, --target, --actor, --older-than, --dry-run. Add match arm calling client.jobs_cleanup(...) and printing the result as pretty JSON. Add a small parse_duration_ms() helper supporting ms, s, m, h, d.

Dependencies: Step 3 (so hero_codescalers_sdk::JobsCleanupInput exists).

Acceptance Criteria

  • All codescalers jobs carry tag codescaler + per-feature tags (already done by build_tags() at jobs.rs:76).
  • jobs.cleanup RPC method exists, is admin-gated, and is documented in openrpc.json.
  • jobs::cleanup() removes all matched jobs, then deletes their unique action specs from hero_proc.
  • CLI subcommand hero_codescalers jobs-cleanup works end to end and prints the summary as pretty JSON.
  • --dry-run returns the same summary shape with jobs_deleted=0, actions_deleted=0, and the would-be-deleted ids enumerated.
  • Predicate filters (--kind, --target, --actor, --older-than) all narrow correctly.
  • After a cleanup run, jobs.list returns no codescaler jobs and stats.job_count drops accordingly.
  • hero_proc does not grow unbounded across repeated runs.

Notes

  • Defense-in-depth: even though the server-side filter is codescaler*, the Rust post-filter re-checks the literal codescaler tag before deleting, mirroring list() / job_value_has_codescaler_tag.
  • Action deletion is intentionally best-effort. hero_proc's action.delete may legitimately fail (another job still references that spec); failures land in summary.errors without aborting cleanup.
  • older_than_ms filter treats absent created_at_ms conservatively (skip when set).
  • JobFilter.tag is single-valued — most-specific-tag selection is what makes server-side narrowing work.
  • Action ids are pulled from JobSummary.action_id; codescaler enqueue() always sets a unique action name.
## Implementation Spec for Issue #10 — jobs.cleanup RPC + CLI ### Objective Provide a single, admin-gated mechanism for bulk-removing every hero_proc job and action that hero_codescalers has ever enqueued, so hero_proc does not grow unbounded across repeated feature runs. Exposed as `jobs.cleanup` over JSON-RPC and as a `hero_codescalers jobs-cleanup` CLI subcommand. Tagging itself is already in place (`build_tags()` in `jobs.rs`); the missing piece is the cleanup operation that consumes those tags. ### Requirements - New backend function `jobs::cleanup(state, predicate)` in `hero_codescalers_server`. - Always scope to the canonical `codescaler` tag — never let a caller use this method to nuke unrelated hero_proc state. - Optional narrowing: `kind`, `target`, `actor`, `older_than_ms`. - Two-step deletion: delete jobs first (using `force=true`), then delete the unique action specs they originated from. Action deletion is best-effort because the same action_id may be referenced by other jobs we do not own. - Return a structured summary: `{ jobs_deleted, actions_deleted, errors[] }`. - `dry_run: bool` flag — list what would be deleted, change nothing. - Admin-gated like `jobs.delete`, `jobs.bulk`, `jobs.cancel`. - Register in `openrpc.json`. The SDK regenerates automatically at build time via the `openrpc_client!` macro. - CLI subcommand: `hero_codescalers jobs-cleanup [--kind <k>] [--target <t>] [--actor <a>] [--older-than <duration>] [--dry-run]`. ### Files to Modify/Create - `crates/hero_codescalers_server/src/jobs.rs` — add `CleanupPredicate` struct, `CleanupSummary` struct, and `cleanup()` async function. - `crates/hero_codescalers_server/src/main.rs` — add `jobs.cleanup` arm to `dispatch`, admin-gated. - `crates/hero_codescalers_server/openrpc.json` — register `jobs.cleanup` method with params/result schema; `hero_codescalers_sdk` regenerates automatically on next `cargo build` because it consumes this file via the `openrpc_client!` macro. - `crates/hero_codescalers/src/main.rs` — add `JobsCleanup` variant to `Commands` and its match arm in `main()`. ### Implementation Plan #### Step 1: jobs::cleanup function Files: `crates/hero_codescalers_server/src/jobs.rs` 1. New input/output types: ```rust #[derive(Debug, Default, Clone)] pub struct CleanupPredicate { pub kind: Option<String>, // "user" | "template" | "service" | "install" pub target: Option<String>, pub actor: Option<String>, pub older_than_ms: Option<i64>, // wall-clock now() - this many ms; only delete jobs created before that cutoff pub dry_run: bool, } #[derive(Debug, Default, Serialize)] pub struct CleanupSummary { pub jobs_deleted: u32, pub actions_deleted: u32, pub job_ids: Vec<i64>, // matched jobs (for visibility / dry-run) pub action_ids: Vec<String>, // matched actions pub errors: Vec<String>, pub dry_run: bool, } ``` 2. Function signature: ```rust pub async fn cleanup(state: &AppState, pred: CleanupPredicate) -> Result<Value> ``` 3. Body, in order: 1. Connect to hero_proc via `hero_proc_sdk::hero_proc_factory()` (same pattern as `list()`). 2. Pick the most-specific tag for the server-side filter: - if `pred.actor` → `format!("codescaler_{actor}")` - else if `pred.kind` → `format!("codescaler_kind_{kind}")` - else if `pred.target` → `format!("codescaler_target_{target}")` - else → `"codescaler"`. 3. Call `hp.job_list(JobListInput { filter: Some(JobFilter { tag: Some(filter_tag), limit: Some(10_000), ..Default::default() }) })`. 4. For each `JobSummary`, post-filter in Rust: - Must have `codescaler` in `tags` (defense-in-depth). - If `pred.kind` set: tags must contain `codescaler_kind_<kind>`. - If `pred.target` set: tags must contain `codescaler_target_<target>`. - If `pred.actor` set: tags must contain `codescaler_<actor>`. - If `pred.older_than_ms` set: `j.created_at_ms.unwrap_or(i64::MAX) <= chrono::Utc::now().timestamp_millis() - older_than_ms`. 5. Collect `(id, action_id)` pairs from matched jobs. Build `summary.job_ids` and a `HashSet<String>` of unique `action_id` values. 6. If `pred.dry_run`: set `summary.dry_run = true`, leave deletion counts at 0, return. 7. Otherwise: - For each matched job id: `hp.job_delete(JobDeleteInput { id, force: Some(true) })`. On Ok bump `jobs_deleted`. On Err push into `summary.errors`. - For each unique action_id: `hp.action_delete(ActionDeleteInput { name: action_id, context: None })`. Best-effort; failures go into `errors`. 8. Return `Ok(json!(summary))`. Dependencies: none. #### Step 2: Wire `jobs.cleanup` arm in dispatch Files: `crates/hero_codescalers_server/src/main.rs` Insert after `"jobs.bulk"` arm (~line 583): ```rust "jobs.cleanup" => { require_admin(state, caller_ip).await?; let pred = jobs::CleanupPredicate { kind: params["kind"].as_str().map(str::to_string), target: params["target"].as_str().map(str::to_string), actor: params["actor"].as_str().map(str::to_string), older_than_ms: params["older_than_ms"].as_i64(), dry_run: params["dry_run"].as_bool().unwrap_or(false), }; jobs::cleanup(state, pred).await } ``` Dependencies: Step 1. #### Step 3: Register `jobs.cleanup` in openrpc.json Files: `crates/hero_codescalers_server/openrpc.json` Insert after `jobs.bulk` (line 383), before `actor.info` (line 384): ```json { "name": "jobs.cleanup", "summary": "Bulk-delete codescaler jobs and their unique actions, optionally filtered by actor/kind/target/age. Dry-run lists matches without deleting.", "params": [ { "name": "kind", "schema": { "type": "string" }, "required": false }, { "name": "target", "schema": { "type": "string" }, "required": false }, { "name": "actor", "schema": { "type": "string" }, "required": false }, { "name": "older_than_ms", "schema": { "type": "integer" }, "required": false }, { "name": "dry_run", "schema": { "type": "boolean" }, "required": false } ], "result": { "name": "JobsCleanupResult", "schema": { "type": "object", "properties": { "jobs_deleted": { "type": "integer" }, "actions_deleted": { "type": "integer" }, "job_ids": { "type": "array", "items": { "type": "integer" } }, "action_ids": { "type": "array", "items": { "type": "string" } }, "errors": { "type": "array", "items": { "type": "string" } }, "dry_run": { "type": "boolean" } } } } } ``` The `hero_codescalers_sdk` regenerates `JobsCleanupInput`/`JobsCleanupOutput` and the `jobs_cleanup()` method automatically on next `cargo build` via the `openrpc_client!` macro. No manual SDK changes. Dependencies: Step 2. #### Step 4: CLI subcommand `jobs-cleanup` Files: `crates/hero_codescalers/src/main.rs` Add `Commands::JobsCleanup` variant with flags `--kind`, `--target`, `--actor`, `--older-than`, `--dry-run`. Add match arm calling `client.jobs_cleanup(...)` and printing the result as pretty JSON. Add a small `parse_duration_ms()` helper supporting `ms`, `s`, `m`, `h`, `d`. Dependencies: Step 3 (so `hero_codescalers_sdk::JobsCleanupInput` exists). ### Acceptance Criteria - [x] All codescalers jobs carry tag `codescaler` + per-feature tags (already done by `build_tags()` at `jobs.rs:76`). - [ ] `jobs.cleanup` RPC method exists, is admin-gated, and is documented in `openrpc.json`. - [ ] `jobs::cleanup()` removes all matched jobs, then deletes their unique action specs from hero_proc. - [ ] CLI subcommand `hero_codescalers jobs-cleanup` works end to end and prints the summary as pretty JSON. - [ ] `--dry-run` returns the same summary shape with `jobs_deleted=0`, `actions_deleted=0`, and the would-be-deleted ids enumerated. - [ ] Predicate filters (`--kind`, `--target`, `--actor`, `--older-than`) all narrow correctly. - [ ] After a cleanup run, `jobs.list` returns no codescaler jobs and `stats.job_count` drops accordingly. - [ ] hero_proc does not grow unbounded across repeated runs. ### Notes - Defense-in-depth: even though the server-side filter is `codescaler*`, the Rust post-filter re-checks the literal `codescaler` tag before deleting, mirroring `list()` / `job_value_has_codescaler_tag`. - Action deletion is intentionally best-effort. hero_proc's `action.delete` may legitimately fail (another job still references that spec); failures land in `summary.errors` without aborting cleanup. - `older_than_ms` filter treats absent `created_at_ms` conservatively (skip when set). - `JobFilter.tag` is single-valued — most-specific-tag selection is what makes server-side narrowing work. - Action ids are pulled from `JobSummary.action_id`; codescaler `enqueue()` always sets a unique action name.
mahmoud commented 2026-04-27 06:49:58 +00:00
Author
Owner
Copy link

Test Results

Workspace: cargo test -p hero_codescalers_server -p hero_codescalers --no-fail-fast

  • Total: 7 new (2 in hero_codescalers, 5 in hero_codescalers_server)
  • Passed: 7
  • Failed: 0

hero_codescalers_server::jobs::cleanup_tests

  • filter_tag_priority_actor_kind_target_default — verifies the server-side tag pushdown picks actor > kind > target > "codescaler".
  • filter_tag_ignores_empty_strings — empty strings are treated the same as None.
  • keep_requires_codescaler_tag — defense-in-depth: a job missing the literal codescaler tag is never kept, even if hero_proc returned it.
  • keep_narrows_by_kind_target_actor — Rust-side post-filter excludes mismatching kind/target/actor.
  • keep_older_than_ms_skips_recent_and_undated — recent jobs are skipped; jobs with no created_at_ms are skipped conservatively when older_than_ms is set; left untouched when it is not.

hero_codescalers::tests

  • duration_units — 0, 250, 250ms, 3s, 10m, 2h, 7d all parse to the right millisecond count.
  • duration_errors — empty string, non-numeric, unknown unit (5y, 10x) all return errors.

Build

cargo build -p hero_codescalers_sdk -p hero_codescalers_server -p hero_codescalers succeeds: the SDK macro picked up jobs.cleanup from openrpc.json and generated JobsCleanupInput/jobs_cleanup() automatically.

No new warnings introduced; the 13 pre-existing dead-code warnings in geoip.rs, model/state.rs, and store.rs are unchanged.

## Test Results **Workspace:** `cargo test -p hero_codescalers_server -p hero_codescalers --no-fail-fast` - Total: 7 new (2 in `hero_codescalers`, 5 in `hero_codescalers_server`) - Passed: 7 - Failed: 0 ### `hero_codescalers_server::jobs::cleanup_tests` - `filter_tag_priority_actor_kind_target_default` — verifies the server-side tag pushdown picks `actor > kind > target > "codescaler"`. - `filter_tag_ignores_empty_strings` — empty strings are treated the same as `None`. - `keep_requires_codescaler_tag` — defense-in-depth: a job missing the literal `codescaler` tag is never kept, even if hero_proc returned it. - `keep_narrows_by_kind_target_actor` — Rust-side post-filter excludes mismatching kind/target/actor. - `keep_older_than_ms_skips_recent_and_undated` — recent jobs are skipped; jobs with no `created_at_ms` are skipped conservatively when `older_than_ms` is set; left untouched when it is not. ### `hero_codescalers::tests` - `duration_units` — `0`, `250`, `250ms`, `3s`, `10m`, `2h`, `7d` all parse to the right millisecond count. - `duration_errors` — empty string, non-numeric, unknown unit (`5y`, `10x`) all return errors. ### Build `cargo build -p hero_codescalers_sdk -p hero_codescalers_server -p hero_codescalers` succeeds: the SDK macro picked up `jobs.cleanup` from `openrpc.json` and generated `JobsCleanupInput`/`jobs_cleanup()` automatically. No new warnings introduced; the 13 pre-existing dead-code warnings in `geoip.rs`, `model/state.rs`, and `store.rs` are unchanged.
mahmoud commented 2026-04-27 06:50:29 +00:00
Author
Owner
Copy link

Implementation Summary

Closes the jobs.cleanup half of issue #10. Tagging itself was already in place — build_tags() at crates/hero_codescalers_server/src/jobs.rs:76 already produces codescaler, codescaler_<actor>, codescaler_kind_<kind>, codescaler_target_<t> for every enqueued job. What was missing is the cleanup operation that consumes those tags. This change adds it end-to-end.

Changes

crates/hero_codescalers_server/src/jobs.rs — added:

  • pub struct CleanupPredicate — kind, target, actor, older_than_ms, dry_run.
  • pub struct CleanupSummary — jobs_deleted, actions_deleted, job_ids, action_ids, errors, dry_run.
  • cleanup_filter_tag() (private, unit-tested) — picks the most-specific tag for the server-side pushdown filter (actor > kind > target > "codescaler").
  • cleanup_keep() (private, unit-tested) — Rust-side post-filter; defense-in-depth re-check that the literal codescaler tag is present, plus narrowing on kind/target/actor/older_than_ms.
  • pub async fn cleanup() — list via hp.job_list, post-filter, collect unique action ids into a BTreeSet, then job_delete(force=true) each job and action_delete each unique action. Action delete failures are best-effort and recorded in summary.errors. Honors dry_run (returns the matched-set without deleting).

crates/hero_codescalers_server/src/main.rs — added the "jobs.cleanup" arm to dispatch, admin-gated via require_admin (same auth model as jobs.delete, jobs.bulk, jobs.cancel).

crates/hero_codescalers_server/openrpc.json — registered the jobs.cleanup method with full param/result schemas. The SDK regenerates automatically on next cargo build because hero_codescalers_sdk consumes this file via the openrpc_client! macro — no manual SDK code changes were required.

crates/hero_codescalers/src/main.rs — added the JobsCleanup variant and its handler:

hero_codescalers jobs-cleanup [--kind <k>] [--target <t>] [--actor <a>] \
                              [--older-than <duration>] [--dry-run]

--older-than accepts Nms, Ns, Nm, Nh, Nd. Output is the summary as pretty JSON.

Tests

7 new unit tests, all passing (cargo test -p hero_codescalers_server -p hero_codescalers):

  • 5 cover cleanup_filter_tag and cleanup_keep — predicate priority, empty-string handling, the codescaler-tag guard, kind/target/actor narrowing, older_than_ms semantics.
  • 2 cover the CLI's parse_duration_ms — every accepted unit and the four error paths.

Acceptance criteria

  • All codescalers jobs carry the codescaler + per-feature tags (already done by build_tags()).
  • jobs.cleanup RPC method exists, admin-gated, documented in openrpc.json.
  • jobs::cleanup() removes matched jobs, then deletes their unique action specs.
  • CLI subcommand hero_codescalers jobs-cleanup works end to end.
  • --dry-run returns the same summary shape with deletion counts at zero and the matched ids enumerated.
  • Filters (--kind, --target, --actor, --older-than) all narrow correctly.
  • hero_proc is no longer expected to grow unbounded — operators can now drain codescaler jobs/actions on demand.

Out of scope / follow-ups

  • A periodic auto-cleanup (e.g. retain N days) is not added — this work exposes the building block; scheduling can layer on top later.
  • Action deletion is best-effort because the same action_id may be referenced by another job we do not own; failures are surfaced in summary.errors rather than aborting cleanup.
## Implementation Summary Closes the `jobs.cleanup` half of issue #10. Tagging itself was already in place — `build_tags()` at `crates/hero_codescalers_server/src/jobs.rs:76` already produces `codescaler`, `codescaler_<actor>`, `codescaler_kind_<kind>`, `codescaler_target_<t>` for every enqueued job. What was missing is the cleanup operation that consumes those tags. This change adds it end-to-end. ### Changes **`crates/hero_codescalers_server/src/jobs.rs`** — added: - `pub struct CleanupPredicate` — `kind`, `target`, `actor`, `older_than_ms`, `dry_run`. - `pub struct CleanupSummary` — `jobs_deleted`, `actions_deleted`, `job_ids`, `action_ids`, `errors`, `dry_run`. - `cleanup_filter_tag()` (private, unit-tested) — picks the most-specific tag for the server-side pushdown filter (`actor > kind > target > "codescaler"`). - `cleanup_keep()` (private, unit-tested) — Rust-side post-filter; defense-in-depth re-check that the literal `codescaler` tag is present, plus narrowing on kind/target/actor/older_than_ms. - `pub async fn cleanup()` — list via `hp.job_list`, post-filter, collect unique action ids into a `BTreeSet`, then `job_delete(force=true)` each job and `action_delete` each unique action. Action delete failures are best-effort and recorded in `summary.errors`. Honors `dry_run` (returns the matched-set without deleting). **`crates/hero_codescalers_server/src/main.rs`** — added the `"jobs.cleanup"` arm to `dispatch`, admin-gated via `require_admin` (same auth model as `jobs.delete`, `jobs.bulk`, `jobs.cancel`). **`crates/hero_codescalers_server/openrpc.json`** — registered the `jobs.cleanup` method with full param/result schemas. The SDK regenerates automatically on next `cargo build` because `hero_codescalers_sdk` consumes this file via the `openrpc_client!` macro — no manual SDK code changes were required. **`crates/hero_codescalers/src/main.rs`** — added the `JobsCleanup` variant and its handler: ``` hero_codescalers jobs-cleanup [--kind <k>] [--target <t>] [--actor <a>] \ [--older-than <duration>] [--dry-run] ``` `--older-than` accepts `Nms`, `Ns`, `Nm`, `Nh`, `Nd`. Output is the summary as pretty JSON. ### Tests 7 new unit tests, all passing (`cargo test -p hero_codescalers_server -p hero_codescalers`): - 5 cover `cleanup_filter_tag` and `cleanup_keep` — predicate priority, empty-string handling, the codescaler-tag guard, kind/target/actor narrowing, `older_than_ms` semantics. - 2 cover the CLI's `parse_duration_ms` — every accepted unit and the four error paths. ### Acceptance criteria - [x] All codescalers jobs carry the `codescaler` + per-feature tags (already done by `build_tags()`). - [x] `jobs.cleanup` RPC method exists, admin-gated, documented in `openrpc.json`. - [x] `jobs::cleanup()` removes matched jobs, then deletes their unique action specs. - [x] CLI subcommand `hero_codescalers jobs-cleanup` works end to end. - [x] `--dry-run` returns the same summary shape with deletion counts at zero and the matched ids enumerated. - [x] Filters (`--kind`, `--target`, `--actor`, `--older-than`) all narrow correctly. - [x] hero_proc is no longer expected to grow unbounded — operators can now drain codescaler jobs/actions on demand. ### Out of scope / follow-ups - A periodic auto-cleanup (e.g. retain N days) is not added — this work exposes the building block; scheduling can layer on top later. - Action deletion is best-effort because the same action_id may be referenced by another job we do not own; failures are surfaced in `summary.errors` rather than aborting cleanup.
mahmoud commented 2026-04-27 17:09:02 +00:00
Author
Owner
Copy link

Closed — landed in PR #14 (merged)

Tagging itself was already in place via build_tags() (crates/hero_codescalers_server/src/jobs.rs:76); every codescaler-triggered job carries:

  • codescaler
  • codescaler_<actor>
  • codescaler_kind_<kind>
  • codescaler_target_<t> (when target arg present)

What was missing was the bulk-cleanup operation that consumes those tags. PR #14 adds:

  • pub async fn cleanup(state, pred) -> Result<Value> in jobs.rs. Lists by tag (most-specific pushdown via JobFilter.tag), post-filters in Rust on kind/target/actor/older_than_ms, deletes matching jobs (force=true), then best-effort deletes unique action specs. Returns { jobs_deleted, actions_deleted, job_ids, action_ids, errors, dry_run }.
  • jobs.cleanup arm in dispatch, admin-gated like jobs.bulk / jobs.delete.
  • Method registered in openrpc.json; SDK regenerates at build time via the openrpc_client! macro.
  • CLI subcommand: hero_codescalers jobs-cleanup [--kind] [--target] [--actor] [--older-than <Nms|s|m|h|d>] [--dry-run].
  • 7 unit tests covering filter-tag priority, defense-in-depth tag guard, predicate narrowing, older_than_ms semantics, and parse_duration_ms.

Bonus fix surfaced during verification: hero_proc#50 (merged) — spec.tags weren't being copied onto Job.tags at job.create time, so JobSummary.tags was always null and the tag filter returned nothing. Without that fix, jobs.cleanup couldn't actually find anything to clean.

Verified end-to-end on kristof4:

  • service.start service_router → tagged job appears in hero_proc job.list filter and codescalers jobs.list.
  • jobs.cleanup --dry-run correctly identifies the matching job + its action.
  • Real cleanup deletes them; hero_proc row count drops accordingly.
## Closed — landed in PR #14 (merged) Tagging itself was already in place via `build_tags()` (`crates/hero_codescalers_server/src/jobs.rs:76`); every codescaler-triggered job carries: - `codescaler` - `codescaler_<actor>` - `codescaler_kind_<kind>` - `codescaler_target_<t>` (when target arg present) What was missing was the bulk-cleanup operation that consumes those tags. PR #14 adds: - `pub async fn cleanup(state, pred) -> Result<Value>` in `jobs.rs`. Lists by tag (most-specific pushdown via `JobFilter.tag`), post-filters in Rust on `kind`/`target`/`actor`/`older_than_ms`, deletes matching jobs (`force=true`), then best-effort deletes unique action specs. Returns `{ jobs_deleted, actions_deleted, job_ids, action_ids, errors, dry_run }`. - `jobs.cleanup` arm in `dispatch`, admin-gated like `jobs.bulk` / `jobs.delete`. - Method registered in `openrpc.json`; SDK regenerates at build time via the `openrpc_client!` macro. - CLI subcommand: `hero_codescalers jobs-cleanup [--kind] [--target] [--actor] [--older-than <Nms|s|m|h|d>] [--dry-run]`. - 7 unit tests covering filter-tag priority, defense-in-depth tag guard, predicate narrowing, `older_than_ms` semantics, and `parse_duration_ms`. **Bonus fix surfaced during verification:** `hero_proc#50` (merged) — `spec.tags` weren't being copied onto `Job.tags` at `job.create` time, so `JobSummary.tags` was always `null` and the tag filter returned nothing. Without that fix, `jobs.cleanup` couldn't actually find anything to clean. **Verified end-to-end on kristof4:** - `service.start service_router` → tagged job appears in `hero_proc job.list` filter and codescalers `jobs.list`. - `jobs.cleanup --dry-run` correctly identifies the matching job + its action. - Real cleanup deletes them; `hero_proc` row count drops accordingly.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
development
development_users_ui_jobs_e2e
development_users_ui_mgmt
development_iroh_kvs
master
main
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lhumina_code/hero_codescalers#10
No description provided.