lhumina_code/hero_slides
2
0
Fork 0
Code Issues 10 Pull requests Projects Releases 3 Packages Wiki Activity Actions

feat(voice): integrate hero_voice_admin for in-app transcription #84

New issue
Open
opened 2026-05-21 09:22:36 +00:00 by casper-stevens · 2 comments
casper-stevens commented 2026-05-21 09:22:36 +00:00
Member
Copy link

Goal

Replace hero_slides internal voice recording (voice.transcribeAsync RPC) with the hero_voice_admin service, which provides a proper transcription pipeline with local STT (sherpa-onnx Parakeet) and Groq cloud fallback.

What was attempted

Commit f38547a wired all per-field mic buttons to POST audio to /hero_voice/ui/transcribe via hero_router. Reverted in 642b4af because hero_voice_admin does not exist yet — only a hero_voice CLI binary is installed, so the endpoint returns 404.

Blocked on

  • hero_voice_admin service needs to be built and deployed (serves /hero_voice/ui/transcribe, /hero_voice/ui/rpc, and the <hero-voice-bar> widget bundle)
  • hero_voiced local STT service (sherpa-onnx Parakeet on 127.0.0.1:8094) needs to be running, or GROQ_API_KEY set in hero_proc secrets as cloud fallback

Integration plan (ready to apply once hero_voice_admin exists)

  1. Load widget assets from /hero_voice/ui/voice-widget/ in base.html
  2. Replace pollAiJob('voice.transcribeAsync', ...) calls in dashboard.js and slide_edit.js with fetch('/hero_voice/ui/transcribe', { method: 'POST', body: FormData }) returning { text }
  3. Optionally add <hero-voice-bar> to the navbar for global focus-aware transcription
## Goal Replace hero_slides internal voice recording (`voice.transcribeAsync` RPC) with the `hero_voice_admin` service, which provides a proper transcription pipeline with local STT (sherpa-onnx Parakeet) and Groq cloud fallback. ## What was attempted Commit f38547a wired all per-field mic buttons to POST audio to `/hero_voice/ui/transcribe` via hero_router. Reverted in 642b4af because `hero_voice_admin` does not exist yet — only a `hero_voice` CLI binary is installed, so the endpoint returns 404. ## Blocked on - `hero_voice_admin` service needs to be built and deployed (serves `/hero_voice/ui/transcribe`, `/hero_voice/ui/rpc`, and the `<hero-voice-bar>` widget bundle) - `hero_voiced` local STT service (sherpa-onnx Parakeet on `127.0.0.1:8094`) needs to be running, or `GROQ_API_KEY` set in hero_proc secrets as cloud fallback ## Integration plan (ready to apply once hero_voice_admin exists) 1. Load widget assets from `/hero_voice/ui/voice-widget/` in `base.html` 2. Replace `pollAiJob('voice.transcribeAsync', ...)` calls in `dashboard.js` and `slide_edit.js` with `fetch('/hero_voice/ui/transcribe', { method: 'POST', body: FormData })` returning `{ text }` 3. Optionally add `<hero-voice-bar>` to the navbar for global focus-aware transcription
scott commented 2026-05-21 17:15:27 +00:00
Owner
Copy link

I would suggest that we use hero-voice-bar everywhere that we need voice functionality. In terms of the slides app, this could go once in the main navbar header. But most of the text input happens in pop-up modals, so I would just suggest to add the bar into each of those modals as well. Then the existing voice functionality in slides can just be stripped out.

I would suggest that we use `hero-voice-bar` everywhere that we need voice functionality. In terms of the slides app, this could go once in the main navbar header. But most of the text input happens in pop-up modals, so I would just suggest to add the bar into each of those modals as well. Then the existing voice functionality in slides can just be stripped out.
❤️ 1
casper-stevens commented 2026-05-22 10:33:05 +00:00
Author
Member
Copy link

Implementation Spec — Issue #84

Objective

Replace the internal voice recording pipeline in hero_slides (which called voice.transcribeAsync via pollAiJob and drove its own MediaRecorder lifecycle) with the hero-voice-bar web component from hero_voice_admin. One instance lives in the navbar (tracks the last-focused field globally), and additional instances are placed inside each modal/overlay that contains text inputs. All internal voice code is deleted.

Requirements

  • Load the widget scripts (components.js, floating.js, boost.js, bar.js) and CSS (voice-widget.css) from /hero_voice/ui/voice-widget/ in both page templates (base.html and slide_edit.html).
  • Place <hero-voice-bar> in the navbar of base.html so it is always visible on the main dashboard page.
  • Place <hero-voice-bar> inside each modal and overlay panel in index.html that contains text inputs where voice input makes sense (5 locations).
  • Place <hero-voice-bar> in slide_edit.html for both the main editor textarea and the instruct panel input.
  • Remove all existing internal voice code from dashboard.js and slide_edit.js (MediaRecorder-based functions, module-level variables, event-listener wires).
  • Remove old mic start/stop button pairs and voice-status span elements from index.html and slide_edit.html.
  • Remove the Transcription Settings admin panel section from index.html.
  • Delete the .slide-edit-voice-status CSS rule from slide_edit.css.
  • No Rust server changes required.

Files to Modify

  • crates/hero_slides_admin/templates/base.html — Add widget CSS/JS, add <hero-voice-bar> in navbar
  • crates/hero_slides_admin/templates/index.html — Add <hero-voice-bar> in 5 locations, remove old mic buttons/status spans, remove Transcription Settings panel
  • crates/hero_slides_admin/static/js/dashboard.js — Delete voice functions and module-level variables
  • crates/hero_slides_admin/templates/slide_edit.html — Add widget CSS/JS, add <hero-voice-bar> in 2 locations, remove old mic buttons/status spans
  • crates/hero_slides_admin/static/js/slide_edit.js — Delete createRecorder factory, recorder instances, event-listener wires
  • crates/hero_slides_admin/static/css/slide_edit.css — Delete .slide-edit-voice-status rule

Implementation Plan

Step 1 — Load widget assets in base.html

Files: crates/hero_slides_admin/templates/base.html
Dependencies: none

  • In <head>, add after the existing connection-status.css link: <link href="/hero_voice/ui/voice-widget/voice-widget.css" rel="stylesheet" />
  • Before </body>, add the four scripts in order before dashboard.js: components.js, floating.js, boost.js, bar.js
  • Add <hero-voice-bar></hero-voice-bar> in the navbar ms-auto div, immediately after <hero-connection-status>

Step 2 — Load widget assets in slide_edit.html

Files: crates/hero_slides_admin/templates/slide_edit.html
Dependencies: none

  • Apply the same CSS link in <head>
  • Add the four JS scripts before slide_edit.js at the bottom of the file

Can run in parallel with Step 1.

Step 3 — Add <hero-voice-bar> and strip old voice UI from slide_edit.html

Files: crates/hero_slides_admin/templates/slide_edit.html
Dependencies: Step 2

  • Replace the two mic button pairs and both voice-status divs with <hero-voice-bar> (one in slide-edit-instruct-controls, one in slide-edit-actions)
  • Remove <div id="voice-status"> and <div id="instruct-voice-status"> elements

Step 4 — Remove voice JS from slide_edit.js

Files: crates/hero_slides_admin/static/js/slide_edit.js, crates/hero_slides_admin/static/css/slide_edit.css
Dependencies: none

  • Delete the createRecorder factory function
  • Delete the mainRecorder and instructRecorder constant declarations
  • Delete the four event-listener wires for the removed mic buttons
  • Delete the .slide-edit-voice-status CSS rule from slide_edit.css

Can run in parallel with Steps 1–3.

Step 5 — Add <hero-voice-bar> and strip old voice UI from index.html

Files: crates/hero_slides_admin/templates/index.html
Dependencies: none

5 locations to update (replace old mic buttons + status spans with <hero-voice-bar>):

  1. Markdown editor overlay topbar (btn-record-start / btn-record-stop + #voice-status)
  2. Markdown editor instruct panel (btn-instruct-record / btn-instruct-record-stop + #instruct-voice-status)
  3. Theme editor overlay topbar (btn-theme-record-start / btn-theme-record-stop + #theme-voice-status)
  4. Theme editor instruct panel (btn-theme-instruct-record / btn-theme-instruct-record-stop + #theme-instruct-voice-status)
  5. Create Slide modal (btn-create-slide-record / btn-create-slide-record-stop + #create-slide-voice-status)

Also remove the Transcription Settings admin panel section (<div class="admin-section" id="section-transcription-settings">).

Can run in parallel with Steps 1–4.

Step 6 — Remove voice JS from dashboard.js

Files: crates/hero_slides_admin/static/js/dashboard.js
Dependencies: none

Delete:

  • startCreateSlideRecording and stopCreateSlideRecording functions
  • startThemeInstructRecording and stopThemeInstructRecording functions
  • startRecording, stopRecording, sendAudioToServer functions
  • saveTranscriptionSettings, onTranscriptionBackendChange, loadTranscriptionSettings functions
  • Module-level variables: createSlideMediaRecorder, mediaRecorder, audioChunks, isRecording

Can run in parallel with Steps 1–5.

Acceptance Criteria

  • <hero-voice-bar> appears in the navbar on the main dashboard page
  • <hero-voice-bar> is visible inside the Create Slide modal, markdown editor topbar, markdown instruct panel, theme editor topbar, and theme instruct panel
  • <hero-voice-bar> is visible in the slide_edit page for both the main editor and the instruct panel
  • No calls to pollAiJob('voice.transcribeAsync', ...) remain in the JS codebase
  • No MediaRecorder-based voice code remains in dashboard.js or slide_edit.js
  • No broken onclick references: all removed functions are fully gone from both HTML and JS files
  • Transcription Settings admin panel is gone from the Admin tab
  • .slide-edit-voice-status CSS rule is deleted
  • No Rust server changes

Notes

  • The widget scripts use absolute paths (/hero_voice/ui/transcribe) internally; no {{ base_path }} prefix needed for the script src attributes.
  • The Rust-side voice RPC infrastructure (voice.transcribeAsync handlers, VoiceTranscribe job type, voice.rs lib, Rhai voice_module.rs, openrpc.json voice entries) is out of scope — only the UI/JS layer is touched.
  • hero_voice_admin does not need to be running for the UI changes to be committed; the widget will gracefully handle unavailability.
## Implementation Spec — Issue #84 ### Objective Replace the internal voice recording pipeline in hero_slides (which called `voice.transcribeAsync` via `pollAiJob` and drove its own MediaRecorder lifecycle) with the `hero-voice-bar` web component from `hero_voice_admin`. One instance lives in the navbar (tracks the last-focused field globally), and additional instances are placed inside each modal/overlay that contains text inputs. All internal voice code is deleted. ### Requirements - Load the widget scripts (`components.js`, `floating.js`, `boost.js`, `bar.js`) and CSS (`voice-widget.css`) from `/hero_voice/ui/voice-widget/` in both page templates (`base.html` and `slide_edit.html`). - Place `<hero-voice-bar>` in the navbar of `base.html` so it is always visible on the main dashboard page. - Place `<hero-voice-bar>` inside each modal and overlay panel in `index.html` that contains text inputs where voice input makes sense (5 locations). - Place `<hero-voice-bar>` in `slide_edit.html` for both the main editor textarea and the instruct panel input. - Remove all existing internal voice code from `dashboard.js` and `slide_edit.js` (MediaRecorder-based functions, module-level variables, event-listener wires). - Remove old mic start/stop button pairs and voice-status span elements from `index.html` and `slide_edit.html`. - Remove the Transcription Settings admin panel section from `index.html`. - Delete the `.slide-edit-voice-status` CSS rule from `slide_edit.css`. - No Rust server changes required. ### Files to Modify - `crates/hero_slides_admin/templates/base.html` — Add widget CSS/JS, add `<hero-voice-bar>` in navbar - `crates/hero_slides_admin/templates/index.html` — Add `<hero-voice-bar>` in 5 locations, remove old mic buttons/status spans, remove Transcription Settings panel - `crates/hero_slides_admin/static/js/dashboard.js` — Delete voice functions and module-level variables - `crates/hero_slides_admin/templates/slide_edit.html` — Add widget CSS/JS, add `<hero-voice-bar>` in 2 locations, remove old mic buttons/status spans - `crates/hero_slides_admin/static/js/slide_edit.js` — Delete `createRecorder` factory, recorder instances, event-listener wires - `crates/hero_slides_admin/static/css/slide_edit.css` — Delete `.slide-edit-voice-status` rule ### Implementation Plan #### Step 1 — Load widget assets in `base.html` Files: `crates/hero_slides_admin/templates/base.html` Dependencies: none - In `<head>`, add after the existing `connection-status.css` link: `<link href="/hero_voice/ui/voice-widget/voice-widget.css" rel="stylesheet" />` - Before `</body>`, add the four scripts in order before `dashboard.js`: `components.js`, `floating.js`, `boost.js`, `bar.js` - Add `<hero-voice-bar></hero-voice-bar>` in the navbar `ms-auto` div, immediately after `<hero-connection-status>` #### Step 2 — Load widget assets in `slide_edit.html` Files: `crates/hero_slides_admin/templates/slide_edit.html` Dependencies: none - Apply the same CSS link in `<head>` - Add the four JS scripts before `slide_edit.js` at the bottom of the file Can run in parallel with Step 1. #### Step 3 — Add `<hero-voice-bar>` and strip old voice UI from `slide_edit.html` Files: `crates/hero_slides_admin/templates/slide_edit.html` Dependencies: Step 2 - Replace the two mic button pairs and both voice-status divs with `<hero-voice-bar>` (one in `slide-edit-instruct-controls`, one in `slide-edit-actions`) - Remove `<div id="voice-status">` and `<div id="instruct-voice-status">` elements #### Step 4 — Remove voice JS from `slide_edit.js` Files: `crates/hero_slides_admin/static/js/slide_edit.js`, `crates/hero_slides_admin/static/css/slide_edit.css` Dependencies: none - Delete the `createRecorder` factory function - Delete the `mainRecorder` and `instructRecorder` constant declarations - Delete the four event-listener wires for the removed mic buttons - Delete the `.slide-edit-voice-status` CSS rule from `slide_edit.css` Can run in parallel with Steps 1–3. #### Step 5 — Add `<hero-voice-bar>` and strip old voice UI from `index.html` Files: `crates/hero_slides_admin/templates/index.html` Dependencies: none 5 locations to update (replace old mic buttons + status spans with `<hero-voice-bar>`): 1. Markdown editor overlay topbar (`btn-record-start` / `btn-record-stop` + `#voice-status`) 2. Markdown editor instruct panel (`btn-instruct-record` / `btn-instruct-record-stop` + `#instruct-voice-status`) 3. Theme editor overlay topbar (`btn-theme-record-start` / `btn-theme-record-stop` + `#theme-voice-status`) 4. Theme editor instruct panel (`btn-theme-instruct-record` / `btn-theme-instruct-record-stop` + `#theme-instruct-voice-status`) 5. Create Slide modal (`btn-create-slide-record` / `btn-create-slide-record-stop` + `#create-slide-voice-status`) Also remove the Transcription Settings admin panel section (`<div class="admin-section" id="section-transcription-settings">`). Can run in parallel with Steps 1–4. #### Step 6 — Remove voice JS from `dashboard.js` Files: `crates/hero_slides_admin/static/js/dashboard.js` Dependencies: none Delete: - `startCreateSlideRecording` and `stopCreateSlideRecording` functions - `startThemeInstructRecording` and `stopThemeInstructRecording` functions - `startRecording`, `stopRecording`, `sendAudioToServer` functions - `saveTranscriptionSettings`, `onTranscriptionBackendChange`, `loadTranscriptionSettings` functions - Module-level variables: `createSlideMediaRecorder`, `mediaRecorder`, `audioChunks`, `isRecording` Can run in parallel with Steps 1–5. ### Acceptance Criteria - [ ] `<hero-voice-bar>` appears in the navbar on the main dashboard page - [ ] `<hero-voice-bar>` is visible inside the Create Slide modal, markdown editor topbar, markdown instruct panel, theme editor topbar, and theme instruct panel - [ ] `<hero-voice-bar>` is visible in the slide_edit page for both the main editor and the instruct panel - [ ] No calls to `pollAiJob('voice.transcribeAsync', ...)` remain in the JS codebase - [ ] No MediaRecorder-based voice code remains in `dashboard.js` or `slide_edit.js` - [ ] No broken `onclick` references: all removed functions are fully gone from both HTML and JS files - [ ] Transcription Settings admin panel is gone from the Admin tab - [ ] `.slide-edit-voice-status` CSS rule is deleted - [ ] No Rust server changes ### Notes - The widget scripts use absolute paths (`/hero_voice/ui/transcribe`) internally; no `{{ base_path }}` prefix needed for the script `src` attributes. - The Rust-side voice RPC infrastructure (`voice.transcribeAsync` handlers, `VoiceTranscribe` job type, `voice.rs` lib, Rhai `voice_module.rs`, openrpc.json voice entries) is out of scope — only the UI/JS layer is touched. - `hero_voice_admin` does not need to be running for the UI changes to be committed; the widget will gracefully handle unavailability.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
development
integration
backup/development-pre-integration-move
fix_update_cargo
development_freeze
backup/development-pptx-mockup-98
backup/integration-pre-move
development_despiegk
development_kristof2
development_kristof
latest
latest-dev
v0.1.0-rc2
v0.1.0-rc1
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lhumina_code/hero_slides#84
No description provided.