lhumina_code/hero_slides
1
0
Fork 0
Code Issues 17 Pull requests Projects Releases 2 Packages Wiki Activity Actions

pdf/image to instructions #67

New issue
Open
opened 2026-05-14 03:26:04 +00:00 by despiegk · 8 comments
despiegk commented 2026-05-14 03:26:04 +00:00
Owner
Copy link

image

  • allow copy paste of image (e.g. got image in clipboard)
  • when doing the instruction, make a job so we can follow
  • improve the prompt, what are good instructions for a theme
  • make sure user can change those instructions (prompt templates)
![image](/attachments/81abd367-80ef-499d-bacc-c044ceeff367) - allow copy paste of image (e.g. got image in clipboard) - when doing the instruction, make a job so we can follow - improve the prompt, what are good instructions for a theme - make sure user can change those instructions (prompt templates)
image.png
100 KiB
casper-stevens commented 2026-05-15 08:22:47 +00:00
Member
Copy link

Implementation Spec: Issue #67 — PDF/Image to Instructions

Objective

Extend the "Extract from image / PDF" feature so it targets the instructions.md file (not just theme.md), adds clipboard paste as an input method, wraps the operation in a trackable background job (via the existing pollAiJob / JobManager infrastructure), improves the AI prompt for instructions extraction, and exposes system_extract_instructions_from_image.md as a user-editable prompt template.

Current State Summary

  • bgExtractThemeFromFile() in dashboard.js opens a file picker, reads base64, calls bg.extractThemeAsync via pollAiJob. This saves to theme.md.
  • bg.extractThemeAsync handler in generate_job.rs calls resolve_extract_theme_source() (accepts {data, mime_type} for inline base64 OR {folder, file} for on-disk files), then submits a CLI job via submit_extract_theme_job().
  • The CLI job runs hero_slides theme extract <unc> --from <file>, which calls extract_theme_from_image() / extract_theme_from_pdf() in generator.rs, backed by system_extract_theme.md prompt.
  • The system_extract_theme.md prompt is in ALL_TEMPLATES and editable via the Templates tab.
  • pollAiJob polls agent.aiJobResult at 800ms intervals; on done it returns status.result containing { content }.
  • There is no equivalent flow for instructions.md from an image/PDF. instruct_instructions modifies existing instructions but there is no "extract instructions from visual reference" path.
  • No clipboard paste handling exists anywhere in the codebase.

Requirements

  • Allow the user to paste an image from the clipboard (Ctrl+V / Cmd+V) as the input for extraction, in addition to the existing file-picker path
  • Add an "Extract instructions from image/PDF" action that sends the image/PDF to the AI and writes the result to instructions.md
  • The extraction must run as a background job tracked by pollAiJob / JobManager — same pattern as bg.extractThemeAsync
  • New AI prompt system_extract_instructions_from_image.md must produce well-structured instructions.md content (audience, tone, structure, key messages, slide outline, goals) from slide images
  • The new prompt template must be user-editable via the Templates tab (added to ALL_TEMPLATES in prompts.rs and TEMPLATE_TO_PROMPT_KEY in dashboard.js)
  • A "Extract instructions from image/PDF" button must appear in the Instructions editor section of the dashboard
  • Clipboard paste (Ctrl+V with an image) should work from both the Instructions and Theme editor contexts

Files to Modify or Create

File Action Description
crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md Create New AI prompt for extracting instructions.md from a visual reference
crates/hero_slides_lib/src/prompts.rs Modify Add const + entry in ALL_TEMPLATES
crates/hero_slides_lib/src/generator.rs Modify Add extract_instructions_from_image() and extract_instructions_from_pdf()
crates/hero_slides_server/src/generate_job.rs Modify Add job submission + handler for bg.extractInstructionsAsync
crates/hero_slides_server/src/rpc.rs Modify Route bg.extractInstructionsAsync to the new handler
crates/hero_slides_admin/static/js/dashboard.js Modify Add bgExtractInstructionsFromFile(), clipboard paste handler, template key
crates/hero_slides_admin/templates/index.html Modify Add "Extract instructions from image/PDF" button + paste hint

Implementation Plan

Step 1: Create the AI prompt template

File: crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md
Dependencies: none

Create the prompt instructing the AI to analyze slide images/PDFs and produce a structured instructions.md covering: audience, tone, key messages, slide structure/outline, and goals. Output must begin with # Deck Instructions and use ## subheadings. No explanations, no code fences, no preamble — only the instructions.md content.

Step 2: Register the template in prompts.rs

File: crates/hero_slides_lib/src/prompts.rs
Dependencies: Step 1

Add pub const DEFAULT_SYSTEM_EXTRACT_INSTRUCTIONS_FROM_IMAGE constant and add it to ALL_TEMPLATES.

Step 3: Add extraction functions to generator.rs

File: crates/hero_slides_lib/src/generator.rs
Dependencies: Step 2

Add extract_instructions_from_image() and extract_instructions_from_pdf() mirroring the existing theme extraction functions, using the new prompt key system_extract_instructions_from_image.md.

Step 4: Add the CLI subcommand

File: crates/hero_slides/src/main.rs + relevant rhai module
Dependencies: Step 3

Add hero_slides instructions extract <unc> --from <file> CLI command mirroring theme extract, calling the new generator functions and writing result to instructions.md.

Step 5: Add backend job submission and handler

File: crates/hero_slides_server/src/generate_job.rs
Dependencies: Step 4

Add submit_extract_instructions_job() and handle_bg_extract_instructions_async() (plus status/logs handlers) mirroring the theme extraction job pattern. Add JobKind::ExtractInstructions variant.

Step 6: Route the new RPC method

File: crates/hero_slides_server/src/rpc.rs
Dependencies: Step 5

Route bg.extractInstructionsAsync, bg.extractInstructionsJobStatus, bg.extractInstructionsJobLogs to the new handlers.

Step 7: Add clipboard paste support + bgExtractInstructionsFromFile to dashboard.js

File: crates/hero_slides_admin/static/js/dashboard.js
Dependencies: Steps 5–6

Add bgExtractInstructionsFromFile() calling pollAiJob('bg.extractInstructionsAsync', ...), add a global paste event listener that detects image clipboard data and routes to instructions or theme extraction based on which editor is active, and register the new template key in TEMPLATE_TO_PROMPT_KEY.

Step 8: Add UI button to index.html

File: crates/hero_slides_admin/templates/index.html
Dependencies: Step 7

Add "Extract from image / PDF" button in the Instructions editor section and a paste hint (Ctrl+V).

Acceptance Criteria

  • Pasting an image (Ctrl+V) while the Instructions editor is open triggers instructions extraction with spinner feedback
  • Pasting an image while the Theme editor is open triggers theme extraction (existing feature now clipboard-accessible)
  • "Extract from image / PDF" button exists in the Instructions editor section
  • File picker accepts PNG, JPEG, WebP, GIF, PDF
  • Extraction runs as a background job visible in the jobs panel
  • On completion, instructions textarea is populated with AI-generated content (Audience, Tone, Key Messages, Slide Outline, Goals)
  • New prompt template appears in Templates tab and is user-editable/resettable
  • Toast notifications for: job started, success, failure
  • No regression to existing theme extraction flow

Notes

  • The pollAiJob pattern already provides job tracking; the new method just needs the same register_with_manager_inputs call with a temp output file in generate_job.rs.
  • resolve_extract_theme_source is reusable for the instructions extraction since the input shape (base64 inline or folder/file) is identical.
  • Adding ExtractInstructions to JobKind requires updating both as_str() and default_timeout() in jobs/types.rs.
## Implementation Spec: Issue #67 — PDF/Image to Instructions ### Objective Extend the "Extract from image / PDF" feature so it targets the **instructions.md** file (not just theme.md), adds clipboard paste as an input method, wraps the operation in a trackable background job (via the existing `pollAiJob` / `JobManager` infrastructure), improves the AI prompt for instructions extraction, and exposes `system_extract_instructions_from_image.md` as a user-editable prompt template. --- ### Current State Summary - `bgExtractThemeFromFile()` in `dashboard.js` opens a file picker, reads base64, calls `bg.extractThemeAsync` via `pollAiJob`. This saves to `theme.md`. - `bg.extractThemeAsync` handler in `generate_job.rs` calls `resolve_extract_theme_source()` (accepts `{data, mime_type}` for inline base64 OR `{folder, file}` for on-disk files), then submits a CLI job via `submit_extract_theme_job()`. - The CLI job runs `hero_slides theme extract <unc> --from <file>`, which calls `extract_theme_from_image()` / `extract_theme_from_pdf()` in `generator.rs`, backed by `system_extract_theme.md` prompt. - The `system_extract_theme.md` prompt is in `ALL_TEMPLATES` and editable via the Templates tab. - `pollAiJob` polls `agent.aiJobResult` at 800ms intervals; on `done` it returns `status.result` containing `{ content }`. - There is no equivalent flow for **instructions.md** from an image/PDF. `instruct_instructions` modifies existing instructions but there is no "extract instructions from visual reference" path. - No clipboard paste handling exists anywhere in the codebase. --- ### Requirements - Allow the user to paste an image from the clipboard (Ctrl+V / Cmd+V) as the input for extraction, in addition to the existing file-picker path - Add an "Extract instructions from image/PDF" action that sends the image/PDF to the AI and writes the result to `instructions.md` - The extraction must run as a background job tracked by `pollAiJob` / `JobManager` — same pattern as `bg.extractThemeAsync` - New AI prompt `system_extract_instructions_from_image.md` must produce well-structured `instructions.md` content (audience, tone, structure, key messages, slide outline, goals) from slide images - The new prompt template must be user-editable via the Templates tab (added to `ALL_TEMPLATES` in `prompts.rs` and `TEMPLATE_TO_PROMPT_KEY` in `dashboard.js`) - A "Extract instructions from image/PDF" button must appear in the Instructions editor section of the dashboard - Clipboard paste (Ctrl+V with an image) should work from both the Instructions and Theme editor contexts --- ### Files to Modify or Create | File | Action | Description | |---|---|---| | `crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md` | Create | New AI prompt for extracting instructions.md from a visual reference | | `crates/hero_slides_lib/src/prompts.rs` | Modify | Add const + entry in ALL_TEMPLATES | | `crates/hero_slides_lib/src/generator.rs` | Modify | Add `extract_instructions_from_image()` and `extract_instructions_from_pdf()` | | `crates/hero_slides_server/src/generate_job.rs` | Modify | Add job submission + handler for `bg.extractInstructionsAsync` | | `crates/hero_slides_server/src/rpc.rs` | Modify | Route `bg.extractInstructionsAsync` to the new handler | | `crates/hero_slides_admin/static/js/dashboard.js` | Modify | Add `bgExtractInstructionsFromFile()`, clipboard paste handler, template key | | `crates/hero_slides_admin/templates/index.html` | Modify | Add "Extract instructions from image/PDF" button + paste hint | --- ### Implementation Plan #### Step 1: Create the AI prompt template **File:** `crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md` **Dependencies:** none Create the prompt instructing the AI to analyze slide images/PDFs and produce a structured `instructions.md` covering: audience, tone, key messages, slide structure/outline, and goals. Output must begin with `# Deck Instructions` and use `##` subheadings. No explanations, no code fences, no preamble — only the instructions.md content. #### Step 2: Register the template in `prompts.rs` **File:** `crates/hero_slides_lib/src/prompts.rs` **Dependencies:** Step 1 Add `pub const DEFAULT_SYSTEM_EXTRACT_INSTRUCTIONS_FROM_IMAGE` constant and add it to `ALL_TEMPLATES`. #### Step 3: Add extraction functions to `generator.rs` **File:** `crates/hero_slides_lib/src/generator.rs` **Dependencies:** Step 2 Add `extract_instructions_from_image()` and `extract_instructions_from_pdf()` mirroring the existing theme extraction functions, using the new prompt key `system_extract_instructions_from_image.md`. #### Step 4: Add the CLI subcommand **File:** `crates/hero_slides/src/main.rs` + relevant rhai module **Dependencies:** Step 3 Add `hero_slides instructions extract <unc> --from <file>` CLI command mirroring `theme extract`, calling the new generator functions and writing result to `instructions.md`. #### Step 5: Add backend job submission and handler **File:** `crates/hero_slides_server/src/generate_job.rs` **Dependencies:** Step 4 Add `submit_extract_instructions_job()` and `handle_bg_extract_instructions_async()` (plus status/logs handlers) mirroring the theme extraction job pattern. Add `JobKind::ExtractInstructions` variant. #### Step 6: Route the new RPC method **File:** `crates/hero_slides_server/src/rpc.rs` **Dependencies:** Step 5 Route `bg.extractInstructionsAsync`, `bg.extractInstructionsJobStatus`, `bg.extractInstructionsJobLogs` to the new handlers. #### Step 7: Add clipboard paste support + `bgExtractInstructionsFromFile` to `dashboard.js` **File:** `crates/hero_slides_admin/static/js/dashboard.js` **Dependencies:** Steps 5–6 Add `bgExtractInstructionsFromFile()` calling `pollAiJob('bg.extractInstructionsAsync', ...)`, add a global `paste` event listener that detects image clipboard data and routes to instructions or theme extraction based on which editor is active, and register the new template key in `TEMPLATE_TO_PROMPT_KEY`. #### Step 8: Add UI button to `index.html` **File:** `crates/hero_slides_admin/templates/index.html` **Dependencies:** Step 7 Add "Extract from image / PDF" button in the Instructions editor section and a paste hint (Ctrl+V). --- ### Acceptance Criteria - [ ] Pasting an image (Ctrl+V) while the Instructions editor is open triggers instructions extraction with spinner feedback - [ ] Pasting an image while the Theme editor is open triggers theme extraction (existing feature now clipboard-accessible) - [ ] "Extract from image / PDF" button exists in the Instructions editor section - [ ] File picker accepts PNG, JPEG, WebP, GIF, PDF - [ ] Extraction runs as a background job visible in the jobs panel - [ ] On completion, instructions textarea is populated with AI-generated content (Audience, Tone, Key Messages, Slide Outline, Goals) - [ ] New prompt template appears in Templates tab and is user-editable/resettable - [ ] Toast notifications for: job started, success, failure - [ ] No regression to existing theme extraction flow --- ### Notes - The `pollAiJob` pattern already provides job tracking; the new method just needs the same `register_with_manager_inputs` call with a temp output file in `generate_job.rs`. - `resolve_extract_theme_source` is reusable for the instructions extraction since the input shape (base64 inline or folder/file) is identical. - Adding `ExtractInstructions` to `JobKind` requires updating both `as_str()` and `default_timeout()` in `jobs/types.rs`.
casper-stevens commented 2026-05-15 08:22:50 +00:00
Member
Copy link

Implementation Spec: Issue #67 — PDF/Image to Instructions

Objective

Extend the "Extract from image / PDF" feature so it targets the instructions.md file (not just theme.md), adds clipboard paste as an input method, wraps the operation in a trackable background job (via the existing pollAiJob / JobManager infrastructure), improves the AI prompt for instructions extraction, and exposes system_extract_instructions_from_image.md as a user-editable prompt template.

Current State Summary

  • bgExtractThemeFromFile() in dashboard.js opens a file picker, reads base64, calls bg.extractThemeAsync via pollAiJob. This saves to theme.md.
  • bg.extractThemeAsync handler in generate_job.rs calls resolve_extract_theme_source() (accepts {data, mime_type} for inline base64 OR {folder, file} for on-disk files), then submits a CLI job via submit_extract_theme_job().
  • The CLI job runs hero_slides theme extract <unc> --from <file>, which calls extract_theme_from_image() / extract_theme_from_pdf() in generator.rs, backed by system_extract_theme.md prompt.
  • The system_extract_theme.md prompt is in ALL_TEMPLATES and editable via the Templates tab.
  • pollAiJob polls agent.aiJobResult at 800ms intervals; on done it returns status.result containing { content }.
  • There is no equivalent flow for instructions.md from an image/PDF. instruct_instructions modifies existing instructions but there is no "extract instructions from visual reference" path.
  • No clipboard paste handling exists anywhere in the codebase.

Requirements

  • Allow the user to paste an image from the clipboard (Ctrl+V / Cmd+V) as the input for extraction, in addition to the existing file-picker path
  • Add an "Extract instructions from image/PDF" action that sends the image/PDF to the AI and writes the result to instructions.md
  • The extraction must run as a background job tracked by pollAiJob / JobManager — same pattern as bg.extractThemeAsync
  • New AI prompt system_extract_instructions_from_image.md must produce well-structured instructions.md content (audience, tone, structure, key messages, slide outline, goals) from slide images
  • The new prompt template must be user-editable via the Templates tab (added to ALL_TEMPLATES in prompts.rs and TEMPLATE_TO_PROMPT_KEY in dashboard.js)
  • A "Extract instructions from image/PDF" button must appear in the Instructions editor section of the dashboard
  • Clipboard paste (Ctrl+V with an image) should work from both the Instructions and Theme editor contexts

Files to Modify or Create

File Action Description
crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md Create New AI prompt for extracting instructions.md from a visual reference
crates/hero_slides_lib/src/prompts.rs Modify Add const + entry in ALL_TEMPLATES
crates/hero_slides_lib/src/generator.rs Modify Add extract_instructions_from_image() and extract_instructions_from_pdf()
crates/hero_slides_server/src/generate_job.rs Modify Add job submission + handler for bg.extractInstructionsAsync
crates/hero_slides_server/src/rpc.rs Modify Route bg.extractInstructionsAsync to the new handler
crates/hero_slides_admin/static/js/dashboard.js Modify Add bgExtractInstructionsFromFile(), clipboard paste handler, template key
crates/hero_slides_admin/templates/index.html Modify Add "Extract instructions from image/PDF" button + paste hint

Implementation Plan

Step 1: Create the AI prompt template

File: crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md
Dependencies: none

Create the prompt instructing the AI to analyze slide images/PDFs and produce a structured instructions.md covering: audience, tone, key messages, slide structure/outline, and goals. Output must begin with # Deck Instructions and use ## subheadings. No explanations, no code fences, no preamble — only the instructions.md content.

Step 2: Register the template in prompts.rs

File: crates/hero_slides_lib/src/prompts.rs
Dependencies: Step 1

Add pub const DEFAULT_SYSTEM_EXTRACT_INSTRUCTIONS_FROM_IMAGE constant and add it to ALL_TEMPLATES.

Step 3: Add extraction functions to generator.rs

File: crates/hero_slides_lib/src/generator.rs
Dependencies: Step 2

Add extract_instructions_from_image() and extract_instructions_from_pdf() mirroring the existing theme extraction functions, using the new prompt key system_extract_instructions_from_image.md.

Step 4: Add the CLI subcommand

File: crates/hero_slides/src/main.rs + relevant rhai module
Dependencies: Step 3

Add hero_slides instructions extract <unc> --from <file> CLI command mirroring theme extract, calling the new generator functions and writing result to instructions.md.

Step 5: Add backend job submission and handler

File: crates/hero_slides_server/src/generate_job.rs
Dependencies: Step 4

Add submit_extract_instructions_job() and handle_bg_extract_instructions_async() (plus status/logs handlers) mirroring the theme extraction job pattern. Add JobKind::ExtractInstructions variant.

Step 6: Route the new RPC method

File: crates/hero_slides_server/src/rpc.rs
Dependencies: Step 5

Route bg.extractInstructionsAsync, bg.extractInstructionsJobStatus, bg.extractInstructionsJobLogs to the new handlers.

Step 7: Add clipboard paste support + bgExtractInstructionsFromFile to dashboard.js

File: crates/hero_slides_admin/static/js/dashboard.js
Dependencies: Steps 5–6

Add bgExtractInstructionsFromFile() calling pollAiJob('bg.extractInstructionsAsync', ...), add a global paste event listener that detects image clipboard data and routes to instructions or theme extraction based on which editor is active, and register the new template key in TEMPLATE_TO_PROMPT_KEY.

Step 8: Add UI button to index.html

File: crates/hero_slides_admin/templates/index.html
Dependencies: Step 7

Add "Extract from image / PDF" button in the Instructions editor section and a paste hint (Ctrl+V).

Acceptance Criteria

  • Pasting an image (Ctrl+V) while the Instructions editor is open triggers instructions extraction with spinner feedback
  • Pasting an image while the Theme editor is open triggers theme extraction (existing feature now clipboard-accessible)
  • "Extract from image / PDF" button exists in the Instructions editor section
  • File picker accepts PNG, JPEG, WebP, GIF, PDF
  • Extraction runs as a background job visible in the jobs panel
  • On completion, instructions textarea is populated with AI-generated content (Audience, Tone, Key Messages, Slide Outline, Goals)
  • New prompt template appears in Templates tab and is user-editable/resettable
  • Toast notifications for: job started, success, failure
  • No regression to existing theme extraction flow

Notes

  • The pollAiJob pattern already provides job tracking; the new method just needs the same register_with_manager_inputs call with a temp output file in generate_job.rs.
  • resolve_extract_theme_source is reusable for the instructions extraction since the input shape (base64 inline or folder/file) is identical.
  • Adding ExtractInstructions to JobKind requires updating both as_str() and default_timeout() in jobs/types.rs.
## Implementation Spec: Issue #67 — PDF/Image to Instructions ### Objective Extend the "Extract from image / PDF" feature so it targets the **instructions.md** file (not just theme.md), adds clipboard paste as an input method, wraps the operation in a trackable background job (via the existing `pollAiJob` / `JobManager` infrastructure), improves the AI prompt for instructions extraction, and exposes `system_extract_instructions_from_image.md` as a user-editable prompt template. --- ### Current State Summary - `bgExtractThemeFromFile()` in `dashboard.js` opens a file picker, reads base64, calls `bg.extractThemeAsync` via `pollAiJob`. This saves to `theme.md`. - `bg.extractThemeAsync` handler in `generate_job.rs` calls `resolve_extract_theme_source()` (accepts `{data, mime_type}` for inline base64 OR `{folder, file}` for on-disk files), then submits a CLI job via `submit_extract_theme_job()`. - The CLI job runs `hero_slides theme extract <unc> --from <file>`, which calls `extract_theme_from_image()` / `extract_theme_from_pdf()` in `generator.rs`, backed by `system_extract_theme.md` prompt. - The `system_extract_theme.md` prompt is in `ALL_TEMPLATES` and editable via the Templates tab. - `pollAiJob` polls `agent.aiJobResult` at 800ms intervals; on `done` it returns `status.result` containing `{ content }`. - There is no equivalent flow for **instructions.md** from an image/PDF. `instruct_instructions` modifies existing instructions but there is no "extract instructions from visual reference" path. - No clipboard paste handling exists anywhere in the codebase. --- ### Requirements - Allow the user to paste an image from the clipboard (Ctrl+V / Cmd+V) as the input for extraction, in addition to the existing file-picker path - Add an "Extract instructions from image/PDF" action that sends the image/PDF to the AI and writes the result to `instructions.md` - The extraction must run as a background job tracked by `pollAiJob` / `JobManager` — same pattern as `bg.extractThemeAsync` - New AI prompt `system_extract_instructions_from_image.md` must produce well-structured `instructions.md` content (audience, tone, structure, key messages, slide outline, goals) from slide images - The new prompt template must be user-editable via the Templates tab (added to `ALL_TEMPLATES` in `prompts.rs` and `TEMPLATE_TO_PROMPT_KEY` in `dashboard.js`) - A "Extract instructions from image/PDF" button must appear in the Instructions editor section of the dashboard - Clipboard paste (Ctrl+V with an image) should work from both the Instructions and Theme editor contexts --- ### Files to Modify or Create | File | Action | Description | |---|---|---| | `crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md` | Create | New AI prompt for extracting instructions.md from a visual reference | | `crates/hero_slides_lib/src/prompts.rs` | Modify | Add const + entry in ALL_TEMPLATES | | `crates/hero_slides_lib/src/generator.rs` | Modify | Add `extract_instructions_from_image()` and `extract_instructions_from_pdf()` | | `crates/hero_slides_server/src/generate_job.rs` | Modify | Add job submission + handler for `bg.extractInstructionsAsync` | | `crates/hero_slides_server/src/rpc.rs` | Modify | Route `bg.extractInstructionsAsync` to the new handler | | `crates/hero_slides_admin/static/js/dashboard.js` | Modify | Add `bgExtractInstructionsFromFile()`, clipboard paste handler, template key | | `crates/hero_slides_admin/templates/index.html` | Modify | Add "Extract instructions from image/PDF" button + paste hint | --- ### Implementation Plan #### Step 1: Create the AI prompt template **File:** `crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md` **Dependencies:** none Create the prompt instructing the AI to analyze slide images/PDFs and produce a structured `instructions.md` covering: audience, tone, key messages, slide structure/outline, and goals. Output must begin with `# Deck Instructions` and use `##` subheadings. No explanations, no code fences, no preamble — only the instructions.md content. #### Step 2: Register the template in `prompts.rs` **File:** `crates/hero_slides_lib/src/prompts.rs` **Dependencies:** Step 1 Add `pub const DEFAULT_SYSTEM_EXTRACT_INSTRUCTIONS_FROM_IMAGE` constant and add it to `ALL_TEMPLATES`. #### Step 3: Add extraction functions to `generator.rs` **File:** `crates/hero_slides_lib/src/generator.rs` **Dependencies:** Step 2 Add `extract_instructions_from_image()` and `extract_instructions_from_pdf()` mirroring the existing theme extraction functions, using the new prompt key `system_extract_instructions_from_image.md`. #### Step 4: Add the CLI subcommand **File:** `crates/hero_slides/src/main.rs` + relevant rhai module **Dependencies:** Step 3 Add `hero_slides instructions extract <unc> --from <file>` CLI command mirroring `theme extract`, calling the new generator functions and writing result to `instructions.md`. #### Step 5: Add backend job submission and handler **File:** `crates/hero_slides_server/src/generate_job.rs` **Dependencies:** Step 4 Add `submit_extract_instructions_job()` and `handle_bg_extract_instructions_async()` (plus status/logs handlers) mirroring the theme extraction job pattern. Add `JobKind::ExtractInstructions` variant. #### Step 6: Route the new RPC method **File:** `crates/hero_slides_server/src/rpc.rs` **Dependencies:** Step 5 Route `bg.extractInstructionsAsync`, `bg.extractInstructionsJobStatus`, `bg.extractInstructionsJobLogs` to the new handlers. #### Step 7: Add clipboard paste support + `bgExtractInstructionsFromFile` to `dashboard.js` **File:** `crates/hero_slides_admin/static/js/dashboard.js` **Dependencies:** Steps 5–6 Add `bgExtractInstructionsFromFile()` calling `pollAiJob('bg.extractInstructionsAsync', ...)`, add a global `paste` event listener that detects image clipboard data and routes to instructions or theme extraction based on which editor is active, and register the new template key in `TEMPLATE_TO_PROMPT_KEY`. #### Step 8: Add UI button to `index.html` **File:** `crates/hero_slides_admin/templates/index.html` **Dependencies:** Step 7 Add "Extract from image / PDF" button in the Instructions editor section and a paste hint (Ctrl+V). --- ### Acceptance Criteria - [ ] Pasting an image (Ctrl+V) while the Instructions editor is open triggers instructions extraction with spinner feedback - [ ] Pasting an image while the Theme editor is open triggers theme extraction (existing feature now clipboard-accessible) - [ ] "Extract from image / PDF" button exists in the Instructions editor section - [ ] File picker accepts PNG, JPEG, WebP, GIF, PDF - [ ] Extraction runs as a background job visible in the jobs panel - [ ] On completion, instructions textarea is populated with AI-generated content (Audience, Tone, Key Messages, Slide Outline, Goals) - [ ] New prompt template appears in Templates tab and is user-editable/resettable - [ ] Toast notifications for: job started, success, failure - [ ] No regression to existing theme extraction flow --- ### Notes - The `pollAiJob` pattern already provides job tracking; the new method just needs the same `register_with_manager_inputs` call with a temp output file in `generate_job.rs`. - `resolve_extract_theme_source` is reusable for the instructions extraction since the input shape (base64 inline or folder/file) is identical. - Adding `ExtractInstructions` to `JobKind` requires updating both `as_str()` and `default_timeout()` in `jobs/types.rs`.
casper-stevens commented 2026-05-15 08:22:53 +00:00
Member
Copy link

Implementation Spec: Issue #67 — PDF/Image to Instructions

Objective

Extend the "Extract from image / PDF" feature so it targets the instructions.md file (not just theme.md), adds clipboard paste as an input method, wraps the operation in a trackable background job (via the existing pollAiJob / JobManager infrastructure), improves the AI prompt for instructions extraction, and exposes system_extract_instructions_from_image.md as a user-editable prompt template.

Current State Summary

  • bgExtractThemeFromFile() in dashboard.js opens a file picker, reads base64, calls bg.extractThemeAsync via pollAiJob. This saves to theme.md.
  • bg.extractThemeAsync handler in generate_job.rs calls resolve_extract_theme_source() (accepts {data, mime_type} for inline base64 OR {folder, file} for on-disk files), then submits a CLI job via submit_extract_theme_job().
  • The CLI job runs hero_slides theme extract <unc> --from <file>, which calls extract_theme_from_image() / extract_theme_from_pdf() in generator.rs, backed by system_extract_theme.md prompt.
  • The system_extract_theme.md prompt is in ALL_TEMPLATES and editable via the Templates tab.
  • pollAiJob polls agent.aiJobResult at 800ms intervals; on done it returns status.result containing { content }.
  • There is no equivalent flow for instructions.md from an image/PDF. instruct_instructions modifies existing instructions but there is no "extract instructions from visual reference" path.
  • No clipboard paste handling exists anywhere in the codebase.

Requirements

  • Allow the user to paste an image from the clipboard (Ctrl+V / Cmd+V) as the input for extraction, in addition to the existing file-picker path
  • Add an "Extract instructions from image/PDF" action that sends the image/PDF to the AI and writes the result to instructions.md
  • The extraction must run as a background job tracked by pollAiJob / JobManager — same pattern as bg.extractThemeAsync
  • New AI prompt system_extract_instructions_from_image.md must produce well-structured instructions.md content (audience, tone, structure, key messages, slide outline, goals) from slide images
  • The new prompt template must be user-editable via the Templates tab (added to ALL_TEMPLATES in prompts.rs and TEMPLATE_TO_PROMPT_KEY in dashboard.js)
  • A "Extract instructions from image/PDF" button must appear in the Instructions editor section of the dashboard
  • Clipboard paste (Ctrl+V with an image) should work from both the Instructions and Theme editor contexts

Files to Modify or Create

File Action Description
crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md Create New AI prompt for extracting instructions.md from a visual reference
crates/hero_slides_lib/src/prompts.rs Modify Add const + entry in ALL_TEMPLATES
crates/hero_slides_lib/src/generator.rs Modify Add extract_instructions_from_image() and extract_instructions_from_pdf()
crates/hero_slides_server/src/generate_job.rs Modify Add job submission + handler for bg.extractInstructionsAsync
crates/hero_slides_server/src/rpc.rs Modify Route bg.extractInstructionsAsync to the new handler
crates/hero_slides_admin/static/js/dashboard.js Modify Add bgExtractInstructionsFromFile(), clipboard paste handler, template key
crates/hero_slides_admin/templates/index.html Modify Add "Extract instructions from image/PDF" button + paste hint

Implementation Plan

Step 1: Create the AI prompt template

File: crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md
Dependencies: none

Create the prompt instructing the AI to analyze slide images/PDFs and produce a structured instructions.md covering: audience, tone, key messages, slide structure/outline, and goals. Output must begin with # Deck Instructions and use ## subheadings. No explanations, no code fences, no preamble — only the instructions.md content.

Step 2: Register the template in prompts.rs

File: crates/hero_slides_lib/src/prompts.rs
Dependencies: Step 1

Add pub const DEFAULT_SYSTEM_EXTRACT_INSTRUCTIONS_FROM_IMAGE constant and add it to ALL_TEMPLATES.

Step 3: Add extraction functions to generator.rs

File: crates/hero_slides_lib/src/generator.rs
Dependencies: Step 2

Add extract_instructions_from_image() and extract_instructions_from_pdf() mirroring the existing theme extraction functions, using the new prompt key system_extract_instructions_from_image.md.

Step 4: Add the CLI subcommand

File: crates/hero_slides/src/main.rs + relevant rhai module
Dependencies: Step 3

Add hero_slides instructions extract <unc> --from <file> CLI command mirroring theme extract, calling the new generator functions and writing result to instructions.md.

Step 5: Add backend job submission and handler

File: crates/hero_slides_server/src/generate_job.rs
Dependencies: Step 4

Add submit_extract_instructions_job() and handle_bg_extract_instructions_async() (plus status/logs handlers) mirroring the theme extraction job pattern. Add JobKind::ExtractInstructions variant.

Step 6: Route the new RPC method

File: crates/hero_slides_server/src/rpc.rs
Dependencies: Step 5

Route bg.extractInstructionsAsync, bg.extractInstructionsJobStatus, bg.extractInstructionsJobLogs to the new handlers.

Step 7: Add clipboard paste support + bgExtractInstructionsFromFile to dashboard.js

File: crates/hero_slides_admin/static/js/dashboard.js
Dependencies: Steps 5–6

Add bgExtractInstructionsFromFile() calling pollAiJob('bg.extractInstructionsAsync', ...), add a global paste event listener that detects image clipboard data and routes to instructions or theme extraction based on which editor is active, and register the new template key in TEMPLATE_TO_PROMPT_KEY.

Step 8: Add UI button to index.html

File: crates/hero_slides_admin/templates/index.html
Dependencies: Step 7

Add "Extract from image / PDF" button in the Instructions editor section and a paste hint (Ctrl+V).

Acceptance Criteria

  • Pasting an image (Ctrl+V) while the Instructions editor is open triggers instructions extraction with spinner feedback
  • Pasting an image while the Theme editor is open triggers theme extraction (existing feature now clipboard-accessible)
  • "Extract from image / PDF" button exists in the Instructions editor section
  • File picker accepts PNG, JPEG, WebP, GIF, PDF
  • Extraction runs as a background job visible in the jobs panel
  • On completion, instructions textarea is populated with AI-generated content (Audience, Tone, Key Messages, Slide Outline, Goals)
  • New prompt template appears in Templates tab and is user-editable/resettable
  • Toast notifications for: job started, success, failure
  • No regression to existing theme extraction flow

Notes

  • The pollAiJob pattern already provides job tracking; the new method just needs the same register_with_manager_inputs call with a temp output file in generate_job.rs.
  • resolve_extract_theme_source is reusable for the instructions extraction since the input shape (base64 inline or folder/file) is identical.
  • Adding ExtractInstructions to JobKind requires updating both as_str() and default_timeout() in jobs/types.rs.
## Implementation Spec: Issue #67 — PDF/Image to Instructions ### Objective Extend the "Extract from image / PDF" feature so it targets the **instructions.md** file (not just theme.md), adds clipboard paste as an input method, wraps the operation in a trackable background job (via the existing `pollAiJob` / `JobManager` infrastructure), improves the AI prompt for instructions extraction, and exposes `system_extract_instructions_from_image.md` as a user-editable prompt template. --- ### Current State Summary - `bgExtractThemeFromFile()` in `dashboard.js` opens a file picker, reads base64, calls `bg.extractThemeAsync` via `pollAiJob`. This saves to `theme.md`. - `bg.extractThemeAsync` handler in `generate_job.rs` calls `resolve_extract_theme_source()` (accepts `{data, mime_type}` for inline base64 OR `{folder, file}` for on-disk files), then submits a CLI job via `submit_extract_theme_job()`. - The CLI job runs `hero_slides theme extract <unc> --from <file>`, which calls `extract_theme_from_image()` / `extract_theme_from_pdf()` in `generator.rs`, backed by `system_extract_theme.md` prompt. - The `system_extract_theme.md` prompt is in `ALL_TEMPLATES` and editable via the Templates tab. - `pollAiJob` polls `agent.aiJobResult` at 800ms intervals; on `done` it returns `status.result` containing `{ content }`. - There is no equivalent flow for **instructions.md** from an image/PDF. `instruct_instructions` modifies existing instructions but there is no "extract instructions from visual reference" path. - No clipboard paste handling exists anywhere in the codebase. --- ### Requirements - Allow the user to paste an image from the clipboard (Ctrl+V / Cmd+V) as the input for extraction, in addition to the existing file-picker path - Add an "Extract instructions from image/PDF" action that sends the image/PDF to the AI and writes the result to `instructions.md` - The extraction must run as a background job tracked by `pollAiJob` / `JobManager` — same pattern as `bg.extractThemeAsync` - New AI prompt `system_extract_instructions_from_image.md` must produce well-structured `instructions.md` content (audience, tone, structure, key messages, slide outline, goals) from slide images - The new prompt template must be user-editable via the Templates tab (added to `ALL_TEMPLATES` in `prompts.rs` and `TEMPLATE_TO_PROMPT_KEY` in `dashboard.js`) - A "Extract instructions from image/PDF" button must appear in the Instructions editor section of the dashboard - Clipboard paste (Ctrl+V with an image) should work from both the Instructions and Theme editor contexts --- ### Files to Modify or Create | File | Action | Description | |---|---|---| | `crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md` | Create | New AI prompt for extracting instructions.md from a visual reference | | `crates/hero_slides_lib/src/prompts.rs` | Modify | Add const + entry in ALL_TEMPLATES | | `crates/hero_slides_lib/src/generator.rs` | Modify | Add `extract_instructions_from_image()` and `extract_instructions_from_pdf()` | | `crates/hero_slides_server/src/generate_job.rs` | Modify | Add job submission + handler for `bg.extractInstructionsAsync` | | `crates/hero_slides_server/src/rpc.rs` | Modify | Route `bg.extractInstructionsAsync` to the new handler | | `crates/hero_slides_admin/static/js/dashboard.js` | Modify | Add `bgExtractInstructionsFromFile()`, clipboard paste handler, template key | | `crates/hero_slides_admin/templates/index.html` | Modify | Add "Extract instructions from image/PDF" button + paste hint | --- ### Implementation Plan #### Step 1: Create the AI prompt template **File:** `crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md` **Dependencies:** none Create the prompt instructing the AI to analyze slide images/PDFs and produce a structured `instructions.md` covering: audience, tone, key messages, slide structure/outline, and goals. Output must begin with `# Deck Instructions` and use `##` subheadings. No explanations, no code fences, no preamble — only the instructions.md content. #### Step 2: Register the template in `prompts.rs` **File:** `crates/hero_slides_lib/src/prompts.rs` **Dependencies:** Step 1 Add `pub const DEFAULT_SYSTEM_EXTRACT_INSTRUCTIONS_FROM_IMAGE` constant and add it to `ALL_TEMPLATES`. #### Step 3: Add extraction functions to `generator.rs` **File:** `crates/hero_slides_lib/src/generator.rs` **Dependencies:** Step 2 Add `extract_instructions_from_image()` and `extract_instructions_from_pdf()` mirroring the existing theme extraction functions, using the new prompt key `system_extract_instructions_from_image.md`. #### Step 4: Add the CLI subcommand **File:** `crates/hero_slides/src/main.rs` + relevant rhai module **Dependencies:** Step 3 Add `hero_slides instructions extract <unc> --from <file>` CLI command mirroring `theme extract`, calling the new generator functions and writing result to `instructions.md`. #### Step 5: Add backend job submission and handler **File:** `crates/hero_slides_server/src/generate_job.rs` **Dependencies:** Step 4 Add `submit_extract_instructions_job()` and `handle_bg_extract_instructions_async()` (plus status/logs handlers) mirroring the theme extraction job pattern. Add `JobKind::ExtractInstructions` variant. #### Step 6: Route the new RPC method **File:** `crates/hero_slides_server/src/rpc.rs` **Dependencies:** Step 5 Route `bg.extractInstructionsAsync`, `bg.extractInstructionsJobStatus`, `bg.extractInstructionsJobLogs` to the new handlers. #### Step 7: Add clipboard paste support + `bgExtractInstructionsFromFile` to `dashboard.js` **File:** `crates/hero_slides_admin/static/js/dashboard.js` **Dependencies:** Steps 5–6 Add `bgExtractInstructionsFromFile()` calling `pollAiJob('bg.extractInstructionsAsync', ...)`, add a global `paste` event listener that detects image clipboard data and routes to instructions or theme extraction based on which editor is active, and register the new template key in `TEMPLATE_TO_PROMPT_KEY`. #### Step 8: Add UI button to `index.html` **File:** `crates/hero_slides_admin/templates/index.html` **Dependencies:** Step 7 Add "Extract from image / PDF" button in the Instructions editor section and a paste hint (Ctrl+V). --- ### Acceptance Criteria - [ ] Pasting an image (Ctrl+V) while the Instructions editor is open triggers instructions extraction with spinner feedback - [ ] Pasting an image while the Theme editor is open triggers theme extraction (existing feature now clipboard-accessible) - [ ] "Extract from image / PDF" button exists in the Instructions editor section - [ ] File picker accepts PNG, JPEG, WebP, GIF, PDF - [ ] Extraction runs as a background job visible in the jobs panel - [ ] On completion, instructions textarea is populated with AI-generated content (Audience, Tone, Key Messages, Slide Outline, Goals) - [ ] New prompt template appears in Templates tab and is user-editable/resettable - [ ] Toast notifications for: job started, success, failure - [ ] No regression to existing theme extraction flow --- ### Notes - The `pollAiJob` pattern already provides job tracking; the new method just needs the same `register_with_manager_inputs` call with a temp output file in `generate_job.rs`. - `resolve_extract_theme_source` is reusable for the instructions extraction since the input shape (base64 inline or folder/file) is identical. - Adding `ExtractInstructions` to `JobKind` requires updating both `as_str()` and `default_timeout()` in `jobs/types.rs`.
casper-stevens commented 2026-05-15 08:23:00 +00:00
Member
Copy link

Implementation Spec: Issue #67 — PDF/Image to Instructions

Objective

Extend the "Extract from image / PDF" feature so it targets the instructions.md file (not just theme.md), adds clipboard paste as an input method, wraps the operation in a trackable background job (via the existing pollAiJob / JobManager infrastructure), improves the AI prompt for instructions extraction, and exposes system_extract_instructions_from_image.md as a user-editable prompt template.

Current State Summary

  • bgExtractThemeFromFile() in dashboard.js opens a file picker, reads base64, calls bg.extractThemeAsync via pollAiJob. This saves to theme.md.
  • bg.extractThemeAsync handler in generate_job.rs calls resolve_extract_theme_source() (accepts {data, mime_type} for inline base64 OR {folder, file} for on-disk files), then submits a CLI job via submit_extract_theme_job().
  • The CLI job runs hero_slides theme extract <unc> --from <file>, which calls extract_theme_from_image() / extract_theme_from_pdf() in generator.rs, backed by system_extract_theme.md prompt.
  • The system_extract_theme.md prompt is in ALL_TEMPLATES and editable via the Templates tab.
  • pollAiJob polls agent.aiJobResult at 800ms intervals; on done it returns status.result containing { content }.
  • There is no equivalent flow for instructions.md from an image/PDF. instruct_instructions modifies existing instructions but there is no "extract instructions from visual reference" path.
  • No clipboard paste handling exists anywhere in the codebase.

Requirements

  • Allow the user to paste an image from the clipboard (Ctrl+V / Cmd+V) as the input for extraction, in addition to the existing file-picker path
  • Add an "Extract instructions from image/PDF" action that sends the image/PDF to the AI and writes the result to instructions.md
  • The extraction must run as a background job tracked by pollAiJob / JobManager — same pattern as bg.extractThemeAsync
  • New AI prompt system_extract_instructions_from_image.md must produce well-structured instructions.md content (audience, tone, structure, key messages, slide outline, goals) from slide images
  • The new prompt template must be user-editable via the Templates tab (added to ALL_TEMPLATES in prompts.rs and TEMPLATE_TO_PROMPT_KEY in dashboard.js)
  • A "Extract instructions from image/PDF" button must appear in the Instructions editor section of the dashboard
  • Clipboard paste (Ctrl+V with an image) should work from both the Instructions and Theme editor contexts

Files to Modify or Create

File Action Description
crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md Create New AI prompt for extracting instructions.md from a visual reference
crates/hero_slides_lib/src/prompts.rs Modify Add const + entry in ALL_TEMPLATES
crates/hero_slides_lib/src/generator.rs Modify Add extract_instructions_from_image() and extract_instructions_from_pdf()
crates/hero_slides_server/src/generate_job.rs Modify Add job submission + handler for bg.extractInstructionsAsync
crates/hero_slides_server/src/rpc.rs Modify Route bg.extractInstructionsAsync to the new handler
crates/hero_slides_admin/static/js/dashboard.js Modify Add bgExtractInstructionsFromFile(), clipboard paste handler, template key
crates/hero_slides_admin/templates/index.html Modify Add "Extract instructions from image/PDF" button + paste hint

Implementation Plan

Step 1: Create the AI prompt template

File: crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md
Dependencies: none

Create the prompt instructing the AI to analyze slide images/PDFs and produce a structured instructions.md covering: audience, tone, key messages, slide structure/outline, and goals. Output must begin with # Deck Instructions and use ## subheadings. No explanations, no code fences, no preamble — only the instructions.md content.

Step 2: Register the template in prompts.rs

File: crates/hero_slides_lib/src/prompts.rs
Dependencies: Step 1

Add pub const DEFAULT_SYSTEM_EXTRACT_INSTRUCTIONS_FROM_IMAGE constant and add it to ALL_TEMPLATES.

Step 3: Add extraction functions to generator.rs

File: crates/hero_slides_lib/src/generator.rs
Dependencies: Step 2

Add extract_instructions_from_image() and extract_instructions_from_pdf() mirroring the existing theme extraction functions, using the new prompt key system_extract_instructions_from_image.md.

Step 4: Add the CLI subcommand

File: crates/hero_slides/src/main.rs + relevant rhai module
Dependencies: Step 3

Add hero_slides instructions extract <unc> --from <file> CLI command mirroring theme extract, calling the new generator functions and writing result to instructions.md.

Step 5: Add backend job submission and handler

File: crates/hero_slides_server/src/generate_job.rs
Dependencies: Step 4

Add submit_extract_instructions_job() and handle_bg_extract_instructions_async() (plus status/logs handlers) mirroring the theme extraction job pattern. Add JobKind::ExtractInstructions variant.

Step 6: Route the new RPC method

File: crates/hero_slides_server/src/rpc.rs
Dependencies: Step 5

Route bg.extractInstructionsAsync, bg.extractInstructionsJobStatus, bg.extractInstructionsJobLogs to the new handlers.

Step 7: Add clipboard paste support + bgExtractInstructionsFromFile to dashboard.js

File: crates/hero_slides_admin/static/js/dashboard.js
Dependencies: Steps 5–6

Add bgExtractInstructionsFromFile() calling pollAiJob('bg.extractInstructionsAsync', ...), add a global paste event listener that detects image clipboard data and routes to instructions or theme extraction based on which editor is active, and register the new template key in TEMPLATE_TO_PROMPT_KEY.

Step 8: Add UI button to index.html

File: crates/hero_slides_admin/templates/index.html
Dependencies: Step 7

Add "Extract from image / PDF" button in the Instructions editor section and a paste hint (Ctrl+V).

Acceptance Criteria

  • Pasting an image (Ctrl+V) while the Instructions editor is open triggers instructions extraction with spinner feedback
  • Pasting an image while the Theme editor is open triggers theme extraction (existing feature now clipboard-accessible)
  • "Extract from image / PDF" button exists in the Instructions editor section
  • File picker accepts PNG, JPEG, WebP, GIF, PDF
  • Extraction runs as a background job visible in the jobs panel
  • On completion, instructions textarea is populated with AI-generated content (Audience, Tone, Key Messages, Slide Outline, Goals)
  • New prompt template appears in Templates tab and is user-editable/resettable
  • Toast notifications for: job started, success, failure
  • No regression to existing theme extraction flow

Notes

  • The pollAiJob pattern already provides job tracking; the new method just needs the same register_with_manager_inputs call with a temp output file in generate_job.rs.
  • resolve_extract_theme_source is reusable for the instructions extraction since the input shape (base64 inline or folder/file) is identical.
  • Adding ExtractInstructions to JobKind requires updating both as_str() and default_timeout() in jobs/types.rs.
## Implementation Spec: Issue #67 — PDF/Image to Instructions ### Objective Extend the "Extract from image / PDF" feature so it targets the **instructions.md** file (not just theme.md), adds clipboard paste as an input method, wraps the operation in a trackable background job (via the existing `pollAiJob` / `JobManager` infrastructure), improves the AI prompt for instructions extraction, and exposes `system_extract_instructions_from_image.md` as a user-editable prompt template. --- ### Current State Summary - `bgExtractThemeFromFile()` in `dashboard.js` opens a file picker, reads base64, calls `bg.extractThemeAsync` via `pollAiJob`. This saves to `theme.md`. - `bg.extractThemeAsync` handler in `generate_job.rs` calls `resolve_extract_theme_source()` (accepts `{data, mime_type}` for inline base64 OR `{folder, file}` for on-disk files), then submits a CLI job via `submit_extract_theme_job()`. - The CLI job runs `hero_slides theme extract <unc> --from <file>`, which calls `extract_theme_from_image()` / `extract_theme_from_pdf()` in `generator.rs`, backed by `system_extract_theme.md` prompt. - The `system_extract_theme.md` prompt is in `ALL_TEMPLATES` and editable via the Templates tab. - `pollAiJob` polls `agent.aiJobResult` at 800ms intervals; on `done` it returns `status.result` containing `{ content }`. - There is no equivalent flow for **instructions.md** from an image/PDF. `instruct_instructions` modifies existing instructions but there is no "extract instructions from visual reference" path. - No clipboard paste handling exists anywhere in the codebase. --- ### Requirements - Allow the user to paste an image from the clipboard (Ctrl+V / Cmd+V) as the input for extraction, in addition to the existing file-picker path - Add an "Extract instructions from image/PDF" action that sends the image/PDF to the AI and writes the result to `instructions.md` - The extraction must run as a background job tracked by `pollAiJob` / `JobManager` — same pattern as `bg.extractThemeAsync` - New AI prompt `system_extract_instructions_from_image.md` must produce well-structured `instructions.md` content (audience, tone, structure, key messages, slide outline, goals) from slide images - The new prompt template must be user-editable via the Templates tab (added to `ALL_TEMPLATES` in `prompts.rs` and `TEMPLATE_TO_PROMPT_KEY` in `dashboard.js`) - A "Extract instructions from image/PDF" button must appear in the Instructions editor section of the dashboard - Clipboard paste (Ctrl+V with an image) should work from both the Instructions and Theme editor contexts --- ### Files to Modify or Create | File | Action | Description | |---|---|---| | `crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md` | Create | New AI prompt for extracting instructions.md from a visual reference | | `crates/hero_slides_lib/src/prompts.rs` | Modify | Add const + entry in ALL_TEMPLATES | | `crates/hero_slides_lib/src/generator.rs` | Modify | Add `extract_instructions_from_image()` and `extract_instructions_from_pdf()` | | `crates/hero_slides_server/src/generate_job.rs` | Modify | Add job submission + handler for `bg.extractInstructionsAsync` | | `crates/hero_slides_server/src/rpc.rs` | Modify | Route `bg.extractInstructionsAsync` to the new handler | | `crates/hero_slides_admin/static/js/dashboard.js` | Modify | Add `bgExtractInstructionsFromFile()`, clipboard paste handler, template key | | `crates/hero_slides_admin/templates/index.html` | Modify | Add "Extract instructions from image/PDF" button + paste hint | --- ### Implementation Plan #### Step 1: Create the AI prompt template **File:** `crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md` **Dependencies:** none Create the prompt instructing the AI to analyze slide images/PDFs and produce a structured `instructions.md` covering: audience, tone, key messages, slide structure/outline, and goals. Output must begin with `# Deck Instructions` and use `##` subheadings. No explanations, no code fences, no preamble — only the instructions.md content. #### Step 2: Register the template in `prompts.rs` **File:** `crates/hero_slides_lib/src/prompts.rs` **Dependencies:** Step 1 Add `pub const DEFAULT_SYSTEM_EXTRACT_INSTRUCTIONS_FROM_IMAGE` constant and add it to `ALL_TEMPLATES`. #### Step 3: Add extraction functions to `generator.rs` **File:** `crates/hero_slides_lib/src/generator.rs` **Dependencies:** Step 2 Add `extract_instructions_from_image()` and `extract_instructions_from_pdf()` mirroring the existing theme extraction functions, using the new prompt key `system_extract_instructions_from_image.md`. #### Step 4: Add the CLI subcommand **File:** `crates/hero_slides/src/main.rs` + relevant rhai module **Dependencies:** Step 3 Add `hero_slides instructions extract <unc> --from <file>` CLI command mirroring `theme extract`, calling the new generator functions and writing result to `instructions.md`. #### Step 5: Add backend job submission and handler **File:** `crates/hero_slides_server/src/generate_job.rs` **Dependencies:** Step 4 Add `submit_extract_instructions_job()` and `handle_bg_extract_instructions_async()` (plus status/logs handlers) mirroring the theme extraction job pattern. Add `JobKind::ExtractInstructions` variant. #### Step 6: Route the new RPC method **File:** `crates/hero_slides_server/src/rpc.rs` **Dependencies:** Step 5 Route `bg.extractInstructionsAsync`, `bg.extractInstructionsJobStatus`, `bg.extractInstructionsJobLogs` to the new handlers. #### Step 7: Add clipboard paste support + `bgExtractInstructionsFromFile` to `dashboard.js` **File:** `crates/hero_slides_admin/static/js/dashboard.js` **Dependencies:** Steps 5–6 Add `bgExtractInstructionsFromFile()` calling `pollAiJob('bg.extractInstructionsAsync', ...)`, add a global `paste` event listener that detects image clipboard data and routes to instructions or theme extraction based on which editor is active, and register the new template key in `TEMPLATE_TO_PROMPT_KEY`. #### Step 8: Add UI button to `index.html` **File:** `crates/hero_slides_admin/templates/index.html` **Dependencies:** Step 7 Add "Extract from image / PDF" button in the Instructions editor section and a paste hint (Ctrl+V). --- ### Acceptance Criteria - [ ] Pasting an image (Ctrl+V) while the Instructions editor is open triggers instructions extraction with spinner feedback - [ ] Pasting an image while the Theme editor is open triggers theme extraction (existing feature now clipboard-accessible) - [ ] "Extract from image / PDF" button exists in the Instructions editor section - [ ] File picker accepts PNG, JPEG, WebP, GIF, PDF - [ ] Extraction runs as a background job visible in the jobs panel - [ ] On completion, instructions textarea is populated with AI-generated content (Audience, Tone, Key Messages, Slide Outline, Goals) - [ ] New prompt template appears in Templates tab and is user-editable/resettable - [ ] Toast notifications for: job started, success, failure - [ ] No regression to existing theme extraction flow --- ### Notes - The `pollAiJob` pattern already provides job tracking; the new method just needs the same `register_with_manager_inputs` call with a temp output file in `generate_job.rs`. - `resolve_extract_theme_source` is reusable for the instructions extraction since the input shape (base64 inline or folder/file) is identical. - Adding `ExtractInstructions` to `JobKind` requires updating both `as_str()` and `default_timeout()` in `jobs/types.rs`.
casper-stevens commented 2026-05-15 08:23:03 +00:00
Member
Copy link

Implementation Spec: Issue #67 — PDF/Image to Instructions

Objective

Extend the "Extract from image / PDF" feature so it targets the instructions.md file (not just theme.md), adds clipboard paste as an input method, wraps the operation in a trackable background job (via the existing pollAiJob / JobManager infrastructure), improves the AI prompt for instructions extraction, and exposes system_extract_instructions_from_image.md as a user-editable prompt template.

Current State Summary

  • bgExtractThemeFromFile() in dashboard.js opens a file picker, reads base64, calls bg.extractThemeAsync via pollAiJob. This saves to theme.md.
  • bg.extractThemeAsync handler in generate_job.rs calls resolve_extract_theme_source() (accepts {data, mime_type} for inline base64 OR {folder, file} for on-disk files), then submits a CLI job via submit_extract_theme_job().
  • The CLI job runs hero_slides theme extract <unc> --from <file>, which calls extract_theme_from_image() / extract_theme_from_pdf() in generator.rs, backed by system_extract_theme.md prompt.
  • The system_extract_theme.md prompt is in ALL_TEMPLATES and editable via the Templates tab.
  • pollAiJob polls agent.aiJobResult at 800ms intervals; on done it returns status.result containing { content }.
  • There is no equivalent flow for instructions.md from an image/PDF. instruct_instructions modifies existing instructions but there is no "extract instructions from visual reference" path.
  • No clipboard paste handling exists anywhere in the codebase.

Requirements

  • Allow the user to paste an image from the clipboard (Ctrl+V / Cmd+V) as the input for extraction, in addition to the existing file-picker path
  • Add an "Extract instructions from image/PDF" action that sends the image/PDF to the AI and writes the result to instructions.md
  • The extraction must run as a background job tracked by pollAiJob / JobManager — same pattern as bg.extractThemeAsync
  • New AI prompt system_extract_instructions_from_image.md must produce well-structured instructions.md content (audience, tone, structure, key messages, slide outline, goals) from slide images
  • The new prompt template must be user-editable via the Templates tab (added to ALL_TEMPLATES in prompts.rs and TEMPLATE_TO_PROMPT_KEY in dashboard.js)
  • A "Extract instructions from image/PDF" button must appear in the Instructions editor section of the dashboard
  • Clipboard paste (Ctrl+V with an image) should work from both the Instructions and Theme editor contexts

Files to Modify or Create

File Action Description
crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md Create New AI prompt for extracting instructions.md from a visual reference
crates/hero_slides_lib/src/prompts.rs Modify Add const + entry in ALL_TEMPLATES
crates/hero_slides_lib/src/generator.rs Modify Add extract_instructions_from_image() and extract_instructions_from_pdf()
crates/hero_slides_server/src/generate_job.rs Modify Add job submission + handler for bg.extractInstructionsAsync
crates/hero_slides_server/src/rpc.rs Modify Route bg.extractInstructionsAsync to the new handler
crates/hero_slides_admin/static/js/dashboard.js Modify Add bgExtractInstructionsFromFile(), clipboard paste handler, template key
crates/hero_slides_admin/templates/index.html Modify Add "Extract instructions from image/PDF" button + paste hint

Implementation Plan

Step 1: Create the AI prompt template

File: crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md
Dependencies: none

Create the prompt instructing the AI to analyze slide images/PDFs and produce a structured instructions.md covering: audience, tone, key messages, slide structure/outline, and goals. Output must begin with # Deck Instructions and use ## subheadings. No explanations, no code fences, no preamble — only the instructions.md content.

Step 2: Register the template in prompts.rs

File: crates/hero_slides_lib/src/prompts.rs
Dependencies: Step 1

Add pub const DEFAULT_SYSTEM_EXTRACT_INSTRUCTIONS_FROM_IMAGE constant and add it to ALL_TEMPLATES.

Step 3: Add extraction functions to generator.rs

File: crates/hero_slides_lib/src/generator.rs
Dependencies: Step 2

Add extract_instructions_from_image() and extract_instructions_from_pdf() mirroring the existing theme extraction functions, using the new prompt key system_extract_instructions_from_image.md.

Step 4: Add the CLI subcommand

File: crates/hero_slides/src/main.rs + relevant rhai module
Dependencies: Step 3

Add hero_slides instructions extract <unc> --from <file> CLI command mirroring theme extract, calling the new generator functions and writing result to instructions.md.

Step 5: Add backend job submission and handler

File: crates/hero_slides_server/src/generate_job.rs
Dependencies: Step 4

Add submit_extract_instructions_job() and handle_bg_extract_instructions_async() (plus status/logs handlers) mirroring the theme extraction job pattern. Add JobKind::ExtractInstructions variant.

Step 6: Route the new RPC method

File: crates/hero_slides_server/src/rpc.rs
Dependencies: Step 5

Route bg.extractInstructionsAsync, bg.extractInstructionsJobStatus, bg.extractInstructionsJobLogs to the new handlers.

Step 7: Add clipboard paste support + bgExtractInstructionsFromFile to dashboard.js

File: crates/hero_slides_admin/static/js/dashboard.js
Dependencies: Steps 5–6

Add bgExtractInstructionsFromFile() calling pollAiJob('bg.extractInstructionsAsync', ...), add a global paste event listener that detects image clipboard data and routes to instructions or theme extraction based on which editor is active, and register the new template key in TEMPLATE_TO_PROMPT_KEY.

Step 8: Add UI button to index.html

File: crates/hero_slides_admin/templates/index.html
Dependencies: Step 7

Add "Extract from image / PDF" button in the Instructions editor section and a paste hint (Ctrl+V).

Acceptance Criteria

  • Pasting an image (Ctrl+V) while the Instructions editor is open triggers instructions extraction with spinner feedback
  • Pasting an image while the Theme editor is open triggers theme extraction (existing feature now clipboard-accessible)
  • "Extract from image / PDF" button exists in the Instructions editor section
  • File picker accepts PNG, JPEG, WebP, GIF, PDF
  • Extraction runs as a background job visible in the jobs panel
  • On completion, instructions textarea is populated with AI-generated content (Audience, Tone, Key Messages, Slide Outline, Goals)
  • New prompt template appears in Templates tab and is user-editable/resettable
  • Toast notifications for: job started, success, failure
  • No regression to existing theme extraction flow

Notes

  • The pollAiJob pattern already provides job tracking; the new method just needs the same register_with_manager_inputs call with a temp output file in generate_job.rs.
  • resolve_extract_theme_source is reusable for the instructions extraction since the input shape (base64 inline or folder/file) is identical.
  • Adding ExtractInstructions to JobKind requires updating both as_str() and default_timeout() in jobs/types.rs.
## Implementation Spec: Issue #67 — PDF/Image to Instructions ### Objective Extend the "Extract from image / PDF" feature so it targets the **instructions.md** file (not just theme.md), adds clipboard paste as an input method, wraps the operation in a trackable background job (via the existing `pollAiJob` / `JobManager` infrastructure), improves the AI prompt for instructions extraction, and exposes `system_extract_instructions_from_image.md` as a user-editable prompt template. --- ### Current State Summary - `bgExtractThemeFromFile()` in `dashboard.js` opens a file picker, reads base64, calls `bg.extractThemeAsync` via `pollAiJob`. This saves to `theme.md`. - `bg.extractThemeAsync` handler in `generate_job.rs` calls `resolve_extract_theme_source()` (accepts `{data, mime_type}` for inline base64 OR `{folder, file}` for on-disk files), then submits a CLI job via `submit_extract_theme_job()`. - The CLI job runs `hero_slides theme extract <unc> --from <file>`, which calls `extract_theme_from_image()` / `extract_theme_from_pdf()` in `generator.rs`, backed by `system_extract_theme.md` prompt. - The `system_extract_theme.md` prompt is in `ALL_TEMPLATES` and editable via the Templates tab. - `pollAiJob` polls `agent.aiJobResult` at 800ms intervals; on `done` it returns `status.result` containing `{ content }`. - There is no equivalent flow for **instructions.md** from an image/PDF. `instruct_instructions` modifies existing instructions but there is no "extract instructions from visual reference" path. - No clipboard paste handling exists anywhere in the codebase. --- ### Requirements - Allow the user to paste an image from the clipboard (Ctrl+V / Cmd+V) as the input for extraction, in addition to the existing file-picker path - Add an "Extract instructions from image/PDF" action that sends the image/PDF to the AI and writes the result to `instructions.md` - The extraction must run as a background job tracked by `pollAiJob` / `JobManager` — same pattern as `bg.extractThemeAsync` - New AI prompt `system_extract_instructions_from_image.md` must produce well-structured `instructions.md` content (audience, tone, structure, key messages, slide outline, goals) from slide images - The new prompt template must be user-editable via the Templates tab (added to `ALL_TEMPLATES` in `prompts.rs` and `TEMPLATE_TO_PROMPT_KEY` in `dashboard.js`) - A "Extract instructions from image/PDF" button must appear in the Instructions editor section of the dashboard - Clipboard paste (Ctrl+V with an image) should work from both the Instructions and Theme editor contexts --- ### Files to Modify or Create | File | Action | Description | |---|---|---| | `crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md` | Create | New AI prompt for extracting instructions.md from a visual reference | | `crates/hero_slides_lib/src/prompts.rs` | Modify | Add const + entry in ALL_TEMPLATES | | `crates/hero_slides_lib/src/generator.rs` | Modify | Add `extract_instructions_from_image()` and `extract_instructions_from_pdf()` | | `crates/hero_slides_server/src/generate_job.rs` | Modify | Add job submission + handler for `bg.extractInstructionsAsync` | | `crates/hero_slides_server/src/rpc.rs` | Modify | Route `bg.extractInstructionsAsync` to the new handler | | `crates/hero_slides_admin/static/js/dashboard.js` | Modify | Add `bgExtractInstructionsFromFile()`, clipboard paste handler, template key | | `crates/hero_slides_admin/templates/index.html` | Modify | Add "Extract instructions from image/PDF" button + paste hint | --- ### Implementation Plan #### Step 1: Create the AI prompt template **File:** `crates/hero_slides_lib/src/prompts/system_extract_instructions_from_image.md` **Dependencies:** none Create the prompt instructing the AI to analyze slide images/PDFs and produce a structured `instructions.md` covering: audience, tone, key messages, slide structure/outline, and goals. Output must begin with `# Deck Instructions` and use `##` subheadings. No explanations, no code fences, no preamble — only the instructions.md content. #### Step 2: Register the template in `prompts.rs` **File:** `crates/hero_slides_lib/src/prompts.rs` **Dependencies:** Step 1 Add `pub const DEFAULT_SYSTEM_EXTRACT_INSTRUCTIONS_FROM_IMAGE` constant and add it to `ALL_TEMPLATES`. #### Step 3: Add extraction functions to `generator.rs` **File:** `crates/hero_slides_lib/src/generator.rs` **Dependencies:** Step 2 Add `extract_instructions_from_image()` and `extract_instructions_from_pdf()` mirroring the existing theme extraction functions, using the new prompt key `system_extract_instructions_from_image.md`. #### Step 4: Add the CLI subcommand **File:** `crates/hero_slides/src/main.rs` + relevant rhai module **Dependencies:** Step 3 Add `hero_slides instructions extract <unc> --from <file>` CLI command mirroring `theme extract`, calling the new generator functions and writing result to `instructions.md`. #### Step 5: Add backend job submission and handler **File:** `crates/hero_slides_server/src/generate_job.rs` **Dependencies:** Step 4 Add `submit_extract_instructions_job()` and `handle_bg_extract_instructions_async()` (plus status/logs handlers) mirroring the theme extraction job pattern. Add `JobKind::ExtractInstructions` variant. #### Step 6: Route the new RPC method **File:** `crates/hero_slides_server/src/rpc.rs` **Dependencies:** Step 5 Route `bg.extractInstructionsAsync`, `bg.extractInstructionsJobStatus`, `bg.extractInstructionsJobLogs` to the new handlers. #### Step 7: Add clipboard paste support + `bgExtractInstructionsFromFile` to `dashboard.js` **File:** `crates/hero_slides_admin/static/js/dashboard.js` **Dependencies:** Steps 5–6 Add `bgExtractInstructionsFromFile()` calling `pollAiJob('bg.extractInstructionsAsync', ...)`, add a global `paste` event listener that detects image clipboard data and routes to instructions or theme extraction based on which editor is active, and register the new template key in `TEMPLATE_TO_PROMPT_KEY`. #### Step 8: Add UI button to `index.html` **File:** `crates/hero_slides_admin/templates/index.html` **Dependencies:** Step 7 Add "Extract from image / PDF" button in the Instructions editor section and a paste hint (Ctrl+V). --- ### Acceptance Criteria - [ ] Pasting an image (Ctrl+V) while the Instructions editor is open triggers instructions extraction with spinner feedback - [ ] Pasting an image while the Theme editor is open triggers theme extraction (existing feature now clipboard-accessible) - [ ] "Extract from image / PDF" button exists in the Instructions editor section - [ ] File picker accepts PNG, JPEG, WebP, GIF, PDF - [ ] Extraction runs as a background job visible in the jobs panel - [ ] On completion, instructions textarea is populated with AI-generated content (Audience, Tone, Key Messages, Slide Outline, Goals) - [ ] New prompt template appears in Templates tab and is user-editable/resettable - [ ] Toast notifications for: job started, success, failure - [ ] No regression to existing theme extraction flow --- ### Notes - The `pollAiJob` pattern already provides job tracking; the new method just needs the same `register_with_manager_inputs` call with a temp output file in `generate_job.rs`. - `resolve_extract_theme_source` is reusable for the instructions extraction since the input shape (base64 inline or folder/file) is identical. - Adding `ExtractInstructions` to `JobKind` requires updating both `as_str()` and `default_timeout()` in `jobs/types.rs`.
casper-stevens commented 2026-05-15 08:38:17 +00:00
Member
Copy link

Implementation Spec (rev 2) — Issue #67: PDF/Image to Theme Instructions

Objective

Enhance the theme editor's "Extract from image / PDF" workflow:

  1. Clipboard paste support (Ctrl+V while theme editor is open → extraction starts immediately)
  2. Non-blocking extraction: loading indicator + AI button lockout during extraction, textarea stays editable
  3. New improved prompt template system_extract_theme_from_image.md producing actionable AI-ready theme instructions
  4. Template registered in ALL_TEMPLATES / TEMPLATE_TO_PROMPT_KEY and user-editable via Templates tab

Requirements

  • Pasting an image (Ctrl+V) while the theme editor overlay is visible starts an extract-theme job immediately — no file picker needed
  • During extraction: Fix / Rewrite / Instruct buttons are disabled/greyed, Extract button shows spinner label
  • The theme editor textarea is NOT disabled — user can keep editing while the job runs
  • On completion: extracted text is appended below existing content (or replaces it if textarea is empty)
  • On failure: toast shown, all buttons re-enabled
  • New prompt file system_extract_theme_from_image.md with improved prompt focused on producing actionable theme instructions (color palette with hex codes, typography, layout, imagery, AI image generation directives)
  • system_extract_theme_from_image.md registered in ALL_TEMPLATES and editable in the Templates tab
  • extract_theme_from_image and extract_theme_from_pdf in generator.rs switched to use new template

Files to Modify / Create

File Action Description
crates/hero_slides_lib/src/prompts/system_extract_theme_from_image.md Create Improved prompt for image-to-theme-instructions extraction
crates/hero_slides_lib/src/prompts.rs Modify Add DEFAULT_SYSTEM_EXTRACT_THEME_FROM_IMAGE const + ALL_TEMPLATES entry
crates/hero_slides_lib/src/generator.rs Modify Switch extract_theme_from_image / extract_theme_from_pdf to new template name
crates/hero_slides_admin/templates/index.html Modify Add id="btn-extract-theme-file" to Extract button
crates/hero_slides_admin/static/js/dashboard.js Modify Non-blocking extract flow, clipboard paste handler, button lockout helpers, new template mapping

Implementation Plan

Step 1: New prompt file

File: crates/hero_slides_lib/src/prompts/system_extract_theme_from_image.md
Dependencies: none

Create an improved prompt instructing the AI to analyze a visual reference and produce actionable theme instructions covering: color palette (exact hex codes), typography, layout rules, imagery style, mood/brand voice, and an ## AI Image Generation Instructions section with bullet-pointed directives. Output starts with # Visual Theme, uses ## sub-sections, no code fences, no preamble.

Step 2: Register in prompts.rs

File: crates/hero_slides_lib/src/prompts.rs
Dependencies: Step 1

Add pub const DEFAULT_SYSTEM_EXTRACT_THEME_FROM_IMAGE: &str = include_str!("prompts/system_extract_theme_from_image.md"); and add ("system_extract_theme_from_image.md", DEFAULT_SYSTEM_EXTRACT_THEME_FROM_IMAGE) to ALL_TEMPLATES.

Step 3: Switch generator to new template

File: crates/hero_slides_lib/src/generator.rs
Dependencies: Step 2

In extract_theme_from_image and extract_theme_from_pdf, change the template name from "system_extract_theme.md" to "system_extract_theme_from_image.md". Old system_extract_theme.md stays registered in ALL_TEMPLATES.

Step 4: Add id to Extract button

File: crates/hero_slides_admin/templates/index.html
Dependencies: none (independent)

Add id="btn-extract-theme-file" to the existing Extract button. Check for Fix, Rewrite, Instruct button IDs and add them if missing.

Step 5: Refactor dashboard.js — lockout helpers + non-blocking extract + clipboard paste

File: crates/hero_slides_admin/static/js/dashboard.js
Dependencies: Step 4

  • Add _lockThemeAiButtons(label) helper: disables Fix/Rewrite/Instruct/Extract buttons, shows spinner label on Extract button
  • Add _unlockThemeAiButtons() helper: re-enables all, restores Extract button label
  • Extract shared job logic into _runExtractThemeJob(mimeType, base64Data): calls lock, runs pollAiJob('bg.extractThemeAsync', ...), on success appends/replaces textarea content, calls _unlockThemeAiButtons() in finally
  • Refactor bgExtractThemeFromFile() to read file and call _runExtractThemeJob
  • Update bgExtractTheme(folder, file) context-menu path to also use lock/unlock + append-or-replace
  • Add document.addEventListener('paste', ...) handler: fires only when theme editor overlay is active, extracts image from clipboard, calls _runExtractThemeJob
  • Add 'system_extract_theme_from_image.md': 'extract_theme' to TEMPLATE_TO_PROMPT_KEY

Acceptance Criteria

  • Ctrl+V with an image in clipboard while theme editor is open starts extraction immediately
  • During extraction: Fix, Rewrite, Instruct buttons are greyed; Extract button shows spinner; textarea remains editable
  • On success: extracted text appended below existing content, or replaces empty textarea
  • On failure: buttons re-enabled, error toast shown
  • system_extract_theme_from_image.md appears in Templates tab and is user-editable/resettable
  • cargo build passes with no errors
  • Existing file-picker extraction flow still works
  • Existing context-menu bgExtractTheme(folder, file) flow still works

Notes

  • No backend changes required — bg.extractThemeAsync already accepts { data, mime_type } base64 payloads. Clipboard path reuses the exact same code path as the file-picker.
  • The old system_extract_theme.md is not removed — stays in ALL_TEMPLATES for users who customized it.
  • Clipboard paste handler checks for theme-editor-overlay having active class before firing — prevents accidental extraction when pasting text elsewhere in the UI.
## Implementation Spec (rev 2) — Issue #67: PDF/Image to Theme Instructions ### Objective Enhance the theme editor's "Extract from image / PDF" workflow: 1. Clipboard paste support (Ctrl+V while theme editor is open → extraction starts immediately) 2. Non-blocking extraction: loading indicator + AI button lockout during extraction, textarea stays editable 3. New improved prompt template `system_extract_theme_from_image.md` producing actionable AI-ready theme instructions 4. Template registered in `ALL_TEMPLATES` / `TEMPLATE_TO_PROMPT_KEY` and user-editable via Templates tab --- ### Requirements - Pasting an image (Ctrl+V) while the theme editor overlay is visible starts an extract-theme job immediately — no file picker needed - During extraction: Fix / Rewrite / Instruct buttons are disabled/greyed, Extract button shows spinner label - The theme editor textarea is NOT disabled — user can keep editing while the job runs - On completion: extracted text is appended below existing content (or replaces it if textarea is empty) - On failure: toast shown, all buttons re-enabled - New prompt file `system_extract_theme_from_image.md` with improved prompt focused on producing actionable theme instructions (color palette with hex codes, typography, layout, imagery, AI image generation directives) - `system_extract_theme_from_image.md` registered in `ALL_TEMPLATES` and editable in the Templates tab - `extract_theme_from_image` and `extract_theme_from_pdf` in `generator.rs` switched to use new template --- ### Files to Modify / Create | File | Action | Description | |------|--------|-------------| | `crates/hero_slides_lib/src/prompts/system_extract_theme_from_image.md` | Create | Improved prompt for image-to-theme-instructions extraction | | `crates/hero_slides_lib/src/prompts.rs` | Modify | Add `DEFAULT_SYSTEM_EXTRACT_THEME_FROM_IMAGE` const + ALL_TEMPLATES entry | | `crates/hero_slides_lib/src/generator.rs` | Modify | Switch `extract_theme_from_image` / `extract_theme_from_pdf` to new template name | | `crates/hero_slides_admin/templates/index.html` | Modify | Add `id="btn-extract-theme-file"` to Extract button | | `crates/hero_slides_admin/static/js/dashboard.js` | Modify | Non-blocking extract flow, clipboard paste handler, button lockout helpers, new template mapping | --- ### Implementation Plan #### Step 1: New prompt file **File:** `crates/hero_slides_lib/src/prompts/system_extract_theme_from_image.md` **Dependencies:** none Create an improved prompt instructing the AI to analyze a visual reference and produce actionable theme instructions covering: color palette (exact hex codes), typography, layout rules, imagery style, mood/brand voice, and an `## AI Image Generation Instructions` section with bullet-pointed directives. Output starts with `# Visual Theme`, uses `##` sub-sections, no code fences, no preamble. #### Step 2: Register in `prompts.rs` **File:** `crates/hero_slides_lib/src/prompts.rs` **Dependencies:** Step 1 Add `pub const DEFAULT_SYSTEM_EXTRACT_THEME_FROM_IMAGE: &str = include_str!("prompts/system_extract_theme_from_image.md");` and add `("system_extract_theme_from_image.md", DEFAULT_SYSTEM_EXTRACT_THEME_FROM_IMAGE)` to `ALL_TEMPLATES`. #### Step 3: Switch generator to new template **File:** `crates/hero_slides_lib/src/generator.rs` **Dependencies:** Step 2 In `extract_theme_from_image` and `extract_theme_from_pdf`, change the template name from `"system_extract_theme.md"` to `"system_extract_theme_from_image.md"`. Old `system_extract_theme.md` stays registered in ALL_TEMPLATES. #### Step 4: Add `id` to Extract button **File:** `crates/hero_slides_admin/templates/index.html` **Dependencies:** none (independent) Add `id="btn-extract-theme-file"` to the existing Extract button. Check for Fix, Rewrite, Instruct button IDs and add them if missing. #### Step 5: Refactor `dashboard.js` — lockout helpers + non-blocking extract + clipboard paste **File:** `crates/hero_slides_admin/static/js/dashboard.js` **Dependencies:** Step 4 - Add `_lockThemeAiButtons(label)` helper: disables Fix/Rewrite/Instruct/Extract buttons, shows spinner label on Extract button - Add `_unlockThemeAiButtons()` helper: re-enables all, restores Extract button label - Extract shared job logic into `_runExtractThemeJob(mimeType, base64Data)`: calls lock, runs `pollAiJob('bg.extractThemeAsync', ...)`, on success appends/replaces textarea content, calls `_unlockThemeAiButtons()` in finally - Refactor `bgExtractThemeFromFile()` to read file and call `_runExtractThemeJob` - Update `bgExtractTheme(folder, file)` context-menu path to also use lock/unlock + append-or-replace - Add `document.addEventListener('paste', ...)` handler: fires only when theme editor overlay is active, extracts image from clipboard, calls `_runExtractThemeJob` - Add `'system_extract_theme_from_image.md': 'extract_theme'` to `TEMPLATE_TO_PROMPT_KEY` --- ### Acceptance Criteria - [ ] Ctrl+V with an image in clipboard while theme editor is open starts extraction immediately - [ ] During extraction: Fix, Rewrite, Instruct buttons are greyed; Extract button shows spinner; textarea remains editable - [ ] On success: extracted text appended below existing content, or replaces empty textarea - [ ] On failure: buttons re-enabled, error toast shown - [ ] `system_extract_theme_from_image.md` appears in Templates tab and is user-editable/resettable - [ ] `cargo build` passes with no errors - [ ] Existing file-picker extraction flow still works - [ ] Existing context-menu `bgExtractTheme(folder, file)` flow still works --- ### Notes - No backend changes required — `bg.extractThemeAsync` already accepts `{ data, mime_type }` base64 payloads. Clipboard path reuses the exact same code path as the file-picker. - The old `system_extract_theme.md` is not removed — stays in ALL_TEMPLATES for users who customized it. - Clipboard paste handler checks for `theme-editor-overlay` having `active` class before firing — prevents accidental extraction when pasting text elsewhere in the UI.
casper-stevens commented 2026-05-15 08:49:26 +00:00
Member
Copy link

Test Results

  • Total: 170
  • Passed: 169
  • Failed: 0
  • Ignored: 1

All tests passed. One test (test_generate_single_slide_ai) was skipped as it requires a live AI backend.

Note: hero_slides_examples failed to compile due to missing tokio dependency in its examples (health.rs, basic_usage.rs). All other crates compiled and tested successfully.

## Test Results - Total: 170 - Passed: 169 - Failed: 0 - Ignored: 1 All tests passed. One test (test_generate_single_slide_ai) was skipped as it requires a live AI backend. Note: hero_slides_examples failed to compile due to missing tokio dependency in its examples (health.rs, basic_usage.rs). All other crates compiled and tested successfully.
casper-stevens commented 2026-05-15 08:50:01 +00:00
Member
Copy link

Implementation Summary

Changes Made

crates/hero_slides_lib/src/prompts/system_extract_theme_from_image.md (created)
New AI prompt for image/PDF-to-theme-instructions extraction. Instructs the AI to produce a structured # Visual Theme document covering exact hex color palette, typography, layout rules, imagery style, mood/brand voice, and a dedicated ## AI Image Generation Instructions section with bullet-pointed directives ready for an image generator.

crates/hero_slides_lib/src/prompts.rs (modified)
Added DEFAULT_SYSTEM_EXTRACT_THEME_FROM_IMAGE constant and registered system_extract_theme_from_image.md in ALL_TEMPLATES so it appears in the Templates tab and is user-editable/resettable.

crates/hero_slides_lib/src/generator.rs (modified)
extract_theme_from_image and extract_theme_from_pdf now use the new system_extract_theme_from_image.md template instead of system_extract_theme.md. The old template stays registered for users who customized it.

crates/hero_slides_admin/templates/index.html (modified)
Added id="btn-extract-theme-file" to the Extract button (Fix/Rewrite/Instruct already had ids).

crates/hero_slides_admin/static/js/dashboard.js (modified)

  • _lockThemeAiButtons(label) / _unlockThemeAiButtons(): helpers that disable Fix, Rewrite, Instruct, and Extract buttons during a job run; Extract button shows a hourglass spinner
  • _runExtractThemeJob(mimeType, base64Data): shared extraction job runner — locks buttons, runs pollAiJob('bg.extractThemeAsync', ...), appends result to theme textarea (or replaces if empty), unlocks on complete/fail
  • bgExtractThemeFromFile() refactored to delegate to _runExtractThemeJob
  • bgExtractTheme(folder, file) updated to use the same lock/unlock and append-or-replace behavior
  • Clipboard paste handler: when the theme editor overlay is active and the user pastes an image (Ctrl+V), extraction starts immediately without a file picker
  • TEMPLATE_TO_PROMPT_KEY updated with system_extract_theme_from_image.md entry

Test Results

169 tests passed, 0 failed, 1 ignored (live AI test).

## Implementation Summary ### Changes Made **`crates/hero_slides_lib/src/prompts/system_extract_theme_from_image.md`** (created) New AI prompt for image/PDF-to-theme-instructions extraction. Instructs the AI to produce a structured `# Visual Theme` document covering exact hex color palette, typography, layout rules, imagery style, mood/brand voice, and a dedicated `## AI Image Generation Instructions` section with bullet-pointed directives ready for an image generator. **`crates/hero_slides_lib/src/prompts.rs`** (modified) Added `DEFAULT_SYSTEM_EXTRACT_THEME_FROM_IMAGE` constant and registered `system_extract_theme_from_image.md` in `ALL_TEMPLATES` so it appears in the Templates tab and is user-editable/resettable. **`crates/hero_slides_lib/src/generator.rs`** (modified) `extract_theme_from_image` and `extract_theme_from_pdf` now use the new `system_extract_theme_from_image.md` template instead of `system_extract_theme.md`. The old template stays registered for users who customized it. **`crates/hero_slides_admin/templates/index.html`** (modified) Added `id="btn-extract-theme-file"` to the Extract button (Fix/Rewrite/Instruct already had ids). **`crates/hero_slides_admin/static/js/dashboard.js`** (modified) - `_lockThemeAiButtons(label)` / `_unlockThemeAiButtons()`: helpers that disable Fix, Rewrite, Instruct, and Extract buttons during a job run; Extract button shows a hourglass spinner - `_runExtractThemeJob(mimeType, base64Data)`: shared extraction job runner — locks buttons, runs `pollAiJob('bg.extractThemeAsync', ...)`, appends result to theme textarea (or replaces if empty), unlocks on complete/fail - `bgExtractThemeFromFile()` refactored to delegate to `_runExtractThemeJob` - `bgExtractTheme(folder, file)` updated to use the same lock/unlock and append-or-replace behavior - Clipboard paste handler: when the theme editor overlay is active and the user pastes an image (Ctrl+V), extraction starts immediately without a file picker - `TEMPLATE_TO_PROMPT_KEY` updated with `system_extract_theme_from_image.md` entry ### Test Results 169 tests passed, 0 failed, 1 ignored (live AI test).
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
development
main
development_kristof2
development_kristof
latest
v0.1.0-rc2
v0.1.0-rc1
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
lhumina_code/hero_slides#67
No description provided.