lhumina_code/hero_slides

Fork 0

Improve background panel: collapse behavior, image upload, and single-file support #7

New issue

Closed

opened 2026-04-13 17:52:02 +00:00 by casper-stevens · 7 comments

casper-stevens commented

2026-04-13 17:52:02 +00:00

Member

Context

The background panel has several issues: it takes up space when empty, image uploads do not work, and files must be inside a subfolder to be recognized as context. These friction points make the background system difficult to use.

Goals

Collapse the background tab by default when it contains no files or folders
Remove the "BG" label when the panel is collapsed
Fix background image upload so uploaded images appear in the file list
Support single files at the root of the background folder without requiring a subfolder
Include root-level files in the context passed to AI slide generation

## Context The background panel has several issues: it takes up space when empty, image uploads do not work, and files must be inside a subfolder to be recognized as context. These friction points make the background system difficult to use. ## Goals - Collapse the background tab by default when it contains no files or folders - Remove the "BG" label when the panel is collapsed - Fix background image upload so uploaded images appear in the file list - Support single files at the root of the background folder without requiring a subfolder - Include root-level files in the context passed to AI slide generation

casper-stevens commented

2026-04-13 21:51:00 +00:00

Author

Member

Implementation Spec

Objective

Fix five usability issues in the background panel: collapse behavior, image upload rejection, and root-level file support.

Requirements

Collapse the background panel by default when empty (no folders, no root files)
Auto-open the panel when the deck has background content
Remove the "BG" label from the collapsed toggle (verify if present)
Accept image uploads (png, jpg, jpeg, webp, gif) on the backend -- currently rejected despite UI allowing them
Support single files at the root of content/background/ without requiring a subfolder
Include root-level .md/.txt files in the AI generation context

Files to Modify

File	Changes
`crates/hero_slides_lib/src/deck.rs`	Fix extension filter (L193), root-aware context loading (L564, L714), root file wrapper, root-aware save
`crates/hero_slides_lib/src/discovery.rs`	New `find_background_root_files()`, update `read_background_content()`, update `has_background`
`crates/hero_slides_lib/src/lib.rs`	Re-export new functions
`crates/hero_slides_server/src/rpc.rs`	Return `root_files` in listFolders, handle empty folder param
`crates/hero_slides_ui/static/js/dashboard.js`	Collapse-by-default, root file rendering, upload-to-root
`crates/hero_slides_ui/templates/index.html`	Verify toggle (currently icon-only, may need no changes)

Implementation Plan

Step 1: Fix image upload backend rejection

Files: crates/hero_slides_lib/src/deck.rs

Change extension whitelist in deck_save_background_file() (line 193) from md|pdf|txt to also include png|jpg|jpeg|webp|gif
Dependencies: none

Step 2: Add root-level file discovery and include in AI context

Files: crates/hero_slides_lib/src/discovery.rs, crates/hero_slides_lib/src/lib.rs

Add find_background_root_files() function to scan content/background/ for files (not dirs)
Update read_background_content() to also read root-level .md/.txt files
Update has_background in deck_info_from_path() to consider root files
Re-export new function from lib.rs
Dependencies: none

Step 3: Update deck generation to include root files in context

Files: crates/hero_slides_lib/src/deck.rs

Update deck_generate() (L564) and slide_generate() (L714) to check for root files when building background_context
Dependencies: Step 2

Step 4: Expose root files through RPC layer

Files: crates/hero_slides_lib/src/deck.rs, crates/hero_slides_server/src/rpc.rs

Add deck_list_background_root_files() wrapper
Update handle_bg_list_folders to return root_files in response
Allow empty string folder param in upload/list/delete/read handlers for root-level operations
Update deck_save_background_file to handle empty folder name
Dependencies: Step 2

Step 5: Update frontend -- root files, collapse behavior, BG label

Files: crates/hero_slides_ui/static/js/dashboard.js, crates/hero_slides_ui/templates/index.html

Default bgPanelOpen = false, auto-open when content exists in loadBgPanel()
Extract and render root_files from bg.listFolders response
Allow uploading to root (empty folder) when no folders exist
Verify collapsed toggle has no "BG" text label (currently icon-only)
Dependencies: Step 4

Acceptance Criteria

Uploading a .png or .jpg via the UI succeeds and file appears in the list
Panel starts collapsed when deck has no background content
Panel auto-opens when deck has folders or root files
Collapsed toggle shows only the folder icon (no text)
A .md file placed directly in content/background/ appears in the UI as a root file
Root-level files are included in AI generation context
cargo build and cargo test pass

Notes

Image files are not included in AI text context (only .md/.txt are read). Images are for visual reference in the file browser only.
PDF extraction remains subfolder-only (no change needed).
The bg.moveFile handler also needs to support empty folder for moving files to/from root.

## Implementation Spec ### Objective Fix five usability issues in the background panel: collapse behavior, image upload rejection, and root-level file support. ### Requirements - Collapse the background panel by default when empty (no folders, no root files) - Auto-open the panel when the deck has background content - Remove the "BG" label from the collapsed toggle (verify if present) - Accept image uploads (png, jpg, jpeg, webp, gif) on the backend -- currently rejected despite UI allowing them - Support single files at the root of `content/background/` without requiring a subfolder - Include root-level `.md`/`.txt` files in the AI generation context ### Files to Modify | File | Changes | |------|---------| | `crates/hero_slides_lib/src/deck.rs` | Fix extension filter (L193), root-aware context loading (L564, L714), root file wrapper, root-aware save | | `crates/hero_slides_lib/src/discovery.rs` | New `find_background_root_files()`, update `read_background_content()`, update `has_background` | | `crates/hero_slides_lib/src/lib.rs` | Re-export new functions | | `crates/hero_slides_server/src/rpc.rs` | Return `root_files` in listFolders, handle empty folder param | | `crates/hero_slides_ui/static/js/dashboard.js` | Collapse-by-default, root file rendering, upload-to-root | | `crates/hero_slides_ui/templates/index.html` | Verify toggle (currently icon-only, may need no changes) | ### Implementation Plan #### Step 1: Fix image upload backend rejection Files: `crates/hero_slides_lib/src/deck.rs` - Change extension whitelist in `deck_save_background_file()` (line 193) from `md|pdf|txt` to also include `png|jpg|jpeg|webp|gif` - Dependencies: none #### Step 2: Add root-level file discovery and include in AI context Files: `crates/hero_slides_lib/src/discovery.rs`, `crates/hero_slides_lib/src/lib.rs` - Add `find_background_root_files()` function to scan `content/background/` for files (not dirs) - Update `read_background_content()` to also read root-level `.md`/`.txt` files - Update `has_background` in `deck_info_from_path()` to consider root files - Re-export new function from `lib.rs` - Dependencies: none #### Step 3: Update deck generation to include root files in context Files: `crates/hero_slides_lib/src/deck.rs` - Update `deck_generate()` (L564) and `slide_generate()` (L714) to check for root files when building `background_context` - Dependencies: Step 2 #### Step 4: Expose root files through RPC layer Files: `crates/hero_slides_lib/src/deck.rs`, `crates/hero_slides_server/src/rpc.rs` - Add `deck_list_background_root_files()` wrapper - Update `handle_bg_list_folders` to return `root_files` in response - Allow empty string folder param in upload/list/delete/read handlers for root-level operations - Update `deck_save_background_file` to handle empty folder name - Dependencies: Step 2 #### Step 5: Update frontend -- root files, collapse behavior, BG label Files: `crates/hero_slides_ui/static/js/dashboard.js`, `crates/hero_slides_ui/templates/index.html` - Default `bgPanelOpen = false`, auto-open when content exists in `loadBgPanel()` - Extract and render `root_files` from `bg.listFolders` response - Allow uploading to root (empty folder) when no folders exist - Verify collapsed toggle has no "BG" text label (currently icon-only) - Dependencies: Step 4 ### Acceptance Criteria - [ ] Uploading a `.png` or `.jpg` via the UI succeeds and file appears in the list - [ ] Panel starts collapsed when deck has no background content - [ ] Panel auto-opens when deck has folders or root files - [ ] Collapsed toggle shows only the folder icon (no text) - [ ] A `.md` file placed directly in `content/background/` appears in the UI as a root file - [ ] Root-level files are included in AI generation context - [ ] `cargo build` and `cargo test` pass ### Notes - Image files are not included in AI text context (only `.md`/`.txt` are read). Images are for visual reference in the file browser only. - PDF extraction remains subfolder-only (no change needed). - The `bg.moveFile` handler also needs to support empty folder for moving files to/from root.

casper-stevens commented

2026-04-14 15:16:24 +00:00

Author

Member

Updated Implementation Spec — scope expanded

Including images as visual input to AI generation (both content and image generation).

Updated Requirements

In addition to the original goals (collapse behavior, image upload fix, root-file support):

Background image files (png, jpg, jpeg, webp, gif) must be passed as multimodal input to the AI slide content generation
Background images must also be passed to the slide image generation as visual reference
All images from both the background root and subfolders are included on every generation call ("model decides" which to use)

Updated Files to Modify

File	Changes
`crates/hero_slides_lib/src/deck.rs`	Fix ext filter; root-file wrapper; root-aware save; root-aware context + images in `deck_generate`/`slide_generate`
`crates/hero_slides_lib/src/discovery.rs`	New `find_background_root_files()`; update `read_background_content()`; update `has_background`; new `collect_background_images()` helper
`crates/hero_slides_lib/src/generator.rs`	Accept `images` param in `generate_slide_content`, `instruct_slide_content`, `generate_slide`; multimodal image generation via `chat_with_raw_model`
`crates/hero_slides_lib/src/lib.rs`	Re-exports
`crates/hero_slides_server/src/rpc.rs`	Return `root_files`; handle empty folder param
`crates/hero_slides_ui/static/js/dashboard.js`	Collapse-by-default; root file rendering; upload-to-root
`crates/hero_slides_ui/templates/index.html`	Verify (likely no changes)

Updated Implementation Plan

Step 1: Fix image upload backend rejection

Files: crates/hero_slides_lib/src/deck.rs

Change extension whitelist in deck_save_background_file() to include images
Dependencies: none

Step 2: Root-level file discovery and text context

Files: crates/hero_slides_lib/src/discovery.rs, crates/hero_slides_lib/src/lib.rs

Add find_background_root_files()
Update read_background_content() to include root .md/.txt files
Update has_background to consider root files
Re-export
Dependencies: none

Step 3: Pass images to AI content generation

Files: crates/hero_slides_lib/src/generator.rs, crates/hero_slides_lib/src/discovery.rs

Add collect_background_images(deck_path) -> Vec<(mime, base64)> helper
Update generate_slide_content() and instruct_slide_content() to accept images parameter
Use Message::user_with_images() when images are present (API already available in herolib_ai)
Models used (Gpt5_4, Gemini3ProPreview) already support vision
Dependencies: Step 2

Step 4: Pass images to slide image generation

Files: crates/hero_slides_lib/src/generator.rs

Update generate_slide() to accept background_images
When images are present, bypass generate_image_to_file and use chat_with_raw_model with google/gemini-3.1-flash-image-preview and multimodal messages, mirroring the existing PDF extraction pattern in deck.rs:308-327
Parse response to extract generated image
Dependencies: Step 3

Step 5: Update deck generation entry points

Files: crates/hero_slides_lib/src/deck.rs

Update deck_generate() (L564) and slide_generate() (L714) to collect and pass images
Dependencies: Steps 2, 3, 4

Step 6: Expose root files through RPC

Files: crates/hero_slides_lib/src/deck.rs, crates/hero_slides_server/src/rpc.rs

Add deck_list_background_root_files() wrapper
Update handle_bg_list_folders to return root_files
Allow empty folder string in upload/list/delete/read handlers
Root-aware deck_save_background_file
Dependencies: Step 2

Step 7: Frontend — root files, collapse behavior, BG label

Files: crates/hero_slides_ui/static/js/dashboard.js, crates/hero_slides_ui/templates/index.html

bgPanelOpen = false default; auto-open when content exists
Render root files in tree
Allow upload to root
Verify no "BG" text label in collapsed state
Dependencies: Step 6

Acceptance Criteria

Uploading .png/.jpg via the UI succeeds
Panel collapsed when empty; auto-opens when content exists
Root-level .md/.txt in content/background/ appears in UI and AI text context
Background images are sent to content generation as multimodal input
Background images are used as reference by the image generator
cargo build and cargo test pass

Notes

herolib_ai::Message::user_with_images(text, &[(mime, base64)]) already exists and is used by extract_pdf_with_ai
Using chat_with_raw_model for multimodal image generation follows the same pattern as PDF extraction
Large numbers of images may exceed model context — this is acceptable for now ("model decides")

## Updated Implementation Spec — scope expanded Including images as visual input to AI generation (both content and image generation). ### Updated Requirements In addition to the original goals (collapse behavior, image upload fix, root-file support): - Background image files (`png`, `jpg`, `jpeg`, `webp`, `gif`) must be passed as multimodal input to the AI slide content generation - Background images must also be passed to the slide image generation as visual reference - All images from both the background root and subfolders are included on every generation call ("model decides" which to use) ### Updated Files to Modify | File | Changes | |------|---------| | `crates/hero_slides_lib/src/deck.rs` | Fix ext filter; root-file wrapper; root-aware save; root-aware context + images in `deck_generate`/`slide_generate` | | `crates/hero_slides_lib/src/discovery.rs` | New `find_background_root_files()`; update `read_background_content()`; update `has_background`; new `collect_background_images()` helper | | `crates/hero_slides_lib/src/generator.rs` | Accept `images` param in `generate_slide_content`, `instruct_slide_content`, `generate_slide`; multimodal image generation via `chat_with_raw_model` | | `crates/hero_slides_lib/src/lib.rs` | Re-exports | | `crates/hero_slides_server/src/rpc.rs` | Return `root_files`; handle empty folder param | | `crates/hero_slides_ui/static/js/dashboard.js` | Collapse-by-default; root file rendering; upload-to-root | | `crates/hero_slides_ui/templates/index.html` | Verify (likely no changes) | ### Updated Implementation Plan #### Step 1: Fix image upload backend rejection Files: `crates/hero_slides_lib/src/deck.rs` - Change extension whitelist in `deck_save_background_file()` to include images - Dependencies: none #### Step 2: Root-level file discovery and text context Files: `crates/hero_slides_lib/src/discovery.rs`, `crates/hero_slides_lib/src/lib.rs` - Add `find_background_root_files()` - Update `read_background_content()` to include root `.md`/`.txt` files - Update `has_background` to consider root files - Re-export - Dependencies: none #### Step 3: Pass images to AI content generation Files: `crates/hero_slides_lib/src/generator.rs`, `crates/hero_slides_lib/src/discovery.rs` - Add `collect_background_images(deck_path) -> Vec<(mime, base64)>` helper - Update `generate_slide_content()` and `instruct_slide_content()` to accept images parameter - Use `Message::user_with_images()` when images are present (API already available in `herolib_ai`) - Models used (`Gpt5_4`, `Gemini3ProPreview`) already support vision - Dependencies: Step 2 #### Step 4: Pass images to slide image generation Files: `crates/hero_slides_lib/src/generator.rs` - Update `generate_slide()` to accept `background_images` - When images are present, bypass `generate_image_to_file` and use `chat_with_raw_model` with `google/gemini-3.1-flash-image-preview` and multimodal messages, mirroring the existing PDF extraction pattern in `deck.rs:308-327` - Parse response to extract generated image - Dependencies: Step 3 #### Step 5: Update deck generation entry points Files: `crates/hero_slides_lib/src/deck.rs` - Update `deck_generate()` (L564) and `slide_generate()` (L714) to collect and pass images - Dependencies: Steps 2, 3, 4 #### Step 6: Expose root files through RPC Files: `crates/hero_slides_lib/src/deck.rs`, `crates/hero_slides_server/src/rpc.rs` - Add `deck_list_background_root_files()` wrapper - Update `handle_bg_list_folders` to return `root_files` - Allow empty folder string in upload/list/delete/read handlers - Root-aware `deck_save_background_file` - Dependencies: Step 2 #### Step 7: Frontend — root files, collapse behavior, BG label Files: `crates/hero_slides_ui/static/js/dashboard.js`, `crates/hero_slides_ui/templates/index.html` - `bgPanelOpen = false` default; auto-open when content exists - Render root files in tree - Allow upload to root - Verify no "BG" text label in collapsed state - Dependencies: Step 6 ### Acceptance Criteria - [ ] Uploading `.png`/`.jpg` via the UI succeeds - [ ] Panel collapsed when empty; auto-opens when content exists - [ ] Root-level `.md`/`.txt` in `content/background/` appears in UI and AI text context - [ ] Background images are sent to content generation as multimodal input - [ ] Background images are used as reference by the image generator - [ ] `cargo build` and `cargo test` pass ### Notes - `herolib_ai::Message::user_with_images(text, &[(mime, base64)])` already exists and is used by `extract_pdf_with_ai` - Using `chat_with_raw_model` for multimodal image generation follows the same pattern as PDF extraction - Large numbers of images may exceed model context — this is acceptable for now ("model decides")

casper-stevens commented

2026-04-14 22:35:22 +00:00

Author

Member

Implementation Spec — Issue #7

Objective

Fix five usability issues in the background panel: image upload rejection, root-level file invisibility, AI context gaps, and collapse behavior.

Scope

This spec covers all goals from the issue including the expanded multimodal scope from comment #2, with one exception: passing background images as visual input to slide image generation is deferred — the current herolib_ai API does not expose a path to send input images alongside image-output modality requests without triggering an EmptyResponse error in send_request. All other goals are implemented.

Files to Modify

File	Changes
`crates/hero_slides_lib/src/deck.rs`	Fix ext filter (L213); support empty folder in file ops (L160–L352)
`crates/hero_slides_lib/src/discovery.rs`	New `find_background_root_files()`; new `read_background_root_content()`; new `collect_background_images()`; update `has_background` (L271); update `deck_info_from_path()`
`crates/hero_slides_lib/src/generator.rs`	Add `background_images` param to `generate_slide_content` and `instruct_slide_content`; use `Message::user_with_images` when images present
`crates/hero_slides_lib/src/lib.rs`	Re-export new functions
`crates/hero_slides_server/src/rpc.rs`	Return `root_files` in `handle_bg_list_folders`; allow empty folder string in upload/list/delete/read handlers
`crates/hero_slides_ui/static/js/dashboard.js`	Collapse-by-default; auto-open when content exists; root file rendering; upload to root

Implementation Plan

Step 1: Fix image upload backend rejection

File: crates/hero_slides_lib/src/deck.rs

Line 213: change extension check from ext != "md" && ext != "pdf" && ext != "txt" to also allow png, jpg, jpeg, webp, gif
Update the error message accordingly
Lines 160–196 (deck_delete_background_file, deck_move_background_file): when folder_name is empty, construct path directly under content/background/ (skip the folder segment in join())
Lines 203–229 (deck_save_background_file): when folder_name is empty, save directly to content/background/; PDF extraction path stays subfolder-only (only apply deck_extract_pdf_to_md when folder_name is non-empty)
Lines 341–352 (deck_read_background_file): when folder_name is empty, read from content/background/ root

Dependencies: none

Step 2: Root-level file discovery and AI text context

File: crates/hero_slides_lib/src/discovery.rs

Add find_background_root_files(deck_path: &Path) -> Vec<BackgroundFile>:

Scan content/background/ for immediate children that are files (not dirs)
Accept extensions: md, pdf, txt, png, jpg, jpeg, webp, gif
Return sorted alphabetically

Add read_background_root_content(deck_path: &Path) -> String:

Read all .md and .txt files from content/background/ root (skip PDFs, images)
Return concatenated content prefixed with ## (root)\n\n
Return empty string if none

Add collect_background_images(deck_path: &Path) -> Vec<(String, String)>:

Scan all image files (png, jpg, jpeg, webp, gif) from:
- content/background/ root
- All immediate subdirectories of content/background/
For each image: read bytes, base64-encode, determine MIME type (image/png, image/jpeg, image/webp, image/gif)
Return Vec<(mime_type, base64_data)>

Update deck_info_from_path() line 271:

has_background should be true if find_background_folders() is non-empty OR find_background_root_files() is non-empty

Dependencies: none (independent of Step 1)

Step 3: Include root files and images in AI generation context

File: crates/hero_slides_lib/src/deck.rs

In deck_generate() (lines 584–590):

After collecting bg_folders and background_context, also call read_background_root_content() and prepend it to background_context if non-empty
After that, call collect_background_images() and store as background_images: Vec<(String, String)>
Pass &background_images to generate_slide()

In slide_generate() (lines 735–740): same additions

Dependencies: Step 2

Step 4: Multimodal images in text content generation

File: crates/hero_slides_lib/src/generator.rs

Update generate_slide_content() (line 138):

Add parameter background_images: &[(String, String)]
When background_images is non-empty, build message as Message::user_with_images(&user_msg, background_images) instead of Message::user(&user_msg)

Update instruct_slide_content() (line 189):

Add parameter background_images: &[(String, String)]
Same pattern as above

Update callers in rpc.rs that call these functions — pass &[] initially (images from deck context not wired through RPC yet; can be extended later)

Update callers in deck.rs to pass the collected background_images

Note: generate_slide() (image generation) does NOT get images passed — deferred as noted in Scope.

Dependencies: Step 3

Step 5: RPC layer — root files and empty folder support

File: crates/hero_slides_server/src/rpc.rs

In handle_bg_list_folders() (line 672):

After fetching folders, also call hero_slides_lib::find_background_root_files(&dir) (re-export needed)
Add root_files: [{ name, extension, size_bytes }] array to the JSON response alongside folders

In handle_bg_list_files() (line 698):

Make folder param optional (default empty string)
Call deck_list_background_files(&dir, &folder) — Step 1 makes this work with empty folder

In handle_bg_upload_file() (line 825):

Make folder param optional (default empty string)

In handle_bg_delete_file() (line 768):

Make folder param optional (default empty string)

In handle_bg_read_file() (line 862):

Make folder param optional (default empty string)

In handle_bg_move_file() (line 807):

Make src_folder and dst_folder params optional (default empty string)

Dependencies: Step 1, Step 2

Step 6: Frontend — collapse behavior, root files, root upload

File: crates/hero_slides_ui/static/js/dashboard.js

Collapse by default: change line 17 let bgPanelOpen = true to let bgPanelOpen = false
Auto-open when content exists: in loadSlidesForDeck() (around line 401), reset bgPanelOpen = false before calling loadBgPanel() so switching decks re-evaluates. In loadBgPanel() after fetching data, if folders.length > 0 || rootFiles.length > 0, set bgPanelOpen = true and update DOM visibility.
Root files state: add let bgRootFiles = [] near line 18
Store root files: in loadBgPanel(), extract result.root_files || [] and store in bgRootFiles
Render root files: update renderBgTree() to render bgRootFiles at the top, before folders. Use a virtual root section with a simple flat list (no folder row, no expand). File key format: file:/:filename (root folder is /).
Root upload:
- In bgTriggerUpload(): remove the "Create a folder first" guard; when bgFolders.length === 0, set bgUploadTargetFolder = '' (root) and trigger file input
- In bgHandleFileInput(): allow folder = '' as valid root target
- In bgUploadFiles(files, folder): pass folder as-is to bg.uploadFile (empty string is valid)
Verify toggle: the bg-panel-toggle div already has only the folder icon — no "BG" text — no HTML change needed

Dependencies: Step 5

Acceptance Criteria

Uploading .png or .jpg via the UI succeeds and the file appears in the file list
Panel starts collapsed when deck has no background content
Panel auto-opens when deck has folders or root files
Collapsed toggle shows only the folder icon (no text label)
A .md file placed directly in content/background/ appears in the UI root section
Root-level .md/.txt files are included in AI generation text context
Background image files are sent as multimodal input to text content generation
cargo build and cargo test pass

Notes

Message::user_with_images(text, &[(mime, base64)]) is already used in extract_pdf_with_ai — same pattern applies
Step 1 and Step 2 are independent and can be implemented in parallel
Step 4 requires Step 3 (deck.rs callers must collect images before passing them)
Step 5 depends on Steps 1 and 2 being in place (uses the same empty-folder path logic)
Step 6 depends on Step 5 (needs root_files in the RPC response)

## Implementation Spec — Issue #7 ### Objective Fix five usability issues in the background panel: image upload rejection, root-level file invisibility, AI context gaps, and collapse behavior. ### Scope This spec covers all goals from the issue including the expanded multimodal scope from comment #2, with one exception: passing background images as *visual input to slide image generation* is deferred — the current `herolib_ai` API does not expose a path to send input images alongside image-output modality requests without triggering an `EmptyResponse` error in `send_request`. All other goals are implemented. ### Files to Modify | File | Changes | |------|--------| | `crates/hero_slides_lib/src/deck.rs` | Fix ext filter (L213); support empty folder in file ops (L160–L352) | | `crates/hero_slides_lib/src/discovery.rs` | New `find_background_root_files()`; new `read_background_root_content()`; new `collect_background_images()`; update `has_background` (L271); update `deck_info_from_path()` | | `crates/hero_slides_lib/src/generator.rs` | Add `background_images` param to `generate_slide_content` and `instruct_slide_content`; use `Message::user_with_images` when images present | | `crates/hero_slides_lib/src/lib.rs` | Re-export new functions | | `crates/hero_slides_server/src/rpc.rs` | Return `root_files` in `handle_bg_list_folders`; allow empty folder string in upload/list/delete/read handlers | | `crates/hero_slides_ui/static/js/dashboard.js` | Collapse-by-default; auto-open when content exists; root file rendering; upload to root | ### Implementation Plan #### Step 1: Fix image upload backend rejection **File:** `crates/hero_slides_lib/src/deck.rs` - Line 213: change extension check from `ext != "md" && ext != "pdf" && ext != "txt"` to also allow `png`, `jpg`, `jpeg`, `webp`, `gif` - Update the error message accordingly - Lines 160–196 (`deck_delete_background_file`, `deck_move_background_file`): when `folder_name` is empty, construct path directly under `content/background/` (skip the folder segment in `join()`) - Lines 203–229 (`deck_save_background_file`): when `folder_name` is empty, save directly to `content/background/`; PDF extraction path stays subfolder-only (only apply `deck_extract_pdf_to_md` when `folder_name` is non-empty) - Lines 341–352 (`deck_read_background_file`): when `folder_name` is empty, read from `content/background/` root **Dependencies:** none #### Step 2: Root-level file discovery and AI text context **File:** `crates/hero_slides_lib/src/discovery.rs` Add `find_background_root_files(deck_path: &Path) -> Vec<BackgroundFile>`: - Scan `content/background/` for immediate children that are *files* (not dirs) - Accept extensions: `md`, `pdf`, `txt`, `png`, `jpg`, `jpeg`, `webp`, `gif` - Return sorted alphabetically Add `read_background_root_content(deck_path: &Path) -> String`: - Read all `.md` and `.txt` files from `content/background/` root (skip PDFs, images) - Return concatenated content prefixed with `## (root)\n\n` - Return empty string if none Add `collect_background_images(deck_path: &Path) -> Vec<(String, String)>`: - Scan all image files (`png`, `jpg`, `jpeg`, `webp`, `gif`) from: - `content/background/` root - All immediate subdirectories of `content/background/` - For each image: read bytes, base64-encode, determine MIME type (`image/png`, `image/jpeg`, `image/webp`, `image/gif`) - Return `Vec<(mime_type, base64_data)>` Update `deck_info_from_path()` line 271: - `has_background` should be true if `find_background_folders()` is non-empty OR `find_background_root_files()` is non-empty **Dependencies:** none (independent of Step 1) #### Step 3: Include root files and images in AI generation context **File:** `crates/hero_slides_lib/src/deck.rs` In `deck_generate()` (lines 584–590): - After collecting `bg_folders` and `background_context`, also call `read_background_root_content()` and prepend it to `background_context` if non-empty - After that, call `collect_background_images()` and store as `background_images: Vec<(String, String)>` - Pass `&background_images` to `generate_slide()` In `slide_generate()` (lines 735–740): same additions **Dependencies:** Step 2 #### Step 4: Multimodal images in text content generation **File:** `crates/hero_slides_lib/src/generator.rs` Update `generate_slide_content()` (line 138): - Add parameter `background_images: &[(String, String)]` - When `background_images` is non-empty, build message as `Message::user_with_images(&user_msg, background_images)` instead of `Message::user(&user_msg)` Update `instruct_slide_content()` (line 189): - Add parameter `background_images: &[(String, String)]` - Same pattern as above Update callers in `rpc.rs` that call these functions — pass `&[]` initially (images from deck context not wired through RPC yet; can be extended later) Update callers in `deck.rs` to pass the collected `background_images` Note: `generate_slide()` (image generation) does NOT get images passed — deferred as noted in Scope. **Dependencies:** Step 3 #### Step 5: RPC layer — root files and empty folder support **File:** `crates/hero_slides_server/src/rpc.rs` In `handle_bg_list_folders()` (line 672): - After fetching folders, also call `hero_slides_lib::find_background_root_files(&dir)` (re-export needed) - Add `root_files: [{ name, extension, size_bytes }]` array to the JSON response alongside `folders` In `handle_bg_list_files()` (line 698): - Make `folder` param optional (default empty string) - Call `deck_list_background_files(&dir, &folder)` — Step 1 makes this work with empty folder In `handle_bg_upload_file()` (line 825): - Make `folder` param optional (default empty string) In `handle_bg_delete_file()` (line 768): - Make `folder` param optional (default empty string) In `handle_bg_read_file()` (line 862): - Make `folder` param optional (default empty string) In `handle_bg_move_file()` (line 807): - Make `src_folder` and `dst_folder` params optional (default empty string) **Dependencies:** Step 1, Step 2 #### Step 6: Frontend — collapse behavior, root files, root upload **File:** `crates/hero_slides_ui/static/js/dashboard.js` 1. **Collapse by default**: change line 17 `let bgPanelOpen = true` to `let bgPanelOpen = false` 2. **Auto-open when content exists**: in `loadSlidesForDeck()` (around line 401), reset `bgPanelOpen = false` before calling `loadBgPanel()` so switching decks re-evaluates. In `loadBgPanel()` after fetching data, if `folders.length > 0 || rootFiles.length > 0`, set `bgPanelOpen = true` and update DOM visibility. 3. **Root files state**: add `let bgRootFiles = []` near line 18 4. **Store root files**: in `loadBgPanel()`, extract `result.root_files || []` and store in `bgRootFiles` 5. **Render root files**: update `renderBgTree()` to render `bgRootFiles` at the top, before folders. Use a virtual root section with a simple flat list (no folder row, no expand). File key format: `file:/:filename` (root folder is `/`). 6. **Root upload**: - In `bgTriggerUpload()`: remove the "Create a folder first" guard; when `bgFolders.length === 0`, set `bgUploadTargetFolder = ''` (root) and trigger file input - In `bgHandleFileInput()`: allow `folder = ''` as valid root target - In `bgUploadFiles(files, folder)`: pass `folder` as-is to `bg.uploadFile` (empty string is valid) 7. **Verify toggle**: the `bg-panel-toggle` div already has only the folder icon — no "BG" text — no HTML change needed **Dependencies:** Step 5 ### Acceptance Criteria - [ ] Uploading `.png` or `.jpg` via the UI succeeds and the file appears in the file list - [ ] Panel starts collapsed when deck has no background content - [ ] Panel auto-opens when deck has folders or root files - [ ] Collapsed toggle shows only the folder icon (no text label) - [ ] A `.md` file placed directly in `content/background/` appears in the UI root section - [ ] Root-level `.md`/`.txt` files are included in AI generation text context - [ ] Background image files are sent as multimodal input to text content generation - [ ] `cargo build` and `cargo test` pass ### Notes - `Message::user_with_images(text, &[(mime, base64)])` is already used in `extract_pdf_with_ai` — same pattern applies - Step 1 and Step 2 are independent and can be implemented in parallel - Step 4 requires Step 3 (deck.rs callers must collect images before passing them) - Step 5 depends on Steps 1 and 2 being in place (uses the same empty-folder path logic) - Step 6 depends on Step 5 (needs `root_files` in the RPC response)

casper-stevens commented

2026-04-14 22:47:13 +00:00

Author

Member

Spec amendment — image generation step added

The "passing background images to slide image generation" step is included in scope. The approach: when background images are present, build a ChatCompletionRequest manually with Message::user_with_images + with_modalities(["image","text"]) + image config, call client.chat_raw(Provider::OpenRouter, request), and extract the image from choices[0].message.images. Falls back to the text-only path if no images are present or on error.

**Spec amendment — image generation step added** The "passing background images to slide image generation" step is included in scope. The approach: when background images are present, build a `ChatCompletionRequest` manually with `Message::user_with_images` + `with_modalities(["image","text"])` + image config, call `client.chat_raw(Provider::OpenRouter, request)`, and extract the image from `choices[0].message.images`. Falls back to the text-only path if no images are present or on error.

casper-stevens commented

2026-04-14 23:00:02 +00:00

Author

Member

Test Results

Total: 59
Passed: 59
Failed: 0
Ignored: 1 (AI integration test — requires live API key)

cargo build and cargo test both pass cleanly. One pre-existing warning about unused imports in hero_slides_ui/src/routes.rs (unrelated to this change).

## Test Results - Total: 59 - Passed: 59 - Failed: 0 - Ignored: 1 (AI integration test — requires live API key) `cargo build` and `cargo test` both pass cleanly. One pre-existing warning about unused imports in `hero_slides_ui/src/routes.rs` (unrelated to this change).

casper-stevens commented

2026-04-14 23:00:20 +00:00

Author

Member

Implementation Summary

All goals from the issue (including the expanded multimodal scope) are implemented.

Changes

crates/hero_slides_lib/src/deck.rs

Extension whitelist in deck_save_background_file now includes png, jpg, jpeg, webp, gif
All file operations (save, delete, move, read, list) accept an empty folder_name to operate at the content/background/ root

crates/hero_slides_lib/src/discovery.rs

find_background_files: handles empty folder name (lists root files)
New find_background_root_files(): lists files directly in content/background/
New read_background_root_content(): reads .md/.txt files from the background root for text context
New collect_background_images(): collects all image files from root and subfolders as (mime, base64) tuples
has_background in deck_info_from_path now returns true when root files exist

crates/hero_slides_lib/src/generator.rs

generate_slide_content and instruct_slide_content: new background_images parameter; uses Message::user_with_images when images are present
generate_slide: new background_images parameter; when images are present, uses ChatCompletionRequest + chat_raw with with_modalities(["image","text"]) for multimodal image generation; falls back to generate_image_to_file when empty

crates/hero_slides_lib/src/lib.rs

Re-exported new discovery functions

crates/hero_slides_rhai/src/deck_module.rs and crates/hero_slides_server/src/agent.rs

Updated callers of generate_slide_content and instruct_slide_content to pass collected background images

crates/hero_slides_server/src/rpc.rs

bg.listFolders response now includes root_files array
All file handlers (listFiles, uploadFile, deleteFile, readFile, moveFile) now accept empty folder string for root operations

crates/hero_slides_ui/static/js/dashboard.js

Panel defaults to collapsed (bgPanelOpen = false); auto-opens when deck has folders or root files
Panel state resets on deck switch
Root files rendered in the tree above folders
Upload to root enabled when no folders exist (no longer requires creating a folder first)

Deferred

None — all original and expanded goals are implemented.

## Implementation Summary All goals from the issue (including the expanded multimodal scope) are implemented. ### Changes **`crates/hero_slides_lib/src/deck.rs`** - Extension whitelist in `deck_save_background_file` now includes `png`, `jpg`, `jpeg`, `webp`, `gif` - All file operations (`save`, `delete`, `move`, `read`, `list`) accept an empty `folder_name` to operate at the `content/background/` root **`crates/hero_slides_lib/src/discovery.rs`** - `find_background_files`: handles empty folder name (lists root files) - New `find_background_root_files()`: lists files directly in `content/background/` - New `read_background_root_content()`: reads `.md`/`.txt` files from the background root for text context - New `collect_background_images()`: collects all image files from root and subfolders as `(mime, base64)` tuples - `has_background` in `deck_info_from_path` now returns `true` when root files exist **`crates/hero_slides_lib/src/generator.rs`** - `generate_slide_content` and `instruct_slide_content`: new `background_images` parameter; uses `Message::user_with_images` when images are present - `generate_slide`: new `background_images` parameter; when images are present, uses `ChatCompletionRequest` + `chat_raw` with `with_modalities(["image","text"])` for multimodal image generation; falls back to `generate_image_to_file` when empty **`crates/hero_slides_lib/src/lib.rs`** - Re-exported new discovery functions **`crates/hero_slides_rhai/src/deck_module.rs`** and **`crates/hero_slides_server/src/agent.rs`** - Updated callers of `generate_slide_content` and `instruct_slide_content` to pass collected background images **`crates/hero_slides_server/src/rpc.rs`** - `bg.listFolders` response now includes `root_files` array - All file handlers (`listFiles`, `uploadFile`, `deleteFile`, `readFile`, `moveFile`) now accept empty folder string for root operations **`crates/hero_slides_ui/static/js/dashboard.js`** - Panel defaults to collapsed (`bgPanelOpen = false`); auto-opens when deck has folders or root files - Panel state resets on deck switch - Root files rendered in the tree above folders - Upload to root enabled when no folders exist (no longer requires creating a folder first) ### Deferred None — all original and expanded goals are implemented.