Video search on top of the Review grid (rebase of #1743) - #1898
Draft
mattdawkins wants to merge 7 commits into
Draft
Video search on top of the Review grid (rebase of #1743)#1898mattdawkins wants to merge 7 commits into
mattdawkins wants to merge 7 commits into
Conversation
Per-dataset search index lifecycle (build via a process_video.py job, status from an index_meta.json sidecar + ITQ file check, delete) and a persistent QueryServiceManager wrapping viame.core.query_service over NDJSON stdio (open index, formulate from image chip + boxes, query with optional warm-start model, refine with +/- feedback, export the SVM as a runnable trained pipeline in DIVE_Pipelines). One index open at a time; the embedded postgres is stopped gracefully on app cleanup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New in-viewer context panel (desktop only) for index-backed video search and rapid model generation: build/rebuild/delete the dataset's search index (via the GPU job queue), query from the selected annotation, an external image file, or a saved .svm model warm-start, adjudicate ranked results +/- with cropped chip thumbnails and click-to-seek, refine iteratively, and save good models as runnable trained pipelines. Shared request/response types live in apispec so web can implement later. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replace per-dataset index directories with a single shared search database (DIVE_SearchIndex): every table keys rows on a per-video stream identifier, so datasets are added, updated, and removed independently while one IQR session searches everything at once. - Add/Update ingests a dataset into the shared database as a job (initializing it on first use), re-using stream identifiers derived from the dataset id (image sequences) or video filename stem. - Remove deletes the dataset's rows through the query service and drops it from the membership metadata; deleting a dataset from DIVE also removes it from the index automatically. - Query results attribute back to their source dataset via stream_id; the panel labels cross-dataset results, seeks within the current dataset, crops thumbnails from each result's own media, and can filter the display to the current dataset. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
New fullscreen 5x4 grid page, opened from the Video Search panel, showing the ranked results across every indexed video as cropped chips with accept/reject buttons; marks share the panel's session state and feed the same IQR refine loop. Chip rendering moves to a shared, concurrency-limited loader (useResultChips) with context padding and the result box outlined, and the cell (AdjudicationChip) is presentation-only so future annotation cluster rows can reuse it. Also: extracted video frames are now cached per source video (hash-keyed) instead of colliding across videos on frame number, and cross-dataset chip media resolves the file the media server actually serves, so transcoded copies work when originals have moved.
Track results load their first-state chip immediately, then the grid queues up to 8 frames evenly sampled along the track (visible page only, at lower priority than primary chips) and each cell cycles through whichever frames have loaded, with a filmstrip badge marking animated cells. Static detection results are unchanged.
Follow renames that landed on main since the branch was cut: the desktop project store type is JsonConfig and is read with loadJsonConfig from the project's datasetFileAbsPath, the frontend metadata loader is loadConfig, the typed IPC invoke is used in the GPU job queue, and a spawn without a pid now fails the index build instead of storing an undefined pid. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
The full-window results grid is now the review chip grid: results map to ReviewItems (searchResultItems.ts), chips are cropped client-side through the review frame sources (a lazy per-dataset registry, so cross-dataset results need no backend frame extraction), and the grid shape, zoom, context and paging come from useReviewGrid / ReviewGridControls with the same persisted settings as the Review page. Cells keep the accept/reject actions through ReviewCell's slots and a highlight border; results without a box show the whole frame. useResultChips.ts and AdjudicationChip.vue are replaced by useSearchChips.ts, which also feeds the side panel's row thumbnails. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1897 (base branch
dev/review-grid). Rebases the video search / IQR work from #1743 onto it and shows search results in the new Review chip grid.Rebase
The five commits of #1743 apply on top of the review branch with these adaptations to changes that landed on main in the meantime (one follow-up commit):
JsonConfig, read withloadJsonConfigfromdatasetFileAbsPathloadConfiginvoke<DesktopJob>undefinedResults in the Review grid
VideoSearchResultsGridis rebuilt on the shared review components: results map toReviewItems (dive-common/review/searchResultItems.ts), and the grid usesReviewGrid/ReviewCell/ReviewGridControlsanduseReviewGrid, so grid shape, zoom, context margin, paging and keyboard behaviour are the same as the Review tab and share its persisted settings.createFrameSourceRegistry), so cross-dataset results load their own media (image sequence or hidden<video>seek) with no backend frame extraction for display. Results without a box show the whole frame.ReviewCell's actions slot, get a green / red highlight border, show the relevancy score as the badge, and clicking a chip from the open dataset seeks the viewer to it.useSearchChips.tsreplacesuseResultChips.tsandAdjudicationChip.vue; it also feeds the side panel's row thumbnails from the same chip store.docs/Review.mdgains a short "Search results" section.Testing
npm run lint,npm run typecheckandnpm testpass.🤖 Generated with Claude Code
https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu