Skip to content

Add a Review tab for bulk annotation auditing as a chip grid - #1897

Draft
mattdawkins wants to merge 3 commits into
mainfrom
dev/review-grid
Draft

Add a Review tab for bulk annotation auditing as a chip grid#1897
mattdawkins wants to merge 3 commits into
mainfrom
dev/review-grid

Conversation

@mattdawkins

Copy link
Copy Markdown
Member

Adds a top-level Review tab (desktop: right of Training; web: right of Models) that shows many annotations at once as a grid of cropped image chips, so a whole class, or everything carrying an attribute, can be audited across one or more datasets and wrong types corrected in place without opening each sequence in the viewer.

What it does

  • Datasets / Grid toggle at the top left; only one view is shown at a time. The Datasets view adds datasets through the platform picker (web) or project listing (desktop), shows load state and track counts, and can reload or drop a dataset. Multicamera parents expand into their cameras. Library (desktop) and the Data page (web) get a Review button for the current selection, and /review?datasetIds=a,b deep links work on both.
  • Grid view: a rows × columns grid (5 × 4 by default; settable, plus zoom in/out that keeps the shape) and a Context slider for the margin around each box (30% by default). Settings persist per browser. Arrow keys / PageUp / PageDown / Home / End page through the results.
  • Query by type at or above a confidence threshold ("Any type" included), or by attribute key / value over track and/or detection attributes. Results can be sorted by dataset + track id, confidence (either direction), or frame. The grid keeps its entries until Show is pressed again, so editing a type never reshuffles the page being worked on.
  • Chips are cropped client-side: image sequences load their frame directly, videos are decoded by a hidden <video> element seeking to the frame (the frame → time mapping is extracted from VideoAnnotator into videoSeek.ts and shared). A concurrency-limited queue renders the first box of every visible entry before any track's extra frames; track entries then cycle through up to 8 boxes sampled along the track with the object kept centred. Chips are rendered at the cell's resolution and aspect ratio.
  • Editing: each cell has a type field (with every seen type as suggestions) and a "mark correct" action; Page actions sets a type for, or accepts, everything on the page. Edits use the same reassignPairs / acceptPairAsCorrect semantics as the viewer, respecting the dataset's type hierarchy. Edits are batched and written with saveDetections on Save; unsaved edits are counted, Discard reloads, and navigating away with unsaved edits prompts.
  • Open in viewer: clicking a chip opens the annotation viewer on that dataset, seeks to the frame and selects the track. The viewer gains initialFrame / initialTrackId props, read from ?frame=&track= on both platforms.

Plumbing

  • Api.peekConfig? (optional): loadConfig without the platform's viewer bookkeeping (desktop recents, web browse location). The web dataset store now shares its config merge (mergeDatasetConfig) with it.
  • Shared code lives in dive-common/review/* (pure item building, chip cropping, frame sources, chip queue), dive-common/use/useReview.ts (service) and dive-common/components/Review/*; platform shells are thin (ReviewPage.vue on desktop, views/Review.vue on web).
  • ReviewCell / ReviewGrid are presentation-only with action / footer slots so other item sources can reuse them; this is the hook a follow-up PR uses to show video-search results (Add optional video search capabilities to DIVE [DRAFT] #1743) in the same panel.
  • Docs: docs/Review.md, linked from the user guide nav.

Testing

  • Unit tests for item building, sorting, vocabularies, chip regions, viewer deep links and the review service (load, multicam expansion, assign/accept, save, failed save, discard, errors).
  • npm run lint, npm run typecheck, npm test and vite build pass.
  • Desktop dev build exercised by hand: image-sequence, video and multicam projects loaded together, chips and track cycling rendered, zoom, a type edit saved (verified in the written result_*.json, then reverted), and chip click → viewer at the right frame with the track selected.
  • Not yet exercised against a running Girder server; the web side is covered by the shared code, typecheck and the build only.

Known limitations

  • Tiled large-image datasets load but their chips cannot be cropped yet (placeholder shown).
  • Web uses the dataset's default annotation set.
  • Type edits only; boxes, attributes and deletion still happen in the viewer.

🤖 Generated with Claude Code

https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu

mattdawkins and others added 3 commits September 9, 2026 01:23
New top-level Review tab (desktop: after Training; web: after Models)
showing many annotations at once as cropped image chips, so a whole class
(or everything carrying an attribute) can be checked across several
datasets and wrong types corrected in place, without opening each
sequence in the viewer.

- Datasets view / Grid view toggle: pick datasets through the platform
  picker or listing (multicam parents expand into their cameras), then page
  through matching annotations in a rows x columns grid (5x4 by default,
  settable, with zoom in/out keeping the shape) and a context slider for
  the margin around each box (30% by default).
- Query by type above a confidence threshold, or by attribute key/value on
  tracks and/or detections; sort by dataset, confidence or frame. The grid
  keeps its entries until the query is re-run so edits never reshuffle it.
- Chips are cropped client-side from image sequences or by seeking a hidden
  video element (shared frame->time mapping extracted from VideoAnnotator
  into videoSeek.ts), through a concurrency-limited queue that renders the
  first box of every visible entry before any track's extra frames. Track
  entries then cycle through up to 8 boxes sampled along the track, with
  the object kept centred.
- Types are edited per cell (assign, or mark correct) or for a whole page,
  batched and saved through saveDetections; unsaved edits are counted and
  guarded on navigation.
- Clicking a chip opens the viewer on that dataset, seeking to the frame
  and selecting the track (new initialFrame / initialTrackId Viewer props
  read from the route query on both platforms).
- Api gains an optional peekConfig for reading dataset configs without the
  viewer bookkeeping (desktop recents, web browse location); the web
  dataset store now shares its config merge with it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
A square chip letterboxed into a wide cell wasted most of the cell, so the
crop region now extends the padded square around the box to the cell's
width/height ratio (coarsened to a tenth so window resizes rarely force a
re-render), keeping the object centred whatever the grid shape.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
Move the review page's paging, zoom, chip-resolution and keyboard handling
into useReviewGrid, its grid-shape/context/pager row into
ReviewGridControls, and the persisted grid settings into gridSettings.ts,
so another view showing ReviewItems (e.g. video search results) gets the
same behaviour without copying the page.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1C4QY6hxjHaUPJfWPfQuu
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant