Skip to content

Unify iolinks generation between first and incremental sync #142

Description

@elainefan331

Description:
iolinks is currently generated through two different paths:

  • First sync reads directly from the CouchDB _design/qq/_view/links view.
  • Incremental sync fetches the raw dataset document and rebuilds links locally using transformLinks().

The CouchDB links view and backend transformLinks() implement the same link-parsing logic separately. Their filename-parsing regexes have diverged, causing first sync and incremental sync to generate inconsistent iolinks records.

Tasks:

  • Add doc._id to the CouchDB links view key so the view can be queried by dataset ID during incremental sync.
  • Update the CouchDB design documents for all databases.
  • Use the CouchDB links view for incremental sync instead of transformLinks().

Acceptance Criteria:

  • First sync and incremental sync both generate iolinks from the CouchDB links view.
  • Incremental sync can retrieve links for a specific dataset using doc._id.
  • transformLinks() is no longer used for incremental iolinks generation.
  • Existing file-type search and matching-file functionality continue to work correctly.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions