Skip to content

feat: add Content Note bulk upload to simply-core and simply-data-core - #181

Merged
ClayChipps merged 2 commits into
mainfrom
feat/content-note-upload
Sep 3, 2026
Merged

feat: add Content Note bulk upload to simply-core and simply-data-core#181
ClayChipps merged 2 commits into
mainfrom
feat/content-note-upload

Conversation

@ClayChipps

Copy link
Copy Markdown
Contributor

Summary

Implements docs/design/0034, converting the scraped UploadContentNotes.md script into library code:

  • simply-core: mapConcurrent() — a streaming worker-pool primitive that bounds concurrency over an AsyncIterable/Iterable source, starting the next item as soon as a slot frees rather than requiring the whole input in memory or waiting for a whole chunk (unlike the existing mapChunked).
  • simply-data-core: createContentNote() / uploadContentNotes() — create a ContentNote and link it to a record via ContentDocumentLink, with uploadContentNotes driving mapConcurrent over a stream of inputs resolved against a caller-supplied external-id lookup Map.

Fixes several bugs found in the original script along the way:

  • CreatedDate was sent as a numeric epoch instead of an ISO-8601 string.
  • An undefined-variable reference in the ContentDocumentLink failure branch (errorWriteStreamWriterStream) would crash the run.
  • That branch also mislabeled which create call actually failed and dropped the already-created ContentNote's id, losing track of an orphaned note. createContentNote now reports stage: 'link' with contentNoteId still populated.
  • Debug console.log of every note's base64 body is gone.

Drops the script's CLI-only concerns (arg parsing, spinner, Org.create(), --output-dir/--use-cached file management) — the library takes an already-resolved Connection and streams typed results back, matching every other -core package here. A CLI command, if wanted, is left as a follow-up per the design doc's open questions.

Also updates docs/design/0034-content-note-upload.md's Status line and corrects two details the implementation clarified: mapConcurrent's source type is AsyncIterable<T> | Iterable<T> (broader than originally drafted), and it propagates a mapper rejection immediately (matching mapChunked's existing convention) rather than the self-contradictory "catches it and yields it" wording in the original draft.

Note: pnpm-lock.yaml's diff is large but is a formatting-only normalization to the pinned pnpm@11.21.0's canonical output, produced by pnpm add @simplysf/simply-core --filter @simplysf/simply-data-core (per CONTRIBUTING.md) — not a dependency version change beyond that one new workspace link.

Test plan

  • pnpm run compile / lint / test:only pass for both simply-core and simply-data-core.
  • lerna run build/test --scope=@simplysf/simply-core --scope=@simplysf/simply-data-core pass.
  • Pre-push hook additionally ran tests for simply-core's dependents (simply-apex-core, simply-community-core, simply-package-core, simply-permissions-core) — all pass, confirming the new export doesn't affect them.
  • test/index.test.ts export-pinning updated in simply-data-core.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ASL1LuYtGCPVDcqtqaNvLy

ClayChipps and others added 2 commits September 3, 2026 09:43
Converts the scraped UploadContentNotes.md script into library code per
docs/design/0034:

- simply-core: mapConcurrent(), a streaming worker-pool primitive that
  bounds concurrency over an AsyncIterable/Iterable source without
  requiring the whole input in memory (unlike mapChunked).
- simply-data-core: createContentNote()/uploadContentNotes(), which
  create a ContentNote and link it to a record via ContentDocumentLink.

Fixes several bugs found in the original script: CreatedDate sent as a
numeric epoch instead of ISO-8601, an undefined-variable crash on a
ContentDocumentLink failure, a mislabeled error, and an orphaned
ContentNote losing its id when linking fails (now reported via
stage: 'link' with contentNoteId still populated).

Drops the script's CLI-only concerns (arg parsing, spinner, Org.create,
--output-dir/--use-cached file management) — the library takes an
already-resolved Connection and streams typed results back, consistent
with every other -core package here.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ASL1LuYtGCPVDcqtqaNvLy
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ASL1LuYtGCPVDcqtqaNvLy
@ClayChipps
ClayChipps merged commit 2696820 into main Sep 3, 2026
9 checks passed
@ClayChipps
ClayChipps deleted the feat/content-note-upload branch September 3, 2026 13:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant