feat: add Content Note bulk upload to simply-core and simply-data-core - #181
Merged
Conversation
Converts the scraped UploadContentNotes.md script into library code per docs/design/0034: - simply-core: mapConcurrent(), a streaming worker-pool primitive that bounds concurrency over an AsyncIterable/Iterable source without requiring the whole input in memory (unlike mapChunked). - simply-data-core: createContentNote()/uploadContentNotes(), which create a ContentNote and link it to a record via ContentDocumentLink. Fixes several bugs found in the original script: CreatedDate sent as a numeric epoch instead of ISO-8601, an undefined-variable crash on a ContentDocumentLink failure, a mislabeled error, and an orphaned ContentNote losing its id when linking fails (now reported via stage: 'link' with contentNoteId still populated). Drops the script's CLI-only concerns (arg parsing, spinner, Org.create, --output-dir/--use-cached file management) — the library takes an already-resolved Connection and streams typed results back, consistent with every other -core package here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASL1LuYtGCPVDcqtqaNvLy
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASL1LuYtGCPVDcqtqaNvLy
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements docs/design/0034, converting the scraped
UploadContentNotes.mdscript into library code:simply-core:mapConcurrent()— a streaming worker-pool primitive that bounds concurrency over anAsyncIterable/Iterablesource, starting the next item as soon as a slot frees rather than requiring the whole input in memory or waiting for a whole chunk (unlike the existingmapChunked).simply-data-core:createContentNote()/uploadContentNotes()— create aContentNoteand link it to a record viaContentDocumentLink, withuploadContentNotesdrivingmapConcurrentover a stream of inputs resolved against a caller-supplied external-id lookupMap.Fixes several bugs found in the original script along the way:
CreatedDatewas sent as a numeric epoch instead of an ISO-8601 string.ContentDocumentLinkfailure branch (errorWriteStreamWriterStream) would crash the run.ContentNote's id, losing track of an orphaned note.createContentNotenow reportsstage: 'link'withcontentNoteIdstill populated.console.logof every note's base64 body is gone.Drops the script's CLI-only concerns (arg parsing, spinner,
Org.create(),--output-dir/--use-cachedfile management) — the library takes an already-resolvedConnectionand streams typed results back, matching every other-corepackage here. A CLI command, if wanted, is left as a follow-up per the design doc's open questions.Also updates
docs/design/0034-content-note-upload.md'sStatusline and corrects two details the implementation clarified:mapConcurrent's source type isAsyncIterable<T> | Iterable<T>(broader than originally drafted), and it propagates amapperrejection immediately (matchingmapChunked's existing convention) rather than the self-contradictory "catches it and yields it" wording in the original draft.Note:
pnpm-lock.yaml's diff is large but is a formatting-only normalization to the pinnedpnpm@11.21.0's canonical output, produced bypnpm add @simplysf/simply-core --filter @simplysf/simply-data-core(perCONTRIBUTING.md) — not a dependency version change beyond that one new workspace link.Test plan
pnpm run compile/lint/test:onlypass for bothsimply-coreandsimply-data-core.lerna run build/test --scope=@simplysf/simply-core --scope=@simplysf/simply-data-corepass.simply-core's dependents (simply-apex-core,simply-community-core,simply-package-core,simply-permissions-core) — all pass, confirming the new export doesn't affect them.test/index.test.tsexport-pinning updated insimply-data-core.🤖 Generated with Claude Code
https://claude.ai/code/session_01ASL1LuYtGCPVDcqtqaNvLy