Skip to content

docs: state the rule that makes the media field decode safe - #688

Open
jkmassel wants to merge 1 commit into
fix/media-uploader-credentials-trapfrom
docs/media-field-decode-invariant
Open

jkmassel wants to merge 1 commit into
fix/media-uploader-credentials-trapfrom
docs/media-field-decode-invariant

Conversation

@jkmassel

@jkmassel jkmassel commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Stacked on #687. Last of nine PRs splitting #621.

What?

Writes down why decoding the editor's form fields as UTF-8 can't corrupt them, and adds tests for the one condition this code could break on its own.

Why?

formFields reads every non-file form value as UTF-8. UTF-8 decoding doesn't fail on bad bytes — it quietly swaps in a replacement character. So if bad bytes ever did arrive, a caption would come back mangled and nothing would report an error.

That can't happen today, but only because of who is on the other end, and nothing in the code says so. Three things have to stay true:

  1. Only the editor's own page can reach this server. It only accepts connections from the device itself, and every request has to carry a per-session token.
  2. Text the editor puts in a form field is already valid Unicode. The browser guarantees that when the value is set.
  3. The only way a browser can put raw bytes in a form is a file or a Blob, and those always arrive with a filename. Anything with a filename is handled as the file, never as a field.

If one of those stops being true, iOS and Android mangle the value differently, so there's no single behaviour that could be documented in place of the rule. Turning off each platform's filename check shows it: the same three bad bytes (ED A0 80) come back as three replacement characters on iOS and one on Android. Both measured, not guessed.

How?

  • The three conditions, as a comment on formFields, on both platforms.
  • Rewrote the comment where the request gets rebuilt — the path used when a processor has changed the file. It said the field bytes were appended "so a non-UTF-8 value is forwarded verbatim rather than coerced", which reads as though bad values were expected there. They can't get there. On iOS the real reason is that String(data:encoding:) returns nil on bad UTF-8, and the ?? "" you'd reach for behind it would throw away the whole field. On both platforms, using the raw bytes keeps this path sending exactly the same request as the plain pass-through it stands in for, so an upload doesn't change shape just because the image got resized.

Testing Instructions

The tests cover condition 3, the only one this code controls. A request carrying a second part that looks like a Blob — it has a filename, and its bytes aren't valid UTF-8 — must not show up in fields, and must not get uploaded in place of the real file.

Both test bodies put file first and the other fields after, which is the order the editor actually sends them. Keep it that way. The server treats the first part that has a filename as the file, so moving the Blob ahead of file makes the Blob the upload and throws the real file away — while the check on fields still passes, so nothing catches it.

The Blob part reaches neither fields nor the request sent on to WordPress; it is dropped. This PR doesn't change that.

  • The Blob test fails if the server stops excluding parts that have a filename — both platforms
  • It also fails if the server takes the last part with a filename instead of the first. The check on the uploaded filename is what catches that; without it, the test still passed under that change
  • swift test — 396 tests green
  • :Gutenberg:testDebugUnitTest — 30 tests in MediaUploadServerTest green
  • SwiftLint clean

@wpmobilebot

wpmobilebot commented Sep 17, 2026

Copy link
Copy Markdown

XCFramework Build

This PR's XCFramework is available for testing. Add the following to your Package.swift:

.package(url: "https://github.com/wordpress-mobile/GutenbergKit", branch: "pr-build/688")

Built from 5620d2b

@jkmassel
jkmassel force-pushed the docs/media-field-decode-invariant branch 2 times, most recently from 9e30369 to 2e73cda Compare September 17, 2026 21:44
`formFields` decodes every non-file form part as UTF-8. That can only be
lossless because of who is on the other end, and nothing in the code enforces
it -- so write it down on both platforms, in plain terms: only the editor's own
page can reach the server, the browser guarantees form-field text is valid
Unicode, and raw bytes always arrive carrying a filename, which routes them to
the file rather than to a field.

Pin the third condition with tests, since it is the one this code could break on
its own. The bodies are ordered the way `uploadToServer` actually emits them --
file first, then additionalData -- and assert both the uploaded filename and the
decoded fields, so the suite fails if either the partition or the file-selection
rule changes. A second test covers the other half: valid UTF-8 round-trips, so
real captions and titles are unaffected.

The `buildMultipart` comments also claimed raw bytes were appended so a non-UTF-8
value would be "forwarded verbatim rather than coerced". That is not why -- such
a value cannot reach them. On iOS they avoid a failable `String(data:encoding:)`
whose `?? ""` would quietly drop a whole field; on both platforms they keep the
re-encode byte-for-byte identical to the passthrough it replaces.
@jkmassel
jkmassel force-pushed the docs/media-field-decode-invariant branch from 2e73cda to 5620d2b Compare September 17, 2026 21:47
@jkmassel jkmassel changed the title docs: state the invariant that makes the media field decode safe docs: state the rule that makes the media field decode safe Sep 17, 2026
@jkmassel
jkmassel marked this pull request as ready for review September 17, 2026 22:00
@jkmassel
jkmassel requested a review from dcalhoun September 17, 2026 22:00

@dcalhoun dcalhoun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Captured a few inline suggestions from Claude, but none of them are blocking.

Comment on lines +473 to +475
/// 3. The only way a browser can put *raw* bytes in a form is a file or a Blob, and
/// those always arrive with a filename. Anything with a filename is handled as
/// the file, never as a field — so raw bytes never reach this decode.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finding from Claude:

Two predicates doing two different jobs on the same array:

  • :210parts.first(where: { $0.filename != nil }) picks the file: the first filename-bearing part.
  • :216parts.filter { $0.filename == nil } picks the fields: only the filename-less ones.

So a second filename-bearing part is neither. first has already returned, and the filter excludes it for having a filename — it falls out of the request entirely. (On this path, at least; the passthrough at :301 relays the original body verbatim, so it survives there.)

The rule's conclusion is still safe — :216 excludes every filename-bearing part, so raw bytes can't reach the decode. It's the stated reason that's off: "never as a field" holds for all of them, but "handled as the file" holds only for the first, and implies a second Blob gets processed as an upload when it's actually dropped.

Suggested change
/// 3. The only way a browser can put *raw* bytes in a form is a file or a Blob, and
/// those always arrive with a filename. Anything with a filename is handled as
/// the file, never as a field — so raw bytes never reach this decode.
/// 3. The only way a browser can put *raw* bytes in a form is a file or a Blob, and
/// those always arrive with a filename. Anything with a filename is excluded from
/// the fields, so raw bytes never reach this decode. (The first such part is the
/// file; on this path any others are dropped.)

Same wording on MediaUploadServer.kt:614-616.

// values are appended as raw bytes rather than through `String(data:encoding:)`, which
// returns nil on bad UTF-8 — and the `?? ""` you'd reach for behind it would quietly
// drop a whole field. Raw bytes also keep this re-encode byte-for-byte identical to
// the plain passthrough it replaces. (Bad bytes can't get here; see `formFields`.)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finding from Claude:

This says bad bytes can't reach multipartBodyStream, but multipartBodyPreservesNonUTF8FieldValue — "forwards a non-UTF-8 field value verbatim", MediaUploadServerTests.swift:836 — calls this function with exactly those bytes. The rationale this replaced was the text that test pointed at.

The risk is someone reads the new sentence and deletes the test as vacuous. Scoping it would keep both true, e.g. "Bad bytes can't reach here via the server; this function's own handling of them is pinned by multipartBodyPreservesNonUTF8FieldValue."

Same on MediaUploadServer.kt:719.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants