Build the well-known-string table at compile time - #13559
Conversation
634f1de to
bf93e5e
Compare
|
[approve ci Autest 1] |
|
[approve ci autest 1] |
There was a problem hiding this comment.
Pull request overview
Warning
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
Moves well-known-string (WKS) construction from runtime to compile time using constexpr, enabling stronger correctness checks and reducing startup work / memory overhead.
Changes:
- Build the WKS table at compile time and populate runtime “hot” parallel arrays from it during
hdrtoken_init(). - Add compile-time validation (initializer/name resolution, hash-slot uniqueness) and shrink the hash table to the reachable slot space.
- Relocate Cache-Control cooked-mask bindings into WKS initialization and remove the MIME startup hook.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| src/proxy/hdrs/MIME.cc | Removes runtime Cache-Control mask initialization; updates WKS/prefix access to const. |
| src/proxy/hdrs/HdrToken.cc | Introduces compile-time WKS table build + checks; updates hashing/slot sizing; keeps runtime init lightweight. |
| include/proxy/hdrs/MIME.h | Removes declaration of the deleted Cache-Control init hook. |
| include/proxy/hdrs/HdrToken.h | Updates field initializer types and makes WKS prefix access read-only (const). |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
hdrtoken_init() built the well-known-string table at startup in an ats_calloc heap, and resolved initializer names through a PCRE2 DFA. Those patterns are start-anchored, so a name matched the first pattern it prefixed. That forced the array to keep the longer entry of every case-insensitive prefix pair at the lower index, which contradicts the rule that new strings must be appended because their indexes are stored on disk for cached objects. An appended name that an existing entry prefixes resolves to that entry and silently overwrites its slot id, presence mask and flags. Build the whole table during translation from a constexpr array of string_view, and resolve each initializer name with an exact case-insensitive comparison, so a prefix pair works in either order. static_assert now enforces what the startup checks did: every initializer name matches an entry, no two rows of a table claim the same entry, and no two strings share a hash slot. Both ink_release_assert range tests are gone, along with the collision scan and its abort(). The table is read-only, so the prefix accessors return const and the cooked Cache-Control masks move in from MIME.cc. The name field held a pointer into the table, which a constant may not do, and nothing read it, so it now lives only in the initializer row type. hdrtoken_hash_table halves as well: hash_to_slot() masks a hash to 15 bits, but the table held 65536 buckets, so half was unreachable.
bf93e5e to
9184ae1
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (2)
src/proxy/hdrs/HdrToken.cc:354
- This fingerprint uses the case-folding token hash, so it does not actually detect every edit that the frozen ledger promises to reject. For example, changing the frozen method
GETtogetleaves this fingerprint unchanged, but thenhdrtoken_method_tokenize()rejectsGETand acceptsget. Hash the original bytes (withouthdrtoken_ascii_toupper) for the ledger; token lookup can remain case-insensitive.
for (char const c : _hdrtoken_strs[i]) {
hval = hdrtoken_hash_step(hval, static_cast<unsigned char>(c));
}
hval = hdrtoken_hash_step(hval, '\0'); // fold in a terminator so entry boundaries matter
src/proxy/hdrs/HdrToken.cc:624
- These range endpoints no longer belong to one character array: every
HdrTokenWksEntry::stris a distinct array subobject.hdrtoken_is_wks()still classifies pointers with built-in>=/<=, whose ordering is unspecified across these subarrays, so a valid WKS pointer from an intermediate entry is not portably guaranteed to fall between these endpoints. Represent and compare the bounds asuintptr_tvalues (consistent withhdrtoken_wks_to_prefix()), or otherwise use a defined total pointer ordering.
const char *_hdrtoken_strs_heap_f = &hdrtoken_wks_table[0].str[0]; // storage first byte
const char *_hdrtoken_strs_heap_l = &hdrtoken_wks_table[std::size(_hdrtoken_strs) - 1].str[HDRTOKEN_WKS_STORAGE - 1];
Computing "wks - sizeof(HdrTokenHeapPrefix)" formed a pointer outside the entry's str array member, which is undefined behavior even though the prefix is physically adjacent. Recover the entry by index through integer arithmetic instead. Also enforce the append-only rule with a fingerprint of the frozen entries, validate the slot, presence-mask, and Cache-Control invariants of the initializer rows, and build the hash table at compile time, replacing pointer buckets with index buckets at half the size and removing hdrtoken_hash_init().
aa8a773 to
e3179c1
Compare
hdrtoken_init()builds the well-known-string (WKS) table at startup. This PR does the work at compile-time, which enables more robust checking, and removes an old restriction regarding prefixes.1. Modernize the well-known-string initialization
Make use of
constexprto initialize WKS at compile time. Remove the unnecessarynamefield from the WKS table at runtime. Trim thehdrtoken_hash_tablesize to just the reachable number of slots, build it at compile time as well, and shrink its buckets from 16 to 8 bytes by storing an index instead of a pointer..rodatahdrtoken_hash_table.rodatasizeof(HdrTokenHeapPrefix)2. Add compile-time checks for WKS correctness
Checks are now done at compile-time, and the new checks are more strict than the old release asserts.
An append-only ledger (
_hdrtoken_strs_frozen) fingerprints the frozen entries: inserting, reordering, or editing them fails to compile, and appending strings requires appending a matching ledger row.3. Remove the restriction that prevents a WKS from being added after its prefix
In order to maintain cache compatibility, new strings needed to be appended to the list. Previously, it was not legal to append a string to WKS if its prefix already exists.
This PR removes the prefix restriction. We can now append strings like
Server-Timing,Accept-CH, andExpect-CT.