initialize: close five holes in the transaction - #101
Open
jll63 wants to merge 5 commits into
Open
Conversation
|
An automated preview of the documentation is available at https://101.openmethod.prtest3.cppalliance.org/libs/openmethod/doc/html/index.html If more commits are pushed to the pull request, the docs will rebuild at the same URL. 2026-09-12 16:31:58 UTC |
boostorg#95 made initialize() transactional, but `transaction.commit()` was called one statement too early: the `++tr << "Installing\n"` that follows it goes through the `output` policy, which is user code and can throw. When it does, the transaction destructor sees a committed transaction and restores nothing, so the policies keep the v-table pointers they read out of `new_dispatch_data` - the local vector unwinding is about to free. The registry still holds the previous dispatch data (the swap never ran), so the next dispatch through the policy state reads freed memory: the exact use-after-free boostorg#95 fixed, reached by a different route. ASan, gcc 13, an `output` policy whose stream throws on "Installing" plus initialize(trace(true)): ERROR: AddressSanitizer: heap-use-after-free READ of size 8 ... in resolve_uni<...> freed by ... write_global_data() Move the trace write above commit(), where a throw rolls the policies back, and extract the writes that follow into `commit_global_data()`, declared `noexcept` so that a throwing statement added there terminates loudly instead of silently reopening this. The test grew a case for that path, and two of its existing assertions were vacuous: the failing initialize saw exactly the input the preceding successful one saw, so `fast_perfect_hash` - which re-seeds a fixed PRNG - recomputed identical factors, and `next<poke_dog>` resolved to the same overrider. Both held whether or not anything was rolled back; reintroducing the dangling-`next` half of boostorg#81 left the suite reporting "No errors detected". Perturb the input between the two calls instead, with two function-local static registrars: an extra class changes the hash factors, the control table and the v-table pointers, and an extra inheritance edge changes what `next<poke_dog>` would be set to. The hash assertion also compares the whole policy state now - factors and control table - rather than `hash_range()` alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RQG6CbE4o2agseE7bDVzHS
The transaction section says the registry is marked as not initialized and that initialize() must be called again, but not what happens if a method is called before it is. Only `runtime_checks` diagnoses that; otherwise the call dispatches through the previous tables, which is harmless for a registry whose classes are all still loaded and a use-after-free for one whose overriders came out of a library that has since been dlclose'd - the case the section is written for. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JQa4fuiwcfsheZYTCyfPPr
…hose boostorg#95 made initialize() transactional by copying the registry's whole `policies` tuple and putting it back if anything throws. The comment justified the width - "restoring an untouched state is harmless, and simpler than picking" - but it is not harmless, in two ways. A state that no `initialize` writes to is not derived data the call is about to replace; it is configuration the caller owns. The error handler is the case that bites: it is *called* from inside the transaction window by design, since fast_perfect_hash reports a search failure through it. A handler that disarms itself - installs another handler with set() before throwing, so the failure is reported once - had that undone on the way out. Reported here as a change to the handler surviving or not: before: handler still installed after the failed initialize = 1 after: handler still installed after the failed initialize = 0 And copying the tuple made copy-constructibility a hard requirement of every policy state in the registry, including states of policies that have no `initialize` at all. That rules out a state holding a std::mutex, std::unique_ptr, std::atomic or std::ostringstream - and an ostringstream is exactly what an `output` policy written to the documented state pattern holds. It compiled before boostorg#95 and stopped compiling after, with 29 lines of deleted-copy-constructor diagnostics on gcc 13 (207 on clang 18) naming the whole policy list 21 times. Filter the saved states to the policies whose `initialize` will actually run, using the same has_initialize test initialize_policy makes, so the two cannot disagree about which policies run. The saved tuple is now a subset of the registry's, so it is filled and restored element-wise. A static_assert spells out the requirement that remains, in the manner of the one preamble.hpp already carries for duplicate state types. One correction to how this was reported to me: it does *not* remove the copy of the control vector and the vptr vector from the success path. Those two policies define `initialize`, so their states must still be saved - that is what makes a rollback possible. What is no longer copied is the states of the policies that do not initialize: in default_registry, the error handler's std::function and the output policy's state. Not addressed: the restore is still a move-assignment in a `noexcept` destructor, so a policy state whose move-assignment throws still terminates. Stock registries are nothrow, so nothing in the suite can show it. The test covers both halves. It fails to compile without this change, because its configuration policy's state is deliberately move-only; with the state made copyable to isolate the other half, it fails 1 != 2 on the rolled-back generation counter. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JQa4fuiwcfsheZYTCyfPPr
…ow on restore Two more ways the transaction could be escaped, both found by the same review that prompted the previous commits. print(report) and print_slots() ran after write_global_data() had committed. Both can throw - the trace goes through the user-supplied `output` policy, and print_slots() builds an unordered_map and two vectors to partition the class graph - and a throw there landed past the commit: the new tables installed, the policies holding the new v-table pointers, and initialize() never reaching `st.initialized = true`. Nothing is corrupted, but it is a fourth outcome the exception-safety contract does not describe, and under runtime_checks the next call aborts with `not_initialized` even though the tables are perfectly good. It survived the previous commit, which moved only the "Installing" write that lives inside write_global_data(). Nothing printed there needs the new tables: the report is gathered during compile(), and the slots and lattices are compiler-local. So report first, install second, and a throw is once again just a failure the transaction rolls back. The last line of a successful trace is now "Installing", which is emitted immediately before the commit. The other is the restore itself. It runs from the transaction's destructor, while an exception is in flight, and a destructor is noexcept by default: a policy state whose move-assignment can throw would terminate the program, destroying the error the transaction exists to let through. That is reachable through the public API - a state holding a std::map with a stateful, non-always-equal allocator degrades to an element-wise move that allocates, and `vptr_map<MapFn>` lets a caller supply exactly that map. Every stock registry is nothrow move-assignable, so no test could ever have shown it. Refuse such a state at compile time, next to the copyability assertion, where the message can say why. Tests: a_throwing_report_does_not_commit trips the trace on "Used slots", which print_slots() writes, and checks the previous dispatch state is intact - it fails all three of those assertions without the reordering. The compile-fail test covers the assertion. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JQa4fuiwcfsheZYTCyfPPr
jll63
force-pushed
the
fix/initialize-commit-order
branch
from
September 12, 2026 13:57
69e2634 to
5ca43eb
Compare
Three places claimed that a failed initialize() leaves the previous dispatch state "complete and consistent", or that "the v-table pointers, `next` pointers and dispatch tables from the previous call all stay in place". Both are false when the preceding call was finalize(): that clears the dispatch data and every policy's state but leaves the classes' static_vptrs set - as documented on static_vptr, which stays valid only until the next initialize() *or finalize()*. After initialize -> finalize -> failed initialize the two halves disagree, and no call is possible until an initialize() succeeds. What the transaction actually guarantees is preservation, not consistency: the call leaves the state it found, whatever that was. Say that instead. The guarantee is unchanged - only the description was too strong. Adds the case to the test, which the suite did not cover: finalize appeared in it only after a failed initialize, never before one. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JQa4fuiwcfsheZYTCyfPPr
jll63
force-pushed
the
fix/initialize-commit-order
branch
from
September 12, 2026 16:25
c6b6584 to
35f4e39
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #101 +/- ##
===========================================
+ Coverage 93.46% 93.62% +0.15%
===========================================
Files 22 22
Lines 1653 1694 +41
Branches 500 505 +5
===========================================
+ Hits 1545 1586 +41
Misses 64 64
Partials 44 44
... and 3 files with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
(Written by Claude Code, on behalf of @jll63.)
Four ways the transaction #95 introduced could be escaped. Rebased onto current
develop, and it now absorbs #105, which is closed in favour of this branch.1. The commit happened one statement too early
transaction.commit()ran before++tr << "Installing\n", which goes through the user-suppliedoutputpolicy and can throw. When it did, the destructor saw a committed transaction and restored nothing, so the policies kept the v-table pointers they had read out ofnew_dispatch_data— the local vector unwinding was about to free. The registry still held the previous dispatch data, so the next dispatch read freed memory: the exact use-after-free #95 fixed, reached by another route.The trace write moves above
commit(), and the writes that follow are extracted intocommit_global_data(), declarednoexceptso a throwing statement added there terminates loudly instead of silently reopening this.2. The reporting ran after the commit
print(report)andprint_slots()ran afterwrite_global_data()had committed. Both can throw — the trace goes through theoutputpolicy, andprint_slots()builds anunordered_mapand two vectors to partition the class graph. A throw there landed past the commit: new tables installed, policies holding the new pointers, andinitialize()never reachingst.initialized = true. Nothing is corrupted, but it is a fourth outcome the contract does not describe, and underruntime_checksthe next call aborts withnot_initializedeven though the tables are good. This survived fix 1, which moved only the write insidewrite_global_data().Nothing printed there needs the new tables — the report is gathered during
compile(), the slots and lattices are compiler-local — so the reporting moves ahead of the installation. The last line of a successful trace is nowInstalling, emitted immediately before the commit.3. The transaction saved configuration, not just derived state (was #105)
It copied the registry's whole
policiestuple. A state that noinitializewrites to is not derived data the call is about to replace; it is configuration the caller owns. The error handler is the case that bites — it is called from inside the window by design, sincefast_perfect_hashreports a search failure through it — so a handler that disarms itself withset()before throwing had that undone:Copying wide also made copy-constructibility a hard requirement of every policy state, including those of policies with no
initializeat all — ruling out thestd::ostringstreamanoutputpolicy written to the documented state pattern holds. It compiled before #95 and stopped after, with 29 lines of diagnostics on gcc 13 (207 on clang 18) naming the policy list 21 times.Now only the states of the policies that will actually be initialized are saved, using the same
has_initializetestinitialize_policymakes.Note this does not remove the copy of the control vector and the vptr vector from the success path — those policies define
initialize, so their states must still be saved; that is what makes rollback possible. What is no longer copied is the error handler'sstd::functionand the output policy's state.4. The restore could terminate
It runs from the destructor, while an exception is in flight, and a destructor is
noexceptby default: a policy state whose move-assignment can throw would callstd::terminate, destroying the error the transaction exists to let through. Reachable through the public API — a state holding astd::mapwith a stateful, non-always-equal allocator degrades to an element-wise move that allocates, andvptr_map<MapFn>lets a caller supply exactly that. Every stock registry is nothrow move-assignable, so no test could have shown it. Refused at compile time now, next to the copyability assertion.5. The exception-safety claim was overstated
Three places said a failed
initialize()leaves the previous dispatch state "complete andconsistent", or that "the v-table pointers,
nextpointers and dispatch tables from theprevious call all stay in place". Both are false when the preceding call was
finalize(),which clears the dispatch data and every policy's state but leaves the classes'
static_vptrs set — as documented onstatic_vptr, which stays valid only until the nextinitialize()orfinalize()*. Afterinitialize→finalize→ failedinitializethetwo halves disagree.
What the transaction guarantees is preservation, not consistency: the call leaves the state
it found, whatever that was. The guarantee is unchanged; only the description was too strong.
The suite did not cover this path —
finalizeappeared in it only after a failedinitialize, never before one — so the case is added.Tests
a_throwing_trace_does_not_commitanda_throwing_report_does_not_committrap the trace on"Installing"and"Used slots"and check the previous dispatch state is intact. The second fails all three of its assertions without the reordering.test_initialize_policy_state_scope.cppfails to compile without fix 3 (its configuration policy's state is deliberately move-only) and fails1 != 2on the generation counter once the state is made copyable.compile_fail_policy_state_throwing_move.cppcovers fix 4.failed_initialize_after_finalize_restores_what_it_foundcovers fix 5. Checked against adeliberately disabled rollback, where it fails on
vptrs.empty()— so it is not one moreassertion that holds either way.
initializesaw exactly the input the preceding successful one saw, sofast_perfect_hashrecomputed identical factors andnext<poke_dog>resolved to the same overrider. Both held whether or not anything was rolled back — reintroducing the dangling-nexthalf of make initialize() transactional (write_global_data is not exception-safe) #81 left the suite reporting "No errors detected". The input is now perturbed between the two calls.Verification
ctestpass.toolset=gcc: no failures.clang-format-22applied.🤖 Generated with Claude Code
https://claude.ai/code/session_01JQa4fuiwcfsheZYTCyfPPr