Skip to content

initialize: close five holes in the transaction - #101

Open
jll63 wants to merge 5 commits into
boostorg:developfrom
jll63:fix/initialize-commit-order
Open

initialize: close five holes in the transaction#101
jll63 wants to merge 5 commits into
boostorg:developfrom
jll63:fix/initialize-commit-order

Conversation

@jll63

@jll63 jll63 commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

(Written by Claude Code, on behalf of @jll63.)

Four ways the transaction #95 introduced could be escaped. Rebased onto current develop, and it now absorbs #105, which is closed in favour of this branch.

1. The commit happened one statement too early

transaction.commit() ran before ++tr << "Installing\n", which goes through the user-supplied output policy and can throw. When it did, the destructor saw a committed transaction and restored nothing, so the policies kept the v-table pointers they had read out of new_dispatch_data — the local vector unwinding was about to free. The registry still held the previous dispatch data, so the next dispatch read freed memory: the exact use-after-free #95 fixed, reached by another route.

ERROR: AddressSanitizer: heap-use-after-free
  READ of size 8 ... in resolve_uni<...>
  freed by ... write_global_data()

The trace write moves above commit(), and the writes that follow are extracted into commit_global_data(), declared noexcept so a throwing statement added there terminates loudly instead of silently reopening this.

2. The reporting ran after the commit

print(report) and print_slots() ran after write_global_data() had committed. Both can throw — the trace goes through the output policy, and print_slots() builds an unordered_map and two vectors to partition the class graph. A throw there landed past the commit: new tables installed, policies holding the new pointers, and initialize() never reaching st.initialized = true. Nothing is corrupted, but it is a fourth outcome the contract does not describe, and under runtime_checks the next call aborts with not_initialized even though the tables are good. This survived fix 1, which moved only the write inside write_global_data().

Nothing printed there needs the new tables — the report is gathered during compile(), the slots and lattices are compiler-local — so the reporting moves ahead of the installation. The last line of a successful trace is now Installing, emitted immediately before the commit.

3. The transaction saved configuration, not just derived state (was #105)

It copied the registry's whole policies tuple. A state that no initialize writes to is not derived data the call is about to replace; it is configuration the caller owns. The error handler is the case that bites — it is called from inside the window by design, since fast_perfect_hash reports a search failure through it — so a handler that disarms itself with set() before throwing had that undone:

before: handler still installed after the failed initialize = 1
after:  handler still installed after the failed initialize = 0

Copying wide also made copy-constructibility a hard requirement of every policy state, including those of policies with no initialize at all — ruling out the std::ostringstream an output policy written to the documented state pattern holds. It compiled before #95 and stopped after, with 29 lines of diagnostics on gcc 13 (207 on clang 18) naming the policy list 21 times.

Now only the states of the policies that will actually be initialized are saved, using the same has_initialize test initialize_policy makes.

Note this does not remove the copy of the control vector and the vptr vector from the success path — those policies define initialize, so their states must still be saved; that is what makes rollback possible. What is no longer copied is the error handler's std::function and the output policy's state.

4. The restore could terminate

It runs from the destructor, while an exception is in flight, and a destructor is noexcept by default: a policy state whose move-assignment can throw would call std::terminate, destroying the error the transaction exists to let through. Reachable through the public API — a state holding a std::map with a stateful, non-always-equal allocator degrades to an element-wise move that allocates, and vptr_map<MapFn> lets a caller supply exactly that. Every stock registry is nothrow move-assignable, so no test could have shown it. Refused at compile time now, next to the copyability assertion.

5. The exception-safety claim was overstated

Three places said a failed initialize() leaves the previous dispatch state "complete and
consistent", or that "the v-table pointers, next pointers and dispatch tables from the
previous call all stay in place". Both are false when the preceding call was finalize(),
which clears the dispatch data and every policy's state but leaves the classes'
static_vptrs set — as documented on static_vptr, which stays valid only until the next
initialize() or finalize()*. After initializefinalize → failed initialize the
two halves disagree.

What the transaction guarantees is preservation, not consistency: the call leaves the state
it found, whatever that was. The guarantee is unchanged; only the description was too strong.
The suite did not cover this path — finalize appeared in it only after a failed
initialize, never before one — so the case is added.

Tests

  • a_throwing_trace_does_not_commit and a_throwing_report_does_not_commit trap the trace on "Installing" and "Used slots" and check the previous dispatch state is intact. The second fails all three of its assertions without the reordering.
  • test_initialize_policy_state_scope.cpp fails to compile without fix 3 (its configuration policy's state is deliberately move-only) and fails 1 != 2 on the generation counter once the state is made copyable.
  • compile_fail_policy_state_throwing_move.cpp covers fix 4.
  • failed_initialize_after_finalize_restores_what_it_found covers fix 5. Checked against a
    deliberately disabled rollback, where it fails on vptrs.empty() — so it is not one more
    assertion that holds either way.
  • Two assertions in the existing tests were vacuous and are repaired: the failing initialize saw exactly the input the preceding successful one saw, so fast_perfect_hash recomputed identical factors and next<poke_dog> resolved to the same overrider. Both held whether or not anything was rolled back — reintroducing the dangling-next half of make initialize() transactional (write_global_data is not exception-safe) #81 left the suite reporting "No errors detected". The input is now perturbed between the two calls.

Verification

  • CMake/Ninja, gcc, Debug: 162/162 ctest pass.
  • b2 toolset=gcc: no failures.
  • Each fix confirmed red before and green after.
  • clang-format-22 applied.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JQa4fuiwcfsheZYTCyfPPr

@cppalliance-bot

cppalliance-bot commented Sep 11, 2026

Copy link
Copy Markdown

An automated preview of the documentation is available at https://101.openmethod.prtest3.cppalliance.org/libs/openmethod/doc/html/index.html

If more commits are pushed to the pull request, the docs will rebuild at the same URL.

2026-09-12 16:31:58 UTC

jll63 and others added 4 commits September 12, 2026 09:50
boostorg#95 made initialize() transactional, but `transaction.commit()` was called
one statement too early: the `++tr << "Installing\n"` that follows it goes
through the `output` policy, which is user code and can throw. When it does,
the transaction destructor sees a committed transaction and restores nothing,
so the policies keep the v-table pointers they read out of `new_dispatch_data`
- the local vector unwinding is about to free. The registry still holds the
previous dispatch data (the swap never ran), so the next dispatch through the
policy state reads freed memory: the exact use-after-free boostorg#95 fixed, reached
by a different route. ASan, gcc 13, an `output` policy whose stream throws on
"Installing" plus initialize(trace(true)):

    ERROR: AddressSanitizer: heap-use-after-free
      READ of size 8 ... in resolve_uni<...>
      freed by ... write_global_data()

Move the trace write above commit(), where a throw rolls the policies back,
and extract the writes that follow into `commit_global_data()`, declared
`noexcept` so that a throwing statement added there terminates loudly instead
of silently reopening this.

The test grew a case for that path, and two of its existing assertions were
vacuous: the failing initialize saw exactly the input the preceding successful
one saw, so `fast_perfect_hash` - which re-seeds a fixed PRNG - recomputed
identical factors, and `next<poke_dog>` resolved to the same overrider. Both
held whether or not anything was rolled back; reintroducing the dangling-`next`
half of boostorg#81 left the suite reporting "No errors detected". Perturb the input
between the two calls instead, with two function-local static registrars: an
extra class changes the hash factors, the control table and the v-table
pointers, and an extra inheritance edge changes what `next<poke_dog>` would be
set to. The hash assertion also compares the whole policy state now - factors
and control table - rather than `hash_range()` alone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RQG6CbE4o2agseE7bDVzHS
The transaction section says the registry is marked as not initialized and
that initialize() must be called again, but not what happens if a method is
called before it is. Only `runtime_checks` diagnoses that; otherwise the call
dispatches through the previous tables, which is harmless for a registry whose
classes are all still loaded and a use-after-free for one whose overriders
came out of a library that has since been dlclose'd - the case the section is
written for.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JQa4fuiwcfsheZYTCyfPPr
…hose

boostorg#95 made initialize() transactional by copying the registry's whole `policies`
tuple and putting it back if anything throws. The comment justified the width
- "restoring an untouched state is harmless, and simpler than picking" - but
it is not harmless, in two ways.

A state that no `initialize` writes to is not derived data the call is about
to replace; it is configuration the caller owns. The error handler is the case
that bites: it is *called* from inside the transaction window by design, since
fast_perfect_hash reports a search failure through it. A handler that disarms
itself - installs another handler with set() before throwing, so the failure
is reported once - had that undone on the way out. Reported here as a
change to the handler surviving or not:

    before: handler still installed after the failed initialize = 1
    after:  handler still installed after the failed initialize = 0

And copying the tuple made copy-constructibility a hard requirement of every
policy state in the registry, including states of policies that have no
`initialize` at all. That rules out a state holding a std::mutex,
std::unique_ptr, std::atomic or std::ostringstream - and an ostringstream is
exactly what an `output` policy written to the documented state pattern
holds. It compiled before boostorg#95 and stopped compiling after, with 29 lines of
deleted-copy-constructor diagnostics on gcc 13 (207 on clang 18) naming the
whole policy list 21 times.

Filter the saved states to the policies whose `initialize` will actually run,
using the same has_initialize test initialize_policy makes, so the two cannot
disagree about which policies run. The saved tuple is now a subset of the
registry's, so it is filled and restored element-wise. A static_assert spells
out the requirement that remains, in the manner of the one preamble.hpp
already carries for duplicate state types.

One correction to how this was reported to me: it does *not* remove the copy
of the control vector and the vptr vector from the success path. Those two
policies define `initialize`, so their states must still be saved - that is
what makes a rollback possible. What is no longer copied is the states of the
policies that do not initialize: in default_registry, the error handler's
std::function and the output policy's state.

Not addressed: the restore is still a move-assignment in a `noexcept`
destructor, so a policy state whose move-assignment throws still terminates.
Stock registries are nothrow, so nothing in the suite can show it.

The test covers both halves. It fails to compile without this change, because
its configuration policy's state is deliberately move-only; with the state
made copyable to isolate the other half, it fails 1 != 2 on the rolled-back
generation counter.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JQa4fuiwcfsheZYTCyfPPr
…ow on restore

Two more ways the transaction could be escaped, both found by the same review
that prompted the previous commits.

print(report) and print_slots() ran after write_global_data() had committed.
Both can throw - the trace goes through the user-supplied `output` policy, and
print_slots() builds an unordered_map and two vectors to partition the class
graph - and a throw there landed past the commit: the new tables installed,
the policies holding the new v-table pointers, and initialize() never reaching
`st.initialized = true`. Nothing is corrupted, but it is a fourth outcome the
exception-safety contract does not describe, and under runtime_checks the next
call aborts with `not_initialized` even though the tables are perfectly good.
It survived the previous commit, which moved only the "Installing" write that
lives inside write_global_data().

Nothing printed there needs the new tables: the report is gathered during
compile(), and the slots and lattices are compiler-local. So report first,
install second, and a throw is once again just a failure the transaction rolls
back. The last line of a successful trace is now "Installing", which is
emitted immediately before the commit.

The other is the restore itself. It runs from the transaction's destructor,
while an exception is in flight, and a destructor is noexcept by default: a
policy state whose move-assignment can throw would terminate the program,
destroying the error the transaction exists to let through. That is reachable
through the public API - a state holding a std::map with a stateful,
non-always-equal allocator degrades to an element-wise move that allocates,
and `vptr_map<MapFn>` lets a caller supply exactly that map. Every stock
registry is nothrow move-assignable, so no test could ever have shown it.
Refuse such a state at compile time, next to the copyability assertion, where
the message can say why.

Tests: a_throwing_report_does_not_commit trips the trace on "Used slots",
which print_slots() writes, and checks the previous dispatch state is intact -
it fails all three of those assertions without the reordering. The
compile-fail test covers the assertion.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JQa4fuiwcfsheZYTCyfPPr
@jll63
jll63 force-pushed the fix/initialize-commit-order branch from 69e2634 to 5ca43eb Compare September 12, 2026 13:57
@jll63 jll63 changed the title initialize: move the last throwing statement before the commit initialize: close four holes in the transaction Sep 12, 2026
@jll63 jll63 changed the title initialize: close four holes in the transaction initialize: close five holes in the transaction Sep 12, 2026
Three places claimed that a failed initialize() leaves the previous dispatch
state "complete and consistent", or that "the v-table pointers, `next`
pointers and dispatch tables from the previous call all stay in place". Both
are false when the preceding call was finalize(): that clears the dispatch
data and every policy's state but leaves the classes' static_vptrs set - as
documented on static_vptr, which stays valid only until the next initialize()
*or finalize()*. After initialize -> finalize -> failed initialize the two
halves disagree, and no call is possible until an initialize() succeeds.

What the transaction actually guarantees is preservation, not consistency: the
call leaves the state it found, whatever that was. Say that instead. The
guarantee is unchanged - only the description was too strong.

Adds the case to the test, which the suite did not cover: finalize appeared in
it only after a failed initialize, never before one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JQa4fuiwcfsheZYTCyfPPr
@jll63
jll63 force-pushed the fix/initialize-commit-order branch from c6b6584 to 35f4e39 Compare September 12, 2026 16:25
@codecov

codecov Bot commented Sep 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.62%. Comparing base (5ec487c) to head (35f4e39).
⚠️ Report is 10 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #101      +/-   ##
===========================================
+ Coverage    93.46%   93.62%   +0.15%     
===========================================
  Files           22       22              
  Lines         1653     1694      +41     
  Branches       500      505       +5     
===========================================
+ Hits          1545     1586      +41     
  Misses          64       64              
  Partials        44       44              
Files with missing lines Coverage Δ
include/boost/openmethod/initialize.hpp 92.96% <100.00%> (+0.30%) ⬆️

... and 3 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ba56242...35f4e39. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants