Skip to content

fix: Publish the data source status before releasing ready waiters - #431

Draft
jsonbailey wants to merge 2 commits into
mainfrom
jb/polling-spec-flake
Draft

fix: Publish the data source status before releasing ready waiters#431
jsonbailey wants to merge 2 commits into
mainfrom
jb/polling-spec-flake

Conversation

@jsonbailey

@jsonbailey jsonbailey commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Requirements

  • I have added test coverage for new or changed functionality
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions

Related issues

Follows #429, which fixed the same ordering problem on the unrecoverable-error path. Reworked from a spec-only change after @kinyoklion pointed out that the ordering itself was the bug — matching the change he has already merged in the Go SDK.

Describe the solution you've provided

1. Publish the status before releasing ready waiters

PollingProcessor#poll set the ready event and only then published VALID. A caller that returned from start could read a data_source_status_provider.status that still said INITIALIZING even though the poll had succeeded and the flags were already in the store. The FDv2 synchronizer loop had the same ordering.

Both now publish the status first and set the ready event afterwards. This is the same argument #429 made for the OFF path, applied to the success path.

2. The success spec no longer needs to wait

This PR started as a spec-only fix that waited for the listener to be notified. With the ordering corrected that wait is unnecessary, so status is set to valid when data is received is back to its original form, unchanged from main.

Worth correcting one claim from the original description: delivery here is not asynchronous. The polling spec uses SynchronousExecutor, whose post just yields, so Broadcaster#broadcast calls listener.update inline. The distinction is which thread — inline on the poll thread, not the spec thread. That is why the old ordering was racy and why the new ordering makes the assertion deterministic: by the time @ready.set runs, the listener has already returned on that same thread.

3. The recoverable-error spec still needs its wait

verify_recoverable_http_error (408, 429, 503) keeps wait_for_count(2). That path publishes INTERRUPTED and never sets the ready event, so there is no event to reorder and the pre-existing ready.wait(1) was only passing time. ListenerSpy's mutex and condition variable stay for the same reason — without an event there is no happens-before for the cross-thread read, and a bare array appended from the poll thread and read from the spec thread is a real data race on JRuby.

wait_for_status is removed; wait_for_count is the only helper still used.

Describe alternatives you've considered

Keeping the spec-only fix. It would have made CI green while leaving an observable ordering problem in the SDK, which is the same trade #429 declined.

Fixing the FDv2 initializer path as well. On success it sets the ready event and publishes no status at all — the status stays INITIALIZING until the synchronizer's first update lands. That looks like a larger bug than the ordering one and is worth its own discussion, so it is deliberately not touched here.

Additional context

The broader problem is that listeners can only be registered after the constructor returns, so the initial status broadcast is unobservable no matter which order these two lines run in — and UpdateSink#update_status drops a repeat of the same state, so it is never replayed. The reliable pattern is subscribe, then read data_source_status_provider.status. Separating start from construction would remove the question entirely; that is being discussed separately.

The new polling spec is a genuine regression guard — it fails deterministically without the production change, because it asserts that the listener does not observe an already-set ready event. The FDv2 spec is a contract assertion rather than a guard: it passes either way on CRuby, since the GIL hides the window there, and the FDv2 status broadcaster uses a real thread pool so a listener-ordering assertion would itself be racy.

The FDv2 sleep flake called out in the original description is now fixed on main by #432.

Validation

  • Full local suite: 1084 examples, 0 failures. The gap against CI is the redis, dynamodb, and consul store specs, which need backing services.
  • polling_spec.rb + fdv2_datasystem_spec.rb in a loop, 15 runs: 15 of 15 passed.
  • New polling spec against the old ordering: fails, as intended.
  • RuboCop clean on all five changed files.

@jsonbailey jsonbailey changed the title fix: Wait for status listener delivery in data source specs chore: Wait for status listener delivery in data source specs Sep 2, 2026
@jsonbailey
jsonbailey marked this pull request as ready for review September 2, 2026 21:50
@jsonbailey
jsonbailey requested a review from a team as a code owner September 2, 2026 21:50
Comment thread spec/impl/data_source/polling_spec.rb Outdated

expect(listener.statuses.count).to eq(1)
expect(listener.statuses[0].state).to eq(Interfaces::DataSource::Status::VALID)
# The poll thread sets the ready event before it publishes the VALID

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason it is in this order? Go was also doing this and I just swapped them because you would expect the status to be set before the ready event.

FDv1 polling and the FDv2 synchronizer loop both set the ready event
before publishing the VALID data source status. A caller that returned
from start could therefore read a status that did not yet reflect the
successful poll.

Both now publish the status first and set the ready event afterwards,
matching the OFF path fixed in #429.

The polling spec no longer needs to wait for the VALID status, because
the broadcaster notifies listeners inline on the poll thread and the
ready event is now set after that. The recoverable-error spec keeps its
wait: that path publishes INTERRUPTED and never sets the ready event, so
there is nothing to synchronize against.
@jsonbailey
jsonbailey force-pushed the jb/polling-spec-flake branch from ea48542 to b3fcc65 Compare September 9, 2026 21:58
@jsonbailey jsonbailey changed the title chore: Wait for status listener delivery in data source specs fix: Publish the data source status before releasing ready waiters Sep 9, 2026
@jsonbailey
jsonbailey marked this pull request as draft September 9, 2026 22:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants