Skip to content

Fix that the cache Switch operator never completes - #1137

Merged
dwcullop merged 5 commits into
reactivemarbles:mainfrom
dwcullop:bugfix/always_completed_connect
Jul 30, 2026
Merged

Fix that the cache Switch operator never completes#1137
dwcullop merged 5 commits into
reactivemarbles:mainfrom
dwcullop:bugfix/always_completed_connect

Conversation

@dwcullop

@dwcullop dwcullop commented Jul 25, 2026

Copy link
Copy Markdown
Member

Fixes #1136.

The cache Switch operator could never complete, could retain data from a source it had already switched away from, and routed delivery through a lock.

Never completing

It relayed changes through a private LockFreeObservableCache:

var destination = new LockFreeObservableCache<TObject, TKey>();
...
.PopulateInto(destination);

destination.Connect().Merge(errors).SubscribeSafe(observer)

A relay only ends when it is disposed, so the terminal event of the source had nowhere to go. Errors were carried across by hand through the merged errors subject; completion had no equivalent path and was silently dropped. A consumer of this operator never received OnCompleted, ever.

The lock

The obvious fix is to let Observable.Switch sit between the sources and the observer and propagate terminal events on its own. That is wrong here, and it took a measurement to see why.

Observable.Switch holds its gate for the whole of the downstream OnNext call. A pipeline that crosses into another cache runs that work under the gate, and a producer on another thread blocks behind it. That is precisely the cross cache deadlock shape the delivery queue exists to avoid, and the operator on main avoids it today by keeping the observer on the far side of the relay cache.

Measured with a subscriber that blocks inside OnNext, writing to the source from another thread:

time to return
main 0 ms
via Observable.Switch 748 ms
this PR 1 ms

So the operator is now written by hand. A SerialDisposable holds the current inner subscription, and every subscription carries an identity, so a superseded source still delivering concurrently is dropped rather than applied on top of the state its replacement has already established. That is the second defect above, fixed by construction rather than by timing. All state changes and all delivery go through the queue, which releases its lock before handing anything downstream, so a second producer enqueues and returns instead of waiting.

The reset that clears the previous source's contribution is enqueued under the same lock acquisition that takes the new identity, which keeps it ordered against the changes that follow it.

Terminal semantics

Completes once the sources and the current inner have both completed. Fails as soon as either does. That matches Observable.Switch, which is what callers reasonably expect from something called Switch.

Tests

Cache/SwitchFixture:

  • CompletesWhenSourcesAndInnerComplete
  • DoesNotCompleteWhileInnerIsStillRunning
  • DoesNotCompleteWhenOnlyASupersededInnerCompletes
  • CompletesWhenSourcesAndInnerCompleteSynchronously
  • DeliversChangesEmittedBeforeSynchronousCompletion
  • PropagatesInnerErrorsRaisedSynchronously
  • IgnoresChangesFromASupersededSource
  • DoesNotHoldALockWhileDeliveringDownstream

That last one blocks a subscriber inside OnNext and asserts that writing to the source from another thread still returns. It fails against an Observable.Switch implementation and passes against this one, so the lock behaviour cannot quietly come back.

SuspendNotificationsFixture, covering #1136 directly:

  • OnCompletedFiresIfCacheDisposedAfterConnectingWhileSuspended
  • OnErrorFiresIfCacheFailsAfterConnectingWhileSuspended
  • OnCompletedFiresIfCacheDisposedAfterWatchingWhileSuspended

Notes

Rebased onto current main and squashed. The branch previously carried a workaround in ObservableCache.Connect() that was later reverted in favour of fixing the operator, and replaying that history over #1132 conflicted repeatedly for no benefit. Nothing outside Switch and its tests is touched now.

The list Switch has the same three defects and is fixed the same way in #1139. The two are independent.

Behavioural change to a public operator: sequences that previously never terminated now terminate when their sources do. main is on 10.0-preview.

Unrelated, and not addressed here: AggregationEx.InvalidateWhen still uses Rx's Switch and therefore still holds a gate during delivery. It operates on plain value streams rather than changesets, so the blast radius is smaller, but it is the same pattern if anyone is worried about it.

@dwcullop dwcullop changed the title Fix that a Connect() deferred by a suspension never completes Fix that the cache Switch operator never completes Jul 25, 2026
@dwcullop
dwcullop requested review from JakenVeina and Copilot July 25, 2026 23:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a long-standing termination bug in DynamicData’s cache Switch operator, restoring Rx-like Observable.Switch completion semantics for IObservable<IObservable<IChangeSet<TObject, TKey>>> pipelines (notably affecting deferred Connect() subscriptions during notification suspension, per #1136).

Changes:

  • Propagate terminal events through an explicit “terminal” channel and end the downstream destination.Connect() stream via TakeUntil, enabling completion to flow correctly.
  • Add operator-level tests for completion behavior (sources vs current/superseded inner sequences).
  • Add regression tests covering deferred Connect() / Watch() during SuspendNotifications(), ensuring completion/error propagation after activation.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
src/DynamicData/Cache/Internal/Switch.cs Reworks termination propagation so the cache Switch can complete/error correctly despite relaying via a cache that otherwise only ends on disposal.
src/DynamicData.Tests/Cache/SwitchFixture.cs Adds targeted unit tests validating correct completion semantics of the cache Switch operator.
src/DynamicData.Tests/Cache/SuspendNotificationsFixture.cs Adds regression coverage for #1136 ensuring deferred subscriptions complete/error once activated.

Comment thread src/DynamicData/Cache/Internal/Switch.cs Outdated
Comment thread src/DynamicData/Cache/Internal/Switch.cs Outdated
Comment thread src/DynamicData/Cache/Internal/Switch.cs Outdated
Comment thread src/DynamicData/Cache/Internal/Switch.cs
Comment thread src/DynamicData/Cache/Internal/Switch.cs Outdated
Comment thread src/DynamicData.Tests/Cache/SuspendNotificationsFixture.UnitTests.cs Outdated
@JakenVeina
JakenVeina force-pushed the bugfix/always_completed_connect branch from c1ad6dd to 3baabaf Compare July 30, 2026 03:29
dwcullop and others added 4 commits July 29, 2026 22:46
…lock

Switch could never complete, could retain data from a source it had already
switched away from, and routed delivery through a lock.

It relayed changes through a private LockFreeObservableCache. That cache only
ends when it is disposed, so the terminal event of the source had nowhere to go.
Errors were carried across by hand through a merged subject; completion had no
equivalent path and was silently dropped. A consumer never received OnCompleted,
ever.

Switching is now explicit rather than delegated to Observable.Switch, which holds
its gate for the whole of the downstream OnNext call. A pipeline that crosses into
another cache runs that work under the gate, and a producer on another thread
blocks behind it, which is the cross cache deadlock shape the delivery queue
exists to avoid. Measured against a subscriber that blocks inside OnNext, writing
to the source from another thread took 748ms through Observable.Switch and 1ms
through the queue, which enqueues and returns.

A SerialDisposable holds the current inner subscription, and each one carries an
identity, so a superseded source still delivering concurrently is dropped rather
than applied on top of the state its replacement has already established. That is
the second defect above. All state changes and all delivery happen through the
queue lock, which is released before anything is handed downstream.

The result completes once the sources and the current inner have both completed,
and fails as soon as either does.
OnNext and OnCompleted both check the captured id against the active source,
because a source that has been switched away from may still be mid-delivery.
OnError went straight to the queue without that check, so a late failure from
a source no longer selected could terminate the output. Rx's own Switch
discards it, so the hand-rolled version was a regression on that point.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The first version subscribed a Subject and failed it after the switch, but
SerialDisposable had already disposed that subscription, so Rx suppressed the
notification and the test passed with or without the guard.

Disposal cannot reach a notification already in flight, which is the case the
guard exists for. RawAnonymousObservable hands back the observer directly, so
the failure can be delivered after the switch without needing a race to land.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Rename active to activeSourceId and id to sourceId, which say what they are.

Use SubscribeSafe throughout, so a source that throws out of its subscribe call
is reported through the queue rather than escaping to whoever happened to be
subscribing.

Replace the CompositeDisposable with an explicit closure. Disposal order matters
here, the queue has to drain before the subscriptions feeding it go away, and
CompositeDisposable does not specify an order.

Drop OnCompletedFiresIfCacheDisposedAfterConnectingWhileSuspended, which is
covered more thoroughly by the test in reactivemarbles#1145.
@JakenVeina
JakenVeina force-pushed the bugfix/always_completed_connect branch from 3baabaf to 27b8a42 Compare July 30, 2026 03:46
@dwcullop
dwcullop enabled auto-merge (squash) July 30, 2026 18:09
The conflict was in SuspendNotificationsFixture, where reactivemarbles#1141 landed three
tests covering deferred subscriptions and this branch had added two of its
own in the same place.

Kept reactivemarbles#1141's three as they are. Dropped this branch's
OnErrorFiresIfCacheFailsAfterConnectingWhileSuspended, which was the same
scenario as reactivemarbles#1141's OnErrorFiresIfCacheFailsAfterResumingWhileConnectionWasSuspended
down to the assertions, differing only in using ObservableCache where reactivemarbles#1141
uses the public IntermediateCache.

Kept the completion-through-Watch test and renamed it to match the naming
reactivemarbles#1141 established for the after-resume cases. Nothing else covers that
corner: the existing OnCompletedFiresIfCacheDisposedWhileSuspended is
Connect while still suspended, and reactivemarbles#1141's Watch test is a failure rather
than a completion.

Normalised the file to CRLF while resolving. It had 58 lines with bare LF
endings before this, which is why the diff for that file looks larger than
the one test being added.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9582bb33-26d3-4aa5-8dd7-57dc55304680
@dwcullop
dwcullop merged commit 41b4d9d into reactivemarbles:main Jul 30, 2026
2 checks passed
dwcullop added a commit to dwcullop/DynamicData that referenced this pull request Jul 31, 2026
Matches what the cache version got in reactivemarbles#1137. SubscribeSafe on both the
outer and inner subscriptions, and an explicit teardown instead of
CompositeDisposable, which does not specify a disposal order. The queue has
to go first so any delivery in flight finishes before the subscriptions
feeding it are torn down.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9582bb33-26d3-4aa5-8dd7-57dc55304680
dwcullop added a commit to dwcullop/DynamicData that referenced this pull request Jul 31, 2026
The only conflict was in BatchIfFixture, where reactivemarbles#1153 added
PauseSelectorOnlyStartsUnpaused at the end of the class and this branch had
added three terminal-event tests in the same place. Both sets kept, they
cover different things.

Checked that the merge did not walk back anything that landed today: reactivemarbles#1141's
SelectMany deferral and SuspensionTracker.Fault are intact, reactivemarbles#1137's Switch
rewrite came through unchanged, and reactivemarbles#1153's BatchIf overloads are all still
there.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9582bb33-26d3-4aa5-8dd7-57dc55304680
dwcullop added a commit to dwcullop/DynamicData that referenced this pull request Jul 31, 2026
Three conflicts, all the same shape: reactivemarbles#1137 and reactivemarbles#1141 changed files whose
using blocks this branch had already removed, so git had nothing to line the
hunks up against.

Resolved by taking main's version of each file and re-running the strip, so
the content is exactly what landed in reactivemarbles#1137 and reactivemarbles#1141 and only the imports
differ. Checked that afterwards: the only difference from main in all three
is the removed usings, plus the byte order mark moving down to the namespace
line now that there is nothing above it.

The repo-wide invariant still holds. The only file-level usings left are the
four opt-in namespaces in the test project.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9582bb33-26d3-4aa5-8dd7-57dc55304680
dwcullop added a commit to dwcullop/DynamicData that referenced this pull request Jul 31, 2026
One conflict, in the cache Switch operator. This branch had removed the Rx
gate around Observable.Switch but kept the surrounding structure, the
SharedDeliveryQueue routing and the LockFreeObservableCache relay. reactivemarbles#1137
went further and dropped Observable.Switch altogether, so main's version
already does what this branch wanted for that file and does it more
thoroughly. Took main's side wholesale.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9582bb33-26d3-4aa5-8dd7-57dc55304680
JakenVeina pushed a commit to dwcullop/DynamicData that referenced this pull request Jul 31, 2026
Matches what the cache version got in reactivemarbles#1137. SubscribeSafe on both the
outer and inner subscriptions, and an explicit teardown instead of
CompositeDisposable, which does not specify a disposal order. The queue has
to go first so any delivery in flight finishes before the subscriptions
feeding it are torn down.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9582bb33-26d3-4aa5-8dd7-57dc55304680
JakenVeina pushed a commit to dwcullop/DynamicData that referenced this pull request Jul 31, 2026
Matches what the cache version got in reactivemarbles#1137. SubscribeSafe on both the
outer and inner subscriptions, and an explicit teardown instead of
CompositeDisposable, which does not specify a disposal order. The queue has
to go first so any delivery in flight finishes before the subscriptions
feeding it are torn down.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9582bb33-26d3-4aa5-8dd7-57dc55304680
JakenVeina pushed a commit to dwcullop/DynamicData that referenced this pull request Jul 31, 2026
Matches what the cache version got in reactivemarbles#1137. SubscribeSafe on both the
outer and inner subscriptions, and an explicit teardown instead of
CompositeDisposable, which does not specify a disposal order. The queue has
to go first so any delivery in flight finishes before the subscriptions
feeding it are torn down.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9582bb33-26d3-4aa5-8dd7-57dc55304680
dwcullop added a commit that referenced this pull request Aug 1, 2026
…1139)

* Fix that the list Switch operator drops completion and throws errors, without taking a lock

Switch relayed changes through a private SourceList and subscribed the observer to
that, so the terminal event of the source had nowhere to go. Completion was
dropped outright, and an error was rethrown out of the subscription rather than
delivered as OnError.

Switching is now explicit rather than delegated to Observable.Switch, which holds
its gate for the whole of the downstream OnNext call. A pipeline that crosses into
another collection runs that work under the gate, and a producer on another thread
blocks behind it, which is the cross cache deadlock shape the delivery queue
exists to avoid. Measured against a subscriber that blocks inside OnNext, writing
to the source from another thread took 748ms through Observable.Switch and 1ms
through the queue, which enqueues and returns.

A SerialDisposable holds the current inner subscription, and each one carries an
identity, so a superseded source still delivering concurrently is dropped rather
than applied on top of the state its replacement has already established. All
state changes and all delivery happen through the queue lock, which is released
before anything is handed downstream.

The result completes once the sources and the current inner have both completed,
and fails as soon as either does.

* Ignore errors from a superseded source in the list Switch operator

The list counterpart of the same gap in the cache operator: OnNext and
OnCompleted check the captured id against the active source, but OnError went
straight to the queue, so a late failure from a source already switched away
from could terminate the output.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Make the superseded-source test actually exercise the guard

Same correction as the cache side. The Subject version was disposed by
SerialDisposable before the failure was raised, so Rx suppressed it and the
test passed with or without the guard. RawAnonymousObservable delivers the
failure after the switch, which is the in-flight case the guard is for.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Use SubscribeSafe and explicit disposal in the list Switch operator

Matches what the cache version got in #1137. SubscribeSafe on both the
outer and inner subscriptions, and an explicit teardown instead of
CompositeDisposable, which does not specify a disposal order. The queue has
to go first so any delivery in flight finishes before the subscriptions
feeding it are torn down.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9582bb33-26d3-4aa5-8dd7-57dc55304680

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 9582bb33-26d3-4aa5-8dd7-57dc55304680
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Deferred Connect() never receives OnCompleted when the source is disposed

3 participants