feat(internal): add flag/segment override layer, overlay store, and change notification diff - #408
Merged
Conversation
This was referenced Jul 7, 2026
kinyoklion
force-pushed
the
rlamb/sdk-2655/internal-overrides
branch
from
September 15, 2026 21:46
3624348 to
cc8c47c
Compare
…ogress A file system that does not respond must not block shutdown. Close now signals the loop and returns, matching the Reloader, and the doc states that onChange can run once more shortly after Close. The doc uses ASCII, states that only modification time and size are compared, and drops the claim about temporarily unreadable files. Tests pin the Close contract, same-size and same-time changes, single firing per change, and changes to the first of several files.
…hange notification diff
kinyoklion
force-pushed
the
rlamb/sdk-2655/internal-overrides
branch
from
September 17, 2026 21:28
b677459 to
5e63a76
Compare
Callers read through the overlay and act on the result, so a separate existence check is not needed.
kinyoklion
changed the base branch from
feat/overrides
to
rlamb/sdk-2654/filedata-poller
September 17, 2026 22:17
kinyoklion
marked this pull request as ready for review
September 17, 2026 22:21
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 5fbd58d. Configure here.
The marked copy was shallow, and the preprocessing it ran wrote caches into the nested slices it still shared with the source's entity. A source that retains an entity and supplies it again while evaluation reads the stored copy raced on those writes. The copy now sets the marker and nothing else. Entities from deserialization or the builders already carry their caches, and the evaluator scans values when a cache is absent, so evaluation is unchanged.
Base automatically changed from
rlamb/sdk-2654/filedata-poller
to
feat/overrides
September 18, 2026 16:23
…e fails A per-key read returns an override entry whatever the state of the base store. An all-flags read returned the base store's error instead and dropped the override entries with it. It now returns the layer's entries alone when the base fails and the layer holds entries. An empty layer still returns the error, so a client without overrides behaves as before.
keelerm84
approved these changes
Sep 21, 2026
The test builds raw entities without preprocessing caches and pristine twins of them. After the layer stores the entities, a deep comparison shows every field of the supplied entities is unchanged, including the caches inside the shared rules, clauses, and targets. The stored copies then evaluate through the accessors' fallbacks. The check is deterministic and does not depend on the race detector.
kinyoklion
added a commit
that referenced
this pull request
Sep 21, 2026
Connects the override layer to the outside world:
- New public seam in `subsystems`: `OverrideSource` (implemented by
sources; `Start(sink)`/`Close`) and `OverrideSink` (implemented by the
SDK; `SetOverrides` takes a full replacement snapshot of flag/segment
collections and the SDK applies the override marker itself). Both are
new interfaces -- nothing is added to any existing exported interface.
- Configuration: a new `OverrideSource` field on
`subsystems.DataSystemConfiguration` and an `Overrides(...)` builder
method on `ldcomponents.DataSystemConfigurationBuilder`, so the eventual
usage is `ldcomponents.DataSystem().Default().Overrides(...)`. A single
source, not a list: combining multiple documents (and the duplicate-key
policy that implies) is the source's job, per the spec's multi-file
semantics.
- FDv2 owns the lifecycle: when a source is configured, it builds the
layer and overlay, returns the overlay from `Store()` (all three client
read paths -- evaluation, prerequisites/segments, all-flags -- go
through that one accessor), starts the source synchronously before the
run loop so a synchronously-loading source has its overrides in place
before the client evaluates anything, and closes it first on `Stop()`.
With no source configured, `Store()` returns the raw store and nothing
about the system changes. The relay data destination and persistence
keep seeing the raw store, and data availability is untouched by
overrides. Overrides are inert in offline/disabled mode.
- The data system exposes no probe of the override layer's contents. The
client decides the not-initialized short-circuit from the store read
itself (next PR), which avoids a check-then-read race across a layer
replacement.
- `internal/sharedtest` gains a programmatic `TestOverrideSource`, which
is both the test vehicle here and the reference implementation of the
seam.
### Configuration in use
An override source is any type that implements
`subsystems.OverrideSource`. The builder takes a
`ComponentConfigurer` so the source is built with the client context,
like every other component.
```go
// pinnedFlags is an override source that forces a fixed set of flags on a running client.
type pinnedFlags struct {
sink subsystems.OverrideSink
}
// Start receives the sink and pushes the initial snapshot. The SDK calls Start before the data
// system runs, so a source that loads synchronously has its overrides in place before the first
// evaluation.
func (p *pinnedFlags) Start(sink subsystems.OverrideSink) {
p.sink = sink
sink.SetOverrides(loadPinnedFlags()) // []ldstoretypes.Collection
}
// Close is called first when the client stops.
func (p *pinnedFlags) Close() error { return nil }
// Build satisfies subsystems.ComponentConfigurer[subsystems.OverrideSource].
func (p *pinnedFlags) Build(subsystems.ClientContext) (subsystems.OverrideSource, error) {
return p, nil
}
config := ld.Config{
DataSystem: ldcomponents.DataSystem().Default().Overrides(&pinnedFlags{}),
}
client, err := ld.MakeCustomClient(sdkKey, config, 10*time.Second)
// Later, replace the whole snapshot. Flags not present in the new data are no longer overridden;
// an empty slice removes every override.
source.sink.SetOverrides(newPinnedFlags())
```
FDv1 behavior is unchanged; overrides are structurally FDv2-only since
the configuration hangs off `Config.DataSystem`.
Based on #408 (the override layer this wires up).
SDK-2655
<!-- CURSOR_SUMMARY -->
---
> [!NOTE]
> **Overview**
> Adds a **public override seam** so operators can pin flag/segment
definitions that win over LaunchDarkly data at read time, without
touching initialization or data-source status.
>
> New `subsystems.OverrideSource` / `OverrideSink` types and
`DataSystemConfiguration.OverrideSource`, plus
`ldcomponents.DataSystemConfigurationBuilder.Overrides(...)` for
configuration.
>
> **FDv2** now optionally builds an override layer and
`overrides.Overlay`, returns the overlay from `Store()` when configured,
starts the source (with an SDK `Sink`) **before** the run loop so
synchronous loads apply before evaluation, closes the source on
`Stop()`, and skips all of this when the data system is disabled.
Persistence, relay, and `DataAvailability()` keep using the underlying
store unchanged.
>
> Includes `internal/sharedtest.TestOverrideSource` and FDv2 integration
tests (overlay vs raw store, flag-change events, lifecycle).
>
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
843e62b. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
internal/overrides, the core of the flag-overrides feature (OVERRIDE spec): a runtime-mutable layer of flag and segment definitions that will take precedence over LaunchDarkly data at the store read boundary. Nothing consumes this package yet -- FDv2 wiring and the public source API come next -- so this PR is the data structures and their semantics in isolation.Layer: thread-safe holder of override entries, replaced wholesale on every update (an update is a full snapshot; entries absent from it are removed). Entries are stored as shallow copies carrying the typed-model override marker. The copy shares its nested slices with the source's entity and the layer never writes to them, so a source may retain its entities and supply them again. Entities from deserialization or the builders already carry their preprocessing caches; the evaluator scans values when a cache is absent. A read of an empty layer is a single atomic check, so a configured-but-unpopulated layer costs almost nothing per evaluation.Overlay: aReadOnlyStorethat merges the layer over a base store -- the override entry wins per key onGet(including over deleted-item tombstones),GetAllis the union, and initialization status delegates to the base untouched. When the base store fails and the layer holds entries,GetAllreturns the layer's entries alone with no error, so an all-flags read during a persistent-store outage still carries the overrides that per-key reads serve; an empty layer still returns the base error. Serving overrides from an uninitialized base works because an uninitialized memory store reports not-found rather than erroring.Sink: applies serialized layer replacements and computes which flags to notify. The seed set is every added, removed, or changed override entry -- added and removed keys always count, because the marker alone changes what an evaluation reports, and same-key changes compare by serialized form since the layer is rebuilt from scratch each time (pointer or version comparison would notify every retained entry, or nothing). Fan-out then runs through dependency trackers built over both the old and the new merged views, because a replacement can rewire dependencies -- e.g. removing a flag override restores the LaunchDarkly definition's prerequisite edges, and dependents of the override's own references only exist in the old view. A base read failure degrades fan-out but never drops the directly changed keys.Depends on go-server-sdk-evaluation carrying the ldmodel override marker (launchdarkly/go-server-sdk-evaluation#57); go.mod points at a pseudo-version of that branch and will be pinned to the released version before merge. Based on #404 (the dependency-tracker consolidation this reuses).
SDK-2655
Note
Overview
Introduces
internal/overrides, the isolated core for runtime flag/segment overrides: a wholesale-replacementLayer(marked shallow copies, no mutation of source entities), aOverlayReadOnlyStorethat wins on per-key reads and unions onGetAll(including overrides when the base is uninitialized orGetAllfails), and aSinkthat applies updates and notifies flag listeners using serialized diff seeds plus dependency fan-out across old and new merged views.Adds
internal/filedata.Poller, an interval-based multi-file watcher (mod time/size, create/delete) meant to complement or replace unreliable fs notifications and feed reload paths.go.modmoves to pseudo-versions ofgo-sdk-commonandgo-server-sdk-evaluationfor the ldmodel override marker; nothing in the SDK wires these packages yet.Reviewed by Cursor Bugbot for commit 8480fc6. Bugbot is set up for automated code reviews on this repo. Configure here.