F# hot reload: Edit-and-Continue delta emission behind --test:HotReloadDeltas#19941
Draft
NatElkins wants to merge 169 commits into
Draft
F# hot reload: Edit-and-Continue delta emission behind --test:HotReloadDeltas#19941NatElkins wants to merge 169 commits into
NatElkins wants to merge 169 commits into
Conversation
…eaps, row indexing, EncLog writer) Ported verbatim from the hot-reload prototype branch onto current main: - ILDeltaHandles.fs: typed handles for EnC delta metadata rows - ILMetadataHeaps.fs: heap accounting shared by baseline and delta passes - ILRowIndexing.fs: row index helpers for delta table emission - ILEncLogWriter.fs: IEncLogWriter abstraction recording EncLog/EncMap entries - ilbinary.fs: additive table/coded-index definitions required by EnC emission - EnvironmentHelpers.fs: FSHARP_HOTRELOAD_* environment variable helpers - Caches.fs: qualify System.Guid to avoid shadowing from new helpers All additions are dormant without the hot reload flag; no behavior change for normal compilation.
…ble name-map state - Generated/GeneratedNames.fs: central helpers for synthesized member/type names - Generated/CompilerGeneratedNameMapState.fs: capture/replay of name-generator state so successive hot reload generations synthesize identical names - CompilerGlobalState.fs: thread name-map state through NiceNameGenerator - IlxGen.fs: route compiler-generated local names through a single freshIlxName helper (replaces direct IlxGenNiceNameGenerator.FreshCompilerGeneratedName call sites) and add IlxGenEnvSnapshot/snapshotIlxGenEnv/restoreIlxGenEnv to capture the codegen environment a delta generation must replay - IlxGen.fsi: expose the snapshot API 3-way merge against current main verified: all 10 freshIlxName sites intact, no new upstream name-generator call sites bypass the helper, net diff vs main matches the prototype footprint exactly (+85/-49).
- TypedTree/SynthesizedTypeMaps.fs: stable maps for synthesized (closure/ state-machine) types across generations - TypedTree/TypedTreeDiff.fs: semantic diff between baseline and updated typed trees - binding/tycon snapshots, FNV-1a digests, rude-edit classification (signature changes, constraint changes, mutable-field toggles, type layout changes) - HotReload/DefinitionMap.fs: maps baseline definitions to updated symbols Static port validation against current main: every TypedTreeOps symbol consumed by TypedTreeDiff.fs resolves in the refactored namespace (upstream split TypedTreeOps.fs into TypedTreeOps.*.fs with [<AutoOpen>] modules inside namespace FSharp.Compiler.TypedTreeOps, so consumer opens are unchanged).
- ilwrite.fs/.fsi: thread IEncLogWriter through metadata emission; expose ILTokenMappings, MetadataHeapSizes, MetadataSnapshot and WriteILBinaryInMemoryWithArtifacts so a compilation can capture the token maps and heap layout a later delta generation must build on; markerForUnicodeBytes made public for delta #US emission (ECMA-335 II.24.2.4) - ilwritepdb.fs: portable PDB hooks for baseline capture - ILBaselineReader.fs: pure F# reader recovering baseline metadata state (row counts, heap sizes, GUID heap start) from emitted images - HotReloadBaseline.fs: baseline snapshot assembly for hot reload sessions - HotReloadPdb.fs: PDB delta support shared across generations 3-way merged cleanly against current main; upstream MethodDefKey/Codebuf changes (compareILTypes etc.) retained. Flag-off emission path is unchanged: full builds use the no-op EncLog writer (createNullEncLogWriter).
EnC delta (#~, #Strings, #US, #Blob, #GUID) construction, ported verbatim: - FSharpSymbolChanges.fs: symbol-level change set consumed by the writer - IlxDeltaStreams.fs: per-generation heap/stream builders (delta-local offsets, generation-aligned Blob/US heaps) - FSharpDefinitionIndex.fs: definition-to-token index over the baseline - DeltaMetadataEncoding.fs / DeltaMetadataTypes.fs / DeltaMetadataTables.fs: ECMA-335 II.22/II.24 row encodings, coded indices, table models - DeltaTableLayout.fs / DeltaIndexSizing.fs: row layout and wide/narrow index sizing for delta images - DeltaMetadataSerializer.fs: serializes EncLog/EncMap-ordered tables - DeltaMetadataSrmWriter.fs: System.Reflection.Metadata-based parity writer (FSHARP_HOTRELOAD_COMPARE_SRM_METADATA) - FSharpDeltaMetadataWriter.fs: top-level per-generation metadata delta writer (EncId/EncBaseId chaining)
- SymbolMatcher.fs: matches baseline symbols to updated tree symbols - HotReloadAccessorTypes.fs: synthesized accessor type support - IlxDeltaEmitter.fs: per-generation IL emission for added/updated methods, async/state-machine @hotreload type synthesis, rude-edit guarded (HotReloadUnsupportedEditException instead of failwith on unresolvable references) - HotReloadState.fs: per-session mutable state (generation chain, name maps) - DeltaBuilder.fs: orchestrates diff -> symbol matching -> emission - HotReloadCapabilities.fs: capability flags (Baseline, AddMethodToExistingType, ...) - HotReloadContracts.fs: cross-layer contracts for tooling integration - RudeEditDiagnostics.fs: rude-edit kinds and diagnostic mapping - EditAndContinueLanguageService.fs: Roslyn-shaped EnC entry point; rejects all rude edits before invoking the emitter - Adapters.fs: adapter layer between FCS session API and EnC service
- CompilerOptions.fs: --enable:hotreloaddeltas (baseline capture emission) and --enable:hotreloadhook (synthesized-name replay only) - CompilerConfig.fs/.fsi: emitCaptureArtifacts config and emit-hook wiring - CompilerEmitHookState.fs / CompilerEmitHookBootstrap.fs / HotReloadEmitHook.fs: pluggable emit hook capturing baseline artifacts (token maps, metadata snapshot, PDB) during normal fsc emission - fsc.fs: invoke the emit hook around main4-main6 binary emission - fsi.fs: qualify System.Guid (shadowing from new compiler-level opens) - FSComp.txt: fscHotReloadRequiresDebugInfo (2026), fscHotReloadIncompatibleWithOptimization (2027) - IDs verified free on current main; xlf regeneration deferred to a follow-up commit (requires a build with /p:UpdateXlfOnBuild=true) Flag-off compilations bypass the hook entirely (null EncLog writer).
- service.fs/.fsi: session surface the IDE/dotnet-watch layer consumes: FSharpChecker.StartHotReloadSession (2 overloads), EmitHotReloadDelta (2 overloads), EndHotReloadSession, HotReloadSessionActive, HotReloadCapabilities; plus FSharpHotReloadError, FSharpHotReloadDelta, FSharpAddedOrChangedMethodInfo, FSharpHotReloadCapability/-ies types - FSharpProjectSnapshot.fs: snapshot support for session baselines - FSharpCheckerResults.fs: expose typed-tree access needed by the differ (sessions require keepAssemblyContents=true) One textual conflict resolved in the service.fs open block (upstream added FSharp.Compiler.Caches at the same spot the prototype added ILDynamicAssemblyWriter/ILPdbWriter; kept all three). Upstream's new single-argument DiagnosticSink signature is unaffected - ported code implements no DiagnosticsLogger. Net diff vs main matches the prototype footprint (+927 service-layer lines).
- tests/FSharp.Compiler.Service.Tests/HotReload/ (18 files): unit tests for TypedTreeDiff, delta metadata writer, coded indices, ILBaselineReader, portable PDB reader, SRM parity, rude edits, thread safety, generated names, error/edge paths (258 tests on the prototype branch) - tests/FSharp.Compiler.ComponentTests/HotReload/ (15 files): integration tests - baseline capture, delta emission, runtime MetadataUpdater ApplyUpdate scenarios, mdv validation, PDB generations (101 tests) - FSharpWorkspace.fs: workspace plumbing used by hot reload session tests - HotReload.runsettings sets DOTNET_MODIFIABLE_ASSEMBLIES=debug and COMPlus_ForceEnc=1 for ApplyUpdate runs; NOTE: repo moved to Microsoft.Testing.Platform since the prototype - verify RunSettingsFilePath is still honored when running these suites - mdv-based tests honor FSHARP_HOTRELOAD_MDV_PATH Test fsproj compile items re-applied onto the MTP-migrated project files (anchors verified; 3-way merge clean).
…hook, release note - tests/scripts/: hot-reload-verify.sh (full pipeline gate), hot-reload-demo-smoke.sh (HOTRELOAD_SMOKE_RUNTIME_APPLY runtime apply), metadata coupling/parity and plugin-boundary guards, check-ilxgen-name-path.sh, main-fsi drift checks + allowlists - tests/projects/HotReloadDemo/: console demo app with hot reload session driver (hotreload-session.json + edited DemoTarget.fs); registered in FSharp.slnx under /Tests/HotReloadDemo/ (upstream migrated .sln -> .slnx since the prototype, so the old FSharp.sln entries were re-expressed) - eng/Build.ps1: Invoke-HotReloadDemoSmokeTest hook after testCoreClr and testDesktop runs, re-derived onto upstream's reworked test sections (TestSplit batching); runs the demo app with DOTNET_MODIFIABLE_ASSEMBLIES=debug and asserts the delta-emitted marker - hot-reload-verify.sh updated to build FSharp.slnx - tools/hot-reload/compare_roslyn.fsx: Roslyn delta comparison helper - docs: debug-emit.md hot reload heap tracing section, hot-reload-tgro-closure-matrix.md, release note entry in 11.0.100.md Deliberately not ported from the prototype branch: tmp.fsx (scratch), HOT_RELOAD_REVIEW_CHECKLIST.md and CLAUDE.md (workflow files, not product); xlf regeneration deferred (needs /p:UpdateXlfOnBuild=true build).
Mirror Roslyn's EditAndContinueCapabilities flags and parser semantics (exact-name matching, unknown capability words ignored, the AddDefinitionToExistingType aggregate) with an immutable typed model in FSharp.Compiler.EditAndContinue. Runtime capability strings are parsed once at the session boundary; a session always carries at least the Baseline capability. Includes parser tests and a design doc.
…bilities Thread the typed capability model through the hot reload session: FSharpChecker.StartHotReloadSession accepts an optional capabilities string sequence (Roslyn WatchHotReloadService parity), parsed once and stored on the session state; when omitted the session is conservative and assumes baseline-only support. Method additions now require the AddMethodToExistingType runtime capability. Without it the diff reports the new RudeEditKind.NotSupportedByRuntime (FSHRDL016) naming the missing capability, mirroring Roslyn's distinction between edits unsupported by hot reload and edits the connected runtime cannot apply. Classification consults a single capabilityForAddition seam so Phase B can flip field additions from always-rude to capability-gated by implementing emission. Rude-edit details now flow into the UnsupportedEdit error so hosts see the missing capability.
… additions Module-level values lower to a static field plus accessor methods with initialization in the startup class constructor, so adding one requires both AddStaticFieldToExistingType and AddMethodToExistingType; module-level functions lower to plain static methods and require only the latter. Both were previously unconditional DeclarationAdded rude edits. Instance fields remain rude until field-row emission lands (Phase B2); an emission-level test pins the cut line so a capability-enabled host gets a clean UnsupportedEdit rather than a half-emitted delta.
Adds Field-table support end-to-end in the delta writer pipeline: FieldDefinitionRowInfo row model, mirror table population, serializer table wiring, and SRM shadow-writer parity (including Field in the tracked parity tables). EncLog follows the Roslyn pattern recorded from a hotreload-delta-gen C# reference delta that adds a static field with an initializer: the parent TypeDef row is logged with the AddField operation immediately followed by the new Field row with the Default operation, and only the Field row enters EncMap. The pairs are spliced between the Module entry and Method entries so per-table EncLog sorting cannot separate them. Writer-level test asserts the exact EncLog sequence, EncMap content, and serialized table stream for an added static int32 field.
Added module-level values (let mutable x = ...) now emit complete, runtime-appliable deltas: the static backing fields on the startup-code class enter the Field table, accessors and the startup constructor are emitted as added methods, and the added value is readable/writable via its accessors after MetadataUpdater.ApplyUpdate. Making the deltas actually apply required aligning the EncLog with what CoreCLR's EnC applier (CMiniMdRW::ApplyDelta) and Roslyn emit for ALL added member kinds: an added member logs its PARENT row tagged with the Add* operation immediately followed by the member row with the Default operation (TypeDef for methods/fields, MethodDef for parameters, PropertyMap/EventMap for properties/events; map rows and MethodSemantics rows are plain Default entries). The previous shape - the Add* op on the member row itself - corrupted parent member lists at apply time and had never been runtime-applied by any test. Further fixes uncovered by the runtime-apply tests: - EditAndContinueOperation numeric values corrected to the CLR/SRM codes (AddParameter=3, AddProperty=4, AddEvent=5; previously 4/5/6). - Baseline #Strings size now uses SRM's trimmed size (StringHeap.TrimEnd semantics, Roslyn EmitBaseline parity); the padded stream size shifted every delta-heap string reference. - Added member names/signatures are written into the delta heaps instead of reusing fresh-compile heap offsets, with signature blobs remapped through remapSignatureBlobWith. - Return-parameter rows are no longer synthesized: the out-of-sequence seq-0 rows forced the CLR's indirect ParamPtr path and made ApplyUpdate reject the delta. ParamList of added methods stays monotone instead. DeltaBuilder resolves the startup-code class constructor as an updated method when the baseline already contains it (it is not a typed-tree binding, so the diff cannot pair it). Added field tokens chain into the next-generation baseline like added methods. Initialization semantics (validated at runtime and documented): the initializer runs lazily if the startup class was not yet type-initialized (the test observes 41); an already-initialized type reads default(T). New tests: checker-level delta validation for the added value, mdv validation of the Field rows and AddField pairing, and runtime ApplyUpdate tests for both added module functions and added module values. Instance field additions remain rejected (Phase B2).
…ule values Generation 2 adds a second mutable module value on top of the generation-1 field addition and applies it with MetadataUpdater.ApplyUpdate. This pins baseline chaining: generation-1 field/method/property tokens resolve in the chained baseline so only the new value is appended, and the startup-code constructor added in generation 1 is re-emitted as an updated method body. Also pins the already-initialized regime of the initialization semantics: the startup class was type-initialized in generation 1, so the second value reads default(int) rather than its initializer, while generation-1 state is preserved across the update.
… the typed-tree diff Replace the blanket lambda-shape digest equality check with a structured per-member lambda occurrence model: - Occurrence identity = traversal ordinal + parent-lambda chain + structural digest (curried arity, per-group parameter type identities, capture identity list, return type identity). Consecutive curried lambdas form one occurrence, matching IlxGen closure formation. Source ranges are diagnostics-only. - Old/new alignment = two LCS passes (full structural digest, then shape-only), so inserting/removing/reordering lambdas aligns the surviving occurrences instead of reporting spurious removed+added pairs. - Capture compatibility per matched pair with C#-parity classification: RenamingCapturedVariable, ChangingCapturedVariableType, ChangingCapturedVariableScope (cross-occurrence move post-pass), plus additions/removals (additions may become applicable in C4 via AddInstanceFieldToExistingType). - Classification: pure body edits within an unchanged lambda set remain plain MethodBody edits; lambda-set changes stay LambdaShapeChange rude edits, now with counts/ordinals in the message and a structured LambdaEdits payload on TypedTreeDiffResult for the C4 emitter. - Quotations, object expressions, local type functions, and types without a computable runtime identity keep the legacy whole-body digest path unchanged. Move the closure-mapping design doc into docs/ with C1 implementation notes.
Add EncMethodDebugInformation (CodeGen), replicating byte for byte the three Roslyn EnC CustomDebugInformation blob formats persisted per method in portable PDBs (EnC Local Slot Map, EnC Lambda and Closure Map, EnC State Machine State Map), including the syntax-offset-baseline optimization and the slot-map kind/ordinal byte packing. In F#, the syntax-offset slots carry C1 occurrence keys packed from the occurrence ordinal chain (16-bit segments, fail-closed past limits). Tests cover round-trips (temps, ordinal-flagged slots, negative baselines, static/this-only closure ordinals, negative state numbers), golden bytes, fail-closed limits, and cross-validation: a C# library built with the repo SDK has its Roslyn-emitted CDI rows decoded with the new decoder and re-encoded byte-identically. PDB writer/reader wiring is the next C2 commit.
… hotreloaddeltas When --enable:hotreloaddeltas is on, the fsc emit path computes per-method lambda occurrence data with the Phase-C1 extraction over the same optimized typed tree the baseline capture snapshots, serializes it with the C2 blob encoders, and carries it into the portable PDB writer through a new methodCustomDebugInfoRows side channel on the IL writer options: - TypedTreeDiff.collectMemberLambdaOccurrences: public C1 extraction over a CheckedImplFile, returning every member binding (empty occurrences for members the model cannot represent). - EncMethodDebugInformation.computeMethodCustomDebugInfoRows: occurrences -> EnC Lambda and Closure Map blobs keyed by IL method (compiled) name, with occurrence keys packed from the C1 ordinal chains; MethodOrdinal stays UndefinedMethodOrdinal; one closure scope per occurrence (IlxGen lowers every occurrence to its own closure class). The EnC Local Slot Map is omitted: the lowered slot layout is an IlxGen artifact, not derivable from the typed tree. - ilwritepdb attaches the rows as CustomDebugInformation on MethodDef parents, but only when the method name identifies exactly one method row; the producer likewise drops compiled names claimed by more than one member, so a map can never attach to the wrong method (overloads fail closed and simply carry no map). Flag-off builds pass the empty map everywhere and emit byte-identical output (EmittedIL gate: 1212 passed, 0 failed). New PdbCdiEmissionTests verify the flag-on PDB carries a decodable map with occurrence keys [0] and [0; 1] for a nested-lambda sample, and that the flag-off PDB contains no EnC CDI rows.
…sion When a baseline is captured, decode every method-level EnC CustomDebugInformation row of the baseline portable PDB (lambda/closure map plus the slot and state-machine maps) into FSharpEmitBaseline.EncMethodDebugInfos, keyed by MethodDef token - the CDI parent is a MethodDef handle, so token keying is unambiguous, unlike the name keying on the write side which exists only because the PDB writer lacks tokens. The fsc emit hook decodes the exact emitted PDB bytes; the checker path additionally reads the on-disk PDB as a sibling input because its in-memory baseline rewrite carries no CDI side channel. Fail safe/fail closed: flag-off and pre-C2 PDBs decode to the empty map (sessions start fine), and a method whose blobs do not decode is omitted rather than guessed. Generation chaining: EmitDeltaForCompilation recomputes per-method occurrence data from the fresh typed tree (computeRefreshedEncMethodDebugInfos, name-to-token resolution fail closed on non-unique names) and chainEncMethodDebugInfos replaces the updated methods' entries in the next-generation baseline, dropping entries the fresh compile did not produce so stale data can never be matched. The delta PDB does not yet re-emit EnC CDI rows; the in-memory chain is what C3 consumes, and PDB persistence across session restarts is the documented remaining gap.
Add ClosureNameAllocator, the F# analogue of Roslyn's
EncVariableSlotAllocator.TryGetPreviousLambda/TryGetPreviousClosure
expressed over the C1 lambda occurrence model: a matched occurrence
(same two-pass LCS as the C1 diff, equal capture sets, recorded
baseline name) reuses the baseline closure class name verbatim; an
unmatched, capture-incompatible, or unmappable occurrence gets a fresh
generation-suffixed name ({base}@hotreload#g{gen}_o{ord}, the
DebugId(ordinal, generation) analogue); removed occurrences fall out of
the chain-forward table so their names are never reused.
The index-pair core of the C1 alignment is extracted into
TypedTreeDiff.alignLambdaOccurrenceIndexPairs and shared by both
consumers, so diff classification and name allocation can never
disagree about pairing. Pure data transformation: no IO, no IlxGen
state; covered by ClosureNameAllocatorTests (match/add/remove/nested/
capture-incompatible/fail-closed plus three-generation chaining).
Gates: service HotReload 365/365 (356 + 9 new), component HotReload
134/134.
Two layers of coverage for the C3 naming machinery: - ClosureNameAllocatorTests now also drives the allocator over REAL C1 extraction (checker compiles via the shared DiffTestHarness, made internal for reuse): adding a filter lambda yields a generation-2 fresh name while the surviving map lambda reuses the baseline name, and generation 3 reuses both names from the chained table. - New component ClosureIdentityTests pins the metadata-level invariant the delta path relies on today (and C4 extends): flag-on recompiles of an unchanged lambda set produce closure classes with identical names across three body-edit generations, so deltas can update the existing closure method bodies in place. Gates: component HotReload 135/135, service HotReload 366/366, EmittedIL 1212 passed / 3 skipped / 0 failed.
Bridge the C1 lambda occurrence model to IlxGen's closure naming seam so the occurrence-keyed allocator can be wired into delta compiles: - LambdaOccurrence gains RootExprStamp, the unique stamp of the occurrence's outermost Expr.Lambda (extraction bookkeeping, never part of the structural digest or alignment). - GetIlxClosureFreeVars records stamp -> emitted closure type name into a new ConditionalWeakTable side channel (ClosureNameAllocationState, mirroring CompilerGeneratedNameMapState); recording is armed only by the emit hook for capture compiles, so flag-off output stays byte-identical. - The fsc emit path joins the recording with the same tree's occurrence extraction (ClosureNameAllocator.computeBaselineClosureNameRows, fail closed on ambiguous compiled names and on members with incompletely recorded occurrences) and threads the per-method chain -> name tables through TryEmitWithArtifacts/CompilerEmitArtifacts into the baseline capture, where they are re-keyed by MethodDef token and stored as FSharpEmitBaseline.EncClosureNames alongside EncMethodDebugInfos. Gates: compiler + both test projects build clean; component HotReload 136/136 (135 + new baseline-capture test), service HotReload 366/366, EmittedIL 1212/1212 (3 skipped).
Thread the C3 allocator into IlxGen lowering for session compiles:
- ICompilerEmitHook.PrepareForCodeGeneration now receives tcGlobals and the
optimized impl files about to be lowered. When a session with baseline
closure-name tables is active, the hook runs
HotReloadBaseline.computeOccurrenceKeyedClosureNames (previous-generation
occurrences vs fresh extraction, allocator per member with
generation = session.CurrentGeneration) and installs the resulting
stamp -> assigned-name table on the compiling CompilerGlobalState.
- The IlxGen closure call site consults the table FIRST and falls back to
sequence replay; the replay-map slot is still consumed unconditionally so
same-basename non-closure names keep their baseline replay positions.
Surviving closures therefore reuse baseline class names verbatim even when
the lambda set changes; added occurrences get {base}@hotreload#g{N}_o{i}.
- DeltaEmissionRequest.RefreshedClosureNameRows carries the allocator's
refreshed per-method tables (recomputed deterministically at delta emission)
and chainClosureNameRows chains them into the next-generation baseline with
the chainEncMethodDebugInfos replace-or-drop semantics.
- Fail closed everywhere: no session, empty baseline tables, ambiguous
compiled names, or unresolvable tokens leave lowering on pure sequence
replay; flag-off compiles see a single failed weak-table lookup.
Gates: compiler + both test projects build clean; component HotReload 138/138,
service HotReload 366/366, EmittedIL 1212/1212 (3 skipped); the
multi-generation closure-identity and body-edit runtime tests now exercise the
allocator path live.
… in delta compiles
Component coverage for the C3 lowering wiring, driven through the session/hook
plumbing (flag-on compiles against an active session):
- three body-edit generations of a 2-lambda method keep closure class names
byte-identical AND keep the chained session tables carrying the same names
(the compiles now take the allocator path, not bare sequence replay);
- a recompile with a lambda ADDED IN FRONT of the survivors — the case
sequence replay cannot handle — keeps every baseline closure name verbatim
and gives the added occurrence the generation-suffixed
{base}@hotreload#g1_o{i} name, which also chains forward for reuse.
Classification still rejects lambda-set changes at delta emission; emitting
the added member is Phase C4, documented in the design doc.
Gates: component HotReload 138/138, service HotReload 366/366.
Phase C4 sub-slice 1: writer + emitter support for ADDED TypeDef rows,
mirroring the Roslyn reference delta for "method gains its first capturing
lambda" (new display class). Reference recorded with a compiler-level
Compilation.EmitDifference harness (hot_reload_poc/src/csharp_enc_reference,
Roslyn 5.9.0) because the hotreload-utils workspace tool needs a net11-only
toolchain; the IDE pipeline ends in the same API, so the delta shape is
identical. Template captured verbatim in docs/hot-reload-member-additions.md.
Writer (FSharpDeltaMetadataWriter.emitWithTypeDefinitions):
- TypeDefinitionRowInfo / NestedClassRowInfo row models; TypeDef rows write
FieldList/MethodList as 0 (Roslyn EnC parity - members are linked through
the AddField/AddMethod EncLog pairs).
- EncLog: new TypeDef rows are plain Default entries placed before every
AddField/AddMethod pair that names them as the parent; NestedClass rows
are plain Default entries trailing the log. Both appear in EncMap.
- Serializer/SRM shadow writer/parity comparer extended for both tables.
Emitter (IlxDeltaEmitter):
- A fresh-compile type with the C3 allocator's generation-suffix marker
({base}@hotreload#g{N}_o{i}) and no baseline TypeDef token allocates the
next delta TypeDef row; its fields/methods (including instance capture
fields and the .ctor/Invoke pair) register as added members parented to
the NEW row. Extends is remapped (TypeRef/TypeDef); TypeSpec base types
rely on baseline passthrough and fail closed past the baseline table, as
do generic closure classes (GenericParam rows unsupported) - both gaps
documented.
- Added type tokens chain into the next-generation baseline TypeTokens and
are reported in UpdatedTypeTokens (Roslyn ChangedTypes parity).
Tests: exact-EncLog writer test mirroring the C# template (service), plus
emitter tests for a hand-constructed nested closure type incl. mdv
validation (component). Component HotReload 140/140, service HotReload
367/367.
Phase C4 sub-slice 2: the payoff slice. A member that gains a lambda now
produces an applicable delta carrying the new closure class.
Classification (TypedTreeDiff): Added-only lambda sets become method-body
edits when the runtime advertises NewTypeDefinition+AddMethodToExistingType,
otherwise NotSupportedByRuntime naming the missing capability (C# parity).
Removed-only sets are allowed at Baseline capabilities (deleted lambda
bodies just become unreachable; the baseline closure class stays unused).
Capture-set changes stay rude.
Emission: the delta compile's fresh rewrite contains the C3 allocator's
{base}@hotreload#g{N}_o{i} closure class; the emitter selects it as an added
TypeDef (sub-slice 1 machinery), and generation-suffixed names are excluded
from basic-name alias buckets (they never alias a baseline closure). The
type token chains into the next-generation baseline, so gen N+1 body-edits
the added lambda in place.
Wiring fixes the runtime test forced:
- checker.StartHotReloadSession rebuilds the baseline from disk, which
cannot carry EncClosureNames (CDI blobs have no name slots); it now
carries the tables over from the in-process capture session it replaces
when the MVID matches. Without this every watch-flow delta compile fell
back to sequence-replay naming.
- MemberRef/TypeSpec token passthrough is now content-validated: an added
lambda shifts the fresh compile's reference-row order, and positional
passthrough silently swapped ListModule.Filter/Map MethodSpec bindings
(BadImageFormatException at invoke). The baseline snapshots MemberRef row
contents and TypeSpec blobs (ILBaselineReader, which also gained the
missing InterfaceImpl row size - assemblies with interface impls had all
later table offsets misread); the remapper validates positional reuse,
falls back to content search, then appends (MemberRef) or fails closed
(TypeSpec). Appended MemberRef rows chain forward for later generations.
- remapTypeSpecBlobWith: TypeSpec blobs are a bare Type (II.23.2.14), not a
calling-convention-prefixed signature.
- The AsyncStateMachineAttribute heuristic now requires a MoveNext method,
so closure classes sharing the {method}@hotreload naming no longer pick
up a spurious attribute.
Runtime evidence (ApplyUpdate succeeds for added lambda creating a new
closure class): gen1 adds a third lambda in front of two survivors -> delta
EncLog carries the new TypeDef row + AddField/AddMethod pairs + NestedClass,
ApplyUpdate accepts, probe observes the new behavior; gen2 body-edits the
ADDED lambda -> method updates only, no new TypeDef rows.
Updated classification tests to the new semantics (added-without-capability
is NotSupportedByRuntime naming NewTypeDefinition; removed-only is a body
edit; async closure-chain shape changes follow the same gating).
Gates: component HotReload 141/141, service HotReload 367/367,
EmittedIL 1212/1212 (3 skipped). Both design docs updated.
Phase C4 sub-slice 3: - Negative runtime gate: a capability-less session rejects an added lambda with NotSupportedByRuntime (FSHRDL016) naming NewTypeDefinition, before any emission. - Removed-lambda runtime test: removing a lambda applies as a plain set of method-body updates (no TypeDef table entries; the baseline closure class stays, unused) and the new behavior takes effect - pinning the C#-parity allowance enabled in sub-slice 2. - mdv/EncLog pattern-parity assertions on the real generation-1 delta against the recorded C# new-display-class template: the new TypeDef row's plain Default entry precedes its Add* entries, every AddField/AddMethod parent entry is immediately followed by the member row with the Default operation, the NestedClass row trails, and EncMap carries the TypeDef and NestedClass rows but never the Add* parent entries. Gates: component HotReload 143/143, service HotReload 367/367.
… conduit The C2 PDB writer wiring added the methodCustomDebugInfoRows option to ilwritepdb.fsi; record the intentional drift and refresh the locked hashes.
Phase B2: instance fields added to existing CLASSES are emitted with the same (TypeDef, AddField) + (Field, Default) EncLog pairing as the B1b static path, validated against a recorded Roslyn EmitDifference reference (csharp_enc_reference field_add scenario) and applied at runtime. Classification: compareEntities now diffs the field segment structurally; a pure field addition on a TFSharpClass is gated on the negotiated AddInstanceFieldToExistingType/AddStaticFieldToExistingType capabilities (RudeEditKind.NotSupportedByRuntime names the missing one) instead of reporting TypeLayoutChange. Struct/record/union/enum layouts stay rude permanently (runtime restriction, C# identical), enforced fail-closed in the emitter via the fresh TypeDef's IsStructOrEnum. Constructor pairing: a mutable class let binding folds its initializer into the primary ctor, whose binding diffs as a plain MethodBody update; ctor return-type identity is now void (the IL truth) so .ctor edits resolve against baseline method tokens. [<DefaultValue>] val mutable needs no pairing: the entity surfaces as a TypeDefinition edit and the delta is a pure Field-row append - the metadata writer's empty-delta short-circuit now keys on row payload, not method updates alone. Runtime tests assert C# EnC semantics: existing instances read zeroed values for the added field, new instances run the updated ctor and see the initializer, and state survives multi-generation chains.
BaselineMetadataReader's small members (TypeRefCount and friends) get cross-module
inlined in Release builds, and the inlined code references the module-private
MetadataContext record directly. The CLR then rejects the field access at runtime
('Attempt by method ... to access field ... RowCounts@ failed') when the inlined
copy lives in another file's state machine, which broke hot reload session start
entirely on Release FSharp.Compiler.Service: every edit silently fell back to
rebuild and restart. Debug builds never inline these members, which is why the
suites stayed green.
Make MetadataContext internal instead of private: same-assembly access is what the
inliner needs, and the type remains hidden outside the assembly.
Measured with the fix on a 72-file app under dotnet watch (Release FCS, in-process
compile flag on): median ~1.6s per edit applied in place with state preserved,
versus ~1.9-2.6s for a plain per-edit dotnet build on the same machine. Debug
suites unchanged: service HotReload 414 passed, component 229 passed.
computeSymbolChanges diffed every file's baseline tree against the fresh tree on every edit, making the diff cost proportional to project size rather than to the edit. When the TransparentCompiler returns the same CheckedImplFile instance the committed baseline holds, nothing in that file can have changed, so the pair is skipped. Reference identity is the sound gate here: source-text equality is not, because F# type inference propagates across files and a downstream file's typed tree can change without its source changing (such files come back as fresh instances and are still diffed). Covered by a session test proving unchanged earlier files are reference-equal between the committed baseline and the fresh check while the edited file is not. Trace under FSHARP_HOTRELOAD_TRACE_METHODS reports skipped vs diffed counts. Service HotReload suite: 415 passed, 0 failed.
…iles computeRefreshedEncMethodDebugInfos and computeOccurrenceKeyedClosureNames walked every implementation file on every edit, even though EmitDelta chains their rows only for the updated method tokens. Both now accept an ImplementationFileScope: ReferenceChanged pairs the committed baseline files with the fresh check's files by qualified name and skips pairs whose CheckedImplFile instances are reference equal, the same sound gate the typed-tree diff uses (source-text equality is not sound under cross-file type inference; reference identity is). Fail-open everywhere: duplicate file keys, added, removed, or renamed files, or a missing committed baseline fall back to the full walk. The codegen-time closure allocator install keeps the full walk; only the delta-emission refresh is scoped. Name-collision note: member-occurrence keying is fail-closed on globally unique compiled names. Under scoping, a fresh-only collision between a changed and an unchanged file would not be detected as ambiguous, but the binding still cannot misfire: baseline token lookup stays baseline-global (duplicates are excluded there), and refreshed rows are only consumed for updated method tokens, which an unchanged file cannot produce. Service HotReload suite: 415 passed, 0 failed. Component HotReload suite: 229 passed, 0 failed.
The in-process compile re-optimized every implementation file on every edit. At the dev-loop settings this path always uses (--optimize- with minimal passes), per-file optimization from a fresh environment is byte-identical to the threaded batch fold - verified by an A/B experiment serializing both variants of a 12-file stress project (shared anonymous records, literals, ordered dependencies) and a 73-file application to identical SHA256s. Whole-assembly IlxGen is unchanged: a prior experiment falsified per-file codegen (assembly-canonical anonymous types, last-file entrypoint semantics, shared PrivateImplementationDetails). Under FSHARP_HOTRELOAD_INCREMENTAL_EMIT, each file's optimized tree is cached in a ConditionalWeakTable keyed by the input CheckedImplFile reference: the checker returns reference-identical instances for unchanged files, so a hit is exactly what re-optimizing would produce, and the table's lifetime rides the checker's own typed-tree cache. Misses optimize that file alone from the fresh environment. Any failure falls open to the threaded batch path; with the flag unset the existing path runs untouched. Nothing accumulated by the batch fold is consumed downstream on this emit path: the optimizer info is discarded and the optimization-data resource list is empty, so inputs to codegen and module creation are identical apart from the cached work. Tests: flag-on bytes equal flag-off bytes after a one-file edit against a warmed cache (pins the independence result); a session with both flags applies two successive edits with exactly one updated method each (cross-generation splice). Service HotReload suite: 417 passed, 0 failed. Component: 229 passed, 0 failed.
Per flag-on edit the fresh module was serialized twice: once by the in-process compile writing the output assembly, and again inside the delta emitter, which re-serialized the whole module in memory purely to obtain PE bytes, a PDB, and token mappings. The session also re-read the freshly written PDB from disk. CompileFromCheckedProject now returns the emitted assembly bytes, PDB bytes, and token mappings alongside the parsed module, and the session threads them into the delta emission request. The emitter consumes them in place of its re-serialization; absent artifacts (the external-build path and the low-level test entry points) keep the existing rewrite, and the sibling-PDB disk read remains the fallback when no PDB bytes were threaded. The writer's own token-mapping closures are not substitutable here: they are keyed by writer-side signatures and can throw when invoked with read-back IL objects. The threaded mappings are instead built from the read-back objects' metadata row indices, which come from parsing the exact emitted bytes, so tokens correspond to the same bytes the emitter's PEReader consumes. The threaded PDB is strictly better than the rewrite's, which carries no sequence points for disk-read modules. No delta byte-equality A/B test: each emit mints a fresh EncId by design, so equivalent emissions are never metadata-byte-identical; coverage comes from the extended session test asserting the delta PDB parses with method debug information on a line-shift edit, plus both suites. Service HotReload: 417 passed, 0 failed. Component HotReload: 229 passed, 0 failed.
Per-edit latency work so far has relied on ad-hoc instrumentation (temporary printfn patches or external log scraping), which cannot decompose a live watch session and keeps getting rewritten. This adds a permanent trace under FSHARP_HOTRELOAD_TRACE_TIMING, following the existing FSHARP_HOTRELOAD_TRACE_* lazy-flag pattern, printing one '[fsharp-hotreload][timing] <stage>=<N>ms' line per stage. Session stages (EmitHotReloadDelta): parseAndCheck, inProcessCompile (when the in-process flag runs it), staleCheck, readIlModule (when the disk path runs), emitDelta, and total. Emission stages (EmitDeltaForCompilation): symbolChanges, buildRequest (symbol-to-token mapping through the refreshed EnC and closure-name tables), and emitCore. The flag is read once lazily; with it unset no stopwatch is created and the only cost is a boolean check per call. The timed bodies are wrapped by two small combinators (timeAsync/timeSync) rather than inline stopwatch plumbing, which re-indents the emission body but does not reorder or alter any operation; the full HotReload service suite (417) and component suite (229) pass unchanged. Motivation: decomposing the remaining per-edit cost between the FCS stages and the dotnet-watch pipeline decides whether further work targets incremental codegen (per-file IlxGen has five documented cross-file couplings to solve) or watch-side overhead. That decision needs stage numbers from real watch sessions, not estimates.
FSHARP_HOTRELOAD_DUMP_DELTAS=<dir> persists each emitted delta's metadata, IL, and PDB blobs plus the updated token list, so a live watch session's output can be decoded offline. Best-effort writes; unset means no behavior change. This was built to investigate a suspected stale-string bug on a real Giraffe app, where it let the actual session delta be decoded and proven correct (the report was a benchmark polling an endpoint the edited handler never served): correct method row in the EncLog, and the updated body's ldstr operand resolving to the appended delta user-string entry carrying the new literal. Also adds a session test decoding an emitted delta's IL to assert the ldstr operand points past the baseline user-string heap into the delta's appended entry with the edited string, pinning the absolute-offset contract for delta user strings against a Giraffe-shaped module (eta-expanded function-typed binding among sibling closures). Service HotReload suite passes.
…ed baselines The in-process hot reload compile cleared all closure naming state and emitted line-based closure names. Against a flag-on baseline (--test:HotReloadDeltas, stable occurrence-derived names plus EnC CDI) those names mis-map: closure-bearing methods hit the ambiguous-synthesized-mapping rude edit, so any project opting into stable baselines lost the in-process fast path entirely. Meanwhile flag-off baselines suffer the line-index problem: a line-adding edit renames every closure below it and falls back to restart, which on closure-dense real-world files (a Giraffe endpoints list) makes restarts the common case. CompileFromCheckedProject now takes a naming mode. Clear mode is the existing behavior for line-based baselines. Preserve mode mirrors the fsc emit hook's PrepareForCodeGeneration discipline: the session installs the project's synthesized-name map, and the compile computes the stamp-keyed closure-name replay over the OPTIMIZED implementation it is about to lower (checked-tree stamps are not the stamps IlxGen consults; computing over optimizedImpls matches the hook and was required for the replay to hit) and installs it via ClosureNameAllocationState before IlxGen. The session selects the mode from the baseline itself: no EncClosureNames means a line-based baseline and clear mode, otherwise preserve. Cleanup runs in a finally exactly like the hook's. Acceptance test: flag-on baseline, in-process compile, a line-adding edit above sibling module-level lambdas emits Ok with exactly one user-authored updated method; on the previous code this shape failed closed as FSHRDL099. Service HotReload suite: 420 passed, 0 failed. Component suite: 229 passed, 0 failed.
Two synthesized-closure families could not be aligned across compiles after a
line-adding edit, so such edits always fell back to restart on closure-dense
real-world files (reproduced on Giraffe's endpoint list, failing identically on
the external and in-process paths):
- debug-pipe closures embed the source line in the basic name itself
('Pipe #1 stage #2 at line 28'), so a line shift changes the name and the
fresh type has no baseline counterpart;
- value-binding list closures use sequential ordinals (endpoints@hotreload-N)
allocated in encounter order, so shifts make one fresh name match several
baseline candidates.
Add a last-resort positional pairing to the delta emitter's synthesized-type
match ladder, consulted only after exact and path-normalized matching fail or
tie. Groups are keyed by enclosing type plus a line-normalized basic name; the
normalizer recognizes only the known generated forms (pipe labels, @hotreload
ordinals, legacy @line ordinals) and refuses everything else, including
occurrence-keyed names, which already match exactly. Pairing requires equal
group counts, distinct ordinals on both sides, an unused baseline token, and a
structural shape match per pair (generic arity, base type, interfaces, field
type multiset, method name/arity set, with self-references normalized), and
produces a bijection; any failure keeps today's fail-closed rude edit, so a
structural change such as adding a pipe stage or list lambda still requires a
rebuild rather than risking a mispair, which would be silent corruption.
Baseline synthesized-type shapes are derived from the module at session start
and chained for delta-added types; no persisted format changes.
This is the recovery layer: it makes line-shift edits on these families apply
in place when the replayed names drift. A follow-up addresses the drift at its
source by keying the replay name map on line-normalized identity so matched
closures keep their generation-0 names, Roslyn's model; exact matching then
succeeds without inference and this pairing remains a guarded fallback.
Component acceptance: in-process and disk-started line-shifted pipe closures
and ordinal-shifted value-list closures each emit with exactly one user-authored
updated method; count-mismatch cases stay UnsupportedEdit. Service suite 423
passed, component 234 passed (2 expected manual-host skips).
Replay of compiler-generated closure names was keyed by the raw basic name, and
debug-pipe closures embed the source line in the basic name itself ('Pipe #1
stage #2 at line 28'), so any line-adding edit changed the bucket key, missed
the recorded generation-0 allocation, and produced fresh names the baseline
could not match. Value-binding list closures shared a bucket whose sequential
ordinals depended on encounter order and drifted similarly. Both families then
depended on the positional-pairing recovery layer or fell back to restart.
Key replay buckets by the line-normalized basic name and keep the original
generation-0 full names as the values, so a matched closure whose code moves
from line 28 to line 30 is handed back its line-28 birth name. This is Roslyn's
Edit and Continue model: synthesized-member identity is established at first
allocation and replayed exactly, making downstream matching exact rather than
inferred. The normalization grammar is extracted to one shared home in
GeneratedNames and reused by the map, the baseline snapshot collector, the
symbol matcher, and the delta emitter's pairing layer, which now acts purely as
a recovery net behind exact matching.
Snapshot compatibility is handled at load: raw legacy keys are normalized,
duplicate buckets merge, replay buckets canonicalize by ordinal, and gapped
buckets recover the unambiguous raw birth basic name from recorded entries
rather than synthesizing a name that would lose the original line text (a
normalized-key fallback remains for genuinely ambiguous recovery). The
ICompilerGeneratedNameMap surface and the persisted Map<string, string[]>
snapshot shape are unchanged; behavior without an installed map is untouched.
Acceptance: the flag-on in-process session test decodes TypeDef names from the
baseline and fresh assemblies after a line-adding edit and asserts exact name
equality for the unchanged closure families. Service HotReload suite: 426
passed, 0 failed. Component suite: 234 passed, 0 failed, 2 expected skips.
…cation Value bindings like an EndpointRouting endpoints list produce MIXED closure allocation: some closures occurrence-keyed, others sequence-replay slots with consume-then-override gaps. The occurrence names for such a binding may not be covered by any reconstructed EncClosureNames table (those tables cover method owners like handlers), but the synthesized-name snapshot preserves the complete consumed-slot layout, including the generation-0 occurrence names, and is sufficient to replay the mixed allocation exactly. The in-process compile's naming-state install was gated on EncClosureNames being non-empty alone. A baseline whose occurrence coverage lives only in the snapshot then compiled in clear mode, consumed sequence slot 0 as a plain replay name where the baseline had an occurrence override, and the divergent name failed the synthesized-type mapping (gracefully, as a rude edit and restart). Widen the gate: install the snapshot when EncClosureNames is non-empty OR when the snapshot itself contains occurrence-keyed closure names, the direct evidence that mixed slot consumption exists. Pure replay snapshots without occurrence names remain insufficient evidence, since broad installation for empty-table baselines risks unsafe slot filling; that shape keeps the current fail-closed behavior, which is also the old-baseline grace path. No CDI, snapshot format, or API changes. Acceptance: a reduced EndpointRouting shaped mixed bucket, verified by decoding baseline PE TypeDefs to show both naming families, applies a line-adding edit with exactly one user-authored updated method and byte-equal endpoint closure names in BOTH the in-memory capture and disk-started topologies; the pure-replay empty-table shape still fails closed. Service HotReload suite: 428 passed, 0 failed. Component suite: 234 passed, 0 failed, 2 expected skips.
…ne PDB Disk-started hot reload sessions previously rebuilt the synthesized-name replay buckets by scanning the baseline module's TypeDef and member names, but IL enumeration order does not record allocation slots, and for buckets that mix sequence-replay names with occurrence-keyed overrides the slot layout is not inferable from the binary at all. Two reconstruction heuristics prove the point by counterexample: keeping raw IL order satisfies the synthetic session shapes but mis-slots a real EndpointRouting endpoints bucket (the fresh compile hands slot 0 a name whose own suffix pins it to slot 2, and the edit degrades to a rude-edit restart), while pinning replay suffixes and filling holes in occurrence-chain order fixes that case but breaks shapes where chain order is not slot order. The layout has to be recorded, not inferred. Record it at the source: flag-on baseline emits (both the fsc capture hook path under --test:HotReloadDeltas and the in-process CompileFromCheckedProject path) now write the name map's allocation-ordered snapshot as a new F#-owned module-level CustomDebugInformation row (kind 49DDB47E-9C74-46EC-8626- 0350676571EB) in the portable PDB, with a deterministic payload (version, sorted bucket keys, names in recorded slot order; no paths, no timestamps). Existing EnC method CDI wire shapes are untouched, and flag-off compiles emit nothing new. Session start prefers the recorded snapshot and loads it verbatim (LoadRecordedSnapshot skips IL-order canonicalization); provenance is a SynthesizedNameSnapshotSource union so the precedence is explicit in types, and a recorded snapshot also satisfies the naming-state install gate. Baselines without the record, produced by older compilers, keep the reconstruction path and its behavior unchanged, degrading at worst to today's fail-closed rude edit. A test-only env gate simulates such old baselines in the fallback tests. Tests: codec roundtrip on a mixed-slot bucket, PDB emission and flag-off no-emission coverage, recorded-preference and old-baseline fallback session tests. Service HotReload suite: 428 passed, 0 failed. Component suite: 236 passed, 0 failed, 2 expected skips.
The recorded synthesized-name snapshot was written after IlxGen from the correct map instance, but both write sites (the fsc driver and the in-process CompileFromCheckedProject path) projected the map through the emitted TypeDef simple-name set before serializing. Slots whose names surface as members rather than types, like the plain endpoints@hotreload method that legitimately holds slot 0 of a mixed bucket, and any allocation history beyond emitted TypeDefs were silently dropped. A real EndpointRouting baseline recorded only three buckets while its module carried the full set of @hotreload names, so the session loaded a recorded-but-incomplete snapshot and the mixed endpoints bucket fell back to nothing, degrading a line-adding edit to a rude-edit restart. Record the map itself: collectRecordedSynthesizedNameSnapshot captures the exact post-IlxGen ICompilerGeneratedNameMap.Snapshot plus the final closure-name overrides from the same compiler global state, with no TypeDef filtering, and both writers use it. Recorded-snapshot alias replay in the delta emitter is adjusted so full mixed buckets participate in mapping without transient fresh aliases stealing baseline rows. The mixed-endpoint disk-started regression now asserts the DECODED record: more than the previously observed three buckets, the exact eight-slot endpoints bucket in allocation order (plain, g0_o0, -2, g0_o1, -4, g0_o2, g0_o3, g0_o4), and equality with the session baseline snapshot after re-read. Service HotReload suite: 428 passed, 0 failed. Component suite: 236 passed, 0 failed, 2 expected skips.
…h hot reload naming Union of main's determinism rework and the hot reload replay machinery, both semantics preserved in full: Main's side kept: per-(basicName, FileIndex) counter buckets, PerFileNamingScope and CompilerGlobalState.NewFileScope for optimizer allocations, the primed rawdata @t counters, Lazy-valued StableNiceNameGenerator, and cross-file closure nameRange bucketing (inlined expressions bucket by the enclosing type's file). Hot reload side kept: the installed ICompilerGeneratedNameMap wins over every allocation path, now including the new FreshCompilerGeneratedNameInScope used by optimizer per-file scopes, so sessions keep replaying baseline names while normal compiles get main's deterministic buckets; ResetCompilerGeneratedNameState composes over the merged counter storage; the primed @t fallback is routed through the nextIlxOrdinal wrapper so the IlxGen architecture guard still enforces replayability; closure replay applies on top of the nameRange-derived name. Reconciliation fixes at the intersection, none of which changes a test expectation: public AddedOrChangedMethods no longer leak compiler-generated helper methods declared on user types, while method-body and PDB paths keep the complete set; active-statement remap and baseline chaining consume full emitted bodies rather than the filtered public list; lambda occurrence extraction skips synthetic zero-range lambdas before consuming an occurrence ordinal; member debug information collects typed-tree inputs so state machine CDI rows can be derived when IlxGen resume-point recording is absent, with recorded rows staying authoritative; baseline closure-name row capture is partial per occurrence instead of all-or-nothing per member; disk-started partial-table replay reuses exact persisted chains only when the in-memory baseline lacks the chain; CDI-derived closure reconstruction fails closed for replay-only closure TypeDefs without misclassifying task state machine helpers. Verification: HotReload service suite 428 passed, 0 failed; component suite 234 passed, 0 failed, 2 expected skips; DeterministicTests guard suite for dotnet#19732 21 passed, 0 failed.
The two merge parents were independently green but their combination broke four task class-state-machine runtime tests with UnsupportedEdit. Complete recorded name snapshots now preserve every allocation slot, including task resumable-code helper names, and the delta emitter consumed those complete buckets as if every alias candidate were closure-chain replay evidence. An inserted let! or do! shifts the resumable helper chain, so a replay-only helper may be the old baseline helper under a new replay name, an exact recorded name may belong to a newly inserted helper, and some replay helpers are genuinely new await machinery with no baseline row. The emitter variously mapped replay helpers onto already-claimed baseline rows, rejected mappings as ambiguous, or appended duplicate helpers under the same string key as the baseline helper; the duplicate case could corrupt rows because method, field, property, and event dictionaries were keyed only by declaring type name. Resumable helper TypeDefs of the updated method are now detected by base name plus ResumableCode shape and reconciled by shape: a replay helper with exactly one shape-compatible unclaimed baseline candidate updates that row; shape-changed exact names and unmatched replay helpers are appended as added helper machinery, with duplicate-name additions isolated behind a synthetic declaring-type key and a suffixed emitted TypeDef name; members declared on an added TypeDef are treated as added before consulting baseline member tokens. Closure-chain reconciliation outside the task resumable-helper case still fails closed, and recorded snapshots keep their completeness and their recorded-over-reconstructed precedence. Contained to IlxDeltaEmitter.fs. Service HotReload suite: 428 passed, 0 failed. Component suite: 236 passed, 0 failed, 2 expected skips.
NatElkins
added a commit
to NatElkins/fsharp
that referenced
this pull request
Jul 2, 2026
…tors Compiler-generated occurrence names (name@line-N) are allocated from process-wide counters on CompilerGlobalState that accumulate across compilations. When a warm checker re-emits the same project in-process, an unchanged closure therefore gets a different occurrence suffix than the previous emit, so consumers that align generated names across compilations (Edit-and-Continue delta emission, dotnet#19941) cannot match them. Add an internal ResetCompilerGeneratedNameState to NiceNameGenerator (clears the per-(name, file) occurrence counters), StableNiceNameGenerator (clears the cached stable names and the inner counters), and an aggregate on CompilerGlobalState that resets all three generators, restoring the fresh-process name layout. Callers must ensure no compilation is concurrently generating names. No in-tree caller yet; the consumer is the hot reload emit path in dotnet#19941. Covered by unit tests proving drift without reset, exact replay after reset, and that the stable-name cache itself is cleared.
NatElkins
added a commit
to NatElkins/fsharp
that referenced
this pull request
Jul 2, 2026
…ethod CDI emission Adds an internal AbstractIL module implementing, byte for byte, the three Portable PDB CustomDebugInformation blob formats Roslyn persists per method for Edit and Continue (EnC Local Slot Map, EnC Lambda and Closure Map, EnC State Machine State Map), with serializers, deserializers, a portable PDB read-back helper, and an occurrence-key packing helper for deterministic syntax-offset slots. Plumbs an optional methodCustomDebugInfoRows side channel through the IL binary writer options into the portable PDB generator so a compilation can attach CDI rows to named methods. Names that do not identify exactly one method row are dropped. All existing writer call sites pass an empty map, so emitted PDBs are byte-identical to before. No in-tree caller populates the map yet; the consumer is the F# hot reload work in dotnet#19941, following the same pattern as dotnet#20017 (land isolated, test-covered infrastructure first, wire the feature later). Tests: blob round-trips, Roslyn golden-byte encodings, cross-validation against CDI blobs emitted by a real Roslyn compilation, fail-closed occurrence-key packing (including an int32-overflow regression where a wrapped negative key previously escaped the bound check), and end-to-end synthetic PDB emission proving correct MethodDef parenting, zero rows for an empty map, and no rows for absent or ambiguous names.
NatElkins
added a commit
to NatElkins/fsharp
that referenced
this pull request
Jul 2, 2026
Adds an internal, standalone ECMA-335 Edit-and-Continue metadata delta writer to AbstractIL: delta #- table stream and heap construction (DeltaMetadataTables, DeltaMetadataSerializer, DeltaTableLayout, DeltaIndexSizing), ECMA-335 II.24.2.6 coded-index encoding (DeltaMetadataEncoding), EncLog/EncMap emission, generation GUID chaining, user-string and standalone-signature token calculators (IlxDeltaStreams), and the coordinating writer (FSharpDeltaMetadataWriter) over a plain row-description input model (DeltaMetadataTypes, ILDeltaHandles, ILMetadataHeaps). The writer's inputs are row records (names, tokens, signatures, RVAs) plus heap offsets; it has no dependency on any semantic diffing or session machinery. It compiles with no in-tree consumer by design: the consumer is the F# hot reload work in dotnet#19941, following the same upstreaming pattern as dotnet#20017 and dotnet#20018 (land isolated, test-covered infrastructure first, wire the feature in a later PR). One line of ilwrite.fsi is touched to expose the pre-existing markerForUnicodeBytes so the delta writer reuses the exact string-marker logic of the full writer. No behavior change for any existing code path. Tests (130): coded-index encodings asserted against the production definitions and ECMA-335 II.24.2.6 order, System.Reflection.Metadata reader parity over emitted deltas, EncLog/EncMap correctness, stream layout, heap and index sizing, multi-generation heap-offset chaining asserted against computed expected values, standalone-signature rows asserted at baseline+1 from a real seeded baseline, and serializer failure paths.
This was referenced Jul 2, 2026
On CI the test projects fail to compile in shapes the local bootstrap accepts: Assert.Equal<string> over string arrays commits overload resolution to the scalar Equal shape (FS0193), and Assert.Equal<int[]> over int arrays commits to a Span overload carrying an unmanaged constraint (FS0001). Switch to Assert.Equal<string[]> and Assert.Equal<int list>, which resolve the same way under both compilers. HotReloadCheckerTests loads baseline assemblies through AssemblyLoadContext, which .NET Framework does not have, and FSharp.Compiler.Service.Tests also compiles for net472 on the Windows Desktop CI legs. Gate the file to .NET Core in the project file, the same way the SurfaceArea test is gated. The EnC CDI cross-validation test resolved the dotnet host by hand from __SOURCE_DIRECTORY__, which misses on CI images that carry no repo-local .dotnet at that depth. Resolve it through TestFramework.initialConfig.DotNetExe like the rest of the framework. Verified: EncMethodDebugInformationTests 14 passed, ThreadSafetyTests 7 passed, both test projects build with 0 errors, fantomas clean on touched files.
String.Contains(string, StringComparison) does not exist on net472 and FSharp.Compiler.Service.Tests also compiles for net472 on the Windows Desktop CI legs. All twenty uses across the session and delta builder tests passed StringComparison.Ordinal, which is already the semantic of the always-available Contains(string) overload, so drop the argument. The synthetic baseline build in TestHelpers computed the dotnet host path by hand from __SOURCE_DIRECTORY__, which misses on CI images that carry no repo-local .dotnet, failing ApplyUpdate tests on macOS agents before the build starts. Resolve it through TestFramework.initialConfig.DotNetExe like the rest of the framework. Verified: session tests 15 passed, DeltaBuilderTests 9 passed, component HotReload 236 passed 0 failed 2 skipped, fantomas clean on touched files.
RoslynBaselineComparisons parses its baseline table with System.Text.Json, which .NET Framework does not have, and FSharp.Compiler.Service.Tests compiles for net472 on every Windows CI job because the solution build covers both target frameworks. Gate the file to .NET Core in the project file like the SurfaceArea test. The nine comparisons keep running on all net10.0 legs. Verified: net10.0 build clean, RoslynBaselineComparisons 9 passed.
The synthesized-alignment and state-machine-shape tests build their updated sources by String.Replace over triple-quoted baseline literals. Triple-quoted literals inherit the source file's line endings, and the repo does not pin an eol for .fs files, so Windows checkouts hand these baselines CRLF while the replace needles use plain newline escapes. The needle then never matches and Replace silently returns the baseline unchanged, so the tests compile identical source twice: the alignment tests fail with NoChanges and the rejection tests invert because there is nothing left to reject. The compiler itself is unaffected since names and line shifts key off sequence-point line numbers. Normalize the affected baselines to plain newlines before any splicing so the fixture surgery is line-ending-agnostic on any host. Verified: component HotReload 236 passed 0 failed 2 skipped, fantomas clean on the touched file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This draft PR presents the complete F# hot reload implementation as a single branch against current main:
dotnet watch(with a companion dotnet-watch change, linked below) patches running F# processes in place via the standard EnC pipeline (MetadataUpdater.ApplyUpdate) — the same runtime contract C# uses. Unsupported edits degrade to the rebuild-and-restart flow watch uses today, so a limited scope still behaves as a complete feature.It is opened as a single draft per discussion with @T-Gro: one branch to review, build, and launch testing from. A decomposition into independently shippable PRs is laid out below and can begin whenever review reaches that stage. Earlier discussion: #11636.
Try it (30–45 min, two clones, copy-paste steps)
docs/hot-reload-quickstart.md walks from
git cloneto editing a running F# app — including adding aList.map (fun s -> s.ToUpper())to a live method and watching it apply in place, state preserved. Short version: build this branch, build NatElkins/sdkfsharp-hotreload-watch-v2(a complete .NET CLI with an F#-awaredotnet watch), copy the freshly builtFSharp.Compiler.Service.dllinto the SDK layout, add<OtherFlags>$(OtherFlags) --test:HotReloadDeltas</OtherFlags>to a console app, anddotnet watch run.What works
async, resume-point-stabletask, and generics--test:HotReloadClassStateMachines): adding or removing alet!/do!intaskandbackgroundTask, which emits class-form (reference-type) state machines so the change is anAddInstanceFieldToExistingTypeplus a method update, matching C#.taskSeqand other resumable CEs share the same lowering path. Off by default; flag-off codegen is byte-identical.[<CLIEvent>]eventsinline-annotation change) degrade to the standard rebuild-and-restart flow with a precise diagnostic. Adding or removing a mid-sequencelet!/do!is rude under the default struct state machines, but becomes a supported edit (anAddInstanceFieldToExistingTypeplus a method update, matching C#) under--test:HotReloadClassStateMachinesIsolation and disabling
The feature is designed so that a compile without the flag is indistinguishable from main, and so that the whole feature can be turned off at one point if something goes wrong:
--test:HotReloadDeltas(requires--debug+, incompatible with--optimize+), with class-form resumable state machines a further opt-in behind--test:HotReloadClassStateMachines. Both are off by default; no MSBuild property or SDK behavior changes in this repo.FSharpProjectSnapshot.fsis byte-identical to main (tracked-input staleness lives in the hot reload session layer instead).ICompilerEmitHookinterface with a no-op default. Guard scripts undertests/scripts/(run by the verification gate) enforce that only the intended files consume the hook and that theIlxGenname-generation path and fsi surface stay on their pinned shapes.DOTNET_WATCH_FSHARP_HOTRELOAD=0) that restores stock restart-on-edit behavior.Architecture
FSharpChecker.CreateHotReloadSessionreturns aFSharpHotReloadSessionholding per-project committed snapshots + emit baselines — theDebuggingSession/CommittedSolutionshape from Roslyn, built onFSharpProjectSnapshot(the snapshot contract, not the experimental workspace surface), so it composes with the FSharpWorkspace direction without depending on it. Solution-wide commit/discard semantics, runtime-capability updates, active-statement intake.AddMethodToExistingType,NewTypeDefinition,GenericUpdateMethod, …). Anything unclassifiable fails closed.EmitDifferencereference deltas, with mdv, and against CoreCLRApplyUpdatein runtime tests.Design documentation (rendered, on this branch)
FSharpHotReloadSession, per-project committed snapshots + baselines onFSharpProjectSnapshot, the FSharpWorkspace relationship, determinism pinsEmitDifferencereference templates (EncLog/EncMap shapes per edit kind) and the F# emission matrix incl. every intentional fail-closed caseEditAndContinueCapabilitiesparity) and per-capability gatingScale and review shape
~118 commits, 159 files, ~60k insertions, of which ~34k are tests and ~2.4k docs. The src/ changes are predominantly new self-contained modules (
IlxDeltaEmitter.fs,TypedTreeDiff.fs,HotReloadBaseline.fs, the AbstractIL EnC readers, the delta writer stack). Pre-existing files carry hook callouts plus one behavior-neutral refactor (ilwrite.fsMetadataTablerecord→class, exposing a baseline-row access seam for the delta writer); total deletions across the branch are 241 lines.Proposed path to merging in pieces
The fail-closed design means scope can grow capability by capability — each unsupported case is already a rude edit with a diagnostic, so every intermediate state is a complete, working feature. The natural sequence:
IlxGen/name-generation paths — reviewed on its own)Evidence
ApplyUpdate+ invocation, multi-generation chains, cross-process (disk-started) sessions#Stringsheap layout #19732, Determinism: closure type names differ between --parallelcompilation- and --parallelcompilation+ #19928 (open; flag-on compiles are insulated),$(ParallelCompilation)never reaches the Fsc task — targets pass the item@(ParallelCompilation)instead of the property #19935 (why the pin is compiler-level)./tests/scripts/hot-reload-verify.shKnown limitations / future work
Companion PRs