Stabilize and migrate SOS harness coverage by steveisok · Pull Request #6001 · dotnet/diagnostics

steveisok · 2026-09-02T23:52:44Z

Summary

Layer 3 of the native SOS test stack. Depends on #6000.

migrates the reviewed command-focused coverage and parsing tests onto the stabilized harness
adds explicit command capability filters and cross-platform corrections without Helix or shard plumbing
enables .NET 11 SingleFile+cDAC configurations while retaining the layer-2 validity rules
adds the ObjectReference oracle for notreachableinrange
retires DivZero.script, NestedExceptionTest.script, and SimpleThrow.script after replacing their exact exception, source-line, stack, thread, live/dump, breakpoint, and CLRMA assertions
retains Reflection.script because reflection-boundary and target-invocation behavior remains specialized legacy coverage

Preservation and split

preserves the newer DumpGen_ArgumentsAndFilters test and FirstObjectOfExactType Assert.NotEmpty guard
preserves concurrent stdout/stderr drainage and all layer-2 harness/process/path/capture work
keeps README.md architecture/usage documentation and COVERAGE.md, updating both surgically
excludes CI, Helix, payload, shard, Arcade, project-local skip, replay attribute, and generated solution changes

Validation

./build.sh -configuration Debug -architecture arm64 -rebuild -skipnative
25 parsing/capability/validity tests
7 focused LLDB live exception and source-stack tests on macOS arm64
1 focused dotnet-dump notreachableinrange test on macOS arm64

Copilot

Copilot review overview

🟡 Changes recommended

The updated PrintExceptionTests/ClrmaExceptionMatrix include heap-less Mini dumps (DumpKind.All) while asserting heap-dependent exception/message data, likely causing failures or flakiness.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review tier: Lite
Findings: 2

New issues introduced by this change (2)

Severity	Finding
	`src/tests/SOS.Tests/DiagnosticCommandTests.cs` — Clrma_ReportsCurrentExceptionChain asserts the current exception chain, which likely depends on…
	`src/tests/SOS.Tests/PrintExceptionTests.cs` — These PrintException assertions require managed heap data (exception object + message strings).…

What changed in this PR

This PR continues the SOS native test-stack migration by retiring several legacy SOS.UnitTests script-driven exception scenarios and re-homing their assertions into the stabilized SOS.Tests harness, while also refining matrix capability filters and improving cross-platform correctness (including .NET 11 + SingleFile + cDAC scenarios).

Changes:

Retires legacy DivZero, SimpleThrow, and NestedExceptionTest script entry points and replaces their coverage with focused SOS.Tests scenarios and oracles.
Introduces explicit capability/matrix filters (heap enumeration, current-thread constraints, ICorDebug stack-walk constraints) and adds targeted macOS/dotnet-dump skips for known platform gaps.
Extends and hardens parsing/inspection coverage (e.g., clru offsets parsing, gcinfo legacy encoding parsing, notreachableinrange slot-range oracle, architecture-correct register assertions).

File	Description
src/tests/SOS.UnitTests/SOS.cs	Removes legacy script-backed test entry points for retired scenarios.
src/tests/SOS.UnitTests/Scripts/SimpleThrow.script	Deletes legacy script now covered by harness-based tests.
src/tests/SOS.UnitTests/Scripts/NestedExceptionTest.script	Deletes legacy script now covered by harness-based tests (incl. CLRMA coverage).
src/tests/SOS.UnitTests/Scripts/DivZero.script	Deletes legacy script now covered by harness-based tests.
src/tests/SOS.UnitTests/Debuggees/SosHarnessScenarios/SosHarnessScenarios.cs	Adds `ObjectReference`/array to provide a deterministic pointer-slot range for `notreachableinrange`.
src/tests/SOS.Tests/TestMatrixCapabilityTests.cs	Adds unit tests validating matrix capability predicates and parsing helpers.
src/tests/SOS.Tests/TestMatrices.cs	Adds capability filters and a macOS/dotnet-dump thread skip helper; expands matrix-building helpers.
src/tests/SOS.Tests/TestConfigValidityTests.cs	Updates validity expectations to allow .NET 11 SingleFile + cDAC.
src/tests/SOS.Tests/StackInspectionTests.cs	Narrows matrix to configs where current-thread commands are meaningful; adds macOS/dotnet-dump skip.
src/tests/SOS.Tests/SpecializedInspectionTests.cs	Adds targeted skip for known macOS .NET 11 legacy-DAC dump limitations.
src/tests/SOS.Tests/RuntimeInfoTests.cs	Makes `registers` assertions architecture-aware (esp/rsp/sp).
src/tests/SOS.Tests/README.md	Updates documentation to reflect SingleFile support with cDAC on .NET 11+.
src/tests/SOS.Tests/PrintExceptionTests.cs	Replaces script assertions with structured, focused printexception tests across matrices.
src/tests/SOS.Tests/ObjectInspectionTests.cs	Makes object size assertion pointer-size aware.
src/tests/SOS.Tests/NativeAddressSpaceTests.cs	Switches `notreachableinrange` oracle to scan an actual object-reference slot range and filters progress lines.
src/tests/SOS.Tests/MiscCommandTests.cs	Ensures dbgout is turned off via `finally`; refines host/DAC exclusions for enummem.
src/tests/SOS.Tests/MemoryAndDecodeTests.cs	Fixes dp/dq assertions for 32-bit by validating split 32-bit halves.
src/tests/SOS.Tests/GcWhereTests.cs	Uses current-thread-capable matrices for gcwhere tests.
src/tests/SOS.Tests/GcInspectionTests.cs	Adjusts stop point for verifyheap and adds targeted macOS legacy-DAC skip.
src/tests/SOS.Tests/EeHeapTests.cs	Strengthens assertions by requiring ranges per generation rather than concatenated segments.
src/tests/SOS.Tests/EeHeapParsing.cs	Refines regions-vs-segments detection and clarifies documentation.
src/tests/SOS.Tests/DumpHeapStringsTests.cs	Uses heap-enumeration capability filtering for dumpheap -strings coverage.
src/tests/SOS.Tests/DiagnosticCommandTests.cs	Adds CLRMA exception-chain coverage and macOS/dotnet-dump skips.
src/tests/SOS.Tests/COVERAGE.md	Updates migration/retirement mapping and rationale for retired scripts.
src/tests/SOS.Tests/CodeInfoTests.cs	Makes transition assertions tolerant of legacy x86 wording differences.
src/tests/SOS.Tests/CodeCommandParsingTests.cs	Adds focused unit tests for clru/gcinfo parsing edge cases (offset/no-offset, legacy encodings).
src/tests/SOS.Tests/CodeCommandParsing.cs	Improves clru parsing to be offset-mode aware; enhances gcinfo parsing for legacy formats and transitions.
src/tests/SOS.Tests/ClrUTests.cs	Adjusts source-annotation expectations and transition marker for x86/legacy encoding.
src/tests/SOS.Tests/ClrStackTests.cs	Applies macOS/dotnet-dump skip and adds capability filtering for GC-root enumeration.
src/tests/SOS.Tests/ClrStackLinesTests.cs	Strengthens source-line assertions (including line-number sets where variability exists) and uses live opt-in.
src/tests/SOS.Tests/ClrStackICorDebugTests.cs	Uses explicit capability filter for known unsupported ICorDebug stack-walk combinations; adds macOS/dotnet-dump skip.
src/tests/SOS.Tests/ClrStackFullTests.cs	Adds macOS/dotnet-dump skip.
src/tests/SOS.Tests/ClrStackFrameCountTests.cs	Adds macOS/dotnet-dump skip.
src/tests/SOS.Tests/ClrStackArgsLocalsTests.cs	Adds macOS/dotnet-dump skip.
src/tests/SOS.Tests/ClrStackAllThreadsTests.cs	Adds macOS/dotnet-dump skip.
src/tests/SOS.TestHarness/TestConfig.cs	Updates DAC support rules to allow SingleFile + cDAC starting at .NET 11.

Suppressed comments (2)

src/tests/SOS.Tests/PrintExceptionTests.cs:26

Same issue here: DumpKind.All introduces heap-less Mini dumps, but this test later asserts exact exception message/source lines which likely require heap data. Restrict the matrix to Heap dumps to avoid Mini rows that can't satisfy the oracle.

    public static TheoryData<TestConfig> NoInnerMatrix { get; } =
        TestMatrices.CurrentThreadCommands(
            [TargetCatalog.DivZero, TargetCatalog.SimpleThrow],
            Liveness.AllValid,
            DumpKind.All);

src/tests/SOS.Tests/PrintExceptionTests.cs:32

DumpKind.All includes Mini dumps; the reflection inner-exception/message assertions are unlikely to be reliable without managed heap memory. Consider using Heap dumps for this matrix.

    public static TheoryData<TestConfig> ReflectionMatrix { get; } =
        TestMatrices.CurrentThreadCommands(
            [TargetCatalog.Reflection],
            Liveness.AllValid,
            DumpKind.All);

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

## Summary - Restores the complete reviewed SOS test harness from [#5979](#5979) after its exact revert in [#6006](#6006). - Re-establishes this change as the new bottom layer beneath [#5999](#5999), [#6000](#6000), [#6001](#6001), and [#6002](#6002). - Temporarily excludes only `SOS.Tests` from local CI via `SkipTests` when `ContinuousIntegrationBuild` is true. Normal local builds and discovery remain enabled until the upper Helix layer moves execution out of local CI. ## Reconstruction proof - The baseline restoration commit `2c7616acc2be6b20cf377574fa52e40196f16ca7` has tree `5d2ab7a6990a8cdc2ff8a3d2ec7a29e373b99d54`, exactly matching the pre-revert commit `ec5af2a1a56c4d9348e7e92438bb2faf8684f15f`. - The only subsequent delta is five added lines in `src/tests/SOS.Tests/SOS.Tests.csproj` for the temporary CI-only skip. ## Validation - `./dotnet.sh build src/tests/SOS.Tests/SOS.Tests.csproj --no-restore --verbosity minimal` — succeeds with 0 warnings and 0 errors. - Focused Microsoft.Testing.Platform discovery for `SOS.Tests.PrintExceptionTests.PrintException_Data` with DotnetDump/Core/net10 constraints — 2 expected rows discovered without `ContinuousIntegrationBuild`. - MSBuild property evaluation: default `SkipTests` is empty; `-p:ContinuousIntegrationBuild=true` evaluates `SkipTests=true`. --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: c34cd9f4-d3b5-4b46-b7fc-e34b34bc88c3

## Summary - add a temporary test-only `sos-lldb` executable on macOS to host SOS through Apple LLDB reliably - keep the driver source and build target with native test helpers while staging the executable for SOS Helix tests - scope LLDB command-result routing with a reentrant RAII guard that restores the previous result - reject invalid LLDB thread IDs at the debugger-service boundary and retain an `E_UNEXPECTED` CLRMA backstop - stop batch execution and return a non-zero exit code when an LLDB command fails ## Why this layer is separate This PR contains only the standalone LLDB test-host layer from #5981. Keeping it separate from Helix sharding, harness, documentation, and dependency work makes the temporary host independently reviewable and provides the bottom layer for subsequent stacked changes. The executable is test infrastructure, is not included in SOS packages, and is temporary until the .NET 11 SDK contains the runtime fix that makes this Apple LLDB workaround unnecessary. ## Validation - `./build.sh -skipmanaged` (macOS arm64 Debug) - successful `sos-lldb --no-lldbinit --batch -o version` returns `0` - failed batch command returns `1` and prevents later `-o` commands from running - an interactive command failure remains nonfatal and later commands still run - invalid and missing argument checks return `2` - loaded `libsosplugin.dylib` through `sos-lldb` and ran `soshelp`, confirming plugin command output is routed through the scoped result --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9a6e90d9-5cd5-4ff1-a05b-11830076408d

Stack layer 2 of the native decomposition for #5981. Depends on #5999 This layer stabilizes the reusable SOS harness across hosts and architectures: bounded child-process capture and stream drainage, clearer child-host failures, bounded LLDB dump hosts, second-chance DbgEng crash handling, macOS `sos-lldb` integration, RID-aware debuggee builds, and focused platform/configuration validity checks. It intentionally excludes Helix submission and pipeline wiring, payload overlays, sharding, command-coverage migration, and legacy test deletion. Validation: - `dotnet build src/tests/SOS.Tests/SOS.Tests.csproj -c Debug -p:TargetArch=arm64 -p:TargetRid=osx-arm64 --no-restore` - focused `BoundedProcessTests`, `HostSlotTests`, and `TestConfigValidityTests` (20 passed) - `dotnet publish` for the `SosHarnessScenarios` single-file `osx-arm64` debuggee - `./build.sh -skipmanaged -configuration Debug -architecture arm64` --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 9a6e90d9-5cd5-4ff1-a05b-11830076408d Copilot-Session: 5281acb1-f8c2-4264-9af1-d8489323b9ba

Roslyn diagnostic exposed three existing negated-pattern bugs - Pinned segments were incorrectly included in the segment count. - Generation 1 objects were incorrectly skipped when finding ephemeral references. - Pinned and frozen segments incorrectly contributed to fragmentation reporting. Co-authored-by: Max Charlamb <maxcharlamb@microsoft.com> Copilot-Session: 7b2ca30d-742d-402c-8ba5-afec8b6d69a1

## Summary - Preserve the existing `eng/build.ps1` `-methodfilter` and `-classfilter` interface. - Pass neutral filter properties into the test traversal. - Translate filters to `-method`/`-class` for the legacy xUnit runner and `--filter-method`/`--filter-class` for xUnit v3 using Microsoft Testing Platform. This allows legacy `SOS.UnitTests` and the new `SOS.Tests` project to run in the same test invocation while using their respective command-line syntax. ## Testing - Ran a filtered legacy `SOS.UnitTests` method through `eng/build.ps1`. - Ran one filtered xUnit v3 method through `eng/build.ps1`. - Ran four filtered xUnit v3 class tests through `eng/build.ps1`. > [!NOTE] > This pull request description was generated with GitHub Copilot. Co-authored-by: Max Charlamb <maxcharlamb@microsoft.com> Copilot-Session: 7b2ca30d-742d-402c-8ba5-afec8b6d69a1

max-charlamb · 2026-09-08T20:56:17Z

SkipUnavailableMacOsDotnetDumpThreads(config) shouldn't be required after #6014 merges

This pull request updates the following dependencies [marker]: <> (Begin:cb58fe07-ae24-4e73-0e84-08d8e40a189f) ## From https://github.com/microsoft/clrmd - **Subscription**: [cb58fe07-ae24-4e73-0e84-08d8e40a189f](https://maestro.dot.net/subscriptions?search=cb58fe07-ae24-4e73-0e84-08d8e40a189f) - **Build**: [20260908.3](https://dev.azure.com/dnceng/internal/_build/results?buildId=3069963) ([331010](https://maestro.dot.net/channel/548/github:microsoft:clrmd/build/331010)) - **Date Produced**: September 8, 2026 8:00:32 PM UTC - **Commit**: [da25fc738ecd61441c93286c7b604b47ed98dc95](microsoft/clrmd@da25fc7) - **Branch**: [main](https://github.com/microsoft/clrmd/tree/main) [DependencyUpdate]: <> (Begin) - **Dependency Updates**: - From [4.1.741901 to 4.1.745803][1] - Microsoft.Diagnostics.Runtime - Microsoft.Diagnostics.Runtime.Utilities [1]: microsoft/clrmd@1ac2ef0...da25fc7 [DependencyUpdate]: <> (End) [marker]: <> (End:cb58fe07-ae24-4e73-0e84-08d8e40a189f) Co-authored-by: dotnet-maestro[bot] <dotnet-maestro[bot]@users.noreply.github.com>

steveisok · 2026-09-09T00:06:22Z

@max-charlamb Addressed in 37073d78: #6001 now drops SkipUnavailableMacOsDotnetDumpThreads and all 17 issue-5987-only call sites, while retaining the unrelated issue #5985 skips. I validated the layer against main with #6014/ClrMD 4.1.745803: the SOS.Tests build and 27 parsing/capability/validity tests passed, along with the focused macOS arm64 dotnet-dump smoke (PrintException_NoInnerException, 2/2 configurations).

Carry the reviewed command tests and cross-platform capability filters onto the stabilized harness layer. Retire the three fully superseded legacy exception scripts while retaining Reflection coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 72ff9a52-4eb6-46f7-a3f9-39b5f99de529

## Summary - migrate command-focused SOS coverage and parsing tests onto the stabilized harness - correct platform and capability matrices while preserving the stabilized process, path, capture, and configuration work - retire the three fully superseded legacy exception scripts while retaining specialized Reflection coverage and issue #5985 skips ## Validation - restored and built `src/tests/SOS.Tests/SOS.Tests.csproj` for macOS arm64 - passed all 27 focused `CodeCommandParsingTests`, `TestMatrixCapabilityTests`, and `TestConfigValidityTests` Supersedes #6001, whose native stacked-PR metadata could not be repaired after its parent was squash-merged. Copilot-Session: 72ff9a52-4eb6-46f7-a3f9-39b5f99de529

steveisok · 2026-09-09T17:28:09Z

Superseded by #6017

steveisok requested a review from a team as a code owner September 2, 2026 23:52

This was referenced Sep 3, 2026

Run the SOS test harness on Helix #6002

Closed

Run SOS test harness on Helix #5981

Closed

steveisok force-pushed the steveisok-sos-coverage-migration branch from 0a60d3a to a92c1df Compare September 3, 2026 12:24

steveisok mentioned this pull request Sep 4, 2026

Reapply new SOS test harness #6008

Merged

Copilot AI lite review requested due to automatic review settings September 4, 2026 02:05

steveisok force-pushed the steveisok-sos-coverage-migration branch from a92c1df to b93dcf9 Compare September 4, 2026 02:05

Copilot started reviewing on behalf of steveisok September 4, 2026 02:05 View session

Copilot AI reviewed Sep 4, 2026

View reviewed changes

Comment thread src/tests/SOS.Tests/DiagnosticCommandTests.cs

Comment thread src/tests/SOS.Tests/PrintExceptionTests.cs

steveisok force-pushed the steveisok-sos-coverage-migration branch 2 times, most recently from 3d5196f to 89a5d09 Compare September 4, 2026 12:40

steveisok force-pushed the steveisok-sos-coverage-migration branch from 89a5d09 to 237278a Compare September 4, 2026 13:07

steveisok force-pushed the steveisok-sos-coverage-migration branch from 237278a to 67dcd35 Compare September 4, 2026 18:04

steveisok force-pushed the steveisok-sos-coverage-migration branch from 67dcd35 to e4c0d90 Compare September 4, 2026 19:03

steveisok force-pushed the steveisok-sos-coverage-migration branch 3 times, most recently from d044a7e to c9b9ceb Compare September 4, 2026 20:21

steveisok force-pushed the steveisok-sos-coverage-migration branch from c9b9ceb to ceae004 Compare September 4, 2026 21:20

steveisok force-pushed the steveisok-sos-coverage-migration branch from ceae004 to 54cb693 Compare September 4, 2026 23:11

steveisok force-pushed the steveisok-sos-coverage-migration branch from 54cb693 to 1bc52eb Compare September 7, 2026 14:42

max-charlamb and others added 2 commits September 8, 2026 20:21

steveisok force-pushed the steveisok-sos-coverage-migration branch 2 times, most recently from 0574248 to 37073d7 Compare September 9, 2026 00:05

max-charlamb approved these changes Sep 9, 2026

View reviewed changes

stack merge was automatically disabled September 9, 2026 16:07
Pull Request is not mergeable

stack merge was automatically disabled September 9, 2026 16:45
Pull Request is not mergeable

stack merge was automatically disabled September 9, 2026 16:51
Pull Request is not mergeable

steveisok force-pushed the steveisok-sos-coverage-migration branch from 37073d7 to d6f2576 Compare September 9, 2026 16:58

steveisok mentioned this pull request Sep 9, 2026

Migrate SOS command coverage #6017

Merged

steveisok closed this Sep 9, 2026

Navigation Menu

Uh oh!

Stabilize and migrate SOS harness coverage - #6001

Stabilize and migrate SOS harness coverage#6001
steveisok wants to merge 6 commits into
steveisok-sos-harness-stabilization-layerfrom
steveisok-sos-coverage-migration

steveisok commented Sep 2, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

max-charlamb commented Sep 8, 2026

Uh oh!

steveisok commented Sep 9, 2026

Uh oh!

steveisok commented Sep 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

steveisok commented Sep 2, 2026

Summary

Preservation and split

Validation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

max-charlamb commented Sep 8, 2026

Uh oh!

steveisok commented Sep 9, 2026

Uh oh!

steveisok commented Sep 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants