Skip to content

Improve E2E coverage across SDKs#1906

Open
stephentoub wants to merge 9 commits into
mainfrom
stephentoub-improve-csharp-e2e-coverage
Open

Improve E2E coverage across SDKs#1906
stephentoub wants to merge 9 commits into
mainfrom
stephentoub-improve-csharp-e2e-coverage

Conversation

@stephentoub

Copy link
Copy Markdown
Collaborator

Summary

The SDK E2E suites had uneven coverage for generated RPCs and option/handler flows, which let gaps such as root collection result metadata and null-setting clears slip through. This PR expands the C# suite and ports the same real-world-ish scenarios across Node, Python, Go, Rust, and Java so the SDKs exercise the same surface area.

This adds coverage for advanced client/session/resume option forwarding, MCP OAuth pending requests, LLM response callbacks, account and user settings RPCs, provider/model/session-state RPCs, visibility, completions fallback, context metadata, subagent settings, and pending handler methods. It also adds shared replay snapshots for the new cross-SDK scenarios.

The tests exposed and fix a few SDK issues: C# source-generation metadata now includes root collection RPC results, Go forwards ExtensionInfo during create/resume, and Java preserves explicit null map values in JSON-RPC params so user setting clears reach the CLI.

Validation

  • Focused E2E suites for Node, Python, Go, Rust, and Java
  • Java ./mvnw.cmd verify

Generated by Copilot

Copilot AI review requested due to automatic review settings July 4, 2026 20:40
@stephentoub stephentoub requested a review from a team as a code owner July 4, 2026 20:40
Comment thread rust/tests/e2e/mcp_oauth.rs Fixed
Comment thread rust/tests/e2e/rpc_server_misc.rs Fixed
Comment thread python/e2e/test_mcp_oauth_e2e.py Fixed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR expands cross-SDK E2E coverage so Node, Python, Go, Rust, Java, and .NET exercise the same RPC surface area and option-forwarding flows, and it includes a few SDK fixes uncovered by the new scenarios (C# source-gen root collection result types, Go ExtensionInfo forwarding, Java JSON null map-value preservation).

Changes:

  • Added/ported E2E scenarios across SDKs for: pending handler RPCs (incl. MCP OAuth / headers refresh), LLM inference response-frame guards, user settings + account auth RPCs, and session-state extras (provider/model/session visibility/completions/context metadata/subagent settings).
  • Added new replay-proxy YAML snapshots for the new scenarios.
  • Fixed SDK wiring gaps: C# source-generation includes root collection RPC results, Go forwards ExtensionInfo on create/resume, Java preserves explicit null map values in JSON-RPC params.
Show a summary per file
File Description
test/snapshots/rpc_session_state_extras/should_update_and_clear_live_subagent_settings.yaml New snapshot fixture for session-state extras subagent settings scenario.
test/snapshots/rpc_session_state_extras/should_return_empty_completions_when_host_does_not_provide_them.yaml New snapshot fixture for “empty completions” scenario.
test/snapshots/rpc_session_state_extras/should_report_visibility_as_unsynced_for_local_session.yaml New snapshot fixture for visibility unsynced scenario.
test/snapshots/rpc_session_state_extras/should_get_context_attribution_and_heaviest_messages_after_turn.yaml New snapshot fixture covering a turn + metadata queries.
test/snapshots/rpc_session_state_extras/should_add_byok_provider_and_model_at_runtime.yaml New snapshot fixture for provider/model add-at-runtime scenario.
test/snapshots/rpc_server_misc/should_login_list_getcurrentauth_and_logout_account.yaml New snapshot fixture for account login/list/logout scenario.
test/snapshots/rpc_server_misc/should_get_set_and_clear_user_settings.yaml New snapshot fixture for user settings get/set/clear scenario.
scripts/codegen/csharp.ts Extends C# RPC source-generation to include root collection result types in the JsonSerializerContext list.
rust/tests/e2e/rpc_tasks_and_handlers.rs Adds Rust E2E coverage for additional pending handler RPCs (UI session limits + MCP headers refresh).
rust/tests/e2e/rpc_session_state_extras.rs Adds Rust E2E coverage for provider add/switch, completions fallback, visibility, metadata, and subagent settings.
rust/tests/e2e/rpc_server.rs Adds Rust E2E coverage for rejecting unknown LLM inference response frames; uses no-snapshot harness path.
rust/tests/e2e/rpc_server_misc.rs Adds Rust E2E coverage for user settings get/set/clear and account auth lifecycle RPCs.
rust/tests/e2e/mcp_oauth.rs Adds Rust E2E coverage for resolving pending MCP OAuth request via direct RPC + small assertion refactors.
rust/tests/e2e/client_options.rs Adds Rust E2E tests validating advanced create/resume option forwarding using a fake stdio CLI capture.
python/e2e/test_rpc_tasks_and_handlers_e2e.py Ports pending handler coverage to Python (session limits + MCP headers refresh).
python/e2e/test_rpc_session_state_extras_e2e.py Ports session-state extras scenarios to Python (provider/model, completions, visibility, metadata, subagent settings).
python/e2e/test_rpc_server_misc_e2e.py Adds Python E2E for user settings get/set/clear and account auth lifecycle with isolated client state.
python/e2e/test_rpc_server_e2e.py Adds Python E2E for rejecting LLM inference response frames for missing request IDs.
python/e2e/test_mcp_oauth_e2e.py Adds Python E2E for resolving MCP OAuth pending requests via direct RPC.
python/e2e/test_client_options_e2e.py Adds Python E2E for advanced create/resume option forwarding via fake CLI capture; updates fake CLI to handle resume.
nodejs/test/e2e/rpc_tasks_and_handlers.e2e.test.ts Ports pending handler scenarios to Node E2E (session limits + MCP headers refresh).
nodejs/test/e2e/rpc_session_state_extras.e2e.test.ts Ports session-state extras scenarios to Node E2E (provider/model, completions, visibility, metadata, subagent settings).
nodejs/test/e2e/rpc_server.e2e.test.ts Adds Node E2E for rejecting unknown LLM inference response frames.
nodejs/test/e2e/rpc_server_misc.e2e.test.ts Adds Node E2E for user settings get/set/clear and account auth lifecycle with isolated client home/env.
nodejs/test/e2e/mcp_oauth.e2e.test.ts Adds Node E2E for resolving MCP OAuth pending requests via direct RPC.
nodejs/test/e2e/client_options.e2e.test.ts Adds Node E2E tests validating advanced create/resume option forwarding via fake CLI capture; updates fake CLI to handle resume.
java/src/test/java/com/github/copilot/RpcTasksAndHandlersE2ETest.java Adds Java E2E coverage for pending handler RPCs (session limits + MCP headers refresh).
java/src/test/java/com/github/copilot/RpcSessionStateExtrasE2ETest.java Adds Java E2E coverage for session-state extras scenarios (provider/model, completions, visibility, metadata, subagent settings).
java/src/test/java/com/github/copilot/RpcServerMiscE2ETest.java Adds Java E2E coverage for LLM response-frame rejection + user settings + account auth lifecycle.
java/src/test/java/com/github/copilot/McpOAuthE2ETest.java Adds Java E2E coverage for resolving pending MCP OAuth request via direct RPC.
java/src/test/java/com/github/copilot/ClientOptionsE2ETest.java Adds Java E2E tests validating create/resume option forwarding via a fake stdio CLI capture.
java/src/main/java/com/github/copilot/JsonRpcClient.java Adjusts Jackson inclusion policy so explicit null map values are serialized (while still omitting null properties).
go/internal/e2e/testharness/context.go Adds Go harness helper to configure proxy without loading a snapshot.
go/internal/e2e/rpc_tasks_and_handlers_e2e_test.go Ports pending handler scenarios to Go E2E (session limits + MCP headers refresh).
go/internal/e2e/rpc_session_state_extras_e2e_test.go Ports session-state extras scenarios to Go E2E (provider/model, completions, visibility, metadata, subagent settings).
go/internal/e2e/rpc_server_misc_e2e_test.go Adds Go E2E for user settings get/set/clear and account auth lifecycle; adds helper for “no token” client.
go/internal/e2e/rpc_server_e2e_test.go Adds Go E2E for rejecting LLM inference response frames with missing request IDs.
go/internal/e2e/mcp_server_helpers_test.go Refactors MCP server status wait helper to allow context + error returns (used by new tests).
go/internal/e2e/mcp_oauth_e2e_test.go Adds Go E2E for resolving pending MCP OAuth request via direct RPC, using no-snapshot proxy setup.
go/internal/e2e/client_options_e2e_test.go Adds Go E2E for advanced create/resume option forwarding via fake CLI capture; adds capture helpers.
go/client.go Forwards ExtensionInfo into session create/resume wire requests.
dotnet/test/E2E/RpcTasksAndHandlersE2ETests.cs Expands .NET E2E pending handler coverage (session limits + MCP headers refresh).
dotnet/test/E2E/RpcSessionStateExtrasE2ETests.cs Expands .NET E2E for session-state extras (provider/model, completions, visibility, metadata, subagent settings).
dotnet/test/E2E/RpcServerMiscE2ETests.cs Expands .NET E2E for user settings get/set/clear and account auth lifecycle; adds isolated-client helper options.
dotnet/test/E2E/RpcServerE2ETests.cs Adds .NET E2E for rejecting missing-request LLM inference response frames; expands discovery RPC coverage assertions.
dotnet/test/E2E/McpOAuthE2ETests.cs Adds .NET E2E for resolving pending MCP OAuth request via direct RPC.
dotnet/test/E2E/ClientOptionsE2ETests.cs Adds .NET E2E for advanced create/resume option forwarding via fake CLI capture (plus provider create test).
dotnet/src/Generated/Rpc.cs Updates generated JsonSerializerContext registrations to include root collection RPC result types.

Review details

  • Files reviewed: 47/48 changed files
  • Comments generated: 3
  • Review effort level: Low

Comment thread rust/tests/e2e/mcp_oauth.rs
Comment thread rust/tests/e2e/rpc_server_misc.rs Outdated
Comment thread go/internal/e2e/client_options_e2e_test.go Outdated
Comment thread dotnet/test/E2E/RpcServerMiscE2ETests.cs Fixed
Comment thread rust/tests/e2e/rpc_server_misc.rs Fixed
Comment thread rust/tests/e2e/rpc_server_misc.rs Fixed
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@stephentoub

Copy link
Copy Markdown
Collaborator Author

Accepted the Java consistency feedback and addressed it in this PR rather than leaving it for follow-up. Java now exposes ExtensionInfo, requestExtensions, and extensionSdkPath on session create/resume config, forwards them on both wire requests, and ClientOptionsE2ETest asserts the serialized create and resume payloads.

Generated by Copilot

@stephentoub

Copy link
Copy Markdown
Collaborator Author

Accepted the remaining actionable consistency observation. Go and Java now also cover the MCP headers refresh none result variant for missing pending request IDs, matching the Node.js and .NET E2E coverage; the Java LLM rejection placement note does not need a code move in this PR because the scenario is already covered.

Generated by Copilot

@github-actions

This comment has been minimized.

@stephentoub

Copy link
Copy Markdown
Collaborator Author

Accepted this parity note and addressed it in 0342d4c4. Node, Python, Go, Rust, and Java now extend their server discovery E2Es with Skills.GetDiscoveryPaths, Agents.Discover, Agents.GetDiscoveryPaths, Instructions.Discover, and Instructions.GetDiscoveryPaths assertions, matching the .NET coverage.

Generated by Copilot

@github-actions

This comment has been minimized.

@stephentoub

Copy link
Copy Markdown
Collaborator Author

Accepted this Java parity gap and addressed it in 29938e7. Java now has a dedicated RpcServerE2ETest for server-scoped RPC coverage, and the Java RPC codegen now keeps explicit sessionId params for server-scoped session RPC wrappers while preserving automatic session ID injection for session-scoped wrappers.

Generated by Copilot

@github-actions

This comment has been minimized.

stephentoub and others added 9 commits July 5, 2026 10:18
Add C# E2E coverage for previously unexercised generated RPC methods, advanced session option forwarding, runtime provider registration, account/user settings flows, session metadata helpers, and handler callbacks.

Fix C# RPC code generation so root collection and dictionary result types are included in the source-generated JSON context.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Replicate the newly added C# E2E coverage across Node, Python, Go, Rust, and Java, including RPC option forwarding, pending handler paths, MCP OAuth handling, account/user settings flows, and session state extras.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Resolve review feedback by removing unused/test-only variables, tightening cleanup exception handling, fixing stale comments, and making the Python MCP OAuth wait explicit.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adjust the Rust account lifecycle E2E assertion so CodeQL can see the matched user binding is used outside an assertion macro.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Make MCP OAuth waits explicit in Python, rewrap Rust OAuth assertions, and avoid CodeQL macro-only bindings in the Rust account lifecycle test.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add Java session and resume support for requestExtensions, extensionSdkPath, and ExtensionInfo so create/resume requests match the other SDKs. Extend the client options E2E coverage to assert those options are forwarded.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add Go and Java E2E assertions for the MCP headers refresh none response variant so their missing-pending-handler coverage matches Node and .NET.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Cover the server discovery-path, agent discovery, and instruction discovery RPCs in the Node, Python, Go, Rust, and Java E2E suites so they match the .NET server RPC coverage.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add a dedicated Java RpcServerE2ETest that exercises server-scoped RPC wrappers in parity with the other SDKs.

Update Java RPC codegen so server-scoped session methods keep their required sessionId params while session-scoped wrappers continue to inject sessionId automatically.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@stephentoub stephentoub force-pushed the stephentoub-improve-csharp-e2e-coverage branch from 29938e7 to 7fd2bfb Compare July 5, 2026 14:18
@github-actions

github-actions Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Cross-SDK Consistency Review ✅

This PR improves cross-SDK consistency rather than creating any gaps. Here's the analysis:

Bug fixes that restore parity

SDK Issue Fixed Already correct in
Go ExtensionInfo was not forwarded to the CLI on CreateSession/ResumeSession (only ExtensionSDKPath was) Node.js, Python, .NET, Rust
Java Same as Go — requestExtensions, extensionSdkPath, extensionInfo were defined on SessionConfig/ResumeSessionConfig but never added to the wire request types or forwarded by SessionRequestBuilder All other SDKs
Java Jackson's NON_NULL policy silently dropped null map values in JSON-RPC params, so user-setting clears (sending null for a key) never reached the CLI N/A — language-specific (Jackson behavior)
C# AOT/trimming source-gen context was missing IDictionary<string,JsonElement> and IList<AccountAllUsers> for root collection RPC results N/A — C#-specific (source generation)

New E2E coverage

New tests are added uniformly across all six SDKs (Node.js, Python, Go, .NET, Java, Rust) with identical test scenario names and shared YAML snapshots, exercising:

  • Advanced client/session/resume option forwarding
  • MCP OAuth pending requests
  • Account/user settings RPCs (including the null-clear fix)
  • Provider/model/session-state RPCs
  • Subagent settings and pending handler methods

No gaps introduced

All three SDK-level bug fixes bring Java and Go into alignment with the already-correct behavior in Node.js, Python, .NET, and Rust. The C# and Java codegen fixes are runtime/toolchain-specific and have no cross-SDK equivalents needed.

Generated by SDK Consistency Review Agent for issue #1906 · sonnet46 2M ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants