Improve E2E coverage across SDKs#1906
Conversation
There was a problem hiding this comment.
Pull request overview
This PR expands cross-SDK E2E coverage so Node, Python, Go, Rust, Java, and .NET exercise the same RPC surface area and option-forwarding flows, and it includes a few SDK fixes uncovered by the new scenarios (C# source-gen root collection result types, Go ExtensionInfo forwarding, Java JSON null map-value preservation).
Changes:
- Added/ported E2E scenarios across SDKs for: pending handler RPCs (incl. MCP OAuth / headers refresh), LLM inference response-frame guards, user settings + account auth RPCs, and session-state extras (provider/model/session visibility/completions/context metadata/subagent settings).
- Added new replay-proxy YAML snapshots for the new scenarios.
- Fixed SDK wiring gaps: C# source-generation includes root collection RPC results, Go forwards
ExtensionInfoon create/resume, Java preserves explicitnullmap values in JSON-RPC params.
Show a summary per file
| File | Description |
|---|---|
| test/snapshots/rpc_session_state_extras/should_update_and_clear_live_subagent_settings.yaml | New snapshot fixture for session-state extras subagent settings scenario. |
| test/snapshots/rpc_session_state_extras/should_return_empty_completions_when_host_does_not_provide_them.yaml | New snapshot fixture for “empty completions” scenario. |
| test/snapshots/rpc_session_state_extras/should_report_visibility_as_unsynced_for_local_session.yaml | New snapshot fixture for visibility unsynced scenario. |
| test/snapshots/rpc_session_state_extras/should_get_context_attribution_and_heaviest_messages_after_turn.yaml | New snapshot fixture covering a turn + metadata queries. |
| test/snapshots/rpc_session_state_extras/should_add_byok_provider_and_model_at_runtime.yaml | New snapshot fixture for provider/model add-at-runtime scenario. |
| test/snapshots/rpc_server_misc/should_login_list_getcurrentauth_and_logout_account.yaml | New snapshot fixture for account login/list/logout scenario. |
| test/snapshots/rpc_server_misc/should_get_set_and_clear_user_settings.yaml | New snapshot fixture for user settings get/set/clear scenario. |
| scripts/codegen/csharp.ts | Extends C# RPC source-generation to include root collection result types in the JsonSerializerContext list. |
| rust/tests/e2e/rpc_tasks_and_handlers.rs | Adds Rust E2E coverage for additional pending handler RPCs (UI session limits + MCP headers refresh). |
| rust/tests/e2e/rpc_session_state_extras.rs | Adds Rust E2E coverage for provider add/switch, completions fallback, visibility, metadata, and subagent settings. |
| rust/tests/e2e/rpc_server.rs | Adds Rust E2E coverage for rejecting unknown LLM inference response frames; uses no-snapshot harness path. |
| rust/tests/e2e/rpc_server_misc.rs | Adds Rust E2E coverage for user settings get/set/clear and account auth lifecycle RPCs. |
| rust/tests/e2e/mcp_oauth.rs | Adds Rust E2E coverage for resolving pending MCP OAuth request via direct RPC + small assertion refactors. |
| rust/tests/e2e/client_options.rs | Adds Rust E2E tests validating advanced create/resume option forwarding using a fake stdio CLI capture. |
| python/e2e/test_rpc_tasks_and_handlers_e2e.py | Ports pending handler coverage to Python (session limits + MCP headers refresh). |
| python/e2e/test_rpc_session_state_extras_e2e.py | Ports session-state extras scenarios to Python (provider/model, completions, visibility, metadata, subagent settings). |
| python/e2e/test_rpc_server_misc_e2e.py | Adds Python E2E for user settings get/set/clear and account auth lifecycle with isolated client state. |
| python/e2e/test_rpc_server_e2e.py | Adds Python E2E for rejecting LLM inference response frames for missing request IDs. |
| python/e2e/test_mcp_oauth_e2e.py | Adds Python E2E for resolving MCP OAuth pending requests via direct RPC. |
| python/e2e/test_client_options_e2e.py | Adds Python E2E for advanced create/resume option forwarding via fake CLI capture; updates fake CLI to handle resume. |
| nodejs/test/e2e/rpc_tasks_and_handlers.e2e.test.ts | Ports pending handler scenarios to Node E2E (session limits + MCP headers refresh). |
| nodejs/test/e2e/rpc_session_state_extras.e2e.test.ts | Ports session-state extras scenarios to Node E2E (provider/model, completions, visibility, metadata, subagent settings). |
| nodejs/test/e2e/rpc_server.e2e.test.ts | Adds Node E2E for rejecting unknown LLM inference response frames. |
| nodejs/test/e2e/rpc_server_misc.e2e.test.ts | Adds Node E2E for user settings get/set/clear and account auth lifecycle with isolated client home/env. |
| nodejs/test/e2e/mcp_oauth.e2e.test.ts | Adds Node E2E for resolving MCP OAuth pending requests via direct RPC. |
| nodejs/test/e2e/client_options.e2e.test.ts | Adds Node E2E tests validating advanced create/resume option forwarding via fake CLI capture; updates fake CLI to handle resume. |
| java/src/test/java/com/github/copilot/RpcTasksAndHandlersE2ETest.java | Adds Java E2E coverage for pending handler RPCs (session limits + MCP headers refresh). |
| java/src/test/java/com/github/copilot/RpcSessionStateExtrasE2ETest.java | Adds Java E2E coverage for session-state extras scenarios (provider/model, completions, visibility, metadata, subagent settings). |
| java/src/test/java/com/github/copilot/RpcServerMiscE2ETest.java | Adds Java E2E coverage for LLM response-frame rejection + user settings + account auth lifecycle. |
| java/src/test/java/com/github/copilot/McpOAuthE2ETest.java | Adds Java E2E coverage for resolving pending MCP OAuth request via direct RPC. |
| java/src/test/java/com/github/copilot/ClientOptionsE2ETest.java | Adds Java E2E tests validating create/resume option forwarding via a fake stdio CLI capture. |
| java/src/main/java/com/github/copilot/JsonRpcClient.java | Adjusts Jackson inclusion policy so explicit null map values are serialized (while still omitting null properties). |
| go/internal/e2e/testharness/context.go | Adds Go harness helper to configure proxy without loading a snapshot. |
| go/internal/e2e/rpc_tasks_and_handlers_e2e_test.go | Ports pending handler scenarios to Go E2E (session limits + MCP headers refresh). |
| go/internal/e2e/rpc_session_state_extras_e2e_test.go | Ports session-state extras scenarios to Go E2E (provider/model, completions, visibility, metadata, subagent settings). |
| go/internal/e2e/rpc_server_misc_e2e_test.go | Adds Go E2E for user settings get/set/clear and account auth lifecycle; adds helper for “no token” client. |
| go/internal/e2e/rpc_server_e2e_test.go | Adds Go E2E for rejecting LLM inference response frames with missing request IDs. |
| go/internal/e2e/mcp_server_helpers_test.go | Refactors MCP server status wait helper to allow context + error returns (used by new tests). |
| go/internal/e2e/mcp_oauth_e2e_test.go | Adds Go E2E for resolving pending MCP OAuth request via direct RPC, using no-snapshot proxy setup. |
| go/internal/e2e/client_options_e2e_test.go | Adds Go E2E for advanced create/resume option forwarding via fake CLI capture; adds capture helpers. |
| go/client.go | Forwards ExtensionInfo into session create/resume wire requests. |
| dotnet/test/E2E/RpcTasksAndHandlersE2ETests.cs | Expands .NET E2E pending handler coverage (session limits + MCP headers refresh). |
| dotnet/test/E2E/RpcSessionStateExtrasE2ETests.cs | Expands .NET E2E for session-state extras (provider/model, completions, visibility, metadata, subagent settings). |
| dotnet/test/E2E/RpcServerMiscE2ETests.cs | Expands .NET E2E for user settings get/set/clear and account auth lifecycle; adds isolated-client helper options. |
| dotnet/test/E2E/RpcServerE2ETests.cs | Adds .NET E2E for rejecting missing-request LLM inference response frames; expands discovery RPC coverage assertions. |
| dotnet/test/E2E/McpOAuthE2ETests.cs | Adds .NET E2E for resolving pending MCP OAuth request via direct RPC. |
| dotnet/test/E2E/ClientOptionsE2ETests.cs | Adds .NET E2E for advanced create/resume option forwarding via fake CLI capture (plus provider create test). |
| dotnet/src/Generated/Rpc.cs | Updates generated JsonSerializerContext registrations to include root collection RPC result types. |
Review details
- Files reviewed: 47/48 changed files
- Comments generated: 3
- Review effort level: Low
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Accepted the Java consistency feedback and addressed it in this PR rather than leaving it for follow-up. Java now exposes Generated by Copilot |
|
Accepted the remaining actionable consistency observation. Go and Java now also cover the MCP headers refresh Generated by Copilot |
This comment has been minimized.
This comment has been minimized.
|
Accepted this parity note and addressed it in Generated by Copilot |
This comment has been minimized.
This comment has been minimized.
|
Accepted this Java parity gap and addressed it in 29938e7. Java now has a dedicated Generated by Copilot |
This comment has been minimized.
This comment has been minimized.
Add C# E2E coverage for previously unexercised generated RPC methods, advanced session option forwarding, runtime provider registration, account/user settings flows, session metadata helpers, and handler callbacks. Fix C# RPC code generation so root collection and dictionary result types are included in the source-generated JSON context. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Replicate the newly added C# E2E coverage across Node, Python, Go, Rust, and Java, including RPC option forwarding, pending handler paths, MCP OAuth handling, account/user settings flows, and session state extras. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Resolve review feedback by removing unused/test-only variables, tightening cleanup exception handling, fixing stale comments, and making the Python MCP OAuth wait explicit. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adjust the Rust account lifecycle E2E assertion so CodeQL can see the matched user binding is used outside an assertion macro. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Make MCP OAuth waits explicit in Python, rewrap Rust OAuth assertions, and avoid CodeQL macro-only bindings in the Rust account lifecycle test. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add Java session and resume support for requestExtensions, extensionSdkPath, and ExtensionInfo so create/resume requests match the other SDKs. Extend the client options E2E coverage to assert those options are forwarded. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add Go and Java E2E assertions for the MCP headers refresh none response variant so their missing-pending-handler coverage matches Node and .NET. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Cover the server discovery-path, agent discovery, and instruction discovery RPCs in the Node, Python, Go, Rust, and Java E2E suites so they match the .NET server RPC coverage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add a dedicated Java RpcServerE2ETest that exercises server-scoped RPC wrappers in parity with the other SDKs. Update Java RPC codegen so server-scoped session methods keep their required sessionId params while session-scoped wrappers continue to inject sessionId automatically. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
29938e7 to
7fd2bfb
Compare
Cross-SDK Consistency Review ✅This PR improves cross-SDK consistency rather than creating any gaps. Here's the analysis: Bug fixes that restore parity
New E2E coverageNew tests are added uniformly across all six SDKs (Node.js, Python, Go, .NET, Java, Rust) with identical test scenario names and shared YAML snapshots, exercising:
No gaps introducedAll three SDK-level bug fixes bring Java and Go into alignment with the already-correct behavior in Node.js, Python, .NET, and Rust. The C# and Java codegen fixes are runtime/toolchain-specific and have no cross-SDK equivalents needed.
|
Summary
The SDK E2E suites had uneven coverage for generated RPCs and option/handler flows, which let gaps such as root collection result metadata and null-setting clears slip through. This PR expands the C# suite and ports the same real-world-ish scenarios across Node, Python, Go, Rust, and Java so the SDKs exercise the same surface area.
This adds coverage for advanced client/session/resume option forwarding, MCP OAuth pending requests, LLM response callbacks, account and user settings RPCs, provider/model/session-state RPCs, visibility, completions fallback, context metadata, subagent settings, and pending handler methods. It also adds shared replay snapshots for the new cross-SDK scenarios.
The tests exposed and fix a few SDK issues: C# source-generation metadata now includes root collection RPC results, Go forwards
ExtensionInfoduring create/resume, and Java preserves explicit null map values in JSON-RPC params so user setting clears reach the CLI.Validation
./mvnw.cmd verifyGenerated by Copilot