Expose the embedded browser as browser_* MCP tools - #332
Merged
Conversation
…-mcp Adds the 85 browser.* socket methods to the MCP tool catalog: 82 browser_* tools plus three focus_browser_* tools for the focus-intent methods. Corrects the tool count in the e2e catalog test, which had drifted past the six agent_* tools, and adds an end-to-end open, read-title, close flow.
open_split, tab_new, and design_mode_toggle do focus the new or target surface inside their workspace even though they never switch workspaces or raise the window; focus_browser_element and focus_browser_tab_switch do not select the workspace. Say so in the schemas, and assert in the e2e flow that the browser tools leave the selected workspace alone.
Member
Author
|
Codex cross-model review: 3 medium, 1 low. Verified against the handlers: the target_surface_id/pane_id finding was wrong (the handlers read both), the two focus-wording findings were right and are fixed in the follow-up commit, which also adds a selection-preservation assertion to the browser e2e flow. Alias-in-required (low) left as is; the aliases mirror the socket contract. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Agents that talk to Programa over MCP can now drive the embedded browser: open a split, navigate, read the accessibility snapshot or a screenshot, click and fill, and close the tab, without a Chrome extension and without moving the user's focus. Before this the 85
browser.*socket methods were only reachable through theprograma browserCLI.Summary
CLI-MCP/Tools/BrowserTools.swiftwith 82browser_*tools, schemas read from the handler bodies inTerminalController+BrowserAutomation.swift.browser.focus_webview,browser.focus,browser.tab.switch) go inFocusTools.swiftasfocus_browser_*, so the disruptive set stays auditable in one file.docs/mcp-server.mdgains a Browser tools section and drops browser automation from the "not exposed" list. Tools that WKWebView cannot support (network routing, viewport, raw input) stay in the list and returnnot_supported, which the doc says.agent_*tools existed in the catalog but not inEXPECTED_TOOL_NAMES. This PR corrects the baseline (102 before, 187 after) and adds a browser open, read-title, close flow.Review order:
BrowserTools.swiftfor schemas, thenFocusTools.swift, then the test.Test plan
test-e2erunstests_v2/test_mcp_server_e2e.pyand passes with 187 tools and 13focus_tools.xcodebuild -scheme programa-mcp buildsucceeds (verified locally).programa-mcpand callbrowser_open_splitwith a URL: a browser split appears next to the pane and the terminal keeps keyboard focus.