feat: add hog harness, a pi distribution#3219
Conversation
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
|
React Doctor found no issues in the changed files. 🎉 Reviewed by React Doctor for commit |
|
|
|
||
| if (parsed.username || parsed.password) { | ||
| return { | ||
| valid: false, | ||
| reason: "URLs with embedded credentials are not supported", | ||
| }; | ||
| } | ||
|
|
||
| const parts = parsed.hostname.split("."); | ||
| if (parts.length < 2) { | ||
| return { valid: false, reason: "URL must have a public hostname" }; | ||
| } | ||
|
|
||
| return { valid: true, url: parsed }; | ||
| } | ||
|
|
||
| interface CacheEntry { | ||
| bytes: number; | ||
| code: number; |
There was a problem hiding this comment.
SSRF via private/loopback IP addresses
validateUrl only checks that the hostname has 2 or more dot-separated segments, which blocks localhost but passes every numeric IP (127.0.0.1, 0.0.0.0, 192.168.x.x, 10.x.x.x, 172.16-31.x.x, 169.254.169.254). A prompt-injected instruction like web_fetch("http://169.254.169.254/latest/meta-data/iam/security-credentials/", ...) would pass validation and reach the AWS metadata endpoint (or any other internal service) from within the agent's network. None of the existing validateUrl tests cover IP literals, confirming the gap.
There was a problem hiding this comment.
Fixed in 0fa4044: validateUrl now rejects loopback/private/link-local/CGNAT IPv4 and IPv6 literal hostnames (127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16 incl. cloud metadata, 100.64.0.0/10, 198.18.0.0/15, ::1, fe80::/10, fc00::/7, IPv4-mapped IPv6). Added test coverage including the exact 169.254.169.254 case. Noted in a code comment that this is a literal-address check only — it doesn't protect against DNS rebinding (a public hostname whose DNS record later resolves to a private address), which would need to be enforced at connection time.
| cached.content, | ||
| params.prompt, | ||
| gateway, | ||
| signal, | ||
| ); | ||
| return { | ||
| content: [{ type: "text" as const, text: summary }], | ||
| details, | ||
| }; | ||
| } | ||
|
|
||
| return returnMarkdown(cached.content, details); | ||
| } | ||
|
|
||
| const controller = new AbortController(); | ||
| const timeout = setTimeout(() => controller.abort(), FETCH_TIMEOUT_MS); | ||
| const combinedSignal = signal | ||
| ? AbortSignal.any([signal, controller.signal]) | ||
| : controller.signal; | ||
|
|
There was a problem hiding this comment.
Content-length header absence bypasses the body-size guard
When a server omits content-length (typical for chunked-encoded or streaming responses), contentLength falls through to 0, making 0 > MAX_CONTENT_LENGTH false and skipping the guard entirely. The subsequent response.text() call then reads the entire body with no ceiling. A server that responds with chunked transfer encoding and streams indefinitely would exhaust process memory.
There was a problem hiding this comment.
Fixed in 0fa4044: added readBodyWithLimit(), which streams the response body via response.body.getReader() and enforces MAX_CONTENT_LENGTH as bytes actually arrive, cancelling the reader and throwing once the cap is exceeded — regardless of whether/what the content-length header says. Kept the header-based check as a fast-fail before reading anything when the server is honest about a huge size, but the streaming cap is now the real backstop. Added tests for both an oversized chunked/no-content-length response (rejected) and a small chunked response with no content-length (still works).
| res.writeHead(200, { "Content-Type": "text/html; charset=utf-8" }); | ||
| res.end(SUCCESS_PAGE); | ||
| cleanup(); | ||
| if (error) { | ||
| reject(new Error(`PostHog OAuth error: ${error}`)); | ||
| } else if (!code) { | ||
| reject(new Error("PostHog OAuth callback missing code")); | ||
| } else if (expectedState && state !== expectedState) { | ||
| reject(new Error("PostHog OAuth state mismatch")); |
There was a problem hiding this comment.
The success page is sent to the browser regardless of whether the code/state validation will pass. If there is a state mismatch or an OAuth error parameter, the user's browser shows "Authentication complete" while the CLI is about to reject the login. Consider serving a different page on the error path.
| res.writeHead(200, { "Content-Type": "text/html; charset=utf-8" }); | |
| res.end(SUCCESS_PAGE); | |
| cleanup(); | |
| if (error) { | |
| reject(new Error(`PostHog OAuth error: ${error}`)); | |
| } else if (!code) { | |
| reject(new Error("PostHog OAuth callback missing code")); | |
| } else if (expectedState && state !== expectedState) { | |
| reject(new Error("PostHog OAuth state mismatch")); | |
| const hasError = | |
| !!error || !code || (expectedState && state !== expectedState); | |
| res.writeHead(200, { "Content-Type": "text/html; charset=utf-8" }); | |
| res.end(hasError ? SUCCESS_PAGE.replace("Authentication complete", "Authentication failed — please return to your terminal") : SUCCESS_PAGE); | |
| cleanup(); | |
| if (error) { | |
| reject(new Error(`PostHog OAuth error: ${error}`)); | |
| } else if (!code) { | |
| reject(new Error("PostHog OAuth callback missing code")); | |
| } else if (expectedState && state !== expectedState) { | |
| reject(new Error("PostHog OAuth state mismatch")); |
There was a problem hiding this comment.
Fixed in 0fa4044, thanks — took your suggestion's approach: the callback handler now determines success/failure first, then serves a distinct failure page ("Authentication failed — please return to your terminal") on error/missing-code/state-mismatch instead of always serving the success page. Added test coverage asserting the served page body for the success path and all three failure paths.
| /** | ||
| * Writes the run's transcript, capped so a very long-running or verbose | ||
| * subagent can't grow an unbounded file on disk. | ||
| */ | ||
| export function writeTranscript( | ||
| runId: string, | ||
| markdown: string, | ||
| maxBytes: number = DEFAULT_MAX_TRANSCRIPT_BYTES, | ||
| ): void { | ||
| const filePath = transcriptPath(runId); | ||
| fs.mkdirSync(path.dirname(filePath), { recursive: true }); | ||
|
|
||
| const byteLength = Buffer.byteLength(markdown, "utf-8"); |
There was a problem hiding this comment.
O(n) character-by-character trim loop for multibyte text
When the transcript contains multibyte characters near the truncation point, the while (Buffer.byteLength(truncated) > maxBytes) truncated = truncated.slice(0, -1) loop removes one character at a time and re-computes Buffer.byteLength over the entire remaining string on each iteration. A binary-search approach or a TextEncoder-based forward scan would be significantly more efficient.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
Fixed in 0fa4044. Style matches our conventions — extracted a shared text-truncate.ts with a single Buffer.from(text, "utf8") encode + at-most-3-byte backoff to the nearest UTF-8 codepoint boundary, used by both lifecycle.ts's writeTranscript and format.ts's truncateForModel (which had the identical O(n) pattern). Added dedicated tests including multi-byte (2-byte é, 4-byte emoji, 3-byte €) boundary cases to confirm no split codepoints/replacement characters.
…or page, transcript truncation perf) - web-fetch: reject loopback/private/link-local IPv4 and IPv6 literal hostnames in validateUrl (e.g. 169.254.169.254 cloud metadata, 10.x/172.16-31.x/192.168.x, ::1, fc00::/7, IPv4-mapped IPv6). Literal-address check only; does not protect against DNS rebinding. - web-fetch: enforce MAX_CONTENT_LENGTH while streaming the response body (readBodyWithLimit), not just via the content-length header, which is absent for chunked/streamed responses and let a server exhaust memory. - oauth: don't serve the OAuth success page before validating the callback; serve a distinct failure page on error/missing-code/state-mismatch so the browser doesn't say "Authentication complete" right before the CLI login rejects. - subagent: replace the O(n) per-iteration byte-length truncation loop in lifecycle.ts/format.ts with a single-pass UTF-8-boundary-aware truncation (text-truncate.ts), shared by both call sites. Addresses review comments from #3219.
Problem
We need a debuggable, stable, and modifiable harness.
Changes
harness, using Pi.dev under the hood. This harness can be launched using either the SDK or the CLI.subagent,web-access,posthog-provider, andhog-branding. I've used this community extension for MCP support: https://github.com/nicobailon/pi-mcp-adapter