feat: expose Otari request IDs

[HareeshBahuleyan]

Why

Successful inference responses expose X-Otari-Request-ID, but the SDK discarded it. Callers could not correlate Otari requests with routing, latency, or usage telemetry.

What changed

Added an opt-in with_response_metadata API for chat completions, Responses API calls, and Messages API calls. Non-streaming calls return OtariResponse; streaming calls return sync or async stream wrappers that expose the request ID without modifying SSE events or the existing default return types.

Notes

• Stream request IDs become available when iteration opens the HTTP response, before the first event is yielded.
• Verified with uv run pytest -q, uv run ruff check ., uv run mypy src/, and uv build.

[Copilot]

Pull request overview

Adds opt-in request ID metadata for synchronous and asynchronous inference calls without changing existing default return types.

Changes:

• Adds response and stream metadata wrappers.
• Exposes metadata-enabled chat, Responses, and Messages APIs.
• Documents and tests request ID extraction.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
src/otari/response_metadata.py	Defines response and stream wrappers.
src/otari/client.py	Adds synchronous metadata APIs.
src/otari/async_client.py	Adds asynchronous metadata APIs.
src/otari/__init__.py	Exports metadata wrappers.
tests/unit/test_client.py	Tests synchronous metadata handling.
tests/unit/test_async_client.py	Tests asynchronous metadata handling.
README.md	Documents metadata usage.

Suppressed comments (4)

src/otari/async_client.py:597

• After awaiting this method, even an omitted stream is typed as a response-or-stream union, preventing direct .data access. Add overloads for literal streaming and non-streaming modes, plus a union overload for a runtime bool, consistent with src/otari/async_client.py:160-178.

    ) -> OtariResponse[Any] | AsyncOtariStream[dict[str, Any]]:

src/otari/async_client.py:617

• The documented async non-streaming usage still infers OtariResponse[MessageResponse] | AsyncOtariStream[...], so callers must narrow before accessing .data. Add overloads that distinguish omitted/false stream, literal True, and a general runtime bool.

    ) -> OtariResponse[MessageResponse] | AsyncOtariStream[dict[str, Any]]:

src/otari/client.py:616

• This public signature leaves non-streaming calls typed as a response-or-stream union even when stream is omitted, so callers cannot access .data without manual narrowing. Add overloads for stream=True, stream=False/omitted, and a general bool, following the streaming overload pattern at src/otari/client.py:167-185.

    ) -> OtariResponse[Any] | OtariStream[dict[str, Any]]:

src/otari/client.py:636

• This signature makes the documented non-streaming call infer OtariResponse[MessageResponse] | OtariStream[...], so the README's result.data access fails static type checking. Add overloads that return OtariResponse for omitted/false stream and OtariStream for literal True, plus a union overload for a runtime bool.

    ) -> OtariResponse[MessageResponse] | OtariStream[dict[str, Any]]:

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

[njbrake]

@HareeshBahuleyan thanks! Does the fix belong here, or should it go into the otari repo which is responsible for auto-generating most of this SDK? I forget if the part you're editing is autogenerated or not.

[HareeshBahuleyan]

@njbrake I think it belongs here.

• Otari already emits X-Otari-Request-ID for both paths:

  • Non-streaming
  • Streaming

• The request ID is HTTP response metadata, not part of the generated response-body models. Non-streaming exposes it through the generated ApiResponse, while streaming receives it through the hand-written httpx path.

• The hand-written SDK shell is therefore the shared layer that can expose this metadata consistently across both paths.

[njbrake]

Approving. Thanks for the contribution, and for the discussion on placement: you were right, and it saved me from sending this to the wrong repo.

I checked your reasoning since I was not sure either. The gateway's codegen workflow (.github/workflows/otari-sdk-codegen.yml) writes only src/otari/_client and sdk-endpoints.txt, on branch sdk-codegen/client-core, so nothing this PR touches is in the regeneration path. The structural half holds too: headers are not in the generated body models, and streaming bypasses the core entirely, which leaves the shell as the only layer that can cover both paths.

Verified locally at e39d7b0: 138 unit tests pass, mypy src/ and ruff clean. Your fixes cover Copilot's overload comments, and its "Responses API has no coverage" comment is stale.

I pushed one doc commit rather than sending you back for it (b984024). The gateway emits this header only on the hybrid path: _pipeline.py:1037 sits inside if hybrid_mode:, and the streaming site at L2561 is gated on platform_request_id. A standalone self-hosted gateway never sends it, so request_id is None there, which a caller cannot tell apart from "iteration has not opened the response yet". The README now says so next to the example. Revert it if you disagree with the wording.

Nit, non-blocking: async completion has no metadata test, while sync covers completion for the non-streaming path.

Two things worth knowing beyond this PR:

This header is the lookup key for the platform's GET /api/v1/request-costs/{request_id}, so you have also made an existing cost API usable. The SDK cannot call it yet, since control_plane is built from the gateway root rather than the platform, but that becomes reachable later.

I filed parity issues for the other three shells and one upstream to make the header a declared contract, since the endpoint-coverage manifest cannot see it: mozilla-ai/otari-sdk-ts#39, mozilla-ai/otari-sdk-go#29, mozilla-ai/otari-sdk-rust#57, mozilla-ai/otari#572. Heads up that rust is harder than a port of your pattern: its generated core returns Result<Model, Error<...>> and discards success-path headers, so there is no *_with_http_info analogue to reach for. TypeScript has *Raw() and Go returns *http.Response, so those two are straightforward.

Note: this review was drafted by Claude Opus 5 via back-and-forth with @njbrake. The reasoning and decisions are his; the prose is Claude's.