Skip to content

fix(server): tolerate malformed responses in triplet export - #574

Open
Rio Yu (rioyu123) wants to merge 2 commits into
microsoft:mainfrom
rioyu123:fix/triplet-malformed-responses
Open

Rio Yu (rioyu123) wants to merge 2 commits into
microsoft:mainfrom
rioyu123:fix/triplet-malformed-responses

Conversation

@rioyu123

Copy link
Copy Markdown
Contributor

Summary

model_request events can contain JSON-valid provider responses whose nested choices, legacy chunks, or server metadata are not objects. The triplet formatter currently calls .get() on those values, failing the entire public format=triplet query.

This change adds narrow shape checks around choices, legacy chunks, token ID lists, and server metadata. Unrecognized fragments are skipped, while valid legacy fragments, HTTP status fields, provider errors, and model/version metadata keep their existing behavior.

Testing

  • Added public event API coverage for null choices, null legacy chunks, invalid choices, invalid legacy token IDs, null server metadata, and mixed valid/invalid legacy chunks.
  • Ran the focused server endpoint tests, Ruff checks, Ruff formatting checks, and Pyright.

Signed-off-by: Rio Yu <52408936+rioyu123@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings August 29, 2026 16:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Hardens triplet event export against malformed provider response shapes.

Changes:

  • Validates choices, legacy chunks, token lists, and server metadata.
  • Adds endpoint coverage for malformed and mixed response fragments.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
agentlightning/server/routes/events.py Adds defensive shape checks during triplet conversion.
tests/server/test_endpoints.py Tests malformed response handling.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

response_token_ids = choices[0].get("token_ids", [])
response_log_probs = _extract_choice_log_probs(choices[0])
prompt_token_ids = choice.get("prompt_token_ids", [])
response_token_ids = choice.get("token_ids", [])

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 58c0a4d. Non-list token_ids from non-streaming choices now normalize to an empty list, matching the legacy path. Added a parameterized public endpoint case; the full suite passes (78 passed, 3 skipped), with Ruff and Pyright clean.

Signed-off-by: Rio Yu <52408936+rioyu123@users.noreply.github.com>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These checks are at the right boundary for a public export formatter: JSON-valid provider payloads can still have the wrong nested shapes, and triplet export should degrade individual unrecognized fragments rather than fail the whole query. The mixed legacy-chunk regression is particularly useful because it proves malformed neighbours do not suppress valid token data. Normalizing non-dict server metadata separately also preserves the existing HTTP/error fields. No blocker.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants