fix(bedrock): return native tool calls from streaming when no available_functions - #7439
alanhuangyoo wants to merge 1 commit into
Conversation
…le_functions The agent executor calls the LLM with available_functions=None and runs the returned tool calls itself. The non-streaming Converse path returns them, but both streaming handlers dropped the call and returned the empty-content fallback, so with stream=True the tool never ran. Collect the tool uses and return them after the stream, as the non-streaming path does. Fixes crewAIInc#7438 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review. 📝 WalkthroughWalkthroughBedrock synchronous and asynchronous streaming handlers now preserve native tool calls when ChangesBedrock streaming tool calls
Suggested reviewers: Priority: ➖ Normal Severity of issue fixed: Medium Merge Risk: ⚪ Minimal · up to Streaming Bedrock tool calls without configured functions are now returned to the executor rather than replaced by a fallback response. The change includes sync and async regression coverage and has no remaining actionable merge risk. 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Related issue
Fixes #7438
Summary
With
stream=True, Bedrock drops native tool calls: the tool never runs and the task answer is the "I apologize, but I couldn't generate a response" fallback.The agent executor calls the LLM with
available_functions=Noneand executes the returned tool calls itself._handle_conversereturns the tool uses in that case (completion.py:741-752). Both streaming handlers only used a tool call whenavailable_functionswas set and otherwise discarded it, so the stream ended with no text and returned the fallback.The streaming handlers now collect the parsed tool uses when
available_functionsis empty (skipping the structured-output tool, as the non-streaming path does) and return them after the stream, with the sameTOOL_CALLcompleted event. Streams that call tools withavailable_functionsset, or return text, are unchanged.Verification
Tests added or updated for the changed behavior
Relevant tests and quality checks pass locally
Two new tests in
tests/llms/bedrock/test_bedrock_streaming_tool_args.py(sync and async) call the streaming handler withoutavailable_functionsand expect the tool use back. Both fail onmain. I moved the async stream helper to module level so both async tests share it.tests/llms/bedrock: 39 passed, 8 skipped.A crew run with a mocked
converse_streamnow runs the tool withstream=True(tool ran with=['Paris']), the same asstream=False.ruff 0.15.1 with the repo config passes. mypy reports the same errors for
completion.pybefore and after (local stubs).Additional context
The OpenAI Responses API streaming handler has the same gap (
_handle_streaming_responsesonly uses collected tool calls whenavailable_functionsis set). I left it out to keep this PR to one provider.This PR was written with AI assistance (Claude Code). I can't add the
llm-generatedlabel as an outside contributor; could a maintainer add it?🤖 Generated with Claude Code