feat(langgraph): Gate prompt/response collection on data_collection option - #7175
Conversation
…ption Modify the LangGraph integration to respect the data_collection config for controlling whether prompts, responses, tool calls, and available tools are captured in spans. When data collection is enabled, the gen_ai.inputs flag controls request messages, tool calls, and available tools, while gen_ai.outputs controls the response text. Tool calls are gated on inputs because they are fed back to the model as input. Available tools are only gated once data collection is configured, since they were never gated on the legacy PII settings. When data collection is not configured, falls back to legacy send_default_pii and include_prompts settings for compatibility. Refs PY-2588 Refs #6748
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit f54e39e. Configure here.
Rename the data collection tests and the invoke span helper so the names state what is being verified, and drop the now-redundant docstrings.
Codecov Results 📊✅ 112296 passed | ⏭️ 6759 skipped | Total: 119055 | Pass Rate: 94.32% | Execution Time: 405m 42s 📊 Comparison with Base Branch
➖ Removed Tests (1)View removed tests
All tests are passing successfully. ✅ Patch coverage is 97.06%. Project has 2482 uncovered lines. Files with missing lines (1)
Coverage diff@@ Coverage Diff @@
## main #PR +/-##
==========================================
+ Coverage 90.20% 90.22% +0.02%
==========================================
Files 193 193 —
Lines 25370 25379 +9
Branches 9310 9320 +10
==========================================
+ Hits 22884 22897 +13
- Misses 2486 2482 -4
- Partials 1433 1437 +4Generated by Codecov Action |
![sentry-warden[bot]](https://avatars.githubusercontent.com/in/2755838?s=60&v=4){kind=small}
|
This integration sucks 😞 . |
|
@alexander-alderman-webb Got it sorted - the changes I had introduced broke the message deltas. Have moved where the input gate is so it doesn't affect that and added regression tests for this scenario 🚀 |
sentrivana
left a comment
sentrivana
left a comment
There was a problem hiding this comment.
Looks good, left some suggestions
| SPANDATA.GEN_AI_REQUEST_MESSAGES, | ||
| messages_data, | ||
| unpack=False, | ||
| if _should_record_inputs(integration): |
There was a problem hiding this comment.
We could move this check above, to line 215, and combine it with the len(args) check so that we only do the input messages manipulation if necessary. Cheap way to save some cycles and remove one level of nesting.
There was a problem hiding this comment.
It was originally there but I had to move it a little further down because input_messages are used in the creation of metadata information (such as token usage) for output messages that aren't gated on either flag (original bugbot error here) 😞
Looking at it again, I could add it move it up to the if input_messages conditional (saves us from the normalize_message_roles call), but not any higher.
| SPANDATA.GEN_AI_REQUEST_MESSAGES, | ||
| messages_data, | ||
| unpack=False, | ||
| if _should_record_inputs(integration): |
There was a problem hiding this comment.
Same here, we could combine this with the args check on L258.
| SPANDATA.GEN_AI_REQUEST_MESSAGES, | ||
| messages_data, | ||
| unpack=False, | ||
| if _should_record_inputs(integration): |
| SPANDATA.GEN_AI_REQUEST_MESSAGES, | ||
| messages_data, | ||
| unpack=False, | ||
| if _should_record_inputs(integration): |
There was a problem hiding this comment.
And here (goodness this integration is a mess)
There was a problem hiding this comment.
goodness this integration is a mess
Yeah, it needs a good overhaul 😭 😅
3 participants
Modify the LangGraph integration to respect the data_collection config for controlling whether prompts, responses, tool calls, and available tools are captured in spans.
When data collection is enabled, the gen_ai.inputs flag controls request messages, tool calls, and available tools, while gen_ai.outputs controls the response text. Tool calls are gated on inputs because they are fed back to the model as input. Available tools are only gated once data collection is configured, since they were never gated on the legacy PII settings.
When data collection is not configured, falls back to legacy send_default_pii and include_prompts settings for compatibility.
Refs #6748
Refs #6748