(WIP) feat(client-reports): report log and metric byte outcomes - #1927
(WIP) feat(client-reports): report log and metric byte outcomes#1927JoshuaMoelans wants to merge 2 commits into
Conversation
Record the byte size of discarded logs and metrics under the `log_byte` and `trace_metric_byte` categories, alongside the existing `log_item` and `trace_metric` counts, as required by https://develop.sentry.dev/sdk/telemetry/client-reports/#log-byte-outcomes Byte sizes come from the serialized payload where an item is already serialized, and from a new allocation-free `sentry_value_t` size estimator on the `before_send_log` / `before_send_metric` and batcher overflow paths. The hooks take ownership of the value, so it is sized before the call. This also fixes the item count for batched envelope items: discarding a `log` or `trace_metric` item now reports every log in the batch via its `item_count` header instead of counting the whole item as one discard. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Instructions and example for changelogPlease add an entry to Example: ## Unreleased
### Features
- report log and metric byte outcomes ([#1927](https://github.com/getsentry/sentry-native/pull/1927))If none of the above apply, you can opt out of this check by adding |
| size_t | ||
| sentry__value_estimate_serialized_size(sentry_value_t value) | ||
| { | ||
| const thing_t *thing = value_as_thing(value); | ||
| if (!thing) { | ||
| if ((value._bits & TAG_MASK) == TAG_INT32) { | ||
| return 11; // upper bound, as in `-2147483648` | ||
| } | ||
| return value._bits == CONST_FALSE ? 5 : 4; // `false`, `true`, `null` | ||
| } | ||
| switch (thing_get_type(thing)) { | ||
| case THING_TYPE_LIST: { | ||
| const list_t *l = thing->payload._ptr; | ||
| size_t size = 2; // [] | ||
| for (size_t i = 0; i < l->len; i++) { | ||
| size += sentry__value_estimate_serialized_size(l->items[i]); | ||
| } | ||
| return l->len ? size + l->len - 1 : size; // separating commas | ||
| } | ||
| case THING_TYPE_OBJECT: { | ||
| const obj_t *o = thing->payload._ptr; | ||
| size_t size = 2; // {} | ||
| for (size_t i = 0; i < o->len; i++) { | ||
| size += strlen(o->pairs[i].k) + 3; // "key": | ||
| size += sentry__value_estimate_serialized_size(o->pairs[i].v); | ||
| } | ||
| return o->len ? size + o->len - 1 : size; // separating commas | ||
| } | ||
| case THING_TYPE_STRING: | ||
| return strlen(thing->payload._ptr) + 2; // surrounding quotes | ||
| case THING_TYPE_DOUBLE: | ||
| case THING_TYPE_INT64: | ||
| case THING_TYPE_UINT64: | ||
| default: | ||
| // upper bound for any 64-bit integer or double rendering | ||
| return 24; | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
Q: I'm a bit afraid of the overhead on this being called for every single log item, especially the recursive nature of the function... I have no insights into what size logs people are usually sending/discarding, so will need to do some benchmarking before committing to this...
any other approaches to keep track of the size are welcome; we could maybe store these things upon creating the sentry_value_t types and adding to the objects, but this is a larger rework of our internals so worth discussing
There was a problem hiding this comment.
(potentially we make this PR just fix the log count numbers, and do the rework with log_bytes counting separately)
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #1927 +/- ##
==========================================
+ Coverage 75.76% 75.82% +0.06%
==========================================
Files 93 93
Lines 22104 22170 +66
Branches 3935 3955 +20
==========================================
+ Hits 16746 16811 +65
+ Misses 4473 4472 -1
- Partials 885 887 +2 🚀 New features to boost your workflow:
|
1 participant
Record the byte size of discarded logs and metrics under the
log_byteandtrace_metric_bytecategories, alongside the existinglog_itemandtrace_metriccounts, as required byhttps://develop.sentry.dev/sdk/telemetry/client-reports/#log-byte-outcomes
Byte sizes come from the serialized payload where an item is already serialized, and from a new allocation-free
sentry_value_tsize estimator on thebefore_send_log/before_send_metricand batcher overflow paths. The hooks take ownership of the value, so it is sized before the call.This also fixes the item count for batched envelope items: discarding a
logortrace_metricitem now reports every log in the batch via itsitem_countheader instead of counting the whole item as one discard.Following https://develop.sentry.dev/sdk/telemetry/client-reports/#log-byte-outcomes