gRPC server instrumentations (grpc-1.5, armeria-grpc) don't make extracted W3C baggage current in the handler

[sy-lee]

Tracer Version(s)

1.56.0, 1.66.0 (observed); master at 02d50d2 (code reading)

Java Version(s)

17.0.13

JVM Vendor

Eclipse Adoptium / Temurin

Bug Report

gRPC server instrumentations (grpc-1.5 and armeria-grpc-0.84) extract inbound W3C baggage but don't make it current while the service runs. As a result:

• Baggage.current() (OTel API, DD_TRACE_OTEL_ENABLED=true) is empty inside the handler.
• Downstream calls from the handler carry no baggage header, even though the trace context (x-datadog-*, traceparent) propagates fine.

The baggage.* span tags still show up on the grpc.server span, which makes the loss easy to miss.

Root cause. Both TracingServerInterceptors call the deprecated AgentPropagation.extractContextAndGetSpanContext (internal-api, v1.66.0 L33-L38).

• That helper extracts the full Context and then returns only the span context, so the extracted Baggage element is discarded.
• The interceptor then runs next.startCall(...) and every listener callback under activateSpan(span), which is a span-only context.
• The tags appear anyway because BaggagePropagator.extract also copies the baggage onto the extracted TagContext (v1.66.0 L130-L138).

Call sites:

• armeria-grpc-0.84: extract L71, activate L94
• grpc-1.5: extract L72, activate L95

HTTP servers don't have this problem. They were moved to full-context extraction in #8820: HttpServerDecorator.extract returns the whole Context, and e.g. JettyServerAdvice attaches parentContext.with(span) (v1.66.0 L32-L61).

Observed (dd-java-agent 1.56.0 and 1.66.0; Armeria 1.33.4 and 1.41.0). An agent-less client sends baggage: jtbd=ap.curl,user.id=u1 plus x-datadog-* headers to a relay server, whose handler logs Baggage.current() and makes one outbound OkHttp call:

relay server	Baggage.current() in handler (OTel bridge on)	baggage header on the relay's outbound call	baggage.jtbd tag on server span	trace joins
Armeria GrpcService	{}	none	yes	yes
Jetty 11 servlet	{jtbd=ap.curl, user.id=u1}	jtbd=ap.curl,user.id=u1	yes	yes

The result was the same with the OTel bridge unset, apart from the Baggage.current() column, which the bridge feeds. I did not run a grpc-java (Netty) server in that harness. For grpc-1.5, the unit test below reproduces the same loss.

Other callers of the deprecated helper (master at 02d50d2, non-test code). I found these by reading the code and have not reproduced them. They look like they have the same shape: extract, keep only the span context, then activate a span-only context.

• RPC: sofarpc-5.0 (ProviderProxyInvokerInstrumentation)
• Messaging consumers: kafka-clients-0.11 and kafka-clients-3.8 (TracingIterator), aws-java-sqs-1.0 and aws-java-sqs-2.0 (TracingIterator), javax-jms-1.1 (JMSMessageConsumerInstrumentation, DatadogMessageListener), rabbitmq-amqp-2.7 (RabbitDecorator), google-pubsub-1.116 (PubSubDecorator)
• API shims: opentracing-0.31 / 0.32 (OTTracer), opentelemetry-0.3 (OtelContextPropagators)

I've seen #11286 and the note there that W3C baggage was initially scoped to HTTP. gRPC carries baggage as an HTTP/2 header, and these servers already extract and tag it, so it seemed worth reporting on its own. Please relabel as a feature request if that fits your conventions better.

Expected Behavior

When a gRPC server span is started from extracted headers, the rest of the extracted context (W3C baggage) is current for the whole call, as it is for HTTP servers. In particular:

• Baggage.current() in the service implementation returns the inbound members.
• Outbound calls made from the handler re-inject the baggage header.

Reproduction Code

Minimal Spock test against grpc-1.5. The same shape against an Armeria GrpcService fails the same way.

class GrpcServerBaggageTest extends InstrumentationSpecification {
  def "inbound baggage is current in unary handler"() {
    setup:
    Map<String, String> seen = null
    def greeter = new GreeterGrpc.GreeterImplBase() {
        @Override
        void sayHello(Helloworld.Request req, StreamObserver<Helloworld.Response> obs) {
          seen = Baggage.fromContext(Context.current())?.asMap()
          obs.onNext(Helloworld.Response.newBuilder().setMessage("hi").build())
          obs.onCompleted()
        }
      }
    def name = InProcessServerBuilder.generateName()
    def server = InProcessServerBuilder.forName(name).addService(greeter).directExecutor().build().start()
    def md = new Metadata()
    md.put(Metadata.Key.of("baggage", Metadata.ASCII_STRING_MARSHALLER), "user.id=abc123")
    def channel = InProcessChannelBuilder.forName(name)
      .intercept(MetadataUtils.newAttachHeadersInterceptor(md)).directExecutor().build()

    when:
    GreeterGrpc.newBlockingStub(channel).sayHello(Helloworld.Request.newBuilder().setName("x").build())

    then:
    seen == ["user.id": "abc123"]   // on master: seen == null (a span is current, baggage is not)

    cleanup:
    channel.shutdownNow(); server.shutdownNow()
  }
}