[BUG] Draw the retry jitter from a per thread generator

[thc1006]

Fixes #4398, the first of the three things on it.

NextRetryTime() multiplies the backoff by a sample from a static engine:

  static std::random_device rd;
  static std::mt19937 gen(rd());
  static std::uniform_real_distribution<float> dis(0.8f, 1.2f);
  ...
  backoff *= dis(gen);

Drawing from an engine advances its state, so dis(gen) is a write. Every HttpClient runs its own background thread and each one calls this from its own retry loop, so two clients retrying at the same time write the same std::mt19937. C++ guarantees the initialisation of a function local static is thread safe; it says nothing about using one afterwards.

The engine becomes thread_local, seeded per thread from a temporary random_device. The distribution becomes an ordinary local, since it was only static out of habit and sharing it bought nothing.

Evidence

NextRetryTime() is public and, unlike IsRetryable(), its body is not behind ENABLE_OTLP_RETRY_PREVIEW, so the static compiles into every build and a test can reach it without a retrying server. Two operations, two threads, 200 draws each:

RetryJitterIsNotSharedAcrossThreads under TSAN	warnings per run
main at f6e4818 with this test added	3, 3, 3
this branch	0, 0, 0

WARNING: ThreadSanitizer: data race
  Read of size 8 by thread T3:
    #0 std::mersenne_twister_engine<...>::operator()() /usr/include/c++/14/bits/random.tcc:458
    #5 HttpOperation::NextRetryTime() ext/src/http/client/curl/http_operation_curl.cc:641

The case passes on main too, because a data race is not a functional failure. It is here so the sanitizer builds keep it fixed.

What this does not touch

#4398 has two more items and neither is here. The retry deadline is redrawn every time doRetrySessions() asks for it, so it is not a stable point in time, and IsRetryable() does not consider whether the request was cancelled. Both are tangled with the retry queue and with the teardown ordering in #4391, and both are worth deciding rather than guessing.

Checks

23 of 23 in curl_http_test. Clean under OTELCPP_MAINTAINER_MODE=ON and clang-format 18.1.8. clang-tidy at zero delta against main under the CI filters. IWYU clean on all three variants, all-options-abiv1, all-options-abiv1-preview and all-options-abiv2-preview, each checked against a deliberately unused include first so the zero means something.

Related

#4392, #4394 and #4395 are open against the same two files. This one only touches NextRetryTime, which none of them go near, so the production sides do not overlap. The test file is shared, and I will rebase whichever of them lands second.

For significant contributions please make sure you have completed the following items:

• CHANGELOG.md updated for non-trivial changes
• Unit tests have been added
• Changes in public API reviewed

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.92%. Comparing base (66d226f) to head (e053d09).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #4399   +/-   ##
=======================================
  Coverage   81.92%   81.92%           
=======================================
  Files         494      494           
  Lines       19658    19658           
=======================================
  Hits        16103    16103           
  Misses       3555     3555           

Files with missing lines	Coverage Δ
ext/src/http/client/curl/http_operation_curl.cc	59.84% <100.00%> (-0.06%)	⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[lalitb]

Thanks.

[marcalff]

LGTM, thanks for the fix.

[marcalff]

Do not auto-merge: Waiting for full ci to see the tsan tests.