[RDMA] Data race and partial state publication during TCP fallback

[legionxiong]

Description

On current master (e0444287), the RDMA handshake state is accessed
concurrently but is stored in a non-atomic field:

// rdma_endpoint.h
State _state;

RdmaEndpoint::_state is read or written by multiple execution contexts,
including:

• the RDMA handshake bthread;
• the TCP event handling path in OnNewDataFromTcp();
• the RDMA completion path in HandleCompletion().

This constitutes a C++ data race.

There is also a dual-state publication issue when RDMA is unavailable.
RdmaConnect::StartConnect() currently publishes the two related states in
this order:

rdma_transport->_rdma_ep->_state = RdmaEndpoint::FALLBACK_TCP;
rdma_transport->_rdma_state = RdmaTransport::RDMA_OFF;

The endpoint state is used by OnNewDataFromTcp() to decide whether normal TCP
message processing can begin:

if (ep->_state == FALLBACK_TCP) {
    InputMessenger::OnNewMessages(m);
    return;
}

A concurrent TCP event can observe FALLBACK_TCP before _rdma_state has been
updated to RDMA_OFF. This exposes a partially published fallback state to the
TCP processing path.

Race sequence

sequenceDiagram
    participant C as Connection/StartConnect bthread
    participant E as TCP event bthread
    participant M as InputMessenger

    Note over C: RDMA is unavailable

    C->>C: ep->_state = FALLBACK_TCP
    Note over C,E: C may be descheduled here

    E->>E: OnNewDataFromTcp(socket)
    E->>E: observe ep->_state == FALLBACK_TCP
    E->>M: InputMessenger::OnNewMessages(socket)
    Note over E,M: rdma_transport->_rdma_state<br/>may still be RDMA_UNKNOWN or RDMA_ON

    C->>C: rdma_transport->_rdma_state = RDMA_OFF
    Note over C: The socket-level state is published too late

Loading

Equivalent thread interleaving:

StartConnect bthread                    TCP event bthread
--------------------                    -----------------
ep->_state = FALLBACK_TCP;

             < context switch >

                                        OnNewDataFromTcp(socket);

                                        if (ep->_state == FALLBACK_TCP) {
                                            InputMessenger::OnNewMessages(
                                                socket);
                                            // _rdma_state may not be RDMA_OFF
                                        }

rdma_transport->_rdma_state = RDMA_OFF;

The invalid intermediate state is:

RdmaEndpoint::_state == FALLBACK_TCP
RdmaTransport::_rdma_state != RDMA_OFF

Expected behavior

Once OnNewDataFromTcp() observes:

ep->_state == FALLBACK_TCP

the corresponding transport state must already be visible as:

rdma_transport->_rdma_state == RdmaTransport::RDMA_OFF

The fallback transition must not expose a partially published state.

Impact

The exact effect depends on timing and the code consuming _rdma_state.
Possible consequences include:

• entering normal TCP message processing while the transport still appears to
  be in an RDMA state;
• inconsistent TCP/RDMA path selection;
• undefined behavior caused by concurrent non-atomic accesses to _state.

An older downstream version detected this inconsistency with an invariant
CHECK in its TCP read path. Current upstream master no longer contains that
CHECK after the transport refactoring, but the underlying data race and
partial publication window remain.