Skip to content

Implements raw PJRT device<->host DMA transfers for PyTorch tensors, mirroring - #884

Open
copybara-service[bot] wants to merge 1 commit into
mainfrom
test_978013337
Open

Implements raw PJRT device<->host DMA transfers for PyTorch tensors, mirroring#884
copybara-service[bot] wants to merge 1 commit into
mainfrom
test_978013337

Conversation

@copybara-service

Copy link
Copy Markdown

Implements raw PJRT device<->host DMA transfers for PyTorch tensors, mirroring
the JAX _raw_transfer API:

  • Adds Python API under tpu_sync.api.torch.raw_transfer (transfer_d2h,
    transfer_h2d, transfer_d2h_async, transfer_h2d_async, and batch variants).
  • Adds RawHostBuffer and PreparedTorchRawTransfer for zero-overhead,
    repeated pinned DMA transfers.
  • Implements readiness synchronization (AwaitReady) using PJRT buffer ready futures
    before issuing DMA.
  • Implements host memory validation with pinned-memory check and fallback warning
    (or error when TPU_RAIDEN_RAW_REQUIRE_PINNED_HOST is set).
  • Adds unsafe_skip_buffer_lock parameter to bypass buffer mutex locking for
    concurrent execution.
  • Consolidates bindings into _tpu_raiden_torch as a raw_transfer submodule
    for wheel distribution, while preserving the standalone _torch_raw_transfer
    extension.
  • Updates build.sh, run_tests.sh, and ci/wheel/BUILD.bazel to package and test
    the raw transfer surface.
  • Adds single-device bandwidth smoke test (torch_raw_transfer_perf_test.py) and
    correctness unit tests (torch_raw_transfer_test.py).

@google-cla

google-cla Bot commented Sep 8, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

…mirroring

the JAX _raw_transfer API:
- Adds Python API under `tpu_sync.api.torch.raw_transfer` (`transfer_d2h`,
  `transfer_h2d`, `transfer_d2h_async`, `transfer_h2d_async`, and batch variants).
- Adds `RawHostBuffer` and `PreparedTorchRawTransfer` for zero-overhead,
  repeated pinned DMA transfers.
- Implements readiness synchronization (`AwaitReady`) using PJRT buffer ready futures
  before issuing DMA.
- Implements host memory validation with pinned-memory check and fallback warning
  (or error when `TPU_RAIDEN_RAW_REQUIRE_PINNED_HOST` is set).
- Adds `unsafe_skip_buffer_lock` parameter to bypass buffer mutex locking for
  concurrent execution.
- Consolidates bindings into `_tpu_raiden_torch` as a `raw_transfer` submodule
  for wheel distribution, while preserving the standalone `_torch_raw_transfer`
  extension.
- Updates `build.sh`, `run_tests.sh`, and `ci/wheel/BUILD.bazel` to package and test
  the raw transfer surface.
- Adds single-device bandwidth smoke test (`torch_raw_transfer_perf_test.py`) and
  correctness unit tests (`torch_raw_transfer_test.py`).

PiperOrigin-RevId: 978013337
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants