Support LoRA publication to remote inference engines

[hershg]

Summary

• Apply the selected LoRA adapter in the routed generation endpoint.
• Stream checksum- and size-verified adapters to disaggregated inference nodes.
• Let FSDP and Megatron select the existing shared-filesystem path or remote upload.
• Fail closed on partial publication: wait for every replica, unload the named adapter on failure, and remove staged files.

Validation

• Focused client/server suite: 80 passed; the full training suite passes.
• Tests cover fan-out, checksums, declared and maximum sizes, partial-replica rollback, staged-file cleanup, routed generation, and successful load.
• The three Qwen3 JAX MoE cases that failed once in broad CI pass locally unchanged (3/3); their code is outside this PR's diff.
• Two-node B300 GLM-5.3 cold replay on SkyRL 4f5ccd8e: zero mean .006000; withheld sampler change 0; post-update mean .000706; held-out mean .000761.
• Export and receiver digests matched for both publications; checkpoint, unload, trainer offload, and runtime release passed.

The live replay required remote_upload: trainer-local storage is not visible on the disaggregated inference node. The upload endpoint follows the same private control-plane trust boundary as SkyRL's existing load, pause, and weight-update endpoints; this PR bounds each streamed file to 128 GiB and cleans failed or abandoned client publications, but does not introduce a separate authentication system for one endpoint.

Replica rollback is fail-closed rather than transactional: if any replica rejects publication, SkyRL removes the named adapter from every reachable replica and raises. A versioned two-phase router cutover would be a broader control-plane change and is not claimed here.

---

Note

Medium Risk
Changes inference control-plane LoRA delivery and generation routing; failures are handled with rollback, but multi-replica publication is not fully transactional and upload endpoints rely on the existing private trust boundary.

Overview
Adds remote_upload as an alternative to shared-disk LoRA sync for disaggregated vLLM inference: training workers can push adapter_model.safetensors and adapter_config.json over HTTP with declared size and SHA-256 checks, then load by upload_id on each replica.

RemoteInferenceClient fans out chunked PUTs, verifies server-returned digests, publishes via /skyrl/v1/load_lora_adapter, and on partial failure unloads the adapter name everywhere and raises; staged upload dirs are always discarded afterward. VLLMServerActor implements the upload/load/discard endpoints (128 GiB cap, streaming validation) and extends /skyrl/v1/generate to honor a model LoRA name instead of rejecting LoRA on that route.

policy.model.lora.transfer_mode (shared_filesystem | remote_upload) is wired from FSDP and Megatron export paths into load_lora_adapter; default behavior stays shared filesystem.

^{Reviewed by Cursor Bugbot for commit 49b65c3. Bugbot is set up for automated code reviews on this repo. Configure here.}

[gemini-code-assist]

Code Review

This pull request introduces a new 'remote_upload' transfer mode for LoRA adapters, enabling training workers to stream and upload adapter files directly to remote vLLM inference servers via HTTP PUT endpoints with SHA256 checksum verification. The review feedback focuses on improving robustness and performance: it recommends replacing Python 3.11's hashlib.file_digest with a chunked fallback for backward compatibility, wrapping JSON parsing in a try-except block to prevent masking HTTP errors, and offloading blocking synchronous I/O operations (such as shutil.rmtree and os.remove) to background threads using asyncio.to_thread to avoid blocking the event loop.

[greptile-apps]

Confidence Score: 2/5

The PR is not safe to merge until remote publication handles partial replica failures and the externally reachable upload route bounds disk consumption.

Findings

1.  Partial Publication Splits Replicas ▶
2.   Uploads Can Exhaust Disk ▶
3.  Failed Uploads Leak Storage ▶

Diagram

sequenceDiagram
    participant W as Training worker
    participant C as RemoteInferenceClient
    participant A as Inference replica A
    participant B as Inference replica B
    participant R as Request router

    W->>C: Publish named LoRA
    par Upload and load replica A
        C->>A: PUT weights + config
        C->>A: POST load(upload_id)
        A-->>C: Adapter loaded
    and Upload and load replica B
        C->>B: PUT weights + config
        B--xC: Upload/load failure
    end
    C--xW: Publication raises
    Note over A,B: Replicas can now retain different adapter versions
    R->>A: "Routed generation(model=LoRA)"
    R->>B: "Routed generation(model=LoRA)"

Loading

_{Reviews (1) · Last reviewed commit: "Record remote LoRA upload digests"}

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit 223abe4. Configure here.}