[FEA] Publish self-contained Java classifier JARs with a statically linked libcuopt, as cuDF does

[ramakrishnap-nv]

Is your feature request related to a problem? Please describe.

The Java bindings (#1524) have no installation path. Nothing is published — no jar, no libcuopt_jni.so — so a user must build the JNI shim from source. That needs libcuopt, its headers, a C++20 compiler, CMake, Ninja, Maven and a JDK.

Conda is effectively mandatory, and not by choice: the JNI build needs the exact rmm and raft headers libcuopt was compiled against, because rmm encodes its version in an inline namespace (rmm::_RMM_26_10). A mismatched copy links cleanly and then fails at dlopen with an undefined symbol. Those headers only ship as conda packages, so a pip wheel is not sufficient.

Verified end to end in a clean nvidia/cuda:13.0.3-base-ubuntu24.04 container: miniforge -> conda env -> build_native.sh -> mvn package -> compile and run a user program. It works, but that is the floor for "hello world".

Describe the solution you'd like

Adopt the cuDF model: publish self-contained per-classifier JARs that bundle a statically linked native library, so a user needs only a Maven dependency and a driver.

cuDF did this in rapidsai/cudf#23261 (merged 2026-07-21):

Link against a static libcudf built from source per CUDA version rather than a conda shared libcudf.

It builds with -DBUILD_SHARED_LIBS=OFF and splits the work into three composable stages — static native build, per-classifier JAR packaging, Maven-repo gather.

The result is genuinely self-contained. libcudf.so inside the published jar has these DT_NEEDED entries and nothing else:

librt, libz, libdl, libpthread, libstdc++, libm, libgcc_s, libc, ld-linux

No libcudart, no librmm, no conda. Published classifiers on Maven Central for ai.rapids:cudf:26.08.1:

cudf-26.08.1-cuda12.jar          707.9 MB
cudf-26.08.1-cuda12-arm64.jar    702.6 MB
cudf-26.08.1-cuda13.jar          321.5 MB
cudf-26.08.1-cuda13-arm64.jar    317.5 MB

For cuOpt this would also defuse #1703 as a distribution hazard. libcuopt_jni.so currently reaches through the private pdlp/cuopt_c_internal.hpp and is coupled to a specific libcuopt build. That is only dangerous because the user supplies libcuopt independently; if both halves are built and shipped together, a cuOpt point release cannot break the jar. It does not remove the need for #1703, but it removes the failure mode.

Describe alternatives you've considered

The cuVS model — a pure-Java jar. com.nvidia.cuvs:cuvs-java is 1.4 MB, ships a NativeDependencyLoader, and expects libcuvs from conda/pip/system. Simpler and far smaller, but it leaves the user to supply and match the native library, which is exactly the fragile configuration given #1703.

Reasonable as a stopgap; poor as an end state.

Additional context

Size is the open risk, and it is specific to cuOpt. libcuopt.so is 554 MB shared with 29 DT_NEEDED entries:

dependency	size
libcublas	517 MB
libnccl	279 MB
libcusparse	155 MB
libcudss	67 MB
libgrpc / protobuf / abseil	~25 MB

A naive static build would likely exceed cuDF's 708 MB. Maven Central documents a 1 GB limit per uploaded bundle ("You can upload an archive up to 1GB in size"), applying to the whole archive rather than a single file.

The scoping work needed is already in flight, and this issue depends on it:

• build(cmake): split libcuopt into cuopt_base / cuopt_routing / cuopt_lp component libs #1622 — split libcuopt into cuopt_base / cuopt_routing / cuopt_lp. The Java bindings expose no routing, so a Java jar should carry only the LP components.
• [FEA] Static-link gRPC, protobuf, abseil, tbb, libgomp into libcuopt.so #1203 — static-link gRPC, protobuf, abseil, tbb, libgomp.
• [FEA] Reduce libcuopt.so binary size #1607 — reduce libcuopt.so binary size.

NCCL (distributed multi-GPU PDLP) and the gRPC server stack are not needed by a Java jar at all, which is ~300 MB of the total.

Publishing infrastructure is being rebuilt upstream and we should adopt it rather than write our own:

• rapidsai/build-infra#379 — cuDF Java release pipeline migration to ADI-owned GitHub Actions. Its stated goal is that "other ADI Java projects (cuVS, cuvs-lucene, etc.) can adopt the same pattern."
• Heuristic Improvements: balance between generation and improvement heuristics #382 sign/stage in Artifactory, Persistent basis_update_mpf_t in the Dual Simplex Phase 2 #383 promote RC to Maven Central, Doc update for container version update and add nvidia-cuda-runtime as dependency #384 promote nightly to Sonatype snapshots — all In Progress.
• Add sanitizer build option #385 provision credentials — Not started.

Worth noting when planning: cuDF's java-publish job is currently skipped in every recent build.yaml run (gated on release builds) and #385 is not started, so that pipeline has not yet run for a real release. The 26.08.1 artifacts on Central came from the legacy Spark RAPIDS Jenkins process. Its assemble_maven_repo.sh gathers all classifiers into a single Maven repo tree published as one bundle, which for 26.08.1 totals 2.69 GB against the documented 1 GB ceiling — unresolved, and raised upstream.

Proposed scope

1. A static libcuopt build (BUILD_SHARED_LIBS=OFF), LP-only where build(cmake): split libcuopt into cuopt_base / cuopt_routing / cuopt_lp component libs #1622 allows, excluding routing, NCCL and the gRPC server.
2. Per-classifier JAR packaging: cuda12 / cuda13 x x86_64 / arm64.
3. A Maven-repo gather step producing the layout the shared publish workflow consumes.
4. Measure the resulting jar. If it approaches 1 GB, that is the signal to narrow scope further rather than to proceed.

Contributes to #1535.

[ramakrishnap-nv]

First working result in #1818: a self-contained classifier JAR that solves from the JAR alone — no libcuopt, no conda environment, no -Dcuopt.native.dir.

405 MB for cuda13 at CUDA arch 75, routing and gRPC excluded:

library	size
libnccl.so.2	278.7 MB
libcuopt_jni.so (cuOpt statically linked in)	172.6 MB
libcudss.so.0	66.7 MB
rmm + tbb + rapids_logger	2.2 MB

One classifier fits the 1 GB bundle limit comfortably; four together would be ~1.6 GB.

Three things worth recording so they are not re-derived:

-DBUILD_SHARED_LIBS=OFF does not work for cuOpt. cuopt is declared add_library(cuopt SHARED ...), so cuDF's approach does not transfer. There was already a cuopt_static target, but only inside if (BUILD_TESTS).

Static linking removes libcuopt.so, not what libcuopt needs. rmm, TBB, NCCL and cuDSS each surfaced as a separate UnsatisfiedLinkError because conda ships them only as shared objects. They are packaged beside the JNI library now.

NCCL cannot usefully be statically linked, and it is 69% of the JAR. conda's nccl package ships no .a at all; libnccl_static.a exists only in apt's libnccl-dev, and at 190 MB against 181 MB shared it is larger, since the bulk is per-architecture device code. NCCL is also not part of the CUDA toolkit — absent from nvidia/cuda:*-base, and in the devel image it comes from the libnccl2 apt package rather than /usr/local/cuda/lib64 — so it cannot be treated as consumer-supplied the way cuBLAS is.

That last point is bigger than packaging. dependencies.yaml makes NCCL a hard dependency for both conda (nccl >=2.19) and wheels (nvidia-nccl-cu1{2,3}>=2.19), and libcuopt.so carries a DT_NEEDED on it, so libcuopt will not load on a plain CUDA runtime image unless NCCL is installed separately — for a feature most users never touch. Every single-GPU pip user carries ~280 MB for distributed PDLP.

The surface is small and already optional at runtime: 14 symbols across 5 files under cpp/src/pdlp/distributed_pdlp/, reached through a forward-declared multi_gpu_engine_t* that defaults to nullptr. Two ways to act on it, both open:

• a switch to compile out distributed PDLP, matching SKIP_ROUTING_BUILD / SKIP_GRPC_BUILD, which would take the JAR from 405 MB to roughly 126 MB;
• loading NCCL lazily via dlopen, which would do the same for conda and wheel users and remove the runtime-image portability problem.

The second needs agreement from whoever owns distributed PDLP, since it touches solver code. Relevant to #1607 and #1622.