[client-v2] Nullable column bound to a primitive POJO setter silently corrupts the row (null marker never consumed)

[alex-clickhouse]

Description

On the compiled-POJO read fast path, a Nullable(T) column bound to a primitive setter decodes without consuming the one-byte null marker. That desyncs the RowBinary stream by one byte per row, so the nullable column and every column after it decode from the wrong offset. Nothing throws — the reader consumes the right total number of bytes, just split in the wrong places — so the caller silently gets plausible-looking wrong numbers.

SerializerUtils.compilePOJOSetter chooses the box-free path on:

https://github.com/ClickHouse/clickhouse-java/blob/9776111f/client-v2/src/main/java/com/clickhouse/client/api/data_formats/internal/SerializerUtils.java#L857

if (targetType.isPrimitive() && BinaryStreamReader.isReadToPrimitive(column.getDataType())) {

Neither half of that condition looks at column.isNullable(). isReadToPrimitive answers for the data type — Int64 is readable to a primitive whether or not the column is nullable — so binaryReaderMethodForType emits a bare BinaryStreamReader.readLongLE() with no marker read.

The failure mode is inverted from what you would expect. With NULLs actually present it throws (a long field cannot hold a NULL, so that binding gets rejected somewhere downstream). It is the case that looks entirely safe — a Nullable column that happens to contain no nulls — that corrupts silently. The marker byte is on the wire because the column is declared Nullable, not because any value is null.

This also means rejecting the binding only when a NULL is encountered would not be a sufficient fix.

Steps to reproduce

1. Query a Nullable(Int64) column containing no nulls, followed by any other column.
2. Register a POJO whose setter for the nullable column takes primitive long.
3. queryAll(sql, Pojo.class, schema) — both columns come back wrong, no exception.

Error Log or Exception StackTrace

No exception. That is the bug.

Expected Behaviour

a = 0,1,2,3,4 and b = 1000..1004. Actual:

column 'a' decoded as [0, 65536, 33554432, 12884901888, 4398046511104]
                  but should be [0, 1, 2, 3, 4]
column 'b' likewise corrupt

Those values are 0, 1<<16, 2<<24, 3<<32, 4<<40 — each row shifted eight more bits than the last, which is the cumulative one-byte-per-row desync made visible.

Code Example

public static class PrimitiveSetterPOJO {
    private long a;
    private long b;
    public long getA() { return a; }
    public void setA(long a) { this.a = a; }   // primitive setter on the Nullable column
    public long getB() { return b; }
    public void setB(long b) { this.b = b; }   // witness for the cascade
}

// Nullable(Int64) containing NO nulls, followed by a plain Int64
String sql = "SELECT toNullable(toInt64(number)) AS a, toInt64(number + 1000) AS b FROM numbers(5)";

TableSchema schema = client.getTableSchemaFromQuery(sql);
client.register(PrimitiveSetterPOJO.class, schema);

for (PrimitiveSetterPOJO row : client.queryAll(sql, PrimitiveSetterPOJO.class, schema)) {
    System.out.println(row.getA() + " " + row.getB());   // 0 256000, 65536 65601536, ...
}

Changing setA(long) to setA(Long) declines the primitive path and the row decodes correctly — which localises the defect to the primitive path rather than the query, the schema or the wire format.

Reproducing test

I have a TestNG integration test in the project's own style — the failing case, a boxed-setter control that passes, and the with-NULLs case for contrast. Happy to open it as a PR if useful; here it is inline:

client-v2/src/test/java/com/clickhouse/client/query/NullablePrimitivePOJOTests.java

package com.clickhouse.client.query;

import com.clickhouse.client.BaseIntegrationTest;
import com.clickhouse.client.ClickHouseNode;
import com.clickhouse.client.ClickHouseProtocol;
import com.clickhouse.client.ClickHouseServerForTest;
import com.clickhouse.client.api.Client;
import com.clickhouse.client.api.ClientException;
import com.clickhouse.client.api.enums.Protocol;
import com.clickhouse.client.api.metadata.TableSchema;
import org.testng.Assert;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;

import java.util.List;
import java.util.stream.Collectors;
import java.util.stream.LongStream;
import java.util.stream.Stream;

public class NullablePrimitivePOJOTests extends BaseIntegrationTest {

    private static final int ROWS = 5;

    public static class PrimitiveSetterPOJO {
        private long a;
        private long b;
        public long getA() { return a; }
        public void setA(long a) { this.a = a; }
        public long getB() { return b; }
        public void setB(long b) { this.b = b; }
    }

    public static class BoxedSetterPOJO {
        private Long a;
        private long b;
        public Long getA() { return a; }
        public void setA(Long a) { this.a = a; }
        public long getB() { return b; }
        public void setB(long b) { this.b = b; }
    }

    private Client client;

    @BeforeMethod(groups = {"integration"})
    public void setUp() {
        ClickHouseNode node = getServer(ClickHouseProtocol.HTTP);
        client = new Client.Builder()
                .addEndpoint(Protocol.HTTP, node.getHost(), node.getPort(), isCloud())
                .setUsername("default")
                .setPassword(ClickHouseServerForTest.getPassword())
                .compressClientRequest(false)
                .compressServerResponse(false)
                .setDefaultDatabase(ClickHouseServerForTest.getDatabase())
                .build();
    }

    @AfterMethod(groups = {"integration"})
    public void tearDown() {
        if (client != null) {
            client.close();
        }
    }

    // a = 0..4 as Nullable(Int64) with no nulls, b = 1000..1004 as plain Int64.
    private static final String SQL_NO_NULLS =
            "SELECT toNullable(toInt64(number)) AS a, toInt64(number + 1000) AS b FROM numbers(" + ROWS + ")";

    // Same, but every other row's `a` is actually NULL.
    private static final String SQL_WITH_NULLS =
            "SELECT if(number % 2 = 0, NULL, toNullable(toInt64(number))) AS a, " +
                    "toInt64(number + 1000) AS b FROM numbers(" + ROWS + ")";

    private <T> List<T> queryAll(String sql, Class<T> pojoClass) {
        TableSchema schema = client.getTableSchemaFromQuery(sql);
        client.register(pojoClass, schema);
        return client.queryAll(sql, pojoClass, schema);
    }

    /** The bug. Asserts the following column too: the cascade is the real signature. */
    @Test(groups = {"integration"})
    public void nullablePrimitiveSetter_WithNoNullsPresent_DoesNotCorruptTheRow() {
        List<PrimitiveSetterPOJO> rows = queryAll(SQL_NO_NULLS, PrimitiveSetterPOJO.class);

        Assert.assertEquals(rows.size(), ROWS);
        assertColumn("a", rows.stream().map(PrimitiveSetterPOJO::getA), 0L);
        assertColumn("b", rows.stream().map(PrimitiveSetterPOJO::getB), 1000L);
    }

    /** Control: boxed setter declines the primitive path, so the marker is read. Passes today. */
    @Test(groups = {"integration"})
    public void boxedSetter_WithNoNullsPresent_ReadsCorrectly() {
        List<BoxedSetterPOJO> rows = queryAll(SQL_NO_NULLS, BoxedSetterPOJO.class);

        Assert.assertEquals(rows.size(), ROWS);
        assertColumn("a", rows.stream().map(BoxedSetterPOJO::getA), 0L);
        assertColumn("b", rows.stream().map(BoxedSetterPOJO::getB), 1000L);
    }

    /** Contrast: with NULLs actually present it throws. Passes today. */
    @Test(groups = {"integration"})
    public void nullablePrimitiveSetter_WithActualNulls_FailsLoudlyUnlikeTheNoNullsCase() {
        Assert.assertThrows(ClientException.class, () -> queryAll(SQL_WITH_NULLS, PrimitiveSetterPOJO.class));
    }

    private static void assertColumn(String name, Stream<? extends Number> actualValues, long first) {
        List<Long> actual = actualValues.map(Number::longValue).collect(Collectors.toList());
        List<Long> expected = LongStream.range(first, first + ROWS).boxed().collect(Collectors.toList());

        Assert.assertEquals(actual, expected,
                "column '" + name + "' decoded as " + actual + " but should be " + expected +
                        "; a wrong value in a column *after* the Nullable one means the null marker was " +
                        "never consumed and the stream is misaligned by one byte per row");
    }
}

Run with (failsafe, not surefire — surefire excludes the integration group):

mvn -pl client-v2 verify -Dit.test=NullablePrimitivePOJOTests -DskipUTs=true

Tests run: 3, Failures: 1

nullablePrimitiveSetter_WithNoNullsPresent_DoesNotCorruptTheRow   FAILS
boxedSetter_WithNoNullsPresent_ReadsCorrectly                     passes
nullablePrimitiveSetter_WithActualNulls_FailsLoudly...            passes

Notes

• Reachable from Client.queryAll(sql, Class, schema), i.e. the documented POJO read API.
• Not specific to Int64 — it should affect any nullable column whose data type passes isReadToPrimitive (BinaryStreamReader.java:1153-1171) bound to a primitive setter.
• Related: Column type 'Bool' cannot be set to a primitive type 'boolean' #2248 touched the same compilePOJOSetter → binaryReaderMethodForType path, for the case where the primitive path threw. The nullable check appears never to have been added.
• Minor, found alongside: POJOSerDe.registerClass rejects the TableSchema a reader builds for itself, because RowBinaryWithNamesAndTypesFormatReader:55 uses new TableSchema(columns), which sets both tableName="" and query="", while POJOSerDe.java:85-91 requires exactly one to be null. Happy to file separately if that is not already known.

Configuration

Environment

• Cloud
• Client version: reproduced on main @ 9776111f; also present in released 0.9.8
• Language version: OpenJDK 17.0.19
• OS: Linux (WSL2)

ClickHouse Server

• ClickHouse Server version: 25.x via the project's own testcontainer (BaseIntegrationTest); also reproduced against 26.6.1
• ClickHouse Server non-default settings, if any: none
• CREATE TABLE statements for tables involved: none — reproduces against numbers(5)