[client-v1] Every compressed read fails on ClickHouse 26.9+: Lz4InputStream hardcodes the LZ4 magic byte but the server default codec is now ZSTD

[polyglotAI-bot]

Description

On ClickHouse Server 26.9+, every compressed read through the legacy v1 client stack
(ClickHouseClient / clickhouse-http-client) fails with Magic is not correct - expect [-126] but got [-112].

The v1 HTTP client requests server-side compression with the native compress=1 framing and then
decompresses it with com.clickhouse.data.stream.Lz4InputStream, which rejects any block whose
method byte is not 0x82 (LZ4). ClickHouse 26.9 changed the default codec of that framing to
ZSTD, whose method byte is 0x90, so the stream now aborts on the very first block.

This affects the v1 client out of the box: ClickHouseClientOption.COMPRESS defaults to
true and COMPRESS_ALGORITHM defaults to LZ4, so no explicit configuration is needed to hit it.
Disabling compression is the only workaround.

Relationship to #3105. #3105 reports the same class of failure in client-v2
(client-v2 ClickHouseLZ4InputStream, message Invalid LZ4 magic byte). This issue is a
separate code path in a different module — clickhouse-data Lz4InputStream, message
Magic is not correct — which PR #3106 does not touch. The v1 stack stays broken after #3106 merges.

Steps to reproduce

1. Run a ClickHouse server 26.9+ (verified on 26.9.1.954).
2. Read any result set with the v1 client using default options (compression on, algorithm LZ4).
3. The read fails on the first block; the same code against a 25.8 server succeeds.

Error Log or Exception StackTrace

FAILED UncheckedIOException: java.io.IOException: Magic is not correct - expect [-126] but got [-112]
  root: java.io.IOException: Magic is not correct - expect [-126] but got [-112]
	at com.clickhouse.data.stream.Lz4InputStream.updateBuffer(Lz4InputStream.java:64)

Expected Behaviour

The v1 client should decompress a compress=1 response according to the method byte the server
actually sent, rather than assuming LZ4 — i.e. dispatch on the block header and decompress
LZ4 (0x82), ZSTD (0x90), or none (0x02).

The server is the source of truth here. Same query, compress=1, two server versions — byte 16 is
the method byte:

# ClickHouse 25.8.28.1  -> 0x82 = LZ4
35 ef b4 f9 60 b0 aa a0 61 3a 75 eb 83 88 4a 0c
82 2e 01 00 00 22 01 00

# ClickHouse 26.9.1.954 -> 0x90 = ZSTD  (payload begins 28 b5 2f fd = ZSTD frame magic)
26 92 15 1b fd 74 9e e7 06 e7 32 c0 b4 28 31 49
90 9c 00 00 00 22 01 00 00 28 b5 2f fd 60 22 00

Code Example

ClickHouseNode node = ClickHouseNode.builder()
        .host("clickhouse").port(ClickHouseProtocol.HTTP, 8123).database("default")
        .credentials(ClickHouseCredentials.fromUserAndPassword("default", "***"))
        .build();

try (ClickHouseClient client = ClickHouseClient.newInstance(ClickHouseProtocol.HTTP);
        ClickHouseResponse response = client.read(node)
                .query("SELECT number FROM numbers(1000)")
                .option(ClickHouseClientOption.COMPRESS, true)   // default is already true
                .executeAndWait()) {
    for (ClickHouseRecord r : response.records()) {
        r.getValue(0).asLong();
    }
}

Observed with the identical program against two servers:

Server	compression disabled	compression enabled (default)	explicit LZ4
25.8.28.1	OK	OK	OK
26.9.1.954	OK	FAILS	FAILS

Root cause

clickhouse-data/src/main/java/com/clickhouse/data/stream/Lz4InputStream.java:64-67:

} else if (header[16] != MAGIC) {   // MAGIC = (byte) 0x82
    throw new IOException(
            ClickHouseUtils.format("Magic is not correct - expect [%d] but got [%d]", MAGIC, header[16]));
}

The method byte is validated as LZ4 instead of being used to select a decompressor. The stream is
reached from com.clickhouse.data.compress.Lz4Support.DefaultImpl.decompress(...), which is
selected whenever the response compression algorithm is LZ4 — the default. The corresponding
request in ClickHouseHttpConnection is what asks for the native framing:

if (config.isResponseCompressed()) {
    if (config.getResponseCompressAlgorithm() == ClickHouseCompression.LZ4) {
        appendQueryParameter(builder, "compress", "1");
    }

Suggested fix

Dispatch on header[16] instead of asserting it, mirroring the approach taken for client-v2 in
PR #3106: accept 0x82 (LZ4), 0x90 (ZSTD), and 0x02 (uncompressed), and raise a clear error
only for a genuinely unknown method byte — which should stay a negative test case.

Two things worth a maintainer decision, which is why this is filed rather than patched:

• Packaging. zstd-jni is an optional dependency of clickhouse-data, so adding a ZSTD
  decode path there needs a decision about whether it becomes required, or whether the ZSTD branch
  fails with an actionable "add zstd-jni" message when the class is absent.
• Scope. The v1 stack and Lz4InputStream are marked @Deprecated. If v1 is not intended to
  support 26.9+ servers, an explicit error telling the user to disable compression or migrate would
  be preferable to the current low-level magic-byte failure.

Configuration

Client Configuration

// defaults only
ClickHouseClientOption.COMPRESS            // true
ClickHouseClientOption.COMPRESS_ALGORITHM  // LZ4

Environment

• Cloud
• Client version: main @ be331d4ed (0.10.0-rc1-SNAPSHOT)
• Language version: OpenJDK 17.0.20
• OS: Linux (x86_64, Docker)

ClickHouse Server

• ClickHouse Server version: 26.9.1.954 (fails) / 25.8.28.1 (works)
• ClickHouse Server non-default settings, if any: none relevant to the framing
• CREATE TABLE statements for tables involved: none — reproduces with SELECT number FROM numbers(1000)
• Sample data for all these tables: n/a

---

Found by automated analysis while working on #3105, and verified against live 26.9 and 25.8 servers
rather than by inspection. The v1 JDBC driver (clickhouse-jdbc) was checked separately and fails
with the client-v2 message instead, so that surface is covered by #3105, not by this issue.