Skip to content

rw2: allow optional gzip and zstd compression (rc.7) #3173

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kgeckhart wants to merge 2 commits into
base: main
Choose a base branch
from
Open

rw2: allow optional gzip and zstd compression (rc.7) #3173

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ccf692b
rw2: allow optional gzip and zstd compression (rc.7)
kgeckhart Oct 30, 2026
efc5723
rw2: address review feedback on gzip and zstd compression
kgeckhart Nov 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 15 additions & 4 deletions docs/specs/prw/remote_write_spec_2_0.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ nav_title: "2.0"
sort_rank: 2
---

* Version: 2.0-rc.6
* Version: 2.0-rc.7
* Status: **Experimental**
* Date: October 2026

Expand Down Expand Up @@ -65,7 +65,7 @@ The protobuf serialization MUST use either of the following Protobuf Messages:
* The `prometheus.WriteRequest` introduced in [the Remote-Write 1.0 specification](./remote_write_spec.md#protocol). As of 2.0, this message is deprecated. It SHOULD be used only for compatibility reasons. Senders and Receivers MAY NOT support the `prometheus.WriteRequest`.
* The `io.prometheus.write.v2.Request` introduced in this specification and defined [below](#protobuf-message). Senders and Receivers SHOULD use this message when possible. Senders and Receivers MUST support the `io.prometheus.write.v2.Request`.

Protobuf Message MUST use binary Wire Format. Then, MUST be compressed with [Google’s Snappy](https://github.com/google/snappy). Snappy's [block format](https://github.com/google/snappy/blob/2c94e11145f0b7b184b831577c93e5a41c4c0346/format_description.txt) MUST be used -- [the framed format](https://github.com/google/snappy/blob/2c94e11145f0b7b184b831577c93e5a41c4c0346/framing_format.txt) MUST NOT be used.
Protobuf Message MUST use binary Wire Format. Then, MUST be compressed with one of the compressions listed in [Content-Encoding](#content-encoding).

Senders MUST send a serialized and compressed Protobuf Message in the body of an HTTP POST request and send it to the Receiver via HTTP at the provided URL path. Receivers MAY specify any HTTP URL path to receive metrics.

Expand All @@ -89,8 +89,19 @@ Content-Encoding: <compression>

<!---
Rationales: https://github.com/prometheus/proposals/blob/alexg/remote-write-20-proposal/proposals/2024-04-09_remote-write-20.md#no-new-compression-added--yet-
Relaxed in 2.0-rc.7: `snappy` stays mandatory, `gzip` and `zstd` are optional for Receivers.
-->
Content encoding request header MUST follow [the RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-content-encoding). Senders MUST use the `snappy` value. Receivers MUST support `snappy` compression. New, optional compression algorithms might come in 2.x or beyond.
Content encoding request header MUST follow [the RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html#name-content-encoding). Senders are allowed to use following values:

* `snappy`: [Google’s Snappy](https://github.com/google/snappy). The [block format](https://github.com/google/snappy/blob/2c94e11145f0b7b184b831577c93e5a41c4c0346/format_description.txt) MUST be used -- [the framed format](https://github.com/google/snappy/blob/2c94e11145f0b7b184b831577c93e5a41c4c0346/framing_format.txt) MUST NOT be used.
* `gzip`: MUST follow [RFC 1952](https://www.rfc-editor.org/rfc/rfc1952.html).
* `zstd`: MUST follow [RFC 8878](https://www.rfc-editor.org/rfc/rfc8878.html) and [RFC 9659](https://www.rfc-editor.org/rfc/rfc9659.html).

@bwplotka bwplotka Nov 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neither RFC 8878 nor RFC 9659 forbids Zstandard dictionaries on zstd content coding, and RW 2.0 has no dictionary negotiation yet. Worth ruling them out explicitly like Snappy's framed format above:

Suggested change
* `zstd`: MUST follow [RFC 8878](https://www.rfc-editor.org/rfc/rfc8878.html) and [RFC 9659](https://www.rfc-editor.org/rfc/rfc9659.html).
* `zstd`: MUST follow [RFC 8878](https://www.rfc-editor.org/rfc/rfc8878.html) and [RFC 9659](https://www.rfc-editor.org/rfc/rfc9659.html). Dictionaries MUST NOT be used.


`gzip` and `zstd` MUST only be used with `io.prometheus.write.v2.Request`. `prometheus.WriteRequest` is unchanged from [1.0](./remote_write_spec.md), `snappy` only.

@bwplotka bwplotka Nov 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since #unsupported-request-content is a MUST, passive MUST only be used here implies dual-version Receivers have to reject prometheus.WriteRequest compressed with gzip or zstd, which breaks HTTP decompression middlewares in client_golang or reverse proxies that decode Content-Encoding before checking Content-Type. Keeping MUST NOT on Senders and MAY on Receivers lets both middleware-first and Content-Type-first Receivers pass compliance:

Suggested change
`gzip` and `zstd` MUST only be used with `io.prometheus.write.v2.Request`. `prometheus.WriteRequest` is unchanged from [1.0](./remote_write_spec.md), `snappy` only.
Senders MUST NOT use `gzip` or `zstd` with `prometheus.WriteRequest`, which remains `snappy` only as in [1.0](./remote_write_spec.md). Receivers MAY reject `prometheus.WriteRequest` compressed with `gzip` or `zstd` with [415 HTTP Unsupported Media Type](https://www.rfc-editor.org/rfc/rfc9110.html#name-415-unsupported-media-type).

@bwplotka bwplotka Nov 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

" snappy only." is weird. Did you forget to finish some sentence?

@bwplotka bwplotka Nov 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also be specific (sender/receiver). Mention e.g. "Sender MUST NOT".


Senders and Receivers MUST support `snappy`. Receivers SHOULD support `gzip` and `zstd`. Senders MAY use `gzip` or `zstd`, and SHOULD use `snappy` unless configured otherwise or the Receiver advertised the encoding with `Accept-Encoding`.

Receivers advertise supported `Content-Encoding` values with the [`Accept-Encoding`](https://www.rfc-editor.org/rfc/rfc9110.html#name-accept-encoding) response header, which applies to the request it answers. Receivers MAY send it on 2xx responses. On a [415 HTTP Unsupported Media Type](https://www.rfc-editor.org/rfc/rfc9110.html#name-415-unsupported-media-type) they SHOULD send it when an unsupported `Content-Encoding` is the cause, and MUST NOT send it otherwise. A Sender that chose its encoding from a 2xx `Accept-Encoding` MUST retry once with `snappy` when it gets a 415, unless the 415 carries `Accept` and no `Accept-Encoding`.

@bwplotka bwplotka Nov 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MUST retry once with snappy only retries that single batch and leaves gzip or zstd active for subsequent requests after a Receiver rollback. Also, phrasing the re-upgrade guard as a cooldown makes it easy to test in prometheus/compliance with back-to-back scrapes right after a 415 fallback:

Suggested change
Receivers advertise supported `Content-Encoding` values with the [`Accept-Encoding`](https://www.rfc-editor.org/rfc/rfc9110.html#name-accept-encoding) response header, which applies to the request it answers. Receivers MAY send it on 2xx responses. On a [415 HTTP Unsupported Media Type](https://www.rfc-editor.org/rfc/rfc9110.html#name-415-unsupported-media-type) they SHOULD send it when an unsupported `Content-Encoding` is the cause, and MUST NOT send it otherwise. A Sender that chose its encoding from a 2xx `Accept-Encoding` MUST retry once with `snappy` when it gets a 415, unless the 415 carries `Accept` and no `Accept-Encoding`.
Receivers advertise supported `Content-Encoding` values with the [`Accept-Encoding`](https://www.rfc-editor.org/rfc/rfc9110.html#name-accept-encoding) response header, which applies to the request it answers. Receivers MAY send it on 2xx responses. On a [415 HTTP Unsupported Media Type](https://www.rfc-editor.org/rfc/rfc9110.html#name-415-unsupported-media-type) they SHOULD send it when an unsupported `Content-Encoding` is the cause, and MUST NOT send it otherwise. A Sender that switched to `gzip` or `zstd` from a 2xx `Accept-Encoding` MUST revert to `snappy` for subsequent requests and retry the failed request once with `snappy` when it gets a 415, unless the 415 carries `Accept` and no `Accept-Encoding`. After reverting on a 415, the Sender SHOULD wait for a cooldown period before switching away from `snappy` via `Accept-Encoding` again.


#### Content-Type

Expand Down Expand Up @@ -189,7 +200,7 @@ Senders SHOULD use Remote-Write to send samples for multiple series in a single

#### Unsupported Request Content

Receivers MUST return [415 HTTP Unsupported Media Type](https://www.rfc-editor.org/rfc/rfc9110.html#name-415-unsupported-media-type) status code if they don't support a given content type or encoding provided by Senders. When returning `415 HTTP Unsupported Media Type`, Receivers SHOULD include the `Accept` (listing supported `Content-Type` values, including supported `;proto=` and `;mode=` parameters) and `Accept-Encoding` (listing supported `Content-Encoding` values) response headers as defined in [RFC 7694](https://www.rfc-editor.org/rfc/rfc7694.html) so Senders can determine what the Receiver supports without guessing.
Receivers MUST return [415 HTTP Unsupported Media Type](https://www.rfc-editor.org/rfc/rfc9110.html#name-415-unsupported-media-type) status code if they don't support a given content type or encoding provided by Senders. If the 415 is due to an unsupported `Content-Type`, Receivers SHOULD include the [`Accept`](https://www.rfc-editor.org/rfc/rfc9110.html#name-accept) response header listing supported `Content-Type` values, including supported `;proto=` and `;mode=` parameters, so Senders can determine what the Receiver supports without guessing. For an unsupported `Content-Encoding`, see [Content-Encoding](#content-encoding).

Senders SHOULD expect [400 HTTP Bad Request](https://www.rfc-editor.org/rfc/rfc9110.html#name-400-bad-request) for the above reasons from 1.x Receivers, for backwards compatibility.

Expand Down
Loading