RDSC-6347: RDI 2.4.0 upgrade notes for Debezium 3.7

content/embeds/rdi-snapshot-modes.md

@@ -51,21 +51,3 @@ monitoring. Check snapshot completion separately from collector connectivity;
 
 An interrupted initial snapshot can restart when the collector starts again, including
 with `initial` and `initial_only`.
-
-For Debezium 3.5, note these connector differences:
-
-- PostgreSQL `no_data` performs no snapshot. It resumes from a saved log sequence
-  number (LSN), or from the replication slot's creation position when no LSN is saved.
-  This requires the relevant write-ahead log (WAL) history to remain available.
-- Other connectors can capture schema or initialize a position without copying
-  existing rows in `no_data` mode.
-- `never` is a deprecated alias for `no_data` in PostgreSQL and MongoDB. MySQL and
-  MariaDB retain `never` with connector-specific prerequisites. Oracle and SQL Server
-  do not offer `never`; use `no_data` to skip existing rows.
-
-See the Debezium 3.5 connector references for [MariaDB](https://debezium.io/documentation/reference/3.5/connectors/mariadb.html),
-[MongoDB](https://debezium.io/documentation/reference/3.5/connectors/mongodb.html),
-[MySQL](https://debezium.io/documentation/reference/3.5/connectors/mysql.html),
-[Oracle](https://debezium.io/documentation/reference/3.5/connectors/oracle.html),
-[PostgreSQL](https://debezium.io/documentation/reference/3.5/connectors/postgresql.html),
-and [SQL Server](https://debezium.io/documentation/reference/3.5/connectors/sqlserver.html).

content/integrate/redis-data-integration/1.19.1/data-pipelines/pipeline-config.md

@@ -199,6 +199,24 @@ section apply to that release.
 
 {{< embed-md "rdi-snapshot-modes.md" >}}
 
+For Debezium 3.5, note these connector differences:
+
+- PostgreSQL `no_data` performs no snapshot. It resumes from a saved log sequence
+  number (LSN), or from the replication slot's creation position when no LSN is saved.
+  This requires the relevant write-ahead log (WAL) history to remain available.
+- Other connectors can capture schema or initialize a position without copying
+  existing rows in `no_data` mode.
+- `never` is a deprecated alias for `no_data` in PostgreSQL and MongoDB. MySQL and
+  MariaDB retain `never` with connector-specific prerequisites. Oracle and SQL Server
+  do not offer `never`; use `no_data` to skip existing rows.
+
+See the Debezium 3.5 connector references for [MariaDB](https://debezium.io/documentation/reference/3.5/connectors/mariadb.html),
+[MongoDB](https://debezium.io/documentation/reference/3.5/connectors/mongodb.html),
+[MySQL](https://debezium.io/documentation/reference/3.5/connectors/mysql.html),
+[Oracle](https://debezium.io/documentation/reference/3.5/connectors/oracle.html),
+[PostgreSQL](https://debezium.io/documentation/reference/3.5/connectors/postgresql.html),
+and [SQL Server](https://debezium.io/documentation/reference/3.5/connectors/sqlserver.html).
+
 A reset clears saved positions without changing `snapshot.mode` or flushing the target
 Redis database. If you need to reload existing rows, select a mode that takes a data
 snapshot. A snapshot can overwrite target records, but it does not remove stale records

content/integrate/redis-data-integration/data-pipelines/pipeline-config.md

@@ -211,6 +211,16 @@ Debezium 3.7.
 
 {{< embed-md "rdi-snapshot-modes.md" >}}
 
+Use the default `initial` mode unless you have a specific reason to choose another.
+The supported modes and their behavior differ by connector. For the modes your source
+supports, see `snapshot.mode` in the Debezium 3.7 reference for
+[MariaDB](https://debezium.io/documentation/reference/3.7/connectors/mariadb.html#mariadb-property-snapshot-mode),
+[MongoDB](https://debezium.io/documentation/reference/3.7/connectors/mongodb.html#mongodb-property-snapshot-mode),
+[MySQL](https://debezium.io/documentation/reference/3.7/connectors/mysql.html#mysql-property-snapshot-mode),
+[Oracle](https://debezium.io/documentation/reference/3.7/connectors/oracle.html#oracle-property-snapshot-mode),
+[PostgreSQL](https://debezium.io/documentation/reference/3.7/connectors/postgresql.html#postgresql-property-snapshot-mode),
+and [SQL Server](https://debezium.io/documentation/reference/3.7/connectors/sqlserver.html#sqlserver-property-snapshot-mode).
+
 A reset clears saved positions without changing `snapshot.mode` or flushing the target
 Redis database. If you need to reload existing rows, select a mode that takes a data
 snapshot. A snapshot can overwrite target records, but it does not remove stale records

content/integrate/redis-data-integration/data-pipelines/prepare-dbs/my-sql-mariadb.md

@@ -245,14 +245,28 @@ certificate. For mutual TLS, also set the source's `CERT` and `KEY` secrets. See
 [Set secrets](/content/integrate/redis-data-integration/data-pipelines/deploy.md#set-secrets)
 for the full list of source database TLS and mTLS secrets.
 
-Select the TLS mode in the source `advanced.source` section, for example:
+Select the TLS mode in the source `advanced.source` section. MySQL and MariaDB sources
+take different values for `database.ssl.mode`. For example, to verify the server
+certificate against the CA certificate, use the following for a MySQL source:
 
 ```yaml
 advanced:
   source:
     database.ssl.mode: verify_ca
 ```
 
+MySQL sources take `disabled`, `preferred`, `required`, `verify_ca`, and `verify_identity`.
+
+For a MariaDB source, use the following:
+
+```yaml
+advanced:
+  source:
+    database.ssl.mode: verify-ca
+```
+
+MariaDB sources take `disable`, `trust`, `verify-ca`, and `verify-full`.
+
 RDI builds the keystore that presents the client certificate to the source database from the
 source's `CERT` and `KEY` secrets, and configures the collector to use it.
 

content/integrate/redis-data-integration/data-pipelines/prepare-dbs/oracle.md

@@ -178,7 +178,7 @@ If you have more than one destination with these settings,
 then you should consult your database administrator about which one to
 choose for Debezium.
 
-Use the `log.mining.archive.destination.name` property in the connector configuration
+Use the `archive.destination.name` property in the connector configuration
 to select the archive log destination for Debezium.
 
 For example, suppose you have two archive destinations, `LOG_ARCHIVE_DEST_2` and
@@ -189,7 +189,7 @@ use the following setting:
 
 ```json
 {
-    "log.mining.archive.destination.name": "LOG_ARCHIVE_DEST_3"
+    "archive.destination.name": "LOG_ARCHIVE_DEST_3"
 }
 ```
 
@@ -953,6 +953,18 @@ PROMPT =====================================================
 
 To support XStream connector, you must create a custom [Docker](https://www.docker.com/) image that includes the required Instant Client package libraries for Linux x64 from the Oracle website.
 
+Build the custom image from the Debezium Server image that your RDI version uses, so that
+the collector keeps the Debezium version and the RDI modifications that your configuration
+is validated against. To find the image, check the collector section of the `rdi-operator`
+configmap:
+
+```bash
+kubectl get configmap rdi-operator -n rdi -o yaml | grep -A 5 'collector:'
+```
+
+The examples below use `redislabs/debezium-server:3.7.0.Final-rdi.2`, the image that
+RDI 2.4.0 uses. Replace it with the repository and tag from your configmap.
+
 1.  On the Docker machine, download the Instant Client package for Linux x64 from the Oracle website
 
     ```bash
@@ -968,7 +980,7 @@ To support XStream connector, you must create a custom [Docker](https://www.dock
 1.  Create a `Dockerfile` in the `./dbz-ora` directory with the following contents:
 
     ```docker
-    FROM debezium/server:3.0.8.Final
+    FROM redislabs/debezium-server:3.7.0.Final-rdi.2
 
     USER root
 
@@ -996,9 +1008,9 @@ To support XStream connector, you must create a custom [Docker](https://www.dock
 1.  Add the image to the K3s image registry using the following commands:
 
     ```bash
-    docker tag dbz-ora quay.io/debezium/server:3.0.8.Final
-    docker image save quay.io/debezium/server:3.0.8.Final -o dbz3.0.8-xstream-linux-amd.tar
-    sudo k3s ctr images import dbz3.0.8-xstream-linux-amd.tar all
+    docker tag dbz-ora redislabs/debezium-server:3.7.0.Final-rdi.2-xstream
+    docker image save redislabs/debezium-server:3.7.0.Final-rdi.2-xstream -o dbz-xstream-linux-amd.tar
+    sudo k3s ctr images import dbz-xstream-linux-amd.tar all
     ```
 
 ### 8. Make RDI use the custom image (VM installation)
@@ -1009,22 +1021,24 @@ Edit the `rdi-operator` configmap:
 kubectl edit configmap rdi-operator -n rdi
 ```
 
-In the editor, find the collector section and change the image settings:
+In the editor, find the collector section and change the image tag to the tag of your
+custom image:
 
 ```yaml
       collector:
         image:
           pullPolicy: IfNotPresent
-          registry: docker.io # change this to `quay.io`
-          repository: redislabs/debezium-server # Change this to `debezium/server`
-          tag: 3.0.8.Final-rdi.1 # Change this to `3.0.8.Final`
+          registry: docker.io
+          repository: redislabs/debezium-server
+          tag: 3.7.0.Final-rdi.2 # Change this to `3.7.0.Final-rdi.2-xstream`
 ```
 
 Save the configmap. Once it is saved, the operator will restart automatically and will apply the changes.
 
 > [!NOTE]
 > After upgrading to another RDI version,
-> the changes to the configmap will be lost. You must repeat the above steps after each upgrade.
+> the changes to the configmap will be lost. You must repeat the above steps after each upgrade,
+> and rebuild the custom image from the Debezium Server image of the new RDI version.
 
 ### 9. Enable the Oracle configuration in RDI
 
@@ -1072,7 +1086,10 @@ you must configure additional libraries for Debezium Server to process these col
 ### Create a custom Debezium Server image
 
 To support `XMLTYPE` columns, you must create a custom [Docker](https://www.docker.com/) image
-that includes the required Oracle XML libraries.
+that includes the required Oracle XML libraries. As with the
+[XStream image](#7-add-a-custom-docker-image-for-the-debezium-server), build it from the
+Debezium Server image that your RDI version uses. The examples below use
+`redislabs/debezium-server:3.7.0.Final-rdi.2`, the image that RDI 2.4.0 uses.
 
 1. Download the required libraries from Maven Central:
 
@@ -1088,7 +1105,7 @@ that includes the required Oracle XML libraries.
 2. Create a `Dockerfile` in the same directory:
 
    ```dockerfile
-   FROM quay.io/debezium/server:3.0.8.Final
+   FROM redislabs/debezium-server:3.7.0.Final-rdi.2
 
    USER root
 
@@ -1101,16 +1118,20 @@ that includes the required Oracle XML libraries.
    ```bash
    cd ..
    docker build -t dbz-xml xml
-   docker tag dbz-xml quay.io/debezium/server:3.0.8.Final
-   docker image save quay.io/debezium/server:3.0.8.Final -o dbz3.0.8-xml-linux-amd.tar
+   docker tag dbz-xml redislabs/debezium-server:3.7.0.Final-rdi.2-xml
+   docker image save redislabs/debezium-server:3.7.0.Final-rdi.2-xml -o dbz-xml-linux-amd.tar
    ```
 
 4. Add the image to your K3s image repository:
 
    ```bash
-   sudo k3s ctr images import dbz3.0.8-xml-linux-amd.tar all
+   sudo k3s ctr images import dbz-xml-linux-amd.tar all
    ```
 
+5. Change the collector image tag in the `rdi-operator` configmap to
+   `3.7.0.Final-rdi.2-xml`, as described in
+   [Make RDI use the custom image](#8-make-rdi-use-the-custom-image-vm-installation).
+
 ### Configure RDI for XMLTYPE support
 
 In your RDI configuration file, set the `lob.enabled` property to `true` in the

content/integrate/redis-data-integration/installation/install-k8s.md

@@ -106,8 +106,8 @@ If you plan to use Snowflake as a source for any of your pipelines, you'll also
 
 In addition, the RDI Helm chart uses the following 3rd party images:
 
--   [redislabs/debezium-server:3.5.0.Final-rdi.3](https://hub.docker.com/r/redislabs/debezium-server),
-    based on `quay.io/debezium/server/3.5.0.Final` with minor modifications:
+-   [redislabs/debezium-server:3.7.0.Final-rdi.2](https://hub.docker.com/r/redislabs/debezium-server),
+    based on `quay.io/debezium/server/3.7.0.Final` with minor modifications:
     [Debezium](https://debezium.io/), an open source distributed platform for change data capture.
 -   [redis/reloader:v1.4.13](https://hub.docker.com/r/redis/reloader), originally `ghcr.io/stakater/reloader:v1.4.13`:
     [Reloader](https://github.com/stakater/Reloader), a K8s controller to watch changes to ConfigMaps

content/integrate/redis-data-integration/installation/upgrade.md

@@ -17,7 +17,8 @@ weight: 30
 ---
 
 > [!NOTE]
-> Before you upgrade, review [Upgrading to RDI 2.2.0](#upgrading-to-rdi-220). If you upgrade
+> Before you upgrade, review [Upgrading to RDI 2.4.0](#upgrading-to-rdi-240). If you upgrade
+> from RDI 2.0.x, also review [Upgrading to RDI 2.2.0](#upgrading-to-rdi-220), and if you upgrade
 > from RDI 1.19.x or earlier, also review [Upgrading to RDI 2.0.0](#upgrading-to-rdi-200).
 
 > [!WARNING]
@@ -181,6 +182,84 @@ your saved `rdi-values.yaml` for that version. This will restore your previous w
 > described in [Uninstall RDI](/content/integrate/redis-data-integration/installation/install-k8s.md#uninstall-rdi),
 > and then install the old version.
 
+## Upgrading to RDI 2.4.0
+
+RDI 2.4.0 upgrades the Debezium collector from Debezium 3.5 to Debezium 3.7. Check the following
+changes before you upgrade from RDI 2.2.x. If you upgrade from RDI 2.0.x, also review
+[Upgrading to RDI 2.2.0](#upgrading-to-rdi-220).
+
+### Replace the Oracle LogMiner batch and sleep properties
+
+Debezium 3.7 no longer sizes each LogMiner iteration by a range of SCNs. Instead, while the
+collector catches up, each iteration reads `log.mining.log.count.min` (default `2`) logs per redo
+thread, and up to `log.mining.log.count.growth.max` (default `4`) while a long-running transaction
+holds back the mining. RDI 2.4.0 rejects the properties that Debezium removed. This example
+shows where to remove them from your configuration:
+
+```yaml
+sources:
+  <source-name>:
+    advanced:
+      source:
+        log.mining.batch.size.min: <value> # Rejected since RDI 2.4.0
+        log.mining.batch.size.max: <value> # Rejected since RDI 2.4.0
+        log.mining.batch.size.default: <value> # Rejected since RDI 2.4.0
+        log.mining.batch.size.increment: <value> # Rejected since RDI 2.4.0
+        log.mining.sleep.time.min.ms: <value> # Rejected since RDI 2.4.0
+        log.mining.sleep.time.max.ms: <value> # Rejected since RDI 2.4.0
+        log.mining.sleep.time.default.ms: <value> # Rejected since RDI 2.4.0
+        log.mining.sleep.time.increment.ms: <value> # Rejected since RDI 2.4.0
+        log.mining.scn.gap.detection.gap.size.min: <value> # Rejected since RDI 2.4.0
+        log.mining.scn.gap.detection.time.interval.max.ms: <value> # Rejected since RDI 2.4.0
+        log.mining.continuous.mine: <value> # Rejected since RDI 2.4.0
+        log.mining.buffer.infinispan.cache.rollbacks: <value> # Rejected since RDI 2.4.0
+        log.mining.buffer.ehcache.rollbacks.config: <value> # Rejected since RDI 2.4.0
+```
+
+A collector that is already running ignores these properties after the upgrade, so it mines with
+the default log counts until you redeploy.
+
+If you set custom values for the removed properties, replace them as follows:
+
+| Removed properties | What to do |
+| :-- | :-- |
+| `log.mining.batch.size.*`, raised so the collector keeps up with the volume of your redo logs | Remove them and raise `log.mining.log.count.min`. |
+| `log.mining.batch.size.*`, raised so mining advances past long-running transactions | Remove them and raise `log.mining.log.count.growth.max`. |
+| `log.mining.sleep.time.*` | Remove them. Debezium 3.7 has no replacement. |
+| `log.mining.scn.gap.detection.*` | Remove them. Debezium 3.7 has no replacement. |
+| `log.mining.continuous.mine`, `log.mining.buffer.infinispan.cache.rollbacks`, and `log.mining.buffer.ehcache.rollbacks.config` | Remove them. Debezium 3.7 has no replacement. |
+
+The new properties count redo logs, not SCNs, so you can't convert an old batch size to a log
+count. Start with the defaults, and raise the values in small steps if the collector falls behind.
+The log count properties don't apply when `log.mining.strategy` is set to `redo_log_catalog`.
+
+`log.mining.strategy: redo_log_catalog` still works in Debezium 3.7, but Debezium has deprecated
+it and plans to remove it. Use the default `online_catalog` instead, or `hybrid` if `lob.enabled`
+is not set to `true`.
+
+### Check MySQL and MariaDB sources
+
+RDI 2.4.0 rejects `snapshot.mode: never` for MySQL and MariaDB sources, and a source that uses it
+fails at startup. Use `no_data`, which also skips the initial snapshot, if it suits your case.
+
+In addition, these sources parse DDL statements with a new parser. If a source fails on a DDL
+statement that it parsed before, set `ddl.parser.type: legacy` in the `advanced.source` section
+of the source.
+
+### Update the database.ssl.mode values of MariaDB sources
+
+MariaDB sources now use the MariaDB connector, which takes MariaDB values for `database.ssl.mode`.
+Replace `disabled` and `preferred` with `disable`, `required` with `trust`, `verify_ca` with
+`verify-ca`, and `verify_identity` with `verify-full`. The default is now `disable`.
+`disable` turns off encryption. `trust` requires encryption but doesn't verify the server
+certificate.
+
+### Check changes in the captured data
+
+PostgreSQL user defined types are reported without their schema, for example `geometry`
+instead of `"inventory"."geometry"`. This changes the metadata that RDI reports, without any
+change to your configuration.
+
 ## Upgrading to RDI 2.2.0
 
 Check the following changes before you upgrade from RDI 2.0.0. If you upgrade from RDI 1.19.x or

content/integrate/redis-data-integration/release-notes/rdi-2-4-0.md

@@ -0,0 +1,28 @@
+---
+Title: Redis Data Integration release notes 2.4.0 (TBD 2026)
+alwaysopen: false
+categories:
+- docs
+- operate
+- rs
+description: |
+  Debezium 3.7 for the Debezium collector.
+linkTitle: 2.4.0 (TBD 2026)
+toc: 'true'
+weight: 964
+---
+
+## What's New in 2.4.0
+
+### Breaking Changes
+
+See [Upgrading to RDI 2.4.0](/content/integrate/redis-data-integration/installation/upgrade.md#upgrading-to-rdi-240) for the breaking changes and the steps to take before and after you upgrade.
+
+- Oracle sources reject the LogMiner batch, sleep, and SCN gap detection properties that Debezium 3.7 removed.
+- MySQL and MariaDB sources no longer accept `snapshot.mode: never`, and parse DDL statements with a new parser.
+- MariaDB sources take MariaDB values for `database.ssl.mode`.
+- PostgreSQL user defined types are reported without their schema.
+
+### Improvements
+
+- **Debezium upgrade**: Upgraded Debezium to version 3.7.0.