Skip to content

chore: the Snowflake Kamelets can log in with a key pair (#3170) - #3208

Merged
davsclaus merged 1 commit into
apache:mainfrom
davsclaus:ci-issue-3170-snowflake
Nov 18, 2026
Merged

davsclaus merged 1 commit into
apache:mainfrom
davsclaus:ci-issue-3170-snowflake

Conversation

@davsclaus

Copy link
Copy Markdown
Contributor

Part of #3170: snowflake-source and snowflake-sink can log in with a key pair, as well as with a password.

The problem

Both Kamelets logged in only with username and password. Snowflake is phasing out sign-in with a password alone:

  • Snowflake's user management page says users of TYPE=SERVICE "cannot log in using a password". The CREATE USER reference describes LEGACY_SERVICE as "similar to SERVICE, but allows password authentication", and says that type is being deprecated.

  • Snowflake's rollout plan (read on 2026-11-18) has three phases:

    • Sep 2025 to Jan 2026: MFA becomes mandatory for people who sign in to Snowsight with a password.
    • May to Jul 2026: new users need strong authentication, and LEGACY_SERVICE can no longer be set.
    • Aug to Oct 2026: "human users cannot authenticate with a password alone", "all non-human users are blocked from using a password", and LEGACY_SERVICE users are migrated to SERVICE.

    Enforcement dates "vary by account, and may be extended". Reader accounts, trial accounts and Snowflake Postgres are exempt.

So a Kamelet that can only send a password stops working for a service user. Snowflake documents key-pair authentication for clients like this one.

The options

The data source bean is still SnowflakeDataSourceFactory.createDataSource(). The driver is snowflake-jdbc 4.3.4, as spec.dependencies declares. Its setPrivateKeyBase64(key, pwd) and setPrivateKeyFile(path, pwd) take two arguments, so a bean property cannot call them. The template therefore puts the driver's connection properties into the data source's properties map. The property names are taken from SFSessionProperty in the 4.3.4 sources.

Option Template mapping (bean dsBean) What it does
privateKey properties[private_key_base64]: "{{?privateKey}}" The private key file in PEM format, encoded in base64 on one line (base64 -w0 rsa_key.p8). format: password.
privateKeyFile properties[private_key_file]: "{{?privateKeyFile}}" The path of the PEM key file, such as a Kubernetes Secret mounted as a file.
privateKeyPassphrase properties[private_key_pwd]: "{{?privateKeyPassphrase}}" The passphrase of an encrypted key. format: password.
password (now optional) password: "{{password:}}" (was "{{password}}") Unchanged when it is set.
  • When a key is set, the driver chooses key-pair login by itself. With no authenticator set, it uses SNOWFLAKE_JWT whenever a private key property is present, and it then does not send the password.
  • Why {{password:}} and not {{?password}}: an optional placeholder that resolves to nothing skips the setter, so the data source's password stays null. The data source then refuses to connect before the driver even looks at the key: "Cannot create connection because password is missing in DataSource properties". This happened in a test with {{?password}}.
    • With an empty default, the setter receives "". For key login, the empty password is never sent.
    • With neither a password nor a key, the driver fails with Missing password. before any request.
  • The key options use {{?x}}: when they are unset, the setter is skipped, so no key property is put into the map. That matters here because the driver treats an empty string as a key. The unset case below proves it: the password login request is identical to main's.
  • The key is text, as asked: private_key_base64 is the driver's only text form. It is the PEM file encoded in base64; the PEM text itself fails with Private key provided is invalid or not supported. The file option covers a Secret mounted as a file.
  • Encrypted keys: driver 4.x decrypts with its bundled Bouncy Castle by default. A PKCS#8 key encrypted with AES-256-CBC, as Snowflake's guide makes it with openssl pkcs8 -topk8 -v2 aes-256-cbc, worked from both options.
  • No pattern:: none of the options goes into a URI. The bean receives them directly.
  • Descriptions:
    • The "Good to know" paragraph of both Kamelets says how to log in, and that a SERVICE user cannot use a password.
    • The password description gives the Missing password error.
    • The option descriptions name the failure cases found in testing.
    • Both partial pages have an "Authentication methods" section and an "Example with a Key Pair".

Verification

There is no local Snowflake. Instead, I captured the driver's login request with a stand-in: a small Python HTTP server that writes each POST /session/v1/login-request body (gunzipped) to a file and answers with Snowflake's login-failure JSON.

  • Pointing the driver at it: instanceUrl was localhost:18099/?ssl=off&account=MYACCOUNT. These are driver connect-string parameters, used for the test only.
  • Keys: two RSA 2048 keys made with the commands from Snowflake's guide, one unencrypted (-nocrypt) and one encrypted with -v2 aes-256-cbc.
  • Checking each JWT: decode its claims, compare the SHA256: fingerprint in iss with the fingerprint of the matching .pub file (SHA-256 of the DER public key, as Snowflake computes it), and check the RS256 signature with openssl dgst -verify.

Tested with Camel JBang 4.24.0-SNAPSHOT, camel run ... --local-kamelet-dir:

Case Kamelet Result
password only, main and this branch sink, source Login JSON identical apart from the driver's cloud-platform probe timings: PASSWORD sent, no AUTHENTICATOR. camel get endpoint shows the same local-sql-snowflake-*-1://...?dataSource=#bean:dsBean-2 URIs on both
privateKey (base64 of unencrypted PEM), no password sink, source AUTHENTICATOR=SNOWFLAKE_JWT, no PASSWORD. The TOKEN is an RS256 JWT, sub=MYACCOUNT.MY_USER, iss=MYACCOUNT.MY_USER.SHA256:K6xv…, which matches rsa_key.pub. Signature Verified OK with that key, Verification failure with the other key
privateKeyFile (unencrypted PEM) sink, source The same: JWT login, signature verified with rsa_key.pub
privateKey (base64 of encrypted PEM) + privateKeyPassphrase sink, source JWT login. iss fingerprint SHA256:5358… matches rsa_key_enc.pub, signature verified
privateKeyFile (encrypted PEM) + privateKeyPassphrase sink, source The same, verified with rsa_key_enc.pub
privateKey set in the route's parameters from a property, value containing +, / and = sink JWT login, signature verified: the base64 value reaches the driver intact
privateKey and password both set sink JWT login, and the password is not sent
Neither a password nor a key sink, source Fails with Missing password.; no request reaches the server
privateKey and privateKeyFile both set sink Fails with Cannot have both private key file and private key base64 string value
privateKey set to the PEM text, not base64 sink Fails with Private key provided is invalid or not supported; no request
Encrypted key, wrong passphrase sink Fails with Private key provided is invalid or not supported: ... Error finalising cipher; no request
Encrypted key, no passphrase sink Fails with a NullPointerException from the driver (privateKeyPwd is null); no request
privateKeyFile that does not exist sink Fails with Invalid parameter value null for parameter type {1}.; no request

./mvnw clean verify -DskipTests and then ./mvnw clean verify passed from the root. Before building, the library copies of both Kamelets were refreshed with ./mvnw -pl library/camel-kamelets resources:copy-resources@copy-resource-one. The build regenerated kamelets-index.json.

Claude Code on behalf of Claus Ibsen

🤖 Generated with Claude Code

davsclaus added a commit to davsclaus/camel-kamelets that referenced this pull request Nov 18, 2026
… JSON array of rows, and size the connection pool (apache#3170)

The PostgreSQL, MySQL, MariaDB, SQL Server and Oracle Database Kamelets
take new optional options. With them unset, the endpoint URIs are the
same as before, apart from oracle-database-source (see below).

Sources (postgresql, mysql, mariadb, sqlserver, oracle-database):
- maxMessagesPerPoll: the most rows one poll sends.
- onConsumeFailed: an SQL statement run for a row whose processing
  failed, such as marking it as failed.
- onConsumeBatchComplete: an SQL statement run once after the rows of
  a poll are sent.
- initialDelay: how long to wait before the first poll.
- useIterator: false sends all the rows of a poll as one JSON array.
- routeEmptyResultSet: with useIterator false, sends [] when no rows
  are found.

oracle-database-source gets the delay option the other sources have,
with the same default of 500 ms. Its endpoint now carries delay=500,
which is the delay it had before.

Sinks: batch accepts a JSON array of objects and writes one row for
each, in one transaction. A single object still works.

All ten: maxTotal, maxIdle and maxWaitMillis set the commons-dbcp2
connection pool.

transacted is left out: without a Spring transaction manager it starts
no transaction, and a failed row stops the source for good.

Snowflake is left out, as apache#3208 changes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
snowflake-source and snowflake-sink logged in only with a username and a
password. A Snowflake user of TYPE=SERVICE cannot log in with a password,
and Snowflake is phasing out sign-in with a password alone for the other
users, so such a Kamelet stops working for a service user.

The Kamelets now take a private key, either as text in privateKey (the PEM
file encoded in base64, the driver's private_key_base64) or as a path in
privateKeyFile (private_key_file), and privateKeyPassphrase
(private_key_pwd) for an encrypted key. They go into the properties map of
the Snowflake data source, as its two-argument setters cannot be bound.
With a key present and no authenticator set, the driver logs in with
SNOWFLAKE_JWT and does not send the password.

password is now optional. It is bound as {{password:}}, an empty default,
because an optional placeholder skips the setter and the data source then
refuses to connect without a password, before the driver looks at the key.
With neither a password nor a key, the driver fails with "Missing password."

With the new options unset, the endpoint URIs and the login request sent
to Snowflake are the same as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@davsclaus
davsclaus force-pushed the ci-issue-3170-snowflake branch from d5d3447 to 2abfb32 Compare November 18, 2026 20:01
@davsclaus
davsclaus merged commit 83a792a into apache:main Nov 18, 2026
6 checks passed
@davsclaus
davsclaus deleted the ci-issue-3170-snowflake branch November 18, 2026 20:39
davsclaus added a commit that referenced this pull request Nov 18, 2026
… JSON array of rows, and size the connection pool (#3170) (#3215)

The PostgreSQL, MySQL, MariaDB, SQL Server and Oracle Database Kamelets
take new optional options. With them unset, the endpoint URIs are the
same as before, apart from oracle-database-source (see below).

Sources (postgresql, mysql, mariadb, sqlserver, oracle-database):
- maxMessagesPerPoll: the most rows one poll sends.
- onConsumeFailed: an SQL statement run for a row whose processing
  failed, such as marking it as failed.
- onConsumeBatchComplete: an SQL statement run once after the rows of
  a poll are sent.
- initialDelay: how long to wait before the first poll.
- useIterator: false sends all the rows of a poll as one JSON array.
- routeEmptyResultSet: with useIterator false, sends [] when no rows
  are found.

oracle-database-source gets the delay option the other sources have,
with the same default of 500 ms. Its endpoint now carries delay=500,
which is the delay it had before.

Sinks: batch accepts a JSON array of objects and writes one row for
each, in one transaction. A single object still works.

All ten: maxTotal, maxIdle and maxWaitMillis set the commons-dbcp2
connection pool.

transacted is left out: without a Spring transaction manager it starts
no transaction, and a failed row stops the source for good.

Snowflake is left out, as #3208 changes it.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant