Skip to content

feat(python): encrypt models into EQL v4 rows - #1499

Merged
yujiyokoo merged 3 commits into
feat/python-sdk-cipherfrom
feat/python-sdk-encrypt
Nov 13, 2026
Merged

yujiyokoo merged 3 commits into
feat/python-sdk-cipherfrom
feat/python-sdk-encrypt

Conversation

@yujiyokoo

Copy link
Copy Markdown
Contributor

Summary

Cipher.encrypt now turns a list of declared models into rows for PostgreSQL with EQL v4. EQL is CipherStash's set of PostgreSQL types for encrypted values. A sealed field becomes a str of EQL JSON. psycopg inserts it with a plain %s and no cast. A passthrough field is the value itself.

Changes

  • Python API: Cipher.encrypt(models) returns one dict per model. The new InvalidValueError is a StashError and a ValueError. A README stub shows the CREATE TABLE, the encrypt call, and the insert.
  • Value checks: the SDK checks the exact type and range of every value before any request. A bad value raises InvalidValueError with the field's name. The error never contains the value. The codes are value_none, value_wrong_type, value_out_of_range, value_not_representable, and value_naive_datetime.
  • Engine client: one client per process, made on the first encrypt of any Cipher. The Python global interpreter lock (GIL) is released during each engine call. The client reads CS_ZEROKMS_HOST. The client's ZeroKMS connection counts requests for the tests.
  • Rust dependencies: the http feature is on. stack-kms, tokio, and zerokms-protocol are new. Every new crate in the Python Cargo.lock has the root lock's version.
  • CI: a new live job in tests-python.yml runs mise run test:live. The job encrypts against ZeroKMS and inserts rows into a postgres:17 service with the EQL v4 bundle from this tree. The job skips on a pull request from a fork.

Verification

  • pytest on Python 3.14 and 3.9: 247 passed, 7 skipped. The skips are the 5 live tests and 2 gate-probe tests.
  • cargo test --locked, clippy, and cargo fmt --check: no failures or warnings.
  • pnpm run test:scripts: 83 of 83 files pass.
  • The live tests have not run yet. This machine has no CipherStash credentials. The first run is the live job on this PR.

Related

Closes #1389. Refs #1341.

Review notes

  • A passthrough field has no "type" key in the output. python: query an equality term #1391 can add it.
  • KmsError is not built yet. A ZeroKMS failure is a StashError.
  • eql-suite-ci.test.mjs now accepts a Rust cache step for a second workspace in the same job. The live job needs one for languages/python and one for packages/eql.

🤖 Generated with Claude Code

@changeset-bot

changeset-bot Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: d1cf50d

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai

coderabbitai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI; Enforced overrides: Organization
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b9c88d2f-944f-4f19-97ad-9d1239ffb1d5

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review

@blacksmith-sh

This comment has been minimized.

@yujiyokoo
yujiyokoo added this pull request to stack #1520 November 4, 2026 02:58
yujiyokoo and others added 3 commits November 5, 2026 00:20
Cipher.encrypt returns one dict per model. A sealed field is a str of
EQL JSON, which psycopg inserts with a plain %s and no cast. A
passthrough field is the value itself, echoed by the engine.

Every value is checked by exact type and range while the models are
written into the engine's input frame. A bad value raises the new
InvalidValueError with the field's name. No request is made, and the
engine client is not built. A property that raises propagates
unchanged.

One engine client is shared per process, with one multi-thread tokio
runtime. It is built on the first encrypt of any Cipher, and the GIL
is released around each engine call. Its ZeroKMS connection counts
requests, so the tests check the 500-keys-per-request rule. The http
feature is on, and every new crate matches the root lock's version.

A live CI job in tests-python.yml encrypts against ZeroKMS and inserts
a row into Postgres with the EQL v4 bundle from the tree. It skips on
a fork's pull request. The EQL cache test now accepts a second
workspace's cache step in the same job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Decimal passthrough now comes back as a Decimal. Its digits go to
Decimal as a tuple, not as bytes.

A zero Decimal with a huge exponent no longer hangs the process with
the GIL held. A zero with a positive exponent returns at once. An
exponent above 38 is refused as value_not_representable.

The shared client now reads CS_ZEROKMS_HOST, then CS_VITUR_HOST, as
the other SDKs do. A bad value is refused as stack_kms_invalid_endpoint,
before the credentials are read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
encrypt failed on every model with a Passthrough field. The extension
wrote the value as a passthrough node. The plan's "passthrough" output
already carries the field's frame as it is, so the engine returned the
value nested in a second passthrough, and reading it failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@yujiyokoo
yujiyokoo force-pushed the feat/python-sdk-encrypt branch from f4a374a to d1cf50d Compare November 4, 2026 15:24
@yujiyokoo
yujiyokoo marked this pull request as ready for review November 10, 2026 00:40
@yujiyokoo
yujiyokoo requested a review from a team as a code owner November 10, 2026 00:40

@cipherstash-agent cipherstash-agent left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review attestation

Reviewed d1cf50d · posted from Claude Code · claude-sonnet-5-5 · effort medium
Sources: code-review · model unknown
Skills: code-review, review-cross-check, review-post

Findings (9): 0 critical, 4 major, 5 minor — 9 inline, 0 below. Each was checked against d1cf50d by reading the code; none was reproduced. No existing review feedback overlapped.


Generated by Claude Code

Comment thread .github/workflows/tests-python.yml Resolved
Comment thread .github/workflows/tests-python.yml Resolved
Comment thread languages/python/tests/test_encrypt.py Resolved
Comment thread languages/python/mise.toml Resolved

@cipherstash-bot cipherstash-bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommendation: 🟡 merge after changes (1 of 8 review job(s) failed)

Nothing blocks this merge. All findings are test gaps or small CI and documentation fixes, and they can wait for a follow-up. The earlier review threads already cover the shared-client concurrency, fork safety and utcoffset() points, so they are not repeated.

Other findings not posted as comments

  • languages/python/crates/core/src/values.rs:162: the Int::I128 branch (a passthrough int wider than 64 bits) has no test. Add i64::MAX + 1, i128::MIN and i128::MAX to a_scalar_written_as_a_passthrough_reads_back.
  • languages/python/tests/test_encrypt_live.py:101: live tests check the returned passthrough for int and Decimal only. Add a parametrized test for str, bytes, float, bool, date and datetime.
  • languages/python/tests/test_encrypt.py:203: no test covers CS_VITUR_HOST, or that CS_ZEROKMS_HOST is read first.
  • languages/python/crates/engine/src/values.rs:174: an int wider than 128 bits says "does not fit 128 bits", while other too-wide values name their kind (for example int16). Use one message that names the declared kind.
  • languages/python/crates/core/src/encrypt.rs:106: the stack_encrypt_unknown branches in read_rows and read_row have no test. They need a forged engine output.
How this review was made
Agent Model Review type Result
claude claude-opus-5-5 test-gap 5 found, 2 posted
claude claude-opus-5-5 rust 2 found, 1 posted
claude claude-opus-5-5 python 1 found, 1 posted
claude claude-opus-5-5 infracode 3 found, 3 posted
codex gpt-5.6-terra test-gap 1 found, 1 posted
codex gpt-5.6-terra rust failed
codex gpt-5.6-terra python 0 found, 0 posted
codex gpt-5.6-terra infracode 0 found, 0 posted

Synthesis: claude-sonnet-5-5 merged the findings, removed duplicates and dropped findings it could not confirm in the code. 0 posted finding(s) were raised by two or more models.

Plain language: the reader test did not run; only the word-and-shape check was applied.

Stack: position 8 of 15 (#1405, #1406, #1414, #1415, #1433, #1437, #1453, *️⃣ #1499, #1509, #1510, #1511, #1513, #1516, #1517, #1582). *️⃣ marks this pull request.

Context loaded: the description, 3 linked issue(s) and 13 discussion entries.

Comment thread languages/python/tests/test_encrypt.py Resolved
Comment thread .github/workflows/tests-python.yml Resolved
Comment thread .github/workflows/tests-python.yml Resolved
Comment thread AGENTS.md Resolved
@yujiyokoo
yujiyokoo deleted the feat/python-sdk-encrypt branch November 13, 2026 03:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants