Repository navigation
Description
Background
EQL (Encrypt Query Language) is the SQL that lets PostgreSQL store and query encrypted values. EQL has two versions. EQL v3 serves the TypeScript SDK. EQL v4 serves Stack Encrypt and the Go SDK.
Each version has an install bundle: one SQL file. The bundle creates two schemas, such as eql_v4 and eql_v4_internal, and fills them with EQL's functions, operators, and query types. The column types for encrypted data are in the public schema instead, so the customer's data never depends on the EQL schemas.
There are two ways to install a bundle today:
stash eql installandstash eql upgrade, in the CLI. These install EQL v3 only.- Directly, with
psql -f. For EQL v4 this is the only way. Each Go release attaches the v4 bundle aseql-v4-install.sql, and the Go README and CHANGELOG tell users to run it withpsql -f.
Problem
The bundle's first two statements drop the EQL schemas with CASCADE, then the rest of the bundle creates them again:
DROP SCHEMA IF EXISTS eql_v4 CASCADE;
DROP SCHEMA IF EXISTS eql_v4_internal CASCADE;CASCADE also drops every customer object that depends on an EQL function, operator, or type. PostgreSQL raises no error. PostgreSQL's notice names only the first 100 of the about 4,400 objects that the drop removes. The operators that queries use, such as public.=, are in public, but they call functions in eql_v4_internal. So a customer object that uses an EQL operator depends on EQL, too.
The customer loses only the objects that use EQL. The customer's other policies, indexes, and constraints stay. I re-installed the rc.5 v4 bundle over a database on PostgreSQL 17, and found:
| Customer object | Uses EQL | After re-install |
|---|---|---|
| Encrypted columns and their rows | No | Kept |
Index on eql_v4.ord_term_ore(col) |
Yes | Dropped. Queries then scan the whole table. |
| View that filters with an EQL operator | Yes | Dropped |
| Restrictive row-level security policy using an EQL operator | Yes | Dropped. More rows of the table become visible. |
Permissive row-level security policy USING (true) |
No | Kept |
CHECK constraint calling an eql_v4 function |
Yes | Dropped |
Generated column calling an eql_v4 function |
Yes | Dropped, with its stored data |
SQL function (BEGIN ATOMIC) calling an eql_v4 function |
Yes | Dropped |
PL/pgSQL (procedural SQL) function calling an eql_v4 function |
Yes | Kept, because PostgreSQL does not track what its body calls |
The policies most likely to use EQL are the ones that filter on encrypted columns, such as a tenant filter. When PostgreSQL drops a restrictive policy, access to the table widens without an error.
For EQL v3, #918 recorded this risk, and #961 made the CLI safe. Before stash eql upgrade or a forced install changes anything, the CLI takes a lock and lists the customer objects that depend on EQL. The CLI saves and rebuilds functional indexes. The CLI refuses to continue if anything else depends on EQL.
That protection covers eql_v3 only. Nothing protects a direct install, of either version.
The problem affects every Go release so far. The rc.3 upgrade note told users to re-install the bundle, and gave no warning about these objects. The rc.5 upgrade note lists the objects with a query, and tells users to save and re-create them by hand.
stash eql install cannot install EQL v4 yet (#1143). One reason is that the @cipherstash/eql npm package does not include the v4 bundle. Version 3.0.6, the latest on npm, contains only cipherstash-encrypt.sql. #1142 planned to add the v4 bundle, and was closed with no linked pull request.
Proposal
The CLI is the preferred way to install EQL. Make it install EQL v4 safely, then deprecate the direct install.
- Add the v4 bundle to
@cipherstash/eql. Include the v4 install and uninstall SQL, with a release manifest, so the CLI can check the bundle's digest as it does for v3. Decide which version the bundle reports. The Go release stamps its copygo-0.1.0-rc.N. - Let the CLI install EQL v4 (
stash eql installcan't install EQL v4 — add--eql-version 3|4|all#1143). Add the--eql-version 3|4|alloption thatstash eql installcan't install EQL v4 — add--eql-version 3|4|all#1143 describes. - Use fix(cli): make EQL reinstall dependency-safe #961's safe reinstall for v4 too. The lock, the list of dependent objects, the index rebuild, and the refusal must cover
eql_v4andeql_v4_internal. Grants andverifymust cover them as well. - Update the Go docs. The Go README and the next Go CHANGELOG point to
stash eql install. The docs say that Go teams need Node.js, fornpx stash. - Deprecate the direct install. The bundle attached to Go releases gets a comment at the top that says the direct install is deprecated, and why. A later release stops attaching the bundle.
- Tell existing Go users. Anyone who followed the rc.3 upgrade note may have lost policies, constraints, views, or generated columns. Tell the Go partner, and say how to check: the query in the rc.5 upgrade note lists what depends on EQL now. Compare its output with the objects their migrations create.
Steps 1 to 3 must be released together. Step 5 must wait until they are released.
Evidence
- The two
DROP SCHEMA … CASCADEstatements are at lines 39 and 47 of the rc.5 v4 bundle, built fromnextat d2af608. These two statements are the only destructive statements in the bundle, apart fromDROP FUNCTION IF EXISTS eql_v4.version. - The v3 bundle has the same two statements, for
eql_v3andeql_v3_internal, from the same source template:packages/eql/src/v3/schema.sql. - The table above comes from a scratch PostgreSQL 17.5 cluster, re-installing the rc.5 v4 bundle over databases with the rc.4 and rc.5 bundles installed.
- The query in the rc.5 upgrade note (
languages/golang/CHANGELOG.md) listed exactly the dropped objects on rc.4 and on rc.5, and nothing on a new install.
Relationship to other work
stash eql installcan't install EQL v4 — add--eql-version 3|4|all#1143: the CLI cannot install EQL v4. This issue adds the safe reinstall and the deprecation to it.@cipherstash/eql4.0: ship the v3 and v4 bundles together and move Stack Encrypt's targets to v4 #1142: closed, but the v4 bundle is not in the npm package.- Make EQL reinstall safe: rebuild indexes and reject unsafe dependencies #918 and fix(cli): make EQL reinstall dependency-safe #961: the v3 decision and the CLI's safe reinstall. Step 3 extends that reinstall to v4.
- docs(go): an rc.5 changelog section #1692: the rc.5 CHANGELOG, whose upgrade note gives the manual steps for now.
Activity