Skip to content

Installing the EQL v4 bundle directly drops customer objects that use EQL — install v4 through stash eql install, and deprecate the direct install #1694

Open

Description

@coderdan

Background

EQL (Encrypt Query Language) is the SQL that lets PostgreSQL store and query encrypted values. EQL has two versions. EQL v3 serves the TypeScript SDK. EQL v4 serves Stack Encrypt and the Go SDK.

Each version has an install bundle: one SQL file. The bundle creates two schemas, such as eql_v4 and eql_v4_internal, and fills them with EQL's functions, operators, and query types. The column types for encrypted data are in the public schema instead, so the customer's data never depends on the EQL schemas.

There are two ways to install a bundle today:

  • stash eql install and stash eql upgrade, in the CLI. These install EQL v3 only.
  • Directly, with psql -f. For EQL v4 this is the only way. Each Go release attaches the v4 bundle as eql-v4-install.sql, and the Go README and CHANGELOG tell users to run it with psql -f.

Problem

The bundle's first two statements drop the EQL schemas with CASCADE, then the rest of the bundle creates them again:

DROP SCHEMA IF EXISTS eql_v4 CASCADE;
DROP SCHEMA IF EXISTS eql_v4_internal CASCADE;

CASCADE also drops every customer object that depends on an EQL function, operator, or type. PostgreSQL raises no error. PostgreSQL's notice names only the first 100 of the about 4,400 objects that the drop removes. The operators that queries use, such as public.=, are in public, but they call functions in eql_v4_internal. So a customer object that uses an EQL operator depends on EQL, too.

The customer loses only the objects that use EQL. The customer's other policies, indexes, and constraints stay. I re-installed the rc.5 v4 bundle over a database on PostgreSQL 17, and found:

Customer object Uses EQL After re-install
Encrypted columns and their rows No Kept
Index on eql_v4.ord_term_ore(col) Yes Dropped. Queries then scan the whole table.
View that filters with an EQL operator Yes Dropped
Restrictive row-level security policy using an EQL operator Yes Dropped. More rows of the table become visible.
Permissive row-level security policy USING (true) No Kept
CHECK constraint calling an eql_v4 function Yes Dropped
Generated column calling an eql_v4 function Yes Dropped, with its stored data
SQL function (BEGIN ATOMIC) calling an eql_v4 function Yes Dropped
PL/pgSQL (procedural SQL) function calling an eql_v4 function Yes Kept, because PostgreSQL does not track what its body calls

The policies most likely to use EQL are the ones that filter on encrypted columns, such as a tenant filter. When PostgreSQL drops a restrictive policy, access to the table widens without an error.

For EQL v3, #918 recorded this risk, and #961 made the CLI safe. Before stash eql upgrade or a forced install changes anything, the CLI takes a lock and lists the customer objects that depend on EQL. The CLI saves and rebuilds functional indexes. The CLI refuses to continue if anything else depends on EQL.

That protection covers eql_v3 only. Nothing protects a direct install, of either version.

The problem affects every Go release so far. The rc.3 upgrade note told users to re-install the bundle, and gave no warning about these objects. The rc.5 upgrade note lists the objects with a query, and tells users to save and re-create them by hand.

stash eql install cannot install EQL v4 yet (#1143). One reason is that the @cipherstash/eql npm package does not include the v4 bundle. Version 3.0.6, the latest on npm, contains only cipherstash-encrypt.sql. #1142 planned to add the v4 bundle, and was closed with no linked pull request.

Proposal

The CLI is the preferred way to install EQL. Make it install EQL v4 safely, then deprecate the direct install.

  1. Add the v4 bundle to @cipherstash/eql. Include the v4 install and uninstall SQL, with a release manifest, so the CLI can check the bundle's digest as it does for v3. Decide which version the bundle reports. The Go release stamps its copy go-0.1.0-rc.N.
  2. Let the CLI install EQL v4 (stash eql install can't install EQL v4 — add --eql-version 3|4|all #1143). Add the --eql-version 3|4|all option that stash eql install can't install EQL v4 — add --eql-version 3|4|all #1143 describes.
  3. Use fix(cli): make EQL reinstall dependency-safe #961's safe reinstall for v4 too. The lock, the list of dependent objects, the index rebuild, and the refusal must cover eql_v4 and eql_v4_internal. Grants and verify must cover them as well.
  4. Update the Go docs. The Go README and the next Go CHANGELOG point to stash eql install. The docs say that Go teams need Node.js, for npx stash.
  5. Deprecate the direct install. The bundle attached to Go releases gets a comment at the top that says the direct install is deprecated, and why. A later release stops attaching the bundle.
  6. Tell existing Go users. Anyone who followed the rc.3 upgrade note may have lost policies, constraints, views, or generated columns. Tell the Go partner, and say how to check: the query in the rc.5 upgrade note lists what depends on EQL now. Compare its output with the objects their migrations create.

Steps 1 to 3 must be released together. Step 5 must wait until they are released.

Evidence

  • The two DROP SCHEMA … CASCADE statements are at lines 39 and 47 of the rc.5 v4 bundle, built from next at d2af608. These two statements are the only destructive statements in the bundle, apart from DROP FUNCTION IF EXISTS eql_v4.version.
  • The v3 bundle has the same two statements, for eql_v3 and eql_v3_internal, from the same source template: packages/eql/src/v3/schema.sql.
  • The table above comes from a scratch PostgreSQL 17.5 cluster, re-installing the rc.5 v4 bundle over databases with the rc.4 and rc.5 bundles installed.
  • The query in the rc.5 upgrade note (languages/golang/CHANGELOG.md) listed exactly the dropped objects on rc.4 and on rc.5, and nothing on a new install.

Relationship to other work

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    SDKbugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions