Skip to content

Enable SQL compilation caching for the ignore-nulls window functions - #647

Open
rebkwok wants to merge 1 commit into
trinodb:masterfrom
rebkwok:rebkwok/enable-caching-on-window-functions
Open

rebkwok wants to merge 1 commit into
trinodb:masterfrom
rebkwok:rebkwok/enable-caching-on-window-functions

Conversation

@rebkwok

@rebkwok rebkwok commented Oct 16, 2026

Copy link
Copy Markdown

Description

FirstValue, LastValue, NthValue, Lead, and Lag (the IGNORE NULLS-capable window functions registered on TrinoSQLCompiler), don't declare _traverse_internals or inherit_cache, which means SQLAlchemy's SQL compilation cache (added in 1.4) can't produce a cache key, falls back to recompiling from scratch every time, and emits a warning ("Class X will not make use of SQL compilation caching...").

Each class now gets its own _traverse_internals, extending Function's (the nearest ancestor in their hierarchy that defines one) with an extra traversal instruction for ignore_nulls.

For backwards compatibility, there's a hasattr check for Function._traverse_internals (doesn't exist pre-1.4)

Added two parametrized tests (across all five functions) confirming a cache key is actually produced with no warning, and that cache keys correctly distinguish both different arguments and different ignore_nulls values. Confirmed both new tests fail without the code fix.

Reproduce with (sqlalchemy >= 1.4):

import warnings
from sqlalchemy import func, table, column
import trino.sqlalchemy.dialect  # registers Lag etc. as SQLAlchemy GenericFunctions

t = table("t", column("x"))

with warnings.catch_warnings(record=True) as w:
    warnings.simplefilter("always")
    stmt = func.lag(t.c.x).over(order_by=t.c.x)
    print(stmt._generate_cache_key())  # None, before this fix
    print([str(x.message) for x in w])  # the SAWarning, before this fix

Non-technical explanation

Queries using these window functions (LAG, LEAD, FIRST_VALUE, LAST_VALUE, NTH_VALUE) couldn't benefit from SQLAlchemy's statement caching, so every execution recompiled the SQL from scratch and printed a performance-warning. This fix lets SQLAlchemy cache and reuse the compiled SQL for these statements, as it already does for most others.

Release notes

( ) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
(x ) Release notes are required, with the following suggested text:

* Fix SQLAlchemy's SQL compilation caching for `first_value`, `last_value`, `nth_value`, `lead`, and `lag`; these now cache by default and avoid emitting SQLAlchemy performance warnings.

FirstValue, LastValue, NthValue, Lead, and Lag are all registered on
the TrinoSQLCompiler, but don't declare `_traverse_internals` or
`inherit_cache`, so SQLAlchemy's SQL compilation cache (introduced in 1.4)
can't produce a cache key, falls back to recompiling from scratch every
time, and emits a warning ("Class X will not make use of SQL compilation
caching...").

Each class now gets its own _traverse_internals, extending Function's (the
nearest ancestor in their hierarchy that defines one) with an extra
traversal instruction for `ignore_nulls`.

For backwards compatibility, there's a hasattr check for
Function._traverse_internals (doesn't exist pre-1.4)
@cla-bot

cla-bot Bot commented Oct 16, 2026

Copy link
Copy Markdown

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to cla@trino.io. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant