Skip to content

test: support --complete in the sqllogictest runner - #324

Draft
andygrove wants to merge 8 commits into
apache:mainfrom
andygrove:slt-complete
Draft

andygrove wants to merge 8 commits into
apache:mainfrom
andygrove:slt-complete

Conversation

@andygrove

Copy link
Copy Markdown
Member

Which issue does this PR close?

Part of #56: the "Support --complete in the sqllogictest runner to regenerate expected output" item. Builds on #316, which adds the simplify and evaluate modes. Until #316 merges, this PR also shows its commit. Only the last commit, test: support --complete in the sqllogictest runner, is new here.

Rationale for this change

DataFusion's sqllogictest runner has --complete, which fills in or rewrites the expected output of a test file from the results of running it. This repository's runner has no such option, so expected results have to be typed by hand.

What changes are included in this PR?

  • --complete: a new flag in tests/sqllogictests.rs. For each matching file it calls sqllogictest's update_test_file, with the same validators and normalizer as a normal run, and prints COMPLETED instead of PASS.
  • One mode: completion writes one set of results, so it runs only simplify mode, or the mode --mode= names. Both modes must give the same results, and a normal run afterwards checks that.
  • Skipped records: records skipped for the mode's label, such as skipif simplify or onlyif spark, are written back unchanged.
  • README: a "Filling in expected results" section in test_files/README.md. It says to review the diff and check the new results against Spark before committing them.

Are these changes tested?

  • No-op round trip: running --complete on all 246 files changes none of them, so completion writes files in the format they're already in.
  • New query: I added SELECT expm1(1::double) to math/expm1.slt with an empty result. --complete filled in 1.718281828459045, and the file then passed in both modes. That edit is not part of this PR.

Are there any user-facing changes?

No. The runner is a test tool.

This PR was written by an LLM (Claude) and reviewed before opening.

🤖 Generated with Claude Code

andygrove and others added 3 commits November 16, 2026 09:11
Run every .slt file in two modes. `simplify` keeps the default optimizer,
so literal-only calls are constant-folded during planning. `evaluate`
removes the `simplify_expressions` rule, so every call goes through
`invoke_with_args`, as it does in engines that build physical expressions
directly. DataFusion's rewrite-only functions such as `arrow_cast` are
still rewritten in `evaluate` mode.

Files and queries that fail in `evaluate` mode are listed or marked
`skipif evaluate` with the issue that tracks each one (apache#305-apache#315).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
time_trunc (apache#310) now works without simplify, so it is no longer a known
failure. Mark the queries in files added or extended since then that need
constant folding: unix_date in date_from_unix_date.slt (apache#306), the
make_dt_interval overflow cases (apache#315), bround with a constant-expression
scale, and positive's EXPLAIN plans, which show the simplify rewrite.
Add a `--complete` flag that rewrites the expected output of the
matching files with the results of running them, as DataFusion's runner
does. Completion runs in one mode, `simplify` unless `--mode` says
otherwise, and leaves records skipped for that mode unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts:
#	crates/sqllogictest/tests/sqllogictests.rs
A file in EVALUATE_KNOWN_FAILURES passed as XFAIL whatever failed in it, and
its first failure stopped the records after it from running. Remove the list
and mark each record that calls a function with no invoke_with_args yet as
`skipif evaluate`, with a note at the top of the file linking its issue. The
other records in those files, such as the unbase64 queries in base64.slt, now
run in both modes.

Narrow the README: evaluate mode only changes the optimizer, so SQL planning
can still evaluate some expressions first, and a call over rows is the
reliable way to test invoke_with_args. Explain why the bround records with a
constant-expression scale are skipped: the planner computes the decimal result
type from literal arguments only.
…mplete

# Conflicts:
#	crates/sqllogictest/tests/sqllogictests.rs
# Conflicts:
#	crates/sqllogictest/test_files/README.md
#	crates/sqllogictest/test_files/conditional/if.slt
#	crates/sqllogictest/test_files/datetime/date_part.slt
#	crates/sqllogictest/test_files/datetime/date_trunc.slt
#	crates/sqllogictest/test_files/datetime/make_dt_interval.slt
#	crates/sqllogictest/test_files/datetime/trunc.slt
#	crates/sqllogictest/tests/sqllogictests.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant