Skip to content

Compare DateTimes at the offset the evaluation actually ran at - #145

Open
bryantaustin13 wants to merge 2 commits into
evaluation-timestamp-and-offsetfrom
datetime-evaluation-offset-comparison
Open

bryantaustin13 wants to merge 2 commits into
evaluation-timestamp-and-offsetfrom
datetime-evaluation-offset-comparison

Conversation

@bryantaustin13

Copy link
Copy Markdown
Contributor

The comparison side of #138, stacked on #142 (the request side). Closes #83 and #84. Supersedes #132.

Base branch: this PR targets evaluation-timestamp-and-offset (#142), so its diff is only this change. Once #142 merges, GitHub retargets it to main.

Problem

A DateTime always has an offset during evaluation. In the CQL Author's Guide: "If no timezone offset is specified, the timezone offset of the evaluation request timestamp is used." Expected outputs are almost always written without one, but an engine must return one, because FHIR requires an offset on a dateTime with a time. The runner compared DateTimes as text, so:

expected: @2005-05-10T10:20:30
actual  : @2005-05-10T10:20:30-06:00     -> fail, although it is the same value at the evaluation offset (#84)

expected: @2014-01-01T12:05:05.955+00:00
actual  : @2014-01-01T12:05:05.955Z      -> fail, although Z and +00:00 are the same offset (#83)

#142 sends the evaluation timestamp and offset, defaulting to UTC. On its own that fixes no test, because the comparison is still textual. This PR changes the comparison.

Change

DateTime equality (src/shared/datetime-comparison.ts):

  • Two DateTimes with at least an hour are equal when, at the same precision, they denote the same instant. So Z = +00:00, and @…T16:20:30Z = @…T10:20:30-06:00.
  • A side written without an offset is read at the evaluation offset.
  • Different precisions stay unequal. Dates, date-precision DateTimes and Times keep plain comparison.
  • With no evaluation offset known, an offset-less value only equals another offset-less value, as before.

Which offset the evaluation ran at:

  • If the server honours the timestamp parameter or the Timezone header (see Send an evaluation request timestamp, defaulting to UTC, with every test #142), it is the test's requested offset: UTC by default, or the test's evaluationTimezoneOffset / evaluationDateTime.
  • If it honours neither, it is the server's own offset. The start-of-run probe evaluates an offset-less DateTime and records the offset it comes back with as evaluationRequest.serverTimezoneOffset, and comparison uses it. This judges each result against the evaluation that actually produced it, rather than against an offset the server never used. The run log states it, e.g. results are compared at the server's own offset -06:00.

time-hasOffset: a result whose _valueDateTime, or Period _start / _end, carries time-hasOffset: false (Using CQL With FHIR) is read as having no offset. This is @brynrhodes's request from #77 / #119.

The evaluation offset is threaded through resultsEqual into lists, tuples and interval boundaries. The README section added in #142 now explains the comparison, and the results schema gains serverTimezoneOffset.

Verification

Against HAPI FHIR 8.10.0 / CQL engine 5.3.0 (clinical-reasoning 4.12.0). This server ignores both mechanisms, and the probe reports serverTimezoneOffset: -06:00. Full cql-tests suite:

pass fail skip error
main 1639 170 14 0
this branch 1665 144 14 0

Exactly 26 tests move from fail to pass, and no other record changes status or actual value across all 1823:

  • 22 where the expected value has no offset and the actual has the server's -06:00, for example DateTimeAdd5Seconds, DateTimeMillisecond, HighBoundaryDateTimeMillisecond, ToDateTime3, DateTimeMin, DateTimeMax;
  • 1 Z vs +00:00 (ToDateTime6);
  • 3 intervals with DateTime boundaries (TestPeriod1, TestPeriod2, DateTimeIntervalTest).

All 22 offset cases carry the same -06:00, including January values when the host's zone is at -07:00. So the engine uses one request-level offset, as CQL specifies, not a zone, which is what makes a single observed server offset sound.

270 unit tests pass (252 before, 18 new). They cover:

  • equality of offsets and instants, and that different instants and precisions stay unequal;
  • the old behaviour when no offset is known;
  • non-DateTime strings left alone;
  • nesting in lists, tuples and intervals;
  • time-hasOffset on values and Period boundaries;
  • the effective-offset rule, and runTest passing or failing against a server that ignores the request.

One bug was caught while testing: instants are built with setUTCFullYear rather than Date.UTC, which reads years 0–99 as 1900–1999 and would have made @0001 equal @1901.

Carried over from #132

#132 is closed in favour of this PR, so its review discussion is reproduced here:

@cmoesel (2026-09-01): Thanks. You can resolve #83 when this gets merged.

This PR covers #83's case (ToDateTime6, Z vs +00:00), so it carries Closes #83, along with #84. Because it is stacked on #142, GitHub closes those issues automatically only if it merges into main. If it merges into evaluation-timestamp-and-offset first, #83 and #84 need closing by hand once the change reaches main.

Notes

🤖 Generated with Claude Code

A DateTime always has an offset during evaluation: one written without an
offset takes the offset of the evaluation request. Expected outputs are
almost always written without one (`@2005-05-10T10:20:30`), while engines
must return one, because FHIR requires it on a dateTime with a time. The
runner compared the two as text, so a correct result failed whenever an
offset was attached, and `Z` never matched `+00:00`.

This is the comparison side of #138, on top of the request side (#142):

- Two DateTimes with at least an hour are equal when, at the same precision,
  they denote the same instant. A side without an offset is read at the
  evaluation offset. Different precisions stay unequal; Dates and Times keep
  plain comparison. This covers `Z` = `+00:00` (#83) and offset vs no offset
  (#84), and supersedes #132.
- The evaluation offset is the one the evaluation actually ran at: the
  requested offset if the server honours the timestamp parameter or Timezone
  header, otherwise the server's own. The start-of-run probe now records the
  latter as `evaluationRequest.serverTimezoneOffset`, observed from an
  offset-less DateTime it evaluates. With neither known, offset-less values
  only equal offset-less values, as before.
- A result whose `_valueDateTime`, or Period `_start` / `_end`, carries
  `time-hasOffset: false` (Using CQL With FHIR) is read as having no offset.
- The offset reaches nested values: lists, tuples and interval boundaries.

Against HAPI FHIR 8.10.0 / CQL engine 5.3.0 (clinical-reasoning 4.12.0),
which ignores both mechanisms and evaluates at -06:00, the full cql-tests
suite goes from 1639/170/14/0 to 1665/144/14/0: exactly 26 tests move from
fail to pass (22 offset vs no offset, 1 `Z` vs `+00:00`, 3 intervals with
DateTime boundaries), and no other record changes status or actual value.
270 unit tests pass (252 before, 18 new).

Instants are built with setUTCFullYear, not Date.UTC, which reads years
0-99 as 1900-1999 and would make @0001 equal @1901.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bryantaustin13

Copy link
Copy Markdown
Contributor Author

I ran this PR against a second engine, the JavaScript cql-execution test-server, to check that the comparison change holds beyond the Java engine.

Setup

Code Pass Fail Skip Error
main, original config 1692 64 86 0
main, runner-issue skips removed 1696 107 39 0
#142 1696 107 39 0
this PR (#145, includes #142) 1736 67 39 0

So this PR fixes 40 tests on cql-execution, with no regressions:

The probe reports that this server ignores both the timestamp parameter and the Timezone header, and evaluates at Z. So these results come from comparing at the server's effective offset, which is the case this PR was designed for.

The 3 #84-skipped tests that still fail aren't comparison problems:

  • ToDateTime2 and ToDateTime3: ToDateTime('2014-01-01T12:05') comes back as @2014-01-01T12:05-06:00, the host offset, while literals get the request offset +00:00. Inside the engine, ToDateTime('2014-01-01T12:05') = @2014-01-01T12:05 is false. CQL 1.5.3, ToDateTime: "If no timezone offset is supplied, the timezone offset of the evaluation request timestamp is assumed." This looks like a cql-execution issue; I can open one there.
  • DateTimeToString2: the result is a string with an offset, and the test expects none. That's Wrong Expected Outcome for ToString of DateTime with No Offset Specified cql-tests#73.

Other findings from this run, not related to this PR:

On the Java engine (HAPI FHIR 8.12.0 / clinical-reasoning 4.13.0 / CQL engine 5.4.0, cql-tests 9e92193), the result is unchanged: #141 + #142 + #145 together fix the same 67 tests as before, with no regressions. The absolute totals in the description came from an older local copy of the tests; the fixes are the same.

Once this merges, the #83/#84 skips can come out of conf/cql-execution-local.json. I'm happy to do that as a follow-up.

@bryantaustin13

Copy link
Copy Markdown
Contributor Author

Follow-up on ToDateTime2 / ToDateTime3 from my comment above: I've opened cqframework/cql-execution#385 for the engine side. With the fix proposed there applied locally, both tests pass with this PR's comparison, and no other test changes (1736 → 1738 pass on cql-execution).

With DateTimes compared at the offset the evaluation ran at, the 40
tests skipped in cql-execution-local.json as "Test Runner Not Detecting
Equal DateTimes (Offset vs No Offset)" (#83) or "(+00:00 vs Z)" (#84)
pass. A full run against cql-execution with the cql-tests submodule as
pinned goes from 1696 pass / 60 fail / 86 skip to 1736 / 60 / 46, and
those 40 skip-to-pass changes are the only difference.

DateTimeToString2 keeps its skip: it still fails, because the test
expects ToString of a DateTime without an offset to print no offset
(cql-tests #73), which is not a comparison problem.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant