Skip to content

Correct uncertainty-arithmetic test fixtures per CQL 1.5.2 §5.5.2 - #176

Open
zacharywhitley wants to merge 1 commit into
cqframework:mainfrom
zacharywhitley:m9-uncertain-arithmetic-fixture-correction
Open

Correct uncertainty-arithmetic test fixtures per CQL 1.5.2 §5.5.2#176
zacharywhitley wants to merge 1 commit into
cqframework:mainfrom
zacharywhitley:m9-uncertain-arithmetic-fixture-correction

Conversation

@zacharywhitley

@zacharywhitley zacharywhitley commented Nov 9, 2026

Copy link
Copy Markdown

Correct five uncertain-arithmetic fixture oracles per CQL §5.5.2

Summary

Five test fixtures in tests/cql/CqlDateTimeOperatorsTest.xml and
tests/cql/CqlTypesTest.xml encode <output> values that diverge from
CQL 1.5.2 §5.5.2 "Determining Difference and Duration". All five test
blocks carry an author-supplied TODO comment acknowledging normative
uncertainty about the oracle. CQFramework's own Kotlin first-party
unit-test oracles (in engine/src/jvmTest/kotlin/... of
cqframework/clinical_quality_language) assert different, spec-correct
values for semantically equivalent expressions, and CQFramework's
engine-fhir test-runner SKIP list formally records that four of the
five fixture oracles are not treated as authoritative.

This PR aligns the published fixture <output> values with CQL 1.5.2
§5.5.2 uncertainty semantics and with CQFramework's own engine first-
party oracles. No test expressions are changed; only the expected
<output> values are corrected.

Normative framework — CQL 1.5.2 §5.5.2

Specification reference: CQL 1.5.2 §5.5.2 "Determining Difference
and Duration" (https://cql.hl7.org/05-languagesemantics.html#determining-difference-and-duration).

Verbatim passages (unchanged between CQL 1.5.2 and CQL 1.5.3 Errata 2):

"the between operator does not return a strict integer, it returns an
uncertainty."

"Uncertainties may also result when calculating duration for a finer
precision than what is specified in the input values."

"the lower bound of that range is found by determining the duration
between the maximum possible value of the first comparand"

"the upper bound is determined using the minimum possible value of the
first comparand"

Worked example from the same section:

"the 17-day and 44-day examples correspond to using Feb 1 and Feb 28."

"The day-precision operand (Date(2014, 1, 15)) appears to be used as
a fixed point, since its maximum and minimum are the same date."

"When both operands are month precision (the 2012 example), both are
expanded."

Two normative rules follow directly from this text.

Rule R1 — uncertainty predicate. An operand is "uncertain at the
requested precision" iff its stored precision is coarser than the
requested precision. The spec phrase "finer precision than what is
specified in the input values" defines a strict less-than relation:
stored_precision < requested_precision. Same-precision operands
(stored == requested) are not uncertain; the worked example makes this
explicit when it says "its maximum and minimum are the same date".

Rule R2 — uncertainty bounds. The duration-uncertainty interval's
lower bound uses the first comparand's maximum against the second's
minimum; the upper bound uses the first's minimum against the second's
maximum. A non-uncertain operand's maximum and minimum are the same
point.

The five fixture oracles corrected below all derive from a "widest"
interpretation that additionally expands DAY- or MONTH-precision
operands at their own precision to a sub-precision millisecond range.
This contradicts Rule R1 (strict less-than) and the §5.5.2 worked
example (DAY-LHS treated as a fixed point).

A separate, orthogonal spec question appears for the Subtract case
alone, discussed in that case's section below (§Changes/
DateTimeDurationBetweenUncertainSubtract): §5.5.3 "Uncertainty
Propagation" is silent on whether Interval<Integer> - Interval<Integer>
uses endpoint-pairwise or full-interval arithmetic. We adopt
CQFramework's reference-engine choice (endpoint-pairwise) for the
proposed corrected oracle and note the standards-clarification
candidate for future HL7 action.

Changes

tests/cql/CqlTypesTest.xml — DateTimeUncertain (lines 125-133)

Expression (unchanged):

days between DateTime(2015, 2, 10) and DateTime(2015, 3)

Correction:

-        <output>Interval [ 18, 49 ]</output>
+        <output>Interval [ 19, 49 ]</output>

Derivation from §5.5.2 primitives:

  • LHS DateTime(2015, 2, 10) — stored precision = DAY. Requested
    precision = DAY. By Rule R1, DAY < DAY is false; LHS is not
    uncertain; fixed point 2015-02-10.
  • RHS DateTime(2015, 3) — stored precision = MONTH. Requested
    precision = DAY. By Rule R1, MONTH < DAY is true; RHS uncertain
    over its month-boundary window [2015-03-01, 2015-03-31] (March 2015
    has 31 days).
  • By Rule R2:
    • low = whole_days(max(LHS), min(RHS)) = whole_days(2015-02-10, 2015-03-01) = 19.
    • high = whole_days(min(LHS), max(RHS)) = whole_days(2015-02-10, 2015-03-31) = 49.
  • Spec-compelled result: Interval[19, 49].

CQFramework first-party Kotlin oracle (confirming):
cqframework/clinical_quality_language →
engine/src/jvmTest/kotlin/org/opencds/cqf/cql/engine/execution/CqlTypesTest.kt:163-165
asserts

value = results["DateTimeUncertain"]!!.value
assertEquals(19.toCqlInteger(), (value as Interval).start, "DateTimeUncertain")
assertEquals(49.toCqlInteger(), value.end, "DateTimeUncertain")

The fixture's current [18, 49] requires treating the DAY-precision
LHS as a sub-day millisecond range, which contradicts Rule R1 and the
§5.5.2 worked example's "its maximum and minimum are the same date".

tests/cql/CqlDateTimeOperatorsTest.xml — DateTimeDurationBetweenUncertainInterval2 (lines 876-885)

Expression (unchanged):

months between DateTime(2005) and DateTime(2006, 5)

Correction:

-            <output>Interval[ 4, 16 ]</output>
+            <output>Interval[ 5, 16 ]</output>

Derivation from §5.5.2 primitives:

  • LHS DateTime(2005) — stored YEAR; requested MONTH. By R1,
    YEAR < MONTH is true; LHS uncertain over
    [2005-01-01, 2005-12-01].
  • RHS DateTime(2006, 5) — stored MONTH; requested MONTH. By R1,
    MONTH < MONTH is false; RHS not uncertain; fixed point
    2006-05-01.
  • By R2:
    • low = whole_months(2005-12-01, 2006-05-01) = 5.
    • high = whole_months(2005-01-01, 2006-05-01) = 16.
  • Spec-compelled result: Interval[5, 16].

CQFramework Kotlin oracle (confirming, semantically equivalent
expression):
cqframework/clinical_quality_language →
engine/src/jvmTest/kotlin/.../CqlDateTimeOperatorsTest.kt:45-48
asserts .start = 5, .end = 16 on
DateTimeDurationBetweenUncertainInterval2.

tests/cql/CqlDateTimeOperatorsTest.xml — DateTimeDurationBetweenUncertainAdd (lines 886-896)

Expression (unchanged):

(days between DateTime(2014, 1, 15) and DateTime(2014, 2))
    + (days between DateTime(2014, 1, 15) and DateTime(2014, 2))

Correction:

-            <output>Interval[ 32, 88 ]</output>
+            <output>Interval[ 34, 88 ]</output>

Derivation:

  • Inner days between DateTime(2014, 1, 15) and DateTime(2014, 2) =
    Interval[17, 44] — this is the explicit §5.5.2 worked example (using
    Feb 1 and Feb 28 as the RHS's uncertainty window).
  • Outer Interval[17, 44] + Interval[17, 44]. For non-null, non-
    negative inputs Add is commutative, so endpoint-pairwise [a+c, b+d]
    and full-interval arithmetic [low, high] both give [17+17, 44+44] = [34, 88]. No §5.5.3 ambiguity on this input.
  • Spec-compelled result: Interval[34, 88].

CQFramework Kotlin oracle (confirming, semantically equivalent
expression):
cqframework/clinical_quality_language →
engine/src/jvmTest/kotlin/.../CqlDateTimeOperatorsTest.kt:52-55
asserts .start = 34, .end = 88 on
DateTimeDurationBetweenUncertainAdd.

tests/cql/CqlDateTimeOperatorsTest.xml — DateTimeDurationBetweenUncertainSubtract (lines 897-907)

Expression (unchanged):

(days between DateTime(2014, 1, 15) and DateTime(2014, 2))
    - (months between DateTime(2005) and DateTime(2006, 5))

Correction (adopting CQFramework's endpoint-pairwise §5.5.3 reading as
the de-facto reference-engine behavior; see note below):

-            <output>Interval[ 0, 40 ]</output>
+            <output>Interval[ 12, 28 ]</output>

Derivation:

  • Inner LHS = Interval[17, 44] (§5.5.2 worked example).
  • Inner RHS = Interval[5, 16] (derivation per
    DateTimeDurationBetweenUncertainInterval2 above).
  • Outer Interval[17, 44] - Interval[5, 16]:
    • CQFramework endpoint-pairwise [start-start, end-end]:
      [17-5, 44-16] = [12, 28]. Source:
      cqframework/clinical_quality_language →
      engine/src/commonMain/kotlin/.../elm/executing/SubtractEvaluator.kt:140-148: 
      } else if (left is Interval && right is Interval) {
    return Interval(
        subtract(leftInterval.start, rightInterval.start, state),
        true,
        subtract(leftInterval.end, rightInterval.end, state),
        true,
    )
}
    • Full-interval arithmetic [low-high, high-low]:
      [17-16, 44-5] = [1, 39].

Normative note on §5.5.3 ambiguity: CQL 1.5.2 §5.5.3 "Uncertainty
Propagation" is silent on which specific interval-arithmetic rule
applies to Interval<Integer> - Interval<Integer>. CQFramework's
reference engine applies endpoint-pairwise; classical numerical-
analysis convention applies full-interval. The two rules coincide on
Add (commutative) and Multiply (monotonic on non-negative inputs) but
differ on Subtract. This PR adopts CQFramework's endpoint-pairwise
reading because it is the only existing first-party reference-engine
behavior for this operator shape. If HL7 subsequently clarifies §5.5.3
in favor of full-interval arithmetic, the corrected value here would
become Interval[1, 39]; we recommend this fixture value be revisited
if the §5.5.3 ambiguity is resolved.

Crucially, the fixture's current Interval[0, 40] matches neither
§5.5.3 reading applied to spec-correct §5.5.2 inner intervals. It is
only consistent with endpoint-pairwise Subtract applied to the "widest-
§15" (R1-violating) inner intervals [16, 44] - [4, 16] = [16-16, 44-4] = [0, 40], which fails R1 at both inner operators. The
§5.5.3 underdetermination does not rehabilitate the fixture value.

CQFramework Kotlin oracle (under endpoint-pairwise reading,
confirming):
cqframework/clinical_quality_language →
engine/src/jvmTest/kotlin/.../CqlDateTimeOperatorsTest.kt:57-60
asserts .start = 12, .end = 28 on
DateTimeDurationBetweenUncertainSubtract.

tests/cql/CqlDateTimeOperatorsTest.xml — DateTimeDurationBetweenUncertainMultiply (lines 908-918)

Expression (unchanged):

(days between DateTime(2014, 1, 15) and DateTime(2014, 2))
    * (days between DateTime(2014, 1, 15) and DateTime(2014, 2))

Correction:

-            <output>Interval[ 256, 1936 ]</output>
+            <output>Interval[ 289, 1936 ]</output>

Derivation:

  • Inner Interval[17, 44] (§5.5.2 worked example).
  • Outer Interval[17, 44] * Interval[17, 44]. For non-negative,
    non-null intervals Multiply is monotonic on both endpoints:
    endpoint-pairwise [a*c, b*d] and full-interval
    [min(a*c, a*d, b*c, b*d), max(…)] give the same result
    [17*17, 44*44] = [289, 1936]. No §5.5.3 ambiguity on this input.
  • Spec-compelled result: Interval[289, 1936].

CQFramework Kotlin oracle (confirming, semantically equivalent
expression):
cqframework/clinical_quality_language →
engine/src/jvmTest/kotlin/.../CqlDateTimeOperatorsTest.kt:62-65
asserts .start = 289, .end = 1936 on
DateTimeDurationBetweenUncertainMultiply.

Cross-engine cross-check

Case Fixture oracle (current) Spec-compelled (§5.5.2) CQFramework Kotlin oracle Proposed PR value
DateTimeUncertain [18, 49] [19, 49] [19, 49] [19, 49]
DateTimeDurationBetweenUncertainInterval2 [4, 16] [5, 16] [5, 16] [5, 16]
DateTimeDurationBetweenUncertainAdd [32, 88] [34, 88] [34, 88] [34, 88]
DateTimeDurationBetweenUncertainSubtract [0, 40] [12, 28] (CQF §5.5.3 reading) / [1, 39] (full-interval §5.5.3 reading) [12, 28] [12, 28]
DateTimeDurationBetweenUncertainMultiply [256, 1936] [289, 1936] [289, 1936] [289, 1936]

All five proposed values match the §5.5.2-derived spec-compelled value.

Further corroboration — CQFramework engine-fhir SKIP list

CQFramework's engine-fhir test runner explicitly records that the
fixture oracles for four of the five cases above are not treated as
authoritative. Source:
cqframework/clinical_quality_language →
engine-fhir/src/jvmTest/kotlin/org/hl7/fhirpath/CQLOperationsR4Test.kt:72-99:

var SKIP: MutableSet<String> =
    Sets.newHashSet(
        ...
        "cql/CqlDateTimeOperatorsTest/Uncertainty tests/DateTimeDurationBetweenUncertainAdd",
        "cql/CqlDateTimeOperatorsTest/Uncertainty tests/DateTimeDurationBetweenUncertainInterval",
        "cql/CqlDateTimeOperatorsTest/Uncertainty tests/DateTimeDurationBetweenUncertainInterval2",
        "cql/CqlDateTimeOperatorsTest/Uncertainty tests/DateTimeDurationBetweenUncertainMultiply",
        "cql/CqlDateTimeOperatorsTest/Uncertainty tests/DateTimeDurationBetweenUncertainSubtract",
        ...
    )

DateTimeUncertain is not on this SKIP list only because it lives in
CqlTypesTest.xml (not CqlDateTimeOperatorsTest.xml); the identical
fixture-author TODO comment (next section) is a fourth-source author
acknowledgement equivalent to a SKIP-list entry.

Fixture-author's own acknowledgement — identical TODO on all five

Each of the five test blocks carries this verbatim author comment:

<!-- TODO: How to handle the fact the question is resulting in an
undertainty interval and that CQL/ELM seem to provide no direct way of
selecting the same value, conceptually an implementation internal;
currently Equivalent() results in null from comparing with an Interval. -->

This PR optionally allows upstream maintainers to either (a) clear
these TODO comments once the corrected oracles land, or (b) rewrite
them to point to the §5.5.2 derivation above. We do not prescribe
either choice; this PR leaves the comments in place.

CQFramework internal source — isUncertain strict-less-than

The isUncertain predicate in the CQFramework runtime implements
exactly the strict-less-than rule derived from §5.5.2 above. Source:
cqframework/clinical_quality_language →
engine/src/commonMain/kotlin/org/opencds/cqf/cql/engine/runtime/DateTime.kt:197-205:

override fun isUncertain(p: Precision): kotlin.Boolean {
    var precision = p
    if (precision == Precision.WEEK) {
        precision = Precision.DAY
    }
    return this.precision!!.toDateTimeIndex() < precision.toDateTimeIndex()
}

This confirms that CQFramework's reference engine's own author
implements R1 as strict less-than, and that the engine's observable
behavior on the five cases follows from this implementation.

Downstream impact

  • CQFramework's engine-fhir CQLOperationsR4Test SKIP list
    (quoted above) currently excludes four of the five cases. Once these
    fixture corrections merge, the SKIP-list entries can be removed in a
    follow-up PR against cqframework/clinical_quality_language —
    the FHIRPath runner's Equivalent() comparison will then produce a
    non-null result matching the engine's Interval output.
  • Implementers aligned with CQFramework's Kotlin first-party oracle
    (CQFramework itself; Cobalt / cql-wasm) will see these fixtures flip
    from FAIL to PASS without any implementation change.
  • Implementers that previously implemented the "widest-§15" rule to
    match the current fixture values will need to re-examine their
    isUncertain semantics against §5.5.2. The direction of the
    correction aligns the published fixtures with the specification text
    and with the first-party reference engine.

Additional recommendation (non-blocking)

The CQL §5.5.3 "Uncertainty Propagation" ambiguity on interval-
arithmetic semantics (noted above under the Subtract case) warrants a
future HL7 specification clarification: should Interval<Integer> OP Interval<Integer> use endpoint-pairwise [start OP start, end OP end]
or full-interval arithmetic? CQFramework's reference engine uses
endpoint-pairwise; numerical-analysis convention uses full-interval.
The two coincide on Add and Multiply but differ on Subtract. This PR
does not require the clarification to land first; it adopts CQF's
endpoint-pairwise reading for the Subtract case to maintain consistency
with the reference engine.

If HL7 subsequently clarifies §5.5.3 in favor of full-interval
arithmetic, the corrected DateTimeDurationBetweenUncertainSubtract
oracle in this PR would move from Interval[12, 28] to
Interval[1, 39]. Either way, the current Interval[0, 40] is
incorrect under any reading of §5.5.3 applied to §5.5.2-correct inner
intervals.

Testing

No test code changes are required for this PR; only <output> values
in the XML fixtures are modified. Implementations that previously
asserted against the old fixture values will need to re-run the
affected tests after this PR merges. CQFramework's own Kotlin unit
tests at engine/src/jvmTest/kotlin/.../CqlDateTimeOperatorsTest.kt
and .../CqlTypesTest.kt are unaffected; their expected values match
the proposed corrected fixture oracles.

Related upstream work (non-blocking)

A follow-up PR against cqframework/clinical_quality_language would
remove the four SKIP-list entries at
engine-fhir/src/jvmTest/kotlin/org/hl7/fhirpath/CQLOperationsR4Test.kt:79-83
once this fixture PR merges.

Zachary Whitley
Correct uncertainty-arithmetic test fixtures per CQL 1.5.2 §5.5.2
dad744c 
Five test-fixture <output> values currently encode a 'widest-§15'
interpretation inconsistent with the strict-less-than rule in
CQL 1.5.2 §5.5.2 'Determining Difference and Duration'.

Affected cases:
- CqlDateTimeOperatorsTest::DateTimeDurationBetweenUncertainAdd
- CqlDateTimeOperatorsTest::DateTimeDurationBetweenUncertainInterval2
- CqlDateTimeOperatorsTest::DateTimeDurationBetweenUncertainMultiply
- CqlDateTimeOperatorsTest::DateTimeDurationBetweenUncertainSubtract
- CqlTypesTest::DateTime::DateTimeUncertain

See PR description for CQL 1.5.2 §5.5.2 worked-example derivation,
§5.5.3 Subtract-case reading disclosure, and before/after
differential evidence.

Corroboration: CQF first-party Kotlin reference engine, CQF
engine-fhir SKIP list, fixture-author TODOs in each affected
block.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@zacharywhitley