Skip to content

[WIP] feat(history): add Junco verified history.match builtin (opt-in feature) - #854

Open
Lef Ioannidis (elefthei) wants to merge 11 commits into
microsoft:fix/number-exact-orderingfrom
elefthei:feature/junco
Open

Lef Ioannidis (elefthei) wants to merge 11 commits into
microsoft:fix/number-exact-orderingfrom
elefthei:feature/junco

Conversation

@elefthei

@elefthei Lef Ioannidis (elefthei) commented Oct 21, 2026 •

Copy link
Copy Markdown

Currently blocked on:

Adds history.match(policy, history, request) -> bool behind the opt-in history feature, which is not part of default or full-opa. A policy pairs a request pattern with a regular expression over past events. The builtin returns whether history (oldest event first) belongs to the pair's language for request. It works in the interpreter and the RVM, including no_std builds. Syntax and semantics are in docs/builtins.md, section "History/request policies".

Specification

HistoryMatchState::evaluate (src/builtins/history/mod.rs) is verified against an independent spec in src/verify/history_specs.rs:

result matches Ok(answer) ==>
    answer == policy_lang(value_view(*policy), value_view(*request), history_view(*history))

Expanding pair_lang gives:

policy_lang(policy, req, history) =
    exists|head: Head, tail: Regex<SrcTest>|
        decodes_policy(policy, head, tail)
        && exists|theta: Subst|
               head_match(head, req, theta)
               && theta.dom() == head_vars(head)
               && lang(|test, event| src_test_holds(theta, test, event), tail, history)
  • decodes_policy decodes the JSON policy object.
  • head_match binds the request variables (theta).
  • lang is ordinary regular-language membership over event sequences (history_specs.rs:112).
  • src_test_holds evaluates eq/neq/and/or on a single event under theta.

An Err result has no Boolean meaning.

Implementation

  • Lazy symbolic derivatives run over a hash-consed bump store (hashconsius, verus-bumpalo, both pinned to git revisions). Each pair's graph is capped at 100 KiB. Timer and memory limits are enforced through ObservationWorld.
  • The interpreter and the RVM each keep a HistoryMatchState. History errors are fatal (VmError::HistoryMatchFailure, HistoryMatchValidationError) and never turn into undefined.
  • Numbers: Data::Number holds a canonical key {rank, units}. Two keys are equal exactly when fix(number): make equality and ordering exact #846's NumberView::cmp_values returns Equal (src/verify/value_numeric.rs, lemma_numeric_of_eq). Array path steps convert floats to indexes with float_array_index, which is proved equal to float_index_spec (Some(i) iff the float is exactly i and 0 <= i <= u64::MAX). Nothing saturates, so 2^64 as a float never selects u64::MAX.
  • The Lean spec (verification/aeneas/history/lean/RegorusHistory/Spec/Data.lean) follows fix(number): make equality and ordering exact #846 for NaN: every NaN keeps its sign and payload, negative NaNs order below -inf, and positive NaNs order above +inf.
  • The Lean/Aeneas extraction scaffold is in verification/aeneas/history (cargo xtask extract-history-lean).

Verification

Verus 0.2026.10.04.426d8b0 with vstd 0.0.0-2026-09-06-0133:

Command Verified Errors
cargo verus verify --locked 109 0
cargo verus verify --locked --features history 1132 2
cargo verus verify --locked --no-default-features --features history,arc,std 1121 2

Both errors are the inherited PartialEq postcondition (obeys_eq_spec() ==> r == self.eq_spec(other)) on Term::eq and Primitive::eq in src/builtins/history/kernel.rs. They come from verus-lang/verus#3298, a declaration-order bug: Verus can check the impl before the axioms of the spec helpers it inherits. The minimized reproducer is verification/verus/inherited_ensures_order.rs. Every other function verifies.

Runtime (Rust 1.95.0):

  • cargo test --locked --features history: 572 passed.
  • cargo test --locked --features history,cache: 572 passed.
  • cargo test --locked --features history,rvm --test history_match: 6 passed.
  • src/builtins/history/tests.rs::float_path_steps_index_arrays_exactly covers the new float index path: 1.0, 0.0, and -0.0 select elements; 1.5, -1.0, subnormals, 2^64, and 1e300 select nothing.
  • cargo xtask ci-debug and cargo xtask test-no-std (thumbv7m-none-eabi) pass.

@elefthei Lef Ioannidis (elefthei) changed the title [WIP] feat(history): add verified history.match builtin (opt-in feature) [WIP] feat(history): add Junco verified history.match builtin (opt-in feature) Oct 22, 2026
Comment thread Cargo.toml Outdated
hashbrown = { version = "0.17", default-features = false, features = ["default-hasher"], optional = true }
lazy_static = { version = "1.4.0", default-features = false }
thiserror = { version = "2.0", default-features = false }
verus-bumpalo = { git = "https://github.com/elefthei/bumpus", rev = "36c8e8723d767934a502fc1d9cea0ba49cff54ab", optional = true }

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lef Ioannidis (@elefthei) This will likely cause problems in internal builds since it is not under microsoft org and can be used (in theory) for supply chain attacks.

Comment thread docs/builtins.md Outdated
"action": {"value": "release"},
"resource": {"var": "r"},
}},
"history": {"concat": [

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice if we can devise a cleaner syntax for this.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The new Git-only dependencies prevent crates.io publication of Regorus even when the history feature is disabled.

1 open issue
What changed in this PR

Adds the opt-in history.match builtin to Regorus for matching prior events against request-bound policies. It builds on #846’s exact numeric equality and integrates with both the interpreter and RVM.

Changes:

  • Adds matcher state, caching, resource-limit handling, and binding feature forwarding.
  • Introduces formal specifications, proof support, semantic tests, and performance benchmarks.
  • Documents policy syntax and verification boundaries, and adds history-enabled Verus CI checks.
File Description
verification/​verus/​inherited_ensures_order.rs Adds an upstream Verus bug reproducer.
tests/​history_match.rs Tests evaluator parity, errors, and serialization.
src/​verify/​value_specs.rs Defines mathematical value views.
src/​verify/​value_numeric.rs Models exact numeric keys and array indexes.
src/​verify/​value_assumptions.rs Specifies trusted value operations.
src/​verify/​mod.rs Registers history verification modules.
src/​verify/​history_examples.rs Connects example policies to evaluator contracts.
src/​verify/​history_assumptions.rs Defines native and storage verification boundaries.
src/​value/​set/​mod.rs Adds set views and structural proofs.
src/​value/​object/​mod.rs Adds object views and structural proofs.
src/​value/​mod.rs Enables verification of the Value representation.
src/​value/​array/​mod.rs Adds array views and structural proofs.
src/​tests/​interpreter/​mod.rs Exposes shared timer test support.
src/​rvm/​vm/​rules.rs Propagates fatal matcher errors.
src/​rvm/​vm/​machine.rs Owns and resets matcher state.
src/​rvm/​vm/​functions.rs Dispatches contextual matching and maps errors.
src/​rvm/​vm/​execution.rs Clarifies fatal-error handling comments.
src/​rvm/​vm/​errors.rs Adds matcher-specific error variants.
src/​lib.rs Enables allocator APIs for verification builds.
src/​interpreter.rs Integrates matcher state and preserves fatal errors.
src/​builtins/​mod.rs Registers and dispatches the builtin.
src/​builtins/​history/​tests.rs Tests semantics, limits, cleanup, and numeric cases.
src/​builtins/​history/​mod.rs Implements matcher state and pair caching.
src/​builtins/​history/​error.rs Defines typed matcher errors.
docs/​builtins.md Documents syntax, semantics, caching, and proofs.
Cargo.toml Adds the feature, dependencies, and benchmark target.
Cargo.lock Locks new matcher dependencies.
bindings/​wasm/​Cargo.toml Forwards the opt-in history feature.
bindings/​wasm/​Cargo.lock Locks matcher dependencies for WASM.
bindings/​ruby/​ext/​regorusrb/​Cargo.toml Forwards the opt-in history feature.
bindings/​ruby/​Cargo.lock Locks matcher dependencies for Ruby.
bindings/​python/​Cargo.toml Forwards the opt-in history feature.
bindings/​python/​Cargo.lock Locks matcher dependencies for Python.
bindings/​java/​Cargo.toml Forwards the opt-in history feature.
bindings/​java/​Cargo.lock Locks matcher dependencies for Java.
bindings/​ffi/​Cargo.toml Forwards the opt-in history feature.
bindings/​ffi/​Cargo.lock Locks matcher dependencies for FFI.
benches/​history_match.rs Measures cold, cached, and incremental matching.
.github/​workflows/​verus.yml Updates Verus and adds history verification profiles.

Review effort: Balanced

Comment thread Cargo.toml Outdated
Comment on lines +114 to +115
verus-bumpalo = { git = "https://github.com/elefthei/bumpus", rev = "36c8e8723d767934a502fc1d9cea0ba49cff54ab", optional = true }
hashconsius = { git = "https://github.com/elefthei/hashconsius", rev = "58ab05c67c34201fa25f27c443cb5b45bc3a6c00", default-features = false, optional = true }
- history: bind Vec::as_mut_slice() to a local before get_mut /
  sort_unstable (clippy::unnecessary_as_slice). Verus still needs the
  explicit slice: Vec auto-deref through DerefMut trips its
  trait-conflict checker.
- char::from_digit instead of deprecated core::char::from_digit.
- question_mark in destructuring planner; values_mut in interpreter.
- Drop redundant format-arg borrows (src, benches, tests).
- Move ast reference_ownership_tests to end of file
  (items_after_test_module).
- is_fatal_builtin_error: avoid bool-literal guard without history.
- Remove redundant ToString import in azure_policy alias tests.
- Use existing module aliases (model, norm, source_model, language,
  progress_model) instead of fully-qualified ghost paths.
- Drop the cfg_attr wrapper on error.rs verus_spec attributes.
- Fix stale value_assumptions module doc.
- Delete 12 unreferenced ghost items (11 proof fns, 1 spec fn).
- Rename generic dedup helpers (test_structural_eq -> structural_eq,
  test_lemma_dedup_* -> lemma_dedup_*) and
  test_lemma_multiset_membership -> lemma_multiset_members; the
  regex/transition multiset wrappers now delegate to it.

No executable code, contract strength, or trusted boundary changes.
Verus: 1404 verified (default) / 1393 (no-default), same 2
pre-existing PartialEq errors in kernel.rs.
A freshly compiled history.match pair built its symbolic derivatives,
atom sets and event-class successors lazily on the first events reaching
each residual, roughly doubling first-decision latency on lock policies.

With `cache`, HistoryMatchState::evaluate now calls the new verified
CompiledPair::precompile right after compile_pair: a depth-first closure
from the root, bounded by 64 residual states and the 256-entry successor
cache (so nothing it publishes is evicted). Transitions whose 2^atoms
classes exceed the remaining budget stay lazy. Exploration that hits
GraphLimit falls back to a freshly compiled lazy pair; the retry after a
retained GraphLimit skips precompilation.

The successor lookup/publication tail of instantiate is extracted into
CompiledPair::successor so lazy and eager entries share one proof, and
the atom-set lookup is shared by atom_set. No new external_body.
Move the owned DSL, flat validation/lowering, interning/history storage and
the numeric compiler/evaluator into src/junco-core, src/history-syntax,
src/history-store and src/history-semantics. Regorus keeps the native Value
parser, VM dispatch/limit adapters and the Program-keyed cache.

hashconsius and verus-bumpalo come from GitHub (elefthei/hashconsius@d7c1748,
elefthei/bumpus@888a698), pinned to vstd 0.0.0-2026-10-04-0306 to match
Verus 0.2026.10.04.
…store

Remove the tagged JSON policy grammar and the native Value parser
(src/languages/history_table/source.rs). history_table::compile now borrows
&[Rule<SourceEvent<'_>>] built with the re-exported Rust DSL and copies exact
string constants into the program; JSON-shape errors become
InvalidRule { index, source: CompileError } and InvalidProgram. Compiled v8
JSON/binary Programs are unchanged.

Back every history_table::HistoryStore wrapper with
history_store::store::HistoryStore::global(): wrappers share byte identities
and accept each other's handles, keeping only their Program-keyed table cache.
RuleRuntime::new takes the store Arc and resume/advance_cursor key cursors by
the locked record's immutable StoredHistory::id(). Typed Store<E> views keep
their own codebooks but draw IDs from the shared allocators.

Port tests, benches and docs to the DSL.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants