Skip to content

std's calls of a closure that have no mapping - #473

Merged
kixelated merged 40 commits into
mainfrom
quest/m3/ts-closure-calls
Nov 3, 2026
Merged

kixelated merged 40 commits into
mainfrom
quest/m3/ts-closure-calls

Conversation

@kixelated

@kixelated kixelated commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Finishes quest/m3/ts-closure-calls.md. Builds on #463, now on main, for rt.iterTryRfold.

Problem

Std functions that take a closure and had no mapping were refused as "unsupported call", whatever the closure held. #438 found them while probing drop order: Iterator::try_fold, try_for_each, partition, and scan, iter::repeat_with, slice::sort_by_cached_key, LazyCell::new, Cell::update, and RefCell::replace_with. scan and replace_with of a value JS copies were refused earlier still, as a &mut their mapping would copy.

Approach

Each maps to the runtime, calls its closure as std's does, and drops what the closure owns where std's does:

  • try_fold and try_for_each share one core with try_rfold (tryFold), reading the iterator as std's try_fold does (folded()) and breaking by an Option's None (rt.None where that is the None), or by Err for a Result or ControlFlow. Each drops its closure as it returns, not the iterator, which it borrows.
  • partition makes two of B's defaults and extends each by the tuple Extend's rt.Part, so it takes every collection a tuple's collect does, not only Vec. It folds the iterator by value, so the closure drops as the fold ends, before what the iterator still holds.
  • scan is an rt.Iter holding its state, lent as an rt.Place where JS copies it. Its next reads on past a None, as std's does, while a fold stops there, as std's try_fold does. Its iterator, closure, and state drop in that order, as std's fields do.
  • repeat_with is an rt.iterFrom of its closure, with std's (usize::MAX, None) hint, dropping the closure with it.
  • sort_by_cached_key makes each key in order, drops the closure (std moves it into the map that makes the keys), sorts (key, index) pairs with the runtime's port of sort_unstable by the pair's lt, size, Freeze, and Copy, permutes the slice as std's does, then drops the keys. The pair's index is a usize: this nightly's std picks a u32 only where it is smaller, which it never is on wasm32. lang's sort case, with a key that fits the network sort alone but not paired, caught the first version picking a u16.
  • LazyCell::new and LazyLock::new make an rt.LazyCell/rt.LazyLock, whose drop already dropped a function that never ran. A closure that Fn or FnMut call becomes rt.once of it, so force's call through FnOnce drops it after, as rustc's shim does.
  • Cell::update calls its function on the value, which a translated function copies before it changes or keeps one JS shares; RefCell::replace_with on the value, or its place where JS copies it, exclusively borrowed until the replacement is in. Both take their function through FnOnce (discards), so an Fn closure becomes rt.once of it.

scan now also takes items whose None is rt.None, so reject's scanned and scanned_unit translate. They move to lang's option_items as differential cases.

Tests (differential against native wasm32):

  • task's new called set, 12 scenarios, with closures that own guards and items that log their drops:
    • try_fold by None, then read on; by Err through map and chain; by Break through rev and filter.
    • try_for_each by each try type.
    • partition of guards.
    • repeat_with through take, and dropped mid-iteration.
    • scan that stops at its first None and is then read on.
    • sort_by_cached_key with keys that drop.
    • LazyCell forced, and dropped unforced.
    • Cell::update and RefCell::replace_with, with closures inferred FnOnce and with let-bound ones that Fn calls.
    • Each over an empty iterator or slice, and a panic mid-try_fold with the log so far.
  • lang's ends folded picks gain try_fold and try_for_each (Option, Result, ControlFlow), partition, and scan over Run, Counted, and a crate's own Overrides. sorts gains sort_by_cached_key at every length and pad type, logging each comparison. option_items gains scan of Option<Option<u8>> and Option<()>.

Mutation checks: each runtime piece, mutated in the translated output and rerun against task and lang, is killed:

  • try_fold: stepping from the back, never releasing its closure on a break or at the end, and releasing it before the result is made;
  • repeat_with: never dropping its closure, and a hint other than (usize::MAX, None);
  • scan: its state dropped before its iterator, reading on past its fold's None, and the inner iterator's hint;
  • partition: the sides swapped, its closure never released or released after the iterator;
  • sort_by_cached_key: the closure dropped late or kept on a short slice, the key's size in place of the pair's, the permutation not chasing its cycles, and the keys dropped in reverse or before the permutation;
  • RefCell::replace_with: not borrowing the cell exclusively, and reading the old value before the call.

Two survive, equivalent for these shapes: folded(true) in try_fold, which differs only for a by-index zip's fold, and scan's folded reading through itself. A third, Cell::update without its copy, survived every case, a mut parameter and a let-bound copy included, since a translated function copies what JS shares before it changes or keeps it. That copy is deleted.

Impact

  • Public API: the translator's output only. These calls translate instead of being refused.
  • New runtime exports: rt.iterTryFold, rt.iterTryForEach, rt.iterPartition, rt.iterScan, rt.iterRepeatWith, rt.sliceSortByCachedKey, rt.cellUpdate, rt.refCellReplaceWith. Nothing existing changes signature, so no MIGRATE.md entry.

Alternatives

  • partition of Vec alone. The tuple Extend parts already cover every collection collect does, at no extra code.
  • sort_by_cached_key picking among u8, u16, u32, and usize indexes, as std did before. This nightly's picks only u32 or usize, and lang's differential proved the older choice wrong.

Follow-ups

None.

Review

quest review (codex gpt-6.1-sol, medium), three rounds, all clean: at 99af98109fa49a38cd1931286af2e665db03bbf1; at f2b3bf8ff7816fa79d0139c40622e82268e03a21, after merging #463's requeued head with #490's rt.iterBox; and at 93536a5f97799e67ff84611d9530bbf08c87da6a, the head being merged. That head follows the fixer's merge of main through #495, which kept both PRs' additions to call.rs's lends list, and a clean merge of #496, #498, and #503. No snapshot changed, and just ci passed at this head.

🤖 Generated with Claude Code

kixelated and others added 30 commits November 2, 2026 15:42
…, count, and last

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lds generic in more than moves

A chain lends its closure to its first half's own fold, which dropped it there.
The crate's fold, translated for one accumulator and closure, now refuses a body
that tells either apart: a call or constant generic over them, or a drop of the
accumulator, in it or its closures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…p of the accumulator

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ure apart through a closure or a type of the crate

A closure is over what it captures, and std's iterators take any accumulator
and closure only outside the crate's own types.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ith, sort_by_cached_key, LazyCell::new, Cell::update, and RefCell::replace_with

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n wasm32

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e closures' drops beside an adaptor's

Snapshots for the last two replace_with and update cases are not yet regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Moves ends' inputs to lang/ends/caller.ts and the fold refusals to reject's iters module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings main's per-module fixture layout through #463's merge of it. The scan cases reject no longer holds move out of reject's iters module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y last of its by-value overrides

Box<I>'s Iterator forwards next, size_hint, nth, last, next_back, nth_back, and len, and folds and counts by next, so a box of the crate's own iterator no longer calls its fold, rfold, or count. lang's ends reads a Box<Overrides> through every consumer and fold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rwards nothing else

A box of a sized iterator, owned or behind a &mut, is an rt.iterBoxed: it forwards next, size_hint, nth, last, next_back, nth_back, and len, as std's Iterator for Box<I> does, and folds, counts, and advances by next. This replaces the last commit's filter of a crate iterator's own methods, which missed a box of an adaptor holding one. lang's ends reads a box of Overrides and of a map of it, and zips reads boxes of each zip, skipped and borrowed. Finishes quest/m3/ts-box-folds.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A translated function already copies a value JS shares before it changes or keeps it, so cellUpdate's own copy could not be observed: the mutant dropping it survived every case, a `mut` parameter and a let-bound copy included. Also regenerates lang's ends and iters snapshots after the last merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kixelated and others added 8 commits November 2, 2026 22:15
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Keeps one box reader, main's rt.iterBox from #490, and deletes this branch's rt.iterBoxed with its own wrapping in into_iter. iterBox already forwards only what std's Box<I> does, so a box of the crate's own iterator, or of an adaptor holding one, never reaches its by-value overrides. ends' boxed Overrides cases and zips' boxed reads stay beside #490's dyns cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#463 landed as 7c41f2b, whose changes this branch already holds: the merged tree is the branch's own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 3, 2026 06:58
@kixelated
kixelated enabled auto-merge November 3, 2026 06:58
@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 999d305f-5fe8-4a08-9a29-064e287b5efe
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.

@kixelated
kixelated added this pull request to the merge queue Nov 3, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to a conflict with the base branch Nov 3, 2026
@moq-bot

moq-bot Bot commented Nov 3, 2026

Copy link
Copy Markdown

quest land watch: the merge queue removed this pull request at 242d714 because it conflicts with main (merge_conflict).

The land fixer works on it in the background, and the watcher re-enqueues it at the back of the queue once it is fixed or judged flaky, or hands it back here with what it needs.

(written by quest land watch)

@moq-bot

moq-bot Bot commented Nov 3, 2026

Copy link
Copy Markdown

codex / Codex land fixer: GitHub refused to re-enqueue 3b93c48b30135a1a005ca20c1f0282145ced7384: GitHub GraphQL error: Pull request has merge conflicts and Pull request not in mergeable state. Recommendation: run /quest-iterate before picking it again.

22 +     if let Some(g) = Some(guard(7)) {
   |

warning: `task-fixture` (lib) generated 12 warnings (run `cargo fix --lib -p task-fixture` to apply 1 suggestion)
    Finished `differential` profile [optimized + debuginfo] target(s) in 7.90s
test share has been running for over 60 seconds
test stock has been running for over 60 seconds
test crates ... ok
test task has been running for over 60 seconds
    Blocking waiting for file lock on build directory
   Compiling rand_core v0.10.1
   Compiling cfg-if v1.0.5
   Compiling getrandom v0.4.3
   Compiling allocator-api2 v0.2.21
   Compiling equivalent v1.0.3
   Compiling foldhash v0.2.0
   Compiling chacha20 v0.10.2
   Compiling hashbrown v0.17.1
   Compiling rand v0.10.3
   Compiling hashlink v0.12.2
   Compiling stock-fixture v0.0.0 (/work/ts/fixtures/stock)
test task ... ok
    Finished `differential` profile [optimized + debuginfo] target(s) in 4.57s
   Compiling share-fixture v0.0.0 (/work/ts/fixtures/share)
    Finished `differential` profile [optimized + debuginfo] target(s) in 4.56s
   Compiling share-fixture v0.0.0 (/work/ts/fixtures/share)
    Finished `differential` profile [optimized + debuginfo] target(s) in 0.25s
   Compiling share-fixture v0.0.0 (/work/ts/fixtures/share)
    Finished `differential` profile [optimized + debuginfo] target(s) in 0.25s
test share ... ok
test stock ... ok

test result: ok. 81 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 207.78s

     Running tests/runtime.rs (target/debug/build/rust2-ts/72f2810aa5d2a132/out/runtime-72f2810aa5d2a132)

running 10 tests
test unicode ... ok
test pointers_apart ... ok
test spawn ... ok
test once_recursive ... ok
test bytes_mut_reserve ... ok
test local_lazy ... ok
test retain ... ok
test nan_payloads ... ok
test fma_allowance ... ok
test btree_script ... ok

test result: ok. 10 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.88s


codex
All 99 tests pass, including the changed iterator, sorting, and task fixtures. I’m committing the merge resolution on `pr-473` and writing the uncommitted result file.

exec
/bin/bash -lc "git diff --check && git diff --cached --check && git -c core.hooksPath=/dev/null commit -m 'Merge base into PR 473, preserving both lending match additions' && git rev-parse HEAD && git merge-base --is-ancestor 242d714cadcbe9f2940124ee909f6fc2eebd1a5c HEAD && git merge-base --is-ancestor base HEAD" in /work
 succeeded in 687ms:
[pr-473 3b93c48b] Merge base into PR 473, preserving both lending match additions
3b93c48b30135a1a005ca20c1f0282145ced7384

exec
/bin/bash -lc "python3 - <<'PY'
import json
from pathlib import Path
result = [{
    'number': 473,
    'action': 'fixed',
    'head': '3b93c48b30135a1a005ca20c1f0282145ced7384',
    'reason': 'Merged base and resolved the call.rs conflict by preserving both sides; passed in ts/: cargo build --locked -p rust2-ts; cargo test --locked -p rust2-ts (99 tests); cargo fmt --all --check; tsc -p runtime, using pinned nightly-2026-10-15.',
    'failure': 'Merge conflict in ts/src/lower/mir/call.rs',
    'blame': True
}]
Path('quest-land-result.json').write_text(json.dumps(result, indent=2) + '\\n')
PY
 git status --short && git branch --show-current && git log -1 --format='%H %P' && cat quest-land-result.json" in /work
 succeeded in 0ms:
?? quest-land-result.json
pr-473
3b93c48b30135a1a005ca20c1f0282145ced7384 242d714cadcbe9f2940124ee909f6fc2eebd1a5c 5294768bf7f16e6c3f53508129415eb1980ac391
[
  {
    "number": 473,
    "action": "fixed",
    "head": "3b93c48b30135a1a005ca20c1f0282145ced7384",
    "reason": "Merged base and resolved the call.rs conflict by preserving both sides; passed in ts/: cargo build --locked -p rust2-ts; cargo test --locked -p rust2-ts (99 tests); cargo fmt --all --check; tsc -p runtime, using pinned nightly-2026-10-15.",
    "failure": "Merge conflict in ts/src/lower/mir/call.rs",
    "blame": true
  }
]

codex
Fixed #473 at `3b93c48b30135a1a005ca20c1f0282145ced7384`, preserving both sides of the merge conflict.

Build, all 99 tests, formatting, and runtime TypeScript checks pass. [quest-land-result.json](/work/quest-land-result.json) is written and uncommitted.
tokens used
59,836
Fixed #473 at `3b93c48b30135a1a005ca20c1f0282145ced7384`, preserving both sides of the merge conflict.

Build, all 99 tests, formatting, and runtime TypeScript checks pass. [quest-land-result.json](/work/quest-land-result.json) is written and uncommitted.
fixer result written

(written by codex / Codex)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated added this pull request to the merge queue Nov 3, 2026
Merged via the queue into main with commit 3732b0a Nov 3, 2026
4 of 5 checks passed
@kixelated
kixelated deleted the quest/m3/ts-closure-calls branch November 3, 2026 08:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant