From 82b27c53abcc8b70a5f566399a4423f726a1cfb6 Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Tue, 3 Nov 2026 23:03:35 -0800 Subject: [PATCH 1/2] chore(quest): plan rs wrappers, the rs rename, and Python call costs Co-Authored-By: Claude Opus 5.5 --- quest/m3/README.md | 8 ++-- quest/m3/ffi-python-loop-lookup.md | 22 ++++++++++ quest/m3/ffi-python-methods.md | 25 ++++++++++++ quest/m3/ts-rs-rename.md | 21 ++++++++++ quest/m3/ts-slice-object.md | 48 +++++++++++++++------- quest/m3/ts-slice-of-one.md | 29 -------------- quest/m3/ts-string-capacity.md | 64 ++++++++++++++---------------- 7 files changed, 137 insertions(+), 80 deletions(-) create mode 100644 quest/m3/ffi-python-loop-lookup.md create mode 100644 quest/m3/ffi-python-methods.md create mode 100644 quest/m3/ts-rs-rename.md delete mode 100644 quest/m3/ts-slice-of-one.md diff --git a/quest/m3/README.md b/quest/m3/README.md index 1c1602c79..6d2241e7c 100644 --- a/quest/m3/README.md +++ b/quest/m3/README.md @@ -13,6 +13,7 @@ generic std gap is planned here even when moq never calls it. ## Required +- [The runtime's namespace is rs](/quest/m3/ts-rs-rename.md) - translated code and hand-written callers import the runtime as `rs`, not `rt` - [Retrial Cranelift on moq's tests](/quest/m3/cranelift-unwind.md) - waits on upstream: moq's tests pass under Cranelift, so its CI can weigh it again - [Relink without rebuilding](/quest/m3/relink.md) - measure upstream's skip of dependency-only rebuilds on moq's edits once it reaches nightly - [Unsafe code the translator can model](/quest/m3/ts-unsafe.md) - a dependency using only `unsafe` the translator can model translates from source @@ -20,13 +21,14 @@ generic std gap is planned here even when moq never calls it. - [Placeholders written to a crate's own fmt::Write](/quest/m3/ts-fmt-calls.md) - `write!` with placeholders into a crate's own writer translates, its `write_str` seeing each call std makes - [Waiting: drop the lang fixture's mul_add allowance](/quest/m3/ts-fma-allowance.md) - the differential fails on any `mul_add` difference again once wasm32's `fma` rounds exactly - [Waiting: type-check in process with tsrs](/quest/m3/ts-tsrs.md) - rust2-ts checks bindings through tsrs instead of `tsc` once it is a stable library -- [Waiting: a slice of one byte behind any &mut](/quest/m3/ts-slice-of-one.md) - `slice::from_mut` and `array::from_mut` of a `&mut u8` translate once a consumer calls them -- [Waiting: a slice object for a mutable part of other values](/quest/m3/ts-slice-object.md) - waits on a moq profile showing the `Proxy` view's traps matter, then decides on an `rt.ByteSlice`-like form +- [rs.Slice](/quest/m3/ts-slice-object.md) - slices and arrays of other values are a runtime type of array, start, and length, splitting in O(1) and comparing by where they point - [Port the workflow guard to kixelated/quest](/quest/m3/quest-workflow-guard.md) - quest's environment guard parses YAML as this repository's does - [A vector keeps the capacity its pushes grew as it shrinks](/quest/m3/ts-vec-clear-capacity.md) - every step that drops a pushed `Vec`'s length keeps its capacity, as std guarantees - [A vector's exact capacity as it shrinks](/quest/m3/ts-vec-exact-capacity.md) - a deque made from a `Vec` splits its slices where std's does -- [A string's exact capacity](/quest/m3/ts-string-capacity.md) - a `String` holds std's capacity, so a program making a `Bytes` of one reads its capacity +- [rs.String](/quest/m3/ts-string-capacity.md) - `String`, `Box`, and `&str` are a runtime type holding std's capacity - [A B-tree's searches call Ord as std's do](/quest/m3/ts-btree-search-order.md) - a B-tree's lookups and inserts call a key's own `Ord` in std's order and number - [Every panic of a hooked program has its location](/quest/m3/ts-panic-sites.md) - no panic in a program that sets a hook lacks the location native code gives it - [The chunk a BufMut lends](/quest/m3/ts-chunk-mut.md) - `BufMut::chunk_mut` and its `UninitSlice`'s writes translate, reaching the buffer's storage - [Slices not yet written](/quest/m3/ts-uninit-slices.md) - `Box::new_uninit_slice` and `write_clone_of_slice` translate +- [Python methods without a wrapper frame](/quest/m3/ffi-python-methods.md) - a method call on a generated Python object enters the extension with no wrapper frame +- [Python calls skip the loop lookup](/quest/m3/ffi-python-loop-lookup.md) - a call while an asyncio loop runs costs what one outside does unless something waits diff --git a/quest/m3/ffi-python-loop-lookup.md b/quest/m3/ffi-python-loop-lookup.md new file mode 100644 index 000000000..ffcb6a4ac --- /dev/null +++ b/quest/m3/ffi-python-loop-lookup.md @@ -0,0 +1,22 @@ +# [S] Python calls skip the loop lookup + +## Goal + +A call into the exporter's [Python](/ffi/README.md) while an asyncio loop runs costs what a +call outside a loop does, unless the thread has a future waiting to be woken. `just ffi +bench` shows the in-loop call back at main's cost before +[#564](https://github.com/moq-dev/rust2/pull/564), and the wake tests #564 added still pass. + +## Plan + +Decided with the maintainer on 2026-11-03: planned from #564's follow-up. #564 resumes a +same-thread wake without the socket pair by looking up the running loop on every call made +while one runs, about +14 ns (104 to 118 ns). Most of those calls wake nothing. Skip the +lookup unless something on the thread is waiting; #564's PR body explains why remembering +only the thread that last ran the loop is unsafe (a stale loop can deadlock a blocking call). + +moq's Python calls run inside a loop, so moq pays this cost. Public API: none. + +## Related + +- [Python methods without a wrapper frame](/quest/m3/ffi-python-methods.md) - the other per-call cost diff --git a/quest/m3/ffi-python-methods.md b/quest/m3/ffi-python-methods.md new file mode 100644 index 000000000..a3ed47e5e --- /dev/null +++ b/quest/m3/ffi-python-methods.md @@ -0,0 +1,25 @@ +# [M] Python methods without a wrapper frame + +## Goal + +A method call on an object the exporter's [Python](/ffi/README.md) generates enters the +extension directly, with no Python wrapper frame in between, saving about 25 ns of a call's +104 ns. `just ffi bench` reports the call against main and +[BoltFFI's](/bench/ffi/README.md#boltffi-comparison) in the same run. The Python API and the +GIL release during a call stay as they are. + +## Plan + +Decided with the maintainer on 2026-11-03: planned from the alternatives +[#560](https://github.com/moq-dev/rust2/pull/560) measured and dropped. That PR found the wrapper +frame costs about 25 ns; native methods (a `PyType_FromSpec` method table, or whatever +CPython offers that is faster) would remove it. It changes how the generated classes are +built, so keep what callers see the same: signatures, keyword arguments, defaults, +docstrings, and the type stubs. Holding the GIL stays rejected, since a call blocking on a +lock a Rust thread holds while calling Python back would deadlock. + +moq's Python bindings are the consumer. Public API: none. + +## Related + +- [Python calls skip the loop lookup](/quest/m3/ffi-python-loop-lookup.md) - the other per-call cost left after #564 diff --git a/quest/m3/ts-rs-rename.md b/quest/m3/ts-rs-rename.md new file mode 100644 index 000000000..35ec2d3b5 --- /dev/null +++ b/quest/m3/ts-rs-rename.md @@ -0,0 +1,21 @@ +# [S] The runtime's namespace is rs + +## Goal + +[rust2-ts](/ts/README.md)'s runtime is imported and named `rs` rather than `rt`, in the +translator's output, the runtime's own files, the fixtures and their snapshots, the tests, +and the README, so a translated Rust type reads as `rs.Vec`, `rs.HashMap`, `rs.u64`. +Nothing named `rt` is left. + +## Plan + +Decided with the maintainer on 2026-11-03: rename, as its own PR, started next. The +wrapper quests ([rs.String](/quest/m3/ts-string-capacity.md), +[rs.Slice](/quest/m3/ts-slice-object.md)) run in parallel rather than waiting on it; +whichever lands second merges the rename. + +It touches every open TS PR, so land it when few are in flight and merge main into +them rather than leaving them to conflict. It's mechanical: the snapshots change by name +only, and the bundles should differ by nothing but the name. + +Public API: the translator's output and every hand-written import of the runtime. diff --git a/quest/m3/ts-slice-object.md b/quest/m3/ts-slice-object.md index adaf16d54..48673f368 100644 --- a/quest/m3/ts-slice-object.md +++ b/quest/m3/ts-slice-object.md @@ -1,23 +1,38 @@ -# [L] Waiting: a slice object for a mutable part of other values +# [L] rs.Slice ## Goal -Decide, from a profile of moq, whether a mutable part of a slice of other values, such as -`&mut v[a..b]`, `split_at_mut`, or `chunks_mut` lends, needs a faster form than the -runtime's `Proxy` view. Waits until a profile of translated moq shows the view's traps -matter, by time spent in `sliceView`'s handler or in code reading through a view. If it -does, a `&mut [T]` becomes an object of its array, start, and length, as `rt.ByteSlice` -is for bytes, which splits in O(1) and reads at near a plain array's speed. Proven by the -lang fixture's differential tests and a benchmark against the view. +[rust2-ts](/ts/README.md) translates `&[T]`, `&mut [T]`, `[T; N]`, and `Box<[T]>` of values +other than bytes as a runtime type of its array, start, and length, as `rs.ByteSlice` is +for bytes, rather than a JS array and the `Proxy` view a mutable part lends today: -Once `&[T]` takes that form too, pointers to a `[T; N]` of other values than bytes, -including those a slice lends by `first_chunk`, `last_chunk`, `split_first_chunk`, -`as_chunks`, or `as_rchunks`, compare where they point, as `ptr::eq` does natively, -lifting the refusal of [moq-dev/rust2#510](https://github.com/moq-dev/rust2/pull/510), -proven by differential tests against native Rust. +- A mutable part, such as `&mut v[a..b]`, `split_at_mut`, or `chunks_mut` lends, splits in + O(1) and reads at near a plain array's speed, proven by the lang fixture's differential + tests and a benchmark against the view. +- Pointers to a `[T; N]` of other values, including those `first_chunk`, `last_chunk`, + `split_first_chunk`, `as_chunks`, and `as_rchunks` lend, compare where they point, as + `ptr::eq` does natively, lifting the refusal of + [moq-dev/rust2#510](https://github.com/moq-dev/rust2/pull/510). +- `slice::from_mut` and `array::from_mut` translate, of a `&mut u8` too, writing the + value through the slice where the caller holds it. + +Hand-written callers convert at the boundary with helpers, as for `rs.Vec`. Each is proven +by differential tests against native Rust. ## Plan +Decided with the maintainer on 2026-11-03: Rust types the runtime can't hold as std's do +get a runtime wrapper, converted at API boundaries, rather than a JS native type +([#555](https://github.com/moq-dev/rust2/pull/555) made `Vec` one). This quest absorbs the +two that waited: on a moq profile of the `Proxy` view, and on a consumer of +`slice::from_mut` of a `&mut u8`. Neither wait holds now. + +A slice of one over a `&mut u8` was found fragile when it meant a one-byte buffer for each +local; a slice whose storage can be an `rs.Place` may avoid that. Measure what each form +costs a hot `&mut u8`, such as `read_exact(slice::from_mut(&mut byte))`. + +Earlier findings: + Decided on 2026-10-30 in [moq-dev/rust2#274](https://github.com/moq-dev/rust2/pull/274): lend the `Proxy` view, since it covers every part, including one returned or stored, and moq's sites run per timer operation or control message, not per frame. The traps cost @@ -55,4 +70,9 @@ whole write. Noting the origin costs, per shared part of 8 of 16 values, in ns: The object carries the array and start, and compares by them as `rt.ByteSlice` does. -Public API: the translator's output and what a caller passes for a slice. +Public API: the translator's output and what a hand-written caller passes for a slice. + +## Related + +- [The runtime's namespace is rs](/quest/m3/ts-rs-rename.md) - runs in parallel +- [rs.String](/quest/m3/ts-string-capacity.md) - the other native mapping becoming a wrapper diff --git a/quest/m3/ts-slice-of-one.md b/quest/m3/ts-slice-of-one.md deleted file mode 100644 index 10ec11de1..000000000 --- a/quest/m3/ts-slice-of-one.md +++ /dev/null @@ -1,29 +0,0 @@ -# [S] Waiting: a slice of one byte behind any &mut - -## Goal - -Condition: a consumer calls `slice::from_mut` or `array::from_mut` of a `&mut u8`, which -[rust2-ts](/ts/README.md) refuses with a span until then. Check: `just ts census` of the -consumer, or its sources for either call over a byte. Then both translate for every -`&mut u8` that consumer passes, proven by a differential test against native Rust that -writes the byte through the slice and reads it where the caller holds it. - -## Plan - -Decided with the maintainer on 2026-11-03: wait for a consumer. A `&mut [u8]` is an -`rt.ByteSlice`, which can only view a `Uint8Array`, while a `&mut u8` is an `rt.Place` -that reads and writes a JS number wherever it is. Giving a local or a field a one-byte -buffer of its own was found fragile: that representation would have to cross -parameters, fields, and map entries. - -- A `&mut u8` parameter reaching `from_mut`: neither option was picked, and the question - waits for the consumer. - - Refuse at the caller, by a whole-program analysis. - - Fall back at run time. -- `u8` struct fields: - - ✅ Refuse fields for now. - -`read_exact(slice::from_mut(&mut byte))` of a local is the common use; measure what each -form costs a hot `&mut u8`. - -Public API: the translator's output. diff --git a/quest/m3/ts-string-capacity.md b/quest/m3/ts-string-capacity.md index 7446eac20..e5c09663c 100644 --- a/quest/m3/ts-string-capacity.md +++ b/quest/m3/ts-string-capacity.md @@ -1,45 +1,41 @@ -# [M] A string's exact capacity +# [L] rs.String ## Goal -[rust2-ts](/ts/README.md) holds a `String`'s exact capacity as std's does, through +[rust2-ts](/ts/README.md) translates `String`, `Box`, and `&str` as a runtime type +rather than a JS `string`, holding what std's do: a `String`'s exact capacity through `with_capacity`, `reserve`, `reserve_exact`, push growth, `shrink_to_fit`, `shrink_to`, `format!`'s estimate, and `into_bytes`, `Vec::from`, and `Bytes::from`, which take it. -Differential tests against native Rust prove it, and the refusal below is lifted. +Differential tests against native Rust prove it, and the refusal of reading a `Bytes`' +capacity where a `String`'s bytes may reach it is lifted. Hand-written callers convert at +the boundary with helpers, as for `rs.Vec`. ## Plan -Decided with the maintainer on 2026-11-03 -([moq-dev/rust2#532](https://github.com/moq-dev/rust2/pull/532)): - -> A `Bytes` of a `String`'s bytes is unique, and truncates to nothing, by the `String`'s -> capacity, which JS keeps none of. How should what reads it behave? -> -> - Throw a `rust2-ts:` error at runtime. -> - Refuse `From for Bytes` at translation. -> - Treat a `String`'s capacity as its length. -> - ✅ Track String capacity, as its own quest. Until then, refuse at translation. - -A `String` is a JS string, so `capacity` is refused and `with_capacity`, `reserve`, and -`shrink_to_fit` do nothing. A `Vec` or a `Bytes` of its bytes gets a buffer as long -as its UTF-8, whose capacity std's differs from after `with_capacity`, `format!`, or a -push that grew it. So where any translated crate makes a `Bytes` or a `Vec` of a -`String`'s bytes, every crate is refused what reads a `Bytes`' capacity (`is_unique`, -`try_into_mut`, `truncate`, `clear`, `From` into a `BytesMut` or a `Vec`, and -`BufMut::put` into a `BytesMut`), since the bytes may reach any of them. Lifting that -needs the capacity on the `String` itself. - -Measure first: a `String` as a wrapper object holding its text and capacity, and a side -table, such as a `WeakMap` from an object, against plain JS strings, on `just ts bench` -and the moq codec in Node, Chrome, Bun, and Firefox, before choosing. A string's text is -a primitive, so a side table needs an object to key on, and a wrapper changes every -string operation's output; weigh both against how few strings ever read a capacity. -[The pushed capacity quest](/quest/m3/ts-vec-clear-capacity.md) measures a `Vec` -wrapper the same way, so share the harness and compare the two. - -Public API: the translator's output. +Decided with the maintainer on 2026-11-03: Rust types the runtime can't hold as std's do +get a runtime wrapper, converted at API boundaries, rather than a JS native type. In +their words on [#555](https://github.com/moq-dev/rust2/pull/555), "trying to support +native types is probably a mistake". `Vec` became one there and `Cow` in +[#567](https://github.com/moq-dev/rust2/pull/567). Primitives, `Option`, tuples, and +closures stay native. + +What it holds: + +- ✅ Measure both, and bring the numbers back as a decision. +- A JS string and its capacity: engine string operations and `Display` stay cheap, but + byte offsets still convert from UTF-16. +- UTF-8 bytes, as `rs.ByteVec` holds: byte indexing and in-place edits are as Rust's, but + every `Display` and JS boundary decodes. + +Measure on `just ts bench` and moq's translated codec in Node, Chrome, Bun, and Firefox, +with an A/A run of main for noise, as #555 did. + +Earlier, on [#532](https://github.com/moq-dev/rust2/pull/532), the maintainer chose to +track capacity rather than refuse it; until this lands, the refusal stays. + +Public API: the translator's output and what a hand-written caller passes for a string. ## Related -- [A vector keeps the capacity its pushes grew as it shrinks](/quest/m3/ts-vec-clear-capacity.md) - measures a `Vec` wrapper holding its own capacity -- [A vector's exact capacity as it shrinks](/quest/m3/ts-vec-exact-capacity.md) - a `Vec`'s exact capacity, which a `String`'s bytes become +- [The runtime's namespace is rs](/quest/m3/ts-rs-rename.md) - runs in parallel +- [rs.Slice](/quest/m3/ts-slice-object.md) - the other native mapping becoming a wrapper From 174ae19dbfc75f25feaa9c71fcfdf629f96faa9f Mon Sep 17 00:00:00 2001 From: Luke Curley <432854+kixelated@users.noreply.github.com> Date: Tue, 3 Nov 2026 23:05:13 -0800 Subject: [PATCH 2/2] chore(quest): prefer an aliased native type where it holds std's semantics Co-Authored-By: Claude Opus 5.5 --- quest/m3/ts-slice-object.md | 9 ++++++--- quest/m3/ts-string-capacity.md | 11 +++++++---- 2 files changed, 13 insertions(+), 7 deletions(-) diff --git a/quest/m3/ts-slice-object.md b/quest/m3/ts-slice-object.md index 48673f368..cb5e72a16 100644 --- a/quest/m3/ts-slice-object.md +++ b/quest/m3/ts-slice-object.md @@ -23,9 +23,12 @@ by differential tests against native Rust. Decided with the maintainer on 2026-11-03: Rust types the runtime can't hold as std's do get a runtime wrapper, converted at API boundaries, rather than a JS native type -([#555](https://github.com/moq-dev/rust2/pull/555) made `Vec` one). This quest absorbs the -two that waited: on a moq profile of the `Proxy` view, and on a consumer of -`slice::from_mut` of a `&mut u8`. Neither wait holds now. +([#555](https://github.com/moq-dev/rust2/pull/555) made `Vec` one). A native type, aliased +or branded, stays where it holds all of std's semantics with no extra allocation, as a +branded `number` does for an integer and a `Uint8Array` for a `[u8; N]`; prefer one if it +holds them, but capacity and the like usually need a wrapper. This quest absorbs the two +that waited: on a moq profile of the `Proxy` view, and on a consumer of `slice::from_mut` +of a `&mut u8`. Neither wait holds now. A slice of one over a `&mut u8` was found fragile when it meant a one-byte buffer for each local; a slice whose storage can be an `rs.Place` may avoid that. Measure what each form diff --git a/quest/m3/ts-string-capacity.md b/quest/m3/ts-string-capacity.md index e5c09663c..50ac6e8f6 100644 --- a/quest/m3/ts-string-capacity.md +++ b/quest/m3/ts-string-capacity.md @@ -13,11 +13,14 @@ the boundary with helpers, as for `rs.Vec`. ## Plan Decided with the maintainer on 2026-11-03: Rust types the runtime can't hold as std's do -get a runtime wrapper, converted at API boundaries, rather than a JS native type. In -their words on [#555](https://github.com/moq-dev/rust2/pull/555), "trying to support -native types is probably a mistake". `Vec` became one there and `Cow` in +get a runtime wrapper, converted at API boundaries, rather than a JS native type. In their +words on [#555](https://github.com/moq-dev/rust2/pull/555), "trying to support native +types is probably a mistake". `Vec` became one there and `Cow` in [#567](https://github.com/moq-dev/rust2/pull/567). Primitives, `Option`, tuples, and -closures stay native. +closures stay native. A native type, aliased or branded, stays where it holds all of std's +semantics with no extra allocation, as a branded `number` does for an integer and a +`Uint8Array` for a `[u8; N]`; prefer one if it holds them, but capacity and the like +usually need a wrapper. What it holds: