Skip to content

feat(ts): a slice of one byte's place - #660

Merged
kixelated merged 14 commits into
mainfrom
quest/m3/ts-byte-slice-place
Nov 7, 2026
Merged

kixelated merged 14 commits into
mainfrom
quest/m3/ts-byte-slice-place

Conversation

@kixelated

@kixelated kixelated commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Claims quest/m3/ts-byte-slice-place.md and deletes it.

Decisions

  • Design checkpoint at review round 3, which found an unsafe fn detached(s: &mut [u8]) -> &'static mut u8 that launders the slice's lifetime, so a write through its result landed only in the mirror. Decided by the coordinator on 2026-11-07 while the maintainer was away, within m3's unsafe policy (quest/m3/README.md: safe Rust is the contract). ✅ Any result that could hold a pointer or a reference may hold the mirrored slice, whatever its lifetimes, which closes every escape through a result. Accepted cost: extra no-op write-backs, and a refusal with a span where such a result is stored through a parameter, returned, or kept across loop rounds. Arguments stay trusted to their signatures, as the thread-local case below is: only unsafe code that fakes a lifetime can break that, and the policy leaves unsafe outside the contract. (Rejected: merging as is, which leaves a silent stale byte through a result; treating a callee that is or reaches unsafe as keeping the slice everywhere, which scans whole call graphs and refuses generic and dyn calls; giving mirrored slices only to std's functions, which refuses the quest's measured case, a field read through a crate reader's read_exact.)

Problem

A &mut [u8] is an rs.ByteSlice, a view of a Uint8Array, while a &mut u8 is an rs.Place wherever the byte is: a JS local, an object's field, an element. So slice::from_mut, array::from_mut, and Option::as_mut_slice of a byte were refused: no Uint8Array could view the byte where the caller holds it.

Approach

Three forms, each picked where the body shows it applies:

  • A view, where the body shows the byte is in a Uint8Array: an element of an array, slice, or Vec of bytes (by a place, a reborrow of one, or index_mut). The slice is new rs.ByteSlice(data, index, 1).
  • A buffer of its own, for a byte variable whose &mut reaches one of these calls. It reuses moq-dev/rust2#640's Slot::Byte, a Uint8Array(1), which the slice then views.
  • A mirror for any other byte, such as a field's, an Option's payload, or one a &mut u8 parameter points at. rs.byteMirror(p) copies the byte into an rs.ByteMirror, or views it if p is an rs.AtIndex into a Uint8Array at run time. rs.byteMirrorFlush writes the copy back to p, but only if the byte changed since it was last written there. Rust reaches the byte only through the slice while the slice is in use. Once nothing uses the slice, the copy never changes again, so a later write-back writes nothing over a later write.

The translator (ts/src/lower/mir/mirror.rs) finds, for each call making a mirror:

  • Which locals may hold the slice or what is made of it (copies, reborrows, iterators, closures, raw pointers, results).
  • Where to write back: after each assignment through one of those locals, and as each block after a call or drop given one starts. A call returning its result as is returns through that block rather than straight away.

Because write-backs are conditional, over-estimating holders only costs no-op calls. A call's result may hold the slice whenever its type could take a pointer, whatever its lifetimes, since a callee's unsafe code can make one up. An argument's memory takes it only where the callee's signature lets borrow checking allow it. A late-bound region in the given argument reaches only arguments that name it. A type or region the caller picks may reach any argument that could take a pointer. For both, std types count only through their type arguments and regions, since their own pointers are private, so &mut Vec<u8> takes nothing and an io::Result<()> result holds nothing. The exceptions are NonNull, AtomicPtr, and the waker types, whose safe API puts in and gives back a raw pointer. A raw pointer names no region, so an argument pointing at memory that could hold one may keep the slice too. The existing raw-pointer trace in holders.rs was too conservative here: it refused a crate function generic over T, and std code calling the crate's trait impls.

Still refused, each with a span:

  • A mirror kept where no variable of the function holds it: returned, stored where a parameter points, or in a coroutine's state across an .await. Its copy would never be written back. This includes a result of a call given one, of a type that could take a pointer, even one whose lifetime says it can't hold the slice.
  • A mirror that may still be used when its call runs again, as when a loop keeps each in a Vec. One variable per call holds it. This applies only to a call in a loop.

Measurement

The quest asked what each form costs a hot &mut u8. The loop is read_exact(slice::from_mut(&mut byte)) over 1 MiB, one byte per call, into a crate-local cursor; std's io::Read for &[u8] doesn't translate. CPU time from process.cpuUsage, Node 24, median of 7 rounds of 20 runs; perf isn't installed on the host, so no instruction counts.

Form ns per byte
[u8; 1] array, the rs.ByteSlice over a Uint8Array every other byte slice is 23.4
byte variable in a buffer of its own (view) 20.0
byte variable through a &mut u8 parameter (mirror) 56.1
struct field (mirror) 65.1

A buffered variable costs the same as the array, so variables take that form. A mirror adds about 35 to 45 ns per call: the rs.place the &mut u8 already was, a Uint8Array(1), and the write-back. Allocating with new Uint8Array(1) rather than Uint8Array.of halved the mirror's extra cost (85 → 56 ns). Reusing a call's previous mirror in a loop saved about 12% on the field case and wasn't kept.

The quest's rs.ByteSlice whose storage can be an rs.Place wasn't built as such. Translated code and about 250 runtime sites read s.data[s.start + i] inline, and runtime functions call Uint8Array methods on data. Every byte slice would have paid for a second storage kind, or for a Proxy, which breaks those methods. The mirror is that idea bounded to the slices that need it.

Choices made without asking

  • Views and buffered variables wherever the body shows a byte is in a Uint8Array, mirrors elsewhere, per the measurement above.
  • Option::as_mut_slice of a byte, refused by the same message, translates by the same mirror.
  • A raw pointer made from a mirrored slice and kept past the slice's last use by the callee's unsafe code, such as in a thread-local, isn't followed. The holder analysis reads a crate callee's signature, not its body. Using such a pointer after the slice is done, but before the byte is next read, is defined in Rust and would read and write the copy. Reading that from callee bodies is what made holders.rs's trace refuse ordinary generic code.
  • The analysis trusts a callee's signature for references it may keep in an argument's memory, and follows raw pointers by type. Results don't trust it, per the decision above.
  • "Could hold a pointer or a reference", for a result, reads std's types through their type arguments and regions, as for arguments, rather than through their private fields. Reading their fields would make every io::Result<()> a holder, and refuse ? after read_exact into a field, since the error would reach the return place. A crate type erased into an io::Error could still carry a reference whose lifetime unsafe code made up: the same trust as the arguments.
  • Std's raw-pointer carriers are named in one list, raw_carrier in holders.rs: NonNull, AtomicPtr, RawWaker, Waker, LocalWaker, and Context. A waker's data can't point at a byte here anyway: the translation refuses a byte pointer cast to *const () at run time.
  • Returning a slice of one of a field's byte, or of a &mut u8 parameter's, stays refused, with no follow-up: its caller would need to know the slice it gets is a copy, and no consumer returns one.

Validation

  • lang::conversions runs lent_one and lent_through for u8 too, which skipped it before. That covers a variable, a field, a Vec element, and &mut &mut, through mem::replace and mem::swap, cells, nested arrays, and clone_from.
  • A new case, conversions::byte_places, covers:
    • fields, a tuple's, an array's, an Option's (Some and None), and a MaybeUninit's;
    • &mut self methods, &mut u8 parameters, and &mut &mut;
    • reads only, closures, a Vec of two, mem::swap, and copy_from_slice;
    • a copy read after the byte is written where it is, which tests the changed-only write-back;
    • the slice made again each loop round, with the byte written between;
    • a raw pointer made from one, returned, stashed in &mut Option<*mut u8>, and stashed as NonNull;
    • round 3's reproducer: a &'static mut u8 an unsafe fn makes from the slice, written through;
    • a function returning, as Ok and as Err, what a call given one returns.
  • Breaking each part makes the differential diverge from native Rust: dropping the write-back (170 differences), writing back unconditionally (163), dropping the write-backs after statements (163) or at block entries (170), ignoring raw-pointer results and receivers (163, measured before the decision above), ignoring NonNull receivers (153), and trusting a result's lifetimes, as before the decision above (153).
  • The reject fixture's two old refusals now translate. Five new cases pin the refusals: returned, stored through a parameter, kept across runs of the call, and a &'static str result of a call given one, stored through a parameter and kept across runs.
  • just ci (fmt, clippy, every fixture, the runtime tests) passes at 731eb51, the branch merged with main.

Impact

  • Translator output: these calls translate where they were refused. A byte variable whose &mut reaches one is now a Uint8Array(1) in generated code.
  • Runtime exports added: rs.ByteMirror, rs.byteMirror, and rs.byteMirrorFlush. A hand-written caller still gets an rs.ByteSlice. No breaking change, no wire change.

Follow-ups

None.

Models

Size: S
Implementer: Claude Code, claude-opus-5-5, high effort

Review

Rewritten by quest review after each round.

  • Reviewer: Codex, gpt-6.1-sol, medium effort
  • Session: 01a1b7f9-2174-7500-a0a8-258485c1c4ad, round 2
  • Head: 731eb51
  • Base: origin/main at eec5aa9
  • Score: 10/10, No remaining defects found; the forwarding fix preserves writeback and results for all 65,536 byte pairs checked, though full CI was not rerun.
  • Findings: none
  • The last round's findings, at 8a43692:
    • Fixed: medium, ts/src/lower/mir/mirror.rs:145: [P2] Flush byte mirrors before forwarding a tail-call result; ts/src/lower/mir/mirror.rs:164 disables early returns when successor writeback is due, and generated conversions.ts:8330 flushes before both returns, with both paths covered by the added differential case.

@pullfrog

pullfrog Bot commented Nov 7, 2026

Copy link
Copy Markdown

Pullfrog  | Review this ➔

kixelated and others added 6 commits November 7, 2026 12:14
…back

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…slices of one byte

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated kixelated changed the title A slice of one byte's place feat(ts): a slice of one byte's place Nov 7, 2026
kixelated and others added 3 commits November 7, 2026 12:55
A raw pointer names no region, so a call's result or a receiver that could
hold one may keep the slice. A call outside any loop can't make a second
mirror, so a live holder there is no longer refused.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NonNull, AtomicPtr, and the waker types hold a raw pointer that safe code puts
in and gets back, so a receiver or result of one may keep the slice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e-place

# Conflicts:
#	ts/src/lower/mir/call.rs
kixelated and others added 3 commits November 7, 2026 13:34
…ce's holder

A callee's unsafe code can return a reference made from the slice with a lifetime
its signature makes up, such as `&'static mut u8`, which names none of the slice's
regions. Any result whose type could take a pointer now may hold the slice, whatever
its regions; arguments stay trusted to their signatures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e-place

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Design change: after review round 3's design checkpoint, the coordinator decided (while the maintainer is away, within m3's unsafe policy) that any result of a call given a mirrored slice, whose type could hold a pointer or a reference, may hold the slice whatever its lifetimes.

  • call_keeps in ts/src/lower/mir/mirror.rs now reads only which arguments' memory may take the slice; every call's result goes through takes_pointer, so a &'static mut u8 an unsafe fn makes from the slice is a holder and gets written back.
  • "Could hold a pointer" reads std's types through their type arguments and regions, as for arguments, so an io::Result<()> from read_exact still holds nothing and ? after it still translates.
  • Arguments stay trusted to their signatures, alongside the thread-local case: only unsafe code that fakes a lifetime can break that.
  • Round 3's reproducer is a differential case in conversions::byte_places; the old analysis diverged on it (153 differences). Two reject cases pin the new refusals: a &'static str result stored through a parameter, and kept across loop rounds.
  • Rejected: merging as is, refusing callees that are or reach unsafe, and mirrored slices only for std callees. The follow-up on returning a mirrored slice is dropped.

The next review starts a new session with quest review --new.

(Written by Claude Opus 5.5 with high effort)

A call into the return place that only goes on to return lowered to a direct
`return`, past its target block, where the mirror of a slice of one byte given to
it is written back. Such a call now returns through that block.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated marked this pull request as ready for review November 7, 2026 21:57
@kixelated
kixelated added this pull request to the merge queue Nov 7, 2026

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed head 731eb51 against base eec5aa9.

One P2 correctness finding: owner can follow a reference's original local borrow after that reference was redirected through a mutable alias, letting a mirrored slice escape through a parameter without future writeback. The inline comment includes a safe-Rust regression case and a suggested guard.

Coverage: all substantive translator/runtime changes, holder propagation and escape checks, loop liveness, direct views and buffered locals, conditional flush placement including returned call results, and the added source/generated/reject fixtures and documentation.

Verification limits: GitHub-only static inspection. I did not run the reproducer, translator, differential tests, benchmarks, or just ci; the PR's reported results were not independently reproduced. This is a comment review, not an approval.

(Written by review (OpenAI))

let Some(at) = place.projection.iter().position(|e| e == mir::ProjectionElem::Deref) else {
return Some(place.local);
};
if at != 0 || self.defs[place.local] != 1 {

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Reject redirected references before resolving their original owner

defs == 1 counts direct assignments, but does not mean the reference still points at its original borrow. Unlike lent_byte/raw_byte, this traversal does not check self.mutated[place.local]. Safe code can redirect a reference through a mutable alias before storing the mirrored slice:

#[inline(never)]
fn slot<T>(p: &mut T) -> &mut T { p }

fn expose<'a>(b: &'a mut u8, out: &mut Option<&'a mut [u8]>) {
    let mut local = None;
    let mut target = &mut local;
    *slot(&mut target) = out;
    *target = Some(std::slice::from_mut(b));
}

Here target still has only its initial direct definition. owner(*target) follows that stale &mut local, so the store is classified as local even though it actually writes out. Flow-insensitive holder propagation does not recover the external destination: the earlier *slot_result = out reads an unheld RHS, and slot has no second argument naming out. The caller can subsequently mutate the saved slice after expose returns, when no mirror flush remains to update b.

Conservatively stop following references that may be mutated (or track their replacement targets), and add this redirection shape to the rejection fixtures, or a differential test if supporting it. This example and the analysis are static; I have not executed the reproducer.

(Written by review (OpenAI))

Merged via the queue into main with commit 41ea814 Nov 7, 2026
6 of 7 checks passed
@kixelated
kixelated deleted the quest/m3/ts-byte-slice-place branch November 7, 2026 22:10
kixelated added a commit that referenced this pull request Nov 7, 2026
…w-behind-ref

Brings #660's slices of one byte's place. ts/README.md keeps this branch's
pointer row with #660's slice-of-one clause; conversions.ts regenerated, where
#660's new cases take this branch's copy shims and `{ v: x }.v` fold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant