[kixelated]

Problem

A translated String is a JS string, which holds no capacity, so String::capacity and everything that takes it (into_bytes, Vec::from, Bytes::from) is refused or guessed. The quest (ts-string-capacity) makes String a runtime type, rs.String, as #555 did for Vec. Its first step is a decision: measure an rs.String over a JS string against one over UTF-8 bytes, and bring the numbers back.

This PR holds the measurement only. The translation waits on the pick below.

Approach

• Workloads: the codec bench gains str/ workloads shaped on moq-net's string paths, each over an ASCII path (demo/room-42/alice/camera) and one with 2-, 3-, and 4-byte chars (démo/salle-42/アリス/📷):

  • str/<path> encode and decode: 256 size-prefixed strings through the fixture's Encoder::string and Decoder::string (moq-net's coding).
  • str/join/<path>: Path::concat's String::with_capacity plus push_str/push, then Path::strip_prefix's byte-boundary check.
  • str/keyed/<path>: 256 format!ed paths as HashMap<String, _> keys, each looked up four times by &str (moq-net finds a broadcast by path this way).
  • fmt/lines (existing): writeln! into one String.

  The hand-written route (bench/strings.ts) uses moq's own Writer.string and Cursor.string.

• Prototypes: ts/bench/prototype/str-routes.py rewrites a translation into each model on the paths those workloads run, as feat(ts): a Vec as rt.Vec, holding std's exact capacity #555's vec-routes.py did. Each edit is what the translator would emit. Every route's output matches native Rust's (the bench checks this).

  • js: rs.String is { s: string, len, cap }: the JS string, its UTF-8 length, and std's exact capacity, grown per push as RawVec::grow_amortized grows it. &str and Box<str> stay plain JS strings, since they hold no capacity. A HashMap<String, _> goes by .s in a JS Map and keeps each key String beside its value.
  • utf8: rs.String is an rs.ByteVec, whose storage length is the exact capacity, and &str is an rs.ByteSlice. A literal's bytes are made once when its module loads. fmt writes encode into the vector's room, and a &str argument is copied as bytes. A HashMap<String, _> hashes the bytes through the runtime's hasher, as an object key does today.

Measured

Host: Ryzen 9 5900XT, shared, load 8 to 20 on 32 threads during the runs. Ratios are route ÷ main's translation, timed in the same run: the median over 3 just ts bench runs per model, with min and max in parentheses. Main's translation here is 90e0f35 plus these workloads, before the merge of #555 and #578; neither changes the string paths.

rs.String over a JS string (js) ÷ main:

workload	Node	Bun	Chrome	Firefox
fmt/lines	1.22 (1.22-1.30)	1.12 (1.00-1.20)	1.07 (0.92-1.08)	1.09 (0.97-1.11)
str/ascii encode	0.99 (0.84-1.00)	0.95 (0.92-1.01)	1.01 (1.01-1.04)	1.05 (0.98-1.12)
str/utf8 encode	1.03 (0.97-1.05)	0.87 (0.79-0.96)	0.99 (0.97-1.04)	1.04 (0.95-1.07)
str/ascii decode	0.31 (0.28-0.35)	0.87 (0.83-1.02)	0.22 (0.21-0.28)	0.91 (0.91-0.95)
str/utf8 decode	0.38 (0.38-0.39)	0.94 (0.65-0.95)	0.24 (0.24-0.27)	0.90 (0.84-0.91)
str/join/ascii	0.75 (0.72-0.82)	0.80 (0.80-0.82)	0.08 (0.07-0.08)	0.57 (0.55-0.74)
str/join/utf8	0.57 (0.50-0.63)	0.85 (0.73-0.88)	0.08 (0.06-0.08)	0.48 (0.45-0.48)
str/keyed/ascii	2.27 (2.00-2.34)	1.25 (1.21-1.46)	1.58 (1.37-1.65)	1.12 (1.08-1.80)
str/keyed/utf8	1.95 (1.91-2.22)	1.13 (1.11-1.22)	1.53 (1.34-1.61)	0.89 (0.86-0.90)
every other workload, geomean	1.00 (0.99-1.00)	1.01 (0.96-1.01)	0.99 (0.98-1.00)	1.00 (0.99-1.00)
A/A ÷ main, geomean (noise)	1.01 (0.99-1.01)	1.00 (0.99-1.03)	1.00 (0.99-1.01)	0.99 (0.99-1.00)

rs.String over UTF-8 bytes (utf8) ÷ main:

workload	Node	Bun	Chrome	Firefox
fmt/lines	0.84 (0.73-0.86)	0.86 (0.79-0.86)	0.93 (0.90-1.00)	0.78 (0.62-0.82)
str/ascii encode	0.15 (0.14-0.17)	0.30 (0.29-0.32)	0.08 (0.07-0.08)	0.32 (0.32-0.39)
str/utf8 encode	0.15 (0.14-0.16)	0.29 (0.28-0.29)	0.08 (0.08-0.08)	0.30 (0.25-0.31)
str/ascii decode	0.19 (0.19-0.20)	0.74 (0.67-0.77)	0.11 (0.10-0.12)	0.67 (0.63-0.70)
str/utf8 decode	0.20 (0.20-0.20)	0.58 (0.57-0.61)	0.12 (0.12-0.12)	0.73 (0.72-0.76)
str/join/ascii	0.10 (0.09-0.11)	0.72 (0.66-0.74)	0.04 (0.03-0.04)	1.00 (0.91-1.03)
str/join/utf8	0.11 (0.10-0.12)	0.69 (0.68-0.77)	0.03 (0.03-0.04)	0.85 (0.84-0.87)
str/keyed/ascii	5.56 (4.80-5.61)	7.29 (7.06-7.49)	9.14 (8.87-10.97)	9.41 (9.01-9.62)
str/keyed/utf8	5.87 (5.60-6.22)	8.54 (6.67-8.55)	9.84 (9.00-10.14)	7.63 (6.75-8.63)
every other workload, geomean	0.99 (0.99-1.00)	1.00 (1.00-1.02)	1.02 (1.01-1.02)	0.99 (0.99-1.00)
A/A ÷ main, geomean (noise)	1.00 (0.99-1.01)	0.99 (0.98-0.99)	1.00 (0.98-1.02)	1.00 (0.99-1.00)

utf8 ÷ js, from the medians above:

workload	Node	Bun	Chrome	Firefox
fmt/lines	0.68	0.76	0.86	0.72
str/* encode	0.14-0.15	0.32-0.33	0.07-0.08	0.29-0.31
str/* decode	0.52-0.62	0.62-0.85	0.51	0.74-0.81
str/join/*	0.14-0.20	0.81-0.90	0.44-0.45	1.76-1.79
str/keyed/*	2.45-3.01	5.83-7.59	5.80-6.44	8.37-8.61

Main is slow on these paths for reasons separate from the model. Its stringLen and stringAsBytes encode the whole string with TextEncoder on every call, and push_str is a concatenation. So much of each model's gain on decode and join is just dropping those calls. The utf8 ÷ js table compares the two models directly.

The keyed gap in utf8 is the map, not the key. Split in Node, per key over an insert plus four gets:

	ns
JS Map of JS strings	~24
utf8 over the runtime's hasher (as measured)	~1100
utf8 over FNV-1a into a Map<number, chain>	~310 (Bun) to ~800 (Node)
utf8 keyed by a byte-per-char JS string	slower than the hasher
formatting the 256 keys: utf8 / main	~250 / ~80

Engines hash and intern JS strings natively. Bytes can't get that without making a JS string first, which is the decode the model exists to avoid.

Bundles, minified by Bun as the bench ships them, in bytes (codec bench fixture; utf8 includes its literals' encodes):

route	raw	gzip
main	78,593	25,003
js	79,151 (+0.7%)	25,153 (+0.6%)
utf8	80,866 (+2.9%)	25,767 (+3.1%)

Trade-offs

	js: JS string + length + capacity	utf8: UTF-8 bytes
Interop	&str and Box<str> stay JS strings, so a hand-written caller passes "name" as is. Only a String crosses as rs.String (rs.stringFrom(s) in, .s out).	Every &str and String crosses as bytes. A caller encodes each argument (rs.strFrom("name")) and decodes each result.
Byte offsets	Still converted from UTF-16 by a scan, as today (str.ts). push_str scans the pushed text for its UTF-8 length; a literal's is known at compile time.	Native: slicing, find, and is_char_boundary index bytes as Rust does.
as_bytes, into_bytes, Bytes::from	Encode a copy each time (TextEncoder), as main does. into_bytes gets the capacity from cap.	Identity. into_bytes and Bytes::from take the storage, so a Bytes' capacity is the String's with no copy. This is the refusal the quest lifts.
Display, {}	Free: the engine's string.	A &str into a String copies bytes. Any other writer (fmt::Formatter to JS, panic!, Debug) decodes.
HashMap<String, _>	Engine string hashing, plus an entry object per key: 0.9-2.3x main.	5.6-9.8x main: hashing bytes in JS.
Hot string ops measured	encode = main, decode 1.1-4.5x faster, join 1.2-13x faster, fmt 1.07-1.22x slower.	encode 3-14x faster than js, decode 1.2-2x, fmt 1.2-1.5x, join mixed (Firefox 1.8x slower).
Bundle	+0.6% gzip	+3.1% gzip

Decisions needed

Which representation should rs.String take?

1. js: a JS string, its UTF-8 length, and its capacity (recommended). &str and Box<str> stay plain JS strings. It keeps native interop and engine string hashing. Its only regressions on main are HashMap<String, _> at 0.9-2.3x (wrapper keys) and fmt/lines at 1.07-1.22x (a UTF-8 length scan per write). The map cost is likely reducible by keeping the key String only when its capacity differs from its length. It gives up utf8's encode speed: as_bytes and into_bytes still copy through TextEncoder, no worse than main today. Byte offsets keep today's UTF-16 scans.
2. utf8: an rs.ByteVec, with &str an rs.ByteSlice. It is the closest to std's semantics: byte offsets are native, and into_bytes/Bytes::from share storage. It is the fastest on moq-net's wire paths (encode 3-14x, decode 1.2-2x over js). But it is 5.6-9.8x slower than main on String-keyed maps, which moq-net uses to find broadcasts. It costs +3.1% bundle, and every hand-written &str argument needs a conversion.
3. Measure a hybrid first: utf8 plus a lazily cached JS string (decoded once, kept while the String is unmutated) for Display, map keys, and the boundary. Map keys are immutable in Rust, so a key's cache never goes stale. A lookup by a &str that borrows a cached String would hit, but one from a wire slice would still decode. This costs another prototype round.

Recommendation: 1. moq-net's wire paths match or beat main under js, its map lookups stay within 2.3x of main, and hand-written TS keeps passing JS strings, which the maintainer's boundary-helper direction on #555 favors. utf8's encode gain is real but is lost back on every String-keyed map.

Impact

Consumer: moq. The str/ workloads time moq-net's own string paths (Encoder::string, Decoder::string, Path::concat, Path::strip_prefix, keying by path).

Public API: none yet. The bench gains str/ workloads. The prototype under ts/bench/prototype/ goes before anything lands, as #555's did.

Alternatives

• FNV-1a over bytes into a Map<number, chain>, for utf8's maps: about 1.4-3.6x faster than the runtime's hasher, still 13-33x a JS string Map. Not timed in the full bench.
• A byte-per-char JS string as the map key for utf8: slower than hashing, as it builds a string a char at a time.

Follow-ups

• Main's rs.stringLen and rs.stringAsBytes call TextEncoder.encode on every use. Under either model len becomes a field, so this goes away with the translation, not separately.

Review

Not yet reviewed: draft awaiting the decision above.

🤖 Generated with Claude Code