[kixelated]

Problem

rust2-ts refused String::from_utf8_lossy (found by #528), and with it the rest of std's stable lossy and chunked UTF-8 decoding. No quest covered it; the maintainer plans all of std.

Survey of the family on main: str::from_utf8, String::from_utf8, Utf8Error::valid_up_to and error_len, and Debug/Display/Clone/== of both errors already translated. Refused were: String::from_utf8_lossy; <[u8]>::utf8_chunks and its Utf8Chunks (Debug, Clone); Utf8Chunk (the type, valid, invalid, Debug, Clone, ==); FromUtf8Error::as_bytes, into_bytes, utf8_error; str::from_utf8_unchecked and String::from_utf8_unchecked; String::into_bytes and Vec::<u8>::from a String or &str.

Approach

• from_utf8_lossy is rt.stringFromUtf8Lossy: a non-fatal TextDecoder with ignoreBOM. WHATWG's UTF-8 decoder writes one U+FFFD per maximal invalid subsequence, the same rule std's Utf8Chunks follows; the differential test confirms it. Its Cow<str> is the text, as every Cow<str> already is.
• utf8_chunks is rt.byteSliceUtf8Chunks, an Iter<rt.Utf8Chunk> that splits where the existing UTF-8 check (now rt.utf8Check, returning undefined for valid input) fails. A sequence cut off by the end of the input is invalid up to the end, as in std. Debug writes Utf8Chunks { source: "..." }, with each invalid byte as \xNN.
• Utf8Chunk is { valid, invalid }. Its derived == lowers from std's MIR, as std's errors' do (Std::error is renamed Std::record to cover it). Clone returns the value, and Debug matches std's derived output.
• FromUtf8Error's accessors read its bytes and error.
• from_utf8_unchecked (of str or String) decodes with the lossy decoder. By contract the input is valid, so the result is exactly the text. String::into_bytes and Vec::<u8>::from a string are rt.stringIntoBytes.

Tests: lang/utf8 gains utf8::lossy. It runs on random raw bytes, on bytes from an alphabet of UTF-8 boundary bytes, and on hand-picked inputs: a BOM, truncated sequences (at the end and cut by another byte), overlongs, encoded surrogates, bytes above F4, empty input, and valid multi-byte text. From every offset it compares the lossy text, every chunk's valid/invalid/Debug/==, the iterator's Debug and clones as it advances, and the unchecked and byte conversions. It also checks FromUtf8Error's accessors. Mutation check: making a truncated chunk 1 byte long fails 10 of 361 cases.

Not covered, because each is a recorded design choice: the Borrowed vs Owned variant of the returned Cow<str> can't be observed. A Cow<str> is a JS string, and matching its variant is refused with a span (reject/src/strs.rs, #105). str::from_utf8_mut stays refused as the README documents, since no JS string is written in place.

Impact

• Public API: the translator's output. New runtime exports: rt.stringFromUtf8Lossy, rt.byteSliceUtf8Chunks, rt.Utf8Chunk, rt.fmtUtf8Chunks, rt.fmtUtf8ChunkDebug, rt.stringIntoBytes, rt.utf8Check. No breaking change.

Alternatives

• Decoding lossily in JS over Utf8Chunks: same output, but slower than the engine's native decoder, which already follows the same replacement rule.
• Giving Cow<str> a variant so the lossy result's Borrowed/Owned can be observed: that changes every Cow<str>, so it belongs in a quest of its own.

Follow-ups

• Suggested for /quest-plan: matching a Cow<str>'s variants, which is stable std that is still refused (see Not covered above).

Review

Codex (gpt-6.1-sol, medium effort) via quest review, round 1, head 7a88ae3346ec24cc87928e2e5c687b7c71df2eba: no findings. Since then only a conflict-free merge of main.

🤖 Generated with Claude Code

[pullfrog]

This run was cancelled before it finished.

  ^{｜ View workflow run ｜ via Pullfrog}

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 54 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: d1bfcf80-147e-4fec-a7ad-932e0c628f99

Commits

Reviewing files that changed from the base of the PR and between 464416d and 69fbfcd.

Files selected for processing: 17

• ts/README.md
• ts/fixtures/lang/utf8/caller.ts
• ts/fixtures/lang/utf8/expected/alloc/string.ts
• ts/fixtures/lang/utf8/expected/core/str/error.ts
• ts/fixtures/lang/utf8/expected/core/str/lossy.ts
• ts/fixtures/lang/utf8/expected/lib.ts
• ts/fixtures/lang/utf8/expected/utf8.ts
• ts/fixtures/lang/utf8/src/lib.rs
• ts/fixtures/lang/utf8/src/utf8.rs
• ts/runtime/rt.ts
• ts/runtime/rt/str.ts
• ts/src/lower/collections.rs
• ts/src/lower/fmt.rs
• ts/src/lower/iter.rs
• ts/src/lower/library.rs
• ts/src/lower/mir/place.rs
• ts/src/lower/mod.rs

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn

CodeRabbit Verify

• Run CodeRabbit Verify to test the changes in this pull request.