[voltcrash]

Repeated unclosed TeX math openers make typing and Markdown rendering rescan the remaining text from each opener. This is a small, focused fix for an obvious performance bug in the existing tokenizer; it changes no math syntax or product behavior.

Cache only failed scans whose end proves there is no closer: text scans stop at their existing EOF/backtick/single-line boundary, and flow scans cache only EOF failures. Keep text caches separate by delimiter and single-line mode, sharing the owning event array across GFM's speculative contexts. Weak keys limit their lifetime to parsing. Content-dependent citation failures and lazy flow boundaries remain uncached.

Proof on upstream main d2c25950b4d9183ea342d5d5fc14c326b9e0172a and this fix:

• The focused operation-count regression fails without the fix and passes with it, without elapsed-time assertions. With GFM, 200 repeated inline openers (1,000 characters) consumed 201,001 characters before and 2,399 after; 400 openers consumed 802,001 before and 4,799 after. The same test covers inline bracket openers and bare display-opener lines.
• Coverage preserves escaping, code/paragraph/container/table boundaries, mixed delimiters, citations, and later valid closers. 93 scoped math/composer tests passed. Separately, 20,000 seeded mixed-delimiter documents produced identical ASTs and offsets; the reviewer independently checked another 30,000.
• vp run check:changed passed: unused patches, formatting, lint, both knip checks, affected-package typechecks, and focused tests (61).
• Integrated web client: pasted 5 KB of repeated inline openers and 700 bare display-opener lines, then edited both; source and edits were retained. Formula source after failed openers remained editable. Used the built-in Browser panel first, then the installed Chromium/Playwright after repeated preview recording timeouts. 11-second recording, inline capture, display capture, operation counts. The free replay turn completed, but its history stayed loading in this dev client, so rendered-message verification is not claimed. State was a worktree-local copy; no writes to the live database and no paid turns.

Duplicate check: inspected current upstream source and open PRs #30658 (single-dollar composer precheck), #32110 (forge math dialect), and #27866 (web rendering parser replacement). None fixes the shared composer tokenizer's repeated failed TeX scans; #27866 leaves that shared parser in place. Keyword searches for math, TeX, micromark, unclosed delimiters, and quadratic parsing found no exact duplicate. These were targeted searches, not an exhaustive review of every open PR.

Scope: the patched dependency serves web/desktop rendering and shared composer parsing, including mobile's composer callers. No provider, protocol, permission, or connection-mode changes. Native mobile, Electron shell, relay/tunnel, and production-build performance were not exercised. Overall Markdown parsing still has other costs; this bounds the failed TeX scan work. The scenario guide explicitly excludes performance proof, so scenarios:prove was not used to substitute a timing threshold for the operation-count regression.

Reviewed with review-loop: 1 round with gpt-6.1-sol on xhigh. Reviewer verdict: approved, no findings.

Created with OpenAI gpt-6.1-sol (xhigh) in the Codex harness through T3 Code.

[macroscopeapp]

Approvability

Verdict: Approved at b656698

Macroscope's review found this PR approvable — The patch avoids repeated scans of unclosed TeX math in Markdown parsing. Its caches are tokenizer-local, with no changes to product defaults, stored content, or external systems.

^{You can add or adjust custom eligibility rules. Learn more.}

[coderabbitai]

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

Files ignored due to path filters: 1

• pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml

Run configuration

• Source mode: Auto
• Repository: pingdotgg/t3code
• Branch: main
• File: .coderabbit.config.ts
• Commit: d380556
• Layers: Repository TS config
• Review profile: CHILL
• Plan: Advanced
• Run ID: 9cf46f23-76da-4663-b544-65f58d8cfaea

Commits

Reviewing files that changed from the base of the PR and between 7745667 and fc768c9.

Files ignored due to path filters: 1

• pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• Trigger review

No actionable comments were generated in the recent review.

Walkthrough

The tokenizer patch changes math delimiter recognition, failed-scan handling, and the package default export target. New tests measure consumption for repeated unclosed delimiters and check that later valid formulas still parse.

Changes: Math tokenizer scan handling

Layer / File(s)	Summary
TeX flow closure patches/micromark-extension-llm-math@3.1.0.patch	The TeX flow tokenizer checks opening metadata and requires a closing fence. It records an EOF failure offset for an unclosed flow.
Text scan recovery and validation patches/micromark-extension-llm-math@3.1.0.patch, packages/shared/src/markdownMath.test.ts	Text scans cache failed opener positions and apply delimiter boundary rules. The patch also changes inline delimiter handling and the default export target. Tests bound consumption for repeated unclosed delimiters and check that later valid formulas still parse.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Possibly related issues

• [Bug]: Typing after a selected skill lags, worse on long prompts #30657 — The issue reports latency from repeated Markdown and math parsing after an unclosed opener. This PR adds failed-scan caching for the tokenizer path.

Possibly related PRs

• pingdotgg/t3code#29739 — That PR uses the shared math parser for mobile TeX delimiter conversion, and this PR changes scan behavior in that parser.

Suggested labels: bugfix, size:M

Suggested reviewers: yash-singh1, macroscopeapp, nikitamgrimm

Merge Risk: Minimal · previous review b6566

The change stops repeated rescans of unclosed TeX math without altering math syntax. No actionable merge-blocking risk was found in the reviewed changes.

Pre-merge checks: 4

Check name	Status	Explanation
Title check		The title is concise, uses the required conventional format, and clearly identifies the primary change: stopping repeated scans of unclosed TeX math.
Description check		The description covers the problem, implementation approach, scope justification, focused verification, observed results, limitations, and agent attribution. It is complete and relevant to the templat…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Description check

Explanation

The description covers the problem, implementation approach, scope justification, focused verification, observed results, limitations, and agent attribution. It is complete and relevant to the template.

Finishing Touches

Generate unit tests (beta)

• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Recent review info

Run configuration

• Source mode: Auto
• Repository: pingdotgg/t3code
• Branch: main
• File: .coderabbit.config.ts
• Commit: d2c2595
• Layers: Repository TS config
• Review profile: CHILL
• Plan: Advanced
• Run ID: 0cbd12f7-edc6-4b30-ad7c-0c061cd97c13

Commits

Reviewing files that changed from the base of the PR and between d2c2595 and b656698.

Files ignored due to path filters: 1

• pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml

Files selected for processing: 2

• packages/shared/src/markdownMath.test.ts
• patches/micromark-extension-llm-math@3.1.0.patch

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 7 remain after this review.

[maria-rcks]

Note

Written by claude-opus-5-5 on behalf of Maria

Verified in a real dev environment

Ran vp run dev on github/main (0629735d) and then on this PR head (b6566980), one server at a time, on a Linux VM. The data was the same in both runs. I added a local git repo as a project through the Add project dialog, then ran a real Claude claude-haiku-5-5 turn that streamed a message with thousands of unclosed \( a openers. The main worktree's .t3 state was copied into the PR worktree, so both runs saw identical threads. I drove the web client with Playwright and Chrome.

Composer (the path this PR speeds up). I typed check plus a @README.md mention, then inserted N copies of \( a , then typed one key at a time. The number below is the time from each keypress to the next painted frame.

	main	PR
1000 openers, per key	5206 to 5354 ms	224 to 288 ms
1500 openers, per key	9490 to 17161 ms	482 to 557 ms
1500 openers, inserting the text	8993 ms	483 ms
1500 openers, longest main-thread task	10562 ms	531 ms

Typed text was kept in both versions (composer ends with \( a typing).

Recordings, trimmed to start once the page is loaded:

Shared parser, same code the app imports (markdownMathRanges, run with node from packages/shared):

input	main	PR
10,000 chars of \( a	1497 ms	141 ms
20,000 chars of \( a	5683 ms	494 ms
6,000 chars of \[+newline	1738 ms	61 ms
12,000 chars of \[+newline	7953 ms	131 ms

Neighboring behavior on the PR:

• I turned on Settings > Appearance > math rendering, then ran a second Haiku turn in the same thread. Inline \(E = mc^2\), $a+b$ and a \[ \int_0^1 x\,dx = \tfrac12 \] display block all rendered with KaTeX, in both the user and assistant messages.
• With math on, opening the thread that holds the ~255 KB assistant message took 2054 ms on the PR. With math off it took 1.7 to 2.0 s on both versions.
• In \( a \( b then closed \(x^2\), the first \( pairs with the first \). That output is identical on main and on the PR, so it is existing behavior.
• collectComposerInlineTokens returns identical mention and formula masking on main and the PR for mentions inside, beside and after unclosed openers.

CI on b6566980 is green.

Not verified

• I did not measure chat rendering with math turned on against main. It is off by default, and the main server was already stopped when I found the setting.
• Native mobile composer (iOS and Android). It uses the same shared parser, but there is no simulator on this Linux VM.
• Electron desktop shell and production builds. Only the Vite dev web client was exercised.
• The first Haiku turn (asked to repeat \( a 1500 times) streamed its text but then kept "Thinking" for more than 14 minutes, so I stopped it. This looks like model behavior, not this PR. It happened on main, before the PR was running.

[juliusmarminge]

Evidence

Same Ubuntu 24.04.4 VM, Chromium 156 headless web-dev client, 1440×1000, dark theme, no throttling. Separate fresh databases and identical synthetic README project.

Steps: Open New thread; clear composer; type check ; type @README; Enter creates the README.md chip; type a space; clipboard-paste repeated openers; press t, y, p, i, n, g separately. Cases: ( a ×1500 control; \( a ×250/500/1000/1500; \[ plus newline ×1000/1500. Three repetitions per case, six individually timed keys per repetition.

Case	Before median ms/key	After median ms/key	Speedup
Inline ×1000	5,542	262	21.2×
Inline ×1500	13,531	572	23.6×
Display lines ×1000	5,665	195	29.0×
Display lines ×1500	10,824	261	41.5×
Plain control ×1500	25.8	21.7	Within noise

Medians pool all 18 keys. Every TeX improvement exceeds both sides’ run-median spread. At inline ×1500, improvement is 12,959 ms versus spreads of 918 ms before and 37 ms after. All 42 resets retain the expected text and README chip; normalized rendered-content hashes match across revisions. Stills and excerpts show repetition 1; videos play at original speed.

Before 4d47d0b76e	After 7745667049
Before: inline paste and typing video	After: inline paste and typing video
Before: display paste and typing video	After: display paste and typing video

Measurement driver · Seed script · Reproduction and method · Raw before measurements · Raw after measurements · Full before recording · Full after recording · Complete evidence bundle: all scripts, data, stills and recordings.

Not shown:

• Tokenizer consumed-character counts and isolated asymptotic complexity were not instrumented; this proof measures real composer performance and preserved content for the reported workloads.
• Physical display scanout was not measured because Chromium was headless; timings end after the next browser rendering opportunity.

[juliusmarminge]

Evidence

Type check , insert the README.md mention with @README → Enter, type a space, clipboard-paste repeated unclosed TeX openers, then press t,y,p,i,n,g individually. Same Linux Chromium web dev client, 1440×1000, light theme, fresh synthetic data. Three repeats per workload, six measured keys per repeat; reported aggregate is the median of the three run medians.

Before	After
d38055656e	fc768c9dfd
Inline ×1500: 12,336.7 ms/key, run medians 10,090.9–12,625.4 ms.	Inline ×1500: 727.8 ms/key, run medians 699.1–786.1 ms; 17.0× faster.
Bare \[ lines ×1500: 14,452.3 ms/key, run medians 10,737.0–14,560.8 ms.	Bare \[ lines ×1500: 418.6 ms/key, run medians 388.4–450.8 ms; 34.5× faster.
Before recording, real time	After recording, real time
Before: every raw sample	After: every raw sample

Every TeX workload improved by more than the entire observed per-key spread. Complete text hashes match across all repeats and both commits, with exactly one README mention chip each time. The control remained comparable: 27.3 ms before versus 25.0 ms after. Stills show the third repeat; media has the same privacy crop on both sides. A reconnect notice appeared on base outside that crop; the local composer retained all input.

Measurement driver · Synthetic fixture script · Method and reproduction commands · Comparison data · Chart script · Video export script

Not shown:

• Exact parser consumed-character counts, linear failed-scan complexity, and parsing equivalence for other math forms were not instrumented; this proof measures the reported real-composer interactions and verifies their complete retained text. Head still has noticeable latency at the largest inline workload.