[maschwenk]

the chrome bake-off from today lived in /tmp and drove the native worker over stdio, so it never went through the mcp layer agents actually use. this makes it a subcommand of computer-use-dev.ts so the next computer use prs (the shared verbs one is stacked on this) all run the same check

node scripts/computer-use-dev.ts bakeoff (add --home .t3 for a worktree dev server)

what it does

• serves scripts/computer-use-bakeoff.html on a loopback port. the page posts every dom event back with isTrusted and where it landed
• opens it in google chrome in the background with the harness's own profile (--disable-backgrounding-occluded-windows + --disable-renderer-backgrounding, otherwise a covered chrome stops painting), puts the front app back if the launch grabbed it, and quits chrome by the pid whose command line names that profile
• runs button, canvas point, hover by ref and by point, the revealed button, typing, return, list scroll, drag and select through computer_snapshot + computer_action over mcp, same calls an agent makes
• a task passes when the page logged the expected trusted events (canvas has to land on 101,60), the final snapshot shows the result, and NSEvent.mouseLocation + the front app are the same before and after every call
• prints the table below: events, each computer_action response's size and via, result
• won't run if another chrome is open, since computer tools find apps by name

baseline on main (88cb172, has the input, diff and cursor prs) with a dev server started from this worktree. 3 runs in a row came out identical except list scroll moving by a few chars

task	page events	computer_action chars, via	pointer, front app	result
button	mousemove, mousedown, btn.click	444 pointer_event	unchanged	pass
canvas click at 101,60	mousemove, mousedown, canvas.click 101,60	466 pointer_event	unchanged	pass
hover by ref	hover.enter, mousemove	4265 pointer_event	unchanged	pass
move away by point	mousemove	4203 pointer_event	unchanged	pass
hover by point	hover.enter, mousemove	4265 pointer_event	unchanged	pass
revealed button	mousedown, secret.click	463 accessibility	unchanged	pass
typing	keydown ×11, input ×11	501 keyboard_event	unchanged	pass
Return	keydown Enter	187 keyboard_event	unchanged	pass
list scroll	wheel	4310 pointer_event	unchanged	pass
drag	mousemove ×2, drag.down, mousedown, drag.up inside	561 pointer_event	unchanged	pass
select	mousedown, select chocolate	1073 accessibility + 837 accessibility	unchanged	pass

11/11, first snapshot 5758 chars, my pointer and helium in front never moved

worth knowing

• hover, move away and scroll come back around 4.2k chars. the diff isnt broken: showing the hidden button makes its row 3 px taller, so every element below gets a ~ line for its new y, including 40 clipped list rows. scroll moves every list row the same way. codex is 125-580 per action, so bounds churn is the next thing to look at for size

• the first button click goes through a pointer event, not ax press. chrome doesn't list press on web elements in the first snapshot after launch (the diffs pr saw this too). the revealed button later on goes through accessibility

• hover only passes because my real cursor sits inside chrome's window area (818,749 in cocoa coords vs a window at 40,40 900x1000). thats the known limit from the background input pr, and the summary line prints the pointer so you can tell

• the chrome profile stays in the temp dir between runs. a managed 1password extension opens a welcome tab on a fresh profile, once 20 s after launch, and that ate a step. so the script closes any other tab before each task, and keeping the profile means it only happens on the first run

• a dev server started from a t3 code (fork) terminal has accessibility + screen recording, macos credits the app that started the shell. so no dev electron app needed, runbook says so now

• docs(computer-use): one vocabulary for apps and browser tabs #38

• fix(computer-use): hover, exact click_at and real typing in background apps #37

• perf(computer-use): computer_action returns what changed, not the whole tree #36

written by claude opus 5.5 in claude code