FR: evaluate agent-browser (native Rust, MCP stdio) as the managed browser tier backend #124

Activity

1. ttmuse commented on Nov 14, 2026

   

   ttmuse

   on Nov 14, 2026

   Author

   More actions

   Clarification: dual install as an option, not a mandatory bundle

   Following up on the proposal above — the intent is not to replace Playwright outright, and definitely not to bundle both backends by default. The suggestion is to allow both to be installed side by side, with a clear division of labour:

   Why coexistence is valuable

   • Complementary coverage. agent-browser fits the everyday agent workload: lightweight, fast to start, and its snapshot + @ref model maps closely to the existing ui_snapshot / ui_action ref model. Playwright remains the heavy-duty option for what agent-browser (Chromium/CDP only) does not cover: network interception, trace/video recording, Firefox/WebKit, and mature multi-context handling.
   • A real fallback while agent-browser is 0.x. Running both during a spike/evaluation period lets the same capability checklist be measured against both backends, so any later decision to deprecate one is based on data rather than a bet.
   • The architecture already supports it. Both are stdio MCP servers, so both can sit behind the same backend interface via the StdioMCPClient / ManagedProcess work in feat: stdio MCP client and managed Playwright tier (browser tiers phase 1, PR 2) #123 — this is configuration, not new infrastructure.

   Conditions that keep dual install from becoming a drawback

   1. Independent, opt-in install choices. Separate checkboxes on the wizard's Browser tools page: agent-browser pre-checked (single ~17 MB native binary), Playwright as an optional advanced tier. Pre-checking / bundling both by default would negate the install-size benefit that motivates this FR.
   2. Lazy, single-backend execution. Neither backend should run as a resident daemon by default; launch on demand via the managed-process layer, and route each task to exactly one backend with a deterministic rule (e.g. needs trace, network interception, or a non-Chromium engine → Playwright; otherwise → agent-browser), rather than leaving the choice to the model per call.
   3. Document the separate profiles. The two backends use separate persistent profiles, so login state is not shared and a user who enables both signs in twice. That should be stated on the wizard page so it is not filed as a bug later.

   In short: dual install is a good option and a bad default bundle. Suggested framing for the tier design in #118: agent-browser as the default managed tier, Playwright as an optional advanced tier, both supported behind the same interface during (and after) evaluation.
2. ttmuse commented on Nov 14, 2026

   

   ttmuse

   on Nov 14, 2026

   Author

   More actions

   Refinement: provider model — the agent selects the provider, the host validates and locks it in

   This refines the routing point in my previous comment. The cleaner model is not host-side rules deciding per call, and not two raw tool sets exposed side by side — it is a provider model, analogous to LLM providers behind one API:

   Shape

   • Both backends register as providers behind a single browser-driver interface. The agent sees one unified browser_* tool surface, not agent-browser's tools and Playwright's tools at the same time.
   • Each provider publishes a capability descriptor: supported engines (Chromium-only vs Chromium/Firefox/WebKit), install size / startup cost, trace & video support, network interception, profile location, and any tool-coverage gaps.
   • The agent selects a provider once, at session start, based on what the task needs — e.g. routine browsing and form work → agent-browser; a task that needs WebKit, trace/video, or network interception → Playwright. If the agent does not specify, a configured default (agent-browser) applies.

   Host responsibilities

   • Maintain the provider registry and only offer providers that are actually installed.
   • Validate the selection (installed, allowed by policy, healthy via the managed-process / stdio layer from feat: stdio MCP client and managed Playwright tier (browser tiers phase 1, PR 2) #123) and lock the session to that provider for its lifetime.
   • If the selected provider is missing or cannot serve a declared capability, fail explicitly at session start so the agent can start a new session on the other provider — never silently switch mid-session.

   Why selection is at session level, not per tool call

   Browser state — tabs, cookies/login, page state, and snapshot refs — lives inside one provider's browser instance and cannot migrate to the other. Exposing both raw MCP tool sets simultaneously (~60 tools, with different names and different ref formats) would bloat the tool surface and invite mid-task provider switches that invalidate all of that state. One selection, one provider, one unified tool surface per session avoids both problems while still letting the agent — which knows the task's requirements better than static host rules — make the choice.
3. ttmuse commented on Nov 14, 2026

   

   ttmuse

   on Nov 14, 2026

   Author

   More actions

   The model in one diagram: install = 2, expose = 1, execute = 1 per session

   Summarising the two comments above into the three-layer rule:

   • Install layer: 2 — both providers may be installed on the same machine at the same time, each with its own directory and persistent profile.
   • Expose layer: 1 — the agent is only ever shown one unified browser_* tool set. The two providers' own MCP tool sets are not both exposed as tools (that would double the tool surface, mix two naming/ref schemes, and invite mid-session provider switches).
   • Execution layer: 1 per session — the agent chooses one provider at session start; the host validates and locks the session to it. (The one-of-two choice is per session, not machine-global: two different sessions may use different providers, but a single session never uses both, because tabs, login state and snapshot refs live in the chosen provider's browser and cannot migrate.)

   flowchart TB
       Agent["Agent<br/>sees ONE unified browser_* tool set"]
       subgraph Host["oab-computer - Browser Driver Host | expose layer: 1 tool set"]
           direction LR
           Registry["Provider Registry<br/>only installed providers are listed"]
           Validate["Validate + session lock<br/>installed? allowed? healthy?"]
           Registry --> Validate
       end
       subgraph Providers["Providers | install layer: 2 may coexist | execution layer: 1 per session"]
           direction LR
           PA["Provider A - agent-browser (default)<br/>Chrome for Testing - own persistent profile"]
           PB["Provider B - Playwright (advanced)<br/>Chromium / Firefox / WebKit - own persistent profile"]
       end
       Agent -->|"1 - choose ONE provider at session start"| Registry
       Validate -->|"2 - selected: every browser_* call goes here"| PA
       Validate -.->|"not selected: never started"| PB
       Validate -.->|"validation fails: explicit error at start,<br/>agent opens a NEW session and re-chooses"| Agent
       PA -.->|"state - tabs / login / refs - cannot migrate"| PB
   

   Loading