Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion apps/docs/content/docs/browser-extension/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ description: Ask Sim about a page, dictate a message, and open your team’s go/
2. Review the attached page by clicking its chip. Use **×** to remove it.
3. Type your question and send it.

To ask about a passage, select it, right-click, and choose **Ask Sim about selection**. To add another page, open it and use **+ → Current page**. You can attach up to three snapshots; each stays as it was when captured.
To ask about a passage, select it, right-click, and choose **Ask Sim about selection**. To add another page, open it and use **+ → Current page**. Each attached page stays as it was when captured.

Choose **+ → Page access → Allow across websites** and approve Chrome’s prompt once to use pages across websites. Your draft stays unsent during setup. To change access later, choose **Page access → Manage in browser**.

Expand Down
3 changes: 2 additions & 1 deletion apps/extension/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ bun run --cwd apps/extension build

In Chrome, open `chrome://extensions`, enable Developer mode, choose **Load unpacked**, and select `apps/extension/dist`. Copy the extension ID into the Sim server's `BROWSER_EXTENSION_IDS` configuration. It accepts comma-separated Chrome extension IDs. Restart the app after changing that configuration. Unconfigured IDs cannot embed the app.

Click the Sim toolbar icon to open chat and attach the current page. **+ → Current page** adds a fresh snapshot; **Selected text** attaches the highlighted passage. Attachments appear as chips inside the composer. Attach up to three pages or selections to summarize, extract facts, or compare sources. To quote a passage, select it and choose **Ask Sim about selection** from the context menu. Review or remove the attachment before sending. **+ → Page access** offers **Allow across websites**. Approve the browser prompt once to use pages across HTTP and HTTPS sites without per-site setup. Manage access in the browser’s extension settings. Permission alone never enables background capture.
Click the Sim toolbar icon to open chat and attach the current page. **+ → Current page** adds a fresh snapshot; **Selected text** attaches the highlighted passage. Attachments appear as chips inside the composer. Attach pages or selections to summarize, extract facts, or compare sources. To quote a passage, select it and choose **Ask Sim about selection** from the context menu. Review or remove the attachment before sending. **+ → Page access** offers **Allow across websites**. Approve the browser prompt once to use pages across HTTP and HTTPS sites without per-site setup. Manage access in the browser’s extension settings. Permission alone never enables background capture.

For local development:

Expand All @@ -36,6 +36,7 @@ SIM_EXTENSION_DEV=1 SIM_EXTENSION_ORIGIN=http://localhost:3000 bun run --cwd app
- Optional HTTP/HTTPS access is requested together by **Allow across websites**, from a direct user action. Chrome's `activeTab` access is temporary and does not follow arbitrary new sites.
- Page capture runs on demand in Chrome's isolated world. It excludes editable fields, inputs, hidden elements, scripts, and embedded documents. It reads only the top frame. PDF viewers and canvas documents may have no accessible text.
- Extraction stops at 50,000 text characters, 150,000 serialized UTF-8 bytes, 15,000 visited nodes, or 150ms. The composer exposes truncation.
- Attached pages share a 150,000-character and 450,000-byte snapshot budget per message, within the general 100-context ceiling. Shorter pages can share the available budget without a separate page-count restriction.
- Source URLs omit credentials, query parameters, and fragments. Human-readable text can still contain sensitive information; it is the user-selected material they review and send into their conversation.
- Snapshots stay in memory until handed to Sim and acknowledged or expired after 30 seconds. Extension storage contains only a short-lived intent with a tab/document identity and a SHA-256 URL digest, never raw URLs, captured text, or credentials. Closing a tab or navigating invalidates its pending intent. Sim applies its normal conversation retention once a message is sent.
- No browser history, bookmarks, cookie access, debugger access, remote extension JavaScript, arbitrary network proxy, or background content collection is requested. Incognito use is disabled.
Expand Down
19 changes: 14 additions & 5 deletions apps/sim/lib/mothership/chat/page-context.ts
Original file line number Diff line number Diff line change
@@ -1,17 +1,26 @@
const MAX_PAGE_CONTEXTS = 3
import type { PageSnapshot } from '@sim/browser-extension-protocol'
import { MAX_CHAT_CONTEXTS } from '@/lib/mothership/chat/context-limits'

const MAX_PAGE_CONTEXT_TEXT_LENGTH = 150_000
const MAX_PAGE_CONTEXT_BYTES = 450_000
const encoder = new TextEncoder()

/** Bounds the combined page budget and refuses ambiguous snapshot identities within a turn. */
export function getPageContextBudgetError(
contexts: readonly { kind: string; snapshot?: { id: string } }[]
contexts: readonly { kind: string; snapshot?: PageSnapshot }[]
): string | undefined {
let count = 0
if (contexts.length > MAX_CHAT_CONTEXTS) return 'Too many attachments'
let textLength = 0
let bytes = 0
const ids = new Set<string>()
for (const context of contexts) {
if (context.kind !== 'browser_page') continue
if (++count > MAX_PAGE_CONTEXTS)
return `At most ${MAX_PAGE_CONTEXTS} page snapshots can be attached`
if (!context.snapshot) return 'A page snapshot is required'
if (ids.has(context.snapshot.id)) return 'Page snapshot IDs must be unique within a message'
ids.add(context.snapshot.id)
textLength += context.snapshot.text.length
Comment thread
waleedlatif1 marked this conversation as resolved.
Resolved
bytes += encoder.encode(JSON.stringify(context.snapshot)).byteLength
if (textLength > MAX_PAGE_CONTEXT_TEXT_LENGTH || bytes > MAX_PAGE_CONTEXT_BYTES)
return 'These pages contain too much text'
}
}
31 changes: 28 additions & 3 deletions apps/sim/lib/mothership/chat/process-contents.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -714,17 +714,42 @@ describe('processContextsServer - browser and terminal selections', () => {
}
)

it('refuses more than three page snapshots before resolving context', async () => {
it('delivers ten short page snapshots without dropping their sources', async () => {
const snapshots = Array.from({ length: 10 }, (_, index) => ({
...snapshot,
id: `11111111-1111-4111-8111-11111111111${index}`,
url: `https://example.com/page-${index}`,
text: `Source ${index}`,
}))
const result = await processContextsServer(
snapshots.map((page, index) => ({
kind: 'browser_page' as const,
label: `Page (${index + 1})`,
snapshot: page,
})),
'user-1'
)
expect(result).toHaveLength(10)
for (const [index, context] of result.entries()) {
const payload = context.content.split('\n').find((line) => line.startsWith('{'))
expect(payload && JSON.parse(payload)).toEqual(snapshots[index])
}
})

it.each([
{ name: 'combined text', text: 'x'.repeat(37_501) },
{ name: 'combined serialized bytes', text: '\u0000'.repeat(18_751) },
])('refuses an oversized page batch before resolving context: $name', async ({ text }) => {
await expect(
processContextsServer(
Array.from({ length: 4 }, (_, index) => ({
kind: 'browser_page' as const,
label: `Page ${index}`,
snapshot: { ...snapshot, id: `11111111-1111-4111-8111-11111111111${index}` },
snapshot: { ...snapshot, text, id: `11111111-1111-4111-8111-11111111111${index}` },
})),
'user-1'
)
).rejects.toThrow('At most 3 page snapshots')
).rejects.toThrow('These pages contain too much text')
})

it('points every browser and terminal mention at its exact tab', async () => {
Expand Down
4 changes: 2 additions & 2 deletions apps/sim/lib/mothership/chat/process-contents.ts
Original file line number Diff line number Diff line change
Expand Up @@ -163,11 +163,11 @@ export async function processContextsServer(
if (contexts.length > MAX_CHAT_CONTEXTS) {
throw new Error(`Context count exceeds the maximum of ${MAX_CHAT_CONTEXTS}`)
}
const pageBudgetError = getPageContextBudgetError(contexts)
if (pageBudgetError) throw new Error(pageBudgetError)
for (const context of contexts) {
if (context.kind === 'browser_page') pageSnapshotSchema.parse(context.snapshot)
}
const pageBudgetError = getPageContextBudgetError(contexts)
if (pageBudgetError) throw new Error(pageBudgetError)
if ((userMessage?.length ?? 0) > MAX_CHAT_MESSAGE_LENGTH) {
throw new Error(`Message exceeds the maximum of ${MAX_CHAT_MESSAGE_LENGTH} characters`)
}
Expand Down
19 changes: 11 additions & 8 deletions apps/sim/scripts/test-browser-extension-e2e.ts
Original file line number Diff line number Diff line change
Expand Up @@ -891,7 +891,7 @@ try {
}
}
await checkNavigation(
'Capture, preview, removal, and the three-page cap use the real composer',
'Capture, preview, and removal support ten pages in the real composer',
async () => {
await capture()
await expect(attachments()).toHaveCount(1)
Expand Down Expand Up @@ -1007,13 +1007,16 @@ try {
await focusBarrier.dispose()
await focusObserver.dispose()
}
await capture()
await capture()
await capture()
await expect(attachments()).toHaveCount(3)
await expect(frame().getByRole('alert').filter({ hasText: 'Remove a page' })).toBeVisible()
await screenshot('page-limit')
await attachments().first().getByRole('button', { name: 'Remove page context' }).click()
for (let count = 2; count <= 10; count++) {
await capture()
await expect(attachments()).toHaveCount(count)
}
await expect(frame().getByRole('alert').filter({ hasText: 'Remove a page' })).toHaveCount(0)
await screenshot('many-pages')
for (let count = 10; count > 2; count--) {
await attachments().first().getByRole('button', { name: 'Remove page context' }).click()
await expect(attachments()).toHaveCount(count - 1)
}
await expect(attachments()).toHaveCount(2)
const stored = await frame().evaluate(() => JSON.stringify(localStorage))
assert(
Expand Down
Loading