Skip to content
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -432,6 +432,11 @@ and versions follow [Semantic Versioning](https://semver.org/).
tell you when something happens, the plan no longer reports on every check, and its check no longer looks at only
the page's heading or first line (so an event added at the bottom is noticed). If the plan does either, the planner
is told why in plain words and writes it again. Asking for "a report every hour" or "the headline" still works.
- **A free "Subscribe" button no longer reads as a purchase.** When Sentient asks before clicking a newsletter,
follow or "notify me" button with no price nearby, the card now says it wants to sign you up, instead of "Make a
purchase" with "Yes, buy it". It still asks under your usual rules. A button with a price, a plan, a trial, an
upgrade or "/month" on it or next to it, or a plain "Subscribe" with nothing saying it is free, still reads as a
purchase and always asks.
- **Telling Sentient something changed updates the memory.** After "Mira starts school in June", saying "Mira's
school now starts in July" replaces June with July instead of being ignored as something Sentient already knew. The
same goes for a new place ("Ravi moved from Pune to Delhi"), a new number or a new name. Things that are only about
Expand Down
4 changes: 3 additions & 1 deletion desktop/src/features/chat/ApprovalCard.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import {
IconTerminal2,
IconTrash,
IconLock,
IconUserPlus,
IconWorldWww,
IconX,
type Icon
Expand Down Expand Up @@ -58,6 +59,7 @@ const RISK_ICONS: Record<RiskLabel, Icon> = {
'Home security': IconLock,
'Runs code': IconTerminal2,
Purchase: IconCreditCard,
'Signs up': IconUserPlus,
Deletes: IconTrash,
'Posts publicly': IconSend,
Submits: IconSend,
Expand All @@ -75,7 +77,7 @@ const RISK_ICONS: Record<RiskLabel, Icon> = {
* reads as a move to the trash, not a loss, and is checked first.
*/
export function effectiveRisk(
approval: Pick<ApprovalView, 'risk' | 'reason' | 'name' | 'arguments'> & Pick<Partial<ApprovalView>, 'preview'>
approval: Pick<ApprovalView, 'risk' | 'reason' | 'name' | 'arguments'> & Pick<Partial<ApprovalView>, 'preview' | 'riskLabel'>
): EffectiveRisk {
// a delete for good became its app's trash version (#142): it can come back, so it reads as a change, not a loss
if (approval.preview?.kind === 'trash' && approval.risk !== 'exec')
Expand Down
6 changes: 6 additions & 0 deletions desktop/src/features/chat/approvalRisk.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ export type RiskLabel =
| 'Home security'
| 'Runs code'
| 'Purchase'
| 'Signs up'
| 'Deletes'
| 'Posts publicly'
| 'Submits'
Expand Down Expand Up @@ -105,6 +106,8 @@ export function approvalRisk(approval: {
* data and never steers the wording.
*/
arguments?: Record<string, unknown>
/** The engine's own label for the call (§10 `describe_fn`), e.g. "Signs up" for a free newsletter button (#378). */
riskLabel?: string | null
}): RiskWording {
// A multi-execute tool's real action lives in its slugs; treat them like tool names.
const name = [approval.name, ...actionSlugs(approval.arguments)].join(' ')
Expand All @@ -114,6 +117,9 @@ export function approvalRisk(approval: {
if (home && (approval.risk === 'send' || approval.risk === 'exec'))
return { label: 'Home security', verb: home.verb, tone: 'danger', ...(home.caution ? { caution: home.caution } : {}) }
if (approval.risk === 'exec') return { label: 'Runs code', verb: 'run code on your computer', tone: 'danger' }
// A free subscribe, follow or "notify me" (#378): the engine saw no price or plan on or near the button.
if (approval.riskLabel === 'Signs up' && approval.risk !== 'read')
return { label: 'Signs up', verb: 'sign you up for something', tone: approval.risk === 'send' ? 'danger' : 'warning' }
if (approval.risk === 'send') {
if (isEmailTool(name))
return { label: 'Sends an email', verb: 'send an email', tone: 'danger' }
Expand Down
23 changes: 23 additions & 0 deletions desktop/tests/approvalRisk.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -177,3 +177,26 @@ test('Home Assistant locks, alarms and garage doors read as home security, with
assert.equal(blinds.label, 'Changes things')
assert.equal(blinds.caution, undefined)
})

// #378: a free subscribe the engine labels "Signs up" reads as a sign-up, not a purchase;
// a priced one keeps the purchase wording.
const click = (riskLabel: string) => ({
risk: 'send' as const,
name: 'browser_click',
reason: 'browser: Click the element with this ref. Buttons that buy, pay, send, post, delete or confirm need approval.',
arguments: { ref: 'e3' },
riskLabel
})

test('a free newsletter button reads as signing up', () => {
const w = approvalRisk(click('Signs up'))
assert.equal(w.label, 'Signs up')
assert.equal(w.verb, 'sign you up for something')
assert.equal(w.tone, 'danger')
})

test('a priced subscribe button still reads as a purchase', () => {
const w = approvalRisk(click('Purchase'))
assert.equal(w.label, 'Purchase')
assert.equal(w.verb, 'make a purchase')
})
9 changes: 8 additions & 1 deletion docs/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -1890,7 +1890,7 @@ Every new tool declares a `Risk`; approvals behave as in section 1.
Outside `run_loop`, use `await app.approvals.requires_approval(tool, arguments, ctx) -> (bool, Risk)`;
`needs_approval(tool, session_id, risk=None, *, arguments=None, ctx=None)` evaluates a synchronous `risk_fn` when
given `arguments` (an async one counts as at least `send` there).
- `approval_request` also carries `risk_label` (e.g. "Purchase", "Deletes", "Posts publicly", "Sends", "Runs code",
- `approval_request` also carries `risk_label` (e.g. "Purchase", "Signs up", "Deletes", "Posts publicly", "Sends", "Runs code",
"Changes something") and `target` (short label of what is acted on, e.g. "Place order", or null). They come from the
optional `Tool.describe_fn(arguments, ctx) -> {risk_label?, target?}` (sync or async, also `@tool(..., describe_fn=...)`);
without it `risk_label` is "Looks something up" / "Changes something" / "Sends" / "Runs code" by effective risk.
Expand Down Expand Up @@ -2272,6 +2272,13 @@ Every new tool declares a `Risk`; approvals behave as in section 1.
pay, place order, checkout, send, post, publish, delete, remove, confirm, subscribe, transfer, or it submits a form
with payment fields → `{ok, clicked, url, message, new_tab?, dialogs?}`. A link opening a new tab makes it active.
`confirm()` dialogs are accepted only for clicks approved as `send`; alerts are accepted; `dialogs` lists them.
Its approval `risk_label` is "Purchase" for buy, pay, order, checkout and the like. A subscribe, follow, sign up or
"notify me" reads "Signs up" (normal approval rules) only when the element or the text around it (its card or
form) says what free thing it is (newsletter, updates, waitlist, alerts, "free" ...) and shows no price, currency,
billing period ("/month", "per seat", an amount beside "monthly" or "annually"), plan, trial or upgrade. When that text stops early at another offer's button
and holds little besides the button itself, the free word is not enough. Anything else, a bare "Subscribe"
included, stays "Purchase", which always asks; a sign-up labelled "Purchase" is always rated `send`. A "Sign up"
with neither a free nor a money signal keeps its plain "Clicks" label and `write` risk.
Defensive check at call time: a send-looking click that approvals did not rate `send` (page changed, or no approval
step ran and approvals mode is not `off`) is refused with an explanation.
- `browser_type(ref, text, submit=False)` write (`send` when submit presses Enter in a payment form, a form whose button
Expand Down
81 changes: 75 additions & 6 deletions sentient/browser/safety.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@

Element descriptors are the dicts produced by the snapshot script (``snapshot.py``):
``{role, name, tag, type, value, href, autocomplete, id, name_attr, label, placeholder,
aria_label, title, in_form, form_payment, form_submit_text, is_submit}``. Missing keys are fine.
aria_label, title, in_form, form_payment, form_submit_text, is_submit, near, near_cut}``. ``near`` is the text around the
element (its card or form); ``near_cut`` says it stopped early at another offer. Missing keys are fine.
"""

from __future__ import annotations
Expand All @@ -33,14 +34,41 @@

# approval wording: first matching rule names the kind of action for the approval card
_LABEL_RULES = [
(re.compile(r"\b(?:buy|purchase|pay|payment|order|check ?out|donate|subscribe|book now|reserve|transfer|"
(re.compile(r"\b(?:buy|purchase|pay|payment|order|check ?out|donate|book now|reserve|transfer|"
r"add money|withdraw)\b", re.IGNORECASE), "Purchase"),
(re.compile(r"\b(?:delete|remove|erase|discard|trash|deactivate|close (?:my )?account)\b", re.IGNORECASE),
"Deletes"),
(re.compile(r"\b(?:post|publish|tweet|share)\b", re.IGNORECASE), "Posts publicly"),
(re.compile(r"\b(?:send|submit|confirm|reply)\b", re.IGNORECASE), "Sends"),
]
_DEFAULT_LABELS = {"click": "Clicks", "type": "Types into the page", "press": "Presses a key"}
SIGNUP_LABEL = "Signs up"

# subscribe, follow, notify me: "Signs up" only with a free signal and no money signal on or around it (#378)
_SIGNUP_RE = re.compile(
r"\b(?:subscribe|sign ?up|follow|notify me|get (?:the )?updates|"
r"join (?:the |our )?(?:newsletter|mailing list|wait ?list))\b",
re.IGNORECASE,
)
# anything that says it may cost money: a price or currency, a billing period, a plan, a trial or an upgrade
_PAID_RE = re.compile(
r"[$¢£¥€₹₩₽₺₫₱฿₦₪₴₨]"
r"|\b(?:usd|eur|gbp|inr|jpy|cad|aud|rs|rupees?|dollars?|euros?|pounds?)\b"
r"|/\s*(?:mo|mon|month|yr|year|wk|week|day)s?\b"
r"|\bper\s+(?:month|year|week|day|user|seat)\b|\b(?:a|each|every)\s+(?:month|year|week)\b"
# an amount next to a billing word ("499 monthly", "billed annually at 4,999"); "monthly newsletter" stays free
r"|\d[\d,.]*\D{0,20}?\b(?:monthly|yearly|annually|annual|quarterly|per\s+annum)\b"
r"|\b(?:monthly|yearly|annually|annual|quarterly|per\s+annum)\b\D{0,20}?\d"
r"|\b(?:plans?|trial|upgrade|premium|pricing|prices?|billed|billing|"
r"paid|pay|payment|buy|purchase|checkout|membership|pro)\b",
re.IGNORECASE,
)
# what a free sign-up says about itself; "Signs up" needs one of these, so a bare "Subscribe" stays a purchase
_FREE_RE = re.compile(
r"\b(?:free|news ?letters?|updates?|wait ?list|notify me|notifications?|follow(?:ing)?|digest|alerts?|"
r"mailing list|no spam|inbox|emails?|blog|podcast|channel)\b",
re.IGNORECASE,
)


def approval_target(el: dict | None) -> str:
Expand All @@ -54,10 +82,45 @@ def approval_target(el: dict | None) -> str:
return str(el.get("role") or el.get("tag") or "")[:80]


def risk_label_for(text: str, kind: str) -> str:
"""'Purchase', 'Deletes', 'Posts publicly', 'Sends', else a neutral label for the action kind."""
for rx, label in _LABEL_RULES:
if rx.search(text or ""):
def _partial_offer(text: str, near: str, near_cut: bool) -> bool:
"""True when the context stopped early and holds little besides the element itself: no whole card was read,
so a price may sit just out of reach."""
if not near_cut:
return False
rest = (near or "").replace(text, " ") if text else (near or "")
return len(re.sub(r"\W+", "", rest)) < 20


def paid_signup(text: str, near: str = "", near_cut: bool = False) -> bool:
"""True when ``text`` is a subscribe, sign up or follow that the approval card calls a 'Purchase'."""
text = text or ""
return bool(_SIGNUP_RE.search(text)) and not _LABEL_RULES[0][0].search(text) and (
risk_label_for(text, "click", near, near_cut) == _LABEL_RULES[0][1])


def risk_label_for(text: str, kind: str, near: str = "", near_cut: bool = False) -> str:
"""'Purchase', 'Signs up', 'Deletes', 'Posts publicly', 'Sends', else a neutral label for the action kind.

``near`` is the text around the element (its card or form). A subscribe, follow or "notify me" reads
'Signs up' only when the element or ``near`` says what free thing it is (a newsletter, updates, a waitlist,
alerts, "free" ...) and neither shows a price, currency, billing period, plan, trial or upgrade. Anything else
may cost money, so it is 'Purchase', which always asks: a bare "Subscribe" is a purchase. A "Sign up" with
neither a free nor a money signal keeps the plain label it had before (#378). ``near_cut`` says the context stopped
early (at another offer's button); then only a whole card of text can show it is free. A 'Purchase' label on a
sign-up comes with ``send`` risk (:func:`paid_signup`, :func:`click_risk`), so its hard stop can't be skipped."""
text = text or ""
purchase_rule, *other_rules = _LABEL_RULES
if purchase_rule[0].search(text):
return purchase_rule[1]
if _SIGNUP_RE.search(text):
context = f"{text} {near or ''}"
paid = _PAID_RE.search(context)
if _FREE_RE.search(context) and not paid and not _partial_offer(text, near, near_cut):
Comment thread
coderabbitai[bot] marked this conversation as resolved.
return SIGNUP_LABEL
if paid or RISKY_CLICK_RE.search(text):
return purchase_rule[1]
for rx, label in other_rules:
if rx.search(text):
return label
return _DEFAULT_LABELS.get(kind, "Changes something")

Expand Down Expand Up @@ -92,6 +155,9 @@ def click_risk(el: dict | None) -> Risk:
return Risk.send
if el.get("is_submit") and el.get("form_payment"):
return Risk.send
# a sign-up the approval card calls a purchase is at least send, so "Purchases always ask" holds (#378)
if paid_signup(approval_target(el), str(el.get("near") or ""), bool(el.get("near_cut"))):
return Risk.send
return Risk.write


Expand All @@ -107,6 +173,9 @@ def type_risk(el: dict | None, submit: bool) -> Risk:
label = " ".join(_norm(el.get(k)) for k in ("name", "aria_label", "label", "placeholder", "name_attr", "id"))
if _MESSAGE_FIELD_RE.search(label):
return Risk.send
context = " ".join([approval_target(el), str(el.get("form_submit_text") or "")])
if paid_signup(context, str(el.get("near") or ""), bool(el.get("near_cut"))):
return Risk.send
return Risk.write


Expand Down
4 changes: 3 additions & 1 deletion sentient/browser/service.py
Original file line number Diff line number Diff line change
Expand Up @@ -1418,7 +1418,9 @@ def describe_for(self, kind: str, arguments: dict | None) -> dict:
el = (self._snap or {}).get("refs", {}).get(ref) if ref else None
target = safety.approval_target(el)
context = " ".join([target, str((el or {}).get("form_submit_text") or "")]) if kind == "type" else target
return {"risk_label": safety.risk_label_for(context, kind), "target": target}
near = str((el or {}).get("near") or "")
cut = bool((el or {}).get("near_cut"))
return {"risk_label": safety.risk_label_for(context, kind, near, cut), "target": target}

def _eval_window(self) -> float:
return float(self.app.config.tools.approvals.timeout_s) + 120.0
Expand Down
39 changes: 39 additions & 0 deletions sentient/browser/snapshot.py
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,42 @@
const submitText = clip(submits.map(b => b.innerText || b.value || b.getAttribute('aria-label') || '').join(' | '), 120);
return { in_form: true, form_payment: payment, form_submit_text: submitText };
}
// The text around a sign-up looking element (its card or form), so a price beside a "Subscribe" button is seen (#378).
// Climbs at most four parents. It stops before a container holding another offer button (the next card on a pricing
// page); in a container with much more text than one card it keeps the text just before and after the element.
const SIGNUP = /\b(subscribe|sign ?up|follow|notify|updates|join|newsletter|wait ?list)\b/i;
const OFFER = /\b(subscribe|sign ?up|follow|notify me|join|buy|purchase|pay|order|check ?out|upgrade|trial|get started|choose|select)\b/i;
const CLICKABLE = 'button, a[href], input[type=submit], input[type=button], input[type=image], [role=button], [role=link]';
function otherOffer(node, el) {
return Array.from(node.querySelectorAll(CLICKABLE)).some(o => o !== el && !o.contains(el) && !el.contains(o) &&
OFFER.test(o.innerText || o.value || o.getAttribute('aria-label') || ''));
}
function windowText(node, el, before) {
const r = document.createRange();
if (before) { r.setStart(node, 0); r.setEndBefore(el); } else { r.setStartAfter(el); r.setEnd(node, node.childNodes.length); }
return r.toString().replace(/\s+/g, ' ').trim();
}
function nearText(el, d) {
const own = [d.name, d.aria_label, d.title, d.label, d.value, d.form_submit_text].join(' ');
if (!SIGNUP.test(own)) return { near: '', cut: false };
// cut: the climb stopped early (another offer button, or a container too big to read), so a price may be out of reach
let near = '', cut = false;
let node = el.parentElement;
for (let i = 0; i < 4 && node && node !== document.documentElement; i++) {
const huge = node === document.body || (node.textContent || '').length > 20000;
if (i > 0 && !huge && otherOffer(node, el)) { cut = true; break; }
const text = (node.textContent || '').length > 4000 ? '' : (node.innerText || '').replace(/\s+/g, ' ').trim();
if (!text || text.length > 400) {
// a larger card or section: keep the text just before and after the element inside it
if (huge) cut = true;
else near = [windowText(node, el, true).slice(-400), near, windowText(node, el, false).slice(0, 200)].join(' ');
break;
Comment thread
coderabbitai[bot] marked this conversation as resolved.
}
near = text;
node = node.parentElement;
}
return { near: clip(near, 700), cut };
}
const els = [];
let n = 0;
const max = args.maxElements || 300;
Expand Down Expand Up @@ -168,6 +204,9 @@
d.value = sel ? clip(sel.label || sel.text, 60) : '';
}
Object.assign(d, formInfo(el));
const around = nearText(el, d);
d.near = around.near;
d.near_cut = around.cut;
d.focused = document.activeElement === el;
els.push(d);
}
Expand Down
2 changes: 1 addition & 1 deletion sentient/tools/rules.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
RULE_TITLES = {
ALLOW: "Allow", IF_ASKED: "Only if I asked", ASK: "Always ask", HAND_OFF: "Hand it to me", NEVER: "Never",
}
# describe_fn label for money leaving the user (browser: buy, pay, order, subscribe, donate, transfer ...)
# describe_fn label for money leaving the user (browser: buy, pay, order, a priced subscribe, donate, transfer ...)
PURCHASE_LABEL = "Purchase"
# describe_fn labels for a call that opens the user's home (#175, ADR 0028): unlocking a door, turning off the alarm,
# opening a garage door, gate or door, or switching something that sounds like one. Like a purchase, it always asks.
Expand Down
14 changes: 14 additions & 0 deletions tests/browser/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,20 @@
"help.html": "<!doctype html><title>Help</title><main><h1>Help center</h1></main>",
"download.html": '<!doctype html><title>Downloads</title><a href="/report.txt" download>Download report</a>',
"report.txt": "Sentient browser download test.\n",
"signup.html": """<!doctype html><meta charset="utf-8"><title>Newsletter</title><main>
<section><h2>Weekly notes</h2><p>Free, no spam.</p><button>Subscribe</button></section>
<section><h2>Pro</h2><p>All the extras, ₹499/month.</p><button>Subscribe</button></section>
<section><h2>Team</h2><div>$20 per seat</div><div><button>Subscribe</button> <a href="/help.html">Learn more</a></div></section>
<section><h2>Studio</h2><p>""" + "Long notes on every plant we grow, with photos and care steps. " * 8 + """</p>
<p>Studio pass, ₹999/month.</p><div><button>Subscribe</button> <a href="/help.html">Learn more</a></div></section>
</main>""",
# a priced card whose context stops early: another offer button ("Choose") shares the row with Subscribe
"signup-choose.html": """<!doctype html><meta charset="utf-8"><title>Team</title><main>
<section><h2>Team plan</h2><p>₹2,999/month for five seats.</p>
<div><div><button>Subscribe</button></div><div><button>Choose</button></div></div></section>
<section><h2>Insider</h2><p>₹1,499/month, billed yearly.</p>
<div><div><button>Subscribe to the newsletter</button></div><div><button>Choose</button></div></div></section>
</main>""",
}


Expand Down
Loading
Loading