Skip to content

feat(assistant): resolve chat links to stable canvas identities (#333) - #389

Open
Xingkai98 wants to merge 10 commits into
ThinkFlowLab:mainfrom
Xingkai98:feat/333-focusnode
Open

Xingkai98 wants to merge 10 commits into
ThinkFlowLab:mainfrom
Xingkai98:feat/333-focusnode

Conversation

@Xingkai98

Copy link
Copy Markdown
Contributor

Summary

reply → provenance/title/identity resolver → validated stored ChatCanvasLink
      → project-reference renderer → current-project focus callback
                                  → foreign-project destination hint

Agent replies can already contain Markdown links, but a relative destination
such as [Finished video](output.mp4) has no project meaning. This change adds
shared resolution and a project-reference renderer that turn such a destination
into a clickable canvas identity. Existing panel/workspace wiring consumes the
stable ID; full viewport behavior still needs canvas-level verification.

The resolver matches a relative filename against the turn's asset provenance
and the canvas items that display the asset, and also accepts explicit
canvas:<uuid>, node:<uuid> and project:<uuid>/canvas:<uuid> targets. It
returns typed outcomes (focused, hidden, materialize, ambiguous,
missing, cross-project, external). Only stable IDs are stored; position,
title and group membership are resolved at click time, so moving, renaming or
regrouping never invalidates a saved link. External URLs keep the existing link
policy and are not persisted as project references.

Existing AssistantReply, AssistantPanel and WorkspaceView components are
modified. The renderer preserves external-link policy and displays a bilingual
destination hint for cross-project references without focusing a local ID.

Changes

  • packages/core/src/opendirector/agent/canvas_links.py (new): Markdown link
    extraction that ignores fenced/inline code, target normalization, provenance
    aware asset→canvas-item resolution, ambiguity/cross-project/materialize
    handling, canonical link formatting, and per-message enrichment.
  • apps/api/src/opendirector_api/agent.py: at turn end, read the final project
    snapshot and persist canvasLinks on the assistant message. Resolution is
    best-effort and never fails an accepted turn.
  • src/backend/chat.ts: use the generated ChatCanvasLink payload type.
  • AssistantReply renders resolved project anchors; AssistantPanel forwards
    navigation to the existing workspace consumer.
  • src/i18n/messages/zh-CN/assistant.ts,
    src/i18n/messages/en-US/assistant.ts: four bilingual hint keys for
    outcomes that cannot focus a node (materialize, ambiguous, missing,
    cross-project). Focused and hidden links focus their node instead of showing
    a hint, so they have no message key.
  • tests/test_canvas_links.py (new): resolution, ambiguity, hidden/materialize,
    cross-project, code-fence, stable-identity and persistence round-trip tests.
  • tests/canvasLinkMessages.test.ts (new): both catalogs resolve the new keys
    without leaking the other language.

The navigation result is generated from a Python domain contract and imported by the frontend. Persisted identities let reply links select and frame visible canvas nodes.

Verification

  • Before: the jsdom fixture tests/canvasLinkRendering.test.mjs reported
    Finished note [blocked], anchors=0 and a cross-project focus:same-id.
    After (current HEAD): that fixture was deleted in 1ab7e5bd; there is no
    browser/DOM-level render or click coverage. Click behavior is covered only at
    the logic level by tests/canvasLinkClick.test.ts (canvasLinkClickAction),
    which asserts focus vs. translated hint keys and that a colliding id is never
    focused. The historical "2 passed" result is not reproducible on HEAD.

Review-r2 repaired tree (starting at 2f275325); paired evidence in
reviews/review-r2.md:

  • pytest tests/test_canvas_links.py -q → 34 passed (existing /tmp/odstubs shim path; media/font inspection is not exercised).
  • pytest tests/test_assistant_conversations.py tests/test_agent_citations.py -q
    → 33 passed.
  • pytest tests/test_agent_gateway.py tests/test_agent_tools_voices.py tests/test_creation_contracts.py tests/test_capability_contract.py -q
    → 253 passed.
  • node --test tests/referenceMentions.test.ts tests/assistantCitations.test.ts tests/i18n.test.ts tests/canvasLinkMessages.test.ts
    → 20 passed.
  • npm run check:i18n → 3020 keys, 0 problem(s).
  • python3 scripts/generate_contracts.py --check → Creation contracts verified;
    base-relative contract diff → exit 1 (2 files, 499 insertions, 2 deletions).
    Generator consistency passes; the functional diff is intentionally nonzero.
  • UI render/click: no browser/DOM-level coverage on HEAD.
    tests/canvasLinkRendering.test.mjs was deleted in 1ab7e5bd; only the
    logic-level tests/canvasLinkClick.test.ts remains. The earlier "2 passed"
    and Chromium checks are historical and not reproducible on HEAD.
  • Mutation testing: 10 Python mutants caught by
    scripts/verify_canvas_link_mutations.py. The two historical UI mutants are
    not reproducible on HEAD because their fixture
    (tests/canvasLinkRendering.test.mjs) was deleted; there is no current UI
    mutation evidence. The older six-kill claim is likewise not reproducible on
    2f275325 (M2 survived; M4 source absent).

Environment limitation: the existing linked node_modules is
incomplete (no react, no mediabunny), so npm test reports 9 failures,
including an ERR_ASSERTION in tests/browserHost.test.ts:121 and a
MODULE_NOT_FOUND in tests/modelBrands.test.ts:28 (historical full-suite run).
Before repair tsc reports 5592 diagnostics, including
src/canvas/AssistantReply.tsx(1,41): TS2307; the repaired count is 5600.
Typechecking fails, including changed UI files. No packages were installed for
this repair; component/browser fixtures reuse the reviewer's existing dependency tree.

Evidence pending real runtime/browser execution

The following are pending, not claimed as passing:

  1. pending — Full canvas selection/highlight/viewport framing. Rendered
    anchors and their focus callback are verified separately.
  2. pending — English and Chinese browser screenshots of the link outcomes
    (missing, ambiguous, hidden/reveal, materialize, cross-project).
  3. pending — Streamed and completed replies through OpenCode, OpenJiuwen,
    Codex, dsh and Bundled real sessions.
  4. pending — Editor-backed subject/timeline/stack/tool/director objects
    resolved through real project fixtures.
  5. pending — Full CI (npm ci, npm run build, full npm test, full
    Python suite) and witnessloop check (the repository is not integrated; see
    the working notes).

Merge Danger

Door: two-way. No migration or destructive project operation is added.

Blast Radius: assistant. Turn-scoped filename links now require matching
provenance; historical references must use stable IDs. Full canvas viewport and
runtime-matrix acceptance remain pending.


中文对照

标题

feat(assistant): 把对话链接解析为稳定的画布对象身份 (#333)

摘要

Agent 回复里可以出现 Markdown 链接,但 [Finished video](output.mp4) 这样的
相对路径本身没有工程含义。本次改动补上共享解析与项目引用渲染器,把目标解析为
稳定画布身份;现有面板和工作区已接线,完整画布取景仍需验证。

解析器会用本轮执行的素材溯源信息,把相对文件名匹配到显示该素材的画布对象;同时
支持显式 canvas:<uuid>、node:<uuid> 和 project:<uuid>/canvas:<uuid>。
结果是带类型的(focused、hidden、materialize、ambiguous、missing、
cross-project、external)。持久化只保存稳定 ID;位置、标题和分组关系在点击
时再解析,因此移动、重命名或重新分组都不会让已保存的链接失效。外部链接继续走
既有策略,不会作为工程引用持久化。

已修改现有 AssistantReply、AssistantPanel 和 WorkspaceView。跨工程引用显示
双语提示与工程 ID,不会聚焦当前工程中碰巧同 ID 的对象。

改动

  • 新增 packages/core/src/opendirector/agent/canvas_links.py:解析 Markdown
    链接(忽略围栏代码和行内代码)、归一化目标、按素材溯源匹配画布对象、处理
    歧义/跨工程/待上画布、规范化链接格式、按消息附加解析结果。
  • apps/api/src/opendirector_api/agent.py:回合结束时读取最终工程快照,把
    canvasLinks 持久化到 assistant 消息;解析失败不影响已接受的回合。
  • src/backend/chat.ts:使用生成的持久化字段类型;现有回复渲染器、面板与工作区消费该身份。
  • 中英文 assistant.ts:新增 4 个无法聚焦时的提示文案键(materialize、
    ambiguous、missing、cross-project)。focused/hidden 链接直接聚焦节点、
    不显示提示,因此没有对应文案键。
  • 新增 tests/test_canvas_links.py 和 tests/canvasLinkMessages.test.ts。

相对 base 有生成契约与既有 UI 组件变更;本次修复没有手改生成文件,也没有修改 CI 或依赖文件。

验证

  • 后端定向测试:34 passed(现有 /tmp/odstubs shim 路径,不代表真实媒体检查)。
  • 助手/引用回归:33 passed。
  • Agent 网关、工具、契约回归:253 passed。
  • 新增及既有 Node 测试:20 passed。
  • npm run check:i18n:0 problem(s)。
  • 契约生成检查:Creation contracts verified;相对 base 的功能 diff 非零(2 文件,499 增、2 删)。
  • UI 渲染/点击:HEAD 上无浏览器/DOM 级覆盖;tests/canvasLinkRendering.test.mjs
    已在 1ab7e5bd 删除,仅保留逻辑层 tests/canvasLinkClick.test.ts。此前的
    “2 passed”与 Chromium 检查属于历史记录,HEAD 不可复现。
  • 当前实现的变异:scripts/verify_canvas_link_mutations.py 捕获 10 个 Python 变异;
    历史 2 个 UI 变异因其 fixture 已删除而无法复现,HEAD 上没有 UI 变异证据。
    历史“6 个全捕获”证据不适用于 2f275325。

需要真实 runtime/浏览器证据的项(一律 pending,不声称已通过)

  1. pending — 完整画布选中/高亮/取景;渲染链接与聚焦回调另有通过证据。
  2. pending — 中英文界面下各结果(missing、ambiguous、hidden/reveal、
    materialize、cross-project)的浏览器截图。
  3. pending — OpenCode、OpenJiuwen、Codex、dsh、Bundled 的真实流式与完成
    回复验证。
  4. pending — 由真实编辑器 fixture 生成的 subject/timeline/stack/tool/
    director 对象解析验证。
  5. pending — 完整 CI(npm ci、npm run build、完整前端与 Python 套件)
    以及 witnessloop check(仓库尚未接入,见工作记录)。

🤖 Generated with Claude Code

Xingkai98 and others added 10 commits November 16, 2026 13:19
…kFlowLab#333)

Add a shared backend resolver for assistant Markdown links. Relative
destinations are matched through the turn's asset provenance and the
canvas items that display the asset; explicit canvas/node/project links
are also supported. Results are typed (focused, hidden, materialize,
ambiguous, missing, cross-project, external) and store only stable IDs so
moving, renaming or regrouping cannot invalidate them.

Resolved links are persisted as canvasLinks on the assistant message at
turn end. Link resolution never fails an accepted turn, and external
links keep the existing policy. Add bilingual outcome copy and tests for
resolution, ambiguity, code fences, persistence and mutation coverage.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
M1: 目录限定链接按完整 name/path 精确匹配,不再静默退化为 basename。
L1: @提及允许标题内标点,仅剥除后随空白/行尾的尾部标点;标题解析匹配最长词前缀。
L2: R1 变异选择器指向重复文件名用例,使其在新行为下红、旧行为下绿。
M2: 将 jsdom 依赖的 UI 点击用例退化为逻辑层断言(canvasLinkClickAction),纳入 node --test;UI 点击未做浏览器级覆盖。

Co-Authored-By: Claude Code <noreply@anthropic.com>
M1: @提及在句末标点后跟空白时按提及边界截断,@finished note. Next step
    等不再解析为 missing;标题内句点(Chapter 2.1)保持完整。
L1: 前端 renderCanvasMentions 对齐后端 (?<!\w) 词边界,foo@Title 与锚点内
    URL 不再被包成 canvas-reference;抽取纯函数并补 node --test 覆盖。
L2: link_targets 过滤空/纯空白 href,避免 [x]() 持久化为 missing 噪声。
M2: 补经 AgentTools 提交 generation.submit 的集成用例,断言产物
    provenance.agentRunId == turn_id 且相对链接命中该产物。

Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
- 未截断的原始提及先做精确标题匹配,未命中再按标点/空格回退,
  避免 @dr. Smith 被截成 @dr 并跳到错误节点
- 全角后置标点(、”…—)」』】》〉)无条件终止提及
- 中文无分隔符场景用已知标题最长前缀解析(仅 CJK 续写)
- materialize/ambiguous 提示改为陈述句,去掉无对应动作的指令措辞
- 抽出 canvasReferences 插件并补 prepare→render→点击映射测试
- 补前一条 H/M 的中句、短前缀共存回归用例

Co-Authored-By: Claude Code <noreply@anthropic.com>
- 先删尾部普通词再做标点截断,@dr. Smith says hello 不再退回 @dr
- 标点前缀按长度从长到短尝试,@dr. Smith. 命中完整标题
- missing 提及的目标按首个边界截断,避免前端把后续句子渲染成链接
- 补对应回归用例

Co-Authored-By: Claude Code <noreply@anthropic.com>
- 用快照中的已知标题做最长前缀匹配,替代标点/空格候选列表
- 边界校验:精确、句末标点、全空白后纯普通词、或紧邻 CJK
- 更长但边界不可用的已知标题会遮蔽短标题,避免 @dr. Smith 99
  回退到 Dr、@标题生成 99 回退到 标题
- 修复 CJK 紧邻完整标题时被短前缀抢先解析的问题
- 补对应回归用例

Co-Authored-By: Claude Code <noreply@anthropic.com>
@github-actions github-actions Bot added area:agent Agent runtime, assistant chat and tools area:canvas Canvas, nodes and direct manipulation area:i18n Localisation and bilingual UI type:feat New capability or behaviour labels Nov 17, 2026
# Link resolution is navigation metadata; it must not
# fail a turn whose accepted result is already saved.
logger.exception('agent.canvas_links.failed project_id=%s turn_id=%s',
project_id, value.id)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:agent Agent runtime, assistant chat and tools area:canvas Canvas, nodes and direct manipulation area:i18n Localisation and bilingual UI type:feat New capability or behaviour

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants