Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -443,7 +443,7 @@ OpenTUI (`@opentui/core`) is the shipping shell; the Ink/React tree has been del
- **Runner host** (`runner-host.ts`) — Runner-facing mount: catalog assembly from live config, chrome pushes on session change, subagent observe resolution, and session teardown (quitting is Ctrl+C twice, owned by the shell).
- **Overlays and pickers** — Resume picker (`src/tui/pick-session.ts`) uses `runListModal` (`src/tui/list-modal.ts`). Slash-command surfaces (`/model`, `/settings`, `/permissions`, `/plugins`, etc.) route through `openCommandSurface` (`src/tui/command-surfaces.ts`).
- **Auto mode** — Toggled by CLI flags only (`--auto` / `--no-auto`); there is currently no in-session key bound to it.
- **Send-path ingest** — Submit, leftover send, and live steer run `ingestOperatorPrompt` (`src/tui/prompt-attachments.ts`): inline image-path mentions become attachments (`findImagePathMentions` / `imageAttachmentFromPath`); `@file` contents are inlined and `@dir` mentions become a directory summary (`resolveAtMentions` in `mention-resolution.ts`). The mention popup splices paths into the prompt (`spliceMentionCompletion`); it does not send. Ctrl+V / Ctrl+P (and `/paste-image`) attach a macOS clipboard PNG (`attachClipboardImage` in `shell.ts` → `readClipboardImage`); Cmd+V stays text (bracketed paste). Linux/Windows bitmap clipboard paste is not supported.
- **Send-path ingest** — Submit, leftover send, and live steer run `ingestOperatorPrompt` (`src/tui/prompt-attachments.ts`): inline image-path mentions become attachments (`findImagePathMentions` / `imageAttachmentFromPath`); `@file` contents are inlined and `@dir` mentions become a directory summary (`resolveAtMentions` in `mention-resolution.ts`). The mention popup splices paths into the prompt (`spliceMentionCompletion`); it does not send. Ctrl+V / Ctrl+P (and `/paste-image`) attach an image from the system clipboard (`attachClipboardImage` in `shell.ts` → `readClipboardImage`); Cmd+V stays text (bracketed paste). Image clipboard reads go through the OpenTUI clipboard service (PNG, JPEG, GIF, WebP) on platforms whose host backend supports them.

Known keybindings: `Ctrl+C` interrupts the in-flight run, and quits on a second press inside a two-second window.

Expand Down
8 changes: 4 additions & 4 deletions docs/IMPLEMENTATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -235,10 +235,10 @@ a path into the prompt (`spliceMentionCompletion`). Attachments land at send
time, not at accept time.

Ctrl+V / Ctrl+P and `/paste-image` call `attachClipboardImage` (`shell.ts`) →
`readClipboardImage` (`image-attachments.ts`), which reads a macOS clipboard
PNG via osascript `«class PNGf»`. Cmd+V is terminal bracketed paste and stays
text. Linux and Windows bitmap clipboard paste is a separate product gap; do
not treat bracketed-paste path text as a paste-time image gesture.
`readClipboardImage` (`image-attachments.ts`), which reads PNG, JPEG, GIF, or
WebP through the OpenTUI clipboard service. Cmd+V is terminal bracketed paste
and stays text. Do not treat bracketed-paste path text as a paste-time image
gesture.

## Configuration

Expand Down
8 changes: 4 additions & 4 deletions docs/TUI.md
Original file line number Diff line number Diff line change
Expand Up @@ -878,11 +878,11 @@ sent messages. Once a real `paste` event has fired even once, the fallback
heuristic is permanently skipped for the rest of the session
(`src/tui/shell/keys.ts`, the `sawBracketedPaste` guard).

Ctrl+V and Ctrl+P attach a PNG from the macOS clipboard
Ctrl+V and Ctrl+P attach an image from the system clipboard
(`attachClipboardImage` in `src/tui/shell/prompt.ts` → `readClipboardImage` in
`image-attachments.ts`).
Cmd+V stays text (bracketed paste above). Clipboard image attach is
macOS-only; Linux/Windows bitmap clipboard paste is not supported.
`image-attachments.ts`, which reads through the OpenTUI clipboard service).
Cmd+V stays text (bracketed paste above). The host clipboard backend reads
PNG, JPEG, GIF, and WebP where the platform supports image clipboard data.
`/paste-image` is the same attach path.

@-mention path completion opens a popup keyed off the `@token` under the
Expand Down
5 changes: 5 additions & 0 deletions src/tui/chrome-state.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -651,6 +651,11 @@ describe("chromeFromSession", () => {
expect(zones.agents?.[0]?.label).toContain("explorer");
});

test("copies a turn-level stall flag onto the live snapshot", () => {
expect(chromeFromSession({ stalled: true }).stalled).toBe(true);
expect(chromeFromSession({}).stalled).toBeUndefined();
});

test("falls back agent id; empty bags hide", () => {
const state = chromeFromSession({
tasks: [],
Expand Down
6 changes: 6 additions & 0 deletions src/tui/chrome-state.ts
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,9 @@ export interface ChromeLiveState {
readonly agentId: string;
readonly description: string;
} | null;
/** Turn-level stall (`isStalledForDisplay`). Absent means the host reads
* the live watchdog instead of this snapshot. */
readonly stalled?: boolean;
}

/** One rendered agents-panel row. `stalled` comes precomputed so the
Expand Down Expand Up @@ -513,6 +516,7 @@ export interface ChromeSessionInput {
readonly tasks?: readonly ChromeSessionTask[] | null;
readonly agents?: readonly ChromeSessionAgent[] | null;
readonly observe?: ChromeLiveState["observe"];
readonly stalled?: boolean;
}

/** Map real session shapes (tasks / subagent store) into ChromeLiveState.
Expand All @@ -521,11 +525,13 @@ export function chromeFromSession(input: ChromeSessionInput): ChromeLiveState {
const task = mapSessionTasks(input.tasks);
const agents = mapSessionAgents(input.agents);
const observe = input.observe ?? null;
const stalled = input.stalled;

return {
...(task !== undefined ? { task } : {}),
...(agents !== undefined ? { agents } : {}),
...(observe !== null && observe !== undefined ? { observe } : {}),
...(stalled !== undefined ? { stalled } : {}),
};
}

Expand Down
41 changes: 41 additions & 0 deletions src/tui/gate-wire.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -412,6 +412,47 @@ describe("operatorChoicesFromOptions / operatorResultFromSelection", () => {
});

describe("wireGates", () => {
test("notifies once when a permission gate reaches the screen", async () => {
const seen: string[] = [];
await withGates(
async ({ emitter }) => {
emitPermission(emitter, { id: "req-9" });
emitPermission(emitter, { id: "req-9" });
},
{
hooks: {
notify: {
notify: (message) => {
seen.push(message);
return true;
},
},
},
},
);
expect(seen).toEqual(["Approval needed"]);
});

test("notifies when an operator gate reaches the screen", async () => {
const seen: string[] = [];
await withGates(
async ({ emitter }) => {
emitOperator(emitter);
},
{
hooks: {
notify: {
notify: (message) => {
seen.push(message);
return true;
},
},
},
},
);
expect(seen).toEqual(["Input needed"]);
});

test("subscribes exactly permission.gate and operator.gate; dispose removes both", async () => {
await withGates(async ({ emitter, disposeGates }) => {
expect(emitter.listenerCount("permission.gate")).toBe(1);
Expand Down
20 changes: 20 additions & 0 deletions src/tui/gate-wire.ts
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,13 @@ export interface GateLifecycleHooks {
readonly onPrimaryOperatorAdmitted?: (event: OperatorGateEvent) => void;
/** A marked primary ask_operator gate settled. */
readonly onPrimaryOperatorSettled?: (event: OperatorGateEvent) => void;
/** Desktop/terminal notification. Fired once per gate id when that gate
* reaches the screen. Absent means no notification. */
readonly notify?: NotifierPort;
}

export interface NotifierPort {
notify(message: string, title?: string): boolean;
}

const NOOP_GATE_HOOKS: GateLifecycleHooks = {
Expand Down Expand Up @@ -275,6 +282,17 @@ export function wireGates(
// tell whether the host has since opened a newer gate; comparing the
// open-time capture against the current value closes only its own overlay.
let overlayGeneration = 0;
const notifiedGateIds = new Set<string>();

function notifyOnce(id: string, message: string, title: string): void {
if (notifiedGateIds.has(id)) return;
notifiedGateIds.add(id);
try {
hooks.notify?.notify(message, title);
} catch {
// A notifier that throws must not block the gate.
}
}

function openHost(open: () => void): void {
overlayGeneration++;
Expand Down Expand Up @@ -388,6 +406,7 @@ export function wireGates(
// screen, distinct from when it was raised (see
// PermissionRequest.markDisplayed).
ev.request.markDisplayed?.();
notifyOnce(ev.id, "Approval needed", "corbits");
if (ev.timeoutMs !== undefined) {
timer = setTimeout(() => {
autoDeny(ev.timeoutMessage ?? "approval timed out; request denied");
Expand Down Expand Up @@ -489,6 +508,7 @@ export function wireGates(

const open = (): void => {
openedGeneration = overlayGeneration;
notifyOnce(ev.id, "Input needed", "corbits");
if (ev.timeoutMs !== undefined) {
timer = setTimeout(() => {
autoCancel();
Expand Down
36 changes: 36 additions & 0 deletions src/tui/image-attachments.clipboard.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
import { describe, expect, test } from "bun:test";

import { readClipboardImage } from "./image-attachments.js";

const PNG = Buffer.from(
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg==",
"base64",
);

describe("readClipboardImage", () => {
test("turns clipboard image bytes into an attachment without a source path", async () => {
const result = await readClipboardImage(() =>
Promise.resolve({
ok: true,
mimeType: "image/png",
bytes: PNG,
}),
);
expect(result.ok).toBe(true);
if (!result.ok) return;
expect(result.attachment.path).toBeUndefined();
expect(result.attachment.contentType).toBe("image/png");
expect(result.attachment.name.startsWith("clipboard-")).toBe(true);
expect(result.attachment.data.byteLength).toBeGreaterThan(0);
});

test("passes a clipboard miss through", async () => {
const result = await readClipboardImage(() =>
Promise.resolve({ ok: false, reason: "no image found on the clipboard" }),
);
expect(result).toEqual({
ok: false,
reason: "no image found on the clipboard",
});
});
});
69 changes: 32 additions & 37 deletions src/tui/image-attachments.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@ import { homedir, tmpdir } from "node:os";
import { readFile, stat, unlink, writeFile } from "node:fs/promises";
import { spawn } from "node:child_process";
import type { MessageAttachment } from "@intx/types/runtime";
import {
readSystemClipboardImage,
type ClipboardImageBytes,
} from "./system-clipboard.js";

export const MAX_IMAGE_ATTACHMENT_BYTES = 10 * 1024 * 1024;

Expand Down Expand Up @@ -335,48 +339,39 @@ function replaceExtension(name: string, contentType: string): string {
return `${dot === -1 ? name : name.slice(0, dot)}.${ext}`;
}

export async function readClipboardImage(): Promise<ClipboardImageResult> {
if (process.platform !== "darwin") {
export async function readClipboardImage(
source: () => Promise<ClipboardImageBytes> = readSystemClipboardImage,
): Promise<ClipboardImageResult> {
const read = await source();
if (!read.ok) return read;

const ext = extensionForMime(read.mimeType);
const tmpPath = join(
tmpdir(),
`corbits-clipboard-${process.pid}-${Date.now()}.${ext}`,
);
try {
await writeFile(tmpPath, read.bytes);
const attachment = await imageAttachmentFromPath(tmpPath);
if (!attachment.ok) return attachment;
const { path: _path, ...clipboardAttachment } = attachment.attachment;
return {
ok: false,
reason: "clipboard image paste is currently supported on macOS",
ok: true,
attachment: {
...clipboardAttachment,
name: `clipboard-${new Date().toISOString().replace(/[:.]/g, "-")}.${ext}`,
},
};
}

const tmpPath = `/tmp/corbits-clipboard-${process.pid}-${Date.now()}.png`;
const script = `
set outPath to POSIX file ${JSON.stringify(tmpPath)}
try
set pngData to the clipboard as «class PNGf»
set outFile to open for access outPath with write permission
set eof of outFile to 0
write pngData to outFile
close access outFile
on error errMsg
try
close access outPath
end try
error errMsg
end try
`;

const result = await runProcess("osascript", ["-e", script]);
if (result.code !== 0) {
} catch {
return { ok: false, reason: "could not read an image from the clipboard" };
} finally {
await unlink(tmpPath).catch(() => undefined);
return { ok: false, reason: "no PNG image found on the clipboard" };
}
}

const attachment = await imageAttachmentFromPath(tmpPath);
await unlink(tmpPath).catch(() => undefined);
if (!attachment.ok) return attachment;
const { path: _path, ...clipboardAttachment } = attachment.attachment;
return {
ok: true,
attachment: {
...clipboardAttachment,
name: `clipboard-${new Date().toISOString().replace(/[:.]/g, "-")}.png`,
},
};
function extensionForMime(mimeType: string): string {
if (mimeType === "image/jpeg") return "jpg";
return mimeType.split("/")[1] ?? "img";
}

export function formatAttachmentSummary(
Expand Down
Loading
Loading