feat(gateway): declare image input for gateway and custom provider models

CHANGELOG.md

@@ -41,6 +41,18 @@ file is the human-readable summary of what changed and why.
 
 ### Provider setup
 
+- **Images reach the gateway instead of being dropped** — the provider block
+  written by the netGate configure flow and the custom-provider flow now
+  declares `attachment: true` and `modalities: { input: ["text", "image"] }`
+  for every model it lists. A config model entry with no modalities resolves to
+  text-only, so an attached image used to be replaced with an "ERROR: Cannot
+  read image" note before the request and the model would answer that it can't
+  see anything. Every gateway model is now declared vision-capable, including
+  text-only ones: for those the endpoint rejects or ignores the image rather
+  than the client silently dropping it. Configs written before this change need
+  one more pass through the configure flow (or a CoDev Hub config rewrite,
+  which does the same since hub PR #219) to pick the declaration up.
+
 - **The gateway provider is now netGate** — the SSO configure flow (and its
   manual key-paste branch) writes the provider block as `provider.netgate`
   with display name `netGate` instead of `provider.aigateway` / `AI Gateway`,

DIVERGENCES.md

@@ -155,6 +155,7 @@ from the squash message first, or use a plain merge.
 | `packages/core/test/ripgrep.test.ts` | `120_000` timeout on both `it.live` tests | First use downloads + extracts the ripgrep binary; GitHub-hosted Windows runners blow bun's 5s default. | Keep our timeouts; take upstream's test changes. |
 | `packages/opencode/test/cli/run/run-process.test.ts`, `packages/opencode/test/cli/acp/lifecycle.test.ts`, `packages/opencode/test/server/httpapi-file.test.ts`, `packages/opencode/test/server/httpapi-sdk.test.ts`, `packages/opencode/test/effect/runner.test.ts`, `packages/opencode/test/session/prompt.test.ts`, `packages/opencode/test/project/instance-bootstrap.test.ts` | Slow-runner headroom: unknown-model test `timeoutMs`/bound 15s → 30s (test timeout 60s); the two unknown-stream-finish `codev run` subprocess tests ("unknown stream finish preserves partial output…", "--format json records partial output for an unknown stream finish") `timeoutMs` 30s → 60s and test timeout 60s → 120s (pure-upstream spawn tests that pay `codev run` cold-start; neither asserts on duration, so headroom only); the three tool-executing `codev run` subprocess tests ("prints each completed text part in order around a tool continuation", "--format json preserves reasoning, tool, and continuation ordering", "rejects requested permissions by default and allows them with the dangerous flag") `timeoutMs` 30s (harness default) → 60s explicitly, test timeout 60s → 120s — 240s for the permissions test, which pays three sequential `codev run` spawns; these pay cold-start *plus* a real `bash` tool spawn and a continuation turn, and were timing out at exactly the 30s harness default on loaded runners; none asserts on duration, so headroom only; ACP stdin-EOF exit timeout 5s → 30s; file-search-index poll 5s → 120s with a 150s test timeout (the search-endpoints test pays ripgrep's first-use download on Windows runners, like ripgrep.test.ts); Runner fiber-coordination `Effect.timeout` guards 250ms/1s → 10s (pure hang-detectors on already-unblocked awaits; Windows runners starve the event loop past 250ms); prompt.test.ts's two `sleep`-based loop/shell tests ("loop waits while shell runs…", "shell completion resumes queued loop callers") moved from `it.instance` to the file's own `unix` helper so they skip on Windows like their `unixNoLLMServer` siblings (`sleep` isn't on Windows, so the shell fiber hangs to the budget — a genuine hang, not slowness); instance-bootstrap's first test 30s → 60s (pays instance cold-start) | Loaded GitHub-hosted runners spawn processes, fetch ripgrep, and build the lazy file index slower than upstream's Blacksmith runners; 250ms–30s budgets flake or fail outright, and Unix-shell tests can't run on Windows at all. Hang regressions are still caught (a hang exceeds the bound; polls return as soon as ready). | Keep our numbers and the `unix` gating; take upstream's test changes. |
 | `packages/opencode/test/tool/external-directory.test.ts`, `packages/opencode/test/tool/read.test.ts` | Windows path-variant tests keep the drive letter in the `alt` form (drop the `.replace(/^[A-Za-z]:/, "")` step) | A drive-less path is drive-relative: it resolves against the CWD's drive (D: on GitHub-hosted Windows) while the temp dir lives on C:, so the file genuinely isn't there. Slash + case variants still exercise normalization. | Keep our `alt` construction; take upstream's test changes. |
+| `packages/opencode/test/provider/provider.test.ts`, `packages/opencode/test/provider/transform.test.ts` | Appended two fork-only blocks at the end of each file: an `it.instance` ("declared image modality makes a custom-provider model attachment capable") asserting a config model with no `modalities` resolves `attachment`/`input.image` to false while one carrying the `@codev/gateway` block resolves both true, and a `describe` ("ProviderTransform.message - image input on a custom provider") asserting `unsupportedParts` passes `image`/`file` parts through when the model declares image input and swaps them for the `ERROR: Cannot read …` note when it doesn't. No upstream test changed. | These pin the two upstream code paths the fork's config writer depends on (`provider.ts` modality→capability resolution, `transform.ts` part filtering); upstream leaves both untested, so a refactor there would silently re-break gateway image attachments. | Keep our appended blocks; take upstream's test changes. Append-only, so conflicts should be end-of-file context alone. |
 | `packages/opencode/test/server/httpapi-exercise/environment.ts` | Exercise config/data dirs `config/opencode`,`data/opencode` → `…/codev`; seeds `codev.json` with `{"share":"manual"}` | Follows the `global.ts` app-dir rename (row above); the fork's `share: "disabled"` default (config.ts row) would 500 the share/unshare scenarios, so the isolated environment opts back in (network stays stubbed by `CODEV_DISABLE_SHARE`). Seed file follows the config-file rename row (global `config.json` is no longer read). | Keep our dir names + seeded config; take upstream's surrounding changes. |
 | `sdks/vscode/package.json`, `sdks/vscode/src/extension.ts`, `sdks/vscode/README.md` | `name` `opencode` → `codev-code` (the extension ID is `publisher.name`); `displayName` "opencode" → "CoDev Code"; `publisher` `sst-dev` → `CoDev` (PLACEHOLDER — no such Marketplace publisher is registered; the mixed-case `CoDev` also isn't a valid Marketplace publisher id, which must be lowercase — fine for the sideloaded `.vsix` the fork ships, but replace with the real fork-owned lowercase publisher id before any `vsce publish`, and keep it in sync with the CLI auto-install target); `description` "opencode for VS Code" → "CoDev Code for VS Code"; `icon` `images/icon.png` is a fork-local generated PNG of the CoDev "c" badge (brand teal `#0891B2` on `#0a0a0a`, matching the `codev --mini` splash), with `images/icon.svg` as its source of record — replacing the earlier symlink to the shared `packages/identity` bracket mark; the two per-theme command `icon` objects (`{light,dark}` → `images/button-{dark,light}.svg`) collapse to a single `"images/button.svg"` string and `extension.ts`'s terminal `iconPath` object collapses to a single `Uri`, both pointing at a fork-local transparent teal `#0891B2` "c" glyph that reads on light+dark themes (replacing the two `packages/identity` bracket-mark symlinks, now deleted); `repository.url` → `github.com/quickbeard/codev-code`; command IDs `opencode.*` → `codev.*` across `contributes.commands`/`menus`/`keybindings` and the `registerCommand` calls; command/keybinding titles "Open opencode…"/"Run opencode"/"opencode: Insert At-Mentioned" → "Open CoDev Code…"/"Run CoDev Code"/"CoDev Code: Insert At-Mentioned" (keys unchanged); `TERMINAL_NAME` `opencode` → `codev` and the terminal launches `codev --port <port>`; scripts gain `"vsix"` (`vsce package --no-dependencies --skip-license -o dist/`) with `@vscode/vsce` added as a pinned devDependency; README body rebranded and trimmed to the details-panel essentials — CoDev Code intro linking the fork repo, `npm install -g codev-code` prerequisite, and feature prose only; upstream's Support and Development/Making-Changes sections are dropped (they're contributor-facing, not end-user extension description) | The fork ships this extension as an installable `.vsix`, so every user-visible surface — palette titles, command IDs, terminal name, launched binary, details-panel README — must say codev, and the launched binary must be the fork's `codev` CLI. Distinct `codev.*` IDs also avoid clashing with upstream's extension if both are installed. `--no-dependencies` because the bundle is self-contained (no runtime deps, bun-managed lockfile); `--skip-license` because `sdks/vscode` has no LICENSE file and vsce aborts without it. | Keep our name + displayName + publisher + repository + description, command IDs + titles, `TERMINAL_NAME`, the `codev --port` launch line, the `vsix` script + `@vscode/vsce` devDependency, and the trimmed README (branding + dropped Support/Development sections); take upstream's extension-logic and manifest-structure changes. |
 | `packages/opencode/src/ide/index.ts` | IDE auto-install target `sst-dev.opencode` → `CoDev.codev-code` (`publisher.name` from `sdks/vscode/package.json`) | The CLI must install the fork's extension ID (`publisher.name` from `sdks/vscode/package.json`), not upstream's Marketplace listing. `Ide.install` currently has no runtime caller (only its test imports the module), and the install only resolves once the extension is actually published — until then the local `.vsix` is the supported install path. | Keep our extension ID (and re-sync it if the registered publisher id changes); take upstream's install/detection logic changes. |

packages/codev-gateway/src/config-block.ts

@@ -1,14 +1,32 @@
 import { GATEWAY_COMPACT_RESERVED, GATEWAY_CONTEXT_WINDOW, GATEWAY_MAX_OUTPUT_TOKENS } from "./const"
 import { GATEWAY_PROVIDER } from "./provider"
 
+// A config model entry with no `modalities` resolves image input to false, and
+// the session then swaps an attached image for an "ERROR: Cannot read image"
+// note before the request instead of sending it. Declare image input so
+// attachments reach the endpoint; a text-only model there rejects or ignores
+// them, which beats the client silently dropping them. `attachment` is what
+// lets the TUI attach files in the first place. The annotation keeps the arrays
+// from widening to string[], which the config schema's modality literals reject.
+type Modality = "text" | "image"
+
+const MODEL_MODALITIES: { attachment: boolean; modalities: { input: Modality[]; output: Modality[] } } = {
+  attachment: true,
+  modalities: {
+    input: ["text", "image"],
+    output: ["text"],
+  },
+}
+
 // The exact codev.json fragment the codev hub writes for CoDev Code (hub
 // src/lib/configure.ts configureOpenCodeKind), so hub-managed and
 // self-configured installs are indistinguishable — the hub marks a config as
 // CoDev-managed by probing `provider.<id>` against its candidate set (saved
 // id, `netgate`, `ai-gateway`, legacy `aigateway`; hub PR #212), so hubs older
 // than that PR cannot attribute a `netgate` config. The per-model `limit` is
 // mandatory: without it opencode sizes the context window to 0 and disables
-// auto-compaction.
+// auto-compaction; the modality block is mandatory for the same reason (hub PR
+// #219) — see MODEL_MODALITIES.
 export function buildGatewayConfig(input: { apiKey: string; baseUrl: string; model: string; models?: string[] }) {
   const ids = [...new Set([input.model, ...(input.models ?? [])])]
   return {
@@ -30,6 +48,7 @@ export function buildGatewayConfig(input: { apiKey: string; baseUrl: string; mod
             id,
             {
               name: id,
+              ...MODEL_MODALITIES,
               limit: {
                 context: GATEWAY_CONTEXT_WINDOW,
                 output: GATEWAY_MAX_OUTPUT_TOKENS,
@@ -48,7 +67,8 @@ export function buildGatewayConfig(input: { apiKey: string; baseUrl: string; mod
 // in by provider id at load) and no npm (the loader defaults to
 // @ai-sdk/openai-compatible). The per-model limit is still mandatory — without
 // it opencode sizes the context window to 0 — so the gateway defaults stand in
-// for the unknown endpoint's real limits.
+// for the unknown endpoint's real limits, and MODEL_MODALITIES applies for the
+// same reason it does to the gateway block.
 export function buildCustomProviderConfig(input: { providerID: string; name: string; baseUrl: string; models: string[] }) {
   return {
     provider: {
@@ -62,6 +82,7 @@ export function buildCustomProviderConfig(input: { providerID: string; name: str
             id,
             {
               name: id,
+              ...MODEL_MODALITIES,
               limit: {
                 context: GATEWAY_CONTEXT_WINDOW,
                 output: GATEWAY_MAX_OUTPUT_TOKENS,

packages/codev-gateway/test/config-block.test.ts

@@ -30,14 +30,35 @@ describe("buildGatewayConfig", () => {
           name: "netGate",
           options: { baseURL: "https://gw.test/v1", apiKey: "sk-test" },
           models: {
-            "MiniMax/MiniMax-M2.7": { name: "MiniMax/MiniMax-M2.7", limit: { context: 196608, output: 65536 } },
-            "other/model": { name: "other/model", limit: { context: 196608, output: 65536 } },
+            "MiniMax/MiniMax-M2.7": {
+              name: "MiniMax/MiniMax-M2.7",
+              attachment: true,
+              modalities: { input: ["text", "image"], output: ["text"] },
+              limit: { context: 196608, output: 65536 },
+            },
+            "other/model": {
+              name: "other/model",
+              attachment: true,
+              modalities: { input: ["text", "image"], output: ["text"] },
+              limit: { context: 196608, output: 65536 },
+            },
           },
         },
       },
     })
   })
 
+  // Without the declaration a custom-provider model resolves to text-only and
+  // the session replaces an attached image with an error note (hub PR #219).
+  test("declares image input so attachments are not stripped", () => {
+    const models = buildGatewayConfig({ apiKey: "k", baseUrl: "https://gw.test", model: "m", models: ["m", "n"] })
+      .provider.netgate.models
+    for (const id of ["m", "n"]) {
+      expect(models[id].attachment).toBe(true)
+      expect(models[id].modalities).toEqual({ input: ["text", "image"], output: ["text"] })
+    }
+  })
+
   test("falls back to a single-model map and dedupes the default", () => {
     const config = buildGatewayConfig({ apiKey: "k", baseUrl: "https://gw.test/v1", model: "m" })
     expect(Object.keys(config.provider.netgate.models)).toEqual(["m"])
@@ -60,8 +81,18 @@ describe("buildCustomProviderConfig", () => {
           name: "My Gateway",
           options: { baseURL: "https://gw.test/v1" },
           models: {
-            m1: { name: "m1", limit: { context: 196608, output: 65536 } },
-            m2: { name: "m2", limit: { context: 196608, output: 65536 } },
+            m1: {
+              name: "m1",
+              attachment: true,
+              modalities: { input: ["text", "image"], output: ["text"] },
+              limit: { context: 196608, output: 65536 },
+            },
+            m2: {
+              name: "m2",
+              attachment: true,
+              modalities: { input: ["text", "image"], output: ["text"] },
+              limit: { context: 196608, output: 65536 },
+            },
           },
         },
       },

packages/opencode/test/provider/provider.test.ts

@@ -668,6 +668,47 @@ it.instance(
   },
 )
 
+// The netGate and custom-provider config blocks written by @codev/gateway
+// (config-block MODEL_MODALITIES) exist to flip these two flags: without the
+// declaration a config model is text-only and ProviderTransform.message drops
+// attached images before the request.
+it.instance(
+  "declared image modality makes a custom-provider model attachment capable",
+  Effect.gen(function* () {
+    const providers = yield* list
+    const models = providers[ProviderV2.ID.make("netgate")].models
+    const bare = models["bare-model"]
+    expect(bare.capabilities.attachment).toBe(false)
+    expect(bare.capabilities.input.image).toBe(false)
+    const declared = models["declared-model"]
+    expect(declared.capabilities.attachment).toBe(true)
+    expect(declared.capabilities.input.image).toBe(true)
+    expect(declared.capabilities.input.text).toBe(true)
+    expect(declared.capabilities.output.image).toBe(false)
+  }),
+  {
+    config: {
+      provider: {
+        netgate: {
+          name: "netGate",
+          npm: "@ai-sdk/openai-compatible",
+          env: [],
+          models: {
+            "bare-model": { name: "bare-model", limit: { context: 196608, output: 65536 } },
+            "declared-model": {
+              name: "declared-model",
+              attachment: true,
+              modalities: { input: ["text", "image"], output: ["text"] },
+              limit: { context: 196608, output: 65536 },
+            },
+          },
+          options: { baseURL: "https://gw.test/v1", apiKey: "sk-test" },
+        },
+      },
+    },
+  },
+)
+
 it.instance(
   "model with custom cost values",
   Effect.gen(function* () {

packages/opencode/test/provider/transform.test.ts

@@ -5247,3 +5247,80 @@ describe("ProviderTransform.options - kimi family adaptive thinking", () => {
     expect(result.thinking).toBeUndefined()
   })
 })
+
+// The behavior the netGate/custom-provider config block buys: a model entry
+// with no `modalities` resolves image input to false, and the image never
+// reaches the endpoint. See @codev/gateway config-block MODEL_MODALITIES.
+describe("ProviderTransform.message - image input on a custom provider", () => {
+  const createModel = (image: boolean) =>
+    ({
+      id: "MiniMax/MiniMax-M2.7",
+      providerID: "netgate",
+      api: {
+        id: "MiniMax/MiniMax-M2.7",
+        url: "https://gw.test/v1",
+        npm: "@ai-sdk/openai-compatible",
+      },
+      name: "MiniMax/MiniMax-M2.7",
+      capabilities: {
+        input: { text: true, audio: false, image, video: false, pdf: false },
+        output: { text: true, audio: false, image: false, video: false, pdf: false },
+      },
+      options: {},
+      headers: {},
+    }) as any
+
+  const pngDataUrl = "data:image/png;base64,iVBORw0KGgo="
+
+  test("passes an image part through when the model declares image input", () => {
+    const result = ProviderTransform.message(
+      [{ role: "user", content: [{ type: "text", text: "what is this" }, { type: "image", image: pngDataUrl }] }] as any[],
+      createModel(true),
+      {},
+    ) as any[]
+    expect(result[0].content).toEqual([
+      { type: "text", text: "what is this" },
+      { type: "image", image: pngDataUrl },
+    ])
+  })
+
+  test("replaces an image part with an error note when it does not", () => {
+    const result = ProviderTransform.message(
+      [{ role: "user", content: [{ type: "image", image: pngDataUrl }] }] as any[],
+      createModel(false),
+      {},
+    ) as any[]
+    expect(result[0].content).toEqual([
+      { type: "text", text: "ERROR: Cannot read image (this model does not support image input). Inform the user." },
+    ])
+  })
+
+  test("passes an attached image file through when the model declares image input", () => {
+    const part = { type: "file", mediaType: "image/png", filename: "shot.png", data: pngDataUrl }
+    const result = ProviderTransform.message(
+      [{ role: "user", content: [part] }] as any[],
+      createModel(true),
+      {},
+    ) as any[]
+    expect(result[0].content).toEqual([part])
+  })
+
+  test("names the dropped file in the error note when it does not", () => {
+    const result = ProviderTransform.message(
+      [
+        {
+          role: "user",
+          content: [{ type: "file", mediaType: "image/png", filename: "shot.png", data: pngDataUrl }],
+        },
+      ] as any[],
+      createModel(false),
+      {},
+    ) as any[]
+    expect(result[0].content).toEqual([
+      {
+        type: "text",
+        text: 'ERROR: Cannot read "shot.png" (this model does not support image input). Inform the user.',
+      },
+    ])
+  })
+})