Repository navigation
feat(gateway): declare image input for gateway and custom provider models - #139
Merged
Merged
Conversation
…dels
A config model entry with no `modalities` resolves image input to false, so
ProviderTransform.message replaced an attached image with an "ERROR: Cannot
read image" note before the request and the model answered that it couldn't
see anything — even when the endpoint supports vision.
Both blocks written by @codev/gateway (netGate configure flow and the custom
provider flow) now declare `attachment: true` and
`modalities: { input: ["text", "image"], output: ["text"] }` per model,
mirroring codev-hub PR #219 so hub-written and self-configured codev.json
stay identical. Every gateway model is now declared vision-capable including
text-only ones: for those the endpoint rejects or ignores the image rather
than the client silently dropping it.
Tests cover the two upstream paths this depends on — config modality →
model capability, and the part filtering in transform — plus the end-to-end
PATCH body from the custom-provider TUI flow. That view test was already
failing on main (it waits on the "Provider id" prompt renamed to "Provider
name" in #137, and predates the provider `name` field); repaired here since
it asserts the payload this changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follows codev-hub PR #219, which fixed the same defect in the hub's config writer. This repo has a mirror of that writer (
packages/codev-gateway/src/config-block.ts), so the fork's own configure flows were still emitting text-only models.A config model entry with no
modalitiesresolvescapabilities.input.imagetofalse(provider/provider.ts:1463), andunsupportedPartsthen swaps the attached image for anERROR: Cannot read image (this model does not support image input)note before the request leaves the client (provider/transform.ts:411). The model replies that it can't see any image — even when the gateway and model support vision.Both blocks the fork writes now declare, per model:
buildGatewayConfig— netGate SSO / key-paste configure flow. Byte-parity with the hub writer is a hard requirement here (hub-managed and self-configured installs must be indistinguishable), so this had to move regardless.buildCustomProviderConfig— the TUI "Custom provider" flow. Not hub territory, but the identical failure mode.Trade-off
Every gateway model is now declared vision-capable, including text-only ones. For those an attached image reaches the endpoint (a possible 400 or a confused reply) instead of being stripped client-side. Intentional, and the same call the hub PR made: the gateway/model decides what to do with an image it can't use, not the client.
Configs written before this change need one more pass through the configure flow, or a CoDev Hub config rewrite, to pick the declaration up. Healing existing installs at load time would mean defaulting custom-provider models to image-capable in
provider.ts— an upstream-file divergence, deliberately not done here.Tests
packages/codev-gateway/test/config-block.test.ts— writer output for both builders, plus the multi-model map.packages/opencode/test/provider/provider.test.ts— newit.instance: anetgatemodel with no modalities resolvesattachment/input.imagefalse; one with the block resolves both true. Pins the config → capability path.packages/opencode/test/provider/transform.test.ts— new describe:imageandfile/image/pngparts pass through when the model declares image input, and become theERROR: Cannot read …note when it doesn't.unsupportedPartshad no coverage.packages/tui/test/custom-provider.view.test.tsx— the end-to-end PATCH body now expects the declaration on every discovered model.Two pre-existing failures repaired in that view test (verified failing on clean
main): they wait on aProvider idprompt that #137 renamed toProvider name (optional), and their expected payload predates the providernamefield. Same file and same assertion this PR touches, so fixed rather than left red.Verified:
bun test test/provider/inpackages/opencode→ 493 pass;packages/codev-gateway→ 36 pass;custom-provider.view→ 4 pass,configure-gate→ 7 pass;bun typecheckclean inopencode,tui,codev-gateway.Bookkeeping
DIVERGENCES.md— new row for the two upstream test files (append-only blocks).packages/codev-gatewayand the TUI view test are fork-only, so untracked there.CHANGELOG.md— Unreleased → Provider setup, including the trade-off and the reconfigure note.🤖 Generated with Claude Code