Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/features/agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,12 +185,15 @@ Each entry in **Settings → AI → Providers** stores one credential. The provi
|---|---|---|---|---|---|
| `anthropic` | Anthropic (Claude) | `apiKey` | API key (`sk-ant-…`) | — | Static `claude-*` catalogue enriched with OpenRouter prices + context windows |
| `openai` | OpenAI | `apiKey` | API key (`sk-…`) | — | Static `gpt-*` / `o*` catalogue enriched with OpenRouter prices + context windows |
| `minimax` | MiniMax | `baseUrl` | Regional OpenAI- or Anthropic-compatible endpoint + API key | — | Live regional `/models` endpoint filtered to MiniMax-M3.1-Flash first, then MiniMax-M3, with provider metadata |
| `openrouter` | OpenRouter | `apiKey` | API key (`sk-or-…`) | — | Live `GET /api/v1/models` (cross-provider; native cost reporting) |
| `ollama` | Ollama (local) | `baseUrl` | Base URL (e.g. `http://localhost:11434`) | API key (bearer, for proxied deployments) | Live `GET {baseUrl}/api/tags`, with `POST {baseUrl}/api/show` capability lookup per model; static fallback list when unreachable |
| `openai-compatible` | Custom Provider | `baseUrl` | Base URL — any host serving the OpenAI `/v1/chat/completions` wire protocol | API key (bearer; cloud services need one, local servers often don't) | Live `GET {baseUrl}/v1/models` (standard OpenAI list shape); model `id` used as label |

**Custom Provider** (id `openai-compatible`) is the generic adapter for any endpoint that speaks the OpenAI chat/completions wire protocol — Groq (`https://api.groq.com/openai`), Together, DeepSeek, Mistral, Fireworks, self-hosted vLLM, LM Studio, and others. Capabilities default to `{ toolCalling: true, visionInput: false, toolResultImages: false, promptCache: false, streaming: true }`; the operator is responsible for selecting a model that actually supports tool calling. Because arbitrary endpoints are not in the OpenRouter catalogue, no context-window enrichment is available and the context meter stays hidden for these models.

**MiniMax** provides presets for the global and China APIs over both supported wire protocols. The OpenAI-compatible endpoints end in `/v1`; the Anthropic-compatible endpoints end in `/anthropic`. The selected model catalogue retains MiniMax-specific context, input capability, and list-price metadata. Flash is the first available choice for new credentials; saved model selections are not rewritten. Flash uses adaptive thinking with the API's default `max` effort. M3 uses the protocol default (adaptive on Chat Completions, disabled on Messages). Instatic does not expose an effort or thinking override. MiniMax reasoning is retained during tool continuation. Direct list prices describe the standard short-context tier; M3 costs increase above 512k context. The gateway catalogue keeps its own prices and context limits, offering `minimax/minimax-m3.1-flash:free` before `minimax/minimax-m3` while preserving saved legacy selections.

**The server is the model-capability authority.** The composer catalogue flags are early UX gates, but they are not trusted for persistence, provider calls, or tool screenshots. `chat.ts` resolves the selected model on every turn through `resolveModelCapabilities`. Providers with stable capabilities (Anthropic, OpenAI, Custom Provider) use their driver default. Model-specific providers own a selected-model lookup: OpenRouter resolves the exact entry's `architecture.input_modalities`, while Ollama sends an authenticated `POST /api/show` for only the selected model. The shared resolver de-duplicates concurrent lookups, applies a ten-second provider timeout, includes credential/backend revisions in its cache key, and caches successful results for five minutes. Missing or unavailable model-specific metadata fails closed for vision input; Custom Provider also remains fail-closed in v1. An image targeting a non-vision model, or an editor-agent turn targeting a model known not to support tools, receives 422 before the user message is stored.

---
Expand Down
127 changes: 91 additions & 36 deletions server/ai/drivers/anthropic.ts
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,13 @@ interface AnthropicToolResultBlock {
content: string | (AnthropicTextBlock | AnthropicImageBlock)[]
is_error?: boolean
}
interface AnthropicThinkingBlock {
type: 'thinking'
thinking: string
signature?: string
}
type AnthropicContentBlock =
| AnthropicThinkingBlock
| AnthropicTextBlock
| AnthropicImageBlock
| AnthropicToolUseBlock
Expand All @@ -216,49 +222,66 @@ export interface AnthropicMessage {
// Adapter
// ---------------------------------------------------------------------------

const anthropicAdapter: ProviderAdapter<AnthropicMessage> = {
export function makeAnthropicMessagesAdapter(opts: {
label: string
endpoint: string
buildHeaders: (req: AiStreamRequest) => Record<string, string>
requestBodyExtras?: (req: AiStreamRequest) => Record<string, unknown>
preserveReasoning?: boolean
}): ProviderAdapter<AnthropicMessage> {
const { label, endpoint, buildHeaders, requestBodyExtras, preserveReasoning } = opts
return {
label,
endpoint,

buildHeaders,

mapHistory(req) {
return mapHistory(req.messages)
},

buildRequestBody(messages, req) {
const body: Record<string, unknown> = {
model: req.modelId,
max_tokens: MAX_OUTPUT_TOKENS,
system: buildSystemBlocks(req.systemPrompt),
messages,
stream: true,
}
if (req.tools.length > 0) {
body.tools = req.tools.map((t) => ({
name: t.name,
description: t.description,
// The TypeBox schema IS JSON Schema — pass it straight through.
input_schema: t.inputSchema,
}))
}
const extra = requestBodyExtras?.(req)
if (extra) Object.assign(body, extra)
return body
},

buildToolResultMessage(results) {
return buildToolResultMessage(results)
},

createTurnTranslator() {
return new AnthropicTurnTranslator(label, preserveReasoning)
},
}
}

const anthropicAdapter = makeAnthropicMessagesAdapter({
label: 'Anthropic',
endpoint: ANTHROPIC_ENDPOINT,

buildHeaders(req) {
return {
'x-api-key': req.credentials.apiKey!,
'anthropic-version': ANTHROPIC_VERSION,
'content-type': 'application/json',
}
},

mapHistory(req) {
return mapHistory(req.messages)
},

buildRequestBody(messages, req) {
const body: Record<string, unknown> = {
model: req.modelId,
max_tokens: MAX_OUTPUT_TOKENS,
system: buildSystemBlocks(req.systemPrompt),
messages,
stream: true,
}
if (req.tools.length > 0) {
body.tools = req.tools.map((t) => ({
name: t.name,
description: t.description,
// The TypeBox schema IS JSON Schema — pass it straight through.
input_schema: t.inputSchema,
}))
}
return body
},

buildToolResultMessage(results) {
return buildToolResultMessage(results)
},

createTurnTranslator() {
return new AnthropicTurnTranslator()
},
}
})

// ---------------------------------------------------------------------------
// System prompt → system blocks
Expand Down Expand Up @@ -432,6 +455,8 @@ const AnthropicSseEventSchema = Type.Object(
id: Type.Optional(Type.String()),
name: Type.Optional(Type.String()),
input: Type.Optional(Type.Unknown()),
thinking: Type.Optional(Type.String()),
signature: Type.Optional(Type.String()),
},
{ additionalProperties: true },
),
Expand All @@ -441,6 +466,8 @@ const AnthropicSseEventSchema = Type.Object(
{
type: Type.Optional(Type.String()),
text: Type.Optional(Type.String()),
thinking: Type.Optional(Type.String()),
signature: Type.Optional(Type.String()),
partial_json: Type.Optional(Type.String()),
stop_reason: Type.Optional(Type.Union([Type.String(), Type.Null()])),
},
Expand Down Expand Up @@ -476,10 +503,19 @@ interface MutableUsage {
}

export class AnthropicTurnTranslator implements TurnTranslator<AnthropicMessage> {
private readonly label: string
private readonly preserveReasoning: boolean

constructor(label = 'Anthropic', preserveReasoning = false) {
this.label = label
this.preserveReasoning = preserveReasoning
}

// Block order as it streams, so the assistant turn rebuilds text/tool_use
// blocks in the sequence the model emitted them.
private readonly order: number[] = []
private readonly textByIndex = new Map<number, string>()
private readonly thinkingByIndex = new Map<number, AnthropicThinkingBlock>()
private readonly toolByIndex = new Map<number, { id: string; name: string; json: string }>()
private readonly toolCalls: TurnToolCall[] = []
private usage: MutableUsage = {}
Expand Down Expand Up @@ -510,6 +546,13 @@ export class AnthropicTurnTranslator implements TurnTranslator<AnthropicMessage>
name: typeof block.name === 'string' ? block.name : 'tool',
json: '',
})
} else if (this.preserveReasoning && block?.type === 'thinking') {
this.order.push(index)
this.thinkingByIndex.set(index, {
type: 'thinking',
thinking: block.thinking ?? '',
...(block.signature !== undefined ? { signature: block.signature } : {}),
})
} else if (block?.type === 'text') {
this.order.push(index)
this.textByIndex.set(index, '')
Expand All @@ -520,6 +563,13 @@ export class AnthropicTurnTranslator implements TurnTranslator<AnthropicMessage>
case 'content_block_delta': {
const index = event.index ?? 0
const delta = event.delta
const thinking = this.thinkingByIndex.get(index)
if (thinking && delta?.type === 'thinking_delta' && typeof delta.thinking === 'string') {
thinking.thinking += delta.thinking
}
if (thinking && delta?.type === 'signature_delta' && typeof delta.signature === 'string') {
thinking.signature = (thinking.signature ?? '') + delta.signature
}
if (delta?.type === 'text_delta' && typeof delta.text === 'string') {
this.textByIndex.set(index, (this.textByIndex.get(index) ?? '') + delta.text)
return [{ type: 'text', text: delta.text }]
Expand Down Expand Up @@ -557,8 +607,8 @@ export class AnthropicTurnTranslator implements TurnTranslator<AnthropicMessage>
return [{
type: 'error',
message: detail
? `Anthropic error: ${detail}`
: 'Anthropic stream failed. Check your credentials in /admin/ai/providers.',
? `${this.label} error: ${detail}`
: `${this.label} stream failed. Check your credentials in /admin/ai/providers.`,
}]
}

Expand All @@ -571,6 +621,11 @@ export class AnthropicTurnTranslator implements TurnTranslator<AnthropicMessage>
finish(): TurnResult<AnthropicMessage> {
const content: AnthropicContentBlock[] = []
for (const index of this.order) {
const thinking = this.thinkingByIndex.get(index)
if (thinking) {
content.push(thinking)
continue
}
const text = this.textByIndex.get(index)
if (text !== undefined) {
if (text) content.push({ type: 'text', text })
Expand Down
26 changes: 22 additions & 4 deletions server/ai/drivers/http/chatCompletions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ import {
} from './toolLoop'
import type { SseFrame } from './sse'
import { parseToolArguments } from './toolArgs'
import type { AiStreamRequest } from '../types'
import { nanoid } from 'nanoid'

// ---------------------------------------------------------------------------
Expand All @@ -44,7 +45,7 @@ export type ChatToolCall = {
export type ChatMessage =
| { role: 'system'; content: string }
| { role: 'user'; content: string | ChatContentPart[] }
| { role: 'assistant'; content: string; tool_calls?: ChatToolCall[] }
| { role: 'assistant'; content: string; tool_calls?: ChatToolCall[]; reasoning_content?: string }
| { role: 'tool'; tool_call_id: string; content: string }

// Each canonical `AiMessage` maps to one or more chat messages (an assistant
Expand Down Expand Up @@ -192,6 +193,7 @@ const ChatChunkSchema = Type.Object(
Type.Object(
{
content: nullable(Type.String()),
reasoning_content: nullable(Type.String()),
tool_calls: nullable(Type.Array(ChatToolCallDeltaSchema)),
},
{ additionalProperties: true },
Expand Down Expand Up @@ -224,7 +226,14 @@ interface MutableToolCall {
}

export class ChatCompletionsTurnTranslator implements TurnTranslator<ChatTurn> {
private readonly preserveReasoning: boolean

constructor(preserveReasoning = false) {
this.preserveReasoning = preserveReasoning
}

private text = ''
private reasoning = ''
// Tool calls accumulate by their streamed `index`; fragments arrive across
// chunks (id + name on the first, arguments piecemeal after).
private readonly toolsByIndex = new Map<number, MutableToolCall>()
Expand Down Expand Up @@ -254,6 +263,9 @@ export class ChatCompletionsTurnTranslator implements TurnTranslator<ChatTurn> {
const events: AiStreamEvent[] = []
const delta = choice.delta
if (delta) {
if (this.preserveReasoning && typeof delta.reasoning_content === 'string') {
this.reasoning += delta.reasoning_content
}
if (typeof delta.content === 'string' && delta.content.length > 0) {
this.text += delta.content
events.push({ type: 'text', text: delta.content })
Expand Down Expand Up @@ -312,10 +324,12 @@ export class ChatCompletionsTurnTranslator implements TurnTranslator<ChatTurn> {
? { role: 'assistant', content: this.text, tool_calls: chatToolCalls }
: { role: 'assistant', content: this.text }

if (this.reasoning) assistant.reasoning_content = this.reasoning

return {
stop: toolCalls.length === 0,
toolCalls,
assistantMessage: this.text || chatToolCalls.length > 0 ? [assistant] : null,
assistantMessage: this.text || this.reasoning || chatToolCalls.length > 0 ? [assistant] : null,
usage: this.usage,
}
}
Expand All @@ -329,8 +343,10 @@ export function makeChatCompletionsAdapter(opts: {
baseUrl: string
apiKey: string | null
label: string
requestBodyExtras?: (req: AiStreamRequest) => Record<string, unknown>
preserveReasoning?: boolean
}): ProviderAdapter<ChatTurn> {
const { baseUrl, apiKey, label } = opts
const { baseUrl, apiKey, label, requestBodyExtras, preserveReasoning } = opts
return {
label,
endpoint: `${normalizeOpenAiBaseUrl(baseUrl)}/v1/chat/completions`,
Expand All @@ -355,6 +371,8 @@ export function makeChatCompletionsAdapter(opts: {
function: { name: t.name, description: t.description, parameters: t.inputSchema },
}))
}
const extra = requestBodyExtras?.(req)
if (extra) Object.assign(body, extra)
return body
},
buildToolResultMessage(results: TurnToolResult[]): ChatTurn {
Expand All @@ -365,7 +383,7 @@ export function makeChatCompletionsAdapter(opts: {
}))
},
createTurnTranslator() {
return new ChatCompletionsTurnTranslator()
return new ChatCompletionsTurnTranslator(preserveReasoning)
},
}
}
3 changes: 2 additions & 1 deletion server/ai/drivers/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,15 @@ import type { AiProvider } from './types'
import type { AiProviderId } from '../runtime/types'
import { anthropicDriver } from './anthropic'
import { openaiDriver } from './openai'
import { minimaxDriver } from './minimax'
import { ollamaDriver } from './ollama'
import { openrouterDriver } from './openrouter'
import { openaiCompatibleDriver } from './openaiCompatible'

const DRIVERS: Record<AiProviderId, AiProvider> = {
anthropic: anthropicDriver,
openai: openaiDriver,
minimax: minimaxDriver,
ollama: ollamaDriver,
openrouter: openrouterDriver,
'openai-compatible': openaiCompatibleDriver,
Expand All @@ -30,4 +32,3 @@ export function resolveDriver(providerId: AiProviderId): AiProvider {
}
return driver
}

Loading