Skip to content

fix(inference): tolerate null role and tool_calls deltas - #1499

Merged
corbits-triage[bot] merged 1 commit into
mainfrom
cl-3010-openai-compatible-sse-parser-rejects-null-roletool_calls
Oct 17, 2026
Merged

corbits-triage[bot] merged 1 commit into
mainfrom
cl-3010-openai-compatible-sse-parser-rejects-null-roletool_calls

Conversation

@TheGreatAxios

Copy link
Copy Markdown
Collaborator

Summary

  • The openai-compatible Chat Completions adapter tolerates explicit null for delta.role and delta.tool_calls on streamed SSE chunks and books them the same as those keys being absent, so vLLM-serializer-relayed endpoints (NEAR AI, Qwen3.8-Flash-Next on vast.ai 2xH200) no longer fail every streamed turn with protocol_mismatch.
  • The non-streaming message schema widens symmetrically (same relays shape both request modes from one serializer), and wrong-type values such as role: 42 still fail schema validation as ProtocolMismatchError — only presence-as-null changes meaning.
  • The vendored divergence carries a ledger entry (PATCHES.md, providers-ts-openai-null-delta-tolerance) with a matching marker comment; regression coverage in src/provider/chat-completions-null-delta.test.ts proves null-equivalence event sequences end to end through the builtin and wrapper adapters.

Verification

  • bun run check passes (lint, typecheck, dead-exports, build, projects-dir guard; sandboxed suite 9188 pass / 0 fail).
  • bun run test passes (9188 pass / 0 fail, --randomize --seed 424242).
  • bin/vendor-patch-diff shows exactly the intended hunks for providers/openai.ts; scripts/vendor-patch-ledger.test.ts is green.
  • Whole-branch review verdict: GO.

Refs CL-3010

@linear-code

linear-code Bot commented Oct 17, 2026

Copy link
Copy Markdown

CL-3010

@corbits-triage

Copy link
Copy Markdown
Contributor

Triage: Pass

No changes needed from you. A maintainer will review this pull request.

@corbits-triage
corbits-triage Bot merged commit f975b9d into main Oct 17, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant