Skip to content

fix(llm_gateway): accept SSE data fields without a space - #272

Draft
Kayvan-Zahiri wants to merge 1 commit into
AssemblyAI:masterfrom
Kayvan-Zahiri:codex/accept-unspaced-gateway-sse-data
Draft

Kayvan-Zahiri wants to merge 1 commit into
AssemblyAI:masterfrom
Kayvan-Zahiri:codex/accept-unspaced-gateway-sse-data

Conversation

@Kayvan-Zahiri

Copy link
Copy Markdown
Contributor

The sync and async LLM Gateway stream readers require the literal prefix data: . SSE permits the space after the colon to be omitted, so valid data:{...} frames are silently skipped. An unspaced data:[DONE] is also ignored, allowing later frames to be emitted instead of ending the iterator.

Accept the data: field prefix and retain the existing payload trimming. Ignore empty data payloads so previously ignored unspaced keepalive frames do not become JSON parse errors. Response parsing, setup-error handling, and the existing spaced format remain unchanged. This is a narrow single-line framing fix, not a full SSE parser rewrite.

Added eight public sync/async Gateway tests covering the Cartesian product of spaced/unspaced chunk and [DONE] fields, empty keepalive data, comments, unrelated fields, CRLF framing, a distinct post-sentinel chunk that must not be yielded, and response closure. Six cases fail against the original code; two are controls.

The optional space is specified by the SSE parsing algorithm. Reproduction uses controlled HTTP responses; it does not establish that the hosted gateway currently emits unspaced frames. No live inference calls were made.

Validation:

  • Final full suite: 749 passed (189 warnings). Local TLS tests require loopback access; the first sandboxed run had five bind-permission failures, and all passed with local socket access.
  • Separately, all 43 LLM Gateway tests pass with Pydantic 1, including the new cases.
  • Changed-file Ruff lint/format and git diff --check pass.
  • Full repository Ruff has 12 existing findings; restoring the original source reproduces the same findings.
  • Wheel and source distribution build successfully.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant