Skip to content

Migration 86's queued-input index rejects user messages whose payload is not valid JSON #36894

Closed

Description

@navidnadali

Note

Written by Claude Opus 5.5 (Claude Code) on behalf of Navid

Migration 86 (086_QueuedTurnInputIndex, from #23420) adds a partial index whose WHERE calls json_extract(payload_json, '$.inputIntent') without a json_valid guard. SQLite evaluates that term on every write of a user_message turn item, so a user message whose payload is not valid JSON can no longer be inserted or updated, and a database that already holds one cannot upgrade past migration 85.

Reproduction with sqlite3 3.54.0, the turn-item table as the projection migration creates it and the index exactly as migration 86 creates it:

CREATE TABLE orchestration_v2_projection_turn_items (
  turn_item_id TEXT PRIMARY KEY, thread_id TEXT NOT NULL, run_id TEXT, node_id TEXT,
  provider_thread_id TEXT, provider_turn_id TEXT, parent_item_id TEXT,
  ordinal INTEGER NOT NULL, type TEXT NOT NULL, status TEXT NOT NULL,
  updated_at TEXT NOT NULL, payload_json TEXT NOT NULL
);
CREATE INDEX orchestration_v2_projection_turn_items_queued_input_idx
  ON orchestration_v2_projection_turn_items(run_id)
  WHERE type = 'user_message' AND json_extract(payload_json, '$.inputIntent') = 'queued_turn';

INSERT INTO orchestration_v2_projection_turn_items
  (turn_item_id, thread_id, run_id, ordinal, type, status, updated_at, payload_json)
  VALUES ('u1', 't1', 'r1', 1, 'user_message', 'completed', '2026-11-14T00:00:00Z', '{not json');
-- Error: malformed JSON

INSERT INTO orchestration_v2_projection_turn_items
  (turn_item_id, thread_id, run_id, ordinal, type, status, updated_at, payload_json)
  VALUES ('a1', 't1', 'r1', 2, 'assistant_message', 'completed', '2026-11-14T00:00:00Z', '{not json');
-- OK

On a database that already holds such a user message, the index itself cannot be built:

-- same table, without the index, holding the 'u1' row above
CREATE INDEX orchestration_v2_projection_turn_items_queued_input_idx
  ON orchestration_v2_projection_turn_items(run_id)
  WHERE type = 'user_message' AND json_extract(payload_json, '$.inputIntent') = 'queued_turn';
-- Error: malformed JSON

Through the server's own migrations on main @ 2137f360e this shows up as Migration "86_QueuedTurnInputIndex" failed … malformed JSON, with the database left at 85. An UPDATE that sets a non-JSON payload on a user message fails the same way.

Assistant messages are unaffected, because the type = 'user_message' term is false before json_extract runs. Migration 89 already guards its expression indexes this way: "json_valid keeps a malformed payload writable, as the other payload indexes do."

Suggested fix: in a new migration, recreate the index with json_valid(payload_json) AND before the json_extract term, and add json_valid(held.payload_json) AND to the two shell-count queries that name it with INDEXED BY; without the matching term SQLite rejects them with "no query solution".

The server normally writes only JSON payloads, so this is defensive.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions