Skip to content

feat(frontend): inspect retrieved chunks in the dashboard - #44

Merged
johnnyhuy merged 1 commit into
mainfrom
agent/issue-43-retrieval-dashboard
Nov 14, 2026
Merged

johnnyhuy merged 1 commit into
mainfrom
agent/issue-43-retrieval-dashboard

Conversation

@johnnyhuy

@johnnyhuy johnnyhuy commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

The dashboard now has a Retrieval tab for inspecting ranked chunk text, source metadata, raw distances and effective settings with optional channel scope. It handles loading, empty results, retries and cancellation, and clears stale results when inputs change.

Part of #5. Closes #43. Parent #5 stays open.

The first browser capture exposed missing Tailwind compilation across the dashboard. This slice includes the PostCSS configuration and the matching @tailwindcss/postcss 4.1.18 dependency, so existing utility classes render as intended. This affects styling across all tabs; navigation wraps on narrow screens. The setup follows the official Next.js integration.

Verified offline with the repository gates.

  • UV_PYTHON=3.13 UV_FROZEN=1 mise run install passed. Frozen mode preserves the Python lockfile.
  • UV_PYTHON=3.13 UV_FROZEN=1 SLACK_BOT_TOKEN=xoxb-fixture-only SLACK_SIGNING_SECRET=fixture-only PYTEST_ADDOPTS=-rP mise run test-backend passed all 153 tests, including the existing local Chroma cutoff transcript. Existing dependency deprecation warnings remain.
  • mise run test-frontend passed all 15 tests, including 8 new retrieval cases. They exercise the actual API client through the dashboard with mocked fetch, covering trimmed and omitted channel scope, rank and raw distance preservation, zero and unset cutoffs, duplicate submission prevention, empty results, retry, stale result clearing, cancellation on tab exit and plain text rendering.
  • mise run test-runtime passed all 7 offline tests.
  • cd frontend && mise exec -- bun run build passed, including TypeScript checking and static page generation.
  • NEXT_PUBLIC_API_URL=http://127.0.0.1:18143 mise run dev-frontend -- --hostname 127.0.0.1 --port 18144 started successfully for the browser capture.
  • PYTHONPATH="$PWD/.tmp/proof:$PWD/backend" UV_PYTHON=3.13 UV_FROZEN=1 RETRIEVAL_MAX_DISTANCE=1 mise run dev-backend -- --host 127.0.0.1 --port 18143 reached application startup and shut down cleanly. Repeated with the cutoff unset. The temporary harness uses local SQLite and Chroma, fixed embeddings, mocked Slack authentication and guards blocking Slack API calls, Socket Mode, generation and remote connections. It is the harness recorded in PR #42's proof, copied into this worktree's .tmp/proof/.
  • git diff --check passed.

Desktop and narrow browser capture succeeded against the fixture backend. The 390-pixel viewport had a 390-pixel document width, with no horizontal overflow. Browser checks also confirmed an empty result for CMISSING, a visible retry after stopping the backend, and successful recovery after restarting it. Captures were saved under .tmp/proof/ and copied into the retrieval documentation for durable review.

Retrieval preview with fixture data

View the narrow screen capture.

For How do I reset VPN? scoped to CTARGET, local curl returned HTTP 200 with two chunks at squared L2 distances 1 and 4 when the cutoff was unset. With cutoff 1, it returned only the first chunk. These fixture distances demonstrate the UI reflecting the existing policy; they do not measure production relevance. Test names, actual curl responses and startup output are recorded in the proof comment.

No capture method was blocked. Live Slack verification is excluded by the schedule's safety rule. No real tokens, messages or Socket Mode were used. No backend API, retrieval ranking or database schema changed. The management API still needs trusted access controls; channel scope is a filter, not authorisation. Preview calls can incur embedding costs with a real provider, and cancelling the browser request cannot guarantee cancellation of backend embedding work already started. Authentication and tuning controls remain outside this slice.

GitHub CI is green for this commit, including tests and all three container builds, CodeQL and Socket checks.

Add a retrieval preview tab with optional channel scope, ranked chunks, raw distances and effective settings. Cover loading, retries, cancellation and safe rendering with offline tests.

Configure the missing Tailwind PostCSS plugin and capture desktop and narrow fixture views.

Co-authored-by: OpenAI Codex <noreply@openai.com>
@socket-security

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addednpm/​@​tailwindcss/​postcss@​4.1.1810010010098100

View full report

@johnnyhuy

Copy link
Copy Markdown
Contributor Author

Captured offline at commit 465c964700132b5dab7cbc2b4f59508dc4f4b5dc. All data below is synthetic. No real Slack or model calls were made.

The frontend gate lists the eight new retrieval tests and the seven existing conversation tests.

[test-frontend] $ bun run test
$ bun test --preload ./tests/setup.ts
bun test v1.3.5 (1e86cebd)

tests/retrieval.test.tsx:
(pass) retrieval waits for a non-blank question and submits trimmed optional scope [77.09ms]
(pass) retrieval omits blank scope, preserves rank and raw distances, and shows an unset cutoff [23.28ms]
(pass) retrieval disables inputs and duplicate submissions while loading [27.04ms]
(pass) retrieval shows an empty result and a zero distance cutoff [34.71ms]
(pass) retrieval replaces a failed request with a successful retry [45.52ms]
(pass) retrieval clears old results when the question or channel changes [36.85ms]
(pass) retrieval aborts on leaving the tab and ignores late responses after reopening [42.53ms]
(pass) retrieval renders chunk and metadata markup as text [58.30ms]

tests/conversations.test.tsx:
(pass) loads only on opening the tab and paginates summaries and chronological turns independently [172.02ms]
(pass) shows loading, an empty page and a list error with a working retry [7.05ms]
(pass) retries missing conversation details without discarding the list [21.48ms]
(pass) ignores stale detail responses after selecting another conversation [24.26ms]
(pass) ignores stale errors after leaving and reopening the tab [8.82ms]
(pass) renders stored markup as text and supports missing metadata [14.94ms]
(pass) encodes conversation IDs in detail request paths [1.69ms]

 15 pass
 0 fail
 92 expect() calls
Ran 15 tests across 2 files. [924.00ms]

The backend gate passed all 153 tests. These existing preview cases exercise the shared retrieval policy used by the new tab.

tests/test_retrieval_preview.py::test_preview_returns_ranked_chunks_metadata_distances_and_settings PASSED [ 90%]
tests/test_retrieval_preview.py::test_preview_applies_channel_scope_before_top_k PASSED [ 91%]
tests/test_retrieval_preview.py::test_preview_exposes_inclusive_distance_cutoff_with_transcript PASSED [ 92%]
tests/test_retrieval_preview.py::test_preview_returns_success_for_empty_results[collection] PASSED [ 92%]
tests/test_retrieval_preview.py::test_preview_returns_success_for_empty_results[channel] PASSED [ 93%]
tests/test_retrieval_preview.py::test_preview_returns_success_for_empty_results[cutoff] PASSED [ 94%]
tests/test_retrieval_preview.py::test_preview_rejects_invalid_input_before_search[payload0] PASSED [ 94%]
tests/test_retrieval_preview.py::test_preview_rejects_invalid_input_before_search[payload1] PASSED [ 95%]
tests/test_retrieval_preview.py::test_preview_rejects_invalid_input_before_search[payload2] PASSED [ 96%]
tests/test_retrieval_preview.py::test_preview_rejects_invalid_input_before_search[payload3] PASSED [ 96%]
tests/test_retrieval_preview.py::test_preview_rejects_invalid_input_before_search[payload4] PASSED [ 97%]
tests/test_retrieval_preview.py::test_preview_rejects_invalid_input_before_search[payload5] PASSED [ 98%]
tests/test_retrieval_preview.py::test_preview_rejects_invalid_input_before_search[payload6] PASSED [ 98%]
tests/test_retrieval_preview.py::test_preview_trims_question_and_channel PASSED [ 99%]
tests/test_retrieval_preview.py::test_preview_reports_unavailable_retrieval_without_exposing_error PASSED [100%]
====================== 153 passed, 727 warnings in 5.89s =======================

The existing Chroma fixture printed this before/after cutoff transcript during the gate. Fixed-vector squared L2 distances prove policy parity, not production relevance.

{
  "before_cutoff": {
    "question": "How do I reset VPN?",
    "channel_id": "CTARGET",
    "top_k": 2,
    "max_distance": null,
    "items": [
      {
        "content": "Reset VPN via IT portal",
        "metadata": {
          "channel_id": "CTARGET",
          "message_ts": "1.0",
          "source": "slack"
        },
        "distance": 1.0
      },
      {
        "content": "Ask IT for VPN access",
        "metadata": {
          "channel_id": "CTARGET",
          "message_ts": "2.0",
          "source": "slack"
        },
        "distance": 4.0
      }
    ]
  },
  "after_cutoff": {
    "question": "How do I reset VPN?",
    "channel_id": "CTARGET",
    "top_k": 2,
    "max_distance": 1.0,
    "items": [
      {
        "content": "Reset VPN via IT portal",
        "metadata": {
          "channel_id": "CTARGET",
          "message_ts": "1.0",
          "source": "slack"
        },
        "distance": 1.0
      }
    ]
  },
  "note": "Fixed-vector squared L2 distances prove policy parity, not production relevance."
}

Cutoff unset. The following is the actual local HTTP request and response.

$ curl --silent --show-error --include --request POST http://127.0.0.1:18143/api/retrieval/preview --header 'Content-Type: application/json' --data '{"question":"How do I reset VPN?","channel_id":"CTARGET"}'
HTTP/1.1 200 OK
date: Sat, 14 Nov 2026 02:24:58 GMT
server: uvicorn
content-length: 345
content-type: application/json

{"question":"How do I reset VPN?","channel_id":"CTARGET","top_k":2,"max_distance":null,"items":[{"content":"Reset VPN via IT portal","metadata":{"channel_id":"CTARGET","message_ts":"1.0","source":"slack"},"distance":1.0},{"content":"Ask IT for VPN access","metadata":{"channel_id":"CTARGET","message_ts":"2.0","source":"slack"},"distance":4.0}]}

Cutoff set to 1. The following is the actual local HTTP request and response.

$ curl --silent --show-error --include --request POST http://127.0.0.1:18143/api/retrieval/preview --header 'Content-Type: application/json' --data '{"question":"How do I reset VPN?","channel_id":"CTARGET"}'
HTTP/1.1 200 OK
date: Sat, 14 Nov 2026 02:22:30 GMT
server: uvicorn
content-length: 221
content-type: application/json

{"question":"How do I reset VPN?","channel_id":"CTARGET","top_k":2,"max_distance":1.0,"items":[{"content":"Reset VPN via IT portal","metadata":{"channel_id":"CTARGET","message_ts":"1.0","source":"slack"},"distance":1.0}]}

Fixture backend lifecycle recorded in backend-startup.txt.

INFO:     Started server process [68104]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Shutting down
{"timestamp": "2026-11-14T02:24:10.825845+00:00", "level": "INFO", "service_name": "wingman-backend", "environment": "development", "service_version": "0.1.0", "logger": "app.main", "message": "Shutting down Wingman backend...", "event_type": "log", "request_id": null, "trace_id": null}
INFO:     Application shutdown complete.
INFO:     Finished server process [68104]

Fixture backend lifecycle recorded in backend-startup-unfiltered.txt.

INFO:     Started server process [3883]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Shutting down
{"timestamp": "2026-11-14T02:25:00.083221+00:00", "level": "INFO", "service_name": "wingman-backend", "environment": "development", "service_version": "0.1.0", "logger": "app.main", "message": "Shutting down Wingman backend...", "event_type": "log", "request_id": null, "trace_id": null}
INFO:     Application shutdown complete.
INFO:     Finished server process [3883]

Desktop and narrow screenshots are committed under docs/images/ and linked in the PR body. The browser also showed an empty result for CMISSING, a retryable connection error while the local fixture backend was stopped, and recovery after restart. At 390 pixels, both viewport and document measured 390 pixels. No browser capture was blocked. Live Slack verification remains excluded by the schedule safety rule.

@johnnyhuy
johnnyhuy merged commit a6eac10 into main Nov 14, 2026
10 checks passed
@johnnyhuy
johnnyhuy deleted the agent/issue-43-retrieval-dashboard branch November 14, 2026 06:19
@github-actions github-actions Bot mentioned this pull request Nov 14, 2026
@johnnyhuy

Copy link
Copy Markdown
Contributor Author

Completed the existing scheduled-run slice and merged it as a6eac10. No additional code changes were needed. The merged main tree is identical to the branch tree verified below.

  • UV_PYTHON=3.13 UV_FROZEN=1 mise run install passed after trusting this worktree's mise configuration.
  • UV_PYTHON=3.13 UV_FROZEN=1 SLACK_BOT_TOKEN=xoxb-fixture-only SLACK_SIGNING_SECRET=fixture-only PYTEST_ADDOPTS=-rP mise run test-backend passed all 153 tests. Existing dependency deprecation warnings remain.
  • mise run test-frontend passed all 15 tests, including all 8 retrieval dashboard cases.
  • mise run test-runtime passed all 7 offline tests.
  • git diff --check passed, and the worktree is clean.
  • All 10 GitHub checks were green before the squash merge.

Reviewed the existing desktop screenshot and the fixture-based behaviour proof. The prior run successfully captured desktop/mobile views and local API transcripts; no capture method was blocked. These captures were retained rather than regenerated because the code did not change.

The merge closed child #43. Parent #5 stays open. The remote branch was deleted, and its clean previous worktree was detached before removing the local branch. Release PR #12 was skipped. No live Slack connection, messages, real tokens or Socket Mode were used in this run. The remaining backlog means the schedule stays active.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Inspect retrieved chunks and distances from the dashboard

1 participant