Skip to content

Add DeepSeek Harness - #2

Open
Jacob953 wants to merge 4 commits into
kagent-dev:mainfrom
Jacob953:feat/deepseek-harness
Open

Jacob953 wants to merge 4 commits into
kagent-dev:mainfrom
Jacob953:feat/deepseek-harness

Conversation

@Jacob953

@Jacob953 Jacob953 commented Oct 16, 2026 •

Copy link
Copy Markdown

Change

Closes #1.

Move the DeepSeek Harness integration developed in kagent-dev/kagent#3195 into harnesses/deepseek/ as an independent community experiment maintained by @Jacob953.

  • Connect DSH through ACP and the existing BYO/A2A contract, with persistent conversations and workspaces.
  • Support HTTP MCP tools, protected-tool approvals, DATA-restored public tasks, settled conversation forks, and Linux process cleanup.
  • Include an optional standalone PostgreSQL-backed approval service, deployment examples, tests and source attribution.
  • Map persisted root-session model calls to kagent usage telemetry, including cache tokens, model identity, timing, partial usage and cancellation; exclude restored history from new reporting.
  • Explicitly disable session-log and plugin-inventory uploads while retaining local persistence; cover the existing completion barrier with delayed/failed session/close regression tests.

Targets DSH 0.2.0-rc.2 and kagent 0a57368aca33, including the merged restoration contract from #3165. No shared kagent runtime/controller or DSH source changes are included.

Incremental output remains unsupported. Model usage is emitted after native-process cleanup and is best effort: subagent calls and crash-time backfill are unsupported, and invalid or oversized logs produce a warning without failing the conversation. Native ACP permissions still require a live process. Pending tasks cannot be checkpointed, uncertain remote outcomes are not automatically replayed, and shared recovery limits in #3010 remain. Supported configuration and conditions for adopting other upstream fixes are documented in the README.

Build

Passed from a clean source export, with the harness directory as the entire context and no credentials or build arguments:

docker build --platform linux/amd64 --tag byo-ext/deepseek:pr harnesses/deepseek

The October 19 follow-up rebuilt the AMD64 image from a clean source export and passed non-root configuration/version/supervisor and readiness checks. The ARM64 production build passed during October 16 migration acceptance.

Manual verification

  • October 19: full Linux arm64 bridge race suite passed 144 unique test/subtest cases (52 top-level tests), with no failures, skips or race reports. Usage parser: 22 cases passed. Host short race tests and host/Linux lint also passed. Details are recorded in VERIFICATION.md.
  • Real DSH usage checks cover cache accounting, process reconstruction, approval restore, stream cancellation and corrupt logs. The production A2A server and SDK exporters also passed a local OTLP HTTP/protobuf test: model spans retain their task/context and execution segment, resumed execution links to its origin, and repeated exports or late approvals add no tokens.
  • October 16: all 15 independent Kind cluster scenarios passed, including persistent conversations/workspaces, MCP approvals, Worker replacement after a DATA snapshot, and settled forks.

Tests used real DSH with local model/MCP and controller fixtures, plus PostgreSQL where required. The OTLP receiver decodes network payloads; it is not a deployed collector/backend. Executor reconstruction retains one SDK provider and does not verify cross-process metric aggregation. No live provider calls or full AMD64 runtime regression were performed, and the October 19 follow-up did not repeat cluster acceptance. Historical results, reproducible commands and coverage boundaries are in VERIFICATION.md.

Checklist

  • My commits include DCO sign-offs (git commit -s).
  • I followed the Code of Conduct, including its guidance on AI assistance.
  • I updated the relevant documentation and, for a new harness, the catalog.
  • I included no credentials or private configuration.

AI assistance was used for implementation, testing, documentation and this description.

Migrate the community harness from kagent #3195 with independent builds, DATA restoration, protected MCP approvals, and documented verification boundaries.

AI assistance: implementation, tests and documentation assisted by OpenAI Codex.

Signed-off-by: Junfeng Yu <64075563+Jacob953@users.noreply.github.com>
@Jacob953
Jacob953 marked this pull request as ready for review October 16, 2026 08:35
Disable session-log and plugin-inventory request attachments while retaining local persistence. Verify delayed and failed session/close handling through A2A settlement, and assert that real DSH model requests omit upload fields across process recovery.

AI assistance: Codex assisted implementation, tests and documentation.

Signed-off-by: Junfeng Yu <64075563+Jacob953@users.noreply.github.com>
@Jacob953
Jacob953 force-pushed the feat/deepseek-harness branch from cc2c809 to 697f102 Compare October 17, 2026 14:29
Map root-session model calls into the existing kagent telemetry contract after native cleanup. Preserve cache accounting, partial usage and cancellation while excluding prior turns after restoration. Cover pinned raw and zstd logs, real DSH approval recovery and stream cancellation.

AI assistance: Codex assisted implementation, tests and documentation.

Signed-off-by: Junfeng Yu <64075563+Jacob953@users.noreply.github.com>
Exercise the production A2A server and SDK exporters with real DSH and a local OTLP receiver. Check cache totals, restored approval span ownership and repeat-export deduplication. Clarify the pinned cluster acceptance and document focused verification commands.

AI assistance: Codex assisted tests and documentation.

Signed-off-by: Junfeng Yu <64075563+Jacob953@users.noreply.github.com>
@Jacob953

Copy link
Copy Markdown
Author

@EItanya, a note on the collaboration behind the recent updates: @foxj77's independent investigation and our discussion in kagent#3194 informed several improvements in this PR. Thank you, @foxj77, for sharing the findings and spike evidence.

  • Usage reporting: his mapping notes and cache-accounting caveat informed the session-log integration. I checked the accounting against our pinned 0.2.0-rc.2 / llm-pi-ai path and added coverage for compressed logs, historical model context, cancellation and recovery without duplicate reporting. The OTLP export test also verifies token totals and span ownership after approval restoration.
  • Upload defaults: his findings prompted explicit disabling of session-log and plugin-inventory uploads, with request-level regression checks while retaining local persistence.
  • Persistence: our follow-up confirmed that the existing session/close flush addresses the completion window he identified. I added delayed/failed-close regression tests around that barrier.

We also clarified the distinction between adapter-owned MCP approvals and native ACP permissions: the durable MCP proxy is already implemented here; native permissions still require a live process.

His spike used 0.2.1-alpha.5 / llm-deepseek on macOS. This PR's Linux validation and remaining limits are recorded separately in VERIFICATION.md. The implementation, validation and ongoing maintenance remain my responsibility; this acknowledgment does not imply a formal review or maintenance commitment from @foxj77.

AI assistance: Codex helped with implementation, testing and this update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Harness]: DeepSeek Harness

1 participant