Skip to content

Implement Rust/C# GraphQL comparison with conformance, benchmarks and reports - #5

Merged
konard merged 7 commits into
mainfrom
issue-4-bf4f2c4ef803
Oct 7, 2026
Merged

konard merged 7 commits into
mainfrom
issue-4-bf4f2c4ef803

Conversation

@konard

@konard konard commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

The repository promised Deep vs Doublets Rust/C# but only contained submodules and a mutation against an obsolete endpoint. This adds runnable Rust and C# GraphQL servers over native Doublets stores, a pinned PostgreSQL/Hasura baseline, and one shared HTTP driver for create/update/delete and five query variants.

The workflow checks conformance before benchmarking each server independently. PRs run B=1,000/N=10; main runs B=10,000/100,000/1,000,000 with N=100. Reports validate raw samples, preserve the handwritten README, and generate medians, deviations, ratios, dependency/CPU/run/date/commit provenance, and linear/log charts. CI uploads PR reports and commits generated results only on main.

Scope: issue #1 has no normative schema attachment, so schema/links.graphql publishes the executable common links subset following Data.Doublets.Gql's field conventions. The baseline reproduces Deep's core links table/indexes at deeplinks@381207d, without its application triggers, permissions, values, relationships or subscriptions. Both adapters are maintained here because the legacy submodule implementations require old runtimes and have unfinished resolvers. Original submodule pointers are preserved. README documents the workload domain and differences in id allocation and durability.

During implementation, repeated mutations exposed an index hang in Rust doublets 0.5.0's split store. A native 20-link test failed within its five-second bound; switching both adapters to united stores makes it pass. experiments/repeated-mutations.md explains reproduction, and experiments/repeat_workload.py retains the bounded HTTP probe. A second reproducing test shows that successful HTTP responses containing unchanged updates must be rejected; every measured response is now validated outside timing.

Validation:

  • 14 Python unit tests and two Rust tests pass, including reproducing regressions for repeated mutations, successful no-op responses, keeping generated raw archives commit-ready, and rejecting non-HTTP initializer URLs.
  • Schema introspection and 26 explicit expected responses pass identically against all three real HTTP servers: each query variant, membership/range/boolean filters, ordering/pagination, partial updates, missing ids, batch deletes, and final state.
  • All eight operations complete locally on each backend with B=1,000/N=10, ten measured samples and two warm-ups; background state remains intact. Generated tables and both chart scales were rendered and inspected.
  • All three backends also complete a bounded million-link run with N=100, two measured samples and one warm-up, including response validation and final-state verification. This checks the largest workload size; it is not the ten-sample published comparison.
  • Rust formatting/Clippy, C# locked restore/format/build, Ruff, actionlint and diff checks pass. Rust, .NET SDK/runtime, package locks, images and CI actions are pinned.

The latest PR workflow passes all five jobs and produces a validated three-backend report artifact. CodeFactor also passes on 13ef6bf.

Published full comparison results will be produced by the first successful main workflow after merge; PR results are available as workflow artifacts.

Fixes #4

Changes

  • 26 file(s) modified
  • 3433 line(s) added
  • 2 line(s) removed
  • Files:
    • .github/workflows/benchmarks.yml
    • .gitignore
    • README.md
    • compose.yml
    • csharp/Doublets.Gql.csproj
    • csharp/Program.cs
    • csharp/packages.lock.json
    • deep/links.sql
    • experiments/compose.local.yml
    • experiments/repeat_workload.py
    • experiments/repeated-mutations.md
    • global.json
    • requirements.txt
    • rust-toolchain.toml
    • rust/Cargo.lock
    • rust/Cargo.toml
    • rust/src/main.rs
    • schema/links.graphql
    • scripts/client.py
    • scripts/conformance.py
    • scripts/init_deep.py
    • scripts/report.py
    • scripts/run.py
    • scripts/workload.py
    • tests/test_comparison.py
    • tests/test_report.py

Adding .gitkeep for PR creation (default mode).
This file will be removed when the task is complete.

Issue: #4
@konard konard self-assigned this Oct 6, 2026
@konard konard changed the title [WIP] Implement the comparison in both Rust and C# with tests, CI and generated results Implement Rust/C# GraphQL comparison with conformance, benchmarks and reports Oct 7, 2026
@konard
konard marked this pull request as ready for review October 7, 2026 00:23
@konard

konard commented Oct 7, 2026

Copy link
Copy Markdown
Member Author

Working session summary

PR #5 is ready for review.

Implemented Rust/C# servers, shared core-links benchmarks, conformance tests, pinned CI and generated reports. All CI checks and CodeFactor pass; million-link runs succeeded on all three backends.

The working tree is clean. Full comparison results will publish after merge to main.


This summary was automatically extracted from the AI working session output.

@konard

konard commented Oct 7, 2026

Copy link
Copy Markdown
Member Author

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

  • Model: GPT-6.1 Sol
  • Provider: OpenAI
  • Public pricing estimate: $4.802329

📊 Context and tokens usage:

  • 365.9K input tokens across requests (cumulative, larger than the 200K context window, so not one request's context), 93.2K / 128K (73%) output tokens

Total: (365.9K + 9.7M cached) input tokens, 93.2K output tokens, $4.802329 cost

🤖 Models used:

  • Tool: OpenAI Codex
  • Requested: gpt-6.1-sol
  • Thinking level: xhigh (~31999 tokens)
  • Model: GPT-6.1 Sol (gpt-6.1-sol)

📎 Log file uploaded as Gist (6173KB)


Now working session is ended, feel free to review and add any feedback on the solution draft.

@konard
konard merged commit 9e4f90a into main Oct 7, 2026
6 checks passed
@konard

konard commented Oct 7, 2026

Copy link
Copy Markdown
Member Author

🎉 Auto-merged

This pull request has been automatically merged by hive-mind.

  • All CI checks have passed

Auto-merged by hive-mind with --auto-merge flag

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement the comparison in both Rust and C# with tests, CI and generated results

1 participant