Skip to content

fix(mcp): use chatbot embedding retrieval for ask security facts - #1155

Open
northdpole wants to merge 3 commits into
feat/oie-rq-fanoutfrom
fix/ask-embedding-retrieval
Open

northdpole wants to merge 3 commits into
feat/oie-rq-fanoutfrom
fix/ask-embedding-retrieval

Conversation

@northdpole

Copy link
Copy Markdown
Collaborator

Summary

  • Share PromptHandler.retrieve_nearest_standard_for_prompt between chatbot and MCP ask so security facts use the same embedding similarity path instead of text_search REST.
  • Stacked on feat/oie-rq-fanout (feat(oie): RQ A→B→C fan-out and Admin OIE queue tab #1142), which includes the lint fix restoring validate_cron_line and exempting admin /rest/v1/api-keys from OpenAPI coverage.

Test plan

  • make lint (openapi-guardrail OK)
  • python -m unittest application.tests.mcp_ask_test (14 tests)

Made with Cursor

@coderabbitai

coderabbitai Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review: 1
  • main

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@northdpole
northdpole force-pushed the fix/ask-embedding-retrieval branch 2 times, most recently from 1129c1f to 7e37a9d Compare October 20, 2026 08:29
@northdpole
northdpole force-pushed the fix/ask-embedding-retrieval branch from 7e37a9d to fdda36d Compare October 20, 2026 08:48
@northdpole
northdpole force-pushed the fix/ask-embedding-retrieval branch from fdda36d to 22f42fa Compare October 20, 2026 09:21
@northdpole
northdpole force-pushed the fix/ask-embedding-retrieval branch from 22f42fa to d6126a4 Compare October 21, 2026 23:09
@northdpole
northdpole force-pushed the fix/ask-embedding-retrieval branch from d6126a4 to d2416dc Compare October 22, 2026 07:59
@ShaanGS

ShaanGS commented Oct 23, 2026

Copy link
Copy Markdown

ask reports setup failures as "no match". On this branch (d2416dc, same on the stack tip), with the stdio server pointed at opencre.org and no local DB, every security prompt returns ok: false with "No CRE/standard embedding match above similarity threshold." The real error only shows up in stderr at INFO: no such table: embeddings, or Missing Gemini API key with the make e2e-db fixture DB.

_retrieve_nearest_standard catches it and returns None, the same as a real miss, and the tests mock that function, so this path isn't covered. Could it return a distinct message or raise (and log at WARNING), or fall back to REST text_search like #1142 did when local retrieval can't run?

Also, the "Security fact pack" section in docs/api/mcp.md still says step 1 is text_search. Happy to help once the stack lands.

northdpole and others added 3 commits October 25, 2026 10:31
Share PromptHandler.retrieve_nearest_standard_for_prompt between chatbot
and MCP ask so security facts use the same similarity path instead of
text_search REST.

Co-authored-by: Cursor <cursoragent@cursor.com>
Harden intent-guard, board year, CRE bait, clarify, and JuiceShop matching so staging MCP ask stops failing the small-24 edge set.

Co-authored-by: Cursor <cursoragent@cursor.com>
ASVS/MASVS/SAMM levels and AIX layers were answered with bare Nest
project_lookup (leaders / project stage). Add node_facts (curated facts
stored as evidence docs), replace aix_facts/cre_facts, and fail closed so
structure asks never resolve to a Nest project pick.

- narrow evidence routing to structural cues, not bare brand tokens
- facts surface on ask items only, gated on structural prompts
- seed_node_facts re-embeds evidence docs only
- add structure product-bad fixtures and Nest-vs-evidence tests

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@northdpole
northdpole force-pushed the fix/ask-embedding-retrieval branch from d2416dc to 2f9187c Compare October 25, 2026 10:35

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants