Repository navigation
fix(api): reject blank document fields before indexing - #46
Merged
Merged
Conversation
Validate document title, source and content before persistence. Preserve valid content formatting and cover rejected input, source defaults and successful indexing with offline tests. Co-authored-by: OpenAI Codex <noreply@openai.com>
Contributor
Author
|
Captured offline against commit The backend gate listed these new tests, all passing. Invalid requests leave the seeded database unchanged and never call indexing. Successful requests test both the default and explicit source, the values passed to indexing, and read-back of the saved document. The frontend gate also passed. No frontend UI changed. The backend started and stopped through The real HTTP capture used curl against the fixture backend. Temporary SQLite stores the documents and temporary Chroma uses fixed embeddings. No model, embedding provider, Pi runtime or Slack connection was used. Empty retrieval before and after rejected writes proves that invalid requests did not add chunks. The final fixed-vector distance demonstrates successful indexing and exact content preservation, not relevance quality. The temporary proof guard was loaded through PYTHONPATH before importing the application. The clean process environment supplied fixture-only Slack credentials, MISE_NO_ENV=1, UV_PYTHON=3.13 and UV_FROZEN=1. The following setup keeps the application singletons and blocks external connections. It is a local capture harness, not a shipped application change. """Local proof guard. Never imported by the shipped application."""
import ipaddress
import socket
from slack_sdk import WebClient
from slack_bolt.adapter.socket_mode import SocketModeHandler
_original_connect = socket.socket.connect
def local_connect(sock, address):
if isinstance(address, tuple):
host = address[0]
try:
local = ipaddress.ip_address(host).is_loopback
except ValueError:
local = host == "localhost"
if not local:
raise AssertionError(f"Offline proof blocked outbound connection to {host}")
return _original_connect(sock, address)
def blocked(*args, **kwargs):
raise AssertionError("Offline proof must not contact Slack or start Socket Mode")
socket.socket.connect = local_connect
WebClient.auth_test = lambda *args, **kwargs: {
"ok": True, "bot_id": "BTEST", "user_id": "UTEST", "team_id": "TTEST",
}
WebClient.api_call = blocked
SocketModeHandler.start = blocked
import os
if os.environ.get("WINGMAN_PROOF_FIXTURES") == "1":
import chromadb
from chromadb.config import Settings as ChromaSettings
from chromadb.telemetry.product.posthog import Posthog
from langchain_community.vectorstores import Chroma
from langchain_core.embeddings import Embeddings
from app.vector_store import vector_store
class FixtureEmbeddings(Embeddings):
def embed_documents(self, texts):
return [[1.0, 0.0] for _ in texts]
def embed_query(self, text):
return [1.0, 0.0]
Posthog.capture = lambda *args, **kwargs: None
vector_store.vector_store = Chroma(
client=chromadb.PersistentClient(
path=os.environ["WINGMAN_PROOF_CHROMA"],
settings=ChromaSettings(anonymized_telemetry=False),
),
collection_name="document-validation-proof",
embedding_function=FixtureEmbeddings(),
)
vector_store._initialized = TrueBackend capture method (a) succeeded. No capture method was blocked; a UI screenshot was not applicable. No real tokens or .env files were read, no Slack message was sent and Socket Mode was never started. |
Contributor
Author
|
Revalidated the existing slice at The following commands passed in a clean process environment with
Backend capture method (a) succeeded again. No capture path was blocked. No frontend UI changed, so a screenshot was not applicable. No live Slack verification or migration was needed. The new document tests passed again. The fixture API used fresh local SQLite and Chroma storage with fixed embeddings, retaining the shared RAG and vector-store singletons. The following curl transcript proves that all three blank fields are rejected with no documents or chunks stored, then valid content is stored and retrieved with formatting intact. Distance zero proves indexing wiring only, not semantic relevance. |
Contributor
Author
|
Merged as 5e381ff. Re-ran mise --no-env run test-backend and mise --no-env run test-frontend on the fetched origin/main commit using the same offline guard and clean fixture environment. Both passed again, with 172 backend tests and 15 frontend tests. The tracked worktree is clean. The remote agent branch and its leftover local branch were deleted; the previous worktree was clean and left detached with its files preserved. Child #45 is closed by this merge; parent #5 remains open. Release PR #12 was skipped. No new slice was started under the existing-agent-PR lease. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reject empty and whitespace-only document titles, content and sources with HTTP 422 before storage or embedding. Trim titles and sources while preserving valid content exactly, including code indentation and trailing newlines.
Part of #5. Closes #45.
Verified with the repository gates and local API capture.
UV_PYTHON=3.13 UV_FROZEN=1 mise --no-env run installpassed. The first unrestricted attempt selected Python 3.14, which has no wheel for locked onnxruntime 1.23.2. Python 3.13 matches mise.toml; the frozen lockfile avoids unrelated dependency changes.mise run test-backendpassed, 172 tests including 19 new document creation cases. Existing dependencies emit 731 deprecation warnings.mise run test-frontendpassed, 15 tests.mise run dev-backend -- --host 127.0.0.1 --port 18745reached application startup complete, served the curl capture, and shut down cleanly.git diff --checkpassed before the commit.The checks used a clean environment with
MISE_NO_ENV=1,UV_PYTHON=3.13,UV_FROZEN=1, fixture Slack credentials and a temporary Python guard that blocks Slack API calls, Socket Mode and non-loopback socket connections. No real credentials or .env files were read. The API proof used temporary local SQLite and Chroma storage with fixed embeddings. The shared RAG and vector-store singletons and their real indexing path were retained.The curl transcript shows all three blank fields returning 422, document and retrieval lists unchanged after rejection, and a valid code document stored and retrieved with its formatting intact. Fixed-vector distance 0 demonstrates indexing wiring only, not semantic relevance. The proof comment contains the full transcript, new test names and capture setup.
Backend capture method (a) succeeded, with an additional fixture retrieval transcript. No frontend UI changed, so a screenshot was not applicable. No capture method was blocked. No migration or live Slack verification is needed for this slice. Existing documents are unchanged; this validation applies to new API submissions.