[johnnyhuy]

A document can now be indexed again with POST /api/documents/{document_id}/index after its initial embedding fails. API-created documents use stable chunk IDs, so retrying replaces owned chunks and removes obsolete fragments after a successful upsert.

Part of #3. Closes #51.

Verification

All commands ran with a cleared inherited environment using env -i HOME="$HOME" PATH="$PATH". This worktree has no root or backend .env; none was read or changed. Python commands selected UV_PYTHON=3.13 and the tests used fixture Slack credentials and DATABASE_URL=sqlite:///.tmp-tests.db.

• UV_PYTHON=3.13 MISE_YES=1 mise run install passed for backend, frontend and runtime. Generated lock-file churn was discarded; dependency declarations and lock files are unchanged.
• UV_PYTHON=3.13 UV_FROZEN=1 MISE_YES=1 SLACK_BOT_TOKEN=xoxb-fixture-only SLACK_SIGNING_SECRET=fixture-only DATABASE_URL=sqlite:///.tmp-tests.db PYTEST_ADDOPTS=-rP mise run test-backend passed, 209 tests including 13 new cases, with 928 existing-library and fixture warnings. The new cases use temporary SQLite and real local Chroma with fixed embeddings.
• MISE_YES=1 mise run test-frontend passed, 22 tests and 187 assertions.
• MISE_YES=1 mise run test-runtime passed, 7 offline tests.
• UV_PYTHON=3.13 UV_FROZEN=1 MISE_YES=1 PYTHONPATH="$PWD/.tmp/offline:$PWD/backend" SLACK_BOT_TOKEN=xoxb-fixture-only SLACK_SIGNING_SECRET=fixture-only DATABASE_URL="sqlite:///$PWD/.tmp/offline/proof.db" mise run dev-backend reached Application startup complete, served the recorded curl requests and was stopped with Ctrl-C, reaching Application shutdown complete. The local sitecustomize.py hook mocked Slack authentication, blocked Slack API calls and Socket Mode startup, and initialised the existing vector-store singleton with local Chroma and fixed embeddings. A marker file injected an embedding outage for the first upload, then was removed before retrying.
• git diff --check passed. No migration is needed.
• GitHub checks passed on fa34ac3, including tests and all three container builds, CodeQL and Socket Security.

Proof

Committed test output, fixture retrieval transcript and local curl transcript. Capture methods (a) and (c) succeeded. No capture method was blocked. This slice changes the API and retrieval indexing; there is no dashboard UI change to capture.

The API transcript shows an empty retrieval result, an upload returning HTTP 503 with saved document ID 1, the saved content remaining readable, two successful index retries and exactly one retrieved chunk at raw squared L2 distance 1.0. The list still contains one document. The missing-ID request returns HTTP 404. The fixture test independently checks one vector chunk after two retries. Fixed-vector distances demonstrate recovery and identity; they do not measure production relevance.

New backend test names and suite result

tests/test_document_indexing.py::test_create_and_retry_keep_one_owned_chunk_and_preserve_saved_record PASSED [ 94%]
tests/test_document_indexing.py::test_retry_recovers_saved_document_after_embedding_failure_with_transcript PASSED [ 94%]
tests/test_document_indexing.py::test_retry_rechunks_and_removes_only_obsolete_owned_chunks PASSED [ 95%]
tests/test_document_indexing.py::test_retry_reads_latest_saved_content_title_and_source PASSED [ 95%]
tests/test_document_indexing.py::test_retry_failure_keeps_previous_chunks_and_saved_content[lookup] PASSED [ 96%]
tests/test_document_indexing.py::test_retry_failure_keeps_previous_chunks_and_saved_content[embed] PASSED [ 96%]
tests/test_document_indexing.py::test_retry_failure_keeps_previous_chunks_and_saved_content[upsert] PASSED [ 97%]
tests/test_document_indexing.py::test_cleanup_failure_is_reported_and_retry_finishes_without_duplicates PASSED [ 97%]
tests/test_document_indexing.py::test_missing_saved_document_does_not_initialise_vector_store PASSED [ 98%]
tests/test_document_indexing.py::test_invalid_legacy_document_is_rejected_without_changing_chunks[content- \n\t ] PASSED [ 98%]
tests/test_document_indexing.py::test_invalid_legacy_document_is_rejected_without_changing_chunks[content-None] PASSED [ 99%]
tests/test_document_indexing.py::test_invalid_legacy_document_is_rejected_without_changing_chunks[title-] PASSED [ 99%]
tests/test_document_indexing.py::test_invalid_legacy_document_is_rejected_without_changing_chunks[source-None] PASSED [100%]
====================== 209 passed, 928 warnings in 6.48s =======================

Review notes

POST /api/documents now returns HTTP 503 with detail.message and detail.document_id when the database save succeeds but indexing fails. Use the index endpoint with that saved ID, rather than posting another copy. Existing successful creation responses are unchanged. Embedding work runs in the shared thread pool.

Retrying does not modify database records. It rejects invalid legacy fields before indexing, preserves other document IDs even when titles and sources match, and leaves Slack chunks and legacy chunks without document ownership untouched. Legacy duplicates can remain and require a deliberate collection rebuild. File indexing without a saved document ID retains its existing behaviour.

Database storage, chunk upsert and obsolete-chunk deletion are separate operations. Failed cleanup is reported as HTTP 503 and can leave a mix of old and new chunks until retry succeeds. Upsert is not a cross-store transaction. Changing the embedding model or vector dimensions still requires the existing collection migration or rebuild process.

The management API retains its existing access-control requirements. This endpoint calls the configured embedding provider in a real deployment. No live Slack connection, real token, remote database, model request or paid embedding call was used for verification. Live embedding-provider behaviour and production dataset performance remain unverified; a live Slack workspace is not needed for this slice. The parent issue remains open, and #51 closes only when this PR merges.