Repository navigation
docs: writing the deploy tag is not deploying — four more copies said it did (C-343) - #497
Merged
Merged
Conversation
… it did (C-343) C-343 was registered in #431 and corrected in scripts/refresh_pipeline.sh and docs/guides/server_quickref.md. Four further copies of the one-step claim were still live, and they are the ones an operator reads: - hetzner_deployment_guide.md said the tag file, then "That's it." This is the file refresh_pipeline.sh itself points to "for full detail". - server_operations.md gave a local `git tag && git push` (which cannot trigger the publish workflow, and cannot be run at all since branch protection) plus the tag file, with no checkout and no `uv sync`. - ADR-022 listed "One-line rollback" as a benefit of the gate, gave a three-step deploy with neither checkout nor `uv sync`, and offered a Verify block naming the tag file and `git describe` but not the installed distribution — the one of the three that was wrong in August. - data_serving_guide.md told newcomers that writing the old tag name back puts you on known-good code. server_quickref.md is now the single operational home and grew the verification it was missing: step 4 checks all three sources, including the installed version, which is the only one that reflects `uv sync`. The four others point there and say the quickref wins on disagreement. ADR-022 carries a dated amendment in the ADR-027 style rather than a rewrite: the Decision it records is sound, and what was wrong was the inference that the gate's own `git checkout` makes the tag file sufficient. C-343 is updated rather than duplicated — same risk, new evidence that the fix reached two of six copies. It stays open; the pre-flight instrument that would catch a seventh copy is still #424. Found because the operator asked, before deploying v1.14.0, whether a one-line tag write was really the procedure. It was not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EfeuULbPTqrx4qtNki5dy9
Polichinel
enabled auto-merge (squash)
October 12, 2026 22:32
Polichinel
added a commit
that referenced
this pull request
Oct 16, 2026
…s read the wrong home (C-343) (#498) Found by running the corrected deploy procedure on the production host. v1.14.0 deployed cleanly, then step 4's first check failed: $ sudo -u views-deploy cat ~/.views-deploy-tag cat: /home/simmaa_prio/.views-deploy-tag: Permission denied The calling shell expands `~` before sudo runs, so the command reads the operator's home rather than views-deploy's. Six such commands were live in the guides; two were added by the C-343 correction in #497. All are now wrapped in `bash -c '...'`, where the inner shell expands `~` as the target user. One of the six failed green rather than loud: sudo -u views-deploy cat ~/.../pipeline_failure.json 2>/dev/null \ || echo "No recent failure" The `||` sat outside `bash -c`, so a permission error on the wrong home printed "No recent failure" — in the command an operator runs to ask whether the last pipeline run broke. The fallback is now inside the quoted command. Step 4's third check also drops the escaped-quote python one-liner for `ls .venv/.../site-packages | grep views_datafactory`, which needs no quoting and prints the dist-info directory — the same evidence form C-343 used in August. C-343 updated with the deploy outcome: `uv sync` reported 1.13.0 -> 1.14.0 on the box, which is direct confirmation that step 3 is load-bearing and the tag file alone would have left 1.13.0 installed. Claude-Session: https://claude.ai/code/session_01EfeuULbPTqrx4qtNki5dy9 Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Polichinel
added a commit
that referenced
this pull request
Oct 16, 2026
… carry (C-343) (#499) The 2026-10-13 consolidation made server_quickref.md the single operational home and told every other guide "if they disagree, the quickref wins." hetzner_deployment_guide.md carried a rule the quickref did not — start tmux before a pipeline run, stated at its line 30 with the reason, and backed by a post-mortem section called "tmux saved us from the SSH drop". Promoting the quickref silently demoted that rule. It cost a run the same day. `client_loop: send disconnect: Broken pipe` killed a foreground pipeline after the 37 GB assembly had completed but before the zarr export, leaving grid.npy current and grid.zarr three weeks stale. Every designed defence held: assemble_grid.py renames from grid.npy.tmp so the grid was whole, .exports_required recorded the owed export, flock released with the process, and check_health.py reports the sentinel as PENDING. Nothing was corrupted and nothing was silent. The cost was a re-run. Every interactive refresh_pipeline.sh invocation in the guides now opens with `tmux new -s pipeline`: - server_quickref.md §"Run the pipeline" — the rule, the dated evidence, detach/reattach, and the detail that catches people out: tmux does not change the shell prompt, so only the bottom status bar tells you whether you are inside it - hetzner_deployment_guide.md §2.1, §4.5, and the rollback section added in #497 - server_operations.md summary table and the full-rebuild procedure Cron invocations are deliberately unchanged — cron has no controlling terminal and no SIGHUP to receive. The ~/.profile wrong-vs-right example in server_operations.md is also unchanged; adding tmux would obscure the point it exists to make. C-343 gets the pattern rather than a new entry: a consolidation is only safe if the new authority absorbs everything the demoted copies knew, and checking that was not part of the 2026-10-13 work. Claude-Session: https://claude.ai/code/session_01EfeuULbPTqrx4qtNki5dy9 Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Polichinel
added a commit
that referenced
this pull request
Oct 20, 2026
…ve times in eight days (C-343) (#509) Covers the export phase instrumentation (#501/#502), four rounds of C-343 correction (#497-#500), the release/publish/deploy of v1.15.0, a fifth C-343 round found while writing this up (#507), and a licensing contradiction found while onboarding an external consumer (#508). The v1.11.0 post-mortem named the spine running through everything it reported: "a written record that was true when written and stopped being true without anything failing." Both it and its companion observation — that the post-mortem convention keeps failing to be applied — recurred. Three of the four releases since are unwritten and now never will be. The spine produced five fresh instances in eight days, all in C-343, the fifth found inside the register entry created to track the first, including in its Trigger: field. A sixth variant was committed by the author of the fourth correction three days later: merging the instrumentation is not deploying the instrumentation. The conclusion recorded is not "try harder". Every instance in this window was found by a human asking an unprompted question or by an author re-reading their own work. Five rounds have produced a procedure that is right and nothing that notices when a copy of it goes wrong. Also records what went well rather than only the failures: block 4 of the deploy procedure executed successfully for the first time since it was written, the #501 falsification criterion was committed before any production figure existed, three guards were sabotaged and confirmed red, and the clean- venv publish check caught a defect in itself rather than a false alarm on the release. Verification: validate_docs.sh PASSED; all 14 register-reading test files 41 passed. Every figure quoted is cited to a file rather than recalled; the release count was corrected against `git tag` before committing. Refs #501, #508. Claude-Session: https://claude.ai/code/session_01EfeuULbPTqrx4qtNki5dy9 Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Before deploying v1.14.0 the operator asked whether the one-line tag write I had given them was really the deploy procedure. It was not — it is the exact mistake C-343 (Tier 2, open) exists to prevent, and the reason the question had to be asked is that C-343's own correction was partial.
#431 registered C-343 and fixed
scripts/refresh_pipeline.shanddocs/guides/server_quickref.md. Four more copies of the one-step claim were still live, and they are the ones an operator reads.docs/guides/hetzner_deployment_guide.mdrefresh_pipeline.shpoints to "for full detail"docs/guides/server_operations.mdgit tag && git push(cannot trigger the publish workflow, and cannot be run at all under branch protection) plus the tag file; no checkout, nouv syncdocs/ADRs/022_tag_based_deployment_gate.mduv sync; a Verify block naming the tag file andgit describebut not the installed distribution — the one of the three that was actually wrong in Augustdocs/guides/data_serving_guide.mdThe ADR is the worst of the four, because it is the governing document and it sold the defect as a feature.
What changed
server_quickref.mdis the single operational home, and grew the verification block it was missing: step 4 checks all three sources — tag file, checked-out tag, and installed distribution version, the only one that reflectsuv sync. Also states that step 1 alone is not a deploy, and that a rollback is a deploy.ADR-027style rather than a rewrite. The Decision it records is sound: the gate does refuse to start without a valid tag and does check that tag out. What was wrong was the inference that the checkout makes the tag file sufficient. The amendment states both reasons (bash has already buffered the script;uv syncis never invoked) and notes the entry is still open.Verification
uv run ruff check .uv run pytest(full, unpiped)FAILEDlinesbash docs/validate_docs.shuv run pytest tests/test_docs_citations.pyfile.sh:Ncitations introducedNo source, test or dependency changes; header counts unchanged (an amendment, not a new entry).
Also worth noting for the record: the first full-suite run failed
test_falsification_deploy_v160.py::TestF2ReleaseTopology::test_main_is_ancestor_of_development. Not a topology break — remotemainanddevelopmentare healthy; the localdevelopmentref was two commits stale at801177d, and this gate reads local refs by design. Fast-forwarded to47fa8b4and it passes. The gate did its job.What this does not fix
Nothing mechanically prevents a seventh copy, or prevents the procedure being skipped on the box. The pre-flight check that would — comparing
$DEPLOY_TAGagainst the HEAD the script started from, failing loud per ADR-011 rather than self-correcting one run late — is still #424 and still unbuilt. C-343 stays open.Refs #424, #402. Closes nothing.
🤖 Generated with Claude Code
https://claude.ai/code/session_01EfeuULbPTqrx4qtNki5dy9