Skip to content

docs: the night's foundation, the quality gate, Spot as a probe, the datasets page, the night's figures (night docs) - #98

Merged
rusholme merged 1 commit into
mainfrom
night/docs
Oct 24, 2026
Merged

rusholme merged 1 commit into
mainfrom
night/docs

Conversation

@rusholme

Copy link
Copy Markdown
Collaborator

night-2026-10-24 task 10's rapid_docs half: the night carried into the design pages. Every page stays DRAFT.

What and why

The brief's task 10: rapid_docs carries Part 1's settings statement and the association amendment, the quality gate, the alert contract, the Spot probe's rules and first data point, the two new datasets' quirk lists, and the null-test and injection figures with their purity floor. Task 7 (PR 97) had already written the association on difference.md, loop.md, alerts.md (the contract, best information at the time) and decisions.md, and left the foundation page's Part 1 and the amendment to this task.

Files

  • system/foundation.md: Runs (the three settings layers resolved at run create); the products table and identity (objects and memberships unbound, official through their source sets); Inputs (two kinds, the object catalog read at execution and recorded, the exception); the official views (memberships_official, objects_official); Why (the chain bullets replaced by the append-only catalog, the reconcile, superseded memberships; forced rows read at execution); the verbs (run create layers, job wait, publish as the recovery sweep, loop --passes --job-queue, --excludes gone); a job on Batch (the Task failed to start retry, the loop's one rerun); Spot, a probe (new); the thin chain table and an Association section in place of associate; packet (best information at the time); The quality gate (new, label foundation-quality-gate); Publication to Kafka rewritten (the packet job's own send, dataset key, offsets on the stamp, not idempotent, the 300 s sweep, the direct consumer check); the procedure proof (test_steps.py, steps 1, 2 and 13 rewritten); On SMDC parts 10 and 11; validation checks 3 and 8 restated for the catalog; Measurements, The second iteration (new); Not decided here updated (the chain-cost, producer-grant, gate-not-built and injection bullets closed or narrowed; the field rule, materialisation, in-flight edge case, exactly-once, interruption capture, OpenUniverse completeness, GBTDS residual added).
  • system/datasets.md (new): HLWAS r00342, GBTDS r00340, OpenUniverse TDS, one section each, data and a quirk/disposition table.
  • system/decisions.md: decision-bright-star-flag and decision-spot-probe (Ben, 2026-10-24 UTC, from the brief's record of his 04:00 to 04:40 rulings); superseded notes on decision-spot-parked and on the chain half of decision-chain-admission-order.
  • system/difference.md: the reference PSF stamp's centring (secant, then grid search), the new metrics, the bright-star footprint flag, the reference_psf setting.
  • system/intake.md: reader, dq_mask_bits, crpix_shift, rederive_wcs in the settings table; the Gaia-offset metric.
  • system/loop.md: --passes, --job-queue, the Kafka step as the recovery sweep with sets_waiting and held_sets, the submission cap, the Spot queue, the one automatic rerun.
  • system/index.md: scope lines for the pages touched, the datasets row and toctree entry.

Divergences from the plan or the brief, with the reason

  • The quirk lists are one page, datasets.md, rather than one page each: the brief allows either, and a third dataset (HLWAS) belongs beside them as the overlay's source of truth.
  • alerts.md is unchanged: PR 97 already states the contract.
  • No local Sphinx build: the docs venv is not on this host and the brief forbids installs; CI's build is the check. scripts/check-public-safety.sh passed locally.

Measurements

The figures on the pages are the runbook's part 11 (rapid PRs 362 to 386), not re-measured here. The ones that decide something:

value source
detection, SCA 8 footprint, AB 21 to 25 0.94 to 0.98 part 11 step 9
alerted of in-field unflagged detections 12,069 / 12,070 step 9
purity AB 21 to 22 / 26 0.91 to 0.93 / 0.20 step 9
false positives per epoch, median 95 (52 bright-flagged, 38 other) step 9
Spot twin vs on-demand 30.2 vs 28.1 min, $1.30 vs $2.03 by lane, 0 reclaims step 9
last difference to last packet 31 s vs 5.2 min chained step 7

Improvements seen

  • The foundation page is 1,300+ lines and narrates every part of the SMDC procedure; its On SMDC and Measurements sections would read better as one short summary per part pointing at the runbook (foundation.md).
  • Spot's rules are stated on the foundation page and summarised on the loop page; if they grow, the loop page should hold them and the foundation point at it.

Session notes

  • The docs venv is absent on the fleet host and building it is a package install, so a docs task here cannot run the brief's Sphinx check locally; CI is the only build.

What the next task must know

The association's field rule is the next science question (foundation, Not decided here); OpenUniverse's chain past intake needs the three pieces on the datasets page.

🤖 Generated with Claude Code

…, the quality gate, Spot as a probe, the datasets page, the night's figures (night docs)

foundation: Part 1's settings statement (three layers resolved at run
create), the association amendment through the inputs, views, why,
thin chain, packet and validation sections, the quality gate, the
packet job's own send with the recovery sweep and the direct consumer
check, Spot's four rules, the procedure proof's steps, On SMDC parts
10 and 11, the second iteration's measurements (the null test with the
bright end flagged, completeness and purity per magnitude with the
purity floor, association timing, Spot's data points) and Not decided
here brought up to date. datasets: new, HLWAS, GBTDS and OpenUniverse
quirks with dispositions. decisions: the bright-star flag and Spot as
a probe (Ben, 2026-10-24), two superseded notes. difference, intake,
loop: the flag, the new metrics and settings, the loop's operations.
Every page stays DRAFT.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rusholme
rusholme merged commit 538ab96 into main Oct 24, 2026
2 checks passed
@rusholme
rusholme deleted the night/docs branch October 26, 2026 04:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant