Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .claude/board/FRUITS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ Generated by `.claude/tools/fruits_index.py --write`. Do not hand-edit:
add or change a `**Fruit:**` line on the ledger row and regenerate.
CI fails when this file is stale.

Open: 24. Closed: 1.
Open: 25. Closed: 1.

## discovery torque

Expand All @@ -14,6 +14,7 @@ _none open_

| tier | row | status | ledger |
|---|---|---|---|
| High-signal | 2026-10-23 — A certificate replaces the deltas: lane 0 keeps the delta, a receipt or seal, and in time a proof reference — never the operations | Open. | `IDEAS.md` |
| High-signal | 2026-10-23 — Lumpability as the closure check: a palette composes under an operation only if its cells are lumpable for it, and purity measures that | Open | `IDEAS.md` |
| High-signal | 2026-10-23 — One replay that checks proof, receipt and commit witness together | Open. | `IDEAS.md` |
| High-signal | 2026-10-23 — OGAR #357's first kill test (TLAC-P01) is now buildable from real guards | Open. | `IDEAS.md` |
Expand Down
114 changes: 114 additions & 0 deletions .claude/board/IDEAS.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,120 @@ Agents filter by `@`-mention or domain to see what's theirs.

(Prepend new ideas here with today's date. Format:)

## 2026-10-23 — A certificate replaces the deltas: lane 0 keeps the delta, a receipt or seal, and in time a proof reference — never the operations

`[P1 @thought-struct-scribe @truth-architect domain:thought-git D-TGA-5 D-TGB-4 D-TGB-7 D-DRMC D-CGW-4]`

Operator direction (provenance): lane 0 is main, 1..63 are thoughts, and each
lane may run on the order of 10^6 operations. Written with session 01G22yT6's
mapping (EXO-50, 2026-10-23).

**Storage arithmetic, not a measurement.** 64 lanes × 10^6 operations is
6.4·10^7 deltas per cycle: 1–2 GB at 16–32 bytes each, or 33 GB at 512-byte
rows. A few cycles a minute reaches terabytes within hours. A receipt per
landed thought is about 64 × ~100 bytes per cycle. So deltas stay in RAM and
die at squash; only the squashed delta, receipts and seals reach disk.

**What exists.**
- **D-TGB-4** (#1674, EXO-58 done) replaces history with a receipt. It
proves only the replay half, not admissibility.
- **egglog** checks a proof for every admissible solution.
- **DR64** refuses every recipe with `NoCertifiedCompiler`. That is the
certified-compiler-boundary row below, and its scope is the operator's call.
- **The joint replay** (proof + receipt + lane read) is the 2026-10-23 IDEAS
row below.
- **The causal verdict** is computed (`pearl.rs:259`) but promotes nothing
(ISS-D-CGW-4-PROMOTION-GATE).

**The rungs.**
1. **The gate:** squash every chain of origin into the most-updated lane,
refusing on contradiction or divergence, with blame kept over lanes.
MEASURED in `entries/2026-10-23-d-tga-5-squash-gate-preregistration.md`:
landing the squash equals landing the finals, and the delta stays N
cells while the history grows k·N. Update count is activity, not
insight, so the target choice is only a tie-free rule. Correctness comes
from the gate.
2. **The determinism falsifier** (01G22yT6). The receipt replaces the
history only if every lane is deterministic given its seeds. Inject one
nondeterministic read (an outside read, thread timing) into the D-TGB-4
replay test. The replay must refuse, not reproduce different writes
silently.
3. **Promotion at squash time.** CE64 bits 59..63 advance (correlation →
causality, MUL-calibrated rather than overconfident) only when the
squash is contradiction-free and blame shows independent lanes
(EXO-138's stamp condition).
4. **Compiled thought.** An egglog / eggcc certificate replaces the order
list. Lane 0 then stores the delta plus a seal that points at the
certificate, and a certified thought is reusable by reference.

**Outside grounding: the accumulation as a large Markov chain** (operator,
2026-10-23). Source: Bittracher & Schütte, "A probabilistic algorithm for
aggregating vastly undersampled large Markov chains", *Physica D* 416
(2021) 132799,
https://publications.imp.fu-berlin.de/2614/1/1-s2.0-S016727891930764X-main.pdf.
The grades are per claim.

- **[G] within the paper's assumptions (Def. 2.1, Lemma 2.3).**
- *Lumpable*: the probability of moving into an aggregate depends only on
the starting state's aggregate.
- *Deflatable*: within an aggregate, the end state is drawn independently
of where the chain came from.
- A chain that is both is *aggregatable*. Its N×N matrix then has exactly
R distinct columns, so it has rank R, and it equals
`P = Π · P̂ · Λ`.
- **[G] (Alg. 3.1, Eq. 18).**
- Sample J columns, take an SVD and read the aggregates off the sign
pattern of the leading R left singular vectors (spectral clustering,
PCCA+ or SEBA).
- J depends only on R and the confidence p, never on N. With equal-sized
aggregates, recomputed here: R = 8 needs J = 51 for p = 0.99, R = 10
needs J = 66, and R = 16 needs J = 115.
- **[G] (Prop. 3.9, Thm. 2.10).**
- The reduced matrix is recovered from the sampled columns alone:
`P̂ = Λ · P_K · Λᵀ_K · K⁻¹`.
- For an ε-almost aggregatable P, the L1 error is ≤ 4ε, also independent
of N.
- **[H] mapping to thought-git.**
- The 64 × 10^6 operations are the trajectory of one large chain over
cell states. Each lane's run is an expensive sample from its starting
state.
- A knowledge-preserving squash is an aggregation. Lane 0 would store the
partition Ω, the R×R matrix P̂ and the in-aggregate distributions π: the
certificate is P̂, not the operations.
- With the 63 thought lanes as J = 63 column samples, recomputed with
Eq. 18: R ≤ 10 aggregates are recovered with p ≥ 0.987, R = 12 with
0.951, and R = 16 with only 0.758.
- The 4ε bound gives the squash gate a quantitative tolerance where it now
has a yes/no refusal.
- **[H] where the mapping can break.** Each item is a falsifier, not an
assumption.
1. The paper's chain is time-homogeneous. Thought lanes are not, and the
aggregated process is in general non-homogeneous (paper §2).
2. Lanes are chosen thoughts, not uniform independent column draws, so
Eq. 18's p does not hold for them as written.
3. Deflatability is a strong extra condition, and nothing yet says
thought transitions satisfy it.
4. SVD / PCCA+ is a floating-point clustering step. It belongs only on the
cold, out-of-fold path (CLAUDE.md, the no-ms scoped exception), never
per query.
- **Ties.** This is the stochastic form of EXO-132's lumpability, the
distinction GPT /08 asked for: almost-lumpability in L1, not deterministic
congruence.
- ⊘ **Superseded 2026-10-23 (same day).** DECISION (operator): the substrate
is not probabilistic; a probabilistic model is a counterfactual that the
egglog/eggcc compiler must certify. The certificate is a checked proof, not
P̂, and the 64 × 10^6 folds are compilation material, not samples. The block
above is kept as the record of a withdrawn framing; see
`entries/2026-10-23-coresearch-thought-git-markov.md` § Correction.

**Next probe:** rung 2, which is cheapest and guards the whole storage
argument. **Kill:** a lane that is deterministic given its seeds cannot be
defined for the planned writers (outside reads are essential). In that case
receipts cannot replace deltas, and the arithmetic above stands against the
design.

**Fruit:** High-signal
**Status:** Open.
## 2026-10-23 — Lumpability as the closure check: a palette composes under an operation only if its cells are lumpable for it, and purity measures that

`[P2 @palette-engineer @truth-architect domain:codec/palette]`
Expand Down
2 changes: 1 addition & 1 deletion .claude/board/STATUS_BOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -385,7 +385,7 @@ Plan: `.claude/plans/ce64-hypothesis-layers-v1.md`. Plateau: `.claude/knowledge/
| **D-TGA-2** | Read plane R and status | Queued | a read outside R is never reported stale |
| **D-TGA-3** | rebase (four masks, no data moved) | Queued | rebase-then-squash == from-scratch re-run on v+1, including a deletion in main (retract then re-derive); disables: no stale re-run, no conflict check, no retraction |
| **D-TGA-4** | land through the revision gate into the next Lance version (D-HL-8) | Queued | landing a thought that is behind is refused |
| **D-TGA-5** | squash_into (partial squash) and blame | Queued | blame returns original sources after slot reuse |
| **D-TGA-5** | squash_into (partial squash) and blame | In PR (lab precursor: `crates/thought-git-probe/tests/squash_gate.rs`, `entries/2026-10-23-d-tga-5-squash-gate-preregistration.md`; most-updated target, contradiction/divergence gate, blame over lanes, 5 disables red) | blame returns original sources after slot reuse (slot reuse not yet covered) |
| **D-TGA-6** | MergeQueue | Queued | independent thoughts == landed by hand in FIFO order; a thought reading another negated lands after it; a cycle through negation refused at enqueue |
| **D-TGB-0** | thought git basics: fixtures per write-intention outcome, squash-stack and anomaly fixtures, absent-not-zero fixture, from-scratch oracle (`thought-git-basics-v1.md`) | Shipped (lab, #1630; probe `crates/thought-git-probe`, 19 tests; `PREREGISTRATION.md`) | 5 oracle disables verified red-then-green; every signal has a can-fire and a can-stay-silent case on real contract code; the pearl half on edges moves to D-TGB-3 |
| **D-TGB-1** | slot pool, autonomous: taken by `branch`, freed by `squash`/`discard`/`land` (the write is the Rubicon decision, 2026-10-20) | Shipped (lab, #1630/#1631; `crates/thought-git-probe/src/slots.rs`). Slot pool only: no production land, rebase or persistence (D-TGB-2..5 Queued) | at 63 taken `branch` picks a FIFO or least-used candidate, squashes it along its dependency chain if it has one, else evicts it (DECISIONS 2026-10-20; was: squash unremarkable first, else refuse); stale handles refused; 6 tests |
Expand Down
114 changes: 114 additions & 0 deletions .claude/board/entries/2026-10-23-coresearch-thought-git-markov.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
# Coresearch 2026-10-23: thought-git accumulation as a large Markov chain

**Status:** exploration map. Nothing is adopted; the operator chooses.
**Question (operator):** "Consider that giant thought-git accumulation as a large Markov chain."
Sources: Bittracher & Schütte, *Physica D* 416 (2021) 132799, read in full;
Askarian, Xu & Faragó, *Algorithms* 9(3):50 (2016), arXiv 1809.06564, read in full.

## Frame (what the scouts established)

- **F1.** A version is a deterministic function of the givens, the prior
version and the landing thought set (`oracle.rs:46-108`, verified in code).
Thought-git is therefore a transducer, or a Markov decision process; it is a
Markov chain only once the policy that chooses thoughts is fixed.
- **F2.** No ruling grants the Markov property to lane accumulation.
E-SUBSTRATE-1 covers bundles and E-MARKOV-TEMPORAL-STREAM-1 covers stream
order. Last-writer replace is neither, and the seal is an order-dependent
fold (`seal-vs-temporal-ordering-information.md`).
- **F3.** Lanes are measured not independent: they share the manuscript, the
tokenizer and the model (`ISSUES.md:3852`).
- **F4.** No code in `crates/` estimates transition probabilities, tests
lumpability or bounds an aggregation error. The search space was a closed
`git grep`; ndarray/hpc was not searched.
- **F5.** Two earlier rejections apply here.
- #1703: a replay capsule is not proof authority, and population evidence
only nominates.
- TORQUE T-2026-10-23-1 (killed): a learned PaletteLut aggregation did not
compose.

## Where the papers stop applying

- **B&S Algorithm 3.1.** It needs a restart from any state to draw a column.
A trajectory log cannot do that. Trajectory-only estimation needs about
p·r samples (Zhu et al., arXiv 1907.00113, Thm 3, section-read), so "J
independent of N" does not transfer [S]. It also requires homogeneity
(fails per F1) and independent samples (fail per F3).
- **AXF Theorem 2.** It needs an irreducible, aperiodic chain. Accumulation
over versions is a transient DAG with no recurrence, so the theorem does
not apply.
- **What survives** without the Markov property and without independence:
- min/max L/U bounds over counts (AXF Def. 3);
- deterministic congruence found by partition refinement;
- the type split: certificate (replay + seal) versus model.

## Exploration map

| idea | verdict | why / probe |
|---|---|---|
| Squash folds only committed writes | **ADOPT-NOW, done** | Found by the falsifier: `Lane::final_state` folded Prune/Veto/Plan writes. Fixed in #1736 with P8 (red before the fix, green after; disable D6 turns only P8 red). |
| D: squash equals chain for every later read | PROBE (S) | Fixtures: (b) contradiction then rewrite; (c) intermediate divergence then agreement; (d) the same value 3× vs 1×. Predicted: the intermediate contradiction and divergence records, and the multiplicity, are lost. Decide whether that loss is acceptable before building a certificate on the squash. Silent twin: `clean_lanes`. |
| D-59..63: the squash preserves the epistemic promotion | PROBE (S), blocked | The promotion reader must be pinned first. Blame is a lane bitmask, not a count, so a multiplicity-reading promotion breaks. |
| I3: counted ε-spread L/U gate | PROBE (M) | Bridge: R2 is ε = 0 with a point mass per lane. Count lanes per (cell, value class) from the blame bitmask; refuse when U−L > ε by integer cross-multiplication. With ε = 0 it must reproduce R2 (the p2/p4 tests). It is a BOUND only when the counts are exhaustive; otherwise it is an empirical interval, a gate annotation and never proof authority. |
| I7: coarsest congruence by partition refinement | PROBE (S–M) | Integer, cold. It supplies I3's blocks. Kill if the reduction is below 2× on real traces. |
| F1: is the process Markov at all, over a chosen state space | PROBE (M), needs real traces | First the revisit rate: below 1% means untestable on that state space, so coarsen. Then an order-1 vs order-2 contingency test against a Jirak band; kill if more than 10% of states differ. A synthetic policy decides the answer in advance, so it does not count. |
| F3: lane independence after conditioning on shared inputs | PROBE (M), needs real traces | Conditional mutual information against a time-shuffled null. Precondition for any cross-lane convergence gate. |
| I12/I13: lossless op-log encoding, tags plus cleanup | PROBE (S) | Use Lance's own column encodings and version tags; no parallel store. This answers the terabyte concern without any Markov assumption. |
| I2 P = Π P̂ Λ; I5 defect; I6 interval propagation; I10 Chapman-Kolmogorov; I11 VAMP; I16 MarkovTripletState | PARK | Each needs F1 to pass (homogeneity) or the dormant VAMPE+Jirak track. All are cold-path and report-only. |
| I1 J-column SVD; I4 AXF convergence; I14 Nova IVC | SKIP | I1: no restart oracle and dependent lanes. I4: transient DAG. I14: licence unknown, far outside the ns law. |
| I8 causal state; I9 certificate vs model | ALREADY-HAVE | `final_state()` is the causal state of a deterministic lane; replay plus seal is the certificate. |

## Head selection (operator follow-up)

Recency is not the criterion. The gate-grade parts are subsumption (R1/R2)
and the exact L/U spread. The others are not:

- cross-lane convergence (R̂) presumes independent lanes, which F3 refutes;
- stability (ρ) and predictive sufficiency are cold-path and report-only.

## Not searched

- ndarray/hpc (clam_compress, cam_pq);
- GOLDSTAUB, identity-temporal-evidence-primer, holographic-temporal-access-projection;
- the IDEAS joint-replay row;
- Buchholz and Franceschinis-Muntz bounding-aggregation papers (not located);
- bisimulation metrics (abstract only).

## ⊘ Correction 2026-10-23: the probabilistic premise was a category error

DECISION (operator, 2026-10-23): the substrate is not probabilistic. SPO-FC
being probabilistic today is legacy. Every probabilistic model counts as a
**counterfactual** and must be certified by the egglog / eggcc reasoning
compiler before anything rests on it. The 64 lanes × ~10^6 folds run per
second are the **material** a thought is compiled from, until it is a proof
that later operations can call by reference. They are not a population to be
aggregated statistically.
SCOPE: this map and the IDEAS row "A certificate replaces the deltas" (its
Markov block). BASIS: the operator's statement; consistent with F1 (versions
are deterministic), #1703 (population evidence only nominates) and
`dr-math-binding-probe` (`admit_recipe` ends in `NoCertifiedCompiler`).
REVISIT WHEN: never on statistical grounds.

What this changes in the map:
- The Bittracher/Schütte and Askarian papers describe at most the **search
over counterfactuals**, never the admission and never the certificate.
All their rows (I1–I6, I10, I11) drop to PARK as counterfactual-layer
descriptions. The certificate/model split (I9) holds and is sharpened: the
certificate is a checked egglog proof, not a replay witness and not P̂.
- The premise gate failed. Phase 0 split certificate from aggregation but
never asked whether the substrate is stochastic at all. F1 (versions are
deterministic) already said it is not; the brief should have stopped there.
- Head selection: the head is the thought whose compiled proof is admitted.
The children's folds are the search trace. After admission they are
discarded, except the seal and the refuted branches the proof cites.
- What actually blocks this: no certificate exists for DR64 (`dr-math-binding-probe`,
`NoCertifiedCompiler`). The open items there are a typed relational statement
language, egglog laws with checked proofs, and a lowering check. The candidate
first law is associativity of Boolean relation composition. `crates/egglog-r2il-probe`
holds the cold compile path; `.arxiv/2026-10_eggcc_effect_safe_extraction_statewalk_r2il_map.md`
holds the effect-safe extraction lesson.
- Still valid: the P8 fix (only committed writes squash) and probe D
(squash = chain for every later read). Both are deterministic.

## Closeout

`coresearch | STATUS: open | OUTCOME: one defect fixed in #1736 (P8); probabilistic premise withdrawn (category error) | OPEN: DR64 certified compiler (typed statement language + checked egglog laws + lowering check); probe D; promotion reader unpinned`
Loading
Loading