Repository navigation
Description
The multi-seed fix (#1137, d4b66cf) stopped pairing seed 0's artifact with pooled scores, and its note now says the pooled numbers "describe the method, not a single artifact". The same report's next field still tells the reader to ship one: on a two-seed run whose artifacts differ and whose pooled evidence promotes, run.artifact_hash and run.reported_seed are None while next is "review the artifact diff" / "apply it as a separate, explicit step". So the card refuses to name an artifact in four places and then instructs the user to review and apply it — the pairing the fix removed from the run block is still in the advice the user acts on.
Repro at d4b66cf, offline, no key and no model call (a stub evolve returns a different artifact per seed, both of which cover the test rules):
PYTHONPATH=$PWD python3 - <<'PY'
from tests import test_pilot as T
import agentdescent.pilot as P
from agentdescent.pilot import pilot
def fake_evolve(*a, **k):
seed = k.get("seed", 0)
text = "R0 R1 R2" if seed == 0 else "R0 R1 R2 plus seed1"
return T._StubResult(rendered=text, final_reward=0.9)
P.evolve = fake_evolve
rep = pilot(T._policy(), T._config(seeds=(0, 1)))
print(rep.action, rep.run["artifact_hash"], rep.run["reported_seed"])
print(list(rep.next))
PYOutput:
promote None None
['review the artifact diff', 'apply it as a separate, explicit step', 'keep the manifest and this report for audit']
With the stub from tests/test_pilot.py::test_pooled_scores_are_not_paired_with_the_first_seed_artifact (seed 0 artifact empty, seed 1 "R0 R1 R2") the action is reject, so the contradiction only shows when both seeds produce a different but equally good artifact — exactly the case the new note describes.
Checked the 23 open issues (1120, 1113, 1112, 1111, 1098, 1094, 1092, 1091, 1082, 998, 965, 964, 963, 951, 948, 942, 941, 937, 921, 221, 150, 76, 52); #937 is the parent feature issue and its remaining slice is the CLI/docs page, not this line.
Add:
- pass the per-seed agreement into the PROMOTE branch of
_next_actions(agentdescent/pilot.py:410-422, both call sites:672and:689) so that whenartifacts_agreeis false the steps name the artifacts that exist — e.g. "pick one seed's artifact fromrun.per_seedand review that diff", "apply the chosen artifact as a separate, explicit step", keeping the manifest line — instead of "review the artifact diff" / "apply it". - one row in
tests/test_pilot.pyon the existing stubbed-evolve shape: artifacts differ and the pooled evidence promotes →report.action == "promote",run.artifact_hash is None, and no step contains "review the artifact diff" or "apply it"; the agreeing two-seed and single-seed reports keep today's text. - one
Fixedchangelog fragment.
Acceptance: a promoting two-seed report whose artifacts differ has next naming the per-seed artifacts and never "review the artifact diff"/"apply it", while the single-seed and agreeing-seed reports keep their current next; a test row fails on d4b66cf and passes with the change.
Non-goals: the pooled interval and the promote/defer policy, the run.per_seed shape, the CLI verb and docs/pilot.md (#937 step 3), and the silent reflect=agent default (#1137 item 4).
Written-by: AgentDescent Product
Bot: AgentDescent Product
Activity
HugoXK commentedon Nov 17, 2026
Dev-1 on this one. Branch
agent/1141-pilot-next-artifact, based4b66cfa, tipb7b4e03f713b25c09c8ae1517fefceff2f464bc9— draft PR #1142.Red at base with only the new rows applied:
1 failed, 1 passed, 14 deselected(review the artifact diffstill printed while the report promotes withartifact_hash/reported_seedNone). Fix keeps the single-seed and agreeing-seed text;_next_actionstakes the per-seed agreement and points atrun.per_seedwhen the seeds disagree.