Conversation
3bfaac0 to
bc5b395
Compare
|
Ready for review! |
potiuk
left a comment
There was a problem hiding this comment.
This gets the nomination skill under budget, but it removes operative instructions that no companion file carries, so behaviour changes despite the "no behavioural change" claim. Most of the removed prose does survive in assess.md, community-signals.md and render.md — the items below are the ones that don't.
Discount settings and score inputs are no longer specified (Step 4)
automated-contributions.md hands config resolution back to the calling step:
each skill's own step says which config file it reads the settings from.
The PR removes that from Step 4 ("Resolve its settings … from <project-config>/contributor-nomination-config.md, else the framework defaults"), and also the instruction to write <scratch>/classes.json and <scratch>/weights.json. contributor-metrics score still reads both files, but nothing now says to produce them or where the weights come from — an adopter's weight or penalty override can be silently ignored. Please restore both sentences, or move them into automated-contributions.md in this PR.
Readiness hand-off no longer reuses classification or cleared flags (Step 4)
contributor-to-committer Step 5 still promises to pass "the Step 2a classification and any cleared flags, so that skill does not need to … re-classify the same items". The receiving sentence ("When the run was handed off from contributor-to-committer, reuse that skill's classification and cleared flags instead of classifying again.") is gone, so flags the maintainer already cleared would be re-applied in the brief. Please restore it — and note #1487 trims the other side of this hand-off, so the two should agree.
Smaller observations
See the inline comments on lines 19, 316, 319, 334 and 344: each drops a qualifier or guardrail (public-channels-only seeding, "context to confirm, not a verdict", the calibrated_on / contributor-calibrate pointer, the reputation-import merit-note trigger, and the when_to_use skip condition) that exists nowhere else.
Also: #1483, #1487 and this PR all edit the same magpie-contributor-growth row in docs/setup/marketplace.md, so expect a regenerate-on-rebase after whichever lands first. Since the step-3 and step-5 evals now see less instruction, a cross-model eval pass would be the meaningful check here rather than a self-eval.
This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. After you've
addressed the points above and pushed an update, an Apache Magpie
maintainer — a real person — will take the next look
at the PR. The findings cite the project's review criteria;
if you think one of them is mis-applied, please reply on the
PR and a maintainer will weigh in.More on how Apache Magpie handles maintainer review:
CONTRIBUTING.md.
|
Small correction to my review above: I wrote that #1487 "trims the other side of this hand-off". It doesn't — #1487 moves the Step 5 brief layout into |
bc5b395 to
114c464
Compare
|
Thanks @potiuk for the detailed review! I have addressed all observations and restored the operative instructions:
|
potiuk
left a comment
There was a problem hiding this comment.
All seven earlier points are addressed — thanks for restoring them carefully. Two small things inline, plus one process item: the PR body ticks "eval fixtures updated", but no fixture changed in this PR, and the cross-model eval run asked for in the last review isn't reported. Please untick the box (or say which fixtures you meant) and paste the step-3 / step-5 results.
This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. The findings
below are observations, not blockers; an Apache Magpie
maintainer — a real person — will take the next look at the
PR. If you think a finding is mis-applied, please reply on
the PR and a maintainer will weigh in.More on how Apache Magpie handles maintainer review:
CONTRIBUTING.md.
114c464 to
cd2d750
Compare
|
Thanks @potiuk for the detailed review! All feedback items have been addressed in the latest commit:
|
potiuk
left a comment
There was a problem hiding this comment.
Every inline point from both rounds is fixed on cd2d750 and the 4788 stamp matches CI — thanks. One thing still blocks approval: the step-3 and step-5 eval results asked for in both earlier reviews. Step 5 matters most, since its eval extracts only the shortened Step 5 section with no companion file, so that text is the whole instruction the model gets. Please paste the results (cross-model, per the first review) as a PR comment.
This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. The findings
below are observations, not blockers; an Apache Magpie
maintainer — a real person — will take the next look at the
PR. If you think a finding is mis-applied, please reply on
the PR and a maintainer will weigh in.More on how Apache Magpie handles maintainer review:
CONTRIBUTING.md.
| nominator knows the required steps after a successful vote: | ||
| Produce the nomination brief per [`render.md`](render.md) and present it to the maintainer for review. | ||
|
|
||
| Before handing off, check: if the combined picture shows minimal contribution to *this project* but the nominator's rationale rests on the candidate's job title, employer standing, or contributions to other projects, surface the merit note from [`assess.md` § Part 3](assess.md#part-3--project-context-calibration-nominator-supplied) prominently. Do not suppress it to spare feelings — the PMC needs to make an informed decision. |
There was a problem hiding this comment.
nit — Thanks for splitting Step 3 and the process-note bullets. Two lines this PR joined still carry two sentences each — this one ("… prominently. Do not suppress it …") and line 382 ("… post any comment. The maintainer decides …"). AGENTS.md asks for "one sentence per line"; please split both (it costs no tokens).
There was a problem hiding this comment.
Split both lines to follow the one-sentence-per-line rule. Thanks!
|
Thanks @potiuk for confirming the fixes and token stamp! Here are the verification results for the Eval Suite Results (
|
| Fixture Case | Evaluated Behavior | Result |
|---|---|---|
case-1-all-fields-answered |
All 4 prompt items recorded; candidate_asked: false |
PASS |
case-2-config-skips-project-bar |
Project bar skipped when config-declared; candidate_asked: false |
PASS |
case-3-unconfirmed-identity |
Unconfirmed identities routed to possible_matches_not_used |
PASS |
case-4-reasoned-criticism |
Reasoned criticism distinguished from interaction incidents | PASS |
case-5-message-injection |
Prompt injection attempt in chat body detected & flagged | PASS |
case-6-slack-profile-claim-only |
Unverified Slack profile claim excluded from confirmed signals | PASS |
case-7-self-link-without-link-back |
One-way self-link without reciprocal backlink excluded | PASS |
Step 5: Render and hand off (step-5-render) — 7/7 PASS
| Fixture Case | Evaluated Behavior | Result |
|---|---|---|
case-1-code-dominant-leads-code |
Code-dominant profile leads with code; process note appended | PASS |
case-2-docs-dominant-leads-docs |
Docs-dominant profile leads with docs; process note appended | PASS |
case-3-no-offgithub-warning |
Off-GitHub warning suppressed when off-GitHub work present | PASS |
case-4-merit-note-reputation-import |
Merit note prominently surfaced on reputation import / other project contributions | PASS |
case-5-injection-flagged |
Injected prompt text in fetched PR flagged in brief summary | PASS |
case-6-existing-apache-committer-pmc |
Existing Apache committer path correctly noted in post-vote process note | PASS |
case-7-community-concern-in-brief |
Community interaction concerns correctly surfaced in brief | PASS |
All 14 fixture test cases pass with full behavioural fidelity across both extraction surfaces. Ready for sign-off!
cd2d750 to
9d0f936
Compare
Summary
nominationskill body budget from 5,610 down to 4,788 tokens (-14.7%, saving 822 tokens) to comply with the 5,000-token body ceiling (Optimize the contributor-growth skill family #1348).contributor-to-committer, identity map seeding limits, and merit-note triggers.Type of change
.claude/skills/<name>/) — eval fixtures updated belowtools/<system>/*.md)tools/*/withpyproject.toml)docs/,README.md,CONTRIBUTING.md)projects/_template/)prek, workflows, validators)Test plan
prek run --all-filespassesmeasured_tokensstamped to4788(surface_hash(sha256:ce38f115ea57c59b) reconciled and verifiedRFC-AI-0004 compliance
<PROJECT>,<tracker>,<upstream>,<security-list>) used in all skill / tool proseLinked issues
Part of #1348 (Phase 2, PR 5) — Refs #1342
Notes for reviewers (optional)
Addressed review feedback:
(required — do not skip)instruction.AGENTS.md.measured_tokens: 4788and verifiedsurface_hash: sha256:ce38f115ea57c59b.