Improve agent setup and evaluation surfaces

[next-devin]

Status: ready for review — campaign-specific framing retained

The hosted prompt is explicitly a Campaign Page Kit quickstart rather than a general NEXT development setup or a new workflow authority. It is ready for review with that narrower framing.

Summary

• publish a versioned /agent-setup/prompt.md and human guide as an optional wrapper around the current Campaigns docs and campaign-init
• preserve developer-supplied template, route, project, package-manager, and agent-context choices; ask for missing project choices instead of silently selecting Apollo/agent-starter
• remove the redundant default next-campaigns-setup skill installation and keep the CLI/docs authoritative for flags, scaffold behavior, conflicts, and exit codes
• retain non-overwrite, no-credential, no-live-store, evidence, and scoped-verification guardrails
• publish /evaluate/prompt.md, a source-grounded prospect evaluation contract that starts from the capability map and focused domain bundles
• advertise both prompts through llms.txt and the sitemap
• correct the capability map so payment-gateway operations and dispute operations are classified separately
• make live-surface checks bounded, canonical-preview-aware, and resilient; verify Brotli, gzip, or Zstandard merchant-search delivery

Boundaries

The campaign quickstart does not decide production architecture, install Campaigns OS, or replace the public setup skill. If an agent's task is platform research, an existing project, storefront themes, apps, or Admin API work, it is routed back to the developer index and capability map without scaffolding a campaign.

/evaluate/prompt.md is a separate docs-grounding primitive: it tells an agent how to research the platform from authoritative sources. It does not decide the marketing-site presentation or implement the hosted consultation/intake flow proposed in NextCommerceCo/nextcommerce-site#154.

Verification

• npm run build — 762 static routes
• npm run check — agent surfaces, 67 authored-page frontmatter records, and links pass
• preview-to-preview npm run check-live-surfaces — 142 passed, 0 failed

The previous fixed Apollo/agent-starter acceptance path was intentionally removed because it made a sample workflow look mandatory. The remaining fresh-agent acceptance run should use developer-confirmed template, slug, name, and agent-context inputs.

Closes #45
Part of NextCommerceCo/nextcommerce-site#92

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	developer-docs	524fdaf	Commit Preview URL Branch Preview URL	Sep 04 2026, 08:23 AM

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

The incremental changes from 29feefc to 524fdaf cleanly retire the human-readable /docs/capabilities projection. Across 81 files (31 insertions, 315 deletions) every reference is removed or updated: the page generator, the components/capability-links.tsx consumer, the sidebar tab and meta.json entries, the llms.txt and 404 navigation links, the source.config.ts schema field, and the capability_ids lines from every authored page frontmatter. The reciprocal scripts/check-frontmatter.mjs capability-id validation and its --write rewriter are removed in the same increment, and scripts/check-agent-surfaces.mjs plus scripts/check-live-surfaces.mjs now assert the projection is absent (build: null file; live: HTTP 404). scripts/generate-capability-map.mjs removes any stale content/docs/capabilities/index.mdx left over from older versions, and AGENTS.md plus the content/capabilities.yaml header accurately describe the new world (relationships derived from URLs in the map; no frontmatter declaration).

The three findings the previous review carried from earlier rounds — content/agent-setup.md:28 (input list), content/agent-setup.md:13 (Node.js requirement), and scripts/generate-agent-setup.mjs:36-41 (H1-to-H2 rewrite) — were fixed in 29feefc and remain resolved; those files are unchanged in this increment.

No new issues were found in this increment.

Files Reviewed (12 changed files, code only)

• .gitignore - 0 issues
• AGENTS.md - 0 issues
• app/docs/[[...slug]]/page.tsx - 0 issues
• app/docs/layout.tsx - 0 issues
• app/llms.txt/route.ts - 0 issues
• app/not-found.tsx - 0 issues
• content/capabilities.yaml - 0 issues
• content/docs/meta.json - 0 issues
• scripts/check-agent-surfaces.mjs - 0 issues
• scripts/check-frontmatter.mjs - 0 issues
• scripts/check-live-surfaces.mjs - 0 issues
• scripts/generate-capability-map.mjs - 0 issues
• source.config.ts - 0 issues

Plus the deletion of components/capability-links.tsx and removal of capability_ids: lines from all authored docs pages (no remaining references in lib/, app/, or scripts).

Previous Review Summaries (5 snapshots, latest commit 29feefc)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 29feefc)

Status: No Issues Found | Recommendation: Merge

The incremental changes from 7ab01ea to 29feefc (1 commit, 2 files) cleanly resolve the three findings raised in the previous review against this PR.

• content/agent-setup.md:28 — the request-for-input list now mentions agent-context preference alongside template, route slug, and campaign name, matching the four-value --ai-context placeholder in the command block. The previous WARNING is resolved.
• content/agent-setup.md:13 — the Node.js prerequisite is now explicit on both directions ("Do not impose a stricter version or relax the documented requirement"), replacing the ambiguous "Do not raise the documented Node.js requirement". The previous SUGGESTION is resolved.
• scripts/generate-agent-setup.mjs:36-41 — the H1-to-H2 rewrite now runs the regex against humanSource first and throws if the expected heading is missing, so a future source drift fails at build time instead of silently emitting the original H1 alongside frontmatter. The previous SUGGESTION is resolved.

No new issues were found in the 9 insertions / 6 deletions of this increment. The other 14 active comments from earlier review rounds are unchanged in scope and remain accurate against the current code.

Files Reviewed (2 changed files)

• content/agent-setup.md - 0 issues
• scripts/generate-agent-setup.mjs - 0 issues

Previous review (commit 7ab01ea)

Status: 3 Issues Found | Recommendation: Address before merge

Overview

Severity	Count
CRITICAL	0
WARNING	1
SUGGESTION	2

The incremental changes from ffe30b9 to 7ab01ea (1 commit, 6 files) narrow the agent-setup prompt from a generic "set up a Next Commerce campaign workspace" guide to a Campaign Page Kit quickstart that preserves developer-supplied choices (template, slug, name, agent context) instead of silently choosing Apollo/agent-starter. The app/llms.txt/route.ts, content/docs/index.mdx, content/docs/skills/index.mdx, and scripts/generate-agent-setup.mjs updates are tightly consistent with this narrower framing.

The new surface checks in scripts/check-agent-surfaces.mjs (lines 140-156) correctly require the placeholders, the table row, and the _site/ build output to appear in the rendered prompt, while explicitly rejecting --template apollo, --api-key, agent-starter, and the duplicate npx skills add workflow. These negative needles are safe against the current source: a substring search for --template apollo does not match --template <selected-template>, and agent-starter no longer appears anywhere in content/.

Three new findings on the incremental diff:

1. content/agent-setup.md:28 (WARNING) — The list of inputs the agent is told to request from the user is missing --ai-context. The non-interactive command on line 37 requires four values (template, slug, name, agent-context), but line 28 only mentions three. A reading agent will either pass the literal <claude|codex|cursor|copilot|none> placeholder (which campaign-init will reject) or silently pick one and generate a context file the user did not ask for.
2. content/agent-setup.md:13 (SUGGESTION) — "Do not raise the documented Node.js requirement" is ambiguous: it could mean "don't exceed what the docs require" or "don't raise the version at all". The previous prompt was explicit (Use Node.js 22 or newer); the replacement should be explicit about not relaxing or hardening the documented minimum.
3. scripts/generate-agent-setup.mjs:38 (SUGGESTION) — The H1 → H2 rewrite on the source heading is unchecked. If the source H1 ever drifts away from ^# Start a Campaign Page Kit project with an AI agent$, String.replace returns the input unchanged and the generated MDX retains the original H1 next to the ---\ntitle: frontmatter. There is no assertion (neither in this generator nor in check-agent-surfaces.mjs) that the rewrite actually happened.

The 5 outstanding issues from the previous review remain resolved (no changes to those files in this increment). The 11 active existing inline comments are unchanged in spirit — the previously raised Retry that exact command once concern is moot because the retry bullet was removed in this rewrite.

Issue Details (click to expand)

WARNING

File	Line	Issue
content/agent-setup.md	28	List of user inputs to request omits --ai-context, but the command block on line 37 requires it.

SUGGESTION

File	Line	Issue
content/agent-setup.md	13	"Do not raise the documented Node.js requirement" is ambiguous in context.
scripts/generate-agent-setup.mjs	38	H1→H2 rewrite is silent; no assertion that the replacement actually fired.

Files Reviewed (6 changed files)

• app/llms.txt/route.ts - 0 issues
• content/agent-setup.md - 2 issues
• content/docs/index.mdx - 0 issues
• content/docs/skills/index.mdx - 0 issues
• scripts/check-agent-surfaces.mjs - 0 issues
• scripts/generate-agent-setup.mjs - 1 issue

Fix these issues in Kilo Cloud

Previous review (commit ffe30b9)

Status: No Issues Found | Recommendation: Merge

The incremental changes from cac4a60 to ffe30b9 (1 commit, 1 file) introduce canonical/preview URL separation for live checks and broaden the in-transit compression check to accept zstd and multi-encoding headers. The new previewUrl() helper at scripts/check-live-surfaces.mjs:99-104 is straightforward and correctly rewrites canonical origins to their preview counterparts while leaving non-canonical URLs untouched. The added compressedInTransit check at line 195 correctly relaxes the previous strict regex so that comma-separated multi-encoding chains like gzip, br are now accepted, while still gating on the 750 KB transfer budget for identity delivery. Sitemap and llms.txt content assertions now reference canonical origins (e.g., DEV_CANONICAL, MERCHANT_CANONICAL), which is the right invariant to test for cross-site linking.

All 5 previously outstanding issues remain resolved in ffe30b9.

Files Reviewed (1 changed file)

• scripts/check-live-surfaces.mjs - 0 issues

Fix these issues in Kilo Cloud

Previous review (commit cac4a60)

Status: No Issues Found | Recommendation: Merge

All 5 issues from the previous review at 1115353f have been resolved in cac4a60:

File	Line	Resolution
scripts/check-live-surfaces.mjs	177	Compression check now allows identity delivery when the decoded index is ≤ 750 KB.
scripts/check-agent-surfaces.mjs	98	Hardcoded === 9 replaced with set comparison against content/capabilities.yaml.
scripts/check-agent-surfaces.mjs	99	Same set-comparison fix as above.
content/agent-setup.md	88	Retry guidance now requires inspecting the working directory first and forbids retry on partial installs.
app/sitemap.ts	11	The two prompt routes no longer emit a misleading build-time lastModified.

No new issues were found on the incremental changes.

Files Reviewed (4 changed files)

• app/sitemap.ts - 0 issues
• content/agent-setup.md - 0 issues
• scripts/check-agent-surfaces.mjs - 0 issues
• scripts/check-live-surfaces.mjs - 0 issues

Previous review (commit 1115353)

Status: 5 Issues Found | Recommendation: Address before merge

Overview

Severity	Count
CRITICAL	0
WARNING	1
SUGGESTION	4

Issue Details (click to expand)

WARNING

File	Line	Issue
scripts/check-live-surfaces.mjs	177	New hard assertion that the merchant search index is compressed in transit will produce false-positive CI failures when Cloudflare serves the response uncompressed (incident, config drift, body-size bypass).

SUGGESTION

File	Line	Issue
scripts/check-agent-surfaces.mjs	98	Hardcoded === 9 operation count for payments-gateways will break CI on any new op added to the tag, even when the YAML mapping is correct.
scripts/check-agent-surfaces.mjs	99	Same hardcoded === 9 brittleness for disputes.
content/agent-setup.md	89	"Retry that exact command once" with no idempotency guard is risky for npm install / npx commands; partial installs may not repair on retry.
app/sitemap.ts	11	lastModified: new Date() reflects build time, not source mtime, so the sitemap advertises a fresh timestamp on every deploy regardless of source changes.

Files Reviewed (17 files)

• .gitignore - 0 issues
• AGENTS.md - 0 issues
• app/agent-setup/prompt.md/route.ts - 0 issues
• app/evaluate/prompt.md/route.ts - 0 issues
• app/llms.txt/route.ts - 0 issues
• app/sitemap.ts - 1 issue
• content/agent-setup.md - 1 issue
• content/capabilities.yaml - 0 issues (9 dispute and 9 payment operations verified against 2024-04-01 spec)
• content/docs/agent-setup/meta.json - 0 issues
• content/docs/index.mdx - 0 issues
• content/docs/meta.json - 0 issues
• content/docs/skills/index.mdx - 0 issues
• content/evaluate-next.md - 0 issues
• package.json - 0 issues
• scripts/check-agent-surfaces.mjs - 2 issues
• scripts/check-live-surfaces.mjs - 1 issue
• scripts/generate-agent-setup.mjs - 0 issues

Fix these issues in Kilo Cloud

---

_{Reviewed by minimax-m3 · Input: 56.1K · Output: 8.4K · Cached: 1.5M}

[alexphelps]

@next-devin I'm not sure I understand the point of this page - https://4103f3ff-developer-docs.next-commerce.workers.dev/docs/capabilities

It's:

1. Not all inclusive and doesnt cover all topics and not human reader friendly
2. It's not a source of truth, content is redundant mostly just links to other pages
3. How is it different than sitmap.xml?
4. If it's for agents, shouldn't it be the llms.txt?

Same goes for https://4103f3ff-developer-docs.next-commerce.workers.dev/docs/agent-setup

If it's a stray page not meant for humans, why not just improve the actual content so we dont need wrappers?

[next-devin]

@alexphelps yeah it's for agents only, I hadn't identified that it wanted to add actual viewable menu items - that's a no-go for sure.

It's basically the result of a lot of probe testing and figuring out what paths agents actually follow or what they believe so to speak. For example a lot of confusion about what/why '29next' everywhere such as APIs so needed to be spelled out more clearly.

Anyway I already shipped significant improvements/updates to llms.txt and AGENTS.md (which was missing actually) - but this is why I flagged this one. I think it's somewhat of a step beyond what is needed. Of course should not appear in nav. So I'll push this one back. Thanks for flagging

[next-devin]

Addressed the capability-page concern in 524fdaf:

• removed /docs/capabilities from the public navigation, generated routes, llms.txt, 404 recovery, and live checks
• kept the actual agent surfaces: /capabilities.json and the six focused /llms/<bundle>.txt bundles
• removed capability_ids from both sites; page relationships now derive from URLs in the map
• removed the developer-side reciprocal capability panels
• removed the legacy-identifiers relationship that caused the irrelevant Get Started footer

The Campaign Page Kit quickstart remains intentionally narrow and optional. It wraps the existing public Campaigns docs and campaign-init flow, preserves developer choices, and routes non-campaign tasks back to the normal docs instead of prescribing a general NEXT workflow.