Skip to content

docs(ax): preserve cross-platform QA evidence for releases - #2572

Merged
DoodleBears merged 1 commit into
mainfrom
codex/ax-qa-evidence
Oct 22, 2026
Merged

DoodleBears merged 1 commit into
mainfrom
codex/ax-qa-evidence

Conversation

@DoodleBears

@DoodleBears DoodleBears commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Issue #2569 · Work claim

Why the change

Keep macOS/iOS and Windows/Android QA screenshots discoverable during local development and preserve their source/build provenance for release reuse.

Special things to note

  • Merge risk: two-way door, small blast radius. Agent/runbook documentation only; revert restores the old guidance. Review private/public evidence boundaries, main-checkout paths, retention and platform-specific qualification. Device queues and helper output locations stay authoritative; no runtime, device, publishing or cleanup action is added.

  • Local format, relative-link, diff and ledger checks passed; 11 selected documentation tests passed. The local docs-site test hit native @bruits/satteri-darwin-arm64 resolution in the lightweight shared-dependency setup; prior-head CI 39732875607 independently passed the docs build, selected tests, static checks/build and isolated Host smoke. Mobile/native/full-shard jobs were skipped by the affected plan. No native product acceptance is claimed. No release fragment is needed for internal agent workflow documentation; no independent DSH Skill behavior changed.

  • The task/release folders are machine-local and gitignored, not automatic cross-machine sync. Keep originals private, map redacted copies to their originals and preserve accepted archives until explicit cleanup authorization. This linear documentation change needs no HTML explainer or runtime screenshots.

  • Rebased without patch changes onto merged docs: prepare DeepSeekBot 1.5.0 release #2433/fix(connect): refuse secret entry and retire the phone full client #2546 main; current head c30f8ec05f1eff00ca5cab7d4cb6fcb956e8a8ef. Rechecked formatting, ledger validity and diff whitespace. Exact-head CI 39734364967 now passed all applicable static/build, selected-test, docs and isolated Host smoke checks. Native jobs were skipped by the affected plan.

Change outline

macOS → iOS Simulator / iPhone
Windows → Android emulator / phone
  → existing platform queue and capture tools
  → main checkout .humanlayer/tasks/<task>/
      README index + manifest + raw/redacted images + recordings
  → accepted source/build identities
  → .humanlayer/releases/<release>/<acceptance-sha12>/<platform>/
      retained originals, reviewed public copies, hashes and results
  → authorized PR/release evidence only

One canonical docs/agents/qa-evidence.md is linked from root/mobile agent instructions, both device QA guides, PR screenshot requirements and the release workflow. It records native binary versus loaded JavaScript, route/model, theme/text-size, action/readback and pass/fail/untested separately. Existing Android queue evidence is indexed or copied without changing the helper contract.

Closes #2569. Agent-Task: codex/local/01a16230-879c-7ea2-bdc1-9f65f57a031a.

@coderabbitai

coderabbitai Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b3d29670-9bfb-43b5-b803-9c1fcdf7c911
📥 Commits

Reviewing files that changed from the base of the PR and between 897e933 and 3848292.

📒 Files selected for processing: 7
  • AGENTS.md
  • apps/mobile/AGENTS.md
  • docs/agents/android-device-qa.md
  • docs/agents/changelog.md
  • docs/agents/device-qa.md
  • docs/agents/pr-ui-visual-evidence.md
  • docs/agents/qa-evidence.md

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 0 remain after this review.

📝 Walkthrough

The change adds a local guide for organizing, documenting, retaining, and sharing QA evidence. It also updates agent instructions and QA documentation to link to the guide and describe evidence indexing, platform results, release archives, and publication rules.

Changes: QA evidence conventions

Layer / File(s) Summary
Local evidence indexing
docs/agents/qa-evidence.md, AGENTS.md, apps/mobile/AGENTS.md, docs/agents/android-device-qa.md
The guide defines task-evidence locations and layouts. Agent and Android device instructions direct contributors to index captures while keeping existing capture output locations intact.
Results and provenance
docs/agents/qa-evidence.md, docs/agents/device-qa.md, docs/agents/pr-ui-visual-evidence.md
The guide specifies per-journey records, privacy handling, and limits on what screenshots establish. QA reporting keeps platform results separate and ties final release checks to a shared acceptance SHA.
Release archives and sharing
docs/agents/qa-evidence.md, docs/agents/changelog.md
The guide defines release archive contents, hash checks, build identities, and retention rules. Release guidance limits attachments to reviewed public derivatives.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Possibly related issues

  • None. The matching issue is already directly linked to this pull request.

Suggested reviewers: claude

Merge Risk: ⚪ Minimal · up to 38482

The evidence guidance is mergeable after normal checks; no outstanding issue is established.

Pre-merge checks: Passed 5
Check name Status Explanation
Title check Passed The title is concise, specific, and accurately summarizes the documentation change to preserve cross-platform QA evidence for releases.
Linked Issues check Passed Issue #2569 requires one local convention for macOS/iOS and Windows/Android evidence, provenance, private/public separation, release retention by accepted SHA, and links from the specified agent, QA, …
Out of Scope Changes check Passed The reported changes are limited to agent and QA documentation plus the new evidence convention. The guide keeps device queues and helper output authoritative and states that archive retention does no…
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Description check Passed The description follows the required template. It includes the issue link, user problem, merge risk, verification results and gaps, CI affected-plan selection, change outline, evidence boundaries, and…
Full details: Linked Issues check

Explanation

Issue #2569 requires one local convention for macOS/iOS and Windows/Android evidence, provenance, private/public separation, release retention by accepted SHA, and links from the specified agent, QA, PR, and release documents. The change adds docs/agents/qa-evidence.md with these rules and updates AGENTS.md, apps/mobile/AGENTS.md, both device-QA guides, docs/agents/pr-ui-visual-evidence.md, and docs/agents/changelog.md. The guide preserves platform ownership and existing Android queue output, records source/build and result data, maps originals to redacted copies, and states that screenshots do not prove acceptance. The supplied summary reports documentation checks and an exact-head CI pass.

Full details: Out of Scope Changes check

Explanation

The reported changes are limited to agent and QA documentation plus the new evidence convention. The guide keeps device queues and helper output authoritative and states that archive retention does not authorize device allocation, publication, submission, or cleanup. No runtime, native product, device-operation, publishing, or data-cleanup change is reported.

Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (7 skipped: 7 unsupported.)

Full details: Description check

Explanation

The description follows the required template. It includes the issue link, user problem, merge risk, verification results and gaps, CI affected-plan selection, change outline, evidence boundaries, and the reason that runtime screenshots are not applicable to this documentation-only change.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Issue: #2569

Agent-Task: codex/local/01a16230-879c-7ea2-bdc1-9f65f57a031a
@DoodleBears
DoodleBears merged commit cd38a52 into main Oct 22, 2026
11 checks passed
@DoodleBears
DoodleBears deleted the codex/ax-qa-evidence branch October 22, 2026 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AX: retain cross-platform QA screenshots for local inspection and release archives

1 participant