Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,10 @@ The dev VPS lives on the Human's Tailscale tailnet. Per-machine values (SSH host

Native checks for `apps/mobile` share devices and QA Hosts among coding agents. Before booting an emulator/Simulator, building the app, starting a Host or touching a phone, follow [shared device QA](docs/agents/device-qa.md): choose the Mac/iOS or Windows/Android queue, submit to its owner-cut batch, use the dedicated QA Host, and report redacted results. Windows emulator and USB-phone checks use the shared `android-qa.ps1` lease and scrcpy capture workflow.

### QA evidence (AX)

Before capturing development screenshots or recordings, handing off macOS/iOS or Windows/Android QA, or retaining release images, follow [local QA evidence and release archives](docs/agents/qa-evidence.md). Keep an indexed task folder in the machine's main checkout and preserve accepted evidence by release and SHA; platform queues still own devices and capture tooling.

### Machine-local secrets (AX)

Files that device QA and store builds need but git must never hold (Firebase `google-services.json`, FCM and Play keys) live in the gitignored `.secrets/` folder of each machine's main checkout; every worktree resolves to the same folder. Read [machine-local secrets](docs/agents/device-secrets.md) for the file list, how the app finds them, local push builds and clean-clone EAS builds.
Expand Down
1 change: 1 addition & 0 deletions apps/mobile/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ Screens follow [DESIGN.md](./DESIGN.md): the Host-as-workspace structure, the sc
- `smoke --ios --no-start --no-reload --port <port> --json`: one exit code for "the app is connected and threw nothing". `--no-start` keeps it from starting Metro, a Simulator or Expo Go.
- Never run `agents:setup`, `new`, `install`, `deploy` or anything with `--eas`: they rewrite `AGENTS.md`/`CLAUDE.md`, change dependencies or spend EAS credits.
- Native devices and QA Hosts are shared by every agent. Read [shared device QA](../../docs/agents/device-qa.md) before builds or device interaction; it routes Mac/iOS and Windows/Android to their own batch queues. Windows emulator/USB phone control and scrcpy evidence use `scripts/android-qa.ps1` from the repo root.
- Screenshots, recordings and release image retention follow [local QA evidence](../../docs/agents/qa-evidence.md): index each platform's actual source/build and results in the main checkout, and retain accepted evidence outside disposable worktrees.
- Cloud sessions have no simulator. They use the web screenshots ("Screenshots without a phone") and reach a Mac through Remote Control for native checks.

## Profiling
Expand Down
4 changes: 4 additions & 0 deletions docs/agents/android-device-qa.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,10 @@ existing route state before changing a shared proxy.

## Phone control, screenshots and video

Index captures for local inspection and release retention using
[local QA evidence and release archives](qa-evidence.md). The helper's queue evidence directory
remains the capture source; link or copy selected evidence into the task/release index with hashes.

The Human connects/unlocks the phone, enables USB debugging and accepts the RSA authorization.
Use `devices` to distinguish `device`, `unauthorized` and `offline`; the helper never chooses
the first device automatically. Install only the development application ID, preserving the
Expand Down
4 changes: 4 additions & 0 deletions docs/agents/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,10 @@ history immutable and describe a restoration, withdrawal, or fix in the next ver

## Preparing a GitHub Release draft

Retain accepted screenshots and recordings using [local QA evidence and release archives](qa-evidence.md).
The archive binds separate macOS/iOS and Windows/Android results to one acceptance SHA and each
artifact's build identity; only reviewed public derivatives are candidates for release attachments.

After archiving `Unreleased` under a dated version, generate a review-only payload from that exact
canonical release section:

Expand Down
4 changes: 4 additions & 0 deletions docs/agents/device-qa.md
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,10 @@ Some steps only the Human can do. Ask once, with the exact command or action and

## Reporting results

Save and index captures using [local QA evidence and release archives](qa-evidence.md).
macOS/iOS and Windows/Android keep separate platform results and share the frozen acceptance SHA
for final release checks; existing runner output locations remain intact.

- **One comment per PR**, posted by the runner with `gh` from the Mac. It gives the PR head SHA
and the batch branch commit it ran on. It gives pass, fail or untested for each step, with the
reason for anything untested (for example "Android untested", "no second Host", or "needs a
Expand Down
4 changes: 4 additions & 0 deletions docs/agents/pr-ui-visual-evidence.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,10 @@ Use this guide when a PR changes a rendered screen or user interaction: layout,

## Capture

Keep local originals, redacted copies and their source/build records according to
[local QA evidence and release archives](qa-evidence.md). Local inspection paths supplement the
public, commit-pinned evidence required below.

1. Run the base revision and the PR revision through the real UI. Capture the baseline before editing when possible; otherwise use a separate base checkout. Take a **Before** and **After** screenshot for each materially changed view or interaction state. Include each viewport whose behavior changes.
2. Match viewport dimensions, theme, locale, test data, and UI state across each pair. Use representative data and remove secrets or personal information before publishing. For a new screen with no direct predecessor, show the prior entry point or absence state as **Before** and label it clearly.
3. Check the images side by side. They should show the actual change without relying on a mockup or a code diff. For motion or a multi-step interaction, capture the key states and add a short recording or exact reproduction steps.
Expand Down
107 changes: 107 additions & 0 deletions docs/agents/qa-evidence.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# Local QA evidence and release archives

Use this convention when capturing development screenshots or recordings, handing native QA
between machines, or retaining images for a release. macOS owns iOS Simulator/iPhone checks;
Windows owns Android emulator/phone checks. Follow each machine's
[iOS queue](device-qa.md) or [Android queue](android-device-qa.md) before operating resources.
An archive does not allocate a device or authorize a release.

## Save where the Human can find it

Default to the **main checkout on the machine that ran the check**, outside disposable
worktrees. Resolve that checkout from the Git common directory and the local queue; independent
clones use the queue's configured main checkout. Use repository-relative paths in shared reports,
not a hard-coded macOS or Windows home directory. A Human-selected folder overrides this default;
record its location privately in the task index.

```text
.humanlayer/tasks/<task>/
README.md # clickable local evidence index and current blockers
screenshots/<platform>/<lane>/<run>/
raw/ # original PNGs; private, unchanged
redacted/ # reviewed copies suitable for the intended audience
recordings/<platform>/<lane>/<run>/
manifest.json # source, build and per-step evidence provenance
```

Use `ios/simulator`, `ios/phone`, `android/emulator` or `android/phone` for platform/lane.
Use a short run ID containing the capture date and source SHA prefix. File names describe the
scene, before/after state, locale, theme and OS text size, for example
`composer-after-zh-dark-max.png`. Record the actual OS text-size setting in the manifest;
`max` alone cannot identify the setting. Reruns get new run IDs rather than overwriting evidence.
Keep feature-showcase images under a named scene in the same layout.

Existing tools keep their output contracts. In particular, `android-qa.ps1` writes captures and
private serial manifests under the Windows queue's `evidence/<batch>/<lane>/`; verification
tools may use their own task directories. Add links from the task index to these originals, or
copy selected files and record their source and SHA-256. Do not move live runner output, alter
its manifests, or duplicate a whole build/cache just to follow the display layout.

## Record what each image proves

Before capture, freeze the source and build identity. Alongside each journey record:

- Issue/PR, task and batch ID; full source SHA, base/candidate SHAs, and acceptance SHA when
assigned. Record every dependency head for an integration batch and any dirty-source diff.
- Platform, simulator/emulator versus phone, device model, OS version, app identity, version,
build number/profile, native input fingerprint and binary hash/build receipt where available.
A reused dev client records both binary source and loaded JavaScript source.
- Capture time with timezone, locale, theme, viewport and exact OS text-size setting; route
type and model identifier when relevant. Use synthetic display names and test content.
- Action, visible result, canonical Host readback when required, and `pass`, `fail` or
`untested` with a reason. Link each screenshot/recording and record its SHA-256.
- Original-to-redacted mapping, redaction/inspection status and any public evidence URLs.

Keep serials, private paths and environment details in the private manifest only. Keep credentials,
pairing links, QR payloads, account secrets and private keys out of captures and manifests.
Separate automated results, Human QA, Simulator/emulator, physical device and real-network
results. A screenshot does not prove a gesture, notification delivery, persistence or a model
reply; retain the corresponding interaction/recording/readback. iOS evidence never qualifies
Android, and a development binary does not qualify a store binary.

Before sharing, inspect every selected frame for status-bar details, real computer/account names,
paths, keyboard suggestions and unrelated personal content. Keep originals local and produce
separate redacted copies. For full-frame showcase images, prepare synthetic names/data and a
privacy-safe status bar before capture; preserve the original resolution and disclose any
redaction. Generated artwork is presentation material, not runtime QA evidence.

## Preserve a release snapshot

After the release coordinator freezes one acceptance SHA, both machines use that SHA for their
final source/build checks. Each machine retains its own platform evidence under:

```text
.humanlayer/releases/<release>/<acceptance-sha12>/
README.md
manifest.json
ios/ # retained on macOS
android/ # retained on Windows
```

For example, a 1.5.0 run uses `release-1.5.0` as the task slug and `1.5.0` as the release folder.
The shared directory shape is not automatic synchronization: the coordinator's index records
which machine holds each platform and which sanitized report is accessible to reviewers.
Mobile app versions/build numbers remain independent of the npm version and are recorded.

Copy the selected originals, redacted derivatives, necessary recordings and provenance before
cleaning up a task worktree. Verify copied file hashes and open the images from the archive index.
The release manifest maps the accepted commit to the npm tag/package and exact mobile builds,
with separate acceptance and publication statuses; leave unknown or pending fields explicit.
If a fix changes the acceptance SHA, create a new archive directory and identify which checks
were rerun or remain untested. Keep prior evidence as prior-SHA evidence.

Release archives and unresolved QA evidence are retained material, not disposable caches. Keep
them until the Human explicitly authorizes deletion or a verified relocation. Do not copy Host
profiles, credentials, app sandboxes, dependencies or native build folders into the archive.

## Publish selected evidence

For PRs, use reviewed derivatives and the commit-pinned image format in
[UI/UX PR visual evidence](pr-ui-visual-evidence.md); the queue may publish its assigned evidence
branch. Record the public URL and read it back to confirm that it renders. A local path is a
local inspection link, not proof that a GitHub reviewer can open the image.

For release notes or store material, select from the accepted archive, check the intended
audience and attach only the approved public derivatives. Image retention does not authorize
uploading private originals, publishing a GitHub Release, submitting a store build or notifying
testers. Keep local inspection links in the Human handoff and public evidence links in GitHub.
Loading