Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ jobs:
run: pnpm build:e2e:desktop

- name: Run desktop E2E suite
run: pnpm test:e2e:desktop:run
run: pnpm test:e2e:desktop:run -- --workers 1

- name: Upload failure evidence
if: failure()
Expand All @@ -144,7 +144,7 @@ jobs:
Start-Sleep -Seconds 10

$orphans = Get-Process -Name leafdown-e2e, msedgedriver -ErrorAction SilentlyContinue
$listeners = Get-NetTCPConnection -LocalPort 4445 -State Listen -ErrorAction SilentlyContinue
$listeners = Get-NetTCPConnection -LocalPort (4445..4448) -State Listen -ErrorAction SilentlyContinue

if ($orphans -or $listeners) {
$orphans | Format-Table -AutoSize Name, Id, StartTime
Expand Down
16 changes: 12 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,22 +80,30 @@ Run the assembled desktop E2E suite, which requires Windows, with:
pnpm test:e2e:desktop
```

This explicit suite is not part of `pnpm check`; CI runs it as its own job on every pull request and every push to `main`, so it is enforced without changing what you run locally. It builds an isolated debug binary with test-only WebDriver capabilities, then runs one embedded WebDriver worker at a time on port 4445 across fresh application sessions. The embedded provider does not require an external WebDriver. Keep port 4445 available while it runs. The test identifier, persisted store, and target directory are separate from ordinary Leafdown builds.
This explicit suite is not part of `pnpm check`; CI runs it as its own job on every pull request and every push to `main`, so it is enforced without changing what you run locally. It builds one isolated debug binary with test-only WebDriver capabilities, then runs fresh application sessions through the embedded provider without requiring an external WebDriver. The runner defaults to one worker on port 4445. CI requests that deterministic single-worker mode explicitly until comparable hosted-run timings support a change.

Run the full suite with two workers against an already built binary:

```powershell
pnpm test:e2e:desktop:run -- --workers 2
```

Worker counts from 1 through 4 are accepted. Workers use consecutive ports beginning at 4445, so keep that range available for the requested count. Each worker receives a distinct runtime application identifier, persisted-data and WebView2 roots, fixture tree, context file, and artifact directory. Independent scenario groups may overlap; the `persistence` target always keeps its write and fresh-process restart sequence ordered on one worker. One local Windows sample on 2026-09-22 ran the already-built full suite in about 123 seconds with one worker and 74 seconds with two. That sample supports opt-in local concurrency but is not treated as hosted-CI evidence.

While iterating on an already built E2E binary, run one target without rebuilding it:

```powershell
pnpm test:e2e:desktop:run -- --scenario folder-watcher
```

Valid targets are `block-selection`, `diagnostics`, `document-lifecycle`, `folder-watcher`, `rendered-images`, `rendered-html`, `missing-document-error`, `persistence`, and `window-lifecycle`. The `persistence` target runs its write and fresh-process restart scenarios in order. Focused runs create the same fixtures and isolated state, preserve the same failure evidence and cleanup, and start a fresh packaged-app process; they do not check whether the binary is current. Run `pnpm build:e2e:desktop` first whenever E2E binary inputs change. The default `pnpm test:e2e:desktop` command and CI still run the full suite.
Valid targets are `block-selection`, `diagnostics`, `document-lifecycle`, `folder-watcher`, `rendered-images`, `rendered-html`, `missing-document-error`, `persistence`, and `window-lifecycle`. Focused runs create the same fixtures and isolated state, preserve the same failure evidence and cleanup, and start a fresh packaged-app process; they do not check whether the binary is current. Run `pnpm build:e2e:desktop` first whenever E2E binary inputs change. The default `pnpm test:e2e:desktop` command and CI still run the full suite.

The runner resets only the isolated E2E persisted store, leaving the application to write its own defaults, creates temporary filesystem fixtures, and removes both after the suite. Each run writes ignored runner, frontend, backend, and focused diagnostic evidence under `e2e/desktop/artifacts/<run>/<scenario>/`. A failed test also captures a screenshot, the real diagnostics summary, the test error, and a semantic UI snapshot that excludes editor content. A failed run additionally writes `fixture-manifest.json` under `e2e/desktop/artifacts/<run>/`, recording each temporary fixture's path, expected and actual hash and size, and modification time before cleanup removes it. Local artifacts are retained until manually deleted. Treat them as potentially sensitive because diagnostics and errors may contain local paths. CI uploads the same evidence only when the job fails, retained for seven days; those artifacts contain runner paths rather than a contributor's.
The runner resets only each worker's isolated E2E persisted store, leaving the application to write its own defaults, creates temporary filesystem fixtures, and removes the worker-owned state after the suite. Each run writes ignored runner, frontend, backend, and focused diagnostic evidence under `e2e/desktop/artifacts/<run>/worker-<n>/<scenario>/`. A failed test also captures a screenshot, the real diagnostics summary, the test error, and a semantic UI snapshot that excludes editor content. A failed worker additionally writes `fixture-manifest.json` under its worker directory, recording its scenario and port plus each temporary fixture's path, expected and actual hash and size, and modification time before cleanup removes it. Other workers finish their queued groups and clean up independently. Local artifacts are retained until manually deleted. Treat them as potentially sensitive because diagnostics and errors may contain local paths. CI uploads the same evidence only when the job fails, retained for seven days; those artifacts contain runner paths rather than a contributor's.

To verify the failure-evidence path, run the suite with the forced-failure flag:

```powershell
$env:LEAFDOWN_E2E_FORCE_FAILURE=1; pnpm test:e2e:desktop; $env:LEAFDOWN_E2E_FORCE_FAILURE=$null
$env:LEAFDOWN_E2E_FORCE_FAILURE=1; pnpm test:e2e:desktop:run -- --workers 2; $env:LEAFDOWN_E2E_FORCE_FAILURE=$null
```

The Diagnostics scenario should fail, retain its evidence, clean its fixture and store state, and return a nonzero exit code.
Expand Down
4 changes: 3 additions & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,9 @@ Automated tests focus on:
- Safe live HTML and literal fallback, attribute and namespace rejection, script/resource-load prevention, exact HTML round trips, and source-projection history and save finalization. Verify parsing and block-atom layout in the desktop WebView as well as the DOM test environment.
- Context popup layout and caret-based marker visibility.

The assembled desktop E2E suite complements those component and boundary tests without replacing them. It requires Windows and stays outside `pnpm check`, running through an explicit command locally and as its own CI job on every pull request and push to `main`. It runs one embedded WebDriver worker at a time against an isolated debug binary and starts fresh application processes for independent scenarios. Developers can run an already built target with `pnpm test:e2e:desktop:run -- --scenario <name>`; the runner validates the target before setup and preserves the selected scenario's fixture, isolated-state, artifact, fresh-process, port-release, and cleanup guarantees. The `persistence` target always runs its write and restart scenarios in order. This focused interface never checks binary freshness, so developers rebuild explicitly when binary inputs change; the default command and CI run every scenario. The suite retains the Help → Diagnostics smoke path, then adds narrow assembled-boundary assertions for the document lifecycle, real folder-watcher refresh, typed backend error propagation, persisted settings across restart, injected frame controls, and the clean window-close handshake.
The assembled desktop E2E suite complements those component and boundary tests without replacing them. It requires Windows and stays outside `pnpm check`, running through an explicit command locally and as its own CI job on every pull request and push to `main`. One isolated debug binary serves every worker. The runner schedules independent scenario groups with configurable bounded concurrency, while each worker receives its own WebDriver port, runtime application identifier, persisted-data and WebView2 roots, fixture tree, context file, and artifact namespace. A worker starts a fresh application process per scenario; the persistence write and restart scenarios remain one ordered, stateful group. Failure evidence and cleanup stay worker-owned so one failed worker cannot remove another worker's state, and healthy workers finish already queued coverage.

Developers can run an already built target with `pnpm test:e2e:desktop:run -- --scenario <name>` or the full suite with `--workers <1-4>`; the runner validates both options before setup and preserves the fixture, isolated-state, artifact, fresh-process, port-release, and cleanup guarantees. This interface never checks binary freshness, so developers rebuild explicitly when binary inputs change. The default command and CI run every scenario with one worker; local concurrency is opt-in until hosted-CI timing and reliability evidence justify changing that deterministic default. The suite retains the Help → Diagnostics smoke path, then adds narrow assembled-boundary assertions for the document lifecycle, real folder-watcher refresh, typed backend error propagation, persisted settings across restart, injected frame controls, and the clean window-close handshake.

User-visible acceptance paths use semantic UI interactions. Scenarios decide that an operation happened from state that outlives it — on-disk contents, menu item state, editor contents, or diagnostic records — rather than from an affordance that dismisses on a timer. Where the notification is itself the reported outcome, the desktop E2E build disables toast auto-dismissal so the assertion reads a settled affordance instead of racing it. Direct bridge execution is limited to corroborating diagnostic state, while Node-side filesystem, persisted-store, log, and process access provides deterministic setup or evidence around the native boundary. WebDriver plugins, permissions, and frontend integration remain limited to the dedicated desktop E2E build and are excluded from ordinary application builds.

Expand Down
Loading