diff --git a/SUMMARY.md b/SUMMARY.md index 8929180..9dfee9c 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -74,6 +74,7 @@ ## Advanced * [Async Execution](advanced/async-execution.md) +* [Cancelling Superseded Runs](advanced/cancel-previous.md) * [Retry Strategies](advanced/retry-strategies.md) * [Chrome Onboarding](advanced/chrome-onboarding.md) * [Exit Codes](advanced/exit-codes.md) diff --git a/advanced/cancel-previous.md b/advanced/cancel-previous.md new file mode 100644 index 0000000..58c753c --- /dev/null +++ b/advanced/cancel-previous.md @@ -0,0 +1,124 @@ +# Cancelling superseded runs + +Push twice in quick succession and the first run's queued tests are dead weight: +nobody is waiting on their verdict, but they still sit in the queue and still +cost you. `--cancel-previous` cancels them when the newer run is submitted. + +It is opt-in and off by default. Without it, nothing about your runs changes. + +## What it does + +When a run carrying the flag is submitted, DeviceCloud looks for the previous +run from the same CI context and cancels the tests of that run which have not +started yet. + +* **Queued tests only.** Anything already running on a device finishes and + reports normally — cancelling it would free no device and refund nothing. +* **Cancelled tests are refunded at 75%**, the same as cancelling a + not-yet-started test by hand in the console. +* **The superseded run exits 0** instead of failing your build, and says so in + its output. It sends no completion email or webhook, and its GitHub check is + closed as `skipped` so it cannot block a pull request. + +## What counts as "the same CI context" + +The context is derived from the metadata your CI integration already attaches: + +| Part | Where it comes from | +|---|---| +| Repository | `gh_repo`, or `bb_repo` on Bitbucket | +| Branch or PR | `gh_pr_number` if present, otherwise `gh_branch` (`bb_` twins on Bitbucket) | +| Job | `gh_check_name` — the check name your integration passes | + +A pull request and a push to the same branch are **different** contexts, which +matches how GitHub Actions treats them. + +If a run has no repository, or no branch and no PR, there is no context to group +by and nothing is cancelled. The run's output says so rather than guessing. + +{% hint style="warning" %} +**Set a check name per job.** The check name is what keeps your iOS job from +cancelling your Android job. If one commit runs tests more than once and those +runs share a check name, they share a group — and the second will cancel the +first's queued tests. +{% endhint %} + +Runs from the **same** CI run never cancel each other, so a job matrix is safe +even before you set check names. Bitrise is the exception: it reports no build +id, so two `dcd cloud` invocations in one Bitrise build need distinct check +names. + +## Usage + +{% tabs %} +{% tab title="GitHub Actions" %} +```yaml +- uses: devicecloud-dev/device-cloud-for-maestro@v2 + with: + api-key: ${{ secrets.DCD_API_KEY }} + app-file: build/app.apk + cancel-previous: true + check-name: Android +``` +{% endtab %} + +{% tab title="CLI" %} +```bash +dcd cloud --app-file build/app.apk --flows ./.maestro --cancel-previous +``` + +Outside CI this does nothing: there is no repository or branch metadata to +derive a context from. +{% endtab %} + +{% tab title="Bitbucket" %} +```yaml +- pipe: docker://moropo/device-cloud-for-bitbucket:latest + variables: + API_KEY: $DCD_API_KEY + APP_FILE: build/app.apk + CANCEL_PREVIOUS: "true" + CHECK_NAME: Android +``` +{% endtab %} + +{% tab title="Bitrise" %} +Set the **Cancel Previous Run** input (`cancel_previous`) to `true`, and give +each invocation its own **GitHub Check Name** (`check_name`) if a build runs the +step more than once. +{% endtab %} + +{% tab title="EAS Workflows" %} +```yaml +- run: | + npx --yes @devicecloud.dev/eas-workflow@v1 \ + --app-file ${{ steps.download.outputs.artifact_path }} \ + --flows ./.maestro \ + --cancel-previous +``` +{% endtab %} +{% endtabs %} + +## How a superseded run reports + +The superseded run's CLI stops as soon as its tests are cancelled and prints: + +``` +! Run superseded by a newer run from the same CI context — exiting 0 +``` + +It exits `0`, so the older CI job goes green rather than red. Under `--json`, +`status` is `SUPERSEDED` — a third value alongside `PASSED` and `FAILED`, so +update any script that switches on it. + +Each result cancelled this way carries a `cancellation_reason` of +`superseded_by:`, visible in the console and in +[`GET /results/{uploadId}`](../api/results.md). + +## Limitations + +* It cannot stop a test that is already running on a device. A run whose tests + had all started is unaffected. +* GitHub's "Re-run failed jobs" reuses the same run id, so a re-run does not + supersede the original attempt. +* A previous run older than 24 hours is not superseded. diff --git a/api/results.md b/api/results.md index 3574ad6..5fa7299 100644 --- a/api/results.md +++ b/api/results.md @@ -39,6 +39,12 @@ curl https://api.devicecloud.dev/results/7e12345f-eb12-12ec-a30b-bb1234f1d12a \ } ``` +A `CANCELLED` result may also carry `cancellation_reason`. The only value +today is `superseded_by:`, set when a newer run from the same CI +context replaced this one — see +[Cancelling superseded runs](../advanced/cancel-previous.md). The field is absent +on every other result. + --- ## Download JUnit report diff --git a/ci-cd/bitbucket-pipelines.md b/ci-cd/bitbucket-pipelines.md index c886016..16dd89f 100644 --- a/ci-cd/bitbucket-pipelines.md +++ b/ci-cd/bitbucket-pipelines.md @@ -71,6 +71,7 @@ The pipe variables map 1:1 to the [`dcd cloud`](../cli/dcd-cloud.md) CLI flags. | `REPORT` | `junit`, `html`, `html-detailed`, `allure`. | | `DOWNLOAD_ARTIFACTS` | `ALL` or `FAILED` — downloads logs/screenshots/videos. | | `ASYNC` | `"true"` to fire-and-forget. | +| `CANCEL_PREVIOUS` | `"true"` to cancel the previous run's still-queued tests when this one starts. See [Cancelling superseded runs](../advanced/cancel-previous.md). | | `RUNNER_TYPE` | `m1` or `m4` for premium runners. | ## Bitbucket context auto-attached diff --git a/ci-cd/bitrise-steps.md b/ci-cd/bitrise-steps.md index f33c3ec..319692a 100644 --- a/ci-cd/bitrise-steps.md +++ b/ci-cd/bitrise-steps.md @@ -19,4 +19,6 @@ DeviceCloud includes a BitRise step to allow automatic triggering of tests via y If your repository is on GitHub and you use [GitHub checks](github-checks.md), the **GitHub Check Name** input (`check_name`) names the check a run posts — `iOS` gives `DeviceCloud / iOS`. Set it when a commit is tested by more than one workflow, so each gets a check you can require separately in branch protection. It applies to runs that also carry `gh_repo` and `gh_sha` metadata. +To cancel the previous run's still-queued tests when a new build starts, set the **Cancel Previous Run** input (`cancel_previous`) to `true` — see [Cancelling superseded runs](../advanced/cancel-previous.md). Bitrise reports no build id to DeviceCloud, so if one build runs the step more than once, give each invocation its own `check_name`; otherwise the second cancels the first. +
diff --git a/ci-cd/eas-workflows.md b/ci-cd/eas-workflows.md index 1a3b85d..98d8faf 100644 --- a/ci-cd/eas-workflows.md +++ b/ci-cd/eas-workflows.md @@ -229,6 +229,7 @@ See the [Devices & OS Versions](../getting-started/devices-configuration.md) pag | Flag | Description | |------|-------------| | `--async` | Exit immediately without waiting for results (exit code `0` regardless). See [Async Execution](../advanced/async-execution.md). | +| `--cancel-previous` | Cancel the still-queued tests of the previous run of this job on the same branch or PR. Needs the `DCD_GH_*` context env vars. See [Cancelling superseded runs](../advanced/cancel-previous.md). | | `--download-artifacts ` | Download logs/screenshots/videos. Options: `ALL`, `FAILED`. | | `--disable-animations` | Disable device animations. See [Animations](../configuration/disable-animations.md). | | `--maestro-chrome-onboarding` | Android only. See [Chrome Onboarding](../advanced/chrome-onboarding.md). | diff --git a/ci-cd/github-actions.md b/ci-cd/github-actions.md index 7fce4a3..85cfba4 100644 --- a/ci-cd/github-actions.md +++ b/ci-cd/github-actions.md @@ -156,6 +156,7 @@ The action automatically attaches Git and pull request metadata to each run, rea | Input | Required | Default | Description | |-------|----------|---------|-------------| | `async` | No | `false` | Exit immediately without waiting for results. Returns exit code `0` regardless of test outcome. See [Async Execution](../advanced/async-execution.md). | +| `cancel-previous` | No | `false` | Cancel the still-queued tests of the previous run of this job on the same branch or PR. See [Cancelling superseded runs](../advanced/cancel-previous.md). | | `quiet` | No | `false` | Reduce console output. Useful in verbose CI environments. | | `download-artifacts` | No | — | Download logs, screenshots, and videos after the run. Options: `ALL`, `FAILED`. | | `json-file` | No | `false` | Write test results to a JSON file (`_dcd.json` or `_dcd.json`). | diff --git a/ci-cd/overview.md b/ci-cd/overview.md index e4aadb7..6ceefb4 100644 --- a/ci-cd/overview.md +++ b/ci-cd/overview.md @@ -13,6 +13,7 @@ DeviceCloud supports a wide range of CI/CD options. If you don't see your provid ### Useful Features For CI/CD * [Async Execution](../advanced/async-execution.md) - Fire-and-forget tests without blocking your pipeline +* [Cancelling Superseded Runs](../advanced/cancel-previous.md) - Drop the previous run's queued tests when a newer commit arrives * [dcd status](../cli/dcd-status.md) - Poll for results after an async run * [Report Formats](../artifacts/report-formats.md) - Generate JUnit/HTML reports for your CI system * [Artifacts & Downloads](../artifacts/artifacts.md) - Access logs, screenshots, and videos diff --git a/cli/dcd-cloud.md b/cli/dcd-cloud.md index 3a7a1c3..df6d66d 100644 --- a/cli/dcd-cloud.md +++ b/cli/dcd-cloud.md @@ -98,6 +98,7 @@ Attach Git and pull request metadata to a run. These values are displayed in the | Flag | Description | |------|-------------| | `--async` | Submit tests and return immediately (exit `0`) without waiting for results (see [Async Execution](../advanced/async-execution.md)) | +| `--cancel-previous` | Cancel the still-queued tests of the previous run from the same CI context (see [Cancelling superseded runs](../advanced/cancel-previous.md)) | | `--quiet`, `-q` | Suppress per-test progress; print only the final summary | | `--json` | Output results as JSON. Exits `0` on success, `2` on test failure, `1` on CLI/infrastructure errors | | `--json-file` | Write JSON results to a file (`_dcd.json` by default). Exits `0` even if the test run fails; infrastructure errors still exit `1` |