diff --git a/.github/workflows/CHANGELOGS/deploy_environment.md b/.github/workflows/CHANGELOGS/deploy_environment.md index 4615cc8..00b5127 100644 --- a/.github/workflows/CHANGELOGS/deploy_environment.md +++ b/.github/workflows/CHANGELOGS/deploy_environment.md @@ -2,6 +2,9 @@ All notable changes to the **deploy_environment** reusable workflow are documented in this file. +> **Deprecated:** replaced by calling [`tf_apply`](tf_apply.md) directly, with the same inputs and +> secrets. No further releases. + ## 0.4.2 ### Changed diff --git a/.github/workflows/CHANGELOGS/deploy_thru_prod.md b/.github/workflows/CHANGELOGS/deploy_thru_prod.md index 48fd1f8..5951cf0 100644 --- a/.github/workflows/CHANGELOGS/deploy_thru_prod.md +++ b/.github/workflows/CHANGELOGS/deploy_thru_prod.md @@ -2,6 +2,17 @@ All notable changes to the **deploy_thru_prod** reusable workflow are documented in this file. +## 0.5.0 + +### Changed + +- Each environment deployment calls `tf_apply` 0.4.0 directly instead of the deprecated + `deploy_environment`, which only passed inputs through. Check names lose one level: for example, + `Apply Dev / Terraform Apply / Terraform Apply` becomes `Apply Dev / Terraform Apply`. +- The concurrency group covers the repository, the three Terraform roots, and the workspace, and is + prefixed with `core-gha-deploy-thru-prod-` so it can't match a caller's own group. Chains for + different roots or workspaces in one repository no longer wait on or replace each other. + ## 0.4.2 ### Changed diff --git a/.github/workflows/CHANGELOGS/tf_apply.md b/.github/workflows/CHANGELOGS/tf_apply.md index dad6093..f8747b5 100644 --- a/.github/workflows/CHANGELOGS/tf_apply.md +++ b/.github/workflows/CHANGELOGS/tf_apply.md @@ -2,6 +2,19 @@ All notable changes to the **tf_apply** reusable workflow are documented in this file. +## 0.4.0 + +### Changed + +- The concurrency group covers the repository, environment, Terraform root, and workspace, so + applies to different states in one environment no longer wait on each other, and a waiting apply + to one workspace is no longer replaced by a run for another. The group is prefixed with + `core-gha-tf-apply-` so it can't match a caller's own group. +- Runs of earlier versions use a different group, so they don't queue behind this version. Upgrade + every caller in a repository together. Terraform's state lock still stops two applies to one state. +- `terraform apply` passes `-lock-timeout=5m`, so it waits for a state lock held by another run, such + as a CI plan, instead of failing. + ## 0.3.2 ### Changed diff --git a/.github/workflows/READMES/deploy_environment.md b/.github/workflows/READMES/deploy_environment.md index a9dc03a..12dd8d4 100644 --- a/.github/workflows/READMES/deploy_environment.md +++ b/.github/workflows/READMES/deploy_environment.md @@ -1,5 +1,10 @@ # Deploy a Commit to One Environment +> **Deprecated:** call [`tf_apply`](tf_apply.md) instead. This workflow gets no further releases. It +> takes the same inputs and secrets and only passes them to `tf_apply`, so switch the `uses:` line +> and keep the `with:` and `secrets:` blocks. Check names lose one level: +> ` / Terraform Apply / Terraform Apply` becomes ` / Terraform Apply`. + This reusable workflow applies one Terraform root to a selected environment by calling the `tf_apply` reusable workflow. `terraform apply` validates and plans as part of the apply, so the workflow has no separate plan job. Callers should validate in Repo Checks and plan in CI before merge, diff --git a/.github/workflows/READMES/deploy_thru_prod.md b/.github/workflows/READMES/deploy_thru_prod.md index 54ae9e2..42e0f54 100644 --- a/.github/workflows/READMES/deploy_thru_prod.md +++ b/.github/workflows/READMES/deploy_thru_prod.md @@ -10,7 +10,7 @@ GitHub release, and posts the final stage, prod, and release status to the assoc ## Pairing plan and apply -This version composes `deploy_environment` 0.4.1 and `tf_apply` 0.3.1, whose `terraform init` +This version composes `tf_apply` 0.4.0, whose `terraform init` installs the provider versions in each root's committed `.terraform.lock.hcl`. Plan in CI with [`tf_plan_single_root`](tf_plan_single_root.md) or [`tf_plan_env_roots`](tf_plan_env_roots.md), which do the same. The deprecated `tf_validate_plan_*` workflows still run @@ -36,7 +36,7 @@ apply differ from what CI showed. ```yaml jobs: deploy-through-prod: - uses: OpenSesame/core-github-actions/.github/workflows/deploy_thru_prod.yml@workflows/deploy_thru_prod/0.4.2 + uses: OpenSesame/core-github-actions/.github/workflows/deploy_thru_prod.yml@workflows/deploy_thru_prod/0.5.0 with: commit-identifier: ${{ github.sha }} oidc-domain: core @@ -68,7 +68,7 @@ jobs: ## Behavior 1. Generates a UTC runner timestamp in `YYYY.MM.DD-HH-MM` form for the release tag. -2. Deploys dev by calling `deploy_environment`. +2. Deploys dev by calling `tf_apply`. 3. Deploys stage only after dev succeeds. 4. Deploys prod only after stage succeeds. 5. Creates a GitHub release for `commit-identifier` with `gh release create`, only after prod @@ -76,8 +76,11 @@ jobs: 6. Posts stage, prod, and release results to the pull request when a PR context is available. The release-tag job only runs when the caller's event is a merged pull request or -`workflow_dispatch`. Other caller event contexts cause the deployment chain to be skipped. Runs for -the same repository share one concurrency group. +`workflow_dispatch`. Other caller event contexts cause the deployment chain to be skipped. + +Runs for the same repository, Terraform roots, and workspace share a concurrency group. One chain +deploys at a time and one waits. A newer run replaces the waiting one, since its commit includes the +older changes. Each environment's apply also takes the `tf_apply` concurrency group. The workflow sets `permissions: {}`, so callers still grant `id-token: write`, `contents: write`, and `pull-requests: write`, and each job requests only what it uses: the deploy jobs @@ -92,4 +95,4 @@ Its direct and composed jobs run on Ubuntu 26.04. ## References -- [Single-environment deploy workflow](deploy_environment.md) +- [Terraform apply workflow](tf_apply.md) diff --git a/.github/workflows/READMES/tf_apply.md b/.github/workflows/READMES/tf_apply.md index 5807b23..549f2dd 100644 --- a/.github/workflows/READMES/tf_apply.md +++ b/.github/workflows/READMES/tf_apply.md @@ -36,7 +36,7 @@ apply differ from what CI showed. ```yaml jobs: terraform-apply: - uses: OpenSesame/core-github-actions/.github/workflows/tf_apply.yml@workflows/tf_apply/0.3.2 + uses: OpenSesame/core-github-actions/.github/workflows/tf_apply.yml@workflows/tf_apply/0.4.0 with: environment: dev oidc-domain: core @@ -80,11 +80,18 @@ jobs: 5. Runs `terraform init`, which installs the provider versions in the required, committed `.terraform.lock.hcl`. 6. Selects or creates `terraform-workspace`. -7. Runs `terraform apply -auto-approve -input=false`. +7. Runs `terraform apply -auto-approve -input=false -lock-timeout=5m`, which waits up to five + minutes for another run's state lock, such as a CI plan's. The apply job sets `TF_VAR_IACDeploymentRef` to the current Actions run URL and -`TF_VAR_release_name` to `release-tag`. Runs for the same repository and environment share a -concurrency group. The workflow sets `permissions: {}`; the apply job requests `id-token: write` +`TF_VAR_release_name` to `release-tag`. + +Runs for the same repository, environment, root, and workspace share a concurrency group, so applies +to different states run in parallel. One run applies at a time and one waits. A newer run replaces +the waiting one, since its commit includes the older changes. A manually dispatched run, such as a +rollback, can be replaced the same way, so check that it ran. + +The workflow sets `permissions: {}`; the apply job requests `id-token: write` and `contents: read`, and the summary job requests none. Both workflow jobs run on Ubuntu 26.04. diff --git a/.github/workflows/TESTS/tf_apply.unit.test.js b/.github/workflows/TESTS/tf_apply.unit.test.js new file mode 100644 index 0000000..000c04a --- /dev/null +++ b/.github/workflows/TESTS/tf_apply.unit.test.js @@ -0,0 +1,44 @@ +const { readWorkflow } = require('./workflow-test-helpers'); + +const concurrency = workflowFile => { + const block = readWorkflow(workflowFile).match(/^concurrency:\n((?: {2}.*\n)+)/m); + if (!block) { + throw new Error(`No workflow-level concurrency block in ${workflowFile}`); + } + return block[1]; +}; + +describe('Terraform apply concurrency', () => { + test('tf_apply locks one Terraform state: repository, environment, root, and workspace', () => { + const block = concurrency('tf_apply.yml'); + + expect(block).toContain( + 'group: core-gha-tf-apply-${{ github.repository }}-${{ inputs.environment }}-${{ inputs.terraform-root }}-${{ inputs.terraform-workspace }}' + ); + expect(block).not.toContain('cancel-in-progress'); + }); + + test('tf_apply waits for a state lock held by another run', () => { + expect(readWorkflow('tf_apply.yml')).toContain( + 'run: terraform apply -auto-approve -input=false -lock-timeout=5m' + ); + }); + + test('deploy_thru_prod locks its roots and workspace and calls tf_apply directly', () => { + const workflow = readWorkflow('deploy_thru_prod.yml'); + const block = concurrency('deploy_thru_prod.yml'); + + expect(block).toMatch(/group: core-gha-deploy-thru-prod-\$\{\{ github\.repository \}\}-/); + for (const input of [ + 'terraform-root-dev', + 'terraform-root-stage', + 'terraform-root-prod', + 'terraform-workspace', + ]) { + expect(block).toContain(`inputs.${input}`); + } + expect(block).not.toContain('cancel-in-progress'); + expect(workflow.match(/uses: \.\/\.github\/workflows\/tf_apply\.yml/g)).toHaveLength(3); + expect(workflow).not.toContain('deploy_environment.yml'); + }); +}); diff --git a/.github/workflows/deploy_thru_prod.yml b/.github/workflows/deploy_thru_prod.yml index 3a2488d..d89c197 100644 --- a/.github/workflows/deploy_thru_prod.yml +++ b/.github/workflows/deploy_thru_prod.yml @@ -37,8 +37,10 @@ on: ORG_GITHUB_PACKAGES_READ_ONLY_TOKEN: required: true - -concurrency: ${{ github.repository }} +# One deployment chain at a time per set of Terraform states. A newer pending run replaces an older +# pending one, since the newer commit includes its changes. +concurrency: + group: core-gha-deploy-thru-prod-${{ github.repository }}-${{ inputs.terraform-root-dev || 'terraform/dev' }}-${{ inputs.terraform-root-stage || 'terraform/stage' }}-${{ inputs.terraform-root-prod || 'terraform/prod' }}-${{ inputs.terraform-workspace }} permissions: {} @@ -58,7 +60,7 @@ jobs: DeployDev: name: Apply Dev needs: [Set-Release-Tag] - uses: ./.github/workflows/deploy_environment.yml + uses: ./.github/workflows/tf_apply.yml permissions: id-token: write # Required to assume the Terraform apply role through OIDC. contents: read # Required to check out the Terraform code. @@ -77,7 +79,7 @@ jobs: DeployStage: name: Apply Stage needs: [Set-Release-Tag, DeployDev] - uses: ./.github/workflows/deploy_environment.yml + uses: ./.github/workflows/tf_apply.yml permissions: id-token: write # Required to assume the Terraform apply role through OIDC. contents: read # Required to check out the Terraform code. @@ -96,7 +98,7 @@ jobs: DeployProd: name: Apply Prod needs: [Set-Release-Tag, DeployStage] - uses: ./.github/workflows/deploy_environment.yml + uses: ./.github/workflows/tf_apply.yml permissions: id-token: write # Required to assume the Terraform apply role through OIDC. contents: read # Required to check out the Terraform code. diff --git a/.github/workflows/tf_apply.yml b/.github/workflows/tf_apply.yml index fd3c92f..32a4c04 100644 --- a/.github/workflows/tf_apply.yml +++ b/.github/workflows/tf_apply.yml @@ -38,8 +38,10 @@ on: ORG_GITHUB_PACKAGES_READ_ONLY_TOKEN: required: true +# One apply at a time per Terraform state. A newer pending run replaces an older pending one, since +# the newer commit includes its changes. concurrency: - group: ${{ github.repository }}-${{ inputs.environment }}-TF-Apply + group: core-gha-tf-apply-${{ github.repository }}-${{ inputs.environment }}-${{ inputs.terraform-root }}-${{ inputs.terraform-workspace }} permissions: {} @@ -128,4 +130,4 @@ jobs: run: terraform workspace select -or-create "$TF_WORKSPACE_NAME" - name: Apply Terraform - run: terraform apply -auto-approve -input=false + run: terraform apply -auto-approve -input=false -lock-timeout=5m