Skip to content

[ci] Harden update_tag_in_docs_and_files.sh against silent staleness - #3276

Merged
VietND96 merged 1 commit into
trunkfrom
ci/harden-tag-rewrite-script
Oct 19, 2026
Merged

VietND96 merged 1 commit into
trunkfrom
ci/harden-tag-rewrite-script

Conversation

@VietND96

Copy link
Copy Markdown
Member

Summary

Follow-up to #3274 / #3275. That issue happened because update_tag_in_docs_and_files.sh's whole-tree sed "s/${LATEST_TAG}/${NEXT_TAG}/g" only matches when LATEST_TAG is the exact string still embedded in the files. Once a wrong LATEST_TAG (or anything else) makes it miss once, every later release searches for a tag that's never actually there, and the chart's default image tags drift silently - for three releases, in this repo's actual history - with nothing in CI to signal it.

This hardens the script itself so the same class of bug can't recur unnoticed:

  1. Fail loudly instead of drifting silently. After the rewrite, assert that charts/selenium-grid/values.yaml's imageTag and nodesImageTag actually equal NEXT_TAG, and exit 1 with a clear message if not. I verified this two ways against a real Linux container (GNU find/sed, matching what the GitHub Actions runners use):

  2. Stop the sed from corrupting files it shouldn't touch. A full end-to-end run (not just the hand-reviewed [ci] Catch up stale default image tags to 4.51.0-20261010 #3275 catch-up) surfaced two real false-positive targets:

    • charts/selenium-grid/multiple-nodes-platform-version.yaml and its test copy tests/charts/ci/multiple-nodes-platform-version-values.yaml pin per-Chrome-version image tags (146.0-<date>, 145.0-<date>, ...), dated independently of the Grid release by their own browser-version release process. Their date collides with the Grid release's date often enough that the bare-date pass was rewriting dates that were never actually built for those specific browser versions.
    • .github/workflows/deploy.yml's only tag-shaped strings are inside one comment narrating this exact past incident, by its specific tag values (explaining why LATEST_TAG resolution moved from git tag --sort to git describe). Every later release's LATEST_TAG eventually matches one of those literals, silently turning the incident report into a false account of its own history - I caught this happening again during testing for [ci] Catch up stale default image tags to 4.51.0-20261010 #3275's catch-up and reverted it there by hand, but the script would keep doing it forever without this exclusion.

Test plan

🤖 Generated with Claude Code

#3274 happened because this script's whole-tree sed only matches when
LATEST_TAG is the exact string still embedded in the files; once a
wrong LATEST_TAG - or anything else - makes it miss, every later
release looks for a tag that's never actually there and the chart's
default image tags drift forever with no signal anything is wrong.

Two changes:

- After the rewrite, assert charts/selenium-grid/values.yaml's
  imageTag and nodesImageTag actually equal NEXT_TAG, and exit 1 with
  a clear message if not. Verified this both fixes the real catch-up
  (exit 0, matching #3275) and catches the exact wrong-LATEST_TAG
  scenario that caused #3274 (exit 1, correct error) when run against
  a container with real GNU find/sed, matching what CI runs.

- Exclude three files from the tag/date sed passes that a real
  end-to-end run showed getting corrupted instead of fixed:
  multiple-nodes-platform-version.yaml and its test copy pin
  per-Chrome-version tags dated independently of the Grid release, and
  their date collides with the Grid release's date often enough that
  the bare-date pass was rewriting dates that were never actually
  built for those browser versions. deploy.yml's only tag-shaped
  strings are inside one comment narrating this exact past incident by
  its specific values - every later release's LATEST_TAG eventually
  matches one of those literals, silently turning the incident report
  into a false account of itself.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@qodo-code-review

Copy link
Copy Markdown
Contributor

ⓘ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing

@VietND96
VietND96 merged commit 36ea99b into trunk Oct 19, 2026
35 checks passed
@VietND96
VietND96 deleted the ci/harden-tag-rewrite-script branch October 19, 2026 12:40
VietND96 added a commit that referenced this pull request Oct 19, 2026
helm-chart-release.yml only triggers on changes to
charts/selenium-grid/Chart.yaml, so the values.yaml fix in this PR
would never actually get published as a chart release on its own.
Patch bump: existing defaults corrected, no new features, no
appVersion change (Grid version is unaffected).

Also fix .github/ISSUE_TEMPLATE/bug_report.yml's version placeholder,
corrupted by a different sed bug: update_tag_in_docs_and_files.sh's
chart-version-bump step uses the chart version as an unescaped sed
pattern, so "0.60.0" (dots as regex wildcards) matched a 6-digit
substring inside "20260905" and replaced it with "0.61.0", producing
"20.61.05". Root-caused; the regex-escaping fix for the script itself
is in #3276.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@VietND96

Copy link
Copy Markdown
Member Author

Added one more fix while bumping the chart for #3275 (PR #3275 bumps charts/selenium-grid/Chart.yaml to publish that fix - helm-chart-release.yml only triggers on changes to that file).

While checking what a chart-coupled release actually does, found that this script's "bump chart version" step has its own version of the same class of bug: sed -i "s/${latest_chart_version}/${next_chart_version}/g" uses the chart version as a sed pattern (regex), where . matches any character. An unescaped three-part version like 0.60.0 can match an unrelated 6-digit substring - and did, repeatedly, in .github/ISSUE_TEMPLATE/bug_report.yml's version placeholder:

  • 4.48.0-20260905? Please use... → (chart 0.60.0→0.61.0) → 4.48.0-20.61.05? Please...
  • → (chart 0.61.0→0.62.0) → 4.48.0-20.62.05? Please...

Fixed by escaping the version string before using it as the sed pattern. Verified by replaying the exact historical scenario that caused the corruption (LATEST_TAG=4.48.0-20260905, NEXT_TAG=4.49.0-20260909, chart 0.60.0) in a real container: Chart.yaml now bumps 0.60.0 → 0.61.0 cleanly and the placeholder updates to 4.49.0-20260909? ... instead of the garbled ...20.61.05....

VietND96 added a commit that referenced this pull request Oct 19, 2026
* [ci] Catch up stale default image tags to 4.51.0-20261010

Fixes #3274. Since the 4.49.0 release, update_tag_in_docs_and_files.sh's
whole-tree sed has matched nothing in these 51 files: the old
sort-based LATEST_TAG resolution picked 4.48.0-20261010 (lexically
highest) instead of the real previous tag 4.48.0-20260905, so the sed
found no match and these files were never rewritten. #3254 fixed the
resolution (switched to ancestry-based `git describe`), but every
release since then searches for the immediately-preceding tag, which
is never what's actually still embedded several releases back - so it
can't self-heal without a one-time manual catch-up.

Ran update_tag_in_docs_and_files.sh 4.48.0-20260905 4.51.0-20261010
locally and reviewed the diff by hand. Reverted four files it touched
that shouldn't be in scope:
- .github/workflows/deploy.yml: the stale tag appears only inside a
  comment narrating this exact past incident by its specific values;
  rewriting it would corrupt the history it's documenting.
- charts/selenium-grid/multiple-nodes-platform-version.yaml and
  tests/charts/ci/multiple-nodes-platform-version-values.yaml: these
  pin per-Chrome-version image tags (146.0, 145.0, ...), each dated
  independently of the Grid release cadence. The date-only sed pass
  matched their "20260905" by coincidence, not because they're stale
  in the same way - a real fix for these needs its own pass.
- tests/ci/test_cleanup_pr_images.py: synthetic example tags in a
  regex-matching unit test, unrelated to live defaults.

Verified the restored tags are real published images (Docker Hub and
the GitHub release) before committing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* [ci] Bump chart to 0.62.1 to publish the stale-tag fix

helm-chart-release.yml only triggers on changes to
charts/selenium-grid/Chart.yaml, so the values.yaml fix in this PR
would never actually get published as a chart release on its own.
Patch bump: existing defaults corrected, no new features, no
appVersion change (Grid version is unaffected).

Also fix .github/ISSUE_TEMPLATE/bug_report.yml's version placeholder,
corrupted by a different sed bug: update_tag_in_docs_and_files.sh's
chart-version-bump step uses the chart version as an unescaped sed
pattern, so "0.60.0" (dots as regex wildcards) matched a 6-digit
substring inside "20260905" and replaced it with "0.61.0", producing
"20.61.05". Root-caused; the regex-escaping fix for the script itself
is in #3276.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant