Repository navigation
fix(deploy): reach the changelog identity service over cluster DNS [CPONETOPS-1176] - #374
Merged
Merged
Conversation
…PONETOPS-1176] The tailnet steps could never have worked where they ran. `create-changelog` is on `arc-arm64-1cpu-2gb-nodocker`, whose pods get neither `/dev/net/tun` nor `NET_ADMIN`, so `tailscaled` never came up — service-ecotap-gateway run 37486823410 shows `failed to connect to local tailscaled` eight times, the step green only because of `continue-on-error`, and the wait step skipped on `outcome == 'failure'`. Of the ARC pools only the legacy `self-hosted-arm64-2xl` has Tailscale. So every release notification since the feature landed has been on the GitHub-profile-email fallback it was meant to replace. The runners are in the same EKS cluster (`internal`) as the identity service, so there was never a perimeter to cross: with a `*.svc.cluster.local` value in `CHANGELOG_IDENTITY_RESOLVE_URL` the lookup resolves directly. Measured from an `arc-arm64-1cpu-2gb-nodocker` runner: 200 in ~80 ms, and ~27 s of dead Tailscale install and retry gone from the job. `TAILSCALE_AUTHKEY` stays declared on both workflows even though nothing reads it. Passing a secret a reusable workflow does not declare is a startup error, and service-ecotap-gateway passes it explicitly in both of its deploy workflows; the input goes once those callers drop it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
cweinberger
requested review from
sudheer-monta
and removed request for
a team
October 7, 2026 07:33
joscdk
approved these changes
Oct 7, 2026
JesperTerkelsen
approved these changes
Oct 7, 2026
morten-andersen
approved these changes
Oct 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The Tailscale steps added in #362 never worked.
create-changelogruns onarc-arm64-1cpu-2gb-nodocker, and those pods get neither/dev/net/tunnorNET_ADMIN, sotailscaledcannot start. From service-ecotap-gateway run 37486823410 (job112350595118, runnerarc-arm64-1cpu-2gb-nodocker-qbwgz-runner-p4gq6):continue-on-error: truereported that step as green, and the wait step correctly skipped onoutcome == 'failure'. Perkube-manifestsinfra/arc/README.mdonly the legacyself-hosted-arm64-2xlpool has Tailscale access. So every release notification since the feature landed has fallen back to GitHub profile emails — the thing the identity service exists to replace.There was never a perimeter to cross: ARC runs on the
internalEKS cluster, which is also where the identity service lives.What
TailscaleandWait for the identity service…steps, and the job-levelTAILSCALE_AUTHKEYenv, fromcreate-changelogin bothdeploy-kotlin.ymlanddeploy-kotlin-v2.yml.vars.CHANGELOG_IDENTITY_RESOLVE_URLis still passed tochangelog-cli-actionexactly as before.docs/workflow-guide.md(both the v1 and v2 sections) andCLAUDE.md.Pairs with changing the org variable to the cluster-local URL:
Note the service name and the port —
project-tracker-internal-internalis the Linkerd mirror name, which exists onmain-production/main-stagingbut not oninternalitself, and the service listens on 3001, not 80.Measured, not assumed
A one-step probe job on
arc-arm64-1cpu-2gb-nodocker:Blast radius
IdentityResolveris only constructed insideReleaseNotificationService, whichChangeLogServiceonly builds when bothreleaseNotifyChannelandreleaseNotifySlackTokenare set. Two repos setchangelog-release-notify-channeltoday, both private, so both land on ARC:service-controldeploy-ocpp-proxy-production.yml#ocpp-proxyservice-ecotap-gatewaydeploy-staging.yml#service-ecotap-gatewayPublic-repo callers run on
ubuntu-24.04-arm, where the internal name does not resolve; the CLI degrades to email lookups and the job still succeeds, same as today.TAILSCALE_AUTHKEYstays declaredNothing reads it any more, but passing a secret a reusable workflow does not declare is a startup error, and
service-ecotap-gatewaypasses it explicitly todeploy-kotlin-v2.yml(deploy-staging.yml:61) anddeploy-kotlin.yml(deploy-production.yml:39). Removing the input here would break both deploys. Its description now says it is unused; the input can go once those callers drop it.sonar-cloud.ymldeclares and uses its own, untouched.🤖 Generated with Claude Code