Repository navigation
CI: make the backend test environment cache deterministic - #13
Merged
Merged
Conversation
Two caches (mamba env and poetry venv) restored into the same env folder, and the venv cache used restore-keys. On nextjs this mixed conda's charset-normalizer 3.5.2 with poetry's 3.1.0, so the same commit passed on the PR and failed on push (charset_normalizer.md has no CharInfo). Use one cache keyed on environment.yml and poetry.lock with no partial matches, and always run poetry install.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What user problem are we solving?
Backend Checks failed on
nextjsafter #12 merged (run 40934362759) withAttributeError: module 'charset_normalizer.md' has no attribute 'CharInfo', even though the identical tree passed on the PR (run 40933573380). The result depended on which cache GitHub served, not on the code.Root cause
backend.ymlrestored two caches into the same conda env folder: a mamba env cache keyed only onenvironment.yml, and a poetry venv cache keyed onpoetry.lockwithrestore-keys(partial matches). On the push run the mamba cache missed, so conda installed charset-normalizer 3.5.2. The venv cache then hit exactly and was extracted on top, bringing poetry's 3.1.0 files, andpoetry installwas skipped. The mixed package brokeimport requests, and that broken env was saved back as a cache.What solution does this PR provide?
dlpenv, keyed on bothtraining/environment.ymlandtraining/poetry.lock, with norestore-keys. A hit is exactly the env those two files built.poetry installalways runs. It is close to a no-op on a correct cache and repairs anything else.dlp-env-...-1-) discards the poisoned caches.Testing Methodology
Backend Checks on this PR (lint and test) must pass from a cold cache. After merge, the push run on
nextjsshould pass too.🤖 Generated with Claude Code
Note
Low Risk
CI-only workflow changes; no runtime application code, though cache behavior affects whether backend lint/tests pass reliably.
Overview
Fixes non-deterministic Backend Checks caused by stacking two GitHub Actions caches on the same conda env path (mamba keyed on
environment.ymlplus poetry keyed onpoetry.lockwith partialrestore-keys), which could mix conda and Poetry package versions and skippoetry install.In
.github/workflows/backend.yml(lint and test jobs), this replaces the dual-cache setup with one cache of/usr/share/miniconda3/envs/dlp, keyed ontraining/environment.ymlandtraining/poetry.lockwith norestore-keys.mamba env updateruns only on cache miss;poetry installalways runs so a valid cache stays fast but inconsistent layers get repaired. The new key prefixdlp-env-...-1-drops previously saved bad caches.Reviewed by Cursor Bugbot for commit 0d68c5d. Bugbot is set up for automated code reviews on this repo. Configure here.