Repository navigation
modifier: fix missing update-ldcache hook in legacy mode - #2150
rootkiller6788 wants to merge 2 commits into
Conversation
blacksoxx
left a comment
There was a problem hiding this comment.
Thanks for tracking this down, the explanation of why the legacy call site passes discover.None{} made it easy to follow.
What I checked on b8a59b7:
- The fix gives the same hook as before cd69ba4:
hookCreator.Create(UpdateLDCacheHook)with no folders is whatldconfig.Hooks()returned for an empty mount list before the guard. With no--folderarguments,update-ldcachestill runsldconfig -f /etc/ld.so.conf, which reads theld.so.conf.ddrop-in thatenable-cuda-compatwrites, so the compat libraries do end up in the cache. Thediscover.Mergeorder keepsenable-cuda-compatfirst. - The guard is in v1.20.0 and v1.20.1, so this is a regression for legacy mode with
cuda-compat-mode = "hook"since v1.20.0. TestGetCudaCompatModeModifierfails with main'sgated.goand passes with yours.
One note on the two failures you mentioned: the branch is 74 commits behind main. I merged it into current main (44f1c6e, clean merge) and go test ./internal/modifier/... ./internal/discover/... passes there, including TestNewCSVModifier and TestDrmDevicesByPath. On macOS I could only reproduce the visible_devices_all failure from a checkout under /tmp, where the expected paths come back with a /private prefix, so it may be environment-related rather than pre-existing on main. A rebase would also let CI run against the current code. golangci-lint run ./internal/modifier/... reports 0 issues on the merge.
Since cd69ba4 the ldconfig discoverer returns no hook when it finds no library folders. The legacy-mode call site in getCudaCompatModeModifier passes discover.None{} deliberately: the CUDA compat libraries are only added to the ldconfig search path by the enable-cuda-compat hook while the container starts, so there is never anything to find here. As a result a legacy runtime with cuda-compat-mode = "hook" got the enable-cuda-compat hook but not the update-ldcache hook, so the compat libraries were written to /etc/ld.so.conf.d/00-compat-<uuid>.conf and ldconfig was never run. Create the hook with the hook creator directly instead of going through the ldconfig discoverer. Signed-off-by: rootkiller6788 <c8688rickowens@outlook.com>
The legacy mode cuda-compat-mode = "hook" case is the one that regressed in cd69ba4, so make sure the update-ldcache hook is injected alongside enable-cuda-compat. Also cover the cuda-compat-mode = "ldconfig" no-op case and the non-legacy modes, which rely on the discovered update-ldcache hook and must not get a second one. The driver version is faked so the test does not need a driver on the host. Signed-off-by: rootkiller6788 <c8688rickowens@outlook.com>
b8a59b7 to
59bc8b0
Compare
|
Thanks for the thorough review, and for confirming the fix restores the pre-cd69ba45 behavior. I've rebased the branch onto current main (1c9a334). The commits are now 8f0a3a7 Re-ran everything on a Linux checkout (Ubuntu 24.04, Go 1.27.1):
You were right about Thanks again — happy to adjust anything else you'd like. |
Legacy mode lost its update-ldcache hook.
getCudaCompatModeModifier() builds that hook through the ldconfig discoverer
with an empty mount discoverer, and it does so on purpose: on a legacy runtime
the CUDA compat libraries only end up in the ldconfig search path after the
enable-cuda-compat hook has run inside the container, so at spec-generation
time there is nothing for the discoverer to find.
cd69ba4 then added a guard so the discoverer returns no hook at all when it
finds no library folders. That guard is right for the CDI and CSV callers, but
it silently killed the legacy one: with cuda-compat-mode = "hook" you'd get
enable-cuda-compat and no update-ldcache, so the compat libraries were written
to /etc/ld.so.conf.d/00-compat-.conf and ldconfig never ran.
I fixed it at the one call site instead of touching the discoverer, since the
other four callers do have real libraries to discover.
Testing: I faked the driver version so the new test in gated_test.go needs no
gpu or driver on the host. It fails on main (only enable-cuda-compat comes
through) and passes with the fix. The two failures remaining in
./internal/modifier/... and ./internal/discover/... are pre-existing and
unrelated: TestNewCSVModifier/visible_devices_all and TestDrmDevicesByPath.