Skip to content

Glibc version incompatibility on Ubuntu 24.04 hosts with NVidia runtime #186

Description

@Hboni

Describe the bug
When I want to launch interface like brainvisa, on my Ubuntu 24.04 computer, I have an import error with the message :

ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /.singularity.d/libs/libGLdispatch.so.0)

This error occurs after an update of the drivers of my computer (NVIDIA). Thanks to @sapetnioc, we saw that libc version inside the container is 2.35, however the error mention the need of 2.38. We suspect that drivers of Ubuntu 24.04 requires recent libc which is not available in the container.
If I specify opengl=software I do not have the issue anymore, but the issue might happens more later.

To Reproduce
Steps to reproduce the behavior:

  1. Install a brainvisa environment on a Ubuntu 24.04
  2. Update nvidia drivers to <=550
  3. Try to launch brainvisa

Expected behavior
A clear and concise description of what you expected to happen.

Environment:

  • Engine: Singularity
  • Version of BrainVISA: master branch
  • casa-dev image: 5.3-19 & 5.3-26

Activity

  1. added
    bugSomething isn't working
    on Sep 9, 2024
  2. denisri commented on Sep 13, 2024

    @denisri
    Contributor

    If libc 2.38 is backward-compatible with 2.35 (which should be OK) then we can try mounting the libc of the host system into the container in this situation. But automating this operation in casa-distro will not be easy, as we must do it only if the libc of the host system is newer than the one of the container. But to begin with, we must do the test.

  3. pinned this issue on Sep 20, 2024
  4. ylep commented on Sep 20, 2024

    @ylep
    Member

    The issue is confirmed on a newly remastered Ubuntu 24.04 workstation. New installs in NeuroSpin are now done with Ubuntu 24.04 unless 22.04 is specifically requested, so we have to find a solution relatively quickly. Pinning the issue.

    From a technical point of view, I do not yet have a clear idea of the implications... but technically, the inclusion of these libraries is prompted by nvidia-container-cli (https://github.com/NVIDIA/libnvidia-container), so they surely are aware of this issue and maybe they already have a solution for it?

  5. changed the title [-]Error libc for PyQt ubuntu 24[/-] [+]Glibc version incompatibility on Ubuntu 24.04 hosts with NVidia runtime[/+] on Sep 20, 2024
  6. denisri commented on Sep 20, 2024

    @denisri
    Contributor

    Could you try mounting the libc of the host system (Ubuntu 24.04) in the container (Ubuntu 22.04) ?

  7. denisri commented on Sep 20, 2024

    @denisri
    Contributor

    I have logged in your machine, and trying:
    bv container_options='-B /usr/lib/x86_64-linux-gnu/libc.so.6,-B /lib64/ld-linux-x86-64.so.2' bash
    it seemed to work.
    However passing these options is not trivial for inexperienced users, and I would not know how to automatize it, knowing that they are needed only in some specific combinations of host/containers systems.

  8. denisri commented on Sep 20, 2024

    @denisri
    Contributor

    While we wait for a cleaner solution, maybe we can just add those libs to the site config in the neurospin install of brainvisa ?

  9. denisri commented on Sep 20, 2024

    @denisri
    Contributor

    I have done that (modified /drf/brainvisa/brainvisa-cea-5.1.2/conf/casa_distro.json on Neurospin network filesystem) and it seems to work. It would probably break the install if run from an older system but I assume all machines in NS are at least in Ubuntu 22.04 (?)
    @ylep, can you test / confirm please ?

  10. ylep commented on Sep 20, 2024

    @ylep
    Member

    I did a quick test, the workaround seems to fix Ubuntu 24.04, unsurprisingly Ubuntu 22.04 still works. However, it seems to have broken Ubuntu 20.04 compatibility (we have few machines, but the graphical frontend of the cluster is one of them):

    /bin/sh: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /bin/sh)
    /bin/sh: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /bin/sh)
    
  11. denisri commented on Sep 20, 2024

    @denisri
    Contributor

    Aha, yes if there are still machines older than the image, then the fix will break them :(
    I didn't remember that there were also such systems on the network. OK I will remove the fix for now.

  12. denisri commented on Sep 20, 2024

    @denisri
    Contributor

    I have attempted a fix: brainvisa/casa-distro@c604093)

  13. denisri commented on Sep 24, 2024

    @denisri
    Contributor
  14. added a commit that references this issue on Jan 8, 2025
  15. ylep commented on Jan 8, 2025

    @ylep
    Member

    The issue was still present on NeuroSpin's workstations, so I have deployed Denis' workaround to the shared install on /drf/brainvisa.

    We need to decide how to proceed with that issue (new release?) because as of now, we cannot use BrainVISA on newly installed or updated NeuroSpin laptops; and external users may run into that issue as well...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions