Uploaded image for project: 'Funtoo Linux'
  1. Funtoo Linux
  2. FL-10920

New eclass "memsaver"

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Severe (Users) Severe (Users)
    • None
    • None
    • Impacts packages that will fail to build in systems with bad balance between the number of cores (and the corresponding optimal MAKEOPTS -jobs setting) and the available memory.

      There is a number of packages that will fail to build on systems where there is not the right balance between the number of physical cores and the amount of memory available.

      Usually these are large packages and the failure happens in the linking phase, but it can also happen in smaller libraries. Rust packages, in particular, are notorious for this kind of issue.

      When such a failure occurs, often the error message is not clear, or could look just wrong. For example, if I have 24 Gb o memory, in theory that should enough to link even the largest packages we have. But still you'll get an "out of memory" error without any clue as to why it's reporting that.

      Worse than that, you could be greeted with “The futex facility returned an unexpected error code.” And it won't even tell you what code was that!

      Those situations can be automatically dealt with by calculating the maximum possible number of jobs for the available memory and setting MAKEOPTS or NINJAOPTS accordingly. A piece of code that does just that exists in our current chromium-91.0.4472.106.ebuild. That code can serve as a reference for the development of something more robust, possibly without even depending on lscpu and awk and made into an eclass, which can then be easily inherited by any ebuilds that suffer from this problem.

      Besides chromium, I guess there is at least one more package that uses code like that to limit parallelism. And I have so far identified 2 more that need it:

      • media-video/rav1e
      • sys-devel/binutils
      • gnome-base/librsvg

            Unassigned Unassigned
            adbosco adbosco
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: