-
Bug
-
Resolution: Fixed
-
Normal
-
None
-
None
-
None
During 1.4 transition, emerge would remain stalled on building catpkg. This process would remain unfinished for hours. Observation was an extremely depressed load average being made on CPU. (6-core AMD Bulldozer w/ persistent load of
avgs 0.5 0.35 0.35 observed).
Review of log of catpkg found that upstream ([source:|https://github.com/python-pillow/Pillow/blob/master/setup.py] detects and limits ninja build. Unlike make, my understanding is that ninja is very sensitive to build order and load. In my case, log file indicated that ninja build was limited to four threads(line 393 and others).
In researching the lethargy, a portage related problem became apparent. catpkg build process is further throttled by ebuild/eclass.
catpkg ebuild is EAPI=6 with python 3.7. Ebuild calls the function distutils-r1_python_compile which then further throttles build based on EAPI and python version.
From file:
if python_is_python3 && [[ ${EPYTHON} != python3.4 ]]; then if [[ ${EAPI} != [56] || ${EPYTHON} != python3.[56] ]]; then local jobs=$(makeopts_jobs "${MAKEOPTS}" INF) if [[ ${jobs} == INF ]]; then local nproc=$(get_nproc) jobs=$(( nproc + 1 )) fi build_args+=( -j "${jobs}" ) fi fi
This code from the distutils-r1.eclass, in conjunction with the catpkg ebuild, unnecessarily throttles the build process that upstream has regulated. The MAKEOPT setting for core usage, -j, is not passed to the package.
Change is to move ebuild EAPI to 7 to avoid implications of distutils-r1 function shown above. Also, clean of unnecessary patch files was undertaken. The lethargy of build is somewhat addressed but more needs to be studied to ascertain instances where upstream using ninja regulates CPU usage.
PR to follow.