Repository navigation
Failed to build v1.17.0 on CentOS7 with default gcc (4.8) #14147
Copy link
Copy link
Closed
Labels
Milestone
Description
Cannot build numpy v1.17.0 on CentOS 7.
Previous version works fine.
I'm seeing a lot of C99/C11 features in error message, and avx512f.
Not sure if the required --std option is set but I don't think gcc 4.8.5 (the system default compiler on CentOS 7) can handle all of that.
Reproducing code example:
Simply build the wheel on CentOS 7 with python3.
Error message:
Some error log attached.
The entire story is too long to be pasted here.
#6 356.1 compile options: '-Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python3.6m -c'
#6 356.1 gcc: _configtest.c
#6 356.1 _configtest.c: In function 'main':
#6 356.1 _configtest.c:5:3: warning: implicit declaration of function '__builtin_mul_overflow' [-Wimplicit-function-declaration]
#6 356.1 __builtin_mul_overflow(5, 5, (int*)5);
#6 356.1 ^
#6 356.1 gcc -pthread _configtest.o -o _configtest
#6 356.1 _configtest.o: In function `main':
#6 356.1 /tmp/pip-req-build-poiltggx/_configtest.c:5: undefined reference to `__builtin_mul_overflow'
#6 356.1 collect2: error: ld returned 1 exit status
#6 356.1 failure.
#6 356.1 removing: _configtest.c _configtest.o _configtest.o.d
#6 356.1 C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC
#6 356.1
#6 356.1 compile options: '-Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python3.6m -c'
#6 356.1 gcc: _configtest.c
#6 356.1 _configtest.c: In function 'main':
#6 356.1 _configtest.c:7:16: warning: unused variable 'r' [-Wunused-variable]
#6 356.1 volatile int r = __builtin_cpu_supports("sse");
#6 356.1 ^
#6 356.1 gcc -pthread _configtest.o -o _configtest
#6 356.1 success!
#6 356.1 removing: _configtest.c _configtest.o _configtest.o.d _configtest
#6 356.1 C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC
#6 356.1
#6 356.1 compile options: '-Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python3.6m -c'
#6 356.1 gcc: _configtest.c
#6 356.1 _configtest.c: In function 'main':
#6 356.1 _configtest.c:7:3: error: Parameter to builtin not valid: avx512f
#6 356.1 volatile int r = __builtin_cpu_supports("avx512f");
#6 356.1 ^
#6 356.1 _configtest.c:7:16: warning: unused variable 'r' [-Wunused-variable]
#6 356.1 volatile int r = __builtin_cpu_supports("avx512f");
#6 356.1 ^
#6 356.1 failure.
#6 356.1 removing: _configtest.c _configtest.o
#6 356.1 C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC
#6 356.1
#6 356.1 compile options: '-Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python3.6m -c'
#6 356.1 gcc: _configtest.c
#6 356.1 gcc -pthread _configtest.o -o _configtest
#6 356.1 success!
#6 356.1 removing: _configtest.c _configtest.o _configtest.o.d _configtest
#6 356.1 C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC
#6 356.1
#6 356.1 compile options: '-Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python3.6m -c'
#6 356.1 gcc: _configtest.c
#6 356.1 gcc -pthread _configtest.o -o _configtest
#6 356.1 success!
#6 356.1 removing: _configtest.c _configtest.o _configtest.o.d _configtest
#6 358.3 C compiler: gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC
#6 358.3
#6 358.3 compile options: '-Ibuild/src.linux-x86_64-3.6/numpy/core/src/common -Inumpy/core/include -Ibuild/src.linux-x86_64-3.6/numpy/core/include/numpy -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python3.6m -Ibuild/src.linux-x86_64-3.6/numpy/core/src/common -Ibuild/src.linux-x86_64-3.6/numpy/core/src/npymath -Ibuild/src.linux-x86_64-3.6/numpy/core/src/common -Ibuild/src.linux-x86_64-3.6/numpy/core/src/npymath -c'
#6 358.3 gcc: build/src.linux-x86_64-3.6/numpy/core/src/npysort/radixsort.c
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'radixsort_bool':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:112:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src:112:5: note: use option -std=c99 or -std=gnu99 to compile your code
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'aradixsort_bool':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:202:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'radixsort_byte':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:112:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'aradixsort_byte':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:202:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'radixsort_ubyte':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:112:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'aradixsort_ubyte':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:202:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'radixsort_short':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:112:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'aradixsort_short':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:202:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'radixsort_ushort':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:112:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'aradixsort_ushort':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:202:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'radixsort_int':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:112:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'aradixsort_int':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:202:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'radixsort_uint':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:112:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'aradixsort_uint':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:202:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'radixsort_long':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:112:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'aradixsort_long':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:202:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'radixsort_ulong':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:112:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'aradixsort_ulong':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:202:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'radixsort_longlong':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:112:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'aradixsort_longlong':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:202:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'radixsort_ulonglong':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:112:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 numpy/core/src/npysort/radixsort.c.src: In function 'aradixsort_ulonglong':
#6 358.3 numpy/core/src/npysort/radixsort.c.src:202:5: error: 'for' loop initial declarations are only allowed in C99 mode
#6 358.3 for (npy_intp i = 1; i < num; i++) {
#6 358.3 ^
#6 358.3 error: Command "gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -Ibuild/src.linux-x86_64-3.6/numpy/core/src/common -Inumpy/core/include -Ibuild/src.linux-x86_64-3.6/numpy/core/include/numpy -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/include/python3.6m -Ibuild/src.linux-x86_64-3.6/numpy/core/src/common -Ibuild/src.linux-x86_64-3.6/numpy/core/src/npymath -Ibuild/src.linux-x86_64-3.6/numpy/core/src/common -Ibuild/src.linux-x86_64-3.6/numpy/core/src/npymath -c build/src.linux-x86_64-3.6/numpy/core/src/npysort/radixsort.c -o build/temp.linux-x86_64-3.6/build/src.linux-x86_64-3.6/numpy/core/src/npysort/radixsort.o -MMD -MF build/temp.linux-x86_64-3.6/build/src.linux-x86_64-3.6/numpy/core/src/npysort/radixsort.o.d" failed with exit status 1
#6 358.3 ----------------------------------------
#6 358.4 ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-poiltggx/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-poiltggx/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-50nzprh1/install-record.txt --single-version-externally-managed --compile Check the logs for full command output.
Numpy/Python version information:
CentOS 7
Default python 3 (3.6)
Default gcc (4.8)
Reactions are currently unavailable
Activity
xkszltl commentedon Jul 29, 2019
Full log attached: numpy.log
eric-wieser commentedon Jul 29, 2019
Numpy 1.17 needs at least
--std=c99.I think the decision was to leave passing that flag up to the distributor rather than doing it automatically, but I forget the rationale. @rgommers, were you part of that discussion?
xkszltl commentedon Jul 29, 2019
That sounds terrible if it's not set by default.
However, it somehow works with Ubuntu 18.04.
Besides, what's the correct way to pass in compiler flags?
I tried
export CC="gcc --std=c11"andexport CFLAGS="--std=c11"but both were ignored.Try numpy wheel due to C99 flag requirement in v1.17.0.
xkszltl commentedon Jul 29, 2019
Looks like on Ubuntu 18.04, gcc 7.4.0 defaults to
gnu11while on CentOS 7, gcc 4.8.5 defaults tognu90.I run
. scl_source enable devtoolset-8to switch to gcc 8, which usesgnu17by default.But seems all related env vars were ignored by numpy.
xkszltl commentedon Jul 29, 2019
Never mind, numpy can pick up the env correctly.
There's a sudo before pip install which drops everything.
But the issue remains: Do we still have the default support for CentOS 7 and its gcc 4.8.5?
paulofbmarcon commentedon Jul 31, 2019
I don't know if it's related but I got a similar problem trying to install on Amazon Linux. The following work around helped me:
update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc48-c99 100 --slave /usr/bin/c89 c89 /usr/bin/gcc48-c89 --slave /usr/bin/c99 c99 /usr/bin/gcc48-c99 --slave /usr/bin/cc cc /usr/bin/gcc48 --slave /usr/bin/gcov gcov /usr/bin/gcov48update-alternatives --config gccsashkab commentedon Jul 31, 2019
@xkszltl You should pass
--std=c99in theCFLAGSenvironmental variable. Following solved build issues for me.xkszltl commentedon Aug 1, 2019
@sashkab I got it resolved for me by simply using gcc-8. But should
--std=c99be added automatically during build if there's no user setting detected?antocuni commentedon Aug 1, 2019
numpy 1.17 is also failing to build on pypy, using a ubuntu 14.04 image: https://travis-ci.org/antocuni/pypy-wheels/jobs/566563056
it seems to be that numpy should try harder to compile even on older systems, unless you have a good reason not to.
tylerjereddy commentedon Aug 5, 2019
I think passing in the std flags by default in a compiler and platform-agnostic manner can be tricky & as with many distutils things, hard to test completely.
rgommers commentedon Aug 5, 2019
That rings a bell. I attempted to fix this before, and concluded there's no good place to insert the fix: #12620 (comment). This attempt has some comment as to what the problem is: rgommers@8aaf4d8
Taking a step back, the
numpy.distutilsdesign problem we run into here is that all flags are (a) taken over from Pythondistutils, then hardcoded per compiler innumpy/distutils/<compilername>.py. And at the time we do that hardcoding, we don't know what comes fromdistutils. And inserting flags dynamically later on (what I tried) is equally tricky.It's possible I missed a sane way to fix this. Suggestions/PRs welcome.
IIRC the minimum gcc version that works for building numpy at all is 4.8. For gcc <5.1, users need to add
--std=c99.This is a duplicate of gh-12766.
rgommers commentedon Aug 6, 2019
This is biting enough people that I'll mark it for 1.17.1 - should be possible to insert a hacky fix somehow.
11 remaining items
mattip commentedon Nov 11, 2019
Fixed in gh-14771, which added both the flag and a CI run with gcc4.8
sh1ng commentedon Mar 18, 2020
The issue is reproducible on
nvidia/cuda-ppc64le:10.0-cudnn7-devel-centos7mattip commentedon Mar 18, 2020
@sh1ng with what version of NumPy? Are you adding the -std=c99 flag?
sh1ng commentedon Mar 18, 2020
Without adding the flag
Adding it do solves the issue, but makes the installation process platform dependent.
sh1ng commentedon Mar 18, 2020
@mattip the issue occurs when miniconda is used, not a centos python package. Most likely conda's issue.
Configure (numpy): added --std=c99 to CFLAGS to fix error
zoj613 commentedon Aug 3, 2021
So, which 1.17.* version is this flag automatically set so 1.17.* build can be platform independent? I am having trouble building numpy 1.17.0 on 3.8 and 3.9 for CI jobs.
sashkab commentedon Aug 3, 2021
@zoj613 did you try my suggestion above with python 3.8? It worked for me for both 1.16.x and 1.17.x on RHEL/Centos 7.
charris commentedon Aug 3, 2021
@zoj613 It isn't automatically set in any 1.17 version, you need 1.18.0+ for that.
zoj613 commentedon Aug 3, 2021
Ah, Okay. @sashkab 's suggestion seems to work fine on 1.17.3
bump numpy verison see: numpy/numpy#14147
bump numpy verison see: numpy/numpy#14147
bump numpy verison see: numpy/numpy#14147
zevaverbach commentedon Jun 8, 2022
just to build on this, for anyone that's lost for a next step:
pip install <the_wheel_the_above_step_creates>.whl