Error: unknown architectural extension 'sve2-aes' when compiling on Graviton3

[fbarbari]

Steps to reproduce:

git clone https://github.com/google/highway.git
cd highway
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cd build
make all

Commit SHA: 326ac6c
Cmake version: 3.26.3
GCC version: 13.1.0

Output of lscpu:

Architecture:        aarch64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              4
On-line CPU(s) list: 0-3
Thread(s) per core:  1
Core(s) per socket:  4
Socket(s):           1
NUMA node(s):        1
Vendor ID:           ARM
Model:               1
Stepping:            r1p1
BogoMIPS:            2100.00
L1d cache:           64K
L1i cache:           64K
L2 cache:            1024K
L3 cache:            32768K
NUMA node0 CPU(s):   0-3
Flags:               fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm ssbs paca pacg dcpodp svei8mm svebf16 i8mm bf16 dgh rng

[jan-wassenberg]

Hi, thanks for reporting this.
Interestingly, in Godbolt GCC 13.1 seems to work fine: https://gcc.godbolt.org/z/5jfdfn7ab

Does anyone know if Graviton3 ships with a GCC tailored to what their core can do? If so, is there any way to detect this?

[fbarbari]

Oddly enough when compiling with -O3 -DNDEBUG -march=armv9-a+sve2-aes I get unrecognized option -march=armv9-a+sve2-aes. Probably, the problem is in the compiler I am using.

Update:
The GCC I was using was installed through spack, which shouldn't be a problem, but if I run g++ -O3 -DNDEBUG -march=native -mtune=native -Q -v -E - < /dev/null to see what architecture is detected I get:

...
COLLECT_GCC_OPTIONS='-O3' '-D' 'NDEBUG'   '-Q' '-v' '-E' '-shared-libgcc' '-mlittle-endian' '-mabi=lp64' '-mtune=zeus'
...

which is correct but there is no flag for the architecture, only the micro-arch.

If I run the system's GCC (7.3.1) with the same command /usr/bin/g++ -O3 -DNDEBUG -march=native -mtune=native -Q -v -E - < /dev/null, I get:

...
COLLECT_GCC_OPTIONS='-O3' '-D' 'NDEBUG'   '-Q' '-v' '-E' '-shared-libgcc' '-mlittle-endian' '-mabi=lp64' '-march=armv8.4-a+crypto+fp16' '-mtune=zeus'
...

which is a lot more promising and also makes highway compile.

[jan-wassenberg]

Nice. It's probably helpful to use a newer GCC than 7.3, so perhaps one good way forward is to build GCC from source, or is there some other package repo available?

[fbarbari]

I just tried with a freshly compiled gcc 13.2.0 and the problem still persists. I will try with clang in the next hours and update you.

[jan-wassenberg]

hm, strange. Is there something about the AWS config that causes compilers built there to be configured for only that CPU?
The above godbolt does work as expected also for gcc 13.2.

[Xarbirus]

Just got the same problem with gcc-11.4 building gemma.cpp

[jan-wassenberg]

Thanks, gcc-11.4 is a different issue. That compiler is too old to know about SVE2 and we should not be attempting to use it. I will update Highway to fix this. In the meantime, -DHWY_DISABLED_TARGETS=(HWY_SVE2|HWY_SVE2_128) should work around this.

[Xarbirus]

@jan-wassenberg thank you for the advice, it helped, this problem disappeared!
But the following appeared:

Assembler messages:
Error: unknown pseudo-op: `.variant_pcs'

I think this problem may be due to the fact that I'm trying to build the library on Ubuntu 18.04 with binutils 2.30. This idea came from here. Wdyt?

[jan-wassenberg]

Sounds like progress :) I think support was added for that in GAS in 2019. Upgrading binutils sounds good!