llvm-elfv2 [FreeBSD WIKI]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
This page contained status information about efforts to bring LLVM to PowerPC64 target, using %%OpenPower%% ELF v2 ABI.  It is now obsolete.

:!: Flag day occurred on Dec 26, 2019, starting from r356111. -- [[/Bdragon28]] 2019-12-27 05:50:24 UTC

===== Converting an existing ELFv1 world to ELFv2 =====

:!: Reinstalling from the latest snapshot is **highly** preferable to doing a source upgrade! You have been warned! :!:

:!: I'm still in the middle of writing this, don't follow these instructions yet! -- [[/Bdragon28]] 2020-01-08 02:12:58 UTC

==== Step 1: Transition build ====

Ensure xtoolchain-llvm90 is installed.

Do an initial world build as a cross compile, and force a gcc 4.2.1-compatible bootstrap (Use the GPL DTC, and use prebuilt tblgen files to avoid building any C++11 code during bootstrap.)

<code>
make CROSS_TOOLCHAIN=llvm90 TARGET=powerpc TARGET_ARCH=powerpc64 \
WITHOUT_BOOT=yes WITHOUT_LIB32=yes \
LLVM_TBLGEN=/usr/local/llvm90/bin/llvm-tblgen \
CLANG_TBLGEN=/usr/local/llvm90/bin/clang-tblgen \
WITH_GPL_DTC=yes \
WITHOUT_GOOGLETEST=yes \
XCFLAGS="-mabi=elfv2 -mcmodel=medium --sysroot=/usr/obj/usr/src/powerpc.powerpc64/tmp/" \
XCXXFLAGS="-mabi=elfv2 -mcmodel=medium --sysroot=/usr/obj/usr/src/powerpc.powerpc64/tmp/" \
buildworld buildkernel KERNCONF=GENERIC64 -j32
</code>

Note: Build is currently failing still.

==== Step 2: Transition install ====

<code>
make installkernel KERNCONF=GENERIC64
</code>

reboot

At this point, you are on a post-flag-day compatible kernel which can execute post-flag-day ELFv2 binaries (This is necessary because of the auxv change.)

:!: DO NOT INSTALL THIS WORLD DIRECTLY! Make will crash during install and leave the system in a broken state! :!:

Prepare a bootable partition of some sort and do a temporary installworld to it.

<code>
newfs /dev/nda1p5
mount /dev/nda1p5 /mnt/newtarget
cp -a /etc /mnt/newtarget
make installkernel installworld DESTDIR=/mnt/newtarget
</code>

Reboot into the temporary volume somehow. (override vfs.root.mountfrom, etc.)

Use the temporary volume to install to your normal volume.

<code>
mkdir /mnt/target
mount (normal root device) /mnt/target
rmdir /usr/src
rmdir /usr/obj
ln -s /mnt/target/usr/src /usr/src
ln -s /mnt/target/usr/obj /usr/obj
cd /usr/src
make installworld DESTDIR=/mnt/target
</code>

At this point, you are in a cross elfv2 world. Rebuild the world and kernel natively.

==== Step 3: Final build ====

<code>
make buildworld buildkernel KERNCONF=GENERIC64 WITHOUT_SYSTEM_COMPILER=yes WITHOUT_SYSTEM_LINKER=yes -j32
</code>

==== Step 4: Final install ====

<code>
make installkernel
</code>

reboot again into final kernel

<code>
make installworld
</code>

===== Status Tracking for Flag Day =====

The table bellow sumarizes status of the hard dependencies for the flag day and duplicates information found on other sections of this document.
All items on this table must have status "accepted" or "commited" for the flagday be declared.

| Description | Target | URL | Status | Plan |
| makes LLVM default compiler | powerpc, powerpcspe, powerpc64 | [[https://reviews.freebsd.org/D20378|D20378]] | committed r356111 | commit on flag day |
| makes ELFv2 default ABI on FreeBSD13.0+ | powerpc64 | [[https://reviews.freebsd.org/D20383|D20383]] | committed r356112 | commit on flag day |
| Flag day parts of D22608 | powerpc, powerpcspe | [[https://reviews.freebsd.org/D22798|D22798]] | committed r356114 | commit on flag day |
| link LIB32 objects with GNU BFD | powerpc64 | [[https://reviews.freebsd.org/D20261|D20261]] | committed r356054 | commit BEFORE flag day |
| LLVM builtin atomics | powerpc, powerpcspe | [[https://reviews.freebsd.org/D22549|D22549]] | committed r356104 | commit BEFORE flag day |
| Switch to Secure-PLT PIC modules, allow MD code to participate in DYNAMIC parsing | powerpc, powerpcspe | [[https://reviews.freebsd.org/D22608|D22608]] | committed r356053 | commit BEFORE flag day |
| Fix predicate handling with SPE | powerpcspe | [[https://reviews.llvm.org/D69483|D69483]] | not accepted | 1. merge changes back to %%FreeBSD%% base code; 2. commit BEFORE flag |
| Relax the restrictions on loading doubles with SPE | powerpcspe | [[https://reviews.llvm.org/D69484|D69484]] | needs revision | 1. merge changes back to %%FreeBSD%% base code; 2. commit BEFORE flag day |
| Fix SPE f64 VAARG handling. | powerpcspe | [[https://reviews.llvm.org/D69486|D69486]] | needs revision | merge changes back to %%FreeBSD%% base code;commit BEFORE flag day |
| Only use PLT annotations if using PIC relocation model | powerpc, powerpcspe | [[https://reviews.llvm.org/D70570|D70570]] | committed, %%FreeBSD%% r356100 | 1. merge changes back to %%FreeBSD%% base code; 2. commit BEFORE flag |
| bump %%FreeBSD%% OSVERSION |  |  | TODO ASAP | to be published on flagday (by jhibbits) |

==== FreeBSD Ports ====

:!: Packages are **not** yet being built for this configuration; for now, you will have to build your own.  Expect to find failures as unlike the [[powerpc/ports|results with gcc]], few ports have really been tested.

:!: Use of the -CURRENT powerpc packages built on %%FreeBSD%%.org may or may not work (it has not been tested as of 20200108).

| Description | URL | Status |
| <del>Mk/bsd.port.mk: detect powerpc64 abi</del> | [[https://reviews.freebsd.org/D22039|D22039]] | committed r518658 |
| <del>llvm-devel on powerpc64: add configure option to select ELFv2 ABI</del> | [[https://reviews.freebsd.org/D22100|D22100]] | committed r515154 |
| <del>backport of LLVM 9 fixes in preparation for building %%FreeBSD%%/PowerPC64</del> | [[https://reviews.freebsd.org/D20375|D20375]] | abandoned - ports/llvm80 is at 8.0.1, which contains all important changes |

=== from ppcdevref ===

There is a [[powerpc/ports/PortsOnClang|more detailed ports summary]] native-built on an IBM Power8 machine (**ppcdevref**) on loan to us by OSU.

----

===== Patches =====

==== FreeBSD Base ====

| Description | URL | Status |
| <del>makes LLVM default on PowerPC64</del> | [[https://reviews.freebsd.org/D20378|D20378]] | commited r356111 |
| <del>LLVM - change default ABI for FreeBSD13.0+</del> | [[https://reviews.freebsd.org/D20383|D20383]] | commited  r356112 |
| fix incorrect clang target triple computation (Makefile.inc1) | [[https://reviews.freebsd.org/D19237|D19237]] | review |
| fix emulation name passed to linker when cross compiling | [[https://reviews.freebsd.org/D21800|D21800]] | review |
| <del>fix LIB32 build (Makefile.libcompat)</del> | [[https://reviews.freebsd.org/D20261|D20261]] | commited r356054 |
| <del>[%%PowerPC%%] enable atomic.c in compiler_rt and makes clang do not emit a call to an external __atomic_is_lock_free__</del> | [[https://reviews.freebsd.org/D22549|D22549]] | commited r356104 |
| <del>Enable TLS usage in system libraries on ELFv2</del> | [[https://reviews.freebsd.org/D22524|D22524]] | committed r355719 |
| <del>rtld: do not try to mmap a zero-sized PT_LOAD</del> | [[https://reviews.freebsd.org/D22634|D22634]] | committed |
| <del>Fix broken kernel modules due to LLD 9+ TOC optimization</del> | [[https://reviews.freebsd.org/D22317|D22317]] | committed r354713 |
| <del>[%%PowerPC%%] force applications linked with lib CSU to have .got</del> | [[https://bugs.llvm.org/show_bug.cgi?id=43148|43148]] [[https://reviews.freebsd.org/D21476|D21476]] | committed r353455 |
| <del>powerpc: Transition to Secure-PLT, like most other OSs (Toolchain part)</del> | [[https://reviews.freebsd.org/rS349351|rS349351]] | committed |
| <del>add ifunc support in libcsu</del> | [[https://reviews.freebsd.org/D21070|D21070]] | committed |
| <del>don't mark ld.bfd as obsolete</del> | [[https://reviews.freebsd.org/D21136|D21136]] | committed |
| <del>fix ifunc indirect call crash</del> | [[https://reviews.freebsd.org/D21102|D21102]] | committed |
| <del>LLVM - backport %%FreeBSD%%/PowerPC64 LLVM9 upstream changes</del> | [[https://reviews.freebsd.org/D20337|D20337]] | abandoned (after 8.0.1 was merged) |
| <del>fix missing powerpc64 in _LLVM_TARGET_FILT</del> | [[https://reviews.freebsd.org/D20756|D20756]] | committed |
| <del>readelf: print description for 'e_flags' in ELF header (ABI type)</del> | [[https://reviews.freebsd.org/D20782|D20782]] | committed |
| <del>add ABI flags to 'file' magic</del> | [[https://reviews.freebsd.org/D20771|D20771]])-- | committed |
| <del>fix panic when loading if_epair.ko built with modern compiler</del> | [[https://reviews.freebsd.org/D20461|D20461]] | committed r349377 |
| <del>fix kernel build with lld8</del> | [[https://reviews.freebsd.org/D19352|D19352]] | committed (r348112) |
| <del>fix stand/bootloader build</del> | [[https://reviews.freebsd.org/D20026|D20026]] | committed |
| <del>ship ld.bfd with base</del> | [[https://reviews.freebsd.org/D20259|D20259]] | committed |
| <del>fix wrong KASSERT in mphyp_pte_insert()</del> | [[https://reviews.freebsd.org/D20012|D20012]] | committed |
| <del>fix pseries-llan (QEMU)</del> | [[https://reviews.freebsd.org/D20008|D20008]] | committed |
| <del>silence cast-align warnings from clang on powerpc64</del> | [[https://reviews.freebsd.org/D18807|D18807]] | committed |
| <del>fix rtld-elf compilation warning for powerpc64 ELFv2 ABI</del> | [[https://reviews.freebsd.org/D18808|D18808]] | committed |
| <del>fix compilation error (contrib/binutils/opcodes/ppc-dis.c)</del> | [[https://reviews.freebsd.org/D19235|D19235]] | committed r344791 |
| <del>fix compilation error (include/pthread.h)</del> | [[https://reviews.freebsd.org/D19236|D19236]] | abandoned after test with r344922 |
| <del>increase ctfconvert buffer size</del> | [[https://reviews.freebsd.org/D19353|D19353]] | committed |
| <del>fix kldxref</del> | [[https://reviews.freebsd.org/D19370|D19370]] | committed |

==== LLVM 9.0 ====

| Description | URL | Status | 9.x backport |
| %%PowerPC%% 32-bit - forces 8 byte lock/lock_free decisions at compiled time | [[https://reviews.llvm.org/D71600|D71600]] | review |
| <del>change default ABI for FreeBSD13.0+</del> | [[https://reviews.llvm.org/D72306|D72306]] [[https://reviews.llvm.org/D72352|D72352]] | merged in LLVM10 (8edf759ca7e1), backported to 9.0.1 |  |
| crash when building graphics/mesa-dri | [[https://bugs.llvm.org/show_bug.cgi?id=44183|44183]] | Fixed in [[https://reviews.llvm.org/rG241cbf201a6f|LLVM10]] | merged in LLVM 9.0.1 (52ac914) |
| <del>Fix openmp on PowerPC64-BE-ELFv2 ABI on %%FreeBSD%%</del> | [[https://reviews.llvm.org/D67190|D67190]] | fixed | merged into %%FreeBSD%% base-LLVM9 (r352763). Merged in LLVM (3a76b8a538c0) |
| <del>[%%PowerPC%%] Do not emit HW loop if the body contains calls to lrint/lround</del> | [[https://bugs.llvm.org/show_bug.cgi?id=43527|43527]] | Fixed in LLVM 10 | Merged b9297dc [[https://bugs.llvm.org/show_bug.cgi?id=43967|43967]] |

| <del>[PPC32] Emit R_PPC_GOT_TPREL16 instead R_PPC_GOT_TPREL16_LO</del> | [[https://bugs.llvm.org/show_bug.cgi?id=43150|43150]] [[https://reviews.llvm.org/D66925|D66925]] | Fixed in LLVM 10 | <del>[[https://bugs.llvm.org/show_bug.cgi?id=43178|43178]]</del> 9.0.0 (r371059) |
| <del>powerpc 32 bit binary segfault due to secure-plt vs bss-plt conflict</del> | [[https://bugs.llvm.org/show_bug.cgi?id=43116|43116]] | Fixed in LLVM 9.0.0 | <del>[[https://bugs.llvm.org/show_bug.cgi?id=43148|43148]]</del> 9.0.0 (r370312, r370313) |

==== LLVM 8.0.0 ====
:!: This list of patch is for reference only. Efforts are now being focused on LLVM9

| Description | URL | Status | 8.x backport |
| change default ABI for FreeBSD13.0+ |  | to be released |  |
| [PPC32] Fix PLT calls for -msecure-plt -fpic | [[https://reviews.llvm.org/D63563|D63563]] | committed in LLVM9/head |  |
| <del>[PowerPC64] lld incorrectly optimizes ifunc TOC relocations</del> | [[https://bugs.llvm.org/show_bug.cgi?id=42759|42759]] | fixed in LLVM10/head | Issue doesn't occur on 8.x |
| <del>Adds ABI parsing when specified on target triple</del> | [[https://reviews.llvm.org/D61950|D61950]] | committed in LLVM9/head | <del>[[https://bugs.llvm.org/show_bug.cgi?id=41981|41981]]</del> unfeasible for 8.0.1 |
| <del>[libunwind] Fix r2 not properly restored</del> | [[https://bugs.llvm.org/show_bug.cgi?id=41050|41050]], [[https://reviews.llvm.org/D59694|D59694]] | committed in LLVM9/head | <del>[[https://bugs.llvm.org/show_bug.cgi?id=41942|41942]]</del> 8.0.1 (r360861 r364217) |
| <del>Consider localentry offset when computing branch distance</del> | [[https://reviews.llvm.org/D61058|D61058]] | committed in LLVM9/head | <del>[[https://bugs.llvm.org/show_bug.cgi?id=42332|42332]]</del> 8.0.1 (r359094 r364209) |
| <del>[PPC] Fix 32-bit build of libunwind</del> | [[https://reviews.llvm.org/D61792|D61792]] | committed in LLVM9/head | <del>[[https://bugs.llvm.org/show_bug.cgi?id=41941|41941]]</del> 8.0.1 (r360862 r363030) |
| <del>llvm-objdump: when ELF st_other field is set, print its value before symbol name</del> | <del>[[https://reviews.llvm.org/D61647|D61647]]</del> [[https://reviews.llvm.org/D61718|D61718]] | committed in LLVM9/head | <del>[[https://bugs.llvm.org/show_bug.cgi?id=41961|41961]]</del> 8.0.1 (r360439 r362668) |
| <del>Implement Thunk Section Spacing</del> | <del>[[https://reviews.llvm.org/D61610|D61610]]</del> [[https://reviews.llvm.org/D61720|D61720]] | committed in LLVM9/head | <del>[[https://bugs.llvm.org/show_bug.cgi?id=41959|41959]]</del> 8.0.1 (r353865 r353866 358975 r360405 r362043 r362273 r362274) |
| <del>Update %%LocalEntry%% from assigned symbols</del> | [[https://reviews.llvm.org/D56586|D56586]] | committed in LLVM9/head | <del>[[https://bugs.llvm.org/show_bug.cgi?id=41960|41960]]</del> 8.0.1 (r361237 r362671) |
| <del>Preserve %%LocalEntry%% when linking</del> | [[https://reviews.llvm.org/D56782|D56782]] | committed in LLVM9/head | <del>[[https://bugs.llvm.org/show_bug.cgi?id=41943|41943]]</del> 8.0.1 (r354184 r361921) |
| <del>Fix missing ELF st_other field on versioned symbols</del> | <del>[[https://bugs.llvm.org/show_bug.cgi?id=41048|41048]]</del>, [[https://reviews.llvm.org/D59436|D59436]] | committed in LLVM9/head | <del>[[https://bugs.llvm.org/show_bug.cgi?id=41968|41968]]</del> 8.0.1 (r360439 r360442 r362668 r362669) |

----

===== Pending Issues =====

List of currently known pending issues. This list assumes above patches are already applied.

==== FreeBSD base ====

| Description | URL | Workaround | Status |
| LIB32 C++ exception is broken when linked with LLVM LLD 9 | [[https://bugs.llvm.org/show_bug.cgi?id=43350|43350]] | Use GNU LD v2.17; with clang9, use -fPIC or -fpic | fixed in LLD 10 [[https://reviews.llvm.org/D73399|D73399]] |
| <del>LIB32 binaries are broken when linked with LLVM LLD 9</del> | [[https://bugs.llvm.org/show_bug.cgi?id=43116|43116]] | [[https://bugs.llvm.org/show_bug.cgi?id=43148|43148]] | fixed |
| <del>LIB32 binaries are broken when linked with LLVM LLD 8</del> |  | Use GNU LD v2.17 | abandoned since support was improved in LLVM9 and %%FreeBSD%% 13 will upgrade to LLVM9 |
| <del>slof loader is broken when linked with LLVM LLD 8</del> |  | Fixed in LLVM LLD | LLD %%PowerPC%% 32-bit support is being improved. See [[https://bugs.llvm.org/show_bug.cgi?id=40888|40888]] |
| <del>LIB32 build is incompatible with WITH_LLVM_LIBUNWIND=yes</del> |  | Fixed | See [[https://reviews.llvm.org/D61792|D61792]] |

==== FreeBSD base compiler ====

| Description | URL | Workaround | Status |
| <del>Integrated AS doesn't implement mulldo, addo and subfo instructions</del> | [[https://bugs.llvm.org/show_bug.cgi?id=42541|42541]] | use GNU AS 2.30+ | Merged in llvm 10.x Backport to 9.0.1 merge ed3f33f [[https://reviews.llvm.org/D66902|D66902]] |
| <del>Invalid PPC CTR loop when building node (clang900-import branch)</del> | [[https://bugs.llvm.org/show_bug.cgi?id=43527|43527]] |  | merged in %%FreeBSD%% base (r353358) |
| <del>enable %%OpenMP%% on powerpc64</del> | [[https://reviews.freebsd.org/D21532|D21532]] |  | fix committed r352763 (clang900-import) |
| <del>LLVM/lld large binaries (like clang itself) may get a wrong/corrupted .init after link</del> | [[https://bugs.llvm.org/show_bug.cgi?id=40740|40740]] | [[https://bugs.llvm.org/attachment.cgi?id=21690|21690]] | <del>[[https://reviews.llvm.org/D61610|D61610]]</del>[[https://reviews.llvm.org/D61720|D61720]] Fix available |
| <del>LLVM/lld doesn't implement dynamic relocations R_PPC_LOCAL24PC and R_PPC_GOT16 used by LIB32</del> | [[https://bugs.llvm.org/show_bug.cgi?id=40888|40888]] | link with GNU LD 2.17 | LLD %%PowerPC%% 32-bit support is being improved. See [[https://bugs.llvm.org/show_bug.cgi?id=40888|40888]] |

==== FreeBSD kernel ====

| Description | URL | Workaround | Status |
| <del>driver virtio drivers are not working when host is Little-Endian</del> | [[https://reviews.freebsd.org/D23401|D23401]] | <del>[[https://reviews.freebsd.org/D19815|D19815]]</del> <del>[[https://lists.nongnu.org/archive/html/qemu-devel/2020-01/msg01496.html|msg01496.html]]</del> | Fixed r357596 |
| <del>'kldload if_epair' causes kernel panic</del> | [[https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232387|232387]] [[https://reviews.freebsd.org/D20461|D20461]] |  | fixed r349377 |
| <del>[ppc][pseries] panic: Error evicting page: -7</del> | [[https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237470|237470]] |  | Fixed |

==== FreeBSD ports ====

| Description | URL | Workaround | Status |
| <del>cannot build rust package (firefox dependency)</del> | [[https://reviews.freebsd.org/D20792|D20792]] | see [[https://reviews.freebsd.org/D20792|D20792]] | fixed rP517954 |

==== Petitiboot - kexec (bare metal) ====

| Description | URL | Workaround | Status |
| <del>kexec can't boot ELFv2 %%FreeBSD%% kernel</del> | [[https://github.com/antonblanchard/kexec-lite/pull/9]] | On petitboot console, download and use http://drop.rtk0.net/kexec instead | committed, is in Talos II/Lite/Blackbird PNOR beta firmware. Only affects platforms using kexec-lite containing [[https://github.com/antonblanchard/kexec-lite/commit/87d044a895b1c004320a2676099a54a5a2a74f2e|this]] commit. Older versions of kexec-lite did not do the erroneous computation and always used the first address of .text as the entry point. There're reports of IBM Power9 and some IBM Power8 machines are affected. |

===== Contact =====

[[IRC/Channels|FreeBSD IRC Channels]]

----

{{tag>Historical}}