====== ARM64 ======

%%FreeBSD%%/arm64 is the %%FreeBSD%% port to the 64-bit ARM architecture, also known as AArch64 or ARMv8.  All [[https://www.freebsd.org/security/#sup|supported FreeBSD releases]] include out-of-the-box support for ARMv8, and over 20,000 packages are available in the 3rd party software collection.
Ampere eMAG and Altra platforms are the primary reference platforms for %%FreeBSD%%/arm64.
arm64 is classified as a [[https://docs.freebsd.org/en/articles/committers-guide/#archs|Tier-1 architecture]] on %%FreeBSD%% as of %%FreeBSD%% 13.0-RELEASE.  Before that it was classified as tier-2.

[[arm64/kyua|Kyua tracking.]]

====== Targets ======

%%FreeBSD%% Release Engineering builds specific images for [[https://cgit.freebsd.org/src/tree/release/arm64|arm64]] including Pine64 and RPi3 & 4.  The Cavium Thunder and %%SoftIron%% Overdrive series are supported via the normal build chain.  Images for certain other boards can be built by [[https://github.com/freebsd/crochet/tree/master/board|crochet]].

====== Source ======

The basic %%FreeBSD%%/arm64 support is available in %%FreeBSD%%-CURRENT, FREEBSD-13-STABLE, FREEBSD-12-STABLE, and FREEBSD-11-STABLE.  (For reference, also see the [[arm64/history|history of FreeBSD/arm64 commits]]).

main (CURRENT) build status:      [[https://ci.freebsd.org/job/FreeBSD-main-aarch64-build/|[[https://ci.freebsd.org/job/FreeBSD-main-aarch64-build/lastCompletedBuild/badge/icon  ]]

stable/13 (13.x-STABLE) build status: [[https://ci.freebsd.org/job/FreeBSD-stable-13-aarch64-build/|[[https://ci.freebsd.org/job/FreeBSD-stable-13-aarch64-build/lastCompletedBuild/badge/icon  ]]

stable/12 (12.x-STABLE) build status: [[https://ci.freebsd.org/job/FreeBSD-stable-12-aarch64-build/|[[https://ci.freebsd.org/job/FreeBSD-stable-12-aarch64-build/lastCompletedBuild/badge/icon  ]]

====== Building ======

===== World =====

The standard buildworld sequence works:

<code>
export MAKEOBJDIRPREFIX=<path-to-suitable-objdir>
make buildworld TARGET=arm64
</code>

===== Kernel =====

The build sequence for the kernel is a fairly standard one:

<code>
export MAKEOBJDIRPREFIX=<path-to-suitable-objdir>
make buildkernel TARGET=arm64
</code>

This results in an image called 'kernel' created in:

<code>
$MAKEOBJDIRPREFIX/arm64.aarch64/<svn-checkout-path**>/sys/GENERIC
</code>

%%**%% See the 'Source' section above for the svn-checkout-path.

====== Running ======

===== Emulation =====

==== ARM Foundation Model ====

The ARMv8 [[http://www.arm.com/products/tools/models/fast-models/foundation-model.php|Foundation Model]] is available from ARM.  You must read and accept a click-through license to obtain it.  The Foundation model may be run in a Linux virtual machine, or by using the experimental 64-bit Linux emulation support.

AndrewTurner has a [[https://github.com/zxombie/fvp-freebsd|GitHub repo]] set up to help run on the Foundation Model.

==== QEMU ====

QEMU now has aarch64 system mode emulation support. It is available by installing the qemu-devel package or emulators/qemu-devel port.

The Release Engineers build periodic snapshots, including for arm64 on QEMU. Instructions on where to find these are sent to the [[https://lists.freebsd.org/mailman/listinfo/freebsd-snapshots|snapshots mailing list]].

See [[arm64/QEMU]] for further details on booting %%FreeBSD%%/arm64 in QEMU, including links to AArch64 EFI firmware files that are not included with the QEMU port.

===== Single Boards =====

==== ESPRESSObin ====

[[arm64/ESPRESSObin]]

==== Gem5 ====

[[gem5|Gem5 instructions]]

==== HiKey ====

The [[arm64/HiKey|HiKey]] is a low cost (~$75 - $100 + shipping) ARMv8 board from Linaro.

==== MACCHIATObin ====

The [[http://macchiatobin.net/MACCHIATObin]] series of boards is supported.

As noted by Greg at unrelenting.technology:

<code>
affordable 4-core (Cortex A72) with DDR4 (takes one full size DIMM), SATA, USB 3.0 and PCIe.
There is excellent firmware support (upstream EDK2+!TrustedFirmware).  However, an affordable 4-core (Cortex A72)
with DDR4 (takes one full size DIMM), SATA, USB 3.0 and PCIe.

However, the PCIe is rather quirky.
</code>

TODO cite email from Mark Millard when it shows up in the archives

==== Allwinner based boards ====

See main Allwinner page at [[arm/Allwinner]]

==== Rockchip based boards ====

This section has moved to [[arm/RockChip]]

==== Raspberry Pi 3 + 4 ( RPI3 + RPI4 + RPI400 ) ====

This section has moved to [[arm/Raspberry Pi]]

===== Complete Systems =====

==== Cavium ThunderX ====

Cavium's %%ThunderX%% is the initial reference target platform for %%FreeBSD%%/arm64.

%%FreeBSD%% supports the %%ThunderX%% EVB (evaluation board) and CRB (customer reference board) in SMP mode (48 CPU cores).
SATA drives, PCIe expansion cards, and the on-chip network interface are fully supported.

%%FreeBSD%% is available on the 2 socket, 96 core [[https://www.packet.net/bare-metal/servers/type-2a/|Type 2A]] %%ThunderX%% systems at [[https://packet.net|Packet.net]].

[[http://youtu.be/lLgc4FJLJ3Y|Demo of SMP kernel on ThunderX]]

==== HoneyComb ====

Although the [[https://www.solid-run.com/arm-servers-networking-platforms/honeycomb-workstation|HoneyComb]] from [[http://www.solid-run.com|SolidRun]] is available as a single board, it is more targeted at the workstation market.

(See [[https://lists.freebsd.org/archives/freebsd-arm/2021-June/000199.html|this mailing list thread]] for current %%FreeBSD%% developer notes.)

==== Lenovo ThinkSystem SR330/SR350 ====

These servers are based on the [[https://amperecomputing.com/product/|Ampere]] [[https://amperecomputing.com/wp-content/uploads/2019/01/eMAG8180_PB_v0.5_20180914.pdf|eMAG chipset]].  There is [[https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=237055|ongoing active work]] to support them.
A number of %%FreeBSD%% developers have these systems, and they are well supported from 13.0-RELEASE onwards.

See [[arm/Ampere]] for more details.

  * [[https://amperecomputing.com/wp-content/uploads/2019/04/Lenovo_ThinkSystem_HR330A_PB_20190409.pdf|SR330]]
  * [[https://amperecomputing.com/wp-content/uploads/2019/04/Lenovo_ThinkSystem_HR350A_20190409.pdf|SR350]]

==== SoftIron Overdrive 1000 ====

The %%FreeBSD%% 12.0R GENERIC (and later) memstick installation images work on the [[https://shop.softiron.com/product/overdrive-1000/|SoftIron Overdrive 1000]] but requires ''%%opensolaris_load="YES"%%'' to be set in ''%%/boot/loader.conf%%''. This can be performed at the end of installation or on first boot by typing ''%%ESC%%'' during the (likely garbled) "Beastie" boot screen, and typing:

<code>
load /boot/kernel/kernel
load /boot/kernel/opensolaris.ko
load /boot/kernel/zfs.ko
boot
</code>

Otherwise, the first boot will hit ''%%mountroot>%%''

Once booted, add ''%%"opensolaris_load="YES"%%'' to ''%%/boot/loader.conf%%'' with the editor of your choice.

To suppress the garbled Beastie boot screen, add ''%%beastie_disable="YES"%%'' to ''%%/boot/loader.conf%%''

Other notes:

The power button is the vertical chrome rectangle on the front.

Be cautious not to frequently cut power to the OD1K as it may corrupt the firmware.

==== SoftIron Overdrive 3000 ====

%%FreeBSD%% mostly works, other than the internal dual 10G networking, however support for this is being worked on. In the interim PCIe NICs work.

===== Cloud Services =====

==== AWS ====
[[arm64/Cloud_Services]]

====== Screenshot ======

{{arm64/arm64_userland.png}}

====== Useful Links ======
  * [[http://infocenter.arm.com/help/topic/com.arm.doc.ihi0055b/IHI0055B_aapcs64.pdf|Procedure call standard]]
  * [[http://people.linaro.org/~riku.voipio/aarch64-talk/|Porting Applications to 64bit ARM]] slides written by Riku Voipio. 

====== Known Limitations ======
  * Userland SDT DTrace probes (usdt) do not work
  * The Cavium %%ThunderX%% vnic driver has several issues, but it's generally functional.  See PRs:
    * [[https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204414|204414]]
    * [[https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204415|204415]]
    * [[https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=206310|206310]]
    * [[https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=223573|223573]]
    * [[https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=223575|223575]]
  * libc does not provide sbrk and some ports have not been updated to cope.
  * Much profiling and performance work is still needed

===== Bugs =====
Search for open PRs with [[https://bugs.freebsd.org/bugzilla/buglist.cgi?bug_status=New&bug_status=Open&bug_status=In%20Progress&list_id=130067&query_format=advanced&rep_platform=arm64|Hardware = arm64]].

====== Ports and Packages ======

  * see [[arm64/ports|the state of ports/packages on arm64]].
  * arm64 port failures are tracked in [[https://bugs.freebsd.org/201763|PR 201763]].