| Both sides previous revision Previous revision Next revision | Previous revision |
| linuxulator [2021/03/28 08:15] – Debugging and Testing are a part of Development, make headings so indented KubilayKocak | linuxulator [2023/01/29 14:30] (current) – man.freebsd.org GrahamPerrin |
|---|
| | ===== Linuxulator (Linux Emulation) ===== |
| | |
| Linux binary compatibility, commonly referred to as Linuxulator, is a mechanism to run unmodified Linux binaries under %%FreeBSD%%. It does not involve virtual machines or emulation; instead, it provides the binaries with kernel interfaces identical to those provided by a real Linux kernel. Technically it's similar to the way 32-bit %%FreeBSD%% binaries run on top of 64-bit %%FreeBSD%% kernel. | Linux binary compatibility, commonly referred to as Linuxulator, is a mechanism to run unmodified Linux binaries under %%FreeBSD%%. It does not involve virtual machines or emulation; instead, it provides the binaries with kernel interfaces identical to those provided by a real Linux kernel. Technically it's similar to the way 32-bit %%FreeBSD%% binaries run on top of 64-bit %%FreeBSD%% kernel. |
| |
| There are two main ways to use Linuxulator. First, documented in the [[https://www.freebsd.org/doc/handbook/linuxemu.html|FreeBSD Handbook]], uses Linux applications provided as %%FreeBSD%% ports and packages, to be installed using [[https://www.freebsd.org/cgi/man.cgi?pkg|pkg(8)]]. Second, described at [[LinuxJails]], provides a full Debian/Ubuntu userspace. The former is more user-friendly, but provides a limited number of available applications. The latter is more cumbersome, but is handy when doing development work. Both can be used at the same time. | There are two main ways to use Linuxulator. First, documented in the [[https://docs.freebsd.org/en/books/handbook/linuxemu|FreeBSD Handbook]], uses Linux applications provided as %%FreeBSD%% ports and packages, to be installed using [[https://man.freebsd.org/cgi/man.cgi?pkg|pkg(8)]]. Second, described at [[LinuxJails]], provides a full Debian/Ubuntu userspace. The former is more user-friendly, but provides a limited number of available applications. The latter method is more involved, but is handy when doing development work. Both can be used at the same time. |
| |
| ===== Status ===== | ===== Status ===== |
| |
| We currently claim compatibility with Linux 3.2.0 in 12-STABLE and 3.17.0 in 13-CURRENT. | We currently claim compatibility with Linux 3.2.0 in 12-STABLE and 4.4.0 in 14-CURRENT. |
| |
| The list of currently tested applications can be found at [[LinuxApps]]. For Steam, we suggest using [[https://www.freshports.org/games/linux-steam-utils/|linux-steam-utils]] package. The status of tested games can be found [[https://github.com/shkhln/linuxulator-steam-utils/wiki/Compatibility|here]]. Instructions on how to watch Netflix can be found in [[https://forums.freebsd.org/threads/linuxulator-how-to-run-google-chrome-linux-binary-on-freebsd.77559/|this forum post]]. | The list of currently tested applications can be found at [[LinuxApps]]. For Steam, we suggest using [[https://www.freshports.org/games/linux-steam-utils/|linux-steam-utils]] package. The status of tested games can be found [[https://github.com/shkhln/linuxulator-steam-utils/wiki/Compatibility|here]]. Instructions on how to watch Netflix can be found in [[https://forums.freebsd.org/threads/linuxulator-how-to-run-google-chrome-linux-binary-on-freebsd.77559/|this forum post]]. |
| The goal is to implement as many Linux syscalls as possible, except for ones that are obsolete or only used for management purposes. | The goal is to implement as many Linux syscalls as possible, except for ones that are obsolete or only used for management purposes. |
| |
| The canonical list of unimplemented syscalls can be found at [[https://svnweb.freebsd.org/base/head/sys/amd64/linux/linux_dummy.c?view=markup|sys/amd64/linux/linux_dummy.c]]. Those marked UNIMPLEMENTED() are not supposed to be ever supported, usually because they have been [[http://www.man7.org/linux/man-pages/man2/unimplemented.2.html|obsoleted upstream]]. A somewhat stale list can be found at [[LinuxulatorMissing]]. | The canonical list of unimplemented syscalls can be found at [[https://cgit.freebsd.org/src/tree/sys/compat/linux/linux_dummy.c|sys/compat/linux/linux_dummy.c]]. Those marked UNIMPLEMENTED() are not supposed to be ever supported, usually because they have been [[http://www.man7.org/linux/man-pages/man2/unimplemented.2.html|obsoleted upstream]]. A somewhat stale list can be found at [[LinuxulatorMissing]]. |
| |
| The [[http://www.kernel.org/doc/man-pages/online/pages/man2/syscalls.2.html|Linux syscalls(2) mange page]] lists all syscalls including references to the Linux kernel version each syscall first appeared. Note that in Linux, there are differences in syscalls supported by each architecture, there's a [[https://fedora.juszkiewicz.com.pl/syscalls.html|website tracking this]]. | The [[http://www.kernel.org/doc/man-pages/online/pages/man2/syscalls.2.html|Linux syscalls(2) man page]] lists all syscalls including references to the Linux kernel version each syscall first appeared. Note that in Linux, there are differences in syscalls supported by each architecture, there's a [[https://fedora.juszkiewicz.com.pl/syscalls.html|website tracking this]]. |
| |
| ==== Testing ==== | ==== Testing ==== |
| |
| * [[http://ltp.sf.net|Linux Test Project]]: README and INSTALL are straight forward, don't forget to mount devfs, linprocfs and linsysfs to the linux-chroot where you compile/test | * [[https://github.com/linux-test-project/ltp|Linux Test Project]]: README and INSTALL are straightforward, don't forget to mount devfs, linprocfs and linsysfs to the linux-chroot where you compile/test |
| * [[http://www.Leidinger.net/FreeBSD/linuxolator/|GNU libc nptl regression test]]: not as easy as LTP, but doable; precompiled stuff available | * [[http://www.Leidinger.net/FreeBSD/linuxolator/|GNU libc nptl regression test]]: not as easy as LTP, but doable; precompiled stuff available |
| * [[http://people.freebsd.org/~jkim/mmap_test.c|mmap fingerprinter]]: little tool by [[MarcinCieslak|Marcin Cieslak]], extended by [[jkim@FreeBSD.org|Jung-uk Kim]] to determine the fingerprint of the mmap() call. Could be used to extend the %%FreeBSD%% regression tests too. Note: Marcin confirmed that mmap_test.c is public domain. | * [[http://people.freebsd.org/~jkim/mmap_test.c|mmap fingerprinter]]: little tool by [[MarcinCieslak|Marcin Cieslak]], extended by [[jkim@FreeBSD.org|Jung-uk Kim]] to determine the fingerprint of the mmap() call. Could be used to extend the %%FreeBSD%% regression tests too. Note: Marcin confirmed that mmap_test.c is public domain. |
| * [[http://git.kernel.org/?p=linux/kernel/git/dvhart/futextest.git|Darren Hart's futex test suite]]: this suite will run various futex related functionality, performance and stress tests. | * [[http://git.kernel.org/?p=linux/kernel/git/dvhart/futextest.git|Darren Hart's futex test suite]]: this suite will run various futex related functionality, performance and stress tests. |
| * [[https://github.com/kernelslacker/trinity|Trinity: a Linux system call fuzz tester]]: a package to call syscalls at random, with random arguments, to find eventual vulnerabilites and bugs. | * [[https://github.com/kernelslacker/trinity|Trinity: a Linux system call fuzz tester]]: a package to call syscalls at random, with random arguments, to find eventual vulnerabilities and bugs. |
| * The ¨ballista¨ testcases of the Linux Test Project should be run when we are finished with fixing bugs. | * [[https://github.com/strace/strace|strace(1)]] test suite - ''%%make check%%'' to run, focused on ''%%ptrace(2)%%'' |
| * The Open POSIX testsuite (comes with the LPI); we need a reference output from Linux to be able to compare this. | * The ¨ballista¨ test cases of the Linux Test Project should be run when we are finished with fixing bugs. |
| * Some of the LTP testcases which are not run by default. | * The Open POSIX test suite (comes with the LPI); we need a reference output from Linux to be able to compare this. |
| | * Some of the LTP test cases that are not run by default. |
| * The glibc regression tests. | * The glibc regression tests. |
| * Applications listed at [[LinuxApps]] and [[WantedLinuxApps]] | * Applications listed at [[LinuxApps]] and [[WantedLinuxApps]] |
| ==== Debugging ==== | ==== Debugging ==== |
| |
| %%FreeBSD%%'s truss(1) and ktrace(1) can trace Linux binaries and decode the syscall names, but they don't support decoding Linux arguments into human-readable form. Linux strace(1) does, though - it can be installed from devel/linux-c7-strace port, or the linux-c7-strace package. | %%FreeBSD%%'s truss(1) and ktrace(1) can trace Linux binaries and decode the syscall names, but they don't support decoding Linux arguments into human-readable form. Linux strace(1) does, though - it can be installed from the linux-c7-strace package, or by ''%%apt install strace%%'' in [[LinuxJails|Ubuntu chroot]]. |
| |
| Linux core dumps are, from what I understand, not functional yet. | Linux core dumps are supported starting with 14.0-CURRENT. Use Linux ''%%gdb%%'' to open them; you might want to consult [[https://wiki.ubuntu.com/Debug%20Symbol%20Packages|Ubuntu documentation on debug symbol packages.]] |
| |
| ===== Roadmap ===== | ===== Roadmap ===== |
| |
| * Futexes lack synchronization with VM and virtual memory independence. | * Implement PI Futexes |
| | * Implement linprocfs ''%%/proc/self/task/$TID/%%'' |
| | * Check ''%%setpriority/getpriority%%'' - seems that Linux allows set/get prio per thread |
| | * Use ''%%uma%%'' instead of ''%%M_LINUX%%'' |
| | * Check ''%%__WALL%%'', ''%%__WCLONE%%'' in ''%%wait%%'' syscalls since Linuxulator's threading model has changed. The ''%%__WALL%%'' since Linux 4.7 automatically implied if the child process is being ptraced |
| * add the compat.linux.strict_emu sysctl (the name is not set in stone) and use it to enable/disable some functionality we allow in %%FreeBSD%% but are not allowed to do in Linux, e.g., allow to open more than one million files, or allow to read() directories | * add the compat.linux.strict_emu sysctl (the name is not set in stone) and use it to enable/disable some functionality we allow in %%FreeBSD%% but are not allowed to do in Linux, e.g., allow to open more than one million files, or allow to read() directories |
| * add detection of unhandled flags to syscalls (e.g. add each detected flag to a variable and compare this variable to the input, if there's a difference print the difference) | * add detection of unhandled flags to syscalls (e.g. add each detected flag to a variable and compare this variable to the input, if there's a difference print the difference) |
| * print a message if a known but unhandled flag (e.g. LINUX_O_NOATIME for open(%%))%% is used (maybe protected by bootverbose or debug; depending upon the severity and frequence of the use of this flag) | * print a message if a known but unhandled flag (e.g. LINUX_O_NOATIME for open(%%))%% is used (maybe protected by bootverbose or debug; depending upon the severity and frequency of the use of this flag) |
| * change the non P1003_1B_MQUEUE part of linux_mq_*() into its own module (the changes for linux aio in p4@114975 can serve as an example) | * change the non P1003_1B_MQUEUE part of linux_mq_*() into its own module (the changes for linux aio in p4@114975 can serve as an example) |
| * add in the normal device driver entries into the device_handler stuff that are common to all Linux machines | * add in the normal device driver entries into the device_handler stuff that are common to all Linux machines |
| * CLK_TCK still valid? / XXX comment: both in linux_misc.c | * CLK_TCK still valid? / XXX comment: both in linux_misc.c |
| * add doxygen comments to the code | * add doxygen comments to the code |
| * Use 1:1 threads instead of processes for Linux threads? | * follow r237433/aea810386d8e and use kernel timekeeping data in the Linuxulator's vdso |
| |
| ===== Bugs ===== | ===== Bugs ===== |
| |
| **A note to users:** Feel free to read the following list of things to fix, but do //not// make conclusions out of it. The linux compatibility environment //runs just fine//. There are some broken edge cases which don't affect the daily use. The following list is only meaningful for developers. Just because something is marked as a bug, it doesn't mean it doesn't work. It may be the case that some obscure error condition does not return the expected error value, or that a seldomly used feature is not implemented. | **A note to users:** Feel free to read the following list of things to fix, but do //not// make conclusions out of it. The linux compatibility environment //runs just fine//. There are some broken edge cases that don't affect daily use. The following list is only meant for developers. Just because something is marked as a bug, it doesn't mean it doesn't work. It may be the case that some obscure error condition does not return the expected error value, or that a seldomly used feature is not implemented. |
| |
| There's an umbrella ticket for tracking Linuxulator-related PRs: [[https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247219|247219]] | There's an umbrella ticket for tracking Linuxulator-related PRs: [[https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247219|247219]] |