| Both sides previous revision Previous revision Next revision | Previous revision |
| ideaspage [2021/12/29 20:17] – remove libmprof now that we have valgrind WarnerLosh | ideaspage [2023/10/04 15:41] (current) – Fix typo. CySchubert |
|---|
| | ===== Ideas Page ===== |
| | |
| ===== Introduction ===== | ===== Introduction ===== |
| |
| * Basic understanding of backup/restore procedures. | * Basic understanding of backup/restore procedures. |
| |
| ===== Filesystem decompression layer ===== | ===== OpenBFS ===== |
| |
| Solaris 10 and newer provide [[http://docs.oracle.com/cd/E19963-01/html/821-1475/dcfs-7fs.html#REFMAN7dcfs-7fs|dcfs]]; a read-only decompression stacking filesystem layer for UFS. Files are initially compressed by a userland [[http://docs.oracle.com/cd/E19963-01/html/821-1462/fiocompress-1m.html|fiocompress]] utility. The filesystem layer is very simple, it is implemented in single [[http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/fs/dcfs/dc_vnops.c|file]] that permits transparent decompression without the end user knowing if the file is compressed or not. While the implementation is really simple it is very useful by making possible to install in systems with little memory or for quick compression of files in typical read-only directories like /usr/bin. While the Solaris fiocompress [[http://src.illumos.org/source/xref/illumos-gate/usr/src/cmd/boot/fiocompress/|implementation]] uses zlib, it would be easy to use more modern algorithms like lz4 or snappy. | [[http://nobius.org/~dbg/practical-file-system-design.pdf|Practical Filesystem Design Book]] |
| | |
| References: [[https://www.isi.edu/~johnh/PAPERS/Heidemann94a.html|File-System Development with Stackable Layers]] | |
| | |
| ===== Bring back the RAIDframe port ===== | |
| | |
| [[https://people.freebsd.org/~scottl/rf/]] | |
| |
| == Description == | == Description == |
| |
| %%NetBSD%% has a software [[https://www.cs.usask.ca/staff/oster/raid.html|RAID framework]] that is very useful for prototyping or otherwise implementing RAID solutions. Around %%FreeBSD%% 5 there was an heroic effort to port it, hovever, the interfaces to devices and disks was changed and the port never caught up so the code was removed from %%FreeBSD%% in [[https://svnweb.freebsd.org/base?view=revision&revision=127066|Revision 127066]]. | Before working at Apple, Dominic Giampaolo wrote a 64 bit journalling filesystem for %%BeOS%%. The objectives were to have a modern filesystem capable of streaming for multimedia applications. It also supports indexing and Extended Attributes for desktop use. [[https://www.haiku-os.org/|Haiku]] has [[https://git.haiku-os.org/haiku/tree/src/add-ons/kernel/file_systems/bfs|implemented]] a version of the filesystem based on the documentation which is used as their base filesystem and is available under an MIT license. Porting it, or rather using the existing implementation as the basis for a new one, would make an interesting case for an alternative filesystem. |
| |
| == Requirements == | == Requirements == |
| |
| * Ability to read and understand foreign C code, familiarity with filesystems is a plus. | * Ability to read and understand foreign C code, familiarity with %%FreeBSD%%'s VFS is a plus. |
| * knowledge of geom(4). | * Ability to interpret results from testsuites and find solutions. |
| * Willingness to experiment and debug filesystems. | * Knowledge of filesystem internals. |
| |
| ===== OpenBFS ===== | ===== Port Hammer2 ===== |
| |
| [[http://nobius.org/~dbg/practical-file-system-design.pdf|Practical Filesystem Design Book]] | [[https://en.wikipedia.org/wiki/HAMMER2|HAMMER2 Filesystem]] |
| |
| == Description == | == Description == |
| |
| Before working at Apple, Dominic Giampaolo wrote a 64 bit journalling filesystem for %%BeOS%%. The objectives were to have a modern filesystem capable of streaming for multimedia applications. It also supports indexing and Extended Attributes for desktop use. [[https://www.haiku-os.org/|Haiku]] has [[https://git.haiku-os.org/haiku/tree/src/add-ons/kernel/file_systems/bfs|implemented]] a version of the filesystem based on the documentation which is used as their base filesystem and is available under an MIT license. Porting it, or rather using the existing implementation as the basis for a new one, would make an interesting case for an alternative filesystem. | The [[https://www.dragonflybsd.org/hammer/|Hammer]] is a recent copy-on-write filesystem developed by Matt Dillon as part of [[https://www.dragonflybsd.org/|DragonFlyBSD]]. |
| | Due to the differences in evolution in %%DragonFly%% it would be very difficult to port the original hammer but it should be possible to port Hammer2, which has most of the same advanced features and is specifically designed to support clustering. |
| | Both filesystems are available under a BSD license. The ports tree carries [[https://www.freshports.org/sysutils/hammer2/|sysutils/hammer2]] a read-only port of the kernel driver: figuring out how to finish the write support is being considered for %%NetBSD%%. |
| |
| == Requirements == | == Requirements == |
| |
| * Ability to read and understand foreign C code, familiarity with %%FreeBSD%%'s VFS is a plus. | * Strong knowledge of C. |
| * Ability to interpret results from testsuites and find solutions. | * Understanding of %%FreeBSD%%'s filesystem interfacing and VFS. |
| * Knowledge of filesystem internals. | * Willingness to dive into %%DragonFly%%BSD's locking semantics. |
| |
| ===== Porting HFS+ ===== | ===== Porting HFS+ ===== |
| * Ability to write C code. | * Ability to write C code. |
| * A good understanding of the %%FreeBSD%% kernel. | * A good understanding of the %%FreeBSD%% kernel. |
| | |
| | ===== Kernel Control Flow Integrity (kCFI) ===== |
| | |
| | Control Flow Integrity (CFI) is a security feature implemented mostly compilers and is fully supported by Clang/LLVM: the original prototype was indeed implemented for %%FreeBSD%%. |
| | CFI has greatly evolved and has been [[https://lwn.net/Articles/810077/|implemented in the Linux kernel]] and [[https://source.android.com/docs/security/test/kcfi|Android]]. |
| | CFI is relatively well [[https://clang.llvm.org/docs/ControlFlowIntegrityDesign.html|documented]] and should be easy to implement on %%FreeBSD%%. Using CFI we shouldn't need to use an additional stack protector at all. |
| | |
| | == Requirements == |
| | |
| | * Ability to read and understand foreign C code. |
| | * Ability to write C code and definitions. |
| | * Interest in compilers, linkers, and how they work. |
| |
| ===== IOCTL command checking tool ===== | ===== IOCTL command checking tool ===== |
| **Contact:** [[DanielEbdrup]] | **Contact:** [[DanielEbdrup]] |
| |
| **Technical Contact:** [[freebsd-virtualization@|freebsd-virtualization@]] | **Technical Contact:** <freebsd-virtualization@freebsd.org> |
| |
| Jail network connectivity has gotten a separate network stack through VNET/VIMAGE, but at present is limited to 10G between jails and the host. Utilizing a netmap based vnic for jails and the vale software switch to make IPC for jails <-> bhyve and the outside world much easier and faster. Additionally, it may very well be possible to adopt a lot of existing code from svn revision 293459. | Jail network connectivity has gotten a separate network stack through VNET/VIMAGE, but at present is limited to 10G between jails and the host. Utilizing a netmap based vnic for jails and the vale software switch to make IPC for jails <-> bhyve and the outside world much easier and faster. Additionally, it may very well be possible to adopt a lot of existing code from svn revision 293459. |
| ===== Profile Guided Optimization (PGO) ===== | ===== Profile Guided Optimization (PGO) ===== |
| |
| URL: [[https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization]] | Both GCC and clang offer the option of generating code that is **optimized with CPU programming counter data** by tools like Linux [[https://perf.wiki.kernel.org/index.php/Main_Page|perf]] or %%FreeBSD%% [[https://man.freebsd.org/cgi/man.cgi?query=pmc&sektion=3&manpath=freebsd-release|pmc(3)]]. |
| | |
| | With Linux, the recommendation is to use the ''%%-b%%'' flag to use the Last Branch Record (LBR) to record call chains. This information is then converted with [[https://github.com/google/autofdo|autofdo]] to generate a profile. |
| | |
| | %%FreeBSD%% has no similar process. We should enhance tooling to enable generation of profiles in a compatible format. |
| | |
| | ==== Documentation ==== |
| | |
| | [[https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization|Profile Guided Optimization]] – Clang Compiler User's Manual |
| | |
| | LLVM SPGO profile generation tool: |
| |
| Both GCC and clang offer the option of generating code optimized with the CPU programming counter data from by tools like linux [[https://perf.wiki.kernel.org/index.php/Main_Page|perf]] or %%FreeBSD%%'s [[https://www.freebsd.org/cgi/man.cgi?query=pmc|pmc(3)]]. | * llvm-profgen14, llvm-profgen(1) https://man.freebsd.org/cgi/man.cgi?query=llvm-profgen14&sektion=1&manpath=freebsd-ports |
| On linux, the recommendation is to use the -b flag to use the Last Branch Record (LBR) to record call chains. | * llvm-profgen15, llvm-profgen(1) https://man.freebsd.org/cgi/man.cgi?query=llvm-profgen15&sektion=1&manpath=freebsd-ports |
| This information is then converted with the [[https://github.com/google/autofdo|autofdo]] tool to generate a profile. | * llvm-profgen16, llvm-profgen(1) manual page installable with [[https://www.freshports.org/devel/llvm16/|devel/llvm16]], the online version is not yet available (2023-07-18). |
| We currently don't have a similar process for %%FreeBSD%%: we need to enhance our tooling to make it possible to generate such profiles in a compatible format. | |
| |
| ===== NDMP data server ===== | ===== NDMP data server ===== |
| If you are interested in working on a project not explicitly mentioned above, you may want to contact one of the potential //Technical Contact//s below: | If you are interested in working on a project not explicitly mentioned above, you may want to contact one of the potential //Technical Contact//s below: |
| |
| * File systems: [[ScottLong|scottl@]], [[AlfredPerlstein|alfred@]] | * File Systems: ScottLong AlfredPerlstein |
| * GEOM: [[PawelJakubDawidek|pjd@]], [[PoulHenningKamp|phk@]], [[AndreyElsukov|ae@]] | * [[GEOM]]: PawelJakubDawidek PoulHenningKamp AndreyElsukov |
| * Networking: [[AlfredPerlstein|alfred@]], [[RobertWatson|rwatson@]], [[AndreyElsukov|ae@]] | * Networking: AlfredPerlstein RobertWatson AndreyElsukov |
| * 802.11 wireless networking: [[AdrianChadd|adrian@]], [[BernhardSchmidt|bschmidt@]] | * Wireless (802.11) Networking: AdrianChadd BernhardSchmidt |
| * Release Engineering / Integration: [[ReleaseEngineering|re@]] | * Release Engineering: ReleaseEngineering |
| * %%TrustedBSD%% / Security: [[RobertWatson|rwatson@]] | * [[Security|Security]] / [[TrustedBSD]]: RobertWatson |
| |
| Additionally, there are a lot of interesting mailing lists that can be used when searching information about specific subjects. | Additionally, there are a lot of interesting mailing lists that can be used when searching information about specific subjects. |