ZFS [FreeBSD WIKI]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== FreeBSD Developer Summit: Advancing ZFS Working Group =====

==== Overview ====

Discussing advancing ZFS on %%FreeBSD%%

==== Topics ====

  * Integrating ZFS features early and often
  * Making %%FreeBSD%% the goto place for the latest ZFS features
  * %%OpenCrypto%% (ZFS native encryption, need AES-CCM)
  * XUIO (Zero Copy, and sendfile, for ZFS)
  * Fault Management
  * Storage management tools

===== ZFS =====
New Features:
  * Compressed ARC
  * ABD
  * RAID-Z performance improvements

Upcoming:
  * Faster scrub/resilver
    * [[http://open-zfs.org/wiki/Scrub/Resilver_Performance|2016 ZFS DevSummit Talk]]
  * Encryption
    * [[https://www.youtube.com/watch?v=frnLiXclAMo&feature=youtu.be|2016 ZFS DevSummit Talk Video]]
  * Channel Programs
  * Device Removal
  * Redacted Send/Recv
  * TRIM support
  * DRAID
  * vdev metadata classes
  * multi-mount support
  * zpool checkpoint
  * persistent L2ARC

==== Zero Copy / Sendfile / XUIO ====

ZFS currently does not play nicely with sendfile(2). The idea with sendfile(2) is to share pages between the buffer cache and the mbuf, to prevent copying. The call results in a range of bytes from an open file descriptor being faulted into the buffer cache, and then those pages being shared all the way through to the network stack until the bytes are transmitted.

With ZFS, since it does not use the OS buffer cache, data is copied from the ZFS ARC to the buffer cache, where it is then shared with the mbuf etc.
Solaris has a solution to solve this issue, and it might make sense to adapt our sendfile mechanism to work in a similar way. [Solaris PSARC: Copy Reduction Interface%%](%%[[https://www.mail-archive.com/opensolaris-arc@mail.opensolaris.org/msg17203.html]])
This creates two new VFS operations, reqzcbuf which asks the filesystem to prepare a UIO for zero copy operations. In the case of ZFS, this involves allocating a new array to track the ARC headers of each buffer that is loaned out from the ZFS ARC.

//The member xu_zc_priv of the extended uio structure for zero-copy is a private handle that may be used by the provider to track its buffer headers or any other private information that is useful to map the loaned iovec entries to its internal buffers. The xu_zc_priv member is private to the provider and should not be changed or interpreted in anyway by the callers.//

Then the retzcbuf VFS operation is used to return that buffer to the filesystem and it can then free the private member and do the necessary housekeeping.
Implementing this same VFS operation for UFS and other filesystems may be a better approach to the future of sendfile(2).

==== OpenCrypto ====
%%OpenCrypto%% does not have an implementation of AES-CCM, which will be required in the future for %%OpenZFS%%'s native storage encryption.

%%OpenZFS%% would appreciate any additional review of the native encryption patch by security and crypto experts.

The Linux approach was to port the entire %%IllumOS%% Crypto Framework, this is an option but I would expect it will take longer to integrate into %%FreeBSD%%, and won't as easily be able to take advantage of accelerators like %%QuickAssist%%.

==== Acceleration ====
%%OpenZFS%% has some patches to vectorize some calculations, like the RAID-Z parity, and some of the checksums. Need someone to integrate these into %%FreeBSD%%. [[https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=218203|PR218203 AVX2 accelerate fletcher4 for ZFS]]

===== sesutil =====
Uses the SES (SCSI Enclosure Services) protocol to query and interact with backplanes to monitor disks and allow setting the locate and fault LEDs. This does not work in the case of direct-attach backplanes, and AHCI devices. In some of these cases there is still a fault/locate LED that could be used. Could mps(4) and mpr(4) be modified to create a virtual ''%%ses%%'' device similar to what AHCI does.

===== mpsutil =====
Native implementation of the LSI/Avago/Broadcom control tools, to view and manage the disk controller. Also allows updating the firmware. Currently does not support toggling the fault/locate LEDs. Just need to figure out how sas2ircu does it, and reimplement it.

==== Notes ====

====== ZFS Working Group - 201709 ======

----

{{tag>Historical}}