Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
zfstuningguide [2018/12/19 12:53] – link to a new website serving ZFS Evil Tuning Guide; the old one is down MateuszPiotrowskizfstuningguide [2023/04/02 17:15] (current) – 2 GB minimum for ZFS, according to OpenZFS documentation. GrahamPerrin
Line 1: Line 1:
----- 
- 
 See also: [[https://www.solaris-cookbook.eu/solaris/solaris-10-zfs-evil-tuning-guide/|Solaris: ZFS Evil Tuning Guide]], [[http://man.freebsd.org/loader.conf|loader.conf(5)]], [[http://man.freebsd.org/sysctl|sysctl(8)]]. See also: [[https://www.solaris-cookbook.eu/solaris/solaris-10-zfs-evil-tuning-guide/|Solaris: ZFS Evil Tuning Guide]], [[http://man.freebsd.org/loader.conf|loader.conf(5)]], [[http://man.freebsd.org/sysctl|sysctl(8)]].
  
 ===== ZFS Tuning Guide ===== ===== ZFS Tuning Guide =====
-(Work in Progress) 
  
-To use ZFS, at least 1 GB of memory is recommended (for all architectures) but more is helpful as ZFS needs *lots* of memory.  Depending on your workload, it may be possible to use ZFS on systems with less memory, but it requires careful tuning to avoid panics from memory exhaustion in the kernel.+[[https://openzfs.github.io/openzfs-docs/Project%20and%20Community/FAQ.html#performance-considerations|OpenZFS documentation recommends a minimum of 2GB of memory for ZFS]]; additional memory is //strongly recommended when the compression and deduplication features are enabled//. 
 + 
 +Depending on your workload, it may be possible to use ZFS on systems with less memory, but it requires careful tuning to avoid panics from memory exhaustion in the kernel.
  
 A 64-bit system is preferred due to its larger address space and better performance on 64-bit variables, which are used extensively by ZFS. 32-bit systems are supported though, with sufficient tuning. A 64-bit system is preferred due to its larger address space and better performance on 64-bit variables, which are used extensively by ZFS. 32-bit systems are supported though, with sufficient tuning.
Line 109: Line 108:
 ==== Application Issues ==== ==== Application Issues ====
  
-ZFS is a copy-on-write filesystem. As such metadata from the top of the hierarchy is copied in order to maintain consistency in case of sudden failure, i.e. loss of power during a write operation. This obviates the need for an fsck-like requirement of ZFS filesystems at boot. However the downside to this is that applications which perform updates in place to large files, e.g. databases, will likely perform poorly in this application of the filesystem due to excessive I/O from copy-on-write (a fast SLOG device -- e.g. a SSD -- can help regarding the write performance of databases or any application which is doing synchronous writes (e.g. open with O_FSYNC) to the FS to make sure the data is on non-volatile storage when the write-call returns). Additionally, database applications, such as Oracle, maintain a large cache (called the SGA in Oracle) in memory will perform poorly due to double caching of data in the ARC and in the application's own cache. Reducing the ARC to a minimum can improve performance of applications which maintain their own cache. At http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide there are some generic recommendations for ZFS on Solaris which mostly apply to %%FreeBSD%% too.+ZFS is a copy-on-write filesystem. As such metadata from the top of the hierarchy is copied in order to maintain consistency in case of sudden failure, i.e. loss of power during a write operation. This obviates the need for an fsck-like requirement of ZFS filesystems at boot. However the downside to this is that applications which perform updates in place to large files, e.g. databases, will likely perform poorly in this application of the filesystem due to excessive I/O from copy-on-write (a fast SLOG device -- e.g. a SSD -- can help regarding the write performance of databases or any application which is doing synchronous writes (e.g. open with O_FSYNC) to the FS to make sure the data is on non-volatile storage when the write-call returns). Additionally, database applications, such as Oracle, maintain a large cache (called the SGA in Oracle) in memory will perform poorly due to double caching of data in the ARC and in the application's own cache. Reducing the ARC to a minimum can improve performance of applications which maintain their own cache. At [[https://archive.vn/ZWZxY|ZFS Best Practices Guide]] there are some generic recommendations for ZFS on Solaris which mostly apply to %%FreeBSD%% too.
  
 ===== General Tuning ===== ===== General Tuning =====
Line 143: Line 142:
 # Set TXG write limit to a lower threshold.  This helps "level out" # Set TXG write limit to a lower threshold.  This helps "level out"
 # the throughput rate (see "zpool iostat").  A value of 256MB works well # the throughput rate (see "zpool iostat").  A value of 256MB works well
-# for systems with 4 GB of RAM, while 1 GB works well for us w/ 8 GB on +# for systems with 4 GB of RAM, while 1 GB works well for us w/ 8 GB on
 # disks which have 64 MB cache. <<BR>> # disks which have 64 MB cache. <<BR>>
  
Line 181: Line 180:
  
 <code> <code>
-vfs.zfs.zil_disable="1" +vfs.zfs.zil_disable="1"
 </code> </code>
  
 the vfs.zfs.zil_disable loader tunable was replaced with the "sync" dataset property. the vfs.zfs.zil_disable loader tunable was replaced with the "sync" dataset property.
  
-Disabling ZIL is not recommended where data consistency is required (such as database servers) but will not result in file system corruption. See http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide#Disabling_the_ZIL_.28Don.27t.29 .+Disabling ZIL is not recommended where data consistency is required (such as database servers) but will not result in file system corruption. See [[https://archive.is/pV9cy|ZFS Evil Tuning Guide]], section "Disabling the ZIL (Don't)".
  
 ZFS is designed to be used with "raw" drives - i.e. not over already created hardware RAID volumes (this is sometimes called "JBOD" or "passthrough" mode when used with RAID controllers), but can benefit greatly from good and fast controllers. ZFS is designed to be used with "raw" drives - i.e. not over already created hardware RAID volumes (this is sometimes called "JBOD" or "passthrough" mode when used with RAID controllers), but can benefit greatly from good and fast controllers.
Line 208: Line 207:
  
 References References
-*[[https://blogs.oracle.com/realneel/entry/mysql_innodb_zfs_best_practices|MySQL Innodb ZFS Best Practices (Oracle)]]+*[[https://web.archive.org/web/20170501084558/https://blogs.oracle.com/realneel/entry/mysql_innodb_zfs_best_practices|MySQL Innodb ZFS Best Practices (Oracle)]]
  
 ===== Scrub and Resilver Performance ===== ===== Scrub and Resilver Performance =====
Line 224: Line 223:
 ---- ----
  
-{{tag>Zfs HowTo}}+{{tag>HowTo}} 
 +{{tag>Stale}} 
 +{{tag>NeedsContent}} 
 +{{tag>Zfs}}
  
  • zfstuningguide.1545224008.txt.gz
  • Last modified: 2018/12/19 12:53
  • by MateuszPiotrowski