networkperformancetuning

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
networkperformancetuning [2016/12/17 08:27] – OlivierCochardLabbénetworkperformancetuning [2018/11/16 08:31] (current) – OlivierCochardLabbé
Line 20: Line 20:
   * net.inet.flowtable.nmbflows   * net.inet.flowtable.nmbflows
 ==== Router ==== ==== Router ====
-This section describes how to reach 10G performance from your server+This section describes how to reach 10G performance from your server. 
 + 
 +**This section is old** you should read the latest [[https://people.freebsd.org/~olivier/talks/2018_AsiaBSDCon_Tuning_FreeBSD_for_routing_and_firewalling-Paper.pdf|Tuning FreeBSD for routing and firewalling (AsiaBSD 2018)]] for more up-to-date data on a %%FreeBSD%% 11.1 
 === Hardware === === Hardware ===
 First, you need good NIC(s) capable of: First, you need good NIC(s) capable of:
Line 36: Line 39:
   * Mellanox   * Mellanox
   * Broadcom   * Broadcom
-  * SolarFlare+  * %%SolarFlare%%
   * Brocade   * Brocade
   * Myricom   * Myricom
Line 53: Line 56:
   * Ensure you're not running with rtsock generating tons of RTM_MISS messages   * Ensure you're not running with rtsock generating tons of RTM_MISS messages
  
-On %%FreeBSD%% older than 11.0, use more compact ip_fastfoward routine. It processes most packets falling back to 'normal' forward routine for fragments, packets with options, etc. Can save you up to 20% speed (but will break IPSec).+On %%FreeBSD%% older than 11.0, use more compact ip_fastfoward routine. It processes most packets falling back to 'normal' forward routine for fragments, packets with options, etc. Can save you up to 20% speed (but will break IPSec). Since %%FreeBSD%% 11.0, fastfordwarding was improved, renamed tryforward (no more break IPSec) and it's the default method.
   * net.inet.ip.fastforwarding=1   * net.inet.ip.fastforwarding=1
 Do not send IP redirects Do not send IP redirects
   * net.inet.ip.redirect=0   * net.inet.ip.redirect=0
-Skip feeding /dev/random from network by adding harvest_mask="351" to /etc/rc.conf or theses line to /etc/sysctl.conf:+Skip feeding /dev/random from network. 
 +Since %%FreeBSD%% 11:0: By adding harvest_mask="351" to /etc/rc.conf. 
 +For older %%FreeBSD%%, add theses line to /etc/sysctl.conf:
   * kern.random.sys.harvest.ethernet=0   * kern.random.sys.harvest.ethernet=0
   * kern.random.sys.harvest.point_to_point=0   * kern.random.sys.harvest.point_to_point=0
   * kern.random.sys.harvest.interrupt=0   * kern.random.sys.harvest.interrupt=0
-Entropy harvest is collecting first 2 bytes of each frame, and doing this under single mutex :-(+Entropy harvest is collecting first 2 bytes of each frame, and doing this under single mutex ([[https://svnweb.freebsd.org/changeset/base/334450|fixed in r334450]])
 Some benchs about this impact here [[http://blog.cochard.me/2015/09/receipt-for-building-10mpps-freebsd.html|Receipt for building a 10Mpps FreeBSD based router]]. Some benchs about this impact here [[http://blog.cochard.me/2015/09/receipt-for-building-10mpps-freebsd.html|Receipt for building a 10Mpps FreeBSD based router]].
  
Line 95: Line 100:
   * ng_mppc   * ng_mppc
   * ng_pipe   * ng_pipe
 +
 +=== if_bridge ===
 +if_bridge(4), specially bridge_input() uses lot's of LOCK: Avoid to use it.
 +[[https://github.com/ocochard/netbenches/blob/master/Atom_C2758_8Cores-Chelsio_T540-CR/bridge/results/fbsd11.1-yandex/README.md|It will reduce by about 50-60% forwarding speed.]]
 +
 === Firewalls === === Firewalls ===
 Firewalling is possibly the one major reasons to use software router on 10G+ links. Firewalling is possibly the one major reasons to use software router on 10G+ links.
Line 131: Line 141:
   * Quagga. Famous cisco-like daemon.   * Quagga. Famous cisco-like daemon.
   * bird. Juniper-like configs, multiple kernel tables, ability to filter kernel routes   * bird. Juniper-like configs, multiple kernel tables, ability to filter kernel routes
 +
 +----
 +
 +{{tag>Project}}
  
  • networkperformancetuning.1481963240.txt.gz
  • Last modified: 2016/12/17 08:27
  • by OlivierCochardLabbé