networking

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
networking [2014/10/01 17:29] – add protocol loading issue JohnMarkGurneynetworking [2021/04/25 08:13] (current) – Fix DevSummit links. JethroNederhof
Line 1: Line 1:
-This topic is 'mission control' for ongoing work on the %%FreeBSD%% networking code. It is by no means exhaustive or authoritative, however, its use as a tool for effective collaboration is encouraged.+This is 'mission control' for ongoing work on the %%FreeBSD%% networking code. It is by no means exhaustive or authoritative, however, its use as a tool for effective collaboration is encouraged.
  
-If you want to work on a task please put your Wiki name next to it. Participation in other forums such as the net@%%FreeBSD%%.org mailing list is strongly encouraged. There is plenty for volunteers of all aptitude levels to do; some of this work even has the potential to be funded open source work. +If you want to work on a task please put your %%WikiName%% next to it. Participation in other forums such as the [[https://lists.freebsd.org/pipermail/freebsd-net/|freebsd-net mailing list]] is strongly encouraged. There is plenty for volunteers of all aptitude levels to do; some of this work even has the potential to be funded open source work.
- +
-As a general rule of thumb it takes around 2 weeks to get good feedback about changes to %%FreeBSD%%-CURRENT.+
  
 The tasks are arranged roughly in descending order of how quickly they can/should be finished. The tasks are arranged roughly in descending order of how quickly they can/should be finished.
 +
 +**Subpages**
 +
 +  * [[Networking/10GbE|10GbE]]
 +  * [[Networking/ReceiveSideScaling|ReceiveSideScaling]]
 +  * [[Networking/WakeOnLan|WakeOnLan]]
  
 ===== Low hanging userland fruit ===== ===== Low hanging userland fruit =====
Line 62: Line 66:
     * The [[http://www.xorp.org/|XORP]] router control plane can use Click as its forwarding path as an alternative to ip_forward().     * The [[http://www.xorp.org/|XORP]] router control plane can use Click as its forwarding path as an alternative to ip_forward().
   * Make 'Zeroconf' local area DNS service discovery ('DNS-SD') work.   * Make 'Zeroconf' local area DNS service discovery ('DNS-SD') work.
-    * A [[http://people.freebsd.org/~bms/dump/kdnssd_avahi.tar|New port]] for kdnssd_avahi is available for testing, which overwrites the default KDE libkdnssd.so.1 library with one which uses Avahi as a back-end for mdns. -- BruceSimpson+    * A ''%%http://people.freebsd.org/~bms/dump/kdnssd_avahi.tar%%'' (broken) New port for kdnssd_avahi is available for testing, which overwrites the default KDE libkdnssd.so.1 library with one which uses Avahi as a back-end for mdns. -- BruceSimpson
   * Teach the net/olsrd port to use IP_ONESBCAST.   * Teach the net/olsrd port to use IP_ONESBCAST.
     * A patch has been sent to the port maintainer. -- BruceSimpson     * A patch has been sent to the port maintainer. -- BruceSimpson
Line 123: Line 127:
   * protosw(9) man page for protocol switch APIs.   * protosw(9) man page for protocol switch APIs.
   * Revisit multi-mbuf allocation, multi-mbuf input.   * Revisit multi-mbuf allocation, multi-mbuf input.
 +    * ether_input() accepts a list of mbufs
   * Investigate solutions to inpcb tear-down races involving timers with tcp -- right now, timers are stopped but not drained, so timers can race with tear-down, hence NULL checks in timer code.   * Investigate solutions to inpcb tear-down races involving timers with tcp -- right now, timers are stopped but not drained, so timers can race with tear-down, hence NULL checks in timer code.
   * Revisit locking and atomicity for uipc_send(), and in particular, with respect to what happens if two threads simultaneously call sendto() on different addresses with the same datagram socket. Currently this is likely prevented by sblock() at the system call layer, but races with connect() aren't, as was shown by the recent race case along the same lines. Somehow this needs to be serialized.   * Revisit locking and atomicity for uipc_send(), and in particular, with respect to what happens if two threads simultaneously call sendto() on different addresses with the same datagram socket. Currently this is likely prevented by sblock() at the system call layer, but races with connect() aren't, as was shown by the recent race case along the same lines. Somehow this needs to be serialized.
Line 128: Line 133:
   * Continue pushing of control of state changes from socket layer to protocol layer so as to improve inter-layer consistency and reduce races between layers on socket state transitions (connect/disconnect/connecting/disconnecting/...)   * Continue pushing of control of state changes from socket layer to protocol layer so as to improve inter-layer consistency and reduce races between layers on socket state transitions (connect/disconnect/connecting/disconnecting/...)
   * Analysis of listen state transitions, which appear poorly defined.  Consider in particular interactions with kqueue, where the order of registering kqueue events vs. calling listen affects the semantics considerably.   * Analysis of listen state transitions, which appear poorly defined.  Consider in particular interactions with kqueue, where the order of registering kqueue events vs. calling listen affects the semantics considerably.
-  * Integrate solutions for RSS (for reference: [[ComparingMutiqueueSupportLinuxvsFreeBSD|ComparingMutiqueueSupportLinuxvsFreeBSD]] [[NetworkRSS|NetworkRSS]]+  * Integrate solutions for RSS (for reference: [[DevSummit/201305/NetworkReceivePerformance/ComparingMutiqueueSupportLinuxvsFreeBSD|ComparingMultiqueueSupportLinuxvsFreeBSD]] [[Networking/ReceiveSideScaling]])
  
 ===== Long term projects ===== ===== Long term projects =====
Line 150: Line 155:
   * Implement source routing policy.   * Implement source routing policy.
     * This is deliberately vague as it needs further research.     * This is deliberately vague as it needs further research.
-    * Original research is currently being conducted in the formal academic world into the issues around implementing DiffServ and IntServ, both of which impact forwarding at all layers of the stack.+    * Original research is currently being conducted in the formal academic world into the issues around implementing %%DiffServ%% and %%IntServ%%, both of which impact forwarding at all layers of the stack.
     * This is an area for funded formal research.     * This is an area for funded formal research.
   * Explore increasing netisr queue depth due to gallatin's report that overflows are causing TCP problems with 10gbps interfaces.   * Explore increasing netisr queue depth due to gallatin's report that overflows are causing TCP problems with 10gbps interfaces.
Line 164: Line 169:
     * Implement API to program FIB for given family     * Implement API to program FIB for given family
     * Implement IPv4 FIB     * Implement IPv4 FIB
-  * Socket / mbuf lifecycle and refcounting 
-    * Netflix have noticed a lot of memory accesses when managing externally refcounted mbufs. The root seems to be the refcount itself - it comes out of a separate UMA pool full of DWORD (4-byte) refcounts, which almost always ends up being a read from memory rather than from L2/L3 cache. glebius@ has a patch to migrate mbuf refcounting into a couple of methods so mbufs that are backed by vmpage/sfbuf can use the refcount already in that structure, rather than adding another layer of refcounting. 
-  * TCP / UDP incoming load balancing across multiple threads/processes 
   * mbuf backing store representation   * mbuf backing store representation
     * Under memory fragmentation, a large send buffer can be represented by a large set of underlying, discontiguous memory pages. In the most pessimistic case, a large read via sendfile() can be decomposed into a small number of 4KiB page entries which are individually converted into separate mbufs and then passed to the network stack to send. If some pages are cached but others aren't, then this also results in a list of mbufs that represent each set of pages. Then in the network driver, this list of mbufs has to be walked again and turned into a gather DMA list for the NIC. This all takes time and touches memory.     * Under memory fragmentation, a large send buffer can be represented by a large set of underlying, discontiguous memory pages. In the most pessimistic case, a large read via sendfile() can be decomposed into a small number of 4KiB page entries which are individually converted into separate mbufs and then passed to the network stack to send. If some pages are cached but others aren't, then this also results in a list of mbufs that represent each set of pages. Then in the network driver, this list of mbufs has to be walked again and turned into a gather DMA list for the NIC. This all takes time and touches memory.
Line 182: Line 184:
 802.11 Wireless is a 'killer app' for open source operating systems. Help %%FreeBSD%% to stay ahead. 802.11 Wireless is a 'killer app' for open source operating systems. Help %%FreeBSD%% to stay ahead.
  
-  * Implement 802.11n 'MIMO' features in net80211. +  * Look at a few things: 
-    * Receive and transmit diversity. +
-  * Develop a virtual 802.11 driver, let's call it wtap(4) for example. +
-    * tap emulates 802.3 Ethernet, it would be useful to have same for 802.11. +
-    * This will be instrumental in developing 802.11s ESS Mesh. +
-    * The if_bridge code serves as a reference for doing "Brouting" like this. +
-      * See [[http://research.sun.com/sunlabsday/docs.2004/talks/1.05_Perlman.pdf|Radia Perlman's "RBridges" paper]] +
-    * Confirm that we support 802.11 frame injection via bpf/pcap from userland. +
-  * Such a virtual 802.11 driver could potentially be used to implement the following:+
     * [[http://en.wikipedia.org/wiki/IEEE_802.11f|IEEE 802.11f Inter Access Point Protocol]]     * [[http://en.wikipedia.org/wiki/IEEE_802.11f|IEEE 802.11f Inter Access Point Protocol]]
     * [[http://en.wikipedia.org/wiki/IEEE_802.11k|IEEE 802.11k Radio Resource Management]]     * [[http://en.wikipedia.org/wiki/IEEE_802.11k|IEEE 802.11k Radio Resource Management]]
Line 225: Line 220:
     * If we want to send 802.1p priority, this should be done as part of a full implementation.     * If we want to send 802.1p priority, this should be done as part of a full implementation.
       * When layers are being crossed, something between the protocol layer and link layer will be responsible for mapping the DSCP byte to 802.1p.       * When layers are being crossed, something between the protocol layer and link layer will be responsible for mapping the DSCP byte to 802.1p.
-    * Until RSVP and DiffServ are fully implemented, the IP_TOS socket option can be used to explicitly set the entire ip_tos byte to a DSCP by an application.+    * Until RSVP and %%DiffServ%% are fully implemented, the IP_TOS socket option can be used to explicitly set the entire ip_tos byte to a DSCP by an application.
   * Create a %%FreeBSD%% port of KOM RSVPD (it interoperates with kernel rsvp code and ALTQ).   * Create a %%FreeBSD%% port of KOM RSVPD (it interoperates with kernel rsvp code and ALTQ).
     * Implement generic packet classification capabilities, with a view to the switching of multimedia calls via SIP over lossy / bandwidth limited networks such as 802.11.     * Implement generic packet classification capabilities, with a view to the switching of multimedia calls via SIP over lossy / bandwidth limited networks such as 802.11.
Line 232: Line 227:
     * Both the Ethernet Slow Protocols (802.3ad) and ATM specify OAM now for managing carrier-grade networks.     * Both the Ethernet Slow Protocols (802.3ad) and ATM specify OAM now for managing carrier-grade networks.
   * Implement Multi-Protocol Label Switching (MPLS)   * Implement Multi-Protocol Label Switching (MPLS)
-    * MPLS is widely deployed on Internet network backbones to optimize forwarding and provide advanced IP services. +    * MPLS is widely deployed on Internet network backbones to optimize forwarding and provide advanced IP services.
     * AlexanderChernikov has begun work with MPLS implementation on %%FreeBSD%%.     * AlexanderChernikov has begun work with MPLS implementation on %%FreeBSD%%.
-      * This MPLS implementation is heavily based on netgraph  +      * This MPLS implementation is heavily based on netgraph 
-      * Modified version of bird routing software is used to provide LDP signalling +      * Modified version of bird routing software is used to provide LDP signalling
       * Current project status/wiki/svn can be viewed at [[http://freebsd.mpls.in]]       * Current project status/wiki/svn can be viewed at [[http://freebsd.mpls.in]]
     * MatthewLuckie has begun porting the [[http://www.ayame.org/|Ayame MPLS stack]] to %%FreeBSD%%.     * MatthewLuckie has begun porting the [[http://www.ayame.org/|Ayame MPLS stack]] to %%FreeBSD%%.
Line 243: Line 238:
     * See http://www.scps.org/     * See http://www.scps.org/
     * This is a 'hot topic' in commercial and academic satellite communications (TCP Performance Enhancing Proxies).     * This is a 'hot topic' in commercial and academic satellite communications (TCP Performance Enhancing Proxies).
-  * Port the MITRE MiniRouter to %%FreeBSD%%.+  * Port the MITRE %%MiniRouter%% to %%FreeBSD%%.
     * This potentially builds on and makes use of SCPS.     * This potentially builds on and makes use of SCPS.
     * See http://www.openchannelsoftware.org/projects/MITRE_Minirouter/     * See http://www.openchannelsoftware.org/projects/MITRE_Minirouter/
     * This is a specialist area of research of interest to those in satellite communications.     * This is a specialist area of research of interest to those in satellite communications.
 +
 ===== See also ===== ===== See also =====
-  * [[http://www.freebsd.org/projects/netperf/cluster.html|Information on the Netperf cluster]]+
   * NetperfClusterPointers   * NetperfClusterPointers
 +  * TestClusterOneReservations
  
  • networking.1412184541.txt.gz
  • Last modified: 2014/10/01 17:29
  • by JohnMarkGurney