This is an old revision of the document!


Thursday May 15th, 9.00-12.00 location TBD

This working group will discuss networking stack topics. Please send proposal for topics to discuss to Luigi Rizzo rizzo@iet.unipi.it , they will be summarized here and then a selection will be done based on interest and relevance. Preference will be given to topics that actually need interactive discussion rather than one-way communications or status updates.

A list of changes and APIs that will help facility FreeBSD better using the above listed features.

The order is just for naming topics

# Topic Description
1 batching
2 parallelism
3 alternate FIBs
4 TCP segmentation and reassembly
5 ifnet abstraction
6 ifnet extensions for multiqueue/ethtool
7 mbuf revisiting
8 backpressure
9 vlan QinQ, vxlan
10 Lightweight refcounters
  • Batching

improve batching support in the network paths, starting from if_input and ether_output. Preferably using backward compatible APIs so we can move on incrementally. Obvious example is to use the m_nextpkt field in mbufs. The solution does not have to be perfect, we need something badly now.

  • Parallelism
    improve parallelism in the network stack. Not my area but there have been various suggestions to reduce contention on tcpcb's etc. Surely taking inspiration on what has been done on other platforms including research ones. See also lock contention issues with short lived connections at http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/183659
  • alternate FIBs (routing tables)
    Low end machines can live with the current, slow routing tables (perhaps with a bit of caching), but we should provide hooks for alternatives such as Marko Zec's DXR, or other solutions which may come up.
  • TCP/UDP segmentation (and reassembly), checksums
    Introduce support for software TCP segmentation, as low as possible in the stack so that we can push large segments down to the driver where it will be taken care of. This saves high repetitive costs for IP and ARP lookups on every segment. GSO is segmentation offloading in software above the driver, but we could also do something more efficient (sometimes called DSO) operating in the driver which could pre-allocate the extra buffers for headers. NIC-based checksums should be used if possible, and in any case support should be moved down the stack.
  • ifnet abstraction - see previous emails from Juniper ?
  • ifnet extensions for multiqueue / ethtool support
    we should provide info on queue numbers and sizes, and other features that might be of use for multiqueue devices, mostly in the control path (so they can be externally linked). In linux these are managed with device-independent tools, which is much better than the custom methods we have now, and avoids polluting the ifnet with extra information.
  • mbuf revisiting
    We need to move forward on this on a number of things.
    • 'ephemeral' mbufs (which must be consumed inline, e.g. allocated on the stack or otherwise owned by the caller). These have a large number of uses.
    • in-mbuf space for mtags (largely used for dummynet and perhaps other things).
    • arrays vs linked lists of mbufs (trying to reduce the cost of chain traversals)
    • L2/L3/L4 pointers in the pkthdr to avoid repeated packet parsing
  • backpressure
    FreeBSD has no good backpressure mechanism to signal sockets that a link is busy. The most evident symptom is that UDP sockets are never blocking on transmission but can generate significant losses. For TCP, we need to backoff when the tx queue is full and hope that incoming acks will wake us up.
    In linux, buffers in the tx queue hold a reference to the socket so completions can be used to notify sockets. Implementing the same mechanism in FreeBSD should be relatively straightforward.

* vlan QinQ, vxlan

* Lightweight refcounters
We have several objects (interfaces, bridges, modules, etc.) that are long lived but do change occasionally, so some form of reference counting is needed. We are looking for solutions that are cheap on acquire/release.

In order to attend you need register for the developer summit as well as by email for the session and be confirmed by the working group organizer. Follow the guidelines described on the main page or what you received by email. For questions or if in doubt ask the session chairs.

Please do NOT add yourself here. Your name will appear automatically once you have received the confirmation email. You need to put your name on the general developer summit attendees list though.

Name Username / Affiliation Topics of Interest Notes
Michael Bentkofsky Verisign all
Kevin Bowling LLNW
Chris Busick
Adrian Chadd adrian
Julien Charbon Verisign TCP stack parallelism
David Christensen broadcom ethtool
Eric Davis edavis
Suresh Gumpula Netapp
John Mark Gurney jmg
Hash Haven isilon OFED, rx livelock
Mike Karels mcafee
Marcus Hoffman Norse
Jeremiah Lott Avere Systems
Jeffrey Meegan isilon
Navdeep Parhar navdeep / chelsio
Luis Otavio O Souza
Alfred Perlstein Norse
Luigi Rizzo luigi / Univ. Pisa
Jonathan Saks xinuos
Hiroki Sato
Mike Silbersack silby
Gleb Smirnoff glebius lightweight refcounters
Matt Smith netgate
Lawrence Stewart lstewart / netflix backpressure
Jim Thompson netgate
Bryan Venteicher bryanv mbuf ext, vlan, vxlan
Claudia Y Norse
Bjoern Zeeb bz

(Add a list or attach slides detailing the achieved results here.)

  • devsummit/201405/networkstack.1400086956.txt.gz
  • Last modified: 2014/05/14 17:02
  • by LuigiRizzo