This is an old revision of the document!
FreeBSD Developer Summit: Network Stack
Thursday May 15th, 9.00-12.00 location TBD
Overview
This working group will discuss networking stack topics. Please send proposal for topics to discuss to Luigi Rizzo rizzo@iet.unipi.it , they will be summarized here and then a selection will be done based on interest and relevance. Preference will be given to topics that actually need interactive discussion rather than one-way communications or status updates.
Goals
A list of changes and APIs that will help facility FreeBSD better using the above listed features.
Topics
The order is just for naming topics
| # | Topic Description |
|---|---|
| 1 | batching |
| 2 | parallelism |
| 3 | alternate FIBs |
| 4 | TCP segmentation and reassembly |
| 5 | ifnet abstraction |
| 6 | ifnet extensions for multiqueue/ethtool |
| 7 | mbuf revisiting |
| 8 | backpressure |
- Batching
improve batching support in the network paths, starting from if_input and ether_output. Preferably using backward compatible APIs so we can move on incrementally. Obvious example is to use the m_nextpkt field in mbufs. The solution does not have to be perfect, we need something badly now.
- Parallelism
improve parallelism in the network stack. Not my area but there have been various suggestions to reduce contention on tcpcb's etc. Surely taking inspiration on what has been done on other platforms including research ones.
- alternate FIBs (routing tables)
Low end machines can live with the current, slow routing tables (perhaps with a bit of caching), but we should provide hooks for alternatives such as Marko Zec's DXR, or other solutions which may come up.
- TCP segmentation (and reassembly)
Introduce support for software TCP segmentation, as low as possible in the stack so that we can push large segments down to the driver where it will be taken care of. This saves high repetitive costs for IP and ARP lookups on every segment. GSO is segmentation offloading in software above the driver, but we could also do something more efficient (sometimes called DSO) operating in the driver which could pre-allocate the extra buffers for headers. NIC-based checksums should be used if possible.
- ifnet abstraction - see previous emails from Juniper ?
- ifnet extensions for multiqueue / ethtool support
we should provide info on queue numbers and sizes, and other features that might be of use for multiqueue devices, mostly in the control path (so they can be externally linked). In linux these are managed with device-independent tools, which is much better than the custom methods we have now, and avoids polluting the ifnet with extra information.
- mbuf revisiting
We need to move forward on this on a numbe of things.- 'ephemeral' mbufs (which must be consumed inline, e.g. allocated on the stack or otherwise owned by the caller). These have a large number of uses.
- in-mbuf space for mtags (largely used for dummynet and perhaps other things).
- arrays vs linked lists of mbufs (trying to reduce the cost of chain traversals)
- backpressure
FreeBSD has no good backpressure mechanism to signal sockets that a link is busy. The most evident symptom is that UDP sockets are never blocking on transmission but can generate significant losses. For TCP, we need to backoff when the tx queue is full and hope that incoming acks will wake us up.
In linux, buffers in the tx queue hold a reference to the socket so completions can be used to notify sockets. Implementing the same mechanism in FreeBSD should be relatively straightforward.
Attending
In order to attend you need register for the developer summit as well as by email for the session and be confirmed by the working group organizer. Follow the guidelines described on the main page or what you received by email. For questions or if in doubt ask the session chairs.
Please do NOT add yourself here. Your name will appear automatically once you have received the confirmation email. You need to put your name on the general developer summit attendees list though.
| Name | Username / Affiliation | Topics of Interest | Notes |
|---|---|---|---|
| Chris Busick | |||
| Adrian Chadd | adrian | ||
| David Christensen | broadcom | ethtool | |
| Eric Davis | edavis | ||
| Suresh Gumpula | Netapp | ||
| John Mark Gurney | jmg | ||
| Hash Haven | isilon | OFED, rx livelock | |
| Mike Karels | mcafee | ||
| Jeffrey Meegan | isilon | ||
| Navdeep Parhar | navdeep / chelsio | ||
| Luis Otavio O Souza | |||
| Luigi Rizzo | luigi / Univ. Pisa | ||
| Mike Silbersack | silby | ||
| Lawrence Stewart | lstewart / netflix | backpressure | |
| Jim Thompson | |||
| Bjoern Zeeb | bz / cambridge |
Notes
Results
(Add a list or attach slides detailing the achieved results here.)