Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
| devsummit:201405:networkstack [2014/05/26 17:39] – AlfredPerlstein | devsummit:201405:networkstack [2021/04/25 09:09] (current) – Fix DevSummit links. JethroNederhof | ||
|---|---|---|---|
| Line 66: | Line 66: | ||
| ==== Attending ==== | ==== Attending ==== | ||
| - | In order to attend you need register for the [[/ | + | In order to attend you need register for the [[/ |
| - | **Please do NOT add yourself here.** Your name will appear automatically once you have received the confirmation email. You need to put your name on the general [[/201405DevSummit|developer summit]] attendees list though. | + | **Please do NOT add yourself here.** Your name will appear automatically once you have received the confirmation email. You need to put your name on the general [[DevSummit/201405|developer summit]] attendees list though. |
| ^ Name ^ Username / Affiliation ^ Topics of Interest ^ Notes ^ | ^ Name ^ Username / Affiliation ^ Topics of Interest ^ Notes ^ | ||
| Line 212: | Line 212: | ||
| For one connection, something like large-receive offload would be similar (reducing the number of packets). Already implemented in some drivers. Batching would improve the case where traffic is received for multiple connections [or if the driver doesn’t have LRO or has limitations]. | For one connection, something like large-receive offload would be similar (reducing the number of packets). Already implemented in some drivers. Batching would improve the case where traffic is received for multiple connections [or if the driver doesn’t have LRO or has limitations]. | ||
| Has to be a two-way contract; there has to be a flag in the ifp that says “I will take packets with the batch flag sent” to ensure they will be processed (m_nextpkt not ignored). Or use a different entry point. | Has to be a two-way contract; there has to be a flag in the ifp that says “I will take packets with the batch flag sent” to ensure they will be processed (m_nextpkt not ignored). Or use a different entry point. | ||
| + | |||
| Parallelism. | Parallelism. | ||
| PR 183659. | PR 183659. | ||
| Line 242: | Line 243: | ||
| Question: Is it just a single listening socket, or multiple? | Question: Is it just a single listening socket, or multiple? | ||
| Have done it with both… | Have done it with both… | ||
| + | |||
| + | Sandvine: tcpinfo locking is too much, has ideas for reducing contention. | ||
| + | * code accepted: New lock for time_wait timers. | ||
| + | * provide detailed analysis: kern/ | ||
| + | * Switch use of read/write lock and locking the inp instead result: *60k -> 130k (200% performance improvement)* | ||
| + | * note: tested in synthetic workload, want more testing. | ||
| + | |||
| + | Patch not to take INP_INFO lock on input for SYN | ||
| + | * Lock order is INP_INFO then INP lock. | ||
| + | |||
| + | Aggressive caching of the route in the tcpcb. | ||
| + | * There are races, but they are “ok” as they will happen anyhow. | ||
| + | * Add a function to allow for plugging in different routing table algorithms. | ||
| + | |||
| Alternative FIBs. | Alternative FIBs. | ||
| Luigi is advocating for an approach which does aggressive caching and is not too bothered if there is wrong information for a while. | Luigi is advocating for an approach which does aggressive caching and is not too bothered if there is wrong information for a while. | ||
| Line 249: | Line 264: | ||
| TCP/UDP segmentation/ | TCP/UDP segmentation/ | ||
| We don’t have software TCP segmentation, | We don’t have software TCP segmentation, | ||
| - | Performance was doubled, without hardware support, by doing segmentation very low in the stack. (Student project.) | + | **Performance was doubled, without hardware support**, by doing segmentation very low in the stack, right before input into driver. (Student project.) |
| Linux calls this approach GSO, pushing large segments through the stack; the hardware can do segmentation if supported, otherwise we do it at the bottom layer. | Linux calls this approach GSO, pushing large segments through the stack; the hardware can do segmentation if supported, otherwise we do it at the bottom layer. | ||
| Simplifies TCP code since you can send arbitrarily large segments. | Simplifies TCP code since you can send arbitrarily large segments. | ||
| Line 256: | Line 271: | ||
| Question: How does this interact with the batching idea? | Question: How does this interact with the batching idea? | ||
| If you get a huge segment, and the NIC does not support it, translate the huge segment into a batch and send that. | If you get a huge segment, and the NIC does not support it, translate the huge segment into a batch and send that. | ||
| + | |||
| ifnet abstraction. | ifnet abstraction. | ||
| Compile-time option to let the procedural interface be inlined… | Compile-time option to let the procedural interface be inlined… | ||
| Line 263: | Line 279: | ||
| George can work to help get this taken care of. | George can work to help get this taken care of. | ||
| Andre’s changes [?] to the ifnet layer have stopped, no more churn. | Andre’s changes [?] to the ifnet layer have stopped, no more churn. | ||
| + | |||
| ifnet extensions for multiqueue/ | ifnet extensions for multiqueue/ | ||
| Linux has their standard ifnet interface, with a single pointer to the extensions; if the interface does not support them, the system still runs. If it does, have interfaces to configure numbers of queues, numbers of buffers, etc. | Linux has their standard ifnet interface, with a single pointer to the extensions; if the interface does not support them, the system still runs. If it does, have interfaces to configure numbers of queues, numbers of buffers, etc. | ||
| Line 273: | Line 290: | ||
| There will be %%FreeBSD%% reference drivers for new hardware. | There will be %%FreeBSD%% reference drivers for new hardware. | ||
| Question: Can we also move the buffering [which?] into this layer? | Question: Can we also move the buffering [which?] into this layer? | ||
| + | |||
| mbuf revisiting. | mbuf revisiting. | ||
| Ephemeral mbufs: Allocated by the caller, e.g. on the stack. | Ephemeral mbufs: Allocated by the caller, e.g. on the stack. | ||
| Line 284: | Line 302: | ||
| Several drivers have to do the same parsing to e.g. determine whether a packet is TCP and find various fields. | Several drivers have to do the same parsing to e.g. determine whether a packet is TCP and find various fields. | ||
| Luigi: Might add a huge number of bits to each packet. The issue is not so much the size as the time it takes to prepare. Is it really useful? | Luigi: Might add a huge number of bits to each packet. The issue is not so much the size as the time it takes to prepare. Is it really useful? | ||
| + | |||
| backpressure. | backpressure. | ||
| In Linux, if you stop and there is a packet in the queue, you notify the socket. In principle it is easy to do this. At end of transmission it might be tricky, it is interrupt context and might conflict with other processing being done on the socket itself. | In Linux, if you stop and there is a packet in the queue, you notify the socket. In principle it is easy to do this. At end of transmission it might be tricky, it is interrupt context and might conflict with other processing being done on the socket itself. | ||
| Line 293: | Line 312: | ||
| Comment (Scott?): In the storage world, we never fail an I/O at entry, we always queue it up (infinitely long queue if necessary). Don’t signal back up the stack with the resulting thundering herd problem. Rather, just process all I/O operations in order when hardware resources are available. | Comment (Scott?): In the storage world, we never fail an I/O at entry, we always queue it up (infinitely long queue if necessary). Don’t signal back up the stack with the resulting thundering herd problem. Rather, just process all I/O operations in order when hardware resources are available. | ||
| Navdeep: There is no hard cap on the amount of memory in the queue. | Navdeep: There is no hard cap on the amount of memory in the queue. | ||
| + | |||
| vlan %%QinQ%%. | vlan %%QinQ%%. | ||
| Encapsulating VLAN in VLAN. Is there interest in this? | Encapsulating VLAN in VLAN. Is there interest in this? | ||
| Line 301: | Line 321: | ||
| Also, some bits in VLAN are for priority. | Also, some bits in VLAN are for priority. | ||
| pfsense already has support for this, there is some overhead since it uses mbuf tags, but it works. | pfsense already has support for this, there is some overhead since it uses mbuf tags, but it works. | ||
| - | vxlan. | + | |
| bryan V: | bryan V: | ||
| * stack vlan interfaces. | * stack vlan interfaces. | ||
| Line 308: | Line 328: | ||
| * also has patch to enable bits for vlan priority. | * also has patch to enable bits for vlan priority. | ||
| * Suggestion from group: perhaps using virtual interface nesting makes it cleaner. | * Suggestion from group: perhaps using virtual interface nesting makes it cleaner. | ||
| + | |||
| + | vxlan. | ||
| lightweight refcounters. | lightweight refcounters. | ||