rwatson's Network Stack TODO

Here are some of the things that have ended up on my todo list, kept here so that I don't forget them. And, in the hopefully unlikely event I am run over by a bus, perhaps someone else will find them useful.

• (IN PROGRESS) if_start needs to become if_startmbuf so that device drivers can either use central queue routines, or use their own queues (such as multi-queue for some ethernet devices, ppp, etc, or no queue for layering such as if_vlan). This will require adding a queue/priority field to the mbuf header, and updating current consumers of multiple queues between link and driver code, such as if_sl, 802.11, etc.
• Generally revisit if_flags, if_drvflags. Look especially hard at the IFF_LINK* flags, which have quite variable semantics and may need to be broken out themselves, as they are sometimes driver flags (and sometimes not).
• More ifaddr, protocol addr locking.
• General review of struct ifnet locking, ifnet API for device drivers, ifnet locking.
• TCPDEBUG appears not to have been updated for TCP timewait support, and therefore may return incorrect data. It needs to be updated.
• so_upcall needs to be broken out from a single upcall with ambiguous locking into several upcalls each with carefully documented locaking and semantics
• Review accept filter locking following so_upcall cleanup.
• The mbuma mbuf allocator fails to drain the mbuf+cluster cache back into component caches when resource limits are approached.
• Revisit UNIX domain socket fine-grained locking in rwatson_proto and decide what to do about it. Proper micro-benchmarking.
• Continue exploring socket locking strategy changes and vertical protocol lock integration in rwatson_resock and rwatson_resock_virtual.
• (DONE) sosend/soreceive/sopoll cleanup – all consumers call sosend/soreceive, and protocols use sosend_generic/soreceive_generic/sopoll_generic (renamed current ones) – sosend/soreceive/sopoll become simply pru_sosend/pru_soreceive/pru_sopoll wrappers.
• (IN PROGRESS) Move pru_* stub functions from uipc_socket2.c to uipc_domain.c (or something along these lines), move some other functions and variables to uipc_socket.c.
• (DONE) Move socket buffer utility routines from uipc_socket2.c to uipc_sockbuf.c via repo-copy.
• For each protosw switch entry, decide whether thread is needed, or if ucred would be sufficient.
• Investigate moving to optimistic locking of accept mutex to avoid frequent accept mutex acquires when not needed.
• socket(9) man page for kernel socket APIs.
• protosw(9) man page for protocol switch APIs.
• Revisit multi-mbuf allocation, multi-mbuf input.
• (DONE) Trim aging mbuf.h types.
• Review read vs. write lock macros for pcbinfo locking, and look at effects of using an rwlock instead of a mutex. Warning: the macro use has never been exercised with shared locking, so may have serious bugs.
• Investigate solutions to inpcb tear-down races involving timers with tcp – right now, timers are stopped but not drained, so timers can race with tear-down, hence NULL checks in timer code.
• (DONE) Create sbdestroy() with a simplified and cleaned up set of socket buffer cleanups, avoiding calls to locking and sorflush().
• Review and clean up sorflush(). Investigate reducing overhead, as this is invoked by a number of apps when they shutdown() a socket for read.
• Revisit locking and atomicity for uipc_send(), and in particular, with respect to what happens if two threads simultaneously call sendto() on different addresses with the same datagram socket. Currently this is likely prevented by sblock() at the system call layer, but races with connect() aren't, as was shown by the recent race case along the same lines. Somehow this needs to be serialized.
• Review Intel TCP offload patches.
• inpcb freeing support so that pcb's can be freed after a load spike, requires moving away from weak consistency sysctl monitoring.
• Continue pushing of control of state changes from socket layer to protocol layer so as to improve inter-layer consistency and reduce races between layers on socket state transitions (connect/disconnect/connecting/disconnecting/…)
• Analysis of listen state transitions, which appear poorly defined.
• Analyze and fix mbuf allocator problem relating to the mbuf+cluster cache growing out of control (with andre).
• SO_NOSIGPIPE appears to be broken again.