This project is about creating wrapper libraries to support monitoring and management applications to avoid direct use of kvm(3) and sysctl(3) interfaces. This approach would allow the kernel implementation to change and monitoring applications to be extended without breaking applications and requiring them to be recompiled. For this project, we propose to provide such a library for the network statistics functions (i.e. libnetstat(3)) as a proof of concept, together with methods for building similar libraries for other subsystems.
The goal of this project is to help creation of wrapper libraries to support monitoring and management applications to avoid direct use of the kvm(3) device and the sysctl(3) variables in the FreeBSD kernel. This would allow the kernel implementation to change and monitoring to be extended without breaking applications and requiring them to be recompiled.
The initial idea is based on Robert Watson's libmemstat(3) library, where he created an abstract interface for collecting and providing information on memory allocation statistics. Its design and development was inspired by the following needs:
struct inpcb (for Protocol Control Blocks) grows frequently as TCP gains new features, but the current sysctl(3) interface is not robust against that growth.For this project, we propose to provide a library for the network statistics functions (i.e. libnetstat(3)). Hopefully, it is just a start, lessons learned during the design of this implementation could be used for creating similar libraries for other subsystems as time goes on.
The reason why libnetstat(3) have been chosen, is that the network-related abstractions (e.g. “sockets”, “socket buffers”) are frequently monitored abstractions, with lot of FreeBSD and third-party tools wanting to show information about them.
There are three parts to be implemented:
For the implementation, we will need to find the abstractions most suitable for the selected applications. For example, the current libmemstat(3) API presents a set for memory types, and allows programs to query various statistics and properties on each type. In our opinion, the same could be implemented for the network statistics.
| Task | Status | Responsible | Description |
|---|---|---|---|
| PCB and listen queues abstractions | Done (to be reviewed) | pgj | Add abstractions for Active sockets (protocol control blocks) and listen queues (unaccepted connections, unaccepted incomplete connections, maximum queued connections) for each network protocol, for particular protocol family, for a single protocol. |
| NIF abstractions | Done (to be reviewed) | pgj | Add abstractions for network interfaces (bytes in/out, dropped packets, watchdog timers, errors, and collisions, packet traffic). |
| MBUF abstractions | Done (to be reviewed) | pgj | Add abstractions for statistics recorded by memory management routines (mbuf(9)). |
| BPF abstractions | Done (to be reviewed) | pgj | Add abstractions for bpf(4) peers (packets matched, dropped, received, buffer size, device states). |
| Routing abstractions | Done (to be reviewed) | pgj | Add abstractions for routing statistics for routing tables, or routing tables for particular address family. |
| MIF abstractions | Partially done (to be reviewed) | pgj | Add abstractions for IP multicast virtual-interfaces, routing tables, multicast group memberships, multicast routing statistics. |
| MIF sysctl(8) support | Done (to be reviewed) | pgj | Add sysctl(8) procedures for live monitoring multicast virtual-interfaces and routing tables. |
| Routing sysctl(8) support | In progress | aman | Add sysctl(8) procedure for live monitoring routing tables. |
| KNDS abstractions | Review in progress (pgj) | aman | Add abstractions for describing netisr(9)-related statistics. |
| netstat(1) adaptation | In progress | pgj | Adapt the netstat(1) utility to use libnetstat(3). |
| nettop(1) | In progress | pgj | Maintain a simple top(1)-like utility for live monitoring. |
| Versioning statistics | In progress | pgj | Introduce platform-independent versioning in networking statistics. |
| libnetstat(3) manual page | Not done | none | Create a manual page for libnetstat(3). |
| Cleaning up | Not done | none | There are some part to be dropped (i.e. not used), and simplified by using macros, for example. It think it would be beneficial to catch the skeleton of the common (allocator, iterator) routines and macrorify it (like in case of queue.h), merge some similar routines. |
| Support for other protocols | Not done | none | Take care of SCTP, Netgraph, IPX, and AppleTalk protocol stuff. |
| MIF export fix | Done | aman | Kernel patches for exporting mifs and mroutes are making witness(4) alerts. |
| Compilation fix | Done | pgj | libnetstat(3) compiles only with -O at the moment due to some pointer tricks. |
| net/stat.h | Done | pgj | Introduce a kernel-level netstat header to collect common declarations. |
| Task | Status | Responsible | Description |
|---|---|---|---|
| NIF sysctl(8) enumeration | In progress | aman | Enumerating network interfaces via sysctl(8) (similarly to ifmib(4)?). |
| 64-bit counters | Deferred | pgj | Add support for multi-platform atomic 64-bit counters for tracking network statistics properly. |
| SNMP support | Not done | none | Add SNMP support to libnetstat(3), so it could be used for remote hosts where bsnmpd(1) is running. |
| VIMAGE support | Not done | none | Respect virtualized network stacks, so they can be monitored independently (for example, by a -J jailname flag in netstat(1)). |
| KVM support | Not done | none | Add support for information extraction from KVM images in an ABI-independent way. |
The current version of the implementation can be found as unified diff against head.
Feel free to contact any or all of the contributors in the project.