libnetstat

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
libnetstat [2010/05/03 19:52] – Link unified diff format distribution. PáliGáborJánoslibnetstat [2021/03/28 07:36] (current) – Remove Perforce repo link (NXDOMAIN) KubilayKocak
Line 1: Line 1:
 ==== Abstract ==== ==== Abstract ====
  
-This project is about creating wrapper libraries to support monitoring and management applications to avoid direct use of the %%FreeBSD%% kernel memory and sysctl(8) interface.  This approach would allow the kernel implementation to change and monitoring applications to be extended without breaking applications and requiring them to be recompiled.  For this project, we propose to provide such a library for the network statistics functions (i.e. //libnetstat(3)//) as a proof of concept, together with methods for building similar libraries for other subsystems.+This project is about creating wrapper libraries to support monitoring and management applications to avoid direct use of kvm(3) and sysctl(3) interfaces.  This approach would allow the kernel implementation to change and monitoring applications to be extended without breaking applications and requiring them to be recompiled.  For this project, we propose to provide such a library for the network statistics functions (i.e. //libnetstat(3)//) as a proof of concept, together with methods for building similar libraries for other subsystems.
  
 ==== Description ==== ==== Description ====
  
-The goal of this project is to help creation of wrapper libraries to support monitoring and management applications to avoid direct use of the kvm(3) device and the sysctl(8) variables in the %%FreeBSD%% kernel.  This would allow the kernel implementation to //change// and monitoring to be extended //without breaking applications// and requiring them to be //recompiled//.+The goal of this project is to help creation of wrapper libraries to support monitoring and management applications to avoid direct use of the kvm(3) device and the sysctl(3) variables in the %%FreeBSD%% kernel.  This would allow the kernel implementation to //change// and monitoring to be extended //without breaking applications// and requiring them to be //recompiled//.
  
 The [[http://unix.derkeiler.com/Mailing-Lists/FreeBSD/current/2005-07/0535.html|initial idea]] is based on Robert Watson's libmemstat(3) library, where he created an abstract interface for collecting and providing information on memory allocation statistics.  Its design and development was inspired by the following needs: The [[http://unix.derkeiler.com/Mailing-Lists/FreeBSD/current/2005-07/0535.html|initial idea]] is based on Robert Watson's libmemstat(3) library, where he created an abstract interface for collecting and providing information on memory allocation statistics.  Its design and development was inspired by the following needs:
  
-  * //Robustness for Application Binary Interface (ABI) over time.//  A library interface, rather than a system call, sysctl(8) variable, or kernel memory interface is the documented interface, and is designed to be robust against common changes to kernel data structures.  For example, the ''%%struct inpcb%%'' (for Protocol Control Blocks) grows frequently as TCP gains new features, but the current sysctl(8) interface is not robust against that growth.+  * //Robustness for Application Binary Interface (ABI) over time.//  A library interface, rather than a system call, sysctl(8) variable, or kernel memory interface is the documented interface, and is designed to be robust against common changes to kernel data structures.  For example, the ''%%struct inpcb%%'' (for Protocol Control Blocks) grows frequently as TCP gains new features, but the current sysctl(3) interface is not robust against that growth.
  
   * //Robustness for different word sizes.//  32-bit monitoring tools should be able to work on a 64-bit kernel.   * //Robustness for different word sizes.//  32-bit monitoring tools should be able to work on a 64-bit kernel.
Line 23: Line 23:
 There are three parts to be implemented: There are three parts to be implemented:
  
-  * Add kernel support for exporting data in a less ABI-sensitive way, with a focus on live monitoring via the sysctl(8) interface.+  * Add kernel support for exporting data in a less ABI-sensitive way, with a focus on live monitoring via the sysctl(3) interface.
  
-  * Write a library to present the information in an extensible way to applications.  This is //libnetstat(3)//.  It is derived from the sources of netstat(1) with the missing kvm(3) and sysctl(8) interfaces added where needed.+  * Write a library to present the information in an extensible way to applications.  This is //libnetstat(3)//.  It is derived from the sources of netstat(1) with the missing kvm(3) and sysctl(3) interfaces added where needed.
  
   * Update applications to use this library instead of reaching directly into the kernel internals.  For now, two applications were selected: netstat(1) and bsnmpd(1).   * Update applications to use this library instead of reaching directly into the kernel internals.  For now, two applications were selected: netstat(1) and bsnmpd(1).
Line 32: Line 32:
  
 ==== Project Tasks ==== ==== Project Tasks ====
 +
 +=== Old Features ===
  
 ^  Task  ^  Status  ^  Responsible  ^  Description  ^ ^  Task  ^  Status  ^  Responsible  ^  Description  ^
Line 40: Line 42:
 | Routing abstractions | Done (to be reviewed) | pgj | Add abstractions for routing statistics for routing tables, or routing tables for particular address family. | | Routing abstractions | Done (to be reviewed) | pgj | Add abstractions for routing statistics for routing tables, or routing tables for particular address family. |
 | MIF abstractions | Partially done (to be reviewed) | pgj | Add abstractions for IP multicast virtual-interfaces, routing tables, multicast group memberships, multicast routing statistics. | | MIF abstractions | Partially done (to be reviewed) | pgj | Add abstractions for IP multicast virtual-interfaces, routing tables, multicast group memberships, multicast routing statistics. |
-| MIF sysctl(8) support | Almost done (to be reviewed) | aman | Add sysctl(8) procedures for live monitoring multicast virtual-interfaces and routing tables. |+| MIF sysctl(8) support | Done (to be reviewed) | pgj | Add sysctl(8) procedures for live monitoring multicast virtual-interfaces and routing tables. |
 | Routing sysctl(8) support | In progress | aman | Add sysctl(8) procedure for live monitoring routing tables. | | Routing sysctl(8) support | In progress | aman | Add sysctl(8) procedure for live monitoring routing tables. |
 +| KNDS abstractions | Review in progress (pgj) | aman | Add abstractions for describing netisr(9)-related statistics. |
 | netstat(1) adaptation | In progress | pgj | Adapt the netstat(1) utility to use libnetstat(3). | | netstat(1) adaptation | In progress | pgj | Adapt the netstat(1) utility to use libnetstat(3). |
 | nettop(1) | In progress | pgj | Maintain a simple top(1)-like utility for live monitoring. | | nettop(1) | In progress | pgj | Maintain a simple top(1)-like utility for live monitoring. |
 | Versioning statistics | In progress | pgj | Introduce platform-independent versioning in networking statistics. | | Versioning statistics | In progress | pgj | Introduce platform-independent versioning in networking statistics. |
-| libnetstat(3) manual page | In progress | nms | Create a manual page for libnetstat(3). |+| libnetstat(3) manual page | Not done | //none// | Create a manual page for libnetstat(3). |
 | Cleaning up | Not done | //none// | There are some part to be dropped (i.e. not used), and simplified by using macros, for example.  It think it would be beneficial to catch the skeleton of the common (allocator, iterator) routines and macrorify it (like in case of queue.h), merge some similar routines. | | Cleaning up | Not done | //none// | There are some part to be dropped (i.e. not used), and simplified by using macros, for example.  It think it would be beneficial to catch the skeleton of the common (allocator, iterator) routines and macrorify it (like in case of queue.h), merge some similar routines. |
 | Support for other protocols | Not done | //none// | Take care of SCTP, Netgraph, IPX, and %%AppleTalk%% protocol stuff. | | Support for other protocols | Not done | //none// | Take care of SCTP, Netgraph, IPX, and %%AppleTalk%% protocol stuff. |
 +| MIF export fix | Done | aman | Kernel patches for exporting mifs and mroutes are making witness(4) alerts. |
 +| Compilation fix | Done | pgj | libnetstat(3) compiles only with //-O// at the moment due to some pointer tricks. |
 +| net/stat.h | Done | pgj | Introduce a kernel-level netstat header to collect common declarations. |
 +
 +=== New Features ===
 +
 +^  Task  ^  Status  ^  Responsible  ^  Description  ^
 | NIF sysctl(8) enumeration | In progress | aman | Enumerating network interfaces via sysctl(8) (similarly to ifmib(4)?). | | NIF sysctl(8) enumeration | In progress | aman | Enumerating network interfaces via sysctl(8) (similarly to ifmib(4)?). |
-| Handling streamed variables | Not done | //none// | Decide how to handle the streamed sysctl variables (e.g. net.inet.tcp.spcblist): replace the original variables (should we maintain compatibility?) or create a netstat.ko with a own mibs (like netstat.inet.tcp.pcblist)? | +| 64-bit counters | Deferred | pgj | Add support for multi-platform atomic 64-bit counters for tracking network statistics properly. |
-| MIF export fix | In progress | aman | Kernel patches for exporting mifs and mroutes are making witness(4) alerts. | +
-| Compilation fix | In progress | pgj | libnetstat(3) compiles only with //-O// at the moment due to some pointer tricks. |+
 | SNMP support | Not done | //none// | Add SNMP support to //libnetstat(3)//, so it could be used for remote hosts where bsnmpd(1) is running. | | SNMP support | Not done | //none// | Add SNMP support to //libnetstat(3)//, so it could be used for remote hosts where bsnmpd(1) is running. |
 | VIMAGE support | Not done | //none// | Respect virtualized network stacks, so they can be monitored independently (for example, by a -J //jailname// flag in netstat(1%%))%%. | | VIMAGE support | Not done | //none// | Respect virtualized network stacks, so they can be monitored independently (for example, by a -J //jailname// flag in netstat(1%%))%%. |
Line 58: Line 66:
 ==== Code ==== ==== Code ====
  
-The current version of the implementation can be found in the project's [[http://p4db.freebsd.org/changeList.cgi?FSPC=%2F%2Fdepot%2Fprojects%2Fsoc2009%2Fpgj_libstat%2F...|Perforce depot]] or as [[http://people.freebsd.org/~pgj/libnetstat/|unified diff against head]].+The current version of the implementation can be found as [[http://people.freebsd.org/~pgj/libnetstat/|unified diff against head]].
  
 ==== Contact ==== ==== Contact ====
Line 64: Line 72:
 Feel free to contact any or all of the contributors in the project. Feel free to contact any or all of the contributors in the project.
  
-  * Robert Watson (initial design, rwatson@freebsd.org) +  * RobertWatson (initial design) 
-  * Gábor Páli (initial implementation (as part of [[PGJSoC2009|Summer of Code 2009]]), effort coordination, pgj@freebsd.org) +  * PáliGáborJános (initial implementation, part of [[PGJSoC2009|Summer of Code 2009]]), coordination) 
-  * Aman Jassal (implementation, aman@freebsd.org) +  * AmanJassal (implementation) <aman@freebsd.org> 
-  * Mark Nesterovych (documentation, nms@freebsd.org)+ 
 +==== Past Contributors ====
  
-We usually have regular weekly meetings on #freebsd-libnetstat @ [[http://freenode.net/|freenode]] on every Saturday, starting from 17:00 UTC.+  * MarkNesterovych
  
  • libnetstat.1272916331.txt.gz
  • Last modified: 2010/05/03 19:52
  • by PáliGáborJános