pmc(3) is a performance measurement framework for FreeBSD which uses the hardware performance measurement counters in modern CPUs to provide low-overhead, non-intrusive, system-wide measurement of production systems.

The project consists in creating a graphical user interface (GUI) analysis tool. It will be based on the performance monitoring counters framework pmc(3) and pmclog(3), using in-cpu measurement counters. The PMC GUI will allow a visual detection of performance bottlenecks in the execution of a program.

Management / welcome screen: The user associates PMCs (measurements) and processes. A set of allowed measurements i.e. according to the CPU - will be proposed to the user. Target processes can be selected by mouse.

Display/monitoring: The user can see in realtime what is happening. Profiling will concern selected processes, threads, and children or descendant processes. Spots/rectangles will correspond to a code segment. There will be a time view of the spots, with 1-2-3D graphes and color codes. Eventually, an interactive link to the function or code of the program should be implemented.

It is committed with the Enlightenment 0.17 libraries, that are BSD licensed. Lastest sources can be obtained by

cvs -z3 -d :pserver:anonymous@anoncvs.enlightenment.org:/var/cvs/e co e17

The following APIs can be found in the directory freshly created e17/libs:

• Imlib2
• Eet
• Evas
• Ecore
• Edje
• Epeg
• Epsilon
• Ewl
• Etk
• Emotion
• Engrave
• Efreet

However Ecore and Evas are the only libraries necessary to implement a UI. Edje is a theming library. It combines complex groups of evas objects into a single evas ‘smart object’. This Edje_Object can be placed on the canvas and used in almost the exact same manner as the primitive Evas objects. It also handles animation.

Evas

Evas is a canvas library that handles all the low level graphics work. Rather than painting pixels yourself, you simply define a set of Evas_Objects such as rectangles, images and gradients. Tell Evas where you want these object to be located, set their color, transparency and stacking, and Evas does all the calulating to blend these together into a single image, ready for display on the screen. Evas includes several engines that output to different display environments. (Software_X11, OpenGL_X11, DirectFB, Buffer, etc).

Ecore

Ecore is really a set of small libraries. It handles a lot of the guts of the EFL; things like Events, Timers and IPC. It also includes Ecore_Evas, which wraps creation of an Evas and a window to put it in into a simple API.

In e17/apps several applications can be found but many tricks and patches are needed to put these application to life. Not all libraries are in the FreeBSD port system, are outdated and, in particular, there is no Python bindings to the APIs.

Download the package (not yet a port) at ?, build and install the UI by:

../autogen.sh; make; make install

To allow the UI to work,you need the following ports:

and you need the following kernel configuration:

Optionally, you can also have:

First, suggestions are welcome. For this, write to me at mprevot@freebsd.org. Many things need to be done yet. Find here a TODO list and ideas for your involvement.

* make a port * update dependencies, ie. APIs on which the UI depends * make a documentation *

* test UI schemes, provide feedback and, suggest new schemes * report bugs

Work flow #1 ~~~~~~~~~~~~

A user wants see the current behaviour of the CPUs of a system; for the sake of the example let us assume that she wants to see the behaviour of three hardware events: “instructions retired”, “branch mispredicts”, and “L1 cache misses” over time.

- She uses the GUI's menus to allocate system-wide, counting PMCs for each of these events. The GUI tool assists her by guiding her through the process:

- It allows her to specify the event being counted. This could be by choosing from a set of cascaded and suitably organized menus, or by presenting a text field where she can type in an event-name, or a combination of the two. The tool also helps her to choose additional modifiers for the event (as supported by pmc(3)), that can be specified for the CPU she is using.

- It allows her to choose which CPUs each such event is to be counted on. She may want specific CPUs to be counting specific events.

- It allows her to choose the display update frequency (once per second, or other). This could be using a suitable 'counter' widget or a slider or other suitable way.

- The tool would allow her to see the current allocation of PMCs and events at any time. This information could include for example, the event name selected, modifiers chosen, the CPU number the event is being counted on, display update frequency, whether the allocated PMC is currently active, and the most recently read value.

- The tool would provide her with a way to start and stop specific PMC. It would also allow her to start and stop all allocated PMCs at once.

- The display would be in the form of a graph with a user selectable display of statistical information such as maximum and minimum values, standard deviation, coefficients of correlation etc.

- Possible graphical views could include:

- A line graph of absolute values of event counts versus time.

- A way to compare event counts on different CPUs (e.g., to see if one CPU is consistently seeing a different event pattern than the other).

- A way to see 'deltas' between successive readings of an event; useful to track small changes.

- For multiple events that are being displayed simultaneously, these graphs could be presented as a set of individual graphs stacked horizontally or vertically. Alternately, a “pseudo 3D” view that allows all values to be seen once, say as a mesh rendered in 3D, would also be useful.

- The GUI should allow specific segments of the graph to be zoomed into for a closer look or a wider time period to be seen by zooming out.

Workflow #2 ~~~~~~~~~~~

The user has a hwpmc(4) log file collected by a pmcstat(8) sampling run. He wants to analyse the behaviour of the system.

- The tool would assist in opening the log file using a conventional sequence of dialogs and menus (File→Open Log→File chooser dialog etc.).

- From the information in the log the tool would display the PMC events that had been allocated. This could be by means of a 'Log Information' dialog that also has statistics such as the total number of samples seen, total number of processes fork()'ed and exec()'ed etc.

- The tool would represent the time period of the log in the form of a “time line”. Various operations (see below) would use the time line.

- From the information in the log the tool would display a `process tree' when asked, showing the processes (pids) that were mentioned in the logs and the children they fork()'ed.

- From the information in the log the tool would display all executable files associated with each process address space seen in the log (i.e., main executable, shared objects opened) etc.

- The tool would show graphically the following views:

- A 'hotspot' view of the entire system. Each executable would be represented as a horizontal bar covering its allocated address space. The bar would be colored with different colors showing the frequency of the samples per 'bucket' of address range.

A crude ASCII representation is below:

:EXECUTABLE NAME: :HOTSPOT GRAPH:

/boot/kernel/kernel | +++**** *** *** | /usr/bin/make | ..+**+… …***..**.. …***..| .. etc …

Here a ' ' shows an area of address space where there were no samples. A '+' shows an area with a light color denoting that a small number of samples fell into that region of address space. A '*' shows an area of address space in which there were a large number of samples and is hence colored with a more forceful hue.

This list would be sorted with the executable object with the most samples topmost. Other sorting orders (e.g., by executable name, or wall clock time of process creation) should also be supported.

Mousing over a specific hot spot should show the function name that had the most number of samples in that hot spot. This could be via a tooltip or in a specific 'reporting pane' of the tool.

The hotspot view should supported 'zooming in' into specific areas of each executable. This expands the view and allows finer grainer display.

- A source code view could be associated with a hotspot view.

Clicking on a hotspot would result in the source code for the associated function being displayed in a source code pane, with the regions of code in which the samples fell marked to indicate their frequency. To aid readability the marking could be using an appropriately colored bar to the side of the displayed text, or alternately the marking could be achieved by changing the foreground or background colors of the text.

When multiple PMCs were active, this information could be displayed by showing one such hotspot bar per executable per PMC. Different colors could be used for different PMCs to aid readability.

- When multiple CPUs were used, the tool should allow filtering to be done on a per CPU basis. For example to display a comparision of hotspots on a per-cpu basis. This would be useful for objects that execute on multiple CPUs concurrently; e.g. the kernel and multi-threaded applications.

- The tool would allow filtering of the displayed information to specific executables (e.g., “show me how /bin/sh behaved”, or “what are the hotspots in /lib/libc.so”).

- The tool would allow filtering based on processes and their children (e.g., “show me the sample for this given PID”, and “show me the behaviour of this process and its descendants”).

- The tool would allow specific segments of the log file to be viewed (i.e., narrowing the views to between a specified begining and ending of the collected information). The displayed 'hotspot' view and 'process tree' information would then update to show processes and pids that had samples recorded differently, perhaps by graying out the others.

- The tool would allow a time period of a given width (say 10s) to be 'swept' through the time line, animating the behaviour of the hotspots.

Workflow #3 ~~~~~~~~~~~

Like workflow #1, the user selects PMCs/events etc. and starts them. However these are sampling PMCs so the user can see a 'live' display of collected sampling information as in Workflow #2.

The data should optionally also be saved in more permanent store for subsequent offline analysis.

Workflow #4 ~~~~~~~~~~~

Like workflow #2 except that the hwpmc(4) log is read over a TCP connection.

The tool would have a slave portion that runs on the remote system and that allows control of PMCs there. The tool itself should act a network server to receive this feed.

Like in workflow #3, there should be a way to optionally save the incoming data for offline analysis.

Workflow #5 ~~~~~~~~~~~

Like workflow #1 and #3 the user selects specific PMCs (counting and/or sampling), but allocates these in 'process mode'. These are subsequently attached to a specific commandline that gets executed. The information gathered is displayed as in #1 and #2.

Workflow #6 ~~~~~~~~~~~

The tool shows the current set of 'live' processes in a manner similar to 'top(1)'.

Then like in #5, the user allocates a process-mode counting or sampling PMC and “attaches” it to one of the 'live' processes on the system. The information collected is displayed as in #1 or #2.