PyCOOLR is a GUI able to plot generic events coming from tools. These events can be filtered to be plotted via metrics the user has to select.
PyCOOLR supports tools such as BEACON (Backplane for Event Control and Notification) and SOS (Scalable Observation System).

Usage:

./pycoolr -tool=<beacon|sos> -platform=<cerberus.nic.uregon.edu|...>

Once the GUI is started, the user can:
- Select the menu "Settings"
- Select the sub menu "Metrics"
- A window labelled "metrics" shows up, then the user can select some metrics from a given list

The GUI contains 6 graphs which can be allocated to the selected metrics. 
For each graph:
- the name of the metric is displayed on the top, as well as the name of the platform the GUI executes on (into brackets)
- X axis represents the time in seconds as units
- Y axis: specific to each metric
- On top left: the power of 10 (for BEACON)

The user also can modify the font size of the GUI:
- Select the menu "Settings"
- Select the sub menu "Fonts"

A window opens where the user can change the font policy and the size.

-------------------------------------

Some other features are specific to each tool PyCOOLR is connected to.

- BEACON: One interesting use case coming with BEACON is to couple runtime introspection and monitoring of MPI libraries, using the MPI Tools Interface (MPI-T). 
MPI-T allows exhibition of MPI internals to the user through Performance Variables PVARs. When coupled to TAU Performance Tools, BEACON proposes a set of PVARs as metrics the user can select, to be monitored. 
It can be done by selection the "Metrics" submenu. 
Also, the user can easily update a list of Control Variables (CVARs) to tune the MPI runtime: In the "Settings" menu, the user can select "CVARS" submenu. A window shows up, and the user can select some CVARs on a given list. 
He then can just update associated values and update them to the MPI library, via the TAU Performance tool. This can be done by the "Update" button. 

- SOS: PyCOOLR reads a database file generated by SOS, reads the events from the file and makes them available to be plotted

--------------------------------------

PyCOOLR reads configuration file where it picks some necessary information such as metrics.

- BEACON: The configuration file associated to BEACON includes a field named "cvars" containing a list a CVARs to be read by the GUI.
- SOS: The configuration file associated to SOS includes a field named "dbfile" containing the path to the database file to be read by PyCOOLR 

--------------------------------------

Use case with BEACON: runtime introspection with MPI-T

We consider here a use case where we can perform runtime introspection of an MPI library. 
- MPI library: MVAPICH2 with MPI-T support
- Application: NAS Parallel Benchmark Suite
- Performance tool: TAU

MPI-T has been shortly supported in MVAPICH2 MPI library. We instrument the NPB benchmark LU with the TAU Performance tool. 
TAU is able to collect the MPI-T PVARs from MVAPICH2 using the MPI-T API, and forward PVARs to BEACON.
PyCOOLR uses BEACON API to subscribe to these PVARs and plot them.

So we proceed this way (done on cerberus.nic.uoregon.edu platform):

1) MVAPICH2 with MPI-T support is installed with the following configure line:

a)   $ ./configure --enable-mpit-pvars=all --prefix=/home/users/aurelem/tau/mpi/mv2-mpit_inst --with-ibverbs-lib=/home/users/aurelem/tau/libs/libibverbs_inst/lib --with-ib-libpath=/home/users/aurelem/tau/libs/libibverbs_inst/lib --with-ib-include=/home/users/aurelem/tau/libs/libibverbs_inst/include --disable-mcast

2) BEACON is installed with the following way

a) In the BEACON source directory, we generate Makefiles based on Autotools Makefiles
[aurelem@cerberus BEACON]$ ./autogen.sh 

b) We install BEACON with the following configure line:
[aurelem@cerberus BEACON]$ ./configure --prefix=<BEACON_installation_path> --with-network-type=sock

[aurelem@cerberus BEACON]$ make install

3) TAU is installed this way:

a) Configure line:
[aurelem@cerberus tau2]$ ./configure  -pdt=/home/users/aurelem/tau/pdtoolkit-3.22.1 -mpi -c++=mpicxx -cc=mpicc -openmp -opari -bfd=download

b) 
[aurelem@cerberus tau2]$ make clean ; make install

4) We start BEACON daemons and set environment variables to be able to track MPI-T PVARs

 After properly set the environment variables (PATH and LD_LIBRARY_PATH), we start the BEACON daemons (server and node agent):
$ beacon_topology_setup_server
$ global_beacon

b) We set the environment variable to track MPI-T PVARs:
$ export TAU_TRACK_MPI_T_PVARS=1

c) We add paths of MVAPICH2 bin and lib directories to PATH and LD_LIBRARY_PATH environment variables 

5) We start the NPB LU benchmarks with TAU instrumentation, using an arbitrary number of 4 MPI processes

$ mpirun -np 4 tau_exec -T mvapich2,beacon,mpi,mpit ./lu.C.4

TAU then starts to collect PVARs and make publish them using BEACON API.

6) We then launch PyCOOLR which subscribes to the PVARs

[aurelem@cerberus bin]$ ./pycoolr -tool=beacon -platform=cerberus.nic.uoregon.edu

And when selecting metrics, we can see values of event plotted and updated on the graphs.
