Tutorial

Getting Started

Profiling Applications allows the user to identify bottlenecks in the application and get function profiling information for templated functions. Generating a profile can show which modules are most time intensive. This section covers installation of the TAU Portable Profiling Package.

Settings

After extracting the files from the distribution in the tau directory, proceed to configure the TAU profiling package and installing it.

Normally, you just `cd' to the directory containing the package's source code and type ./configure. Running `configure' takes a minute or two. While it is running, it prints some messages that tell what it is doing. configure takes the following optional arguments: 



% ./configure -help

TAU Configuration Utility 
***********************************************************************
Usage: configure [OPTIONS]
  where [OPTIONS] are:
-c++=<compiler>  ............................ specify the C++ compiler.
  options [CC|KCC|g++|xlC|cxx|NCC|egcs|pgCC|FCC|guidec++|aCC|c++|ecpc].
-cc=<compiler> ................................ specify the C compiler.
                              options [cc|gcc|KCC|pgcc|guidec|xlc|ecc].
-pdt_c++=<compiler>  ............ specify a different PDT C++ compiler.
 options [CC|KCC|g++|xlC|cxx|egcs|pgCC|FCC|guidec++|aCC|c++|ecpc|icpc].
-fortran=<compiler> ..................... specify the Fortran compiler.
   options    [gnu|sgi|ibm|ibm64|hp|cray|pgi|absoft|fujitsu|sun|compaq|
                                                   kai|nec|hitachi|intel]
-useropt='<parameters>' ............... list of commandline parameters.
-pthread .................................. Use pthread thread package.
-sproc .................................. Use SGI sproc thread package.
-tulipthread=<dir> .......... Specify location of Tulip/Smarts package.
-smarts .................. Use SMARTS API for threads (use with above).
-openmp ........................................... Use OpenMP threads.
-opari=<dir>... Specify location of Opari OpenMP tool (use with above).
-opari_region ......... Report performance data for all OpenMP regions.
-opari_construct ... Report performance data for all OpenMP constructs.
-pcl=<dir> ..... Specify location of PCL (Performance Counter Library).
-papi=<dir> ............... Specify location of PAPI (Performance API).
-pdt=<dir> ........ Specify location of PDT (Program Database Toolkit).
-jdk=<dir> ...... Specify location of JAVA 2 Development Kit (jdk1.2+).
-dyninst=<dir> ................... Specify location of DynInst Package.
-mpi .......................... Specify use of TAU MPI wrapper library.
-mpiinc=<dir> ............. Specify location of MPI include dir and use
                           the TAU MPI Profiling and Tracing Interface.
-mpilib=<dir> ............. Specify location of MPI library dir and use
                           the TAU MPI Profiling and Tracing Interface.
-mpilibrary=<library> ................ Specify a different MPI library.
            e.g., -mpilibrary=-lmpi_r                                  
-epilog=<dir>  ............ Specify location of EPILOG Tracing package.
-TRACE ..................................... Generate TAU event traces.
-PROFILE ............ Generate profiles (summary statistics) (default).
-PROFILECALLPATH ......................... Generate call path profiles.
-PROFILESTATS .................. Enable standard deviation calculation.
-MULTIPLECOUNTERS ............ Use multiple hardware counters and time.
-SGITIMERS .......... Use fast nanosecond timers on SGI R10000 systems.
-LINUXTIMERS ......... Use low overhead TSC Counter for wallclock time.
-CPUTIME .......... Use usertime+system time instead of wallclock time.
-PAPIWALLCLOCK ........ Use PAPI to access wallclock time. Needs -papi.
-PAPIVIRTUAL   .......... Use PAPI for virtual (user) time calculation.
-noex .................. Use no exceptions while compiling the library.
-help ...................................... display this help message.
***********************************************************************
Note: By default if 
   % ./configure
is invoked, it uses wall-clock time based profiling.

The following command-line options are available to configure: 

-prefix=<directory>
Specifies the destination directory where the header, library and binary files are copied. By default, these are copied to subdirectories <arch>/bin and <arch>/lib in the TAU root directory. 
-arch=<architecture>
Specifies the architecture. If the user does not specify this option, configure determines the architecture. For SGI, the user can specify either of sgi32, sgin32 or sgi64 for 32, n32 or 64 bit compilation modes respectively. The files are installed in the <architecture>/bin and <architecture>/lib directories. 
-c++=<C++ compiler>
Specifies the name of the C++ compiler. Supported C++ compilers include KCC, (from KAI/Intel), CC, g++ (from GNU), FCC (from Fujitsu), xlC(from IBM), guidec++ (from KAI/Intel), aCC (from HP), c++ (from Apple, NEC), pgCC (from PGI), and ecpc/icpc (from Intel). 
-cc=<C Compiler>
Specifies the name of the C compiler. Supported C compilers include cc, gcc (from GNU), pgcc (from PGI), fcc (from Fujitsu), xlc (from IBM), ecc/icc (from Intel) and KCC (from KAI/Intel). 
-pdt_c++=<C++ Compiler>
Specifies a different C++ compiler for PDT (tau_instrumentor). This is typically used when the library is compiled with a C++ compiler (specified with -c++) and the tau_instrumentor is compiled with a different <pdt_c++> compiler. For e.g., 
 -c++=pgCC -cc=pgcc -pdt_c++=KCC -openmp ...
uses PGI's OpenMP compilers for TAU's library and KCC for tau_instrumentor. 
-fortran=<Fortran Compiler>
Specifies the name of the Fortran90 compiler. Valid options are: gnu, sgi, ibm, ibm64, hp, cray, pgi, absoft, fujitsu, sun, compaq, nec, hitachi and kai. 
-pthread
Specifies pthread as the thread package to be used. In the default mode, no thread package is used. 
-tulipthread=<directory>
Specifies Tulip threads (HPC++) as the threads package to be used as well as the location of the root directory where the package is installed. [ Ref: http://www.acl.lanl.gov/tulip ] 
-tulipthread=<directory> -smarts
Specifies SMARTS (Shared Memory Asynchronous Runtime System) as the threads package to be used. <directory> gives the location of the SMARTS root directory. [ Ref: http://www.acl.lanl.gov/smarts ] 
-openmp
Specifies OpenMP as the threads package to be used. [ Ref: http://www.openmp.org ] 
-opari=<dir>
Specifies the location of the Opari OpenMP directive rewriting tool. The use of Opari source-to-source instrumentor in conjunction with TAU exposes OpenMP events for instrumentation. See examples/opari directory. [ Ref: http://www.fz-juelich.de/zam/kojak/opari/ ] 
-opari_region
Report performance data for only OpenMP regions and not constructs. By default, both regions and constructs are profiled with Opari. 
-opari_construct
Report performance data for only OpenMP constructs and not regions. By default, both regions and constructs are profiled with Opari. 
-pdt=<directory>
Specifies the location of the installed PDT (Program Database Toolkit) root directory. PDT is used to build tau_instrumentor, a C++, C and F90 instrumentation program that automatically inserts TAU annotations in the source code. If PDT is configured with a subdirectory option (-compdir=<opt>) then TAU can be configured with the same option by specifying -pdt=<dir> -pdtcompdir=<opt>. [ Ref: http://www.acl.lanl.gov/pdtoolkit ] 
-pcl=<directory>
Specifies the location of the installed PCL (Performance Counter Library) root directory. PCL provides a common interface to access hardware performance counters on modern microprocessors. The library supports Sun UltraSparc I/II, PowerPC 604e under AIX, MIPS R10000/12000 under IRIX, Compaq Alpha 21164, 21264 under Tru64Unix and Cray Unicos (T3E) and the Intel Pentium family of microprocessors under Linux. This option specifies the use of hardware performance counters for profiling (instead of time). To measure floating point instructions, set the environment variable PCL_EVENT to PCL_FP_INSTR (for example). Refer to the TAU User's Guide or PCL Documentation (pcl.h) for other event names. [ Ref : http://www.fz-juelich.de/zam/PCL ] 
-papi=<directory>
Specifies the location of the installed PAPI (Performance API) root directory. PAPI specifies a standard application programming interface (API) for accessing hardware performance counters available on most modern microprocessors similar. To measure floating point instructions, set the environment variable PAPI_EVENT to PAPI_FP_INS (for example). Refer to the TAU User's Guide or PAPI Documentation for other event names. [ Ref : http://icl.cs.utk.edu/projects/papi/api/ ] 
-jdk=<directory>
Specifies the location of the Java 2 development kit (jdk1.2+). See README.JAVA on instructions on using TAU with Java 2 applications. This option should only be used for configuring TAU to use JVMPI for profiling and tracing of Java applications. It should not be used for configuring jracy, which uses java from the user's path. 
-dyninst=<directory>
Specifies the location of the DynInst (dynamic instrumentation) package. See README.DYNINST for instructions on using TAU with DynInstAPI for binary runtime instrumentation (instead of manual instrumentation). [ Ref: http://www.dyninst.org ] 
-mpiinc=<dir>
Specifies the directory where mpi header files reside (such as mpi.h and mpif.h). This option also generates the TAU MPI wrapper library that instruments MPI routines using the MPI Profiling Interface. See the examples/NPB2.3/config/make.def file for its usage with Fortran and MPI programs and examples/pi/Makefile for a C++ example that uses MPI. 
-mpilib=<dir>
Specifies the directory where mpi library files reside. This option should be used in conjunction with the -mpiinc=<dir> option to generate the TAU MPI wrapper library. 
-mpilibrary=<lib>
Specifies the use of a different MPI library. By default, TAU uses -lmpi or -lmpich as the MPI library. This option allows the user to specify another library. e.g., -mpilibrary=-lmpi_r for specifying a thread-safe MPI library. 
-epilog=<dir>
Specifies the directory where the EPILOG tracing package [FZJ] is installed. This option should be used in conjunction with the -TRACE option to generate binary EPILOG traces (instead of binary TAU traces). EPILOG traces can then be used with other tools such as EXPERT. EPILOG comes with its own implementation of the MPI wrapper library and the POMP library used with Opari. Using option overrides TAU's libraries for MPI, and OpenMP. 
-PROFILE
This is the default option; it specifies summary profile files to be generated at the end of execution. Profiling generates aggregate statistics (such as the total time spent in routines and statements), and can be used in conjunction with the profile browser jracy to analyse the performance. Wallclock time is used for profiling program entities. 
-PROFILECALLPATH
This option generates call path profiles which shows the time spent in a routine when it is called by another routine in the calling path. "a => b" stands for the time spent in routine "b" when it is invoked by routine "a". This option is an extension of -PROFILE, the default profiling option. See Call Path Profiling for more information. 
-PROFILESTATS
Specifies the calculation of additional statistics, such as the standard deviation of the exclusive time/counts spent in each profiled block. This option is an extension of -PROFILE, the default profiling option. 
-PROFILECOUNTERS
Specifies use of hardware performance counters for profiling under IRIX using the SGI R10000 perfex counter access interface. The use of this option is deprecated in favor of the -pcl=<dir> and -papi=<dir> options described above. 
-MULTIPLECOUNTERS
Allows TAU to track more than one quantity (multiple hardware counters, CPU time, wallclock time, etc.) Configure with other options such as -papi=<dir>, -pcl=<dir>, -LINUXTIMERS, -SGITIMERS, -CPUTIME, -PAPIVIRTUAL, etc. See examples/multicounters/README file for detailed instructions on setting the environment variables for this option. If -MULTIPLECOUNTERS is used with the -TRACE option, tracing employs the COUNTER1 variable for wallclock time. 
-SGITIMERS
Specifies use of the free running nano-second resolution on-chip timer on the MIPS R10000. This timer has a lower overhead than the default timer on SGI, and is recommended for SGIs. 
-CPUTIME
Uses usertime + system time instead of wallclock time. It gives the CPU time spent in the routines. This currently works only on LINUX systems for multi-threaded programs and on all systems for single-threaded programs. 
-PAPIWALLCLOCK
Uses PAPI (must specify -papi=<dir> also) to access high resolution CPU timers for wallclock time. The default case uses gettimeofday() which has a higher overhead than this. 
-PAPIVIRTUAL
Uses PAPI (must specify -papi=<dir> also) to access process virtual time. This represents the user time for measurements. 
-TRACE
Generates event-trace logs, rather than summary profiles. Traces show when and where an event occurred, in terms of the location in the source code and the process that executed it. Traces can be merged and converted using tau_merge and tau_convert utilities respectively, and visualized using Vampir, a commercial trace visualization tool [ Ref http://www.pallas.de ] or Paraver [ CEPBA]. 
-noex
Specifies that no exceptions be used while compiling the library. This is relevant for C++. 
-useropt=<options-list>
Specifies additional user options such as -g or -I. For multiple options, the options list should be enclosed in a single quote. 
-help
Lists all the available configure options and quits. Examples: 
   % ./configure -c++=KCC
Use TAU with KCC 
   % ./configure -c++=CC -useropt='-g -I/local/apps/STL/'
Use TAU with SGI CC and add the above user defined options to the commandline. 
   % ./configure -TRACE -PROFILE
Enable both profiling and tracing. 
   % ./configure -c++=KCC -SGITIMERS -tulipthread=/home/smarts/build/smarts-1.0
     -smarts -arch=sgin32 -prefix=/usr/local/packages/tau
Use TAU with KCC and fast nanosecond timers on SGI and use SMARTS with -n32 options and install the files in /usr/local/packages/tau 
   % ./configure -c++=KCC -cc=cc -arch=sgi64 -mpiinc=/local/apps/mpich/include
     -mpilib=/local/apps/mpich/lib/IRIX64/ch_p4 -SGITIMERS -pdt=/local/apps/pdt
Use TAU with KCC, and cc on 64 bit SGI systems and use MPI wrapper libraries with SGI's low cost timers and use PDT for automated source code instrumentation. 
   % ./configure -c++=guidec++ -cc=guidec -papi=/usr/local/packages/papi -openmp
     -mpiinc=/usr/packages/mpich/include -mpilib=/usr/packages/mpich/lib
Use OpenMP+MPI using KAI's Guide compiler suite and use PAPI for accessing hardware performance counters for measurements. Configure changes Makefiles in all subdirectories. It replaces system dependent options and profiling options.

Compiling TAU

After configuring TAU, the user needs to build TAU. This is done by typing 
  % make install
This makes the profiling library, pprof - the parallel profile display tool, racy and jracy - the graphical user interface for displaying profile data. Make installs the libraries in the $(TAUROOT)/lib/$(TAU_ARCH) subdirectory and installs utilities such as pprof and racy in the $(TAU_ROOT)/bin/$(TAU_ARCH) subdirectory. In your .cshrc file, add the bin/$(TAU_ARCH) subdirectory to the PATH environment variable. e.g., 
   # in .cshrc file
   set path=($path /usr/local/packages/tau/bin/sgi8k)
This completes the configuration stage of TAU and it is ready to profile applications. Make also compiles a sample application in the examples/instrument subdirectory to illustrate the use of TAU.

Note for Pooma r1 users :

The TAU profiling package is integrated in the POOMA r1 framework, so POOMA users do not need to download TAU separately. Instructions on configuring profiling options in POOMA r1 can be found here.