Instrumenting source files.

The TAU Compiler provides a simple way to automatically instrument an entire project. The TAU Compiler can be used on C, C++, fixed form Fortran, and free form Fortran.

-optVerbose Turn on verbose debugging messages. -optMemDbg Enable TAU’s runtime memory debugger. -optDetectMemoryLeaks Instructs TAU to detect any memory leaks in C/C programs.TAU then tracks the source location of the memory leak as wellas the place in the callstack where the memory allocation wasmade. `-optPdtDir=<dir>` The PDT architecture directory. Typically `$(PDTDIR)/$(PDTARCHDIR)` . `-optPdtF95Opts=<opts>` Options for Fortran parser in PDT (f95parse). `-optPdtF95Reset=<opts>` Reset options to the Fortran parser to the given list. `-optPdtCOpts=<opts>` Options for C parser in PDT (cparse). Typically `$(TAU_MPI_INCLUDE) $(TAU_INCLUDE) $(TAU_DEFS)` . `-optPdtCReset=<opts>` Reset options to the C parser to the given list `-optPdtCxxOpts=<opts>` Options for C parser in PDT (cxxparse). Typically $(TAU_MPI_INCLUDE) $(TAU_INCLUDE) $(TAU_DEFS) . -optPdtCxxReset=<opts> Reset options to the C parser to the given list `-optPdtF90Parser=<parser>` Specify a different Fortran parser. For e.g., `f90parse` instead of `f95parse` . `-optPdtCxxParser=<parser>` Specify a different C parser. For e.g., cxxparse401 instead of cxxparse . -optGnuFortranParser=<parser> Specify the GNU gfortran Fortran parser gfparse instead of f95parse -optGnuCleanscapeParser Uses the Cleanscape Fortran parser f95parse instead of GNU’s gfparse -optPdtUser=<opts> Optional arguments for parsing source code. -optTauInstr=<path> Specify location of tau_instrumentor. Typically $(TAUROOT)/$(CONFIG_ARCH)/bin/tau_instrumentor . -optContinueBeforeOMP Insert a CONTINUE statement before !$OMP directives. -optIncludeMemory Forinteral use only -optTrackUPCR Adds tracking of the UPC runtime library. -optTrackDMAPP Specify wrapping of Pthread library calls at link time. -optTrackPthread Adds tracking of the UPC runtime library. -optNoTrackGOMP Disable wrapping of GOMP library calls at link time -optTrackMPCThread Specify wrapping of MPC Thread library calls at link time. -optPreProcess Preprocess the source code before parsing. Uses /usr/bin/cpp-P by default. -optCPP=<path> Specify an alternative preprocessor and pre-process the sources. -optCPPOpts=<options> Specify additional options to the C pre-processor. -optCPPReset=<options> ResetC preprocessor options to the specified list. -optFPP=<path> Specify an alternative preprocessor and pre-process for Fortran sources. -optFPPOpts=<options> Specify additional options to the Fortran pre-processor. -optTauSelectFile=<file> Specify selective instrumentation file for tau_instrumentor -optPDBFile=<file> Specify PDB file for tau_instrumentor. Skips parsing stage. -optTau=<opts> Specify options for tau_instrumentor. -optCompile=<opts> Options passed to the compiler. Typically $(TAU_MPI_INCLUDE) $(TAU_INCLUDE) $(TAU_DEFS) . -optTauDefs=<opts> Options passed to the compiler by TAU. Typically $(TAU_DEFS) . -optTauIncludes=<opts> Options passed to the compiler by TAU. Typically $(TAU_MPI_INCLUDE) $(TAU_INCLUDE) . -optReset=<opts> Reset options to the compiler to the given list -optLinking=<opts> Options passed to the linker. Typically $(TAU_MPI_FLIBS) $(TAU_LIBS) $(TAU_CXXLIBS) . -optLinkReset=<opts> Reset options to the linker to the given list. -optLinkPreserveLib=<opts> Libraries which TAU should preserve the order of on the link line see "Moving these libraries to the end of the link line:". Default: none. -optTauCC=<cc> Specifies the C compiler used by TAU. -optUseReturnFix Specifies the use of a bug fix with ROSE parser using EDG v3.x -optLinkOnly Disable instrumentation during compilation, do link in the TAU libs -optOpariTool=<path/opari> Specifies the location of the Opari tool. -optOpariDir=<path> Specifies the location of the Opari directory. -optOpariOpts=<opts> Specifies optional arguments to the Opari tool. -optOpariNoInit Do not initlize the POMP2 regions. -optOpariReset=<opts> Resets options passed to the Opari tool. -optOpariLibs=<> Specifies the libraries that have POMP2 regions. (Overrides optOpariNoInit). -optOpari2Tool=<path to opari2> Specifies the location of the Opari tool. -optOpari2ConfigTool=<path/opari2-config> Specifies the location of the Opari tool configuration file -optOpari2Opts=<opts> Specifies optional arguments to the Opari tool. -optOpari2Reset=<opts> Resets options passed to the Opari tool. -optOpari2Dirs=<opts> Specifies the location of the Opari directory -optNoMpi Removes -l*mpi* libraries during linking (default). -optMpi Does not remove -l*mpi* libraries during linking. -optNoRevert Exit on error. THIS IS CRAZY Does not revert to the original compilation rule on error. -optRevert Revert to the original compilation rule on error (default). -optKeepFiles Does not remove intermediate .pdb and .inst.* files. -optReuseFiles Reuses a pre-instrumented file and preserves them. -optAppCC Sets the failsafe C compiler. -optAppCXX Sets the failsafe C compiler. `-optAppF90` Sets the failsafe F90 compiler `-optShared` Use shared library version of TAU `-optCompInst` Use compiler-based instrumentation `-optNoCompInst` Do not revert to compiler instrumentation if source instrumentation fails. `-optPDTInst` Use PDT-based instrumentation `-optHeaderInst` Enable instrumentation of headers `-optDisableHeaderInst` Disable instrumentation of headers `-optTrackIO` Specify wrapping of POSIX I/O calls at link time. `-optMICOffload` Links code for Intel MIC offloading, requires both host and MIC TAU libraries `-optWrappersDir=""` Specify the location of the link wrappers directory. `-optTauUseCXXForC` Specifies the use of a C compiler for compiling C code -optTauWrapFile=<filename> Specify path to the link_options.tau file generated by tau_wrap -optFixHashIf

Generate a TAU profile set from a vampir trace file

vtf2profile is created when TAU is configured with the -vtf=<vtf_dir> option. This tool converts a VTF trace file (*.vpt) to a tau profile set (profile.A.B.C where A, B and C are the node, context and thread numbers respectively). The vtf file to be read is specified in the command line by the -f flag followed by the file’s location. The VTF tracefile specified may be in gzipped form, eg app.vpt.gz. -p is similarly used to specify the relative path to the directory where the profile files should be stored. If no output directory is specified the current directory will be used. A contiguous interval within the vtf file may be selected for conversion by using the -i flag followed by two integers, representing the timestamp of the start and end of the desired interval respectively. The entire vtf file is converted if no interval is given.

-f tracefile -Specify the Vampir tracefile to be converted. -p profile -Specify the location where the profile file(s) should be written. -i interval_start interval_end -Limit the profile produced to the specified interval within the vampir trace file. -c -Opens a command line interface for the program. -h -Displays a help message.

To convert a vampir tracefile, trace.vpt, to an equivalent TAU profile, use the following: ---- vtf2profile -f trace.vpt To produce a TAU profile in the ./profiles directory representing only the events from the start of the tracefile to timestamp 6000, use: ---- To produce a TAU profile in the ./profiles directory representing only the events from the start of the tracefile to timestamp 6000, use: ---- vtf2profile -f trace.vpt -p ./profiles -i 0 6000 ----

tau2vtf , trace2profile

convert TAU tracefiles to vampir tracefiles

This program is generated when TAU is configured with the -vtf=<vtf_dir> option. The tau2vtf trace converter takes a single tau_tracefile (.trc) and tau_eventfile (.edf) and produces a corresponding vtf_tracefile (*.vtf). The input files and output file must be specified in that order. Multi-file TAU traces must be merged before conversion. The default output file format is VTF3 binary. If the output filename is given as the .vpt.gz type, rather than .vpt, the output file will be gzipped. There are two additional output format options. The command line argument '-a' produces the vtf file output in ASCII VTF3 format. The command line argument '-fa' produces the vtf file output in the FAST ASCII VTF3 format. Note that these arguments are mutually exclusive.

-nomessage Suppresses printing of message information in the trace. -v Verbose mode sends trace event descriptions to the standard output as they are converted. -a Print the vtf file output in the human-readable VTF3 ASCII format -fa Print the vtf file in the simplified human-readable FAST ASCII VTF3 format

The program must be run with the TAU trace, TAU event and vtf output files specified in the command line in that order. Any additional arguments follow. The following will produce a VTF, app.vpt, from the TAU trace and event files merged.trc and tau.edf trace file: ---- tau2vtf merged.trc tau.edf app.vpt The following will convert merged.trc and tau.edf to a gzipped FAST ASCII vampir tracefile app.vpt.gz, with message events omitted: ---- The following will convert merged.trc and tau.edf to a gzipped FAST ASCII vampir tracefile app.vpt.gz, with message events omitted: ---- tau2vtf merged.trc tau.edf app.vpt.gz -nomessage -fa ----

vtf2profile , trace2profile , tau_merge , tau_convert

convert TAU tracefiles to TAU profile files

This program is generated when TAU is configured with the -TRACE option. The trace2profile converter takes a single tau_tracefile (.trc) and tau_eventfile (.edf) and produces a corresponding series of profile files. The input files must be specified in that order, with optinal parameters coming afterward. Multi-file TAU traces must be merged before conversion.

-d Output profile files to the specified 'directory' rather than the current directory. -s Output a profile snapshot showing the state of the profile data accumulated from the trace every 'snapshot_interval' time units. The snapshot profiles are placed sequentially in directories labled 'snapshot_n' where 'n' is an integer ranging from 0 to to the total number of snapshots -1.

The program must be run with the TAU trace and tau event files specified in the command line in that order. Any additional arguments follow. The following will produce a profile file array, from the TAU trace and event files merged.trc and tau.edf trace file: ---- trace2profile merged.trc tau.edf The following will convert merged.trc and tau.edf to a series of profiles one directory higher. It will also produce a profile snapshot every 250,000 time units: ---- The following will convert merged.trc and tau.edf to a series of profiles one directory higher. It will also produce a profile snapshot every 250,000 time units: ---- trace2profile merged.trc tau.edf -d ./.. -s 250000 ----

vtf2profile , tau2vtf , tau2otf , tau_merge , tau_convert

convert TAU tracefiles to Epilog tracefiles

This program is generated when TAU is configured with the -epilog=<epilog_dir> option. The tau2elg trace converter takes a tau trace file (.trc) and event definition file (.edf) and produces a corresponding epilog binary trace file (*.elg). Multi-file TAU traces must be merged before conversion.

-nomessage Suppresses printing of message information in the trace. -v Verbose mode sends trace event descriptions to the standard output as they are converted.

The program must be run with the TAU trace, TAU event and elg output files specified in the command line in that order. Any additional arguments follow. The following would convert merged.trc and tau.edf to the Epilog tracefile app.elg, with message events omitted: ---- ./tau2vtf merged.trc tau.edf app.elg -nomessage ----

tau_merge

convert TAU tracefiles to SLOG2 tracefiles

This program is generated when TAU is configured with the -slog2 or -slog2=<slog2_dir> option. The tau2slog2 trace converter takes a single tau trace file (.trc) and event definition file (.edf) and produces a corresponding slog2 binary trace file (*.slog2). The tau2slog2 converter is called from the command line with the locations of the tau trace and event files. These arguments must be followed by the -o flag and the name of the slog2 file to be written. tau2slog 2 accepts no other arguments.

[-h|--h|-help|--help] Display HELP message. [-tc] Check increasing endtime order, exit when 1st violation occurs. [-tcc] Check increasing endtime order,continue when violations occur. [-nc number] Number of childern per node (default is 2) [-ls number] Max byte size of leaf nodes (default is 65536) [-o output.slog2] Output filename with slog2 suffix

A typical invocation of the converter, to create app.slog2, is as follows: ---- tau2slog2 app.trc tau.edf -o app.slog2 ----

tau_merge , tau_convert

convert TAU tracefiles to OTF tracefiles for Vampir/VNG

This program is generated when TAU is configured with the -otf=<otf_dir> option. The tau2otf trace converter takes a TAU formatted tracefile (.trc) and a TAU event description file (.edf) and produces an output trace file in the Open Trace Format (OTF). The user may specify the number of output streams for OTF. The input files and output file must be specified in that order. TAU traces should be merged using tau_merge prior to conversion.

-n streams Specifies the number of output streams (default is 1). -nomessage Suppresses printing of message information in the trace. -v Verbose mode sends trace event descriptions to the standard output as they are converted.

The program must be run with the tau trace, tau event and otf output files specified in the command line in that order. Any additional arguments follow. The following will produce an OTF file, a pp.otf and other related event and definition files, from the TAU trace and event files merged.trc and tau.edf: ---- tau2otf merged.trc tau.edf app.otf ----

tau2vtf(1), trace2profile(1), vtf2profile(1), tau_merge(1), tau_convert(1)

convert TAU tracefiles to OTF2 tracefiles for Vampir/VNG

This program is generated when TAU is configured with the -otf=<otf_dir> option. The tau2otf2 trace converter takes a TAU formatted tracefile (.trc) and a TAU event description file (.edf) and produces an output trace file in the Open Trace Format (OTF2). The user may specify the number of output streams for OTF2. The input files and output file must be specified in that order. TAU traces should be merged using tau_merge prior to conversion.

-n streams Specifies the number of output streams (default is 1). -nomessage Suppresses printing of message information in the trace. -v Verbose mode sends trace event descriptions to the standard output as they are converted.

The program must be run with the tau trace, tau event and otf2 output files specified in the command line in that order. Any additional arguments follow. The following will produce an OTF2 file, a pp.otf2 and other related event and definition files, from the TAU trace and event files tau.trc and tau.edf: ---- tau2otf2 merged.trc tau.edf app.otf2 ----

tau2vtf(1), trace2profile(1), vtf2profile(1), tau_merge(1), tau_convert(1)

convert TAU tracefiles to json tracefiles for Chrome tracing or other viewers

The tau_trace2json trace converter takes a TAU formatted tracefile (.trc) and a TAU event description file (.edf) and produces an output trace file in json. The user may specify the creation of a json file readable by Chrome’s trace viewer. If no output file name is specified with the -o option output will be created in an events.json file in the current directory.

-chrome Output Chrome readable trace output. -ignoreatomic Do not include atomic events in json output. TAU traces include metadata as atomic events so using this option is advised in general. -o Specify an output file other than the default events.json -nomessage Suppresses printing of message information in the trace. -v Verbose mode sends trace event descriptions to the standard output as they are converted.

The program must be run with the tau trace, tau event input first followed by any arguments. The following will produce a json file, trace.json, from the TAU trace and event files tau.trc and tau.edf: ---- tau_trace2json ./tau.trc ./tau.edf -chrome -ignoreatomic -o trace.json ----

tau2vtf(1), trace2profile(1), vtf2profile(1), tau_merge(1), tau_convert(1)

converts PerfLib profiles to TAU profile files

Converts perflib data to TAU format. If an argument is not specified, it checks the perf_data_directory environment variable. Then opens perf_data.timing directory to read perflib data If no args are specified, it tries to read perf_data.<current_date> file.

-h Display the help information. -flat Suppresses callpath profiles, each callpath profile will be flattened to show only the function profile.

---- %> perf2tau timing ----

vtf2profile , tau2vtf , tau2otf , tau_merge , tau_convert

combine multiple node and or thread TAU tracefiles into a merged tracefile

tau_merge is generated when TAU is configured with the -TRACE option. This tool assembles a set of tau trace and event files from multiple multiple nodes or threads across a program’s execution into a single unified trace file. Many TAU trace file tools operate on merged trace files. Minimally, tau_merge must be invoked with a list of unmerged trace files followed by the desired name of the merged trace file or the - flag to send the output to the standard out. Typically the list can be designated by giving the shared name of the trace files to be merged followed by desired range of thread or node designators in brackets or the wild card character '' to encompass variable thread and node designations in the filename (trace.A.B.C.trc where A, B and C are the node, context and thread numbers respectively). For example tautrace..trc would represent all tracefiles in a given directory while tautrace.[0-5].0.0.trc would represent the tracefiles of nodes 0 through 5 with context 0 and thread 0. tau_merge will generate the specified merged trace file and an event definition file, tau.edf by default. The event definition file can be given an alternative name by using the '-m' flag followed by the desired filename. A list of event definition files to be merged can be designated explicitly by using the '-e' flag followed by a list of unmerged .edf files, specified in the same manner as the trace file list. If computational resources are insufficient to merge all trace and event files simultaneously the process may be undertaken hierarchically. Corresponding subsets of the tracefiles and eventfiles may be merged in sequence to produce a smaller set of files that can then be to merged into a singular fully merged tracefile and eventfile. E.g. for a 100 node trace, trace sets 1-10, 11-20, …​, 91-100 could be merged into traces 1a, 2a, …​, 10a. Then 1a-10a could be merged to create a fully merged tracefile.

-e eventfile_list explicitly define the eventfiles to be merged -m output_eventfile explicitly name the merged eventfile to be created - send the merged tracefile to the standard out -a adjust earliest timestamp time to zero -r do not reassemble long events -n do not block waiting for new events. By default tau_merge will block and wait for new events to be appended if a tracefile is incomplete. This command allows offline merging of (potentially) incomplete tracefiles.

To merge all TAU tracefiles into app.trc and produce a merged tau.edf eventfile: ---- tau_merge .trc app.trc To merge all eventfiles 0-255 into ev0_255merged.edf and TAU tracefiles for nodes 0-255 into the standard out: ---- To merge all eventfiles 0-255 into ev0_255merged.edf and TAU tracefiles for nodes 0-255 into the standard out: ---- tau_merge -e events.[0-255].edf -m ev0_255merged.edf \ tautrace.[0-255]..trc - To merge eventfiles 0, 5 and seven info ev057.edf and tau tracefiles for nodes 0, 5 and 7 with context and thread 0 into app.trc: ---- To merge eventfiles 0, 5 and seven info ev057.edf and tau tracefiles for nodes 0, 5 and 7 with context and thread 0 into app.trc: ---- tau_merge -e events.0.edf events.5.edf events.7.edf -m ev057.edf \ tautrace.0.0.0.trc tautrace.5.0.0.trc tautrace.7.0.0.trc app.trc ----

tau_convert trace2profile tau2vtf tau2elg tau2slog2

combine multiple node and or thread TAU tracefiles into a merged tracefile

tau_treemerge.pl is generated when TAU is configured with the -TRACE option. This tool assembles a set of tau trace and event files from multiple multiple nodes or threads across a program’s execution into a single unified trace file. Many TAU trace file tools operate on merged trace files. tau_treemerge.pl will generate the specified merged trace file and an event definition file, tau.edf by default.

-n break_amount set the maximum number of trace files to merge in each invocation of tau_merge. If we need to merge 2000 trace files and if the maximum number of open files specified by unix is 250, tau_treemerge.pl will incrementally merge the trace files so as not to exceed the number of open file descriptors.

tau_merge tau_convert trace2profile tau2vtf tau2elg tau2slog2

convert TAU tracefiles into various alternative trace formats

tau_convert is generated when TAU is configured with the -TRACE option. This program requires specification of a TAU tracefile and eventfile. It will convert the given TAU traces to the ASCII-based trace format specified in the first argument. The conversion type specification may be followed by additional options specific to the conversion type. It defaults to the single threaded vampir format if no other format is specified. tau_convert also accepts specification of an output file as the last argument. If none is given it prints the converted data to the standard out.

-alog convert TAU tracefile into the alog format (This format is deprecated. The SLOG2 format is recommended.) -SDDF convert TAU tracefile into the SDDF format -dump convert TAU tracefile into multi-column human readable text -paraver convert TAU tracefile into paraver format -t indicate conversion of multi threaded TAU trace into paraver format -pv convert single threaded TAU tracefile into vampir format (all -vampir options apply) (default) -vampir convert multi threaded TAU tracefile into vampir format -longsymbolbugfix make the first characters of long, similar identifier strings unique to avoid a bug in vampir -compact abbreviate individual event entries -all compact all entries (default) -user compact user entries only -class compact class entries only -nocomm disregard communication events [outputtrc] specify the name of the output tracefile to be produced

To print the contents of a TAU tracefile to the screen: ---- tau_convert -dump app.trc tau.edf To convert a merged, threaded TAU tracefile to paraver format: ---- To convert a merged, threaded TAU tracefile to paraver format: ---- tau_convert -paraver -t app.trc tau.edf app.pv ----

tau_merge , tau2vtf , trace2profile , tau2slog2

generates selective instrumentation rules based on profile data

tau_reduce is an application that will apply a set of user-defined rules to a pprof dump file ( pprof -d ) in order to create a select file that will include an exclude list for selective implementation for TAU. The user must specify the name of the pprof dump file that this application will use. This is done with the -f filename flag. If no rule file is specified, then a single default rule will be applied to the file. This rule is: numcalls > 1000000 & usecs/call < 2, which will exclude all routines that are called at least 1,000,000 times and average less then two microseconds per call. If a rule file is specified, then this rule is not applied. If no output file is specified, then the results will be printed out to the screen.

Users can specify a set of rules for tau_reduce to apply. The rules should be specified in a separate file, one rule per line, and the file name should be specifed with the appropriate option on the command line. The grammar for a rule is: [GROUPNAME:]FIELD OPERATOR NUMBER. The GROUPNAME followed by the colon (:) is optional. If included, the rule will only be applied to routines that are a member of the group specified. Only one group name can be applied to each rule, and a rule must follow a groupname. If only a groupname is given, then an unrecognized field error will be returned. If the desired effect is to exclude all routines that belong to a certain group, then a trivial rule, such as GROUP:numcalls > -1 may be applied. If a groupnameis given, but the data does not contain any groupname data, then then an error message will be given, but the rule will still be applied to the date ignoring the groupname specification. A FIELD is any of the routine attributes listed in the following table:

Some FIELDS are only available for certain files. If hardware counters are used, then usec, cumusec, usecs/per call are not applicable and a error is reported. The opposite is true if timing data is used rather than hardware counters. Also, stddev is only available for certain files that contain that data. An OPERATOR is any of the following: < (less than), > (greater than), or = (equals). A NUMBER is any number. A compound rule may be formed by using the & (and) symbol in between two simple rules. There is no "OR" because there is an implied or between two separate simple rules, each on a separate line. (ie the compound rule usec < 1000 OR numcalls = 1 is the same as the two simple rules "usec < 1000" and "numcalls = 1").

-f filename specify filename of pprof dump file -p print out all functions with their attributes -o filename specify filename for select file output (default: print to screen -r filename specify filename for rule file -v verbose mode (for each rule, print out rule and all functions that it excludes)

To print to the screen the selective instrumentation list for the paraprof dump file app.prf with default selection rules use: ---- tau_reduce -f app.prf To create a selection file, app.sel, from the paraprof dump file app.prf using rules specified in foo.rlf use: ---- To create a selection file, app.sel, from the paraprof dump file app.prf using rules specified in foo.rlf use: ---- tau_reduce -f app.prf -r foo.rlf -o app.sel ----

Completes uncompleted do/for/parallel omp directives

Finds uncompleted do/for omp directives and inserts closing directives for each one uncompleted. do/for directives are expected immediately before a do/for loop. Closing directives are then placed immediately following the same do/for loop.

pdbfile A pdbfile generated from the source file you wish to check. This pdbfile must contain comments from which the omp directives are gathered. See pdbcomment for information on how to obtain comment from a pdbfile. sourcefile A fortran, C or C++ source file to analyzed. -o write the output to the specified outfile. -v verbose output, will say which directive where added. -d debuging information, we suggest you pipe this unrestrained output to a file.

To check file: source.f90 do: (you will need pdtoolkit/<arch>/bin and tau/utils/ in your path). ---- %>f95parse source.f90 %>pdbcomment source.pdb > source.comment.pdb %>tau_omp source.comment.pdb source.f90 -o source.chk.f90 ----

f95parse pdbcomment

Instruments a MPI application while it is being executed with poe.

This tool dynamically instruments a mpi application by loading a specific mpi library file.

tauOptions To instrument a mpi application a specific TAU library file is loaded when the application is executed. To select which library is loaded use this option. The library files are build according to the options set when TAU is configured. The library file that have been build and thus available for use are in the [TAU_HOME]/[arch]/lib directory. The file are listed as libTAUsh-*.so where * is the instrumentation options. For example to use the libTAUsh-pdt-openmp-opari.so file let the comman line option be -XrunTAUsh-pdt-openmp-opari.

Instrument a.out wit the currently configured options and then run it on four nodes: ---- %>tau_poe ./a.out -procs 4 ---- Select the libTAUsh-mpi.so library to instrument a.out with: ---- %>tau_poe -XrunTAUsh-mpi ./a.out -procs 4 ----

Validates a TAU installation by performing various tests on each TAU stub Makefile

tau_validate will attempt to validate a TAU installation by performing various tests on each TAU stub Makefile. Some degree of logic exists to know where a given test applies to a given makefile, but it’s not perfect.

v Verbose output html Output results in HTML build Only build run Only run tag Only check configurations containing the tag. ie. --tag papi checks only libraries with the -papi in their name. arch directory Specify an arch directory (e.g. rs6000), or the lib directory (rs6000/lib), or a specific makefile. Relative or absolute paths are ok.

There is a few examples: ---- bash : ./tau_validate --html x86_64 &> results.html tcsh : ./tau_validate --html x86_64 >& results.html ----

Allows you to choose a TAU configuration at runtime

Use this script to dynamically load a TAU profiling/tracing library or to select which papi events/domain to use during execuation of the application. At runtime tauex will set the LD_LIBRARY_PATH and pass any other parameters (or papi events) to the program and execute it with the specified TAU measurement options.

Defaults if unspecified: -U -T MPI,PROFILE -e P_WALL_CLOCK_TIME MPI is assumed unless SERIAL is specified PROFILE is assumed unless one of TRACE, VAMPIRTRACE or EPILOG is specified P_WALL_CLOCK_TIME means count real time using fastest available timer

mpirun -np 2 tauex -e PAPI_TOT_CYC -e PAPI_FP_OPS -T MPI,PROFILE — ./ring

TAU execution wrapping script

Use this script to perform runtime performance tracking on either an instrumented or uninstrumented executable. Options include memory and IO tracking, event based sampling, hardware accelerator tracking and data collection from library-provided instrumention API’s such as mpi communication events and RAJA and Kokkos instrumention hooks.

Defaults if unspecified: -T MPI. MPI is assumed unless SERIAL is specified CUDA kernel tracking is included, if A CUDA SYNC call is made after each kernel launch and cudaThreadExit() is called before the exit of each thread that uses CUDA. OPENCL kernel tracking is included, if A OPENCL SYNC call is made after each kernel launch and clReleaseContext() is called before the exit of each thread that uses CUDA. tau_python is similar to tau_exec and can replace the 'python' command when launching a python application. The -tau_python_interpreter=<interpreter> argument allows specification of a python interpreter other than the one used to configure TAU.

mpirun -np 2 tau_exec -io ./ring mpirun -np 8 tau_exec -ebs -ebs_period=1000000 -ebs_source=PAPI_FP_INS ./ring tau_exec -T serial,cupti -cupti ./matmult (Preferred for CUDA 4.1 or later) tau_exec -T serial -cuda ./matmult (Preferred for CUDA 4.0 or earlier) tau_exec -T serial -opencl (OPENCL)

Corrects and reorders the records of TAU trace files.

This program takes in TAU trace files, reorders and corrects the times of these records and then outputs the records to new trace files. The time correction algorithm uses a logical clock algorithm with amortization. This is done by adjusting the times of events such that the product of an effect happens after the cause of that effect.

trace input file EDF input file trace output file EDF output file

This tool generates a selective instrumentation file (called throttle.tau) from a program output that has "Disabling" messages.

This tools will auto-generates a selective instrumenation file basied on output from a program that has the profiling of some its functions throttled.

This tool is design to interact with the TAU web portal (http://tau.nic.uoregon.edu). There are commands for uploading or downloading packed profile files form the TAU portal.

Each command will initate a transfer to profile data btween the TAU portal and either the filesytem (to be stored as ppk file) or to a PerfDMF database. See tau_portal --help for more information.

Configuration program for a PerfDMF database.

This configuration script will create a new TAUdb database.

-h, --help show help --create-default creates a H2 database with all the default values -g, --configFile configFile specify the path to the file that defines the TAUdb configuration. -c, --config configuration_name specify the name of the TAUdb configuration -c foo is equalivent to -g <home>/.ParaProf/perfdmf.cfg.foo . -t, --tauroot path Path to the root directory of tau.

This script will create a new application in the perfdmf database.

-g, --configFile configFile specify the path to the file that defines the perfdmf configuration. -c, --config configuration_name specify the name of the perfdmf configuration -c foo is equalivent to -g <home>/.ParaProf/perfdmf.cfg.foo . -a, --applicationid applicationID specify the id number of the newly added application (default uses auto-increment). -n, --name name the name of the application.

This script will create a new experiment in the perfdmf database.

-g, --configFile configFile specify the path to the file that defines the perfdmf configuration. -c, --config configuration_name specify the name of the perfdmf configuration -c foo is equalivent to -g <home>/.ParaProf/perfdmf.cfg.foo. -a, --applicationid applicationID specify the id number of the application to associate with the new experiment. -n, --name name the name of the application.

Command line tool to load a trial into the TAUdb database.

This script will create a new trial in the TAUdb database.

-n, --name name the name of the application. -a, --applicationname name specify associated application name for this trial -x, --experimentname experimentName specify the name of the experiment to associate with newly uploaded trial. -e, --experimentid experimentID specify the id number of the experiment to associate with the new trial. -g, --configFile configFile specify the path to the file that defines the TAUdb configuration. (overrides -c) -c, --config configuration_name specify the name of the TAUdb configuration -c foo is equalivent to -g <. -t, --trialid experimentID specify the id number of the newly uploaded trial. -m, --metadata filename specify the filename of the XML metadata for this trial. -f, --filetype filetype Specify type of performance data, options are: profiles (default), pprof, dynaprof, mpip, gprof, psrun, hpm, packed, cube, hpc, ompp, snap, perixml, gptl, paraver, ipm, google -i, --fixnames Use the fixnames option for gprof

For the TAU profiles type, you can specify either a specific set of profile files on the commandline, or you can specify a directory (by default the current directory). The specified directory will be searched for profile...* files, or, in the case of multiple counters, directories named MULTI_* containing profile data.

taudb_loadtrial -e 12 -n "Batch 001" This will load profile.* (or multiple counters directories MULTI_*) into experiment 12 and give the trial the name "Batch 001" taudb_loadtrial -e 12 -n "HPM data 01" -f hpm perfhpm* This will load perfhpm* files of type HPMToolkit into experiment 12 and give the trial the name "HPM data 01" taudb_loadtrial -a "NPB2.3" -x "parametric" -n "64" par64.ppk This will load packed profile par64.ppk into the experiment named "parametric" under the application named "NPB2.3" and give the trial the name "64". The application and experiment will be created if not found.

Launches TAU’s Performance Data Mining Analyzer.

Complete documentation can be found at http://www.cs.uoregon.edu/research/tau/tau-usersguide.pdf

Configures a TAUdb database for use with perfexplorer, and installs necessary JAR files.

Configures a TAUdb database for use with perfexplorer, and installs necessary JAR files.

C compiler wrapper for TAU

If the -tau:makefile option is not used, the TAU_MAKEFILE environment variable will be checked, if it is not specified, then the -tau:<options> will be used to identify a binding.

taucc foo.c -o foo taucc -tau:MPI,OPENMP,TRACE foo.c -o foo taucc -tau:verbose -tau:PTHREAD foo.c -o foo

Complete documentation can be found at http://www.cs.uoregon.edu/research/tau/tau-usersguide.pdf

UPC wrapper for TAU

If the -tau:makefile option is not used, the TAU_MAKEFILE environment variable will be checked, if it is not specified, then the -tau:<options> will be used to identify a binding.

Complete documentation can be found at http://www.cs.uoregon.edu/research/tau/tau-usersguide.pdf

C++ compiler wrapper for TAU

If the -tau:makefile option is not used, the TAU_MAKEFILE environment variable will be checked, if it is not specified, then the -tau:<options> will be used to identify a binding.

taucxx foo.cpp -o foo taucxx -tau:MPI,OPENMP,TRACE foo.cpp -o foo taucxx -tau:verbose -tau:PTHREAD foo.cpp -o foo

Complete documentation can be found at http://www.cs.uoregon.edu/research/tau/tau-usersguide.pdf

Fortran compiler wrapper for TAU

If the -tau:makefile option is not used, the TAU_MAKEFILE environment variable will be checked, if it is not specified, then the -tau:<options> will be used to identify a binding.

tauf90 foo.f90 -o foo tauf90 -tau:MPI,OPENMP,TRACE foo.f90 -o foo tauf90 -tau:verbose -tau:PTHREAD foo.f90 -o foo

Complete documentation can be found at http://www.cs.uoregon.edu/research/tau/tau-usersguide.pdf

Launches TAU’s Java-based performance data viewer.

For the TAU profiles type, you can specify either a specific set of profile files on the commandline, or you can specify a directory (by default the current directory). The specified directory will be searched for profile...* files, or, in the case of multiple counters, directories named MULTI_* containing profile data.

Complete documentation can be found at http://www.cs.uoregon.edu/research/tau/tau-usersguide.pdf

Quickly diplays profile data.

-a Show all location information available -c Sort according to number of Calls -b Sort according to number of suBroutines called by a function -m Sort according to Milliseconds (exclusive time total) -t Sort according to Total milliseconds (inclusive time total) (default) -e Sort according to Exclusive time per call (msec/call) -i Sort according to Inclusive time per call (total msec/call) -v Sort according to Standard Deviation (excl usec) -r Reverse sorting order -s print only Summary profile information -n num print only first num number of functions -f filename specify full path and Filename without node ids -p suPpress conversion to hhmmssmmm format -l List all functions and exit -d Dump output format (for tau_reduce) [node numbers] prints only info about all contexts/threads of given node numbers

automaticly instruments a source basied on information provided by pdt.

-a Show all location information available -c Sort according to number of Calls -b Sort according to number of suBroutines called by a function -m Sort according to Milliseconds (exclusive time total) -t Sort according to Total milliseconds (inclusive time total) (default) -e Sort according to Exclusive time per call (msec/call) -i Sort according to Inclusive time per call (total msec/call) -v Sort according to Standard Deviation (excl usec) -r Reverse sorting order -s print only Summary profile information -n num print only first num number of functions -f filename specify full path and Filename without node ids -p suPpress conversion to hhmmssmmm format -l List all functions and exit -d Dump output format (for tau_reduce) [node numbers] prints only info about all contexts/threads of given node numbers

%> tau_instrumentor foo.pdb foo.cpp -o foo.inst.cpp -f select.tau

Converts VTF profile to TAU profiles and launches an interactive VTF prompt.

-c Opens command line interface. -f Converts trace [file] to TAU profiles. -p Places the resulting profiles in the directory [path]. -i States that the interval [from],[to] should be profiled.

Launches GUI interface to configure TAU.

-v Verbose output. --html Output results in HTML. --build Only build. --run Only run.

Instruments an external library with TAU without needing to recompile

---- %> tau_wrap hdf5.h.pdb hdf5.h -o hdf5.inst.c -f select.tau -g hdf5 ---- This specifies the instrumented wrapper library source (hdf5.inst.c), the instrumentation specification file (select.tau) and the group (hdf5). It creates the wrapper/ directory.

Generates a wrapper library that can intercept at link time or at runtime routines specified in a header file

---- %> tau_gen_wrapper hdf5.h /usr/lib/libhdf5.a ---- This generates a wrapper library that may be linked in using TAU_OPTIONS -optTauWrapFile=<wrapperdir>/link_options.tau

tau_gen_wrapper reads the TAU_MAKEFILE environment variable to get PDT settings

Instruments application at run time using Intel’s PIN library

To profile routines in mytest.exe with prefix "myf": ---- tau_pin -r myf.* — mytest.exe ---- To profile all routines in mpitest.exe ( no need to specify any rule for all ): ---- tau_pin mpitest.exe ---- to profile only MPI routines in mpitest.exe by launching two instances: ---- tau_pin -n 2 -r MPI.* — mpitest.exe ---- .Wildcards * for anything, for example *MPI means any string having MPI in between any other characters. * ? It’s a placeholder wild card ?MPI* means any character followed by MPI and followed by any string, example: ??Try could be __Try or MyTry or MeTry etc.

Instruments java applications at runtime using JVMTI

Detects the available CUPTI counters on the a each GPU device.

Instruments and executes binaries to generate performance data. (DyninstAPI based instrumentor)

Rewrites binaries using Maqao if TAU is configured using PDT 3.17+ at the routine level. If it doesn’t find the Maqao package from PDT 3.17, it reverts to tau_run (DyninstAPI based instrumentor).

Defaults if unspecified: -T MPI MPI is assumed unless SERIAL is specified

---- tau_rewrite -T papi,pdt a.out -o a.inst ---- ---- mpirun -np 4 ./a.inst ----

Launches PySpark applications with TAU instrumentation

TAU can profile PySpark applications using Spark 2.2 or later and Python 2.7 or later with the numpy package installed. TAU must be configured with the -pythoninc and -pythonlib options specifying an appropriate Python installation. The SPARK_HOME environment variable must be set to the location of your Spark installation. Replace spark-submit in your normal Spark application invocation with tau_spark-submit. Options for tau_spark-submit can be set using the TAU_SPARK_PYTHON_ARGS environment variable. A PySpark application profiled using tau_spark-submit will generate one profile file per task executed.

---- export TAU_SPARK_PYTHON_ARGS="-T serial,python" ---- ---- tau_spark-submit --master local[4] ./als.py ----

Additional documentation and examples can be found in the pyspark subdirectory of the examples directory in your TAU installation.