This file contains information for installing EVH1 with TAU.

Steps:
------
1) For the NERSC IBM SP (seaborg)
% module load tcltk GNU gcc 

Some packages (PDT) require GNU make instead of IBM's make so this step is important.

2) Install HDF that comes with EVH1.  
   a) Go into ./HDF
   b) ./configure
   c) make
   d) make -test
   e) make install

3) Install PDT (http://www.cs.uoregon.edu/research/paracomp/pdtoolkit)
% configure -XLC
% make 
% make install

4) Install TAU (http://www.acl.lanl.gov/tau)
% cd tau-2.xx
% configure -pdt=<pdt_dir_above> -fortran=ibm -mpi -useropt=-O3
for PROFILING 

% configure -pdt=<pdt_dir_above> -fortran=ibm -mpi -useropt=-O3 -papi=<papidir>
for using with PAPI

% configure -pdt=<pdt_dir_above> -fortran=ibm -mpi -useropt=-O3 -TRACE
for generating event traces.

% make clean install

% set path=(<TAUDIR>/rs6000/bin $path)
This ensures that there is no name clash with IBM's pprof. 

5) Install EVH1
Modify the Makefile. Set HOME and TAU_DIR variables to point to the EVH1 and 
TAU directories. Note: EVH1 Makefile assumes that HDF is installed in the same
directory level as the top-most EVH1 directory. If this is not the case, you may
have to modify the location of the HDF directory. 

To enable TAU, keep the line
USE_TAU = 1

To disable TAU, comment this line
# USE_TAU = 1

% make
produces xtest.
Modify job (LoadLeveler script) accordingly.

% llsubmit job
See output/rh.log and compare it with rh.log in the saved_output directory. 

If you're using PAPI, it is important to add 

or some similar PAPI event before 
./xtest
in the job script that is used to submit the LoadLeveler job. 

NOTE: If you're using PAPI on IBM, I'd recommend using papi-1.1.5 on seaborg 
(See the attached papi-1.1.5 directory).

Once profiles are created, they can be viewed using
% pprof
(Ensure that you're using TAU's pprof.)
% jracy

If traces are generated,
% tau_merge *.trc evh1.trc
% tau_convert -pv evh1.trc tau.edf evh1.pv
% vampir evh1.pv


*** NOTE: By default a selective instrumentation file in EVH1's compile 
directory is specified. Edit this "select.dat" file to exclude routines for 
instrumentation. We have added "PARABOLA" routine to illustrate this. If you 
wish to instrument it, please remove it from the exclude list. An include 
list can be specified similarly (see TAU's examples/autoinstrument/select file).

CHANGES:
********
To parse EVH1 with PDT, the following changes were made:
1) Added the "parameter" keyword to evh1_modules.f for the initialization of 
boltzman, avgmass, G, bmass, and nmfx variables. 

2) Changed each occurence of real*8 to real in several files. 

3) In prin.f, the Mutek parser did not like the declaration of the array with
   a variable size.  
      
      real :: mass_enc(imax)
  was changed to
      real :: mass_enc(ii)
  as imax is initialized with the constant ii. 


If you need any assistance with running EVH1 with TAU, please send an e-mail
to Sameer Shende <sameer@cs.uoregon.edu>.