TDL/POET - a Basic Tool for Accessing Event Traces

Based on the logical structure introduced in the last subsection, we designed and implemented the event trace access interface function library POET. The basic idea is to consider the measured event trace a generic abstract data structure. The analysis tools can access the traces only via a uniform and standardized set of generic procedures.

In order to be able to decode the different trace formats, the POET functions use a trace description file called key file (see fig. 3), as discussed in approach (5) above. For efficiency, the key file is in a binary and compact format. It not only describes data formats and representation of the single values, but also includes user-defined (problem-oriented) identifiers for each record field (field names) and the interpretations for the values of record fields. In order to access event record fields efficiently, in POET a field type is assigned to each record field. These are the following basic field types:

TOKEN:

Record fields of type token contain only one value from a fixed and well-defined set of constant values. It is a construction similar to the enumeration types in the usual programming languages. They can be used to describe encoded information such as event or processor identifications. Each value has a special fixed meaning called interpretation.

FLAGS:

Record fields of type flags are like token record fields, but they can contain more than one value out of a fixed well-defined set. This is done by encoding the individual values as bits which are set or not set. Similar to token values, each bit can have a special meaning, also called interpretation.

TIME:

Record fields of type time are used to describe timing information contained in an event record. This timing information can be of arbitrary resolution and mode (point in time or distance from previous time value).

DATA:

Record fields of type data contain the value of a variable of the monitored application, or the contents of a register of the object system. They can be compared with variables in programming languages. It is only specified how to interpret their value. This format specification is a simple data type such as integer, unsigned or string.

Additionally, there are other types of event record fields which are only relevant to the decoding system: First, there are record length fields, which contain the length of the following record field or of the current or previous event record, and checksums. Second, fields containing irrelevant or uninteresting data, such as blank fields, are called filler.

POET provides the following five types of functions for trace initialization, information retrieval and positioning (see fig. 3):

init functions:

The init functions have to be used to get access to an event trace and to bind the corresponding key file to it. They return a trace number which has to be used in the other functions for further reference. In this way, it is possible to have access to more than one trace at the same time. All the information from the key file is read once and stored in internal variables for later use. This is for efficiency and to save file descriptors.

information functions:

This group of functions can be used by analysis tools to obtain all useful information about a certain event trace, e.g. the number, the types and the names of record fields, or a list of interpretations defined for a token record field.

move functions:

These functions move the current decoding position (shown as high-lighted box in fig. 3) in the event trace. With these functions it is possible to process the event records in an event trace in the order they have been recorded (get_next), or to move the decoding position in the event trace relative (forward/backward) or absolute (goto_record) to a desired event record. In moving, the event trace is decoded and the contents of the record fields of the current event record are stored in an internal value buffer.

get functions:

For each record field type, POET provides an efficient and representation-independent way of accessing the decoded values of a certain record field (get_token, get_flags, get_time, and get_value) stored in the value buffer. But it is also possible to get the decoded values in a generalized form (get_value).

special functions:

In addition to the routines described above there are some functions to support the user in often needed tasks. This includes routines such as functions for interpreting time values in different resolutions or for error handling.

As an illustration, see the C program fragment in fig. 4. It steps once through an event trace stored in the file trc_file and prints for each event record the record number and the problem-oriented interpretation of the event occurred. The program text is not complete; the declarations are missing and there is no error handling (e.g. it should print an error message, when record field "EVENT" does not exist). But it demonstrates how POET can be used to write a program for analyzing event traces.

In order to make the construction of the access key file more user-friendly, we developed the event trace description language TDL which is designed for a problem-oriented description of event traces. The TDL compiler checks the TDL description for syntactic and semantic correctness and transforms it into the corresponding binary key file (see fig. 5). In this way, the initialization of POET can be much faster as there is no need for error checking.

The development of TDL had two principal aims: the first was to make a language available which clearly and naturally reflects the fundamental structure of an event trace. The second was that even a user not familiar with all details of the language should be able to read and understand a given TDL description. Therefore, TDL is largely adapted to the English language. The notation of syntactic elements of the language and the general structure of a TDL description are closely related to similar constructs in the programming languages PASCAL and C. By writing an event trace description in TDL, one provides at the same time a documentation of the performed measurement.

Beyond that, we use a similar approach for filtering event records depending on the values of their record fields. For efficiency, the filtering layer is integrated into POET. There is an additional function to the POET library (get_next_filtered) which can be used to move the current decoding position within the event trace to the next event record which matches the user-specified restrictions given in a so-called filter file. These rules can be specified in the Filter Description Language FDL. Since the FDL compiler does not only read the filter description but also the key file, the problem-oriented identifiers for field names and interpretations can be used for specifying the filter rules. The FDL compiler can also check for syntactical correctness and consistency.

A prototype of the tool TDL/POET was designed and implemented under the operating system UNIX in the programming language C in 1987. A redesign took place in 1989. The now available version 5.3 is much faster and provides more functions than the prototype. For details, see the comprehensive user's guide [22]. The tools enable us to analyze event traces which were recorded by ZM4 or other monitor systems such as network analyzers, logic analyzers, software monitors or even traces generated by simulation tools (e.g. QNAP). POET is an open interface. This means that the user can build his own customized analysis tools using the POET function library.

Using a complex data interface like POET does not automatically mean loosing performance. Using UNIX profiling we measured that our analysis tools spend about 5%of their time in POET functions. If the tool uses graphical output (e.g. X windows), this fraction was even less than 1%. Therefore, the win in performance when substituting the POET functions with access routines optimized for a special trace format is small, and the object-independence of the tool is lost.