Performance engineering technology for scientific component software should take into account not only how performance is observed, but also what techniques and tools are needed to improve the performance of component applications. In general, performance tuning is the process of diagnosing performance problems by comparing measured performance to expected performance, and then removing the inefficiencies based on an analysis of those performance properties of components and component ensembles best matching execution conditions. We define performance tuning in this way because there are many factors that can affect the performance of component applications. These include the performance characteristics of individual components, the performance effects related to component coupling, and the resources available for component execution. Building and executing component applications will be guided by a set of choices that take these factors into account. These choices represent a complex performance space. Our goal is to develop performance technology that can assist application developers and users in making judicious decisions that will lead to high performance component codes.
The approach we will take provides a means to store performance knowledge about components and component compositions, and to make that knowledge available to applications at the times of component selection, deployment, and replacement during execution. A performance knowledgebase (PerfKB) that can be used to record performance data for components measured from multiple performance experiments. In addition, the PerfKB can be able to store results from empirical data analysis in the form of statistical tables and regression models. Analytical models used to express component performance with respect to algorithm characteristics will also be represented in the knowledgebase.
Once we have a means to capture and store performance knowledge, we need to support access to this information in ways that are consistent with component technology. As part of our plan, we will integrate the performance knowledgebase in the CCA framework services used for component registration and discovery. The idea is to extend component repository services with interfaces to the PerfKB where queries can be made about component performance. This will allow smart choices of which components and versions to use during application construction based on the known system parameters. Inquires can also be made to discover appropriate choice of parameters for component execution (e.g., number of threads or processors). Work within the CCTTSS project is underway to define a XML file specification to store component information, and we will work with this effort to incorporate what is needed for performance.
However, we believe this is only part of the solution. To control or improve the performance of an application during execution (i.e., performance steering), we must be able to make decisions based on how well a component is performing with respect to what is expected. We can think of performance aware components, which are able to monitor their own performance (using the performance measurement mechanisms discussed above) and adapt their operation in response to the performance feedback, but this depends on knowing what to change to effect positive performance gain. The performance knowledge to make the decision must be packaged in such a way so as to be easily accessed and processed. Requiring the component to return to the performance repository is too coarse-grained. Furthermore, the steering decisions are only with respect to local component performance. For intelligent performance control at the level of component ensembles, it is necessary to conceive of representations of performance knowledge that can be efficiently and actively processed.
In this context, a performance knowledge component (PKC) can be implemented to capture specific performance information about a deployed component and to provide access to that information during execution using standard CCA interfaces. In essence, the PKC functions as a performance estimation (prediction) service for its respective component, where the virtual interfaces are specialized for the component type. Performance aware components can be created using this approach. More importantly, the approach makes it possible to build higher-level performance models by combining the PKCs of the components employed in an application. These composition performance models provide performance awareness at the application level and can be used to guide performance steering decisions. Their CCA-based implementation using PKCs means performance estimations will be able to be made efficiently and with the benefit of existing CCA framework services.