To achieve high-performing CCA applications, it is critically important to characterize and understand component performance and the performance of component compositions. For the most part, component applications are constructed in a functional manner by connecting components based on their offered services. However, opportunities for performance optimization are few without performance information. The goal of performance modeling is to build up performance knowledge in a form for that can be applied in performance decisions regarding component selection and configuration, component connections, scheduling of components, online component adaptation, and so on.
The performance of individual components can be studied in a straightforward manner by conducting a series of performance experiments on expected target platforms, varying algorithm- or system-specific parameters. In this way, a performance measurement space is enumerated, reflecting observed performance for chosen parameter and platform settings. The main complication of this approach, with respect to scientific components, is the complexity of the component implementations. For instance, parallel components will require performance experimentation for different numbers of processors. Component performance may also be sensitive to problem size and data layout. System features, such as processor and memory architecture, can lead to significant performance differences between machines. An experimentation harness will help to conduct performance measurements and could use a performance database to store the results. Analysis tools are also needed to synthesize empirical models from the performance experiments. The models will characterize performance behavior across experimental parameters and will serve as performance estimators of points not sampled in the performance space.
Difficulties arise when attempting to characterize the performance of component compositions. In general, it should not be expected that the principle of superposition holds when two components are connected. That is, if you combine the ``best'' performing variants of each component, the component composition might not perform best or even well. This may be due, for instance, to data structure mismatch or conflicting memory usage. The experimentation approach then should involve testing of component combinations to characterize their performance interactions. However, as the number of connected components increase, the performance space grows and the interactions can become more complex. Again, performance experimentation studies are important, but techniques will need to be developed to help narrow the performance search.