High performance computing (HPC) systems are an important enabling tool for modern scientific discovery. These large scale computing systems have, since the 1990s, been increasing built as clusters of commodity computers. The operational energy needs of these clusters has lead the HPC community to focus on energy efficient hardware and programming practices. One of the major side effects of introducing energy efficient hardware is variability in power consumption between components within the cluster. In practice, power variability at scale has resulted in poor power utilization and challenges for energy providers contracted to provide the needed power. Hardware overprovisioned HPC systems have been proposed to improve power utilization however production deployment of such a system would compound the challenge for energy providers.

This dissertation presents foundational work on HPC power scheduling, a technique that reduces the risks associated with operating hardware overprovisioned HPC systems. Power scheduling is formalized using the power scheduling invariant. Generalized application behavior, for applications running under a power cap, are experimentally studied. Study insights are used to develop a power scheduler and a power capping cluster simulator. Comparative behavior of different power scheduling strategies as also examined.

Utilizing the power scheduling invariant, the safety of any power scheduler for deployment can be proven through analyzing scheduler’s algorithm and mechanism. A general trend exists in power capped application performance that can be related to application progress, the underlying physics of the hardware, and expected runtime dilation. PowSched provides a proof by construction that power scheduling can be done safely and effectively without application specific models using a simple feedback mechanism. Experimentally, PowSched was shown to produce a 14% improvement in throughput compared to a fair distribution of power between cluster components. PowSim provides a proof by construction that the generalized effects on runtime can be efficiently simulated at scale, providing critical simulation infrastructure for researchers exploring power scheduling at scale. Using simulation, power scheduling strategies are studied and dynamic power scheduling appears to out perform static and reservation based techniques.

This dissertation includes previously published and unpublished co-authored material.