Human Activity Recognition (HAR) has a growing research interest due to the widespread presence of motion sensors on user personal devices. The performance of HAR system deployed on large-scale is often significantly lower than reported due to the sensor-, device-, and person-specific heterogeneities. In this work, we develop a new approach for clustering such heterogeneous data, represented as a time series, which incorporates different level of heterogeneities in the data within the model. Our method is based on representing the heterogeneities as a hierarchy where each hierarchy denotes a specific heterogeneity (e.g. a sensor-specific heterogeneity). Experimental evaluation on an EMG sensor dataset with heterogeneities shows that our method performs favourably compared to other time series clustering approaches.