A large number of optimization problems have been identified as computationally challenging and/or intractable to solve within a reasonable amount of time. Due to the NP-hard nature of these problems, in practice, heuristics account for the majority of existing algorithms. Metaheuristics are one very popular type of heuristics used for many of these optimization problems. In this paper, we present a novel parallel-metaheuristic framework, which effectively enables to devise parallel metaheuristics, particularly with heterogeneous metaheuristics. The core component of the proposed framework is its harmony-search-based coordinator. Harmony search is a recent breed of metaheuristic that mimics the improvisation process of musicians. The coordinator facilitates heterogeneous metaheuristics (forming a parallel metaheuristic) to escape local optima. Specifically, best solutions generated by these worker metaheuristics are maintained in the harmony memory of the coordinator, and they are used to form new-possibly better-harmonies (solutions) before actual solution sharing between workers occurs; hence, their solutions are harmonized with each other. For the applicability validation and the performance evaluation, we have implemented a parallel hybrid metaheuristic using the framework for the task scheduling problem on multiprocessor computing systems (e.g., computer clusters). Experimental results verify that the proposed framework is a compelling approach to parallelize heterogeneous metaheuristics.