Predictive clustering is based on ideas from two machine learning subareas, predictive modeling and clustering. Methods for predictive clustering enable us to construct models for predicting multiple target variables, which are normally simpler and more comprehensible than the corresponding collection of models, each predicting a single variable. To this end, predictive clustering has been restricted to decision tree methods. Our goal is to extend this approach to methods for learning rules. We have developed a generalized version of the covering algorithm that enables learning of ordered or unordered rules, on single or multiple target classification or regression domains. Performance of the new method compares favorably to existing methods. Comparison of single target and multiple target prediction models shows that multiple target models offer comparable performance and drastically lower complexity than the corresponding collections of single target models.