Ordered Classifier Chains for Multi-label Classification

Maryam Keikha, Sattar Hashemi


Classifier chains method is introduced recently in multi-label classification scope as a high predictive performance technique aims to exploit label dependencies and in the meantime preserving the computational complexity in a desirable level. In this paper, we present a method for chain's order, called Ordered Classifier Chains (OCC), elaborating that the sequence of labels in the chain plays an important role in predictive performance of corresponding multi-label classifiers. OCC proposes making use of correlation of every class label with that of features. OCC renders an ordering of class labels in their descending order. Once the ordering of labels is determined, the features along with every label are fed to binary classifier. In the classifier chain model the feature space of every binary classifier is extended with the new order of labels. In order to specify association of each sample with the set of class labels, it is given to all of classifiers. Empirical evaluations include an extensive range of multi-label datasets reveal that OCC manages to improve the classification performance compared to existing approaches.


Multi-Label Classification; Label Dependency; Classifier Chains; Binary Relevance.

Full Text:



Dimou, Anastasios, Grigorios Tsoumakas, Vasileios Mezaris, Ioannis Kompatsiaris, and L. Vlahavas. "An empirical study of multi-label learning methods for video annotation." In Content-Based Multimedia Indexing, 2009. CBMI'09. Seventh International Workshop on, pp. 19-24. IEEE, 2009.



Fürnkranz, Johannes, Eyke Hüllermeier, Eneldo Loza Mencía, and Klaus Brinker. "Multilabel classification via calibrated label ranking." Machine learning 73, no. 2 (2008): 133-153.



Wieczorkowska, Alicja, Piotr Synak, and Zbigniew W. Raś. "Multi-label classification of emotions in music." In Intelligent Information Processing and Web Mining, pp. 307-315. Springer Berlin Heidelberg, 2006.



Read, Jesse, Bernhard Pfahringer, Geoff Holmes, and Eibe Frank. "Classifier chains for multi-label classification." In: ECML'09: 20th European Conference on Machine Learning, Springer (2009): 254–269.



Read, Jesse, Bernhard Pfahringer, Geoff Holmes, and Eibe Frank. "Classifier chains for multi-label classification." Machine learning 85, no. 3 (2011): 333-359.



Tsoumakas, Grigorios, and Ioannis Katakis. "Multi-label classification: An overview." Dept. of Informatics, Aristotle University of Thessaloniki, Greece (2006).


Schapire, Robert E., and Yoram Singer. "BoosTexter: A boosting-based system for text categorization." Machine learning 39, no. 2-3 (2000): 135-168.Vens, C., Struyf, J., Schietgat, L., Dˇzeroski, S., Blockeel, H.: Decision trees for hierarchical multi-label classification. Machine Learning 2(73), 185–214 (2008).


Vens, Celine, Jan Struyf, Leander Schietgat, Sašo Džeroski, and Hendrik Blockeel. "Decision trees for hierarchical multi-label classification." Machine Learning 73, no. 2 (2008): 185-214.



Zhang, Min-Ling, and Zhi-Hua Zhou. "ML-KNN: A lazy learning approach to multi-label learning." Pattern recognition 40, no. 7 (2007): 2038-2048.



Read, Jesse, Bernhard Pfahringer, and Geoffrey Holmes. "Multi-label classification using ensembles of pruned sets." In Data Mining, 2008. ICDM'08. Eighth IEEE International Conference on, pp. 995-1000. IEEE, 2008.



Tsoumakas, Grigorios, Anastasios Dimou, Eleftherios Spyromitros, Vasileios Mezaris, Ioannis Kompatsiaris, and Ioannis Vlahavas. "Correlation-based pruning of stacked binary relevance models for multi-label learning." In Proceedings of the 1st International Workshop on Learning from Multi-Label Data, pp. 101-116. 2009.


Cheng, Weiwei, and Eyke Hüllermeier. "Combining instance-based learning and logistic regression for multilabel classification." Machine Learning 76, no. 2-3 (2009): 211-225.Tsoumakas, G., Vlahavas, I.P.: Random k-labelsets: An ensemble method for multilabel classification. In: ECML '07: 18th European Conference on Machine Learning, pp. 406–417. Springer (2007).



Trohidis, Konstantinos, Grigorios Tsoumakas, George Kalliris, and Ioannis P. Vlahavas. "Multi-Label Classification of Music into Emotions." In ISMIR, vol. 8, pp. 325-330. 2008.


Witten, Ian H., and Eibe Frank. Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann, 2005.Friedman, M. :The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association 32(200) (1973).


Friedman, Milton. "The use of ranks to avoid the assumption of normality implicit in the analysis of variance." Journal of the American Statistical Association 32, no. 200 (1937): 675-701.



Demšar, Janez. "Statistical comparisons of classifiers over multiple data sets." The Journal of Machine Learning Research 7 (2006): 1-30.


Dunn, Olive Jean. "Multiple comparisons among means" Journal of the American Statistical Association 56, no. 293 (1961): 52-64.



DOI: http://dx.doi.org/10.21174/jomi.v1i1.23


  • There are currently no refbacks.

ISSN: 2377-2220   

CC BY Google Scholar DOAJ  Crossref logo