Class switching ensembles for ordinal regression

Research areas:

• Methodology - Machine learning

Year:

2017

Type of Publication:

In Proceedings

Keywords:

Class switching, ordinal regression, ensemble learning

Authors:

• Gutiérrez, Pedro Antonio
• Pérez-Ortiz, María
• Suárez, Alberto

Volume:

10305

Book title:

14th International Work-Conference on Artificial and Natural Neural Networks (IWANN2017)

Series:

Lecture Notes in Computer Science (LNCS)

Pages:

408-419

Organization:

Cádiz, Spain

Month:

14th-16th June

ISBN:

978-3-319-59152-0

BibTex:

@conference{Gutierrez2017IWANN, author = "Pedro Antonio Guti{\'e}rrez and Mar{\'i}a P{\'e}rez-Ortiz and Alberto Su{\'a}rez", abstract = "The term ordinal regression refers to classification tasks in which the categories have a natural ordering. The main premise of this learning paradigm is that the ordering can be exploited to generate more accurate predictors. The goal of this work is to design class switching ensembles that take into account such ordering so that they are more accurate in ordinal regression problems. In standard (nominal) class switching ensembles, diversity among the members of the ensemble is induced by injecting noise in the class labels of the training instances. Assuming that the classes are interchangeable, the the labels are modified at random. In ordinal class switching, the ordering between classes is taken into account by reducing the transition probabilities to classes that are further apart. In this manner smaller label perturbations in the ordinal scale are favoured. Two different specifications of these transition probabilities are considered; namely, an arithmetic and a geometric decrease with the absolute difference of the class ranks. These types of ordinal class switching ensembles are compared with an ensemble method that does not consider class-switching, a nominal class-switching ensemble, an ordinal variant of boosting, and two state-of-the-art ordinal classifiers based on support vector machines and Gaussian processes, respectively. These methods are evaluated and compared in a total of $15$ datasets, using three different performance metrics. From the results of this evaluation one concludes that ordinal class-switching ensembles are more accurate than standard class-switching ones and than the ordinal ensemble method considered. Furthermore, their performance is comparable to the state-of-the-art ordinal regression methods considered in the analysis. Thus, class switching ensembles with specifically designed transition probabilities, which take into account the relationships between classes, are shown to provide very accurate predictions in ordinal regression problems.", booktitle = "14th International Work-Conference on Artificial and Natural Neural Networks (IWANN2017)", doi = "10.1007/978-3-319-59153-7_36", isbn = "978-3-319-59152-0", keywords = "Class switching, ordinal regression, ensemble learning", month = "14th-16th June", organization = "C{\'a}diz, Spain", pages = "408-419", publisher = "Springer International Publishing", series = "Lecture Notes in Computer Science (LNCS)", title = "{C}lass switching ensembles for ordinal regression", url = "dx.doi.org/10.1007/978-3-319-59153-7_36", volume = "10305", year = "2017", }

Abstract:

The term ordinal regression refers to classification tasks in which the categories have a natural ordering. The main premise of this learning paradigm is that the ordering can be exploited to generate more accurate predictors. The goal of this work is to design class switching ensembles that take into account such ordering so that they are more accurate in ordinal regression problems. In standard (nominal) class switching ensembles, diversity among the members of the ensemble is induced by injecting noise in the class labels of the training instances. Assuming that the classes are interchangeable, the the labels are modified at random. In ordinal class switching, the ordering between classes is taken into account by reducing the transition probabilities to classes that are further apart. In this manner smaller label perturbations in the ordinal scale are favoured. Two different specifications of these transition probabilities are considered; namely, an arithmetic and a geometric decrease with the absolute difference of the class ranks. These types of ordinal class switching ensembles are compared with an ensemble method that does not consider class-switching, a nominal class-switching ensemble, an ordinal variant of boosting, and two state-of-the-art ordinal classifiers based on support vector machines and Gaussian processes, respectively. These methods are evaluated and compared in a total of $15$ datasets, using three different performance metrics. From the results of this evaluation one concludes that ordinal class-switching ensembles are more accurate than standard class-switching ones and than the ordinal ensemble method considered. Furthermore, their performance is comparable to the state-of-the-art ordinal regression methods considered in the analysis. Thus, class switching ensembles with specifically designed transition probabilities, which take into account the relationships between classes, are shown to provide very accurate predictions in ordinal regression problems.