Unimodal regularisation based on beta distribution for deep ordinal regression

Research areas:

• Methodology - Ordinal classification

Year:

2022

Type of Publication:

Article

Keywords:

Ordinal regression, Unimodal distribution, Convolutional network, Beta distribution, Stick-breaking

Authors:

• Vargas, Víctor Manuel
• Gutiérrez, Pedro Antonio
• Hervás-Martínez, César

Journal:

Pattern Recognition

Volume:

122

Pages:

108310

Month:

February

ISSN:

0031-3203

BibTex:

@article{PRUnimodal2022, author = "V{\'i}ctor Manuel Vargas and Pedro Antonio Guti{\'e}rrez and C{\'e}sar Herv{\'a}s-Mart{\'i}nez", abstract = "Currently, the use of deep learning for solving ordinal classification problems, where categories follow a natural order, has not received much attention. In this paper, we propose an unimodal regularisation based on the beta distribution applied to the cross-entropy loss. This regularisation encourages the distribution of the labels to be a soft unimodal distribution, more appropriate for ordinal problems. Given that the beta distribution has two parameters that must be adjusted, a method to automatically determine them is proposed. The regularised loss function is used to train a deep neural network model with an ordinal scheme in the output layer. The results obtained are statistically analysed and show that the combination of these methods increases the performance in ordinal problems. Moreover, the proposed beta distribution performs better than other distributions proposed in previous works, achieving also a reduced computational cost.", awards = "JCR(2022): 8.0 Position: 22/145 (Q1) Category: COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE", comments = "JCR(2022): 8.0 Position: 22/145 (Q1) Category: COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE", doi = "10.1016/j.patcog.2021.108310", issn = "0031-3203", journal = "Pattern Recognition", keywords = "Ordinal regression, Unimodal distribution, Convolutional network, Beta distribution, Stick-breaking", month = "February", note = "JCR(2022): 8.0 Position: 22/145 (Q1) Category: COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE", pages = "108310", title = "{U}nimodal regularisation based on beta distribution for deep ordinal regression", url = "doi.org/10.1016/j.patcog.2021.108310", volume = "122", year = "2022", }

Note:

JCR(2022): 8.0 Position: 22/145 (Q1) Category: COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE

Abstract:

Currently, the use of deep learning for solving ordinal classification problems, where categories follow a natural order, has not received much attention. In this paper, we propose an unimodal regularisation based on the beta distribution applied to the cross-entropy loss. This regularisation encourages the distribution of the labels to be a soft unimodal distribution, more appropriate for ordinal problems. Given that the beta distribution has two parameters that must be adjusted, a method to automatically determine them is proposed. The regularised loss function is used to train a deep neural network model with an ordinal scheme in the output layer. The results obtained are statistically analysed and show that the combination of these methods increases the performance in ordinal problems. Moreover, the proposed beta distribution performs better than other distributions proposed in previous works, achieving also a reduced computational cost.

Comments:

JCR(2022): 8.0 Position: 22/145 (Q1) Category: COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE