A two dimensional accuracy-based measure for classification performance

Áreas de investigación:

• Methodology - Machine learning

Año:

2017

Tipo de publicación:

Artículo

Autores:

• Carbonero-Ruz, Mariano
• Martínez-Estudillo, Francisco José
• Fernandez-Navarro, Francisco
• Becerra-Alonso, David
• Martínez-Estudillo, Alfonso Carlos

Journal:

Information Sciences

Volumen:

382-383

Páginas:

60-80

BibTex:

@article{52016, author = "Mariano Carbonero-Ruz and Francisco Jos{\'e} Mart{\'i}nez-Estudillo and Francisco Fernandez-Navarro and David Becerra-Alonso and Alfonso Carlos Mart{\'i}nez-Estudillo", abstract = "Accuracy has been used traditionally to evaluate the performance of classifiers. However, it is well known that accuracy is not able to capture all the different factors that characterize the performance of a multiclass classifier. In this manuscript, accuracy is studied and analyzed as a weighted average of the classification rate of each class. This perspective allows us to propose the dispersion of the classification rate of each class as its complementary measure. In this sense, a graphical performance metric, which is defined in a two dimensional space composed by accuracy and dispersion, is proposed to evaluate the performance of classifiers. We show that the combined values of accuracy and dispersion must fall within a clearly bounded two dimensional region, different for each problem. The nature of this region depends only on the a priori probability of each class, and not on the classifier used. Thus, the performance of multiclassifiers is represented in a two dimensional space where the models can be compared in a more fair manner, providing greater awareness of the strategies that are more accurate when trying to improve the performance of a classifier. Furthermore we experimentally analyse the behaviour of seven different performance metrics based on the computation of the confusion matrix values in several scenarios, identifying clusters and relationships between measures. As shown in the experimentation, the graphical metric proposed is specially suitable in challenging, highly imbalanced and with a high number of classes datasets. The approach proposed is a novel point of view to address the evaluation of multiclassifiers and it is an alternative to other evaluation measures used in machine learning.", awards = "JCR(2017): 4.305 Position: 12/148 (Q1) Category: COMPUTER SCIENCE, INFORMATION SYSTEMS", comments = "JCR(2017): 4.305 Position: 12/148 (Q1) Category: COMPUTER SCIENCE, INFORMATION SYSTEMS", doi = "http://dx.doi.org/10.1016/j.ins.2016.12.005", journal = "Information Sciences", note = "JCR(2017): 4.305 Position: 12/148 (Q1) Category: COMPUTER SCIENCE, INFORMATION SYSTEMS", pages = "60-80", title = "{A} two dimensional accuracy-based measure for classification performance", url = "http://www.sciencedirect.com/science/article/pii/S0020025516319181", volume = "382-383", year = "2017", }

Nota:

JCR(2017): 4.305 Position: 12/148 (Q1) Category: COMPUTER SCIENCE, INFORMATION SYSTEMS

Abstract:

Accuracy has been used traditionally to evaluate the performance of classifiers. However, it is well known that accuracy is not able to capture all the different factors that characterize the performance of a multiclass classifier. In this manuscript, accuracy is studied and analyzed as a weighted average of the classification rate of each class. This perspective allows us to propose the dispersion of the classification rate of each class as its complementary measure. In this sense, a graphical performance metric, which is defined in a two dimensional space composed by accuracy and dispersion, is proposed to evaluate the performance of classifiers. We show that the combined values of accuracy and dispersion must fall within a clearly bounded two dimensional region, different for each problem. The nature of this region depends only on the a priori probability of each class, and not on the classifier used. Thus, the performance of multiclassifiers is represented in a two dimensional space where the models can be compared in a more fair manner, providing greater awareness of the strategies that are more accurate when trying to improve the performance of a classifier. Furthermore we experimentally analyse the behaviour of seven different performance metrics based on the computation of the confusion matrix values in several scenarios, identifying clusters and relationships between measures. As shown in the experimentation, the graphical metric proposed is specially suitable in challenging, highly imbalanced and with a high number of classes datasets. The approach proposed is a novel point of view to address the evaluation of multiclassifiers and it is an alternative to other evaluation measures used in machine learning.

Comentarios:

JCR(2017): 4.305 Position: 12/148 (Q1) Category: COMPUTER SCIENCE, INFORMATION SYSTEMS