Optimising Convolutional Neural Networks using a Hybrid Statistically-driven Coral Reef Optimisation algorithm

Research areas:

• Methodology - Evolutionary artificial neural networks

Year:

2020

Type of Publication:

Article

Keywords:

Neuroevolution, Convolutional Neural Networks, Optimisation, Hybridisation, Coral Reef Optimisation

Authors:

• Martín, Alejandro
• Vargas, Víctor Manuel
• Gutiérrez, Pedro Antonio
• Camacho, David
• Hervás-Martínez, César

Journal:

Applied Soft Computing

Volume:

90

Pages:

106144

Month:

May

ISSN:

1568-4946

BibTex:

@article{DeepSCRO2020, author = "Alejandro Mart{\'i}n and V{\'i}ctor Manuel Vargas and Pedro Antonio Guti{\'e}rrez and David Camacho and C{\'e}sar Herv{\'a}s-Mart{\'i}nez", abstract = "Convolutional Neural Networks stands at the front of many solutions which deal with computer vision related tasks. The use and the applications of these models are growing unceasingly, as well as the complexity required to deal with bigger and highly complex problems. However, hitting the most suitable model for solving a specific task is not trivial. A very manually intensive and time consuming trial-and-error experimentation is needed in order to find an architecture, hyperparameters and parameters which reach a certain level of performance. Moreover, this process leads to oversized models, diminishing their generalisation capacity. In this paper, we leverage a metaheuristic and a hybridisation process to optimise the reasoning block of CNN models, composed by fully connected and dropout layers, conducting a full reconstruction that leads to lighter models with better performance. Our approach is architecture-independent and operates at the topology, hyperparameters and parameters (connection weights) levels. For that purpose, we have implemented the Hybrid Statistically-driven Coral Reef Optimisation (HSCRO) algorithm as an extension of SCRO, a metaheuristic which does not require to adjust any parameter since they are automatically and dynamically chosen based on the statistical characteristics of the evolution. In addition, a hybridisation process employs the backpropagation algorithm to make a final fine-grained weights adjustment. In the experiments, the VGG-16 model is successfully optimised in two different scenarios (the CIFAR-10 and the CINIC-10 datasets), resulting in a lighter architecture, with an 88% reduction of the connection weights, but without losing its generalisation performance.", awards = "JCR(2020): 6.725 Position: 11/112 (Q1) Category: COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS", comments = "JCR(2020): 6.725 Position: 11/112 (Q1) Category: COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS", doi = "10.1016/j.asoc.2020.106144", issn = "1568-4946", journal = "Applied Soft Computing", keywords = "Neuroevolution, Convolutional Neural Networks, Optimisation, Hybridisation, Coral Reef Optimisation", month = "May", note = "JCR(2020): 6.725 Position: 11/112 (Q1) Category: COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS", pages = "106144", title = "{O}ptimising {C}onvolutional {N}eural {N}etworks using a {H}ybrid {S}tatistically-driven {C}oral {R}eef {O}ptimisation algorithm", url = "doi.org/10.1016/j.asoc.2020.106144", volume = "90", year = "2020", }

Note:

JCR(2020): 6.725 Position: 11/112 (Q1) Category: COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS

Abstract:

Convolutional Neural Networks stands at the front of many solutions which deal with computer vision related tasks. The use and the applications of these models are growing unceasingly, as well as the complexity required to deal with bigger and highly complex problems. However, hitting the most suitable model for solving a specific task is not trivial. A very manually intensive and time consuming trial-and-error experimentation is needed in order to find an architecture, hyperparameters and parameters which reach a certain level of performance. Moreover, this process leads to oversized models, diminishing their generalisation capacity. In this paper, we leverage a metaheuristic and a hybridisation process to optimise the reasoning block of CNN models, composed by fully connected and dropout layers, conducting a full reconstruction that leads to lighter models with better performance. Our approach is architecture-independent and operates at the topology, hyperparameters and parameters (connection weights) levels. For that purpose, we have implemented the Hybrid Statistically-driven Coral Reef Optimisation (HSCRO) algorithm as an extension of SCRO, a metaheuristic which does not require to adjust any parameter since they are automatically and dynamically chosen based on the statistical characteristics of the evolution. In addition, a hybridisation process employs the backpropagation algorithm to make a final fine-grained weights adjustment. In the experiments, the VGG-16 model is successfully optimised in two different scenarios (the CIFAR-10 and the CINIC-10 datasets), resulting in a lighter architecture, with an 88% reduction of the connection weights, but without losing its generalisation performance.

Comments:

JCR(2020): 6.725 Position: 11/112 (Q1) Category: COMPUTER SCIENCE, INTERDISCIPLINARY APPLICATIONS