A structural distance based crossover for neural network classifiers

Research areas:

• Methodology - Machine learning

Year:

2012

Type of Publication:

Article

Keywords:

Artificial Neural Networks, multiclass classification, product units, evolutionary algorithms, memetic algorithms, crossover, niching techniques

Authors:

• Moreno, Manuel
• Gutiérrez, Pedro Antonio
• Hervás-Martínez, César

Journal:

International Journal of Pattern Recognition and Artificial Intelligence

Volume:

26

Number:

6

Pages:

1250012

ISSN:

0218-0014

BibTex:

@article{DBCMoreno2012, author = "Manuel Moreno and Pedro Antonio Guti{\'e}rrez and C{\'e}sar Herv{\'a}s-Mart{\'i}nez", abstract = "This paper presents a structural distance based crossover for neural network classifiers, which is applied as part of a Memetic Algorithm (MA) for evolving simultaneously the structure and weights of neural network models applied to multiclass problems. Previous researchers have shown that this simultaneous evolution is a way to avoid the noisy fitness evaluation. The MA incorporates a crossover operator that shows to be useful for ameliorating the permutation problem of the network representation (i.e. different genotypes can be used to represent the same neural network phenotype), increasing the structural diversity of the individuals and improving the accuracy of the results. Instead of a recombination probability, the crossover operator considers a similarity parameter (the minimum structural distance), which allows to maintain a trade-off between global and local search. The neural network models selected in this work are the Product-Unit Neural Networks (PUNNs), due to their increasing relevance in those classification problems which show a high order relationship between the input variables. The proposed MA is intended to reduce the possible overtraining problems which can raise in some datasets for this kind of models. The evolutionary system is applied to eight classification benchmarks and the results of an Analysis of Variance contrast (ANOVA) show the effectiveness of the structural based crossover operator and the capacity of our algorithm to obtain evolved PUNNs with a higher classification accuracy than those obtained using other evolutionary techniques. On the other hand, the results obtained are compare with popular effective machine learning classification methods, resulting in a competitive performance. ", awards = "JCR (2012): 0.562 (category COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE, position 94/115 Q4)", comments = "JCR (2012): 0.562 (category COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE, position 94/115 Q4)", doi = "10.1142/S0218001412500127", issn = "0218-0014", journal = "International Journal of Pattern Recognition and Artificial Intelligence", keywords = "Artificial Neural Networks, multiclass classification, product units, evolutionary algorithms, memetic algorithms, crossover, niching techniques", note = "JCR (2012): 0.562 (category COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE, position 94/115 Q4)", number = "6", pages = "1250012", title = "{A} structural distance based crossover for neural network classifiers", url = "http://dx.doi.org/10.1142/S0218001412500127 ", volume = "26", year = "2012", }

Note:

JCR (2012): 0.562 (category COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE, position 94/115 Q4)

Abstract:

This paper presents a structural distance based crossover for neural network classifiers, which is applied as part of a Memetic Algorithm (MA) for evolving simultaneously the structure and weights of neural network models applied to multiclass problems. Previous researchers have shown that this simultaneous evolution is a way to avoid the noisy fitness evaluation. The MA incorporates a crossover operator that shows to be useful for ameliorating the permutation problem of the network representation (i.e. different genotypes can be used to represent the same neural network phenotype), increasing the structural diversity of the individuals and improving the accuracy of the results. Instead of a recombination probability, the crossover operator considers a similarity parameter (the minimum structural distance), which allows to maintain a trade-off between global and local search. The neural network models selected in this work are the Product-Unit Neural Networks (PUNNs), due to their increasing relevance in those classification problems which show a high order relationship between the input variables. The proposed MA is intended to reduce the possible overtraining problems which can raise in some datasets for this kind of models. The evolutionary system is applied to eight classification benchmarks and the results of an Analysis of Variance contrast (ANOVA) show the effectiveness of the structural based crossover operator and the capacity of our algorithm to obtain evolved PUNNs with a higher classification accuracy than those obtained using other evolutionary techniques. On the other hand, the results obtained are compare with popular effective machine learning classification methods, resulting in a competitive performance.

Comments:

JCR (2012): 0.562 (category COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE, position 94/115 Q4)