Publication
A. Ramírez, R. Barbudo and J.R. Romero*. “An experimental comparison of metaheuristic optimization frameworks for multi-objective optimization”. Expert Systems, vol. 40(4), e12672. 2023.
Abstract
Multi-objective optimization problems frequently appear in many diverse research areas and application domains. Metaheuristics, as efficient techniques to solve them, need to be easily accessible to users with different expertise and programming skills. In this context, metaheuristic optimization frameworks are helpful, as they provide popular algorithms, customizable components and additional facilities to conduct experiments. Due to the broad range of available tools, this paper presents a systematic evaluation and experimental comparison of ten frameworks, covering from multi-purpose mature tools to recent libraries specifically designed for multi-objective optimization. The evaluation is organized around seven characteristics: search components and techniques, configuration, execution, utilities, external support and community, software implementation and runtime performance. An analysis of code measures and a series of experiments serve to assess the last two features. Open issues and lessons learned are also discussed as part of the comparative study. The outcomes of the evaluation process reveal a contrasted support to recent advances in multi-objective optimization, with a lack of novel algorithms and types of metaheuristics others than evolutionary algorithms. The experimental comparison also reports significant differences in terms of both execution time and memory usage under demanding configurations.
Frameworks
The following table shows the list of frameworks under evaluation:
| Framework | Version | Year | Website |
|---|---|---|---|
| DEAP | 2.13 | 2019 | https://github.com/DEAP/deap |
| ECJ | 27 | 2019 | https://cs.gmu.edu/~eclab/projects/ecj/ |
| EvA | 2.2.0 | 2015 | http://www.ra.cs.uni-tuebingen.de/software/eva2/ |
| HeuristicLab | 3.3.15 | 2019 | http://dev.heuristiclab.com/ |
| JCLEC-MO | 1.0 | 2019 | http://www.uco.es/grupos/kdis/jclec-mo |
| jMetal | 5.9 | 2019 | http://jmetal.github.io/jMetal/ |
| MOEA Framework | 2.13 | 2019 | http://moeaframework.org/ |
| Opt4J | 3.1.4 | 2015 | http://opt4j.sourceforge.net/ |
| ParadisEO-MOEO | 2.0.1 | 2012 | http://paradiseo.gforge.inria.fr/ |
| Platypus | 1.0.4 | 2020 | https://github.com/Project-Platypus/Platypus |
Additional material
The full comparative study is provided as a technical report, which includes extended information for some of the characteristics under analysis and the complete experimental results.
Technical report available for download here (PDF, 411 KB)