Page 51 - SMILESENG

Intl. Summer School on Search- and Machine Learning-based Software Engineering
Improving Search-based Test Case Generation by means of Interactive Evolutionary Computation
Pedro Delgado-Pe´rez Dpto. de Ingenier´ıa Informa´tica University of Ca´diz, Spain
pedro.delgado@uca.es
Inmaculada Medina-Bulo Dpto. de Ingenier´ıa Informa´tica University of Ca´diz, Spain
inmaculada.medina@uca.es
Abstract—This talk summarizes our research efforts towards getting more human-like and competitive results in search-based software testing (SBST). Our ongoing work seeks to enhance automated SBST tools like EvoSuite by incorporating the tester’s knowledge and preferences through interactive evolutionary com- putation. In our envisioned tester-centered interactive search approach, the tester and the evolutionary algorithm cooperate to reach more satisfactory test suites in terms of their detection capability and readability. Here, we present our current ideas and initial steps in this direction, and discuss current challenges.
I. CONTEXT AND MOTIVATION
Automated and efficient testing tools have been developed in the last years to guide the generation of effective test suites, thus reducing the costs of manual testing [1]. In this context, search-based software testing (SBST) has been revealed as a fruitful approach, clearly illustrated by the success of Evo- Suite [2]. EvoSuite provides several genetic algorithms that can be applied to evolve a population of whole test suites or individual test cases for Java classes. Individuals represent test cases encoded as sequences of method invocations, and are iteratively modified by crossover and mutation operators. The quality of the test cases is assessed by a configurable fitness function that maximizes different coverage targets.
As EvoSuite has gained popularity and maturity, its ability to detect real faults on industrial systems has been analyzed too [3]. This study shows that SBST tools still have some limitations to reveal complex faults due to the difficulty to reach some parts of the code or to generate convenient com- binations of input values to detect the fault. Furthermore, other works have evaluated the comprehension of automatically generated test suites and the willingness of testers to accept them [4]. These studies reveal that testers perceive automatic test suites as less helpful than those manually designed, which clearly hamper their possible adoption in industrial contexts. To overcome this, some authors have proposed fully automated techniques to improve the appearance of the final test suite [5], with the ultimate goal of gaining testers’ acceptance.
Jose´ Rau´l Romero
Dpto. Informa´tica y Ana´lisis Nume´rico University of Co´rdoba, Spain
jrromero@uco.es
Our vision is that one alternative option to address both issues —detection capability and readability— is to put the tester in the loop. Interactive search-based software engi- neering (iSBSE) [6] proposes showing intermediate results to the engineer and incorporate their feedback into the search process. Designing an effective interactive experience is a hard task, requiring a careful design of the algorithm to prevent fatigue and information overload. Interactive approaches for SBST have only been explored in the context of test data generation for embedded software [7]. Similarly, combining the best of automated tools and the knowledge of testers about the system under test (SUT) seems to be a better approach to take one step further in the test case generation process. We have high expectations on the possibilities that an interactive approach could bring to such process, improving the tester’s experience when using SBST tools and fostering a wider acceptance from the software industry.
II. INTERACTIVE OPPORTUNITIES IN SBST
The design of an interactive search algorithm involves many decisions that should be adapted to the particularities of the problem domain and the purpose of the interaction. In a recent work [8], we analyzed the general interactive options compiled by the iSBSE review [6], mapping them to the test case generation problem. Figure 1 summarizes the proposed framework to design algorithm variants depending on whether the interaction is focused on improving test case readability, fault detection capability, or both.
Involving the tester in the evaluation could be a proper way to assess the readability of the test cases, while his/her knowledge about the SUT could be exploited to guide the search towards the detection of hard faults. During an inter- action, the tester could be asked to evaluate characteristics of the candidate test cases, but also modify their statements and parameters, or choose between similar test cases (e.g., those that detect the same mutant). As for the interaction scheduling, internal information about the evolution could be considered. For instance, the level of achieved coverage can be useful
Aurora Ram´ırez
Dpto. Informa´tica y Ana´lisis Nume´rico University of Co´rdoba, Spain
aramirez@uco.es
Kevin J. Valle-Go´mez Dpto. de Ingenier´ıa Informa´tica University of Ca´diz, Spain
kevin.valle@uca.es
39