Interactivity in the Generation of Test Cases with Evolutionary Computation
Publication
Currently under review.
Abstract
TO-DO
Additional material
Introduction to EvoSuite
TO-DO
Step-by-step illustrative execution
1. Start of the search until secondary objective activation
The search starts with the new secondary objective disabled. Hopefully, the fitness of the candidates –i.e., their coverage of goals– will be increased overall with each new generation. When the coverage of the best candidate is greater or equal to Revise_after_percentage_of_coverage, the secondary objective is enabled. In our example, this happens at the 15th generation:
2. Selection of candidates
In our example, 26 out of 50 candidates present the same best fitness. However, according to Percentage_to_revise, only 3 of them can be selected for revision. These three candidates are then chosen at random:
3. Minimization
The selected candidates are minimized. In our example, two of the three candidates present exactly the same minimization. Therefore, they are grouped together and only two test suites are shown to the tester:
4. Interaction with the tester for readability assignment
At this point, EvoSuite is prepared for an interaction. The two minimized versions are:
The execution is paused to wait for the tester’s feedback. In our example, the second minimized test suite is given a better score (8) than the first one (6):
5. Information integration and sorting
The readability scores are transferred to their respective candidate test suites, including those in the same minimization group. These candidates are re-ordered in the population going from highest to lowest legibility score. In our example, the test suite with id=119 has the best score (legibilityValue=8), and, therefore, is placed at the first position:
6. Evolution continues until next interaction
The search is resumed, checking if new interactions are needed to break ties at the configured Frequency (10 generations). In our example, no ties are produced at the 25th generation, so the next interaction happens at the 35th generation. Since the maximun number of interactions (Max_times_sort) has not been exhausted, a new interaction is prepared. As in step 3, minimization is executed for the three candidate solutions selected. In this case, each test suite has a different minimized version:
7. Second interaction with the tester for readability assignment
The three minimized test suites are shown to the tester, who assigns new readability scores:
8. Final test suite returned after the search
No additional interactions are scheduled, so the evolutionary algorithm ends the search. The returned test suite corresponds to the candidate solution with highest coverage and best readability coverage (the 3rd candidate test suite in the last interaction). Notice that EvoSuite adds some test cases to cover additional goals at the end of the search, using an archive of pairs (test case, test goal) taken from any individual within the population. Final statistics are computed, showing that including the readability assessment does not impede coverage improvement.
