Page 60 - SMILESENG

Intl. Summer School on Search- and Machine Learning-based Software Engineering
Start
No
Learning
Generation of n requests
Execution of requests
Predictor training
Generation of 100n requests
Selection of the best ones
Execution of requests
Predictor training
predictor accurate? Yes
is violated: we rule out the possibility that errors are due to individual values using predefined data dictionaries). Finally, the predictor is trained on these observations.
B. Learning
As long as the accuracy of the predictor is less than a con- figurable threshold, active learning is performed by iterations. In each iteration, 100n random requests are generated. From these, the n requests with the highest uncertainty are chosen, where uncertainty is greater the closer the probabilities of the request being valid or invalid are. For example, of the requests shown in the Table, the one with the highest uncertainty is number 3, since the probabilities of being valid (Pv) and invalid (Pi) are very similar (0.49 and 0.51, respectively). Finally, the API is invoked with the chosen requests, the requests are labeled and the predictor is retrained, obtaining another accuracy value.
TABLE I
EXAMPLE OF THREE UNLABELED REQUESTS.
TABLE II EXPERIMENTAL RESULTS.
Fig. 1. Approach diagram.
API
GitHub Stripe
Media
Valid requests (%)
Random AL
62.10 98.65 55.75 99.30
58.93 98.98
Failures Random AL 313 1159
104 284 209 722
ID type visibility
affiliation
‘collaborator’ ‘owner’ ‘collaborator’
sort
- ‘created’ -
direction Pv Pi ‘asc’ 0.9 0.1
- 0.3 0.7 - 0.49 0.51
In this article we present a new technique for the automatic generation of more efficient test cases in REST APIs. Prelimi- nary results show that the technique, based on active learning, achieves up to a ∼ 98% of valid requests, compared to the ∼ 59% achieved by random techniques, in commercial APIs from GitHub and Stripe, tripling the number of errors detected and demonstrating the potential of artificial intelligence for the automatic generation of test cases in REST APIs.
REFERENCES
[1] R. T. Fielding, “Architectural Styles and the Design of Network-based Software Architectures,” Ph.D. dissertation, 2000.
[2] “GitHub API,” accessed January 2020. [Online]. Available: https: //developer.github.com/v3/
[3] A. Martin-Lopez, S. Segura, and A. Ruiz-Cortés, “A Catalogue of Inter-Parameter Dependencies in RESTful Web APIs,” in International Conference on Service-Oriented Computing, 2019, pp. 399–414.
[4] “OpenAPI Specification,” accessed April 2020. [Online]. Available: https://www.openapis.org
[5] A. G. Mirabella, A. Martin-Lopez, S. Segura, L. Valencia-Cabrera, and A. Ruiz-Cortés, “Deep Learning-Based Prediction of Test Input Validity for RESTful APIs,” in International Workshop on Testing for Deep Learning and Deep Learning for Testing, 2021.
[6] B. Settles, “Active learning literature survey,” University of Wisconsin– Madison, Computer Sciences Technical Report 1648, 2009.
[7] X. Zhu, J. Lafferty, and R. Rosenfeld, “Semi-supervised learning with graphs,” Ph.D. dissertation, USA, 2005, aAI3179046.
[8] T. K. Ho, “Random decision forests,” in Proceedings of 3rd international
V. CONCLUSIONS
1 2 3
- ‘all’ ‘private’ - ‘public’ -
IV. EVALUATION
We evaluated the proposed technique on two commercial API operations: reading repositories on GitHub and creating a product on Stripe. For each operation, we generated 2000 requests and tried to maximise the number of valid ones (those getting a status code 2XX). The table shows the percentage of valid requests obtained with our technique (AL) versus those obtained with random techniques (Random). The percentage of valid requests increases a lot with our proposal, reaching a ∼ 98%. Thanks to this, we are able to detect up to three times more errors (failures) of non-conformity with the specification. The proposal has been implemented in Python with pandas and scikit-learn, and the classification technique used is the Random Forest [8].
conference on document analysis and recognition, vol. 1. pp. 278–282.
IEEE, 1995,
48