MT-Forward Algorithm for Prediction of Project Categories Based on Selection of Mutants

Sasa Ani Arnomo, Noraini Ibrahim

Abstract


Mutation testing is an effective technique for errors. Although effective, mutation testing has the main limitation that it is awfully expensive because it requires a series of tests on each mutant. Therefore, many researchers have focused on presenting various techniques to reduce the cost of mutation testing. In this study, the MT-Forward algorithm was developed. The concept of MT-Forward is mutation testing that connects the selection and prediction methods. As a result of the selection, the mutant operator can make a more efficient selection of the number of mutants obtained. The shrinkage of the selected mutants was from 1,019 live mutants to 749 live mutants as a priority. There are 15 features used in this study. The required features are obtained from the Locmetrix Test, Coverage Test, and Mutation Testing. The accuracy value of each method is almost the same, which distinguishes this study is the selection of important operators who prioritize improvement programs without reducing the level of accuracy. Where the PMS Method uses 43 Mutant Operators, PMT Method 22 Mutant Operators, while MT-Forward uses 7 mutant operators as priority mutant operators. The method used to evaluate the performance of the algorithm is 5-fold cross-validation. The accuracy of each method is PMS 57.14%, PMT Method 57.14%, and MT-Forward is 95.00%. MT-Forward states that combining mutant selection techniques and project category prediction is important for fixing faults.

Keywords


Prediction; selection; mutants; mutation testing; MT-Forward.

Full Text:

PDF PDF

References


A. Mustafa, W. M. N. Wan-Kadir, and N. Ibrahim, “Comparative evaluation of the state-of-art requirements-based test case generation approaches,†Int. J. Adv. Sci. Eng. Inf. Technol., vol. 7, no. 4–2 Special Issue, pp. 1567–1573, 2017.

F. F. Ismail, R. Razali, and Z. Mansor, “Considerations for cost estimation of software testing outsourcing projects,†Int. J. Adv. Sci. Eng. Inf. Technol., vol. 9, no. 1, pp. 142–152, 2019.

A. Aghamohammadi, S. H. Mirian-Hosseinabadi, and S. Jalali, “Statement frequency coverage: A code coverage criterion for assessing test suite effectiveness,†Inf. Softw. Technol., vol. 129, no. September 2020, p. 106426, 2021.

A. Usman, N. Ibrahim, and I. A. Salihu, “TEGDroid: Test case generation approach for android apps considering context and GUI events,†Int. J. Adv. Sci. Eng. Inf. Technol., vol. 10, no. 1, pp. 16–23, 2020.

R. Romli, S. Sarker, M. Omar, and M. Mahmod, “Automated test cases and test data generation for dynamic structural testing in automatic programming assessment using MC/DC,†Int. J. Adv. Sci. Eng. Inf. Technol., vol. 10, no. 1, pp. 120–127, 2020.

P. Ma, H. Cheng, J. Zhang, and J. Xuan, “Can this fault be detected: A study on fault detection via automated test generation,†J. Syst. Softw., vol. 170, p. 110769, 2020.

J. M. Zhang, L. Zhang, D. Hao, L. Zhang, and M. Harman, “An empirical comparison of mutant selection assessment metrics,†Proc. - 2019 IEEE 12th Int. Conf. Softw. Testing, Verif. Valid. Work. ICSTW 2019, pp. 90–101, 2019.

M. A. Guimaraes, L. Fernandes, M. Ribeiro, M. D’Amorim, and R. Gheyi, “Optimizing Mutation Testing by Discovering Dynamic Mutant Subsumption Relations,†Proc. - 2020 IEEE 13th Int. Conf. Softw. Testing, Verif. Validation, ICST 2020, pp. 198–208, 2020.

E. Guerra, J. Sánchez Cuadrado, and J. De Lara, “Towards Effective Mutation Testing for ATL,†Proc. - 2019 ACM/IEEE 22nd Int. Conf. Model Driven Eng. Lang. Syst. Model. 2019, no. label 1, pp. 78–88, 2019.

X. Dang, X. Yao, D. Gong, T. Tian, and B. Sun, “Multi-Task Optimization-Based Test Data Generation for Mutation Testing via Relevance of Mutant Branch and Input Variable,†IEEE Access, vol. 8, pp. 144401–144412, 2020.

A. V. Pizzoleto, F. C. Ferrari, L. D. Dallilo, and J. Offutt, “SiMut: Exploring Program Similarity to Support the Cost Reduction of Mutation Testing,†Proc. - 2020 IEEE 13th Int. Conf. Softw. Testing, Verif. Valid. Work. ICSTW 2020, pp. 264–273, 2020.

Z. Cui, M. Jia, X. Chen, L. Zheng, and X. Liu, “Improving software fault localization by combining spectrum and mutation,†IEEE Access, vol. 8, pp. 172296–172307, 2020.

N. Chetouane, F. Wotawa, H. Felbinger, and M. Nica, “On Using k-means Clustering for Test Suite Reduction,†Proc. - 2020 IEEE 13th Int. Conf. Softw. Testing, Verif. Valid. Work. ICSTW 2020, pp. 380–385, 2020.

A. Derezińska, “Soft Computing in Computer and Information Science,†vol. 342, 2015.

J. Zhang, L. Zhang, M. Harman, D. Hao, Y. Jia, and L. Zhang, “Predictive Mutation Testing,†IEEE Trans. Softw. Eng., vol. 14, no. 8, 2018.

A. Derezińska, “A quality estimation of mutation clustering in C# programs,†Adv. Intell. Syst. Comput., vol. 224, pp. 119–129, 2013.

M. Al-Hajjaji, J. Krüger, F. Benduhn, T. Leich, and G. Saake, “Efficient Mutation Testing in Configurable Systems,†Proc. - 2017 IEEE/ACM 2nd Int. Work. Var. Complex. Softw. Des. VACE 2017, pp. 2–8, 2017.

Dana H Halabi & Adnan Shaout, “Mutation Testing Tools for Java Programs – a Survey,†Int. J. Comput. Sci. Eng. , vol. 5, no. 4, pp. 11–22, 2016.

A. Duque-Torres, N. Doliashvili, D. Pfahl, and R. Ramler, “Predicting Survived and Killed Mutants,†Proc. - 2020 IEEE 13th Int. Conf. Softw. Testing, Verif. Valid. Work. ICSTW 2020, pp. 274–283, 2020.

L. Madeyski and N. Radyk, “Judy – a mutation testing tool for Java,†IET Softw., vol. 4, no. 1, pp. 32–42, 2010.

S. A. Arnomo, N. B. Ibrahim, and A. Maslan, “Mutation score prediction based on traditional operator of MUJAVA using logistic regression,†J. Adv. Res. Dyn. Control Syst., vol. 11, no. 8 Special Issue, pp. 1230–1238, 2019.

M. Kintis, M. Papadakis, Y. Jia, N. Malevris, Y. Le Traon, and M. Harman, “Detecting Trivial Mutant Equivalences via Compiler Optimisations,†IEEE Trans. Softw. Eng., vol. 44, no. 4, pp. 308–333, 2018.

A. S. Ghiduk, M. R. Girgis, and M. H. Shehata, “Employing Dynamic Symbolic Execution for Equivalent Mutant Detection,†IEEE Access, vol. 7, pp. 163767–163777, 2019.

X. Dang, X. Yao, D. Gong, and T. Tian, “Efficiently Generating Test Data to Kill Stubborn Mutants by Dynamically Reducing the Search Domain,†IEEE Trans. Reliab., vol. 69, no. 1, pp. 334–348, 2020.

H. Gu, J. Zhang, M. Chen, T. Wei, L. Lei, and F. Xie, “Specification-Driven Conformance Checking for Virtual/Silicon Devices Using Mutation Testing,†IEEE Trans. Comput., vol. 70, no. 3, pp. 400–413, 2021.

A. Panichella, F. M. Kifetew, and P. Tonella, “Automated Test Case Generation as a Many-Objective Optimisation Problem with Dynamic Selection of the Targets,†IEEE Trans. Softw. Eng., vol. 44, no. 2, pp. 122–158, 2018.

P. Delgado-Pérez and F. Chicano, “An experimental and practical study on the equivalent mutant connection: An evolutionary approach,†Inf. Softw. Technol., vol. 124, no. April, 2020.

S. A. Arnomo and N. Binti Ibrahim, “Priority path for mutant repairs on mutation testing,†Proc. ICAITI 2019 - 2nd Int. Conf. Appl. Inf. Technol. Innov. Explor. Futur. Technol. Appl. Inf. Technol. Innov., pp. 71–76, 2019.

K. Jalbert and J. S. Bradbury, “Predicting mutation score using source code and test suite metrics,†2012 1st Int. Work. Realiz. AI Synerg. Softw. Eng. RAISE 2012 - Proc., pp. 42–46, 2012.

M. R. Naeem, T. Lin, H. Naeem, F. Ullah, and S. Saeed, “Scalable Mutation Testing Using Predictive Analysis of Deep Learning Model,†IEEE Access, vol. 7, pp. 158264–158283, 2019.

M. Deon Bordignon and R. A. Silva, “Mutation Operators for Concurrent Programs in Elixir,†21st IEEE Latin-American Test Symp. LATS 2020, 2020.

X. Yao, G. Zhang, F. Pan, D. Gong, and C. Wei, “Orderly Generation of Test Data via Sorting Mutant Branches Based on Their Dominance Degrees for Weak Mutation Testing,†IEEE Trans. Softw. Eng., vol. 5589, no. c, pp. 1–17, 2020.

G. Grano, F. Palomba, and H. C. Gall, “Lightweight Assessment of Test-Case Effectiveness Using Source-Code-Quality Indicators,†IEEE Trans. Softw. Eng., vol. 47, no. 4, pp. 758–774, 2019.

J. Zhang et al., “A Study of Programming Languages and Their Bug Resolution Characteristics,†IEEE Trans. Softw. Eng., vol. PP, no. X, p. 1, 2019.

L. Villalobos-Arias, C. Quesada-López, A. Martínez, and M. Jenkins, “Evaluation of a model-based testing platform for Java applications,†IET Softw., vol. 14, no. 2, pp. 115–128, 2020.

J. Lee, S. Kang, and P. Jung, “Test coverage criteria for software product line testing: Systematic literature review,†Inf. Softw. Technol., vol. 122, no. January, p. 106272, 2020.

S. Yang, S. Huang, and Z. Hui, “Theoretical Analysis and Empirical Evaluation of Coverage Indictors for Closed Source APP Testing,†IEEE Access, vol. 7, pp. 162323–162332, 2019.

C. Lu, J. Zhong, Y. Xue, L. Feng, and J. Zhang, “Ant Colony System with Sorting-Based Local Search for Coverage-Based Test Case Prioritization,†IEEE Trans. Reliab., vol. 69, no. 3, pp. 1004–1020, 2020.

K. Amanullah and T. Bell, “Evaluating the Use of Remixing in Scratch Projects Based on Repertoire, Lines of Code (LOC), and Elementary Patterns,†Proc. - Front. Educ. Conf. FIE, vol. 2019-Octob, pp. 1–8, 2019.




DOI: http://dx.doi.org/10.18517/ijaseit.12.5.14296

Refbacks

  • There are currently no refbacks.


Published by INSIGHT - Indonesian Society for Knowledge and Human Development