Automatic Test-Case Generation Based on Rule-Based Behavioral Specification
Subject Areas : electrical and computer engineeringAli Habibi 1 , Ramtin Khosravi 2
1 - School of ECE, University of Tehran, Tehran, Iran
2 - University of Tehran
Keywords: Model-based testing, search-based testing, formal specification,
Abstract :
With the increasing use of software in safety-critical applications, such as the automotive, defense, and medical industries, achieving high levels of assurance regarding the quality of these software systems is essential. Model-based testing is an automated test-case generation method that, on one hand, provides relative assurance by covering a formal description of the system’s behavior, ensuring that various execution scenarios of the program are tested. On the other hand, by automating the generation of these test cases, it significantly reduces the cost of test production. In this research, a model-based testing framework is presented that utilizes a rule-based model and can generate test cases based on two criteria: rule coverage and active rule condition coverage. To generate test cases, this framework employs a search-based approach using a genetic algorithm. The proposed method enables the definition of a system with a large state space and the generation of test cases for it. The framework has been evaluated through a case study on an embedded industrial software, and the evaluation results demonstrate its applicability to real-world problems in the industry.
[1] J. Zander, Model-Based Testing of Real-Time Embedded Systems in the Automotive Domain, Ph.D Thesis, Technical University of Berlin, 2009.
[2] R. N. Charette, "This car runs on code," IEEE Spectrum, vol. 46, no. 3, p. 3, Feb. 2009. [3] J. Babic, M. Siniša, and I. Petrovic, "Introducing model-based techniques into development of real-time embedded applications," Automatika, vol. 52, no. 4, pp. 329-338, Oct. 2011.
[4] A. Kramer and B. Legeard, 2019 Model-Based Testing User Survey: Results, Technical Report, Comité Français des Tests Logiciels, Jan. 2020.
[5] W. Li, F. Le Gall, and N. Spaseski, "A survey on model-based testing tools for test case generation," in Proc. 4th Int. Conf. on Tools and Methods of Program Analysis, pp. 77-89, Moscow, Russia, 3-4 Mar. 2017.
[6] A. Shaout and S. Pattela, "Model based approach for automotive embedded systems," in Proc. 22nd Int. Arab Conf. on Information Technology, 7 pp., Muscat, Oman, 21-23 Dec. 2021.
[7] M. N. Zafar, W. Afzal, and E. Enoiu, "Evaluating system-level test generation for industrial software: a comparison between manual, combinatorial and model-based testing," in Proc. of the 3rd ACM/IEEE Int. Conf. on Automation of Software Test, pp. 148-159, Pittsburgh, PA, USA, 21-22 May 2022.
[8] V. Garousi, A. B. Keleş, Y. Balaman, Z. Özdemir Güler, and A. Arcuri, "Model-based testing in practice: an experience report from the web applications domain," J. of Systems and Software, vol. 180, Article ID: 111032, Oct. 2021.
[9] A. Zakeriyan, R. Khosravi, H. Safari, E. Khamespanah, and S. M. Shamsabadi, "Automated testing of an industrial stock market trading platform based on functional specification," Science of Computer Programming, vol. 225, Article ID: 102908, Jan. 2023.
[10] H. R. Asaadi, R. Khosravi, M. Mousavi, and N. Noroozi, "Towards model-based testing of electronic funds transfer systems," in Proc. Int. Conf. on Fundamentals of Software Engineering, pp. 253-267, Tehran, Iran, 20-22 Apr. 2011.
[11] J. Tretmans, "Test generation with inputs, outputs and repetitive quiescence," Software-Concepts and Tools, vol. 17, no. 3, pp. 103-120, 1996.
[12] M. van der Bijl, A. Resnik, and J. Tretmans, "Compositional testing with ioco," in Proc. Third Int. Workshop on Formal Approaches to Testing of Software, pp. 86-100, Montreal, Canada, 6-6 Oct. 2003.
[13] P. Daca, T. Henzinger, W. Krenn, and D. Nickovic, "Compositional specifications for ioco testing," in Proc. IEEE 7th Int. Conf. on Software Testing, Verification and Validationpp. 373-382, Cleveland, OH, USA, 31 Mar.-4 Apr. 2014.
[14] M. L. Mohd-Shafie, et al., "An EFSM-based test data generation approach in model-based testing," Computers, Materials & Continua, vol. 71, no. 3, pp. 4337-4354, Jan. 2022.
[15] W. Huang, N. Krafczyk, and J. Peleska, "Exhaustive property-oriented model-based testing with symbolic finite state machines," Science of Computer Programming, vol. 231, Article ID: 103005, Aug. 2023.
[16] N. Van Kelecom, S. Silverans, and M. Dutre, "A traceable development and testing methodology for embedded software," in Proc. 12th Graz Symp. on Virtual Vehicle, 8 pp., Graz, Austria, 7-8 May 2019.
[17] M. L. Mohd-Shafie, W. M. N. Wan-Kadir, H. Lichter, M. Khatibsyarbini, and M. Adham-Isa, "Model-based test case generation and prioritization: a systematic literature review," Software and Systems Modeling, vol. 21, no. 2, pp. 751-753, Apr. 2022.
[18] R. Ferdous, C. Hung, F. Kifetew, D. Prandi, and A. Susi, "EvoMBT: evolutionary model-based testing," Science of Computer Programming, vol. 227, Article ID: 102942, Mar. 2023.
[19] U. C. Türker, R. M. Hierons, K. El-Fakih, M. R. Mousavi, and I. Y. Tyukin, "Accelerating finite state machine-based testing using reinforcement learning," IEEE Trans. on Software Engineering, vol. 50, no. 3, pp. 574-597, Mar. 2024.
[20] -, intel/fMBT: Free Model Based tool. URL: https://github.com/intel/fMBT, accessed: Dec. 2022.
[21] M. N. Zafar, et al., "Model-based testing in practice: an industrial case study using graphwalker," in Proc. 14th Innovations in Software Engineering Conf., 11 pp., Bhubaneswar, India, 25-27 Feb. 2021.
[22] V. Aravantinos, S. Voss, S. Mavin, F. Hölzl, and B. Schätz, "AutoFOCUS 3: Tooling Concepts for Seamless, Model-based Development of Embedded Systems", in Joint Proc. of the 8th Int. Workshop on Model-based Architecting of Cyber-physical and Embedded Systems and 1st Int. Workshop on UML Consistency Rules, pp. 19-26, Ottawa, Canada, 28-28 Sept. 2015.
[23] J. Tretmans and P. van de Laar, "Model-based testing with torXakis," in Proc. 30th Central European Conf. on Information and Intelligent Systems, pp. 247-258, Varaždin, Croatia, 2-4 Oct. 2019.
[24] P. Kaur and G. Gupta, "Automated model-based test path generation from UML diagrams via graph coverage tech-niques," International J. of Computer Science and Mobile Computing, vol. 2, no. 7, pp. 302-311, Jul. 2013.
[25] A. Huima, "Implementing conformiq qtronic," in Joint Proc. of 19th IFIP TC 6/WG 6.1 Int. Conf., TestCom 2007, and 7th Int. Workshop Testing of Software and Communicating Systems, 12 pp., Tallin, Estonia, 26-29 Jun. 2007.
[26] G. J. Holzmann, The Spin Model Checker: Primer and Reference Manual, Addison-Wesley, 2004.
[27] Rebeca Modeling Language, https://rebeca-lang.org, accessed Feb. 2024.
[28] Event-B, https://www.event-b.org, accessed Feb. 2024.
[29] M. Atif and J. F. Groote, Understanding Behaviour of Distributed Systems Using mCRL2, Springer, 2023.
[30] A. Mavin, P. Wilkinson, A. Harwood, and M. Novak, "Easy approach to requirements syntax (EARS)," in Proc. 17th IEEE Int. Requirements Engineering Conf., pp. 317-322, Atlanta, GA, USA, 31 Aug.-4 Sep. 2009.
[31] P. McMinn, "Search-based software testing: past, present and future," in ¬Proc. IEEE 4th Int. Conf. on Software Testing, Verification and Validation Workshops, pp. 153-163, Berlin, Germany, 21-25 Mar. 2011.
[32] M. Khari and P. Kumar, "An extensive evaluation of search-based software testing: a review," Soft Computing, vol. 23, pp. 1933-1946, Mar. 2019.
[33] P. Ammann and J. Offutt, Introduction to Software Testing, 2nd Ed., Cambridge University Press, 2016.
[34] K. Forsberg and H. Mooz, "The relationship of systems engineering to the project cycle," Engineering Management J., vol. 4, no. 3, pp. 36-43, 1992.
