Towards Multi-Faceted Test Cases Optimization
ABSTRACT
The target of software engineering is to produce high quality software product at low cost. Software testing is labour-intensive, ambiguous and error prone activity of software development. How to provide cost-effective strategies for software test cases optimization problem such as classification, minimization, selection, and prioritization has been one of the research focuses in software testing for a long time. Many researchers and academicians have addressed the effectiveness/fitness and optimization of test cases, and obtained many interesting results. However, one issue of paramount importance in software testing i.e. the intrinsic imprecise and uncertainty of test cases fitness, fitness parameters, multi-objective optimization, is left unaddressed. Test cases fitness depends on several parameters. Vagueness of fitness of test cases and their fitness parameters have created the uncertainty in test cases optimization. Cost and adequacy values are incorporated into multi-faceted optimization of test cases. This paper argues test cases optimization requires multi-faceted optimization in order to adequately cater realistic software testing. In this paper, authors have identified several parameters for test cases fitness and multiple objectives for test cases optimization. In addition above, authors have formulated the test cases optimization problem in three different ways using multi-faceted concept. These formulations can be used in future by authors and researchers.
The target of software engineering is to produce high quality software product at low cost. Software testing is labour-intensive, ambiguous and error prone activity of software development. How to provide cost-effective strategies for software test cases optimization problem such as classification, minimization, selection, and prioritization has been one of the research focuses in software testing for a long time. Many researchers and academicians have addressed the effectiveness/fitness and optimization of test cases, and obtained many interesting results. However, one issue of paramount importance in software testing i.e. the intrinsic imprecise and uncertainty of test cases fitness, fitness parameters, multi-objective optimization, is left unaddressed. Test cases fitness depends on several parameters. Vagueness of fitness of test cases and their fitness parameters have created the uncertainty in test cases optimization. Cost and adequacy values are incorporated into multi-faceted optimization of test cases. This paper argues test cases optimization requires multi-faceted optimization in order to adequately cater realistic software testing. In this paper, authors have identified several parameters for test cases fitness and multiple objectives for test cases optimization. In addition above, authors have formulated the test cases optimization problem in three different ways using multi-faceted concept. These formulations can be used in future by authors and researchers.
Cite this paper
nullM. Kumar, A. Sharma and R. Kumar, "Towards Multi-Faceted Test Cases Optimization," Journal of Software Engineering and Applications, Vol. 4 No. 9, 2011, pp. 550-557. doi: 10.4236/jsea.2011.49064.
nullM. Kumar, A. Sharma and R. Kumar, "Towards Multi-Faceted Test Cases Optimization," Journal of Software Engineering and Applications, Vol. 4 No. 9, 2011, pp. 550-557. doi: 10.4236/jsea.2011.49064.
References
[1] D. S. Alberts, “The Economics of Software,” Proceedings of National Computer Conference on Quality Assurance, Montvale, Vol. 45, 1976, pp. 433-442. [2] S. Hema and L. Williams, “On the Economics of Requirements Based Test Case Prioritization,” ACM SIGSOFT Software Engineering Notes, Vol. 30, No. 4, 2005, pp. 1-3. [3] Y. Shin and M. Harman, “Pareto Efficient Multi-Objective Test Case Selection,” Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), London, ACM press, 2007, pp. 140-150. [4] D. J. Mala and V. Mohan, “ABC Tester—Artificial Bee Colony Based Software Test Suite Optimization Approach,” International Journal of Software Engineering, Vol. 2, No. 2, 2009, pp. 15-43. [5] D. J. Mala, V. Mohan, “Quality Improvement and Optimization of Test Cases—A Hybrid Genetic Algorithm Based Approach,” ACM SIGSOFT Software Engineering Notes, Vol. 35, No. 3, 2010, pp. 1-14. [6] R. Mohanty, V. Ravi and M. R. Patra, “The Application of Intelligent and Soft-Computing Techniques to Software Engineering Problems: A Review,” International Journal of Information and Decision Sciences, Vol. 2, No. 3, 2010, pp. 233-272. [7] S. Elbaum, A. G. Malishevsky and G. Rothermel, “Incorporating Varying Test Costs and Fault Severities into Test Case Prioritization,” Proceedings of the 23rd International Conference on Software Engineering (ICSE-01), IEEE Computer Society, May 2001, pp. 329-338. [8] W. E. Wong, J. R. Horgan, A. P. Mathur and A. Pasquini, “A Test Set Size Minimization and Fault Detection Effectiveness: A Case Study in Space Application,” Proceedings of IEEE 21st Annual International Computer Software and Application Conference (COMPSAC-97), IEEE Press, Los Alamitos, 1997, pp. 552-528. [9] S. Tallam and N. Gupta, “A Concept Analysis Inspired Greedy Algorithm for Test Suite Minimization,” The Sixth ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, New York, ACM Press, 2005, pp. 35-42. [10] P. McMin, “Search-Based Software Test Data Generation: A Survey,” Proceedings in Software Testing, Verification and Reliability, Vol. 14, 2004, pp. 105-156. doi:10.1002/stvr.294 [11] J. M. Kim and A. Porter, “A History-Based Test Prioritization Technique for Regression Testing in Resource Constrained Environments,” Proceedings of the 24th International Conference on Software Engineering, Orlando, 2002, pp. 119-129. [12] K.Varun, Sujata and M. Kumar, “Test Case Prioritization Using Fault Severity,” International Journal of Computer Science and Technology (IJCST), Vol. 1, No. 1, September 2010, pp. 67-71. [13] J. A. Jones and M. J. Harrold, “Test-Suite Reduction and Prioritization for Modified Condition/Decision Coverage,” IEEE Transactions on Software Engineering, Vol. 29, No. 3, March 2003, pp. 195-209. doi:10.1109/TSE.2003.1183927 [14] W. E. Wong, J. R. Horgan, S. London and A. P. Mathur, “Effect of Test Set Size Minimization and Fault Detection Effectiveness,” 17th International Conference on Software Engineering (ICSE’95), Seattle, 23-30 April 1995. [15] H. Mark and J. Wegener, “Getting Results with Search- Based Software Engineering: Tutorial,” 26th IEEE International Conference and Software Engineering (ICSE 2004), Los Alamitos, 2004, IEEE Computer Society Press, pp. 728-729. [16] K. R. Walcott, M. L. Soffa, G. M. Kapfhammer and R. S. Roos, “Time Aware Test Suite Prioritization,” International Symposium on Software Testing and Analysis (ISSTA 06), ACM Press, Portland, 2006, pp. 1-12. [17] M. Bozkurt, M. Harman and Y. Hassoun, “Testing Web Services: A Survey,” Technical Report TR-10-01, Department of Computer Science, King’s College London, April 2010. [18] Z. Hong, “Axiomatic Assessment of Control Flow-Based Software Test Adequacy Criteria,” Software Engineering Journal, Vol. 10, No. 5, September 1995, pp. 194-204. [19] D. Leon and A. Podgurski, “A Comparison of Coverage-Based and Distribution-Based Techniques for Filtering and Prioritizing Test Cases,” Proceedings of the 14th International IEEE Symposium on Software Reliability Engineering (ISSRE’03), Denver, 2003, pp. 442-453. [20] Y. Shin, M. Harman, P. Tonella and A. Susi, “Clustering Test Cases to Achieve Effective and Scalable Prioritisation Incorporating Expert Knowledge,” ACM International Conference on Software Testing and Analysis (ISSTA 09), Chicago, 19-23 July 2009, pp. 201-212. [21] E. Emelie, P. Runeson and M. Skoglund, “A Systematic Review on Regression Test Selection Techniques,” Information & Software Technology, Vol. 52, No. 1, 2010, pp. 14-30. doi:10.1016/j.infsof.2009.07.001 [22] L. Zheng, M. Harman and R. Hierons, “Meta-Heuristic Search Algorithms for Regression Test Case Prioritization,” IEEE Transactions on Software Engineering, Vol. 33, No. 4, 2007, pp. 225-237. doi:10.1109/TSE.2007.38
[1] D. S. Alberts, “The Economics of Software,” Proceedings of National Computer Conference on Quality Assurance, Montvale, Vol. 45, 1976, pp. 433-442. [2] S. Hema and L. Williams, “On the Economics of Requirements Based Test Case Prioritization,” ACM SIGSOFT Software Engineering Notes, Vol. 30, No. 4, 2005, pp. 1-3. [3] Y. Shin and M. Harman, “Pareto Efficient Multi-Objective Test Case Selection,” Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), London, ACM press, 2007, pp. 140-150. [4] D. J. Mala and V. Mohan, “ABC Tester—Artificial Bee Colony Based Software Test Suite Optimization Approach,” International Journal of Software Engineering, Vol. 2, No. 2, 2009, pp. 15-43. [5] D. J. Mala, V. Mohan, “Quality Improvement and Optimization of Test Cases—A Hybrid Genetic Algorithm Based Approach,” ACM SIGSOFT Software Engineering Notes, Vol. 35, No. 3, 2010, pp. 1-14. [6] R. Mohanty, V. Ravi and M. R. Patra, “The Application of Intelligent and Soft-Computing Techniques to Software Engineering Problems: A Review,” International Journal of Information and Decision Sciences, Vol. 2, No. 3, 2010, pp. 233-272. [7] S. Elbaum, A. G. Malishevsky and G. Rothermel, “Incorporating Varying Test Costs and Fault Severities into Test Case Prioritization,” Proceedings of the 23rd International Conference on Software Engineering (ICSE-01), IEEE Computer Society, May 2001, pp. 329-338. [8] W. E. Wong, J. R. Horgan, A. P. Mathur and A. Pasquini, “A Test Set Size Minimization and Fault Detection Effectiveness: A Case Study in Space Application,” Proceedings of IEEE 21st Annual International Computer Software and Application Conference (COMPSAC-97), IEEE Press, Los Alamitos, 1997, pp. 552-528. [9] S. Tallam and N. Gupta, “A Concept Analysis Inspired Greedy Algorithm for Test Suite Minimization,” The Sixth ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, New York, ACM Press, 2005, pp. 35-42. [10] P. McMin, “Search-Based Software Test Data Generation: A Survey,” Proceedings in Software Testing, Verification and Reliability, Vol. 14, 2004, pp. 105-156. doi:10.1002/stvr.294 [11] J. M. Kim and A. Porter, “A History-Based Test Prioritization Technique for Regression Testing in Resource Constrained Environments,” Proceedings of the 24th International Conference on Software Engineering, Orlando, 2002, pp. 119-129. [12] K.Varun, Sujata and M. Kumar, “Test Case Prioritization Using Fault Severity,” International Journal of Computer Science and Technology (IJCST), Vol. 1, No. 1, September 2010, pp. 67-71. [13] J. A. Jones and M. J. Harrold, “Test-Suite Reduction and Prioritization for Modified Condition/Decision Coverage,” IEEE Transactions on Software Engineering, Vol. 29, No. 3, March 2003, pp. 195-209. doi:10.1109/TSE.2003.1183927 [14] W. E. Wong, J. R. Horgan, S. London and A. P. Mathur, “Effect of Test Set Size Minimization and Fault Detection Effectiveness,” 17th International Conference on Software Engineering (ICSE’95), Seattle, 23-30 April 1995. [15] H. Mark and J. Wegener, “Getting Results with Search- Based Software Engineering: Tutorial,” 26th IEEE International Conference and Software Engineering (ICSE 2004), Los Alamitos, 2004, IEEE Computer Society Press, pp. 728-729. [16] K. R. Walcott, M. L. Soffa, G. M. Kapfhammer and R. S. Roos, “Time Aware Test Suite Prioritization,” International Symposium on Software Testing and Analysis (ISSTA 06), ACM Press, Portland, 2006, pp. 1-12. [17] M. Bozkurt, M. Harman and Y. Hassoun, “Testing Web Services: A Survey,” Technical Report TR-10-01, Department of Computer Science, King’s College London, April 2010. [18] Z. Hong, “Axiomatic Assessment of Control Flow-Based Software Test Adequacy Criteria,” Software Engineering Journal, Vol. 10, No. 5, September 1995, pp. 194-204. [19] D. Leon and A. Podgurski, “A Comparison of Coverage-Based and Distribution-Based Techniques for Filtering and Prioritizing Test Cases,” Proceedings of the 14th International IEEE Symposium on Software Reliability Engineering (ISSRE’03), Denver, 2003, pp. 442-453. [20] Y. Shin, M. Harman, P. Tonella and A. Susi, “Clustering Test Cases to Achieve Effective and Scalable Prioritisation Incorporating Expert Knowledge,” ACM International Conference on Software Testing and Analysis (ISSTA 09), Chicago, 19-23 July 2009, pp. 201-212. [21] E. Emelie, P. Runeson and M. Skoglund, “A Systematic Review on Regression Test Selection Techniques,” Information & Software Technology, Vol. 52, No. 1, 2010, pp. 14-30. doi:10.1016/j.infsof.2009.07.001 [22] L. Zheng, M. Harman and R. Hierons, “Meta-Heuristic Search Algorithms for Regression Test Case Prioritization,” IEEE Transactions on Software Engineering, Vol. 33, No. 4, 2007, pp. 225-237. doi:10.1109/TSE.2007.38