Starting with the goal and significance of software security testing,this paper introduces the main methods of software security testing in the open network environment,including formal security testing,white box test...Starting with the goal and significance of software security testing,this paper introduces the main methods of software security testing in the open network environment,including formal security testing,white box testing,fuzzy testing,model testing,and fault injection testing.A software security testing method based on a security target model is proposed.This paper provides new ideas for software security testing,better adapts to the open network environment,improves the efficiency and quality of testing,and builds a good software application environment.展开更多
作者以他个人的观点,对该文献中重要部分进行了点评,指出:空间软件地面可靠性测试尽管困难重重且有诸多限制,但不能放松甚至放弃执行现有的软件地面测试标准;应从"精神号"火星探测器飞行失败中吸取教训,加强软件地面测试,包...作者以他个人的观点,对该文献中重要部分进行了点评,指出:空间软件地面可靠性测试尽管困难重重且有诸多限制,但不能放松甚至放弃执行现有的软件地面测试标准;应从"精神号"火星探测器飞行失败中吸取教训,加强软件地面测试,包括传统软件黑盒和白盒测试方法中的典型测试项目(如:健壮测试、应力测试、极值测试、随机测试等);已经公布的新软件研发标准——Aerospace Report No.TOR-2004(3909)-3537,Software Development Standard for Space Systems值得我们参考。展开更多
Modeling technology has been introduced into software testing field. However, how to carry through the testing modeling effectively is still a difficulty. Based on combination of simulation modeling technology and emb...Modeling technology has been introduced into software testing field. However, how to carry through the testing modeling effectively is still a difficulty. Based on combination of simulation modeling technology and embedded real-time software testing method, the process of simulation testing modeling is studied first. And then, the supporting environment of simulation testing modeling is put forward. Furthermore, an approach of embedded real-time software simulation testing modeling including modeling of cross-linked equipments of system under testing （SUT）, test case, testing scheduling, and testing system service is brought forward. Finally, the formalized description and execution system of testing models are given, with which we can realize real-time, closed loop, mad automated system testing for embedded real-time software.展开更多
The reliability of real-time embedded software directly determines the reliability of the whole real-time embedded sys- tem, and the effective software testing is an important way to ensure software quality and reliab...The reliability of real-time embedded software directly determines the reliability of the whole real-time embedded sys- tem, and the effective software testing is an important way to ensure software quality and reliability. Based on the analysis of the characteristics of real-time embedded software, the formal method is introduced into the real-time embedded software testing field and the real-time extended finite state machine （RT-EFSM） model is studied firstly. Then, the time zone division method of real-time embedded system is presented and the definition and description methods of time-constrained transition equivalence class （timeCTEC） are presented. Furthermore, the approaches of the testing sequence and test case generation are put forward. Finally, the proposed method is applied to a typical avionics real- time embedded software testing practice and the examples of the timeCTEC, testing sequences and test cases are given. With the analysis of the testing result, the application verification shows that the proposed method can effectively describe the real-time embedded software state transition characteristics and real-time requirements and play the advantages of the formal methods in accuracy, effectiveness and the automation supporting. Combined with the testing platform, the real-time, closed loop and automated simulation testing for real-time embedded software can be realized effectively.展开更多
Robustness testing for safety-critical embedded software is still a challenge in its nascent stages. In this paper, we propose a practical methodology and implement an environment by employing model-based robustness t...Robustness testing for safety-critical embedded software is still a challenge in its nascent stages. In this paper, we propose a practical methodology and implement an environment by employing model-based robustness testing for embedded software systems. It is a system-level black-box testing approach in which the fault behaviors of embedded software is triggered with the aid of modelbased fault injection by the support of an executable model-driven hardware-in-loop （HIL） testing environment. The prototype implementation of the robustness testing environment based on the proposed approach is experimentally discussed and illustrated by industrial case studies based on several avionics-embedded software systems. The results show that our proposed and implemented robustness testing method and environment are effective to find more bugs, and reduce burdens of testing engineers to enhance efficiency of testing tasks, especially for testing complex embedded systems.展开更多
Software security poses substantial risks to our society because software has become part of our life. Numerous techniques have been proposed to resolve or mitigate the impact of software security issues. Among them, ...Software security poses substantial risks to our society because software has become part of our life. Numerous techniques have been proposed to resolve or mitigate the impact of software security issues. Among them, software testing and analysis are two of the critical methods, which significantly benefit from the advancements in deep learning technologies. Due to the successful use of deep learning in software security, recently,researchers have explored the potential of using large language models(LLMs) in this area. In this paper, we systematically review the results focusing on LLMs in software security. We analyze the topics of fuzzing, unit test, program repair, bug reproduction, data-driven bug detection, and bug triage. We deconstruct these techniques into several stages and analyze how LLMs can be used in the stages. We also discuss the future directions of using LLMs in software security, including the future directions for the existing use of LLMs and extensions from conventional deep learning research.展开更多
Through reusing software test components, automated software testing generally costs less than manual software testing. There has been much research on how to develop the reusable test components, but few fall on how ...Through reusing software test components, automated software testing generally costs less than manual software testing. There has been much research on how to develop the reusable test components, but few fall on how to estimate the reusability of test conlponents for automated testing. The purpose of this paper is to present a method of minimum reusability estimation for automated testing based on the return on investment （ROI） model. Minimum reusability is a benchmark for the whole automated testing process. If the reusability in one test execution is less than the minimum reusability, some new strategies must be adopted ill the next test execution to increase the reusability. Only by this way, we can reduce unnecessary costs and finally get a return on the investment of automated testing.展开更多
This paper studies the software scenario testing, which is commonly used in black-box testing at present. In the paper, the workflow model based on task-driven, which is very common in scenario testing, is analyzed. A...This paper studies the software scenario testing, which is commonly used in black-box testing at present. In the paper, the workflow model based on task-driven, which is very common in scenario testing, is analyzed. According to test adequacy criteria in scenario testing, the model is designed to correspond test cases in the light of logic block(LB). The final test cases that conform to the test adequacy criteria can be obtained through test case combination and test case reduction. In the last part of the paper, example of actual workflow is to design the efficient test case. Therefore the method is proved to be effective.展开更多
Software configuration testing is used to test a piece of software with all kinds of hardware to ensure that it can run properly on them.This paper generates test cases for configuration testing with several common me...Software configuration testing is used to test a piece of software with all kinds of hardware to ensure that it can run properly on them.This paper generates test cases for configuration testing with several common methods,such as multiple single-factor experiments,uniform design,and orthogonal experiment design used in other fields.This paper analyzes their merits and improves the orthogonal experiment design method with pairwise testing,and decreases the testing risk caused by incomplete testing with a method of multiple-factors-covering.It presents a simple factor cover method which can cover all the factors and pairwise combinations to the greatest degree.Some comparisons of these methods are made on the aspects of test suite scale,coverage,and usability,etc..展开更多
In recent decades,many software reliability growth models（SRGMs） have been proposed for the engineers and testers in measuring the software reliability precisely.Most of them is established based on the non-homogene...In recent decades,many software reliability growth models（SRGMs） have been proposed for the engineers and testers in measuring the software reliability precisely.Most of them is established based on the non-homogeneous Poisson process（NHPP）,and it is proved that the prediction accuracy of such models could be improved by adding the describing of characterization of testing effort.However,some research work indicates that the fault detection rate（FDR） is another key factor affects final software quality.Most early NHPPbased models deal with the FDR as constant or piecewise function,which does not fit the different testing stages well.Thus,this paper first incorporates a multivariate function of FDR,which is bathtub-shaped,into the NHPP-based SRGMs considering testing effort in order to further improve performance.A new model framework is proposed,and a stepwise method is used to apply the framework with real data sets to find the optimal model.Experimental studies show that the obtained new model can provide better performance of fitting and prediction compared with other traditional SRGMs.展开更多
The meanings of parameters of software reliabi-lity models are investigated in terms of the process of the software testing and in terms of other measurements of software.Based on the investigation,the empirical estim...The meanings of parameters of software reliabi-lity models are investigated in terms of the process of the software testing and in terms of other measurements of software.Based on the investigation,the empirical estimation of the parameters is addressed.On one hand,these empirical estimates are also measurements of the software,which can be used to control and to optimize the process of the software development.On the other hand,by treating these empirical estimates as Bayes priors,software reliability models are extended such that the engineers’experience can be integrated into and hence to improve the models.展开更多
Software-related security aspects are a growing and legitimate concern,especially with 5G data available just at our palms.To conduct research in this field,periodic comparative analysis is needed with the new techniq...Software-related security aspects are a growing and legitimate concern,especially with 5G data available just at our palms.To conduct research in this field,periodic comparative analysis is needed with the new techniques coming up rapidly.The purpose of this study is to review the recent developments in the field of security integration in the software development lifecycle(SDLC)by analyzing the articles published in the last two decades and to propose a way forward.This review follows Kitchenham’s review protocol.The review has been divided into three main stages including planning,execution,and analysis.From the selected 100 articles,it becomes evident that need of a collaborative approach is necessary for addressing critical software security risks(CSSRs)through effective risk management/estimation techniques.Quantifying risks using a numeric scale enables a comprehensive understanding of their severity,facilitating focused resource allocation and mitigation efforts.Through a comprehensive understanding of potential vulnerabilities and proactive mitigation efforts facilitated by protection poker,organizations can prioritize resources effectively to ensure the successful outcome of projects and initiatives in today’s dynamic threat landscape.The review reveals that threat analysis and security testing are needed to develop automated tools for the future.Accurate estimation of effort required to prioritize potential security risks is a big challenge in software security.The accuracy of effort estimation can be further improved by exploring new techniques,particularly those involving deep learning.It is also imperative to validate these effort estimation methods to ensure all potential security threats are addressed.Another challenge is selecting the right model for each specific security threat.To achieve a comprehensive evaluation,researchers should use well-known benchmark checklists.展开更多
Generally,software testing is considered as a proficient technique to achieve improvement in quality and reliability of the software.But,the quality of test cases has a considerable influence on fault revealing capabi...Generally,software testing is considered as a proficient technique to achieve improvement in quality and reliability of the software.But,the quality of test cases has a considerable influence on fault revealing capability of software testing activity.Test Case Prioritization(TCP)remains a challenging issue since prioritizing test cases is unsatisfactory in terms of Average Percentage of Faults Detected(APFD)and time spent upon execution results.TCP ismainly intended to design a collection of test cases that can accomplish early optimization using preferred characteristics.The studies conducted earlier focused on prioritizing the available test cases in accelerating fault detection rate during software testing.In this aspect,the current study designs aModified Harris Hawks Optimization based TCP(MHHO-TCP)technique for software testing.The aim of the proposed MHHO-TCP technique is to maximize APFD and minimize the overall execution time.In addition,MHHO algorithm is designed to boost the exploration and exploitation abilities of conventional HHO algorithm.In order to validate the enhanced efficiency of MHHO-TCP technique,a wide range of simulations was conducted on different benchmark programs and the results were examined under several aspects.The experimental outcomes highlight the improved efficiency of MHHO-TCP technique over recent approaches under different measures.展开更多
Software reliability was estimated based on NHPP software reliability growth models. Testing reliability and operational reliability may be essentially different. On the basis of analyzing similarities and differences...Software reliability was estimated based on NHPP software reliability growth models. Testing reliability and operational reliability may be essentially different. On the basis of analyzing similarities and differences of the testing phase and the operational phase, using the concept of operational reliability and the testing reliability, different forms of the comparison between the operational failure ratio and the predicted testing failure ratio were conducted, and the mathematical discussion and analysis were performed in detail. Finally, software optimal release was studied using software failure data. The results show that two kinds of conclusions can be derived by applying this method, one conclusion is to continue testing to meet the required reliability level of users, and the other is that testing stops when the required operational reliability is met, thus the testing cost can be reduced.展开更多
In this paper, the testing technology of event-driven software is focused. It is first analyzed the difference between event-driven software and the traditional procedure-oriented software, and based on the above anal...In this paper, the testing technology of event-driven software is focused. It is first analyzed the difference between event-driven software and the traditional procedure-oriented software, and based on the above analysis, the mechanism of event-driven and the effect of introduction of event-driven mechanism on software testing are unveiled. Then based on the characteristic of the event-driven software, the traditional software testing method is improved, and testing policy of event based test is presented in this paper.Moreover the event coverage criteria are defined and given here. At the same time the event executing rule are further uncovered, such as ordinal event, non-ordinal event, predecessor event and concurrent event etc., and also the methods of testing according to event executing rule are studied.展开更多
This study addresses the challenge of assessing gaps among the differences of test people in eight groups by matching them based on four scenarios. The proposed model called Test Employee Capability Maturity Model (TE...This study addresses the challenge of assessing gaps among the differences of test people in eight groups by matching them based on four scenarios. The proposed model called Test Employee Capability Maturity Model (TEC-MM) helps find the gaps and measure the capability and maturity levels of each sub-maturity models in four dimensions as technicality, management, business, and personality. We applied TEC-MM to a software company which has instituted test people improvement strategy and plans to be a front runner in software testing industry. The findings reveal that gaps existed and consequently affect software testing process quality.展开更多
Testing-effort（TE） and imperfect debugging（ID） in the reliability modeling process may further improve the fitting and prediction results of software reliability growth models（SRGMs）. For describing the S-shaped...Testing-effort（TE） and imperfect debugging（ID） in the reliability modeling process may further improve the fitting and prediction results of software reliability growth models（SRGMs）. For describing the S-shaped varying trend of TE increasing rate more accurately, first, two S-shaped testing-effort functions（TEFs）, i.e.,delayed S-shaped TEF（DS-TEF） and inflected S-shaped TEF（IS-TEF）, are proposed. Then these two TEFs are incorporated into various types（exponential-type, delayed S-shaped and inflected S-shaped） of non-homogeneous Poisson process（NHPP）SRGMs with two forms of ID respectively for obtaining a series of new NHPP SRGMs which consider S-shaped TEFs as well as ID. Finally these new SRGMs and several comparison NHPP SRGMs are applied into four real failure data-sets respectively for investigating the fitting and prediction power of these new SRGMs.The experimental results show that：（i） the proposed IS-TEF is more suitable and flexible for describing the consumption of TE than the previous TEFs;（ii） incorporating TEFs into the inflected S-shaped NHPP SRGM may be more effective and appropriate compared with the exponential-type and the delayed S-shaped NHPP SRGMs;（iii） the inflected S-shaped NHPP SRGM considering both IS-TEF and ID yields the most accurate fitting and prediction results than the other comparison NHPP SRGMs.展开更多
Technical debt(TD)happens when project teams carry out technical decisions in favor of a short-term goal(s)in their projects,whether deliberately or unknowingly.TD must be properly managed to guarantee that its negati...Technical debt(TD)happens when project teams carry out technical decisions in favor of a short-term goal(s)in their projects,whether deliberately or unknowingly.TD must be properly managed to guarantee that its negative implications do not outweigh its advantages.A lot of research has been conducted to show that TD has evolved into a common problem with considerable financial burden.Test technical debt is the technical debt aspect of testing(or test debt).Test debt is a relatively new concept that has piqued the curiosity of the software industry in recent years.In this article,we assume that the organization selects the testing artifacts at the start of every sprint.Implementing the latest features in consideration of expected business value and repaying technical debt are among candidate tasks in terms of the testing process(test cases increments).To gain the maximum benefit for the organization in terms of software testing optimization,there is a need to select the artifacts(i.e.,test cases)with maximum feature coverage within the available resources.The management of testing optimization for large projects is complicated and can also be treated as a multi-objective problem that entails a trade-off between the agile software’s short-term and long-term value.In this article,we implement a multi-objective indicatorbased evolutionary algorithm(IBEA)for fixing such optimization issues.The capability of the algorithm is evidenced by adding it to a real case study of a university registration process.展开更多
To address the problems of insufficient number of personalized exercises and cases and teachers’lack of grasp of students’weak knowledge points in the current software testing online courses,we study the strategy of...To address the problems of insufficient number of personalized exercises and cases and teachers’lack of grasp of students’weak knowledge points in the current software testing online courses,we study the strategy of establishing and updating intelligent exercise sets and case libraries and analyze the answers and dig out the weak points of knowledge through group intelligence reasoning and interactive machine learning methods.This will help teachers to make uniform and targeted explanations,reduce manual judgment,and achieve intelligent teaching quality reform,and implement the educational concepts of“keeping up with the times”and“teaching according to students’abilities”.展开更多
Customizing applications through program configuration options has been proved by many open-source and commercial projects as one of the best practices in software engineering. However, traditional performance testing...Customizing applications through program configuration options has been proved by many open-source and commercial projects as one of the best practices in software engineering. However, traditional performance testing is not in synch with this industrial practice. Traditional performance testing techniques consider program inputs as the only external factor. It ignores the performance influence of configuration options. This study aims to stimulate research interest in performance testing in the context of configurable software systems by answering three research questions. That is, why it is necessary to conduct research in performance testing, what are the state-of-the-art techniques, and how do we conduct performance testing research in configurable software systems. In this study, we examine the unique characteristics and challenges of performance testing research in configurable software systems. We review and discuss research topics on the performance bug study, performance anti-patterns, program analysis, and performance testing. We share the research findings from the empirical study and outline the opening opportunities for new and advanced researchers to contribute to the research community.展开更多
文摘Starting with the goal and significance of software security testing,this paper introduces the main methods of software security testing in the open network environment,including formal security testing,white box testing,fuzzy testing,model testing,and fault injection testing.A software security testing method based on a security target model is proposed.This paper provides new ideas for software security testing,better adapts to the open network environment,improves the efficiency and quality of testing,and builds a good software application environment.
文摘作者以他个人的观点,对该文献中重要部分进行了点评,指出:空间软件地面可靠性测试尽管困难重重且有诸多限制,但不能放松甚至放弃执行现有的软件地面测试标准;应从"精神号"火星探测器飞行失败中吸取教训,加强软件地面测试,包括传统软件黑盒和白盒测试方法中的典型测试项目(如:健壮测试、应力测试、极值测试、随机测试等);已经公布的新软件研发标准——Aerospace Report No.TOR-2004(3909)-3537,Software Development Standard for Space Systems值得我们参考。
文摘Modeling technology has been introduced into software testing field. However, how to carry through the testing modeling effectively is still a difficulty. Based on combination of simulation modeling technology and embedded real-time software testing method, the process of simulation testing modeling is studied first. And then, the supporting environment of simulation testing modeling is put forward. Furthermore, an approach of embedded real-time software simulation testing modeling including modeling of cross-linked equipments of system under testing （SUT）, test case, testing scheduling, and testing system service is brought forward. Finally, the formalized description and execution system of testing models are given, with which we can realize real-time, closed loop, mad automated system testing for embedded real-time software.
基金supported by the Aviation Science Foundation of China
文摘The reliability of real-time embedded software directly determines the reliability of the whole real-time embedded sys- tem, and the effective software testing is an important way to ensure software quality and reliability. Based on the analysis of the characteristics of real-time embedded software, the formal method is introduced into the real-time embedded software testing field and the real-time extended finite state machine （RT-EFSM） model is studied firstly. Then, the time zone division method of real-time embedded system is presented and the definition and description methods of time-constrained transition equivalence class （timeCTEC） are presented. Furthermore, the approaches of the testing sequence and test case generation are put forward. Finally, the proposed method is applied to a typical avionics real- time embedded software testing practice and the examples of the timeCTEC, testing sequences and test cases are given. With the analysis of the testing result, the application verification shows that the proposed method can effectively describe the real-time embedded software state transition characteristics and real-time requirements and play the advantages of the formal methods in accuracy, effectiveness and the automation supporting. Combined with the testing platform, the real-time, closed loop and automated simulation testing for real-time embedded software can be realized effectively.
基金the Aeronautics Science Foundation of China(No.2011ZD51055)Science and Technology on Reliability&Environmental Engineering Laboratory(No.302367)the National Pre-Research Foundation of China(No.51319080201)
文摘Robustness testing for safety-critical embedded software is still a challenge in its nascent stages. In this paper, we propose a practical methodology and implement an environment by employing model-based robustness testing for embedded software systems. It is a system-level black-box testing approach in which the fault behaviors of embedded software is triggered with the aid of modelbased fault injection by the support of an executable model-driven hardware-in-loop （HIL） testing environment. The prototype implementation of the robustness testing environment based on the proposed approach is experimentally discussed and illustrated by industrial case studies based on several avionics-embedded software systems. The results show that our proposed and implemented robustness testing method and environment are effective to find more bugs, and reduce burdens of testing engineers to enhance efficiency of testing tasks, especially for testing complex embedded systems.
文摘Software security poses substantial risks to our society because software has become part of our life. Numerous techniques have been proposed to resolve or mitigate the impact of software security issues. Among them, software testing and analysis are two of the critical methods, which significantly benefit from the advancements in deep learning technologies. Due to the successful use of deep learning in software security, recently,researchers have explored the potential of using large language models(LLMs) in this area. In this paper, we systematically review the results focusing on LLMs in software security. We analyze the topics of fuzzing, unit test, program repair, bug reproduction, data-driven bug detection, and bug triage. We deconstruct these techniques into several stages and analyze how LLMs can be used in the stages. We also discuss the future directions of using LLMs in software security, including the future directions for the existing use of LLMs and extensions from conventional deep learning research.
基金Foundation item： the National Natural Science Foundation of China （No. 90718037）
文摘Through reusing software test components, automated software testing generally costs less than manual software testing. There has been much research on how to develop the reusable test components, but few fall on how to estimate the reusability of test conlponents for automated testing. The purpose of this paper is to present a method of minimum reusability estimation for automated testing based on the return on investment （ROI） model. Minimum reusability is a benchmark for the whole automated testing process. If the reusability in one test execution is less than the minimum reusability, some new strategies must be adopted ill the next test execution to increase the reusability. Only by this way, we can reduce unnecessary costs and finally get a return on the investment of automated testing.
基金National Torch Project, China ( No. 2009GH510068 )National High-Tech R & D Program of China ( 863 ) ( No.2007AA010401)
文摘This paper studies the software scenario testing, which is commonly used in black-box testing at present. In the paper, the workflow model based on task-driven, which is very common in scenario testing, is analyzed. According to test adequacy criteria in scenario testing, the model is designed to correspond test cases in the light of logic block(LB). The final test cases that conform to the test adequacy criteria can be obtained through test case combination and test case reduction. In the last part of the paper, example of actual workflow is to design the efficient test case. Therefore the method is proved to be effective.
基金The Natronal Natural Science Foundation of China(No.60373066)Opening Foundation of State Key Laboratory of Software Engineering in Wuhan UniversityOpening Foundation ofJiangsu Key Laboratory of Computer Information ProcessingTechnology in Soochow University.
文摘Software configuration testing is used to test a piece of software with all kinds of hardware to ensure that it can run properly on them.This paper generates test cases for configuration testing with several common methods,such as multiple single-factor experiments,uniform design,and orthogonal experiment design used in other fields.This paper analyzes their merits and improves the orthogonal experiment design method with pairwise testing,and decreases the testing risk caused by incomplete testing with a method of multiple-factors-covering.It presents a simple factor cover method which can cover all the factors and pairwise combinations to the greatest degree.Some comparisons of these methods are made on the aspects of test suite scale,coverage,and usability,etc..
基金supported by the National Natural Science Foundation of China(61070220)the Anhui Provincial Natural Science Foundation(1408085MKL79)
文摘In recent decades,many software reliability growth models（SRGMs） have been proposed for the engineers and testers in measuring the software reliability precisely.Most of them is established based on the non-homogeneous Poisson process（NHPP）,and it is proved that the prediction accuracy of such models could be improved by adding the describing of characterization of testing effort.However,some research work indicates that the fault detection rate（FDR） is another key factor affects final software quality.Most early NHPPbased models deal with the FDR as constant or piecewise function,which does not fit the different testing stages well.Thus,this paper first incorporates a multivariate function of FDR,which is bathtub-shaped,into the NHPP-based SRGMs considering testing effort in order to further improve performance.A new model framework is proposed,and a stepwise method is used to apply the framework with real data sets to find the optimal model.Experimental studies show that the obtained new model can provide better performance of fitting and prediction compared with other traditional SRGMs.
基金Supported by the National Natural Science Foun-dation of China(60173063)
文摘The meanings of parameters of software reliabi-lity models are investigated in terms of the process of the software testing and in terms of other measurements of software.Based on the investigation,the empirical estimation of the parameters is addressed.On one hand,these empirical estimates are also measurements of the software,which can be used to control and to optimize the process of the software development.On the other hand,by treating these empirical estimates as Bayes priors,software reliability models are extended such that the engineers’experience can be integrated into and hence to improve the models.
文摘Software-related security aspects are a growing and legitimate concern,especially with 5G data available just at our palms.To conduct research in this field,periodic comparative analysis is needed with the new techniques coming up rapidly.The purpose of this study is to review the recent developments in the field of security integration in the software development lifecycle(SDLC)by analyzing the articles published in the last two decades and to propose a way forward.This review follows Kitchenham’s review protocol.The review has been divided into three main stages including planning,execution,and analysis.From the selected 100 articles,it becomes evident that need of a collaborative approach is necessary for addressing critical software security risks(CSSRs)through effective risk management/estimation techniques.Quantifying risks using a numeric scale enables a comprehensive understanding of their severity,facilitating focused resource allocation and mitigation efforts.Through a comprehensive understanding of potential vulnerabilities and proactive mitigation efforts facilitated by protection poker,organizations can prioritize resources effectively to ensure the successful outcome of projects and initiatives in today’s dynamic threat landscape.The review reveals that threat analysis and security testing are needed to develop automated tools for the future.Accurate estimation of effort required to prioritize potential security risks is a big challenge in software security.The accuracy of effort estimation can be further improved by exploring new techniques,particularly those involving deep learning.It is also imperative to validate these effort estimation methods to ensure all potential security threats are addressed.Another challenge is selecting the right model for each specific security threat.To achieve a comprehensive evaluation,researchers should use well-known benchmark checklists.
基金The authors extend their appreciation to the Deanship of Scientific Research at King Khalid University for funding this work under Grant Number(RGP.1/127/42)Princess Nourah bint Abdulrahman University Researchers Supporting Project Number(PNURSP2022R237),Princess Nourah bint Abdulrahman University,Riyadh,Saudi Arabia.
文摘Generally,software testing is considered as a proficient technique to achieve improvement in quality and reliability of the software.But,the quality of test cases has a considerable influence on fault revealing capability of software testing activity.Test Case Prioritization(TCP)remains a challenging issue since prioritizing test cases is unsatisfactory in terms of Average Percentage of Faults Detected(APFD)and time spent upon execution results.TCP ismainly intended to design a collection of test cases that can accomplish early optimization using preferred characteristics.The studies conducted earlier focused on prioritizing the available test cases in accelerating fault detection rate during software testing.In this aspect,the current study designs aModified Harris Hawks Optimization based TCP(MHHO-TCP)technique for software testing.The aim of the proposed MHHO-TCP technique is to maximize APFD and minimize the overall execution time.In addition,MHHO algorithm is designed to boost the exploration and exploitation abilities of conventional HHO algorithm.In order to validate the enhanced efficiency of MHHO-TCP technique,a wide range of simulations was conducted on different benchmark programs and the results were examined under several aspects.The experimental outcomes highlight the improved efficiency of MHHO-TCP technique over recent approaches under different measures.
基金the PhD Programs Foundation for Young Researchers of Ministry of Education of China (Grant No.20070217051)Major Program of National Natural Science Foundation of China (Grant No.90718003)
文摘Software reliability was estimated based on NHPP software reliability growth models. Testing reliability and operational reliability may be essentially different. On the basis of analyzing similarities and differences of the testing phase and the operational phase, using the concept of operational reliability and the testing reliability, different forms of the comparison between the operational failure ratio and the predicted testing failure ratio were conducted, and the mathematical discussion and analysis were performed in detail. Finally, software optimal release was studied using software failure data. The results show that two kinds of conclusions can be derived by applying this method, one conclusion is to continue testing to meet the required reliability level of users, and the other is that testing stops when the required operational reliability is met, thus the testing cost can be reduced.
文摘In this paper, the testing technology of event-driven software is focused. It is first analyzed the difference between event-driven software and the traditional procedure-oriented software, and based on the above analysis, the mechanism of event-driven and the effect of introduction of event-driven mechanism on software testing are unveiled. Then based on the characteristic of the event-driven software, the traditional software testing method is improved, and testing policy of event based test is presented in this paper.Moreover the event coverage criteria are defined and given here. At the same time the event executing rule are further uncovered, such as ordinal event, non-ordinal event, predecessor event and concurrent event etc., and also the methods of testing according to event executing rule are studied.
文摘This study addresses the challenge of assessing gaps among the differences of test people in eight groups by matching them based on four scenarios. The proposed model called Test Employee Capability Maturity Model (TEC-MM) helps find the gaps and measure the capability and maturity levels of each sub-maturity models in four dimensions as technicality, management, business, and personality. We applied TEC-MM to a software company which has instituted test people improvement strategy and plans to be a front runner in software testing industry. The findings reveal that gaps existed and consequently affect software testing process quality.
基金supported by the Pre-research Foundation of CPLA General Equipment Department
文摘Testing-effort（TE） and imperfect debugging（ID） in the reliability modeling process may further improve the fitting and prediction results of software reliability growth models（SRGMs）. For describing the S-shaped varying trend of TE increasing rate more accurately, first, two S-shaped testing-effort functions（TEFs）, i.e.,delayed S-shaped TEF（DS-TEF） and inflected S-shaped TEF（IS-TEF）, are proposed. Then these two TEFs are incorporated into various types（exponential-type, delayed S-shaped and inflected S-shaped） of non-homogeneous Poisson process（NHPP）SRGMs with two forms of ID respectively for obtaining a series of new NHPP SRGMs which consider S-shaped TEFs as well as ID. Finally these new SRGMs and several comparison NHPP SRGMs are applied into four real failure data-sets respectively for investigating the fitting and prediction power of these new SRGMs.The experimental results show that：（i） the proposed IS-TEF is more suitable and flexible for describing the consumption of TE than the previous TEFs;（ii） incorporating TEFs into the inflected S-shaped NHPP SRGM may be more effective and appropriate compared with the exponential-type and the delayed S-shaped NHPP SRGMs;（iii） the inflected S-shaped NHPP SRGM considering both IS-TEF and ID yields the most accurate fitting and prediction results than the other comparison NHPP SRGMs.
基金The authors would like to thank the Deanship of Scientific Research at Umm Al-Qura University for supporting this work by Grant Code:(22UQUyouracademicnumberDSRxx).
文摘Technical debt(TD)happens when project teams carry out technical decisions in favor of a short-term goal(s)in their projects,whether deliberately or unknowingly.TD must be properly managed to guarantee that its negative implications do not outweigh its advantages.A lot of research has been conducted to show that TD has evolved into a common problem with considerable financial burden.Test technical debt is the technical debt aspect of testing(or test debt).Test debt is a relatively new concept that has piqued the curiosity of the software industry in recent years.In this article,we assume that the organization selects the testing artifacts at the start of every sprint.Implementing the latest features in consideration of expected business value and repaying technical debt are among candidate tasks in terms of the testing process(test cases increments).To gain the maximum benefit for the organization in terms of software testing optimization,there is a need to select the artifacts(i.e.,test cases)with maximum feature coverage within the available resources.The management of testing optimization for large projects is complicated and can also be treated as a multi-objective problem that entails a trade-off between the agile software’s short-term and long-term value.In this article,we implement a multi-objective indicatorbased evolutionary algorithm(IBEA)for fixing such optimization issues.The capability of the algorithm is evidenced by adding it to a real case study of a university registration process.
文摘To address the problems of insufficient number of personalized exercises and cases and teachers’lack of grasp of students’weak knowledge points in the current software testing online courses,we study the strategy of establishing and updating intelligent exercise sets and case libraries and analyze the answers and dig out the weak points of knowledge through group intelligence reasoning and interactive machine learning methods.This will help teachers to make uniform and targeted explanations,reduce manual judgment,and achieve intelligent teaching quality reform,and implement the educational concepts of“keeping up with the times”and“teaching according to students’abilities”.
文摘Customizing applications through program configuration options has been proved by many open-source and commercial projects as one of the best practices in software engineering. However, traditional performance testing is not in synch with this industrial practice. Traditional performance testing techniques consider program inputs as the only external factor. It ignores the performance influence of configuration options. This study aims to stimulate research interest in performance testing in the context of configurable software systems by answering three research questions. That is, why it is necessary to conduct research in performance testing, what are the state-of-the-art techniques, and how do we conduct performance testing research in configurable software systems. In this study, we examine the unique characteristics and challenges of performance testing research in configurable software systems. We review and discuss research topics on the performance bug study, performance anti-patterns, program analysis, and performance testing. We share the research findings from the empirical study and outline the opening opportunities for new and advanced researchers to contribute to the research community.
