Test coverage analysis is a structural testing technique, which helps to evaluate the sufficiency of software testing. This letter presents two test generation algorithms based on binary decision diagrams to produce t...Test coverage analysis is a structural testing technique, which helps to evaluate the sufficiency of software testing. This letter presents two test generation algorithms based on binary decision diagrams to produce tests for the Multiple-Condition Criterion(M-CC) and the Modified Condition/Decision Criterion(MC/DC), and describes the design of the C program Coverage Measurement Tool (CCMT), which can record dynamic behaviors of C programs and quantify test coverage.展开更多
Deep learning has been recently studied to generate high-quality prediction intervals(PIs)for uncertainty quantification in regression tasks,including recent applications in simulation metamodeling.The high-quality cr...Deep learning has been recently studied to generate high-quality prediction intervals(PIs)for uncertainty quantification in regression tasks,including recent applications in simulation metamodeling.The high-quality criterion requires PIs to be as narrow as possible,whilst maintaining a pre-specified level of data(marginal)coverage.However,most existing works for high-quality PIs lack accurate information on conditional coverage,which may cause unreliable predictions if it is significantly smaller than the marginal coverage.To address this problem,we propose an end-to-end framework which could output high-quality PIs and simultaneously provide their conditional coverage estimation.In doing so,we design a new loss function that is both easy-to-implement and theoretically justified via an exponential concentration bound.Our evaluation on real-world benchmark datasets and synthetic examples shows that our approach not only achieves competitive results on high-quality PIs in terms of average PI width,but also accurately estimates conditional coverage information that is useful in assessing model uncertainty.展开更多
Symbolic execution is an effective way of systematically exploring the search space of a program,and is often used for automatic software testing and bug finding.The program to be analyzed is usually compiled into a b...Symbolic execution is an effective way of systematically exploring the search space of a program,and is often used for automatic software testing and bug finding.The program to be analyzed is usually compiled into a binary or an intermediate representation,on which symbolic execution is carried out.During this process,compiler optimizations influence the effectiveness and efficiency of symbolic execution.However,to the best of our knowledge,there exists no work on compiler optimization recommendation for symbolic execution with respect to(w.r.t.)modified condition/decision coverage(MC/DC),which is an important testing coverage criterion widely used for mission-critical software.This study describes our use of a state-of-the-art symbolic execution tool to carry out extensive experiments to study the impact of compiler optimizations on symbolic execution w.r.t.MC/DC.The results indicate that instruction combining(IC)optimization is the important and dominant optimization for symbolic execution w.r.t.MC/DC.We designed and implemented a support vector machine based optimization recommendation method w.r.t.IC(denoted as auto).The experiments on two standard benchmarks(Coreutils and NECLA)showed that auto achieves the best MC/DC on 67.47%of Coreutils programs and 78.26%of NECLA programs.展开更多
文摘Test coverage analysis is a structural testing technique, which helps to evaluate the sufficiency of software testing. This letter presents two test generation algorithms based on binary decision diagrams to produce tests for the Multiple-Condition Criterion(M-CC) and the Modified Condition/Decision Criterion(MC/DC), and describes the design of the C program Coverage Measurement Tool (CCMT), which can record dynamic behaviors of C programs and quantify test coverage.
基金the National Science Foundation under grants CAREER CMMI-1834710 and IS-1849280The research of Ziyi Huang and Haofeng Zhang is supported in part by the Cheung-Kong Innovation Doctoral Fellowship.
文摘Deep learning has been recently studied to generate high-quality prediction intervals(PIs)for uncertainty quantification in regression tasks,including recent applications in simulation metamodeling.The high-quality criterion requires PIs to be as narrow as possible,whilst maintaining a pre-specified level of data(marginal)coverage.However,most existing works for high-quality PIs lack accurate information on conditional coverage,which may cause unreliable predictions if it is significantly smaller than the marginal coverage.To address this problem,we propose an end-to-end framework which could output high-quality PIs and simultaneously provide their conditional coverage estimation.In doing so,we design a new loss function that is both easy-to-implement and theoretically justified via an exponential concentration bound.Our evaluation on real-world benchmark datasets and synthetic examples shows that our approach not only achieves competitive results on high-quality PIs in terms of average PI width,but also accurately estimates conditional coverage information that is useful in assessing model uncertainty.
基金Project supported by the National Key R&D Program of China(No.2017YFB1001802)the National Natural Science Foundation of China(Nos.61472440,61632015,61690203,and 61532007)。
文摘Symbolic execution is an effective way of systematically exploring the search space of a program,and is often used for automatic software testing and bug finding.The program to be analyzed is usually compiled into a binary or an intermediate representation,on which symbolic execution is carried out.During this process,compiler optimizations influence the effectiveness and efficiency of symbolic execution.However,to the best of our knowledge,there exists no work on compiler optimization recommendation for symbolic execution with respect to(w.r.t.)modified condition/decision coverage(MC/DC),which is an important testing coverage criterion widely used for mission-critical software.This study describes our use of a state-of-the-art symbolic execution tool to carry out extensive experiments to study the impact of compiler optimizations on symbolic execution w.r.t.MC/DC.The results indicate that instruction combining(IC)optimization is the important and dominant optimization for symbolic execution w.r.t.MC/DC.We designed and implemented a support vector machine based optimization recommendation method w.r.t.IC(denoted as auto).The experiments on two standard benchmarks(Coreutils and NECLA)showed that auto achieves the best MC/DC on 67.47%of Coreutils programs and 78.26%of NECLA programs.
