This paper proposes a novel reinforcement-learning-based intelligent fault-tolerant assistance control framework for Air-breathing Hypersonic Vehicles(AHVs).Considering that Reinforcement Learning(RL)has the advantage...This paper proposes a novel reinforcement-learning-based intelligent fault-tolerant assistance control framework for Air-breathing Hypersonic Vehicles(AHVs).Considering that Reinforcement Learning(RL)has the advantage of exploring approximate optimal strategies,an RL-based assistance controller parallel to the fundamental controller is introduced to generate the assistance control signal.Specifically,the Incremental model-based Dual Heuristic Programming(IDHP)method is adopted to design the RL-based assistance control law.In order to extend the IDHP method to the assistance control scenario,a novel linear time-varying incremental model of the closed-loop augmented system is constructed and identified in real time,which consists of the AHV plant,the fundamental controller,and the command generator.The RL agent continuously updates its neural-network weights according to the real-time identification information,and adjusts its control policy,i.e.,the assistance control signal,after detecting sudden model changes.Simulation results have validated the effectiveness of the proposed intelligent fault-tolerant control scheme under various types of elevator faults and aerodynamic/configuration parameter uncertainties.The fault-tolerant ability of the whole control system with the proposed RL-based assistance controller is validated in both inner-loop attitude and outer-loop altitude tracking tasks.展开更多
The operational demands of a wide range significantly exacerbate combustion instability issues within ramjet combustor.To suppress combustion oscillations,an open-loop control system utilizing Linear Genetic Programmi...The operational demands of a wide range significantly exacerbate combustion instability issues within ramjet combustor.To suppress combustion oscillations,an open-loop control system utilizing Linear Genetic Programming(LGP)has been developed for a full-scale annular ramjet combustor.The LGP is used to generate control laws that include multi-frequency forcing.These laws are then transformed into square waves to actuate the solenoid valve,which modulates the kerosene supply for open-loop control.The results show that the duty cycle has little effect on instability amplitude,whereas an increase in frequency leads to a remarked reduction in combustion amplitude.After five generations evolvements,the pressure amplitude is reduced by 40.6% under the optimal control law generated by LGP.Furthermore,the machine learning process is depicted using a proximity map of control law similarity,with the search pathway visualized by the steepest descent.All individuals go forward to the upper left corner of the map with the evolution process,terminating at the optimal individual of the fifth generation.展开更多
Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and ...Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.展开更多
This paper introduces a self-learning control approach based on approximate dynamic programming. Dynamic programming was introduced by Bellman in the 1950's for solving optimal control problems of nonlinear dynami...This paper introduces a self-learning control approach based on approximate dynamic programming. Dynamic programming was introduced by Bellman in the 1950's for solving optimal control problems of nonlinear dynamical systems. Due to its high computational complexity, the applications of dynamic programming have been limited to simple and small problems. The key step in finding approximate solutions to dynamic programming is to estimate the performance index in dynamic programming. The optimal control signal can then be determined by minimizing (or maximizing) the performance index. Artificial neural networks are very efficient tools in representing the performance index in dynamic programming. This paper assumes the use of neural networks for estimating the performance index in dynamic programming and for generating optimal control signals, thus to achieve optimal control through self-learning.展开更多
Heat integration is important for energy-saving in the process industry.It is linked to the persistently challenging task of optimal design of heat exchanger networks(HEN).Due to the inherent highly nonconvex nonlinea...Heat integration is important for energy-saving in the process industry.It is linked to the persistently challenging task of optimal design of heat exchanger networks(HEN).Due to the inherent highly nonconvex nonlinear and combinatorial nature of the HEN problem,it is not easy to find solutions of high quality for large-scale problems.The reinforcement learning(RL)method,which learns strategies through ongoing exploration and exploitation,reveals advantages in such area.However,due to the complexity of the HEN design problem,the RL method for HEN should be dedicated and designed.A hybrid strategy combining RL with mathematical programming is proposed to take better advantage of both methods.An insightful state representation of the HEN structure as well as a customized reward function is introduced.A Q-learning algorithm is applied to update the HEN structure using theε-greedy strategy.Better results are obtained from three literature cases of different scales.展开更多
Owing to extensive applications in many fields,the synchronization problem has been widely investigated in multi-agent systems.The synchronization for multi-agent systems is a pivotal issue,which means that under the ...Owing to extensive applications in many fields,the synchronization problem has been widely investigated in multi-agent systems.The synchronization for multi-agent systems is a pivotal issue,which means that under the designed control policy,the output of systems or the state of each agent can be consistent with the leader.The purpose of this paper is to investigate a heuristic dynamic programming(HDP)-based learning tracking control for discrete-time multi-agent systems to achieve synchronization while considering disturbances in systems.Besides,due to the difficulty of solving the coupled Hamilton–Jacobi–Bellman equation analytically,an improved HDP learning control algorithm is proposed to realize the synchronization between the leader and all following agents,which is executed by an action-critic neural network.The action and critic neural network are utilized to learn the optimal control policy and cost function,respectively,by means of introducing an auxiliary action network.Finally,two numerical examples and a practical application of mobile robots are presented to demonstrate the control performance of the HDP-based learning control algorithm.展开更多
Pair programming has been widely acclaimed the best way to go in computer programming. Recently, collaboration involving more subjects has been shown to produce better results in programming environments. However, the...Pair programming has been widely acclaimed the best way to go in computer programming. Recently, collaboration involving more subjects has been shown to produce better results in programming environments. However, the optimum group size needed for the collaboration has not been adequately addressed. This paper seeks to inculcate and acquaint the students involved in the study with the spirit of team work in software projects and to empirically determine the effective (optimum) team size that may be desirable in programming/learning real life environments. Two different experiments were organized and conducted. Parameters for determining the optimal team size were formulated. Volunteered participants of different genders were randomly grouped into five parallel teams of different sizes ranging from 1 to 5 in the first experiment. Each team size was replicated six times. The second experiment involved teams of same gender compositions (males or females) in different sizes. The times (efforts) for problem analysis and coding as well as compile-time errors (bugs) were recorded for each team size. The effectiveness was finally analyzed for the teams. The study shows that collaboration is highly beneficial to new learners of computer programming. They easily grasp the programming concepts when the learning is done in the company of others. The study also demonstrates that the optimum team size that may be adopted in a collaborative learning of computer programming is four.展开更多
Programming difficulties are one of the common problems faced by software engineering students,which can lead to a rapid decline in motivation and even drop out.Probing students’programming difficulties is a crucial ...Programming difficulties are one of the common problems faced by software engineering students,which can lead to a rapid decline in motivation and even drop out.Probing students’programming difficulties is a crucial step in understanding their current programming situation and implementing appropriate instructional interventions.However,how to detect students’programming difficulties accurately without students’awareness remains a big challenge.Address the issues above;this paper adopts a sensor-free difficulties detecting method based on a deep neural network which employs a recurrent neural network(RNN)model and uses the sequential timing data from programming behaviour.The method can detect students’programming difficulties in real-time with 93%accuracy without interference in the programming process.In the long term,this method is the first step for establishing an automated intelligent programming environment.At the same time,it can assist teachers in noticing the difficulties that students encounter.Then,teachers can adjust their teaching plans and provide manual tutoring intervention more quickly.展开更多
Students often face difficulties while taking basic programming courses due to several factors. In response, research has presented subjective assessments for diagnosing learning problems to improve the teaching of pr...Students often face difficulties while taking basic programming courses due to several factors. In response, research has presented subjective assessments for diagnosing learning problems to improve the teaching of programming in higher education. In this paper, the authors propose an Object Oriented conceptual map model and organize this approach into three levels： constructing a Concept Effect Propagation Table, constructing Test Item-Concept Relationships and diagnosing Student Learning Problems with Matrix Composition. The authors＇ work is a modification of the approaches of Chert and Bai as well as Chu et al., as the authors use statistical methods, rather than fuzzy sets, for the authors＇ analysis. This paper includes a statistical summary, which has been tested on a small sample of students in King Abdulaziz University, Jeddah, Saudi Arabia, illustrating the learning problems in an Object Oriented course. The experimental results have demonstrated that this approach might aid learning and teaching in an effective way.展开更多
Approximate dynamic programming (ADP) is a general and effective approach for solving optimal control and estimation problems by adapting to uncertain and nonconvex environments over time.
<span style="font-family:Verdana;">Students face difficulties in programming languages learning (PLL) which encourages many scholars to investigate the factors behind that. Although there a number of p...<span style="font-family:Verdana;">Students face difficulties in programming languages learning (PLL) which encourages many scholars to investigate the factors behind that. Although there a number of positive and negative factors found to be effective in PLL procedure, utilising online tools in PLL were recognized as a positive recommended means. This motivates many researchers to provide solutions and proposals which result in a number of choices and options. However, categorising those efforts and showing what has been done, would provide a better and clear image for future studies. Therefore, this paper aims to conduct a systematic literature review to show what studies have been done and then categorise them based on the type of online tools and the aims of the research. The study follows Kitchenham and Charters guidelines for writing SLR (Systematic Literature Review). The search result reached 1390 publications between 2013-09/2018. After the filtration which has been done through selected criteria, 160 publications were found to be adequate to answer the review questions. The main results of this systematic review are categorizing the aims of the studies in online PLL tools, classifying the tools and finding the current trends of the online PLL tools.</span>展开更多
In view of the current situation that offline teaching is the main mode of teaching Java Programming in higher vocational schools,this paper introduces the online and offline hybrid teaching method and expounds it fro...In view of the current situation that offline teaching is the main mode of teaching Java Programming in higher vocational schools,this paper introduces the online and offline hybrid teaching method and expounds it from the aspects of blended learning design,teaching organization,and implementation.At the same time,combined with the characteristics of blended learning,this paper proposes that under the new mode,teachers should actively change the form of teaching and research,the teaching mode,and the role of teachers,take students as the center,and build an independent and effective classroom.展开更多
The use of dynamic programming(DP)algorithms to learn Bayesian network structures is limited by their high space complexity and difficulty in learning the structure of large-scale networks.Therefore,this study propose...The use of dynamic programming(DP)algorithms to learn Bayesian network structures is limited by their high space complexity and difficulty in learning the structure of large-scale networks.Therefore,this study proposes a DP algorithm based on node block sequence constraints.The proposed algorithm constrains the traversal process of the parent graph by using the M-sequence matrix to considerably reduce the time consumption and space complexity by pruning the traversal process of the order graph using the node block sequence.Experimental results show that compared with existing DP algorithms,the proposed algorithm can obtain learning results more efficiently with less than 1%loss of accuracy,and can be used for learning larger-scale networks.展开更多
Reinforcement learning(RL)has been widely studied as an efficient class of machine learning methods for adaptive optimal control under uncertainties.In recent years,the applications of RL in optimised decision-making ...Reinforcement learning(RL)has been widely studied as an efficient class of machine learning methods for adaptive optimal control under uncertainties.In recent years,the applications of RL in optimised decision-making and motion control of intelligent vehicles have received increasing attention.Due to the complex and dynamic operating environments of intelligent vehicles,it is necessary to improve the learning efficiency and generalisation ability of RL-based decision and control algorithms under different conditions.This survey systematically examines the theoretical foundations,algorithmic advancements and practical challenges of applying RL to intelligent vehicle systems operating in complex and dynamic environments.The major algorithm frameworks of RL are first introduced,and the recent advances in RL-based decision-making and control of intelligent vehicles are overviewed.In addition to self-learning decision and control approaches using state measurements,the developments of DRL methods for end-to-end driving control of intelligent vehicles are summarised.The open problems and directions for further research works are also discussed.展开更多
The following sections of this article are the background of the experiences described in the book Creative Journals in a Bottle.Out-of-the-Box Activities That Help Teenagers Become Sensitive and Self-Confident Adults...The following sections of this article are the background of the experiences described in the book Creative Journals in a Bottle.Out-of-the-Box Activities That Help Teenagers Become Sensitive and Self-Confident Adults(Cuccu,2024).Being a teacher in a classroom of young people involves more than just being able to tell them about a topic they have to study,they are also educators and play an important role in their development in a critical period of their lives.The following sections deal with things to do and not to do in order to create an ideal environment characterized by empathy,motivation,and learning together.展开更多
A mixed adaptive dynamic programming(ADP)scheme based on zero-sum game theory is developed to address optimal control problems of autonomous underwater vehicle(AUV)systems subject to disturbances and safe constraints....A mixed adaptive dynamic programming(ADP)scheme based on zero-sum game theory is developed to address optimal control problems of autonomous underwater vehicle(AUV)systems subject to disturbances and safe constraints.By combining prior dynamic knowledge and actual sampled data,the proposed approach effectively mitigates the defect caused by the inaccurate dynamic model and significantly improves the training speed of the ADP algorithm.Initially,the dataset is enriched with sufficient reference data collected based on a nominal model without considering modelling bias.Also,the control object interacts with the real environment and continuously gathers adequate sampled data in the dataset.To comprehensively leverage the advantages of model-based and model-free methods during training,an adaptive tuning factor is introduced based on the dataset that possesses model-referenced information and conforms to the distribution of the real-world environment,which balances the influence of model-based control law and data-driven policy gradient on the direction of policy improvement.As a result,the proposed approach accelerates the learning speed compared to data-driven methods,concurrently also enhancing the tracking performance in comparison to model-based control methods.Moreover,the optimal control problem under disturbances is formulated as a zero-sum game,and the actor-critic-disturbance framework is introduced to approximate the optimal control input,cost function,and disturbance policy,respectively.Furthermore,the convergence property of the proposed algorithm based on the value iteration method is analysed.Finally,an example of AUV path following based on the improved line-of-sight guidance is presented to demonstrate the effectiveness of the proposed method.展开更多
Ransomware,particularly crypto-ransomware,remains a significant cybersecurity challenge,encrypting victim data and demanding a ransom,often leaving the data irretrievable even if payment is made.This study proposes an...Ransomware,particularly crypto-ransomware,remains a significant cybersecurity challenge,encrypting victim data and demanding a ransom,often leaving the data irretrievable even if payment is made.This study proposes an early detection approach to mitigate such threats by identifying ransomware activity before the encryption process begins.The approach employs a two-tiered approach:a signature-based method using hashing techniques to match known threats and a dynamic behavior-based analysis leveraging Cuckoo Sandbox and machine learning algorithms.A critical feature is the integration of the most effective Application Programming Interface call monitoring,which analyzes system-level interactions such as file encryption,key generation,and registry modifications.This enables the detection of both known and zero-day ransomware variants,overcoming limitations of traditional methods.The proposed technique was evaluated using classifiers such as Random Forest,Support Vector Machine,and K-Nearest Neighbors,achieving a detection accuracy of 98%based on 26 key ransomware attributes with an 80:20 training-to-testing ratio and 10-fold cross-validation.By combining minimal feature sets with robust behavioral analysis,the proposed method outperforms existing solutions and addresses current challenges in ransomware detection,thereby enhancing cybersecurity resilience.展开更多
BACKGROUND Stage IV pancreatic cancer(PC)has a poor prognosis and lacks individualized prognostic tools.Current survival prediction models are limited,and there is a need for more accurate,personalized methods.The Sur...BACKGROUND Stage IV pancreatic cancer(PC)has a poor prognosis and lacks individualized prognostic tools.Current survival prediction models are limited,and there is a need for more accurate,personalized methods.The Surveillance,Epidemiology,and End Results(SEER)database offers a valuable resource for studying large patient cohorts,yet machine learning-based nomograms for stage IV PC prognosis remain underexplored.This study hypothesizes that a machine learning-based nomogram can predict cancer-specific survival(CSS)and overall survival(OS)with high accuracy in stage IV PC patients.AIM To construct and validate a machine learning-based nomogram for predicting survival in stage IV PC patients using real-world data.METHODS Clinical data from stage IV PC patients diagnosed via pathology from 2000 to 2019 INTRODUCTION Pancreatic cancer(PC)is a significant human health issue and,by 2025,is projected to surpass breast cancer as the third leading cause of cancer-related deaths[1].In the United States,an estimated 66440 new cases and 51750 deaths due to PC were reported in 2024.PC is often asymptomatic in its early stages,with more than half of patients presenting with distant organ metastasis at the time of initial diagnosis[2].Consequently,the prognosis is very poor,with a 5-year relative survival rate of only 12.8%[2]In clinical practice,considerable heterogeneity in survival outcomes has been observed among patients with stage IV PC,highlighting the need for an individualized survival prediction tool for this population.Nomograms,which are visual tools incorporating multiple prognostic factors to predict patient survival,aid in person-alized treatment planning and clinical decision-making and are widely used in cancer prognosis evaluation[3-6].Machine learning,a core technique within artificial intelligence,employs algorithms to analyze data,learn from patterns,and predict real-world events with high accuracy,and is increasingly applied in health assessment,medical decision-making,prognosis,and personalized treatment[7-9].This study leverages the large sample size and comprehensive clinical data from the United State Surveillance,Epidemiology,and End Results(SEER)database to develop a prognostic nomogram for stage IV PC patients using machine learning,with the aim of providing individualized prognostic assessments to improve clinical decision-making.展开更多
基金co-supported by the Aeronautical Science Foundation of China(Nos.20220048051001,20230013051002)the“1912 Project”of China。
文摘This paper proposes a novel reinforcement-learning-based intelligent fault-tolerant assistance control framework for Air-breathing Hypersonic Vehicles(AHVs).Considering that Reinforcement Learning(RL)has the advantage of exploring approximate optimal strategies,an RL-based assistance controller parallel to the fundamental controller is introduced to generate the assistance control signal.Specifically,the Incremental model-based Dual Heuristic Programming(IDHP)method is adopted to design the RL-based assistance control law.In order to extend the IDHP method to the assistance control scenario,a novel linear time-varying incremental model of the closed-loop augmented system is constructed and identified in real time,which consists of the AHV plant,the fundamental controller,and the command generator.The RL agent continuously updates its neural-network weights according to the real-time identification information,and adjusts its control policy,i.e.,the assistance control signal,after detecting sudden model changes.Simulation results have validated the effectiveness of the proposed intelligent fault-tolerant control scheme under various types of elevator faults and aerodynamic/configuration parameter uncertainties.The fault-tolerant ability of the whole control system with the proposed RL-based assistance controller is validated in both inner-loop attitude and outer-loop altitude tracking tasks.
基金support from the National Natural Science Foundation of China(No.12002372)the Young Elite Scientists Sponsorship Program by China Association for Science and Technology(No.2022QNRC001)the Natural Science Foundation of Hunan Province,China(No.2021JJ40674)。
文摘The operational demands of a wide range significantly exacerbate combustion instability issues within ramjet combustor.To suppress combustion oscillations,an open-loop control system utilizing Linear Genetic Programming(LGP)has been developed for a full-scale annular ramjet combustor.The LGP is used to generate control laws that include multi-frequency forcing.These laws are then transformed into square waves to actuate the solenoid valve,which modulates the kerosene supply for open-loop control.The results show that the duty cycle has little effect on instability amplitude,whereas an increase in frequency leads to a remarked reduction in combustion amplitude.After five generations evolvements,the pressure amplitude is reduced by 40.6% under the optimal control law generated by LGP.Furthermore,the machine learning process is depicted using a proximity map of control law similarity,with the search pathway visualized by the steepest descent.All individuals go forward to the upper left corner of the map with the evolution process,terminating at the optimal individual of the fifth generation.
基金supported in part by the National Natural Science Foundation of China(62222301, 62073085, 62073158, 61890930-5, 62021003)the National Key Research and Development Program of China (2021ZD0112302, 2021ZD0112301, 2018YFC1900800-5)Beijing Natural Science Foundation (JQ19013)。
文摘Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.
基金Supported by the National Science Foundation (U.S.A.) under Grant ECS-0355364
文摘This paper introduces a self-learning control approach based on approximate dynamic programming. Dynamic programming was introduced by Bellman in the 1950's for solving optimal control problems of nonlinear dynamical systems. Due to its high computational complexity, the applications of dynamic programming have been limited to simple and small problems. The key step in finding approximate solutions to dynamic programming is to estimate the performance index in dynamic programming. The optimal control signal can then be determined by minimizing (or maximizing) the performance index. Artificial neural networks are very efficient tools in representing the performance index in dynamic programming. This paper assumes the use of neural networks for estimating the performance index in dynamic programming and for generating optimal control signals, thus to achieve optimal control through self-learning.
基金The financial support provided by the Project of National Natural Science Foundation of China(U22A20415,21978256,22308314)“Pioneer”and“Leading Goose”Research&Development Program of Zhejiang(2022C01SA442617)。
文摘Heat integration is important for energy-saving in the process industry.It is linked to the persistently challenging task of optimal design of heat exchanger networks(HEN).Due to the inherent highly nonconvex nonlinear and combinatorial nature of the HEN problem,it is not easy to find solutions of high quality for large-scale problems.The reinforcement learning(RL)method,which learns strategies through ongoing exploration and exploitation,reveals advantages in such area.However,due to the complexity of the HEN design problem,the RL method for HEN should be dedicated and designed.A hybrid strategy combining RL with mathematical programming is proposed to take better advantage of both methods.An insightful state representation of the HEN structure as well as a customized reward function is introduced.A Q-learning algorithm is applied to update the HEN structure using theε-greedy strategy.Better results are obtained from three literature cases of different scales.
基金This work was supported by Tianjin Natural Science Foundation under Grant 20JCYBJC00880Beijing key Laboratory Open Fund of Long-Life Technology of Precise Rotation and Transmission MechanismsGuangdong Provincial Key Laboratory of Intelligent Decision and Cooperative Control.
文摘Owing to extensive applications in many fields,the synchronization problem has been widely investigated in multi-agent systems.The synchronization for multi-agent systems is a pivotal issue,which means that under the designed control policy,the output of systems or the state of each agent can be consistent with the leader.The purpose of this paper is to investigate a heuristic dynamic programming(HDP)-based learning tracking control for discrete-time multi-agent systems to achieve synchronization while considering disturbances in systems.Besides,due to the difficulty of solving the coupled Hamilton–Jacobi–Bellman equation analytically,an improved HDP learning control algorithm is proposed to realize the synchronization between the leader and all following agents,which is executed by an action-critic neural network.The action and critic neural network are utilized to learn the optimal control policy and cost function,respectively,by means of introducing an auxiliary action network.Finally,two numerical examples and a practical application of mobile robots are presented to demonstrate the control performance of the HDP-based learning control algorithm.
文摘Pair programming has been widely acclaimed the best way to go in computer programming. Recently, collaboration involving more subjects has been shown to produce better results in programming environments. However, the optimum group size needed for the collaboration has not been adequately addressed. This paper seeks to inculcate and acquaint the students involved in the study with the spirit of team work in software projects and to empirically determine the effective (optimum) team size that may be desirable in programming/learning real life environments. Two different experiments were organized and conducted. Parameters for determining the optimal team size were formulated. Volunteered participants of different genders were randomly grouped into five parallel teams of different sizes ranging from 1 to 5 in the first experiment. Each team size was replicated six times. The second experiment involved teams of same gender compositions (males or females) in different sizes. The times (efforts) for problem analysis and coding as well as compile-time errors (bugs) were recorded for each team size. The effectiveness was finally analyzed for the teams. The study shows that collaboration is highly beneficial to new learners of computer programming. They easily grasp the programming concepts when the learning is done in the company of others. The study also demonstrates that the optimum team size that may be adopted in a collaborative learning of computer programming is four.
基金supported by the 2018-2020 Higher Education Talent Training Quality and Teaching Reform Project of Sichuan Province(Grant No.JG2018-46)the Science and Technology Planning Program of Sichuan University and Luzhou(Grant No.2017CDLZG30)the Postdoctoral Science fund of Sichuan University(Grant No.2019SCU12058).
文摘Programming difficulties are one of the common problems faced by software engineering students,which can lead to a rapid decline in motivation and even drop out.Probing students’programming difficulties is a crucial step in understanding their current programming situation and implementing appropriate instructional interventions.However,how to detect students’programming difficulties accurately without students’awareness remains a big challenge.Address the issues above;this paper adopts a sensor-free difficulties detecting method based on a deep neural network which employs a recurrent neural network(RNN)model and uses the sequential timing data from programming behaviour.The method can detect students’programming difficulties in real-time with 93%accuracy without interference in the programming process.In the long term,this method is the first step for establishing an automated intelligent programming environment.At the same time,it can assist teachers in noticing the difficulties that students encounter.Then,teachers can adjust their teaching plans and provide manual tutoring intervention more quickly.
文摘Students often face difficulties while taking basic programming courses due to several factors. In response, research has presented subjective assessments for diagnosing learning problems to improve the teaching of programming in higher education. In this paper, the authors propose an Object Oriented conceptual map model and organize this approach into three levels： constructing a Concept Effect Propagation Table, constructing Test Item-Concept Relationships and diagnosing Student Learning Problems with Matrix Composition. The authors＇ work is a modification of the approaches of Chert and Bai as well as Chu et al., as the authors use statistical methods, rather than fuzzy sets, for the authors＇ analysis. This paper includes a statistical summary, which has been tested on a small sample of students in King Abdulaziz University, Jeddah, Saudi Arabia, illustrating the learning problems in an Object Oriented course. The experimental results have demonstrated that this approach might aid learning and teaching in an effective way.
文摘Approximate dynamic programming (ADP) is a general and effective approach for solving optimal control and estimation problems by adapting to uncertain and nonconvex environments over time.
文摘<span style="font-family:Verdana;">Students face difficulties in programming languages learning (PLL) which encourages many scholars to investigate the factors behind that. Although there a number of positive and negative factors found to be effective in PLL procedure, utilising online tools in PLL were recognized as a positive recommended means. This motivates many researchers to provide solutions and proposals which result in a number of choices and options. However, categorising those efforts and showing what has been done, would provide a better and clear image for future studies. Therefore, this paper aims to conduct a systematic literature review to show what studies have been done and then categorise them based on the type of online tools and the aims of the research. The study follows Kitchenham and Charters guidelines for writing SLR (Systematic Literature Review). The search result reached 1390 publications between 2013-09/2018. After the filtration which has been done through selected criteria, 160 publications were found to be adequate to answer the review questions. The main results of this systematic review are categorizing the aims of the studies in online PLL tools, classifying the tools and finding the current trends of the online PLL tools.</span>
基金This study was supported by the General Project of Ganzhou Social Science Research in 2021-Research on the Transformation of Teaching and Research Form of Professional Teachers in the Blending Learning Mode of Colleges and Universities-Taking the Course“Java Programming”as an Example(Project Number:2021-028-0323).
文摘In view of the current situation that offline teaching is the main mode of teaching Java Programming in higher vocational schools,this paper introduces the online and offline hybrid teaching method and expounds it from the aspects of blended learning design,teaching organization,and implementation.At the same time,combined with the characteristics of blended learning,this paper proposes that under the new mode,teachers should actively change the form of teaching and research,the teaching mode,and the role of teachers,take students as the center,and build an independent and effective classroom.
基金Shaanxi Science Fund for Distinguished Young Scholars,Grant/Award Number:2024JC-JCQN-57Xi’an Science and Technology Plan Project,Grant/Award Number:2023JH-QCYJQ-0086+2 种基金Scientific Research Program Funded by Education Department of Shaanxi Provincial Government,Grant/Award Number:P23JP071Engineering Technology Research Center of Shaanxi Province for Intelligent Testing and Reliability Evaluation of Electronic Equipments,Grant/Award Number:2023-ZC-GCZX-00472022 Shaanxi University Youth Innovation Team Project。
文摘The use of dynamic programming(DP)algorithms to learn Bayesian network structures is limited by their high space complexity and difficulty in learning the structure of large-scale networks.Therefore,this study proposes a DP algorithm based on node block sequence constraints.The proposed algorithm constrains the traversal process of the parent graph by using the M-sequence matrix to considerably reduce the time consumption and space complexity by pruning the traversal process of the order graph using the node block sequence.Experimental results show that compared with existing DP algorithms,the proposed algorithm can obtain learning results more efficiently with less than 1%loss of accuracy,and can be used for learning larger-scale networks.
基金supported by the National Natural Science Foundation of China under Grant T2521006,Grant 62403483,Grant 62533021 and Grant U24A20279.
文摘Reinforcement learning(RL)has been widely studied as an efficient class of machine learning methods for adaptive optimal control under uncertainties.In recent years,the applications of RL in optimised decision-making and motion control of intelligent vehicles have received increasing attention.Due to the complex and dynamic operating environments of intelligent vehicles,it is necessary to improve the learning efficiency and generalisation ability of RL-based decision and control algorithms under different conditions.This survey systematically examines the theoretical foundations,algorithmic advancements and practical challenges of applying RL to intelligent vehicle systems operating in complex and dynamic environments.The major algorithm frameworks of RL are first introduced,and the recent advances in RL-based decision-making and control of intelligent vehicles are overviewed.In addition to self-learning decision and control approaches using state measurements,the developments of DRL methods for end-to-end driving control of intelligent vehicles are summarised.The open problems and directions for further research works are also discussed.
文摘The following sections of this article are the background of the experiences described in the book Creative Journals in a Bottle.Out-of-the-Box Activities That Help Teenagers Become Sensitive and Self-Confident Adults(Cuccu,2024).Being a teacher in a classroom of young people involves more than just being able to tell them about a topic they have to study,they are also educators and play an important role in their development in a critical period of their lives.The following sections deal with things to do and not to do in order to create an ideal environment characterized by empathy,motivation,and learning together.
基金National Key Research and Development Program of China,Grant/Award Number:2021YFC2801700Defense Industrial Technology Development Program,Grant/Award Numbers:JCKY2021110B024,JCKY2022110C072+6 种基金Science and Technology Innovation 2030-“New Generation Artificial Intelligence”Major Project,Grant/Award Number:2022ZD0116305Natural Science Foundation of Hefei,China,Grant/Award Number:202321National Natural Science Foundation of China,Grant/Award Numbers:U2013601,U20A20225Yangtze River Delta S&T Innovation Community Joint Research Project,Grant/Award Number:2022CSJGG0900Anhui Province Natural Science Funds for Distinguished Young Scholar,Grant/Award Number:2308085J02State Key Laboratory of Intelligent Green Vehicle and Mobility,Grant/Award Number:KFY2417State Key Laboratory of Advanced Design and Manufacturing Technology for Vehicle,Grant/Award Number:32215010。
文摘A mixed adaptive dynamic programming(ADP)scheme based on zero-sum game theory is developed to address optimal control problems of autonomous underwater vehicle(AUV)systems subject to disturbances and safe constraints.By combining prior dynamic knowledge and actual sampled data,the proposed approach effectively mitigates the defect caused by the inaccurate dynamic model and significantly improves the training speed of the ADP algorithm.Initially,the dataset is enriched with sufficient reference data collected based on a nominal model without considering modelling bias.Also,the control object interacts with the real environment and continuously gathers adequate sampled data in the dataset.To comprehensively leverage the advantages of model-based and model-free methods during training,an adaptive tuning factor is introduced based on the dataset that possesses model-referenced information and conforms to the distribution of the real-world environment,which balances the influence of model-based control law and data-driven policy gradient on the direction of policy improvement.As a result,the proposed approach accelerates the learning speed compared to data-driven methods,concurrently also enhancing the tracking performance in comparison to model-based control methods.Moreover,the optimal control problem under disturbances is formulated as a zero-sum game,and the actor-critic-disturbance framework is introduced to approximate the optimal control input,cost function,and disturbance policy,respectively.Furthermore,the convergence property of the proposed algorithm based on the value iteration method is analysed.Finally,an example of AUV path following based on the improved line-of-sight guidance is presented to demonstrate the effectiveness of the proposed method.
基金funded by the National University of Sciences and Technology(NUST)supported by the Basic Science Research Program through the National Research Foundation of Korea(NRF),funded by the Ministry of Education(2021R1IIA3049788).
文摘Ransomware,particularly crypto-ransomware,remains a significant cybersecurity challenge,encrypting victim data and demanding a ransom,often leaving the data irretrievable even if payment is made.This study proposes an early detection approach to mitigate such threats by identifying ransomware activity before the encryption process begins.The approach employs a two-tiered approach:a signature-based method using hashing techniques to match known threats and a dynamic behavior-based analysis leveraging Cuckoo Sandbox and machine learning algorithms.A critical feature is the integration of the most effective Application Programming Interface call monitoring,which analyzes system-level interactions such as file encryption,key generation,and registry modifications.This enables the detection of both known and zero-day ransomware variants,overcoming limitations of traditional methods.The proposed technique was evaluated using classifiers such as Random Forest,Support Vector Machine,and K-Nearest Neighbors,achieving a detection accuracy of 98%based on 26 key ransomware attributes with an 80:20 training-to-testing ratio and 10-fold cross-validation.By combining minimal feature sets with robust behavioral analysis,the proposed method outperforms existing solutions and addresses current challenges in ransomware detection,thereby enhancing cybersecurity resilience.
基金Supported by Mianyang Health and Health Committee 2023 Scientific Research Project,No.202309Chengdu University of Traditional Chinese Medicine University-Hospital Joint Innovation Fund,No.LH202402010Mianyang Chinese Medicine Association 2024 Traditional Chinese Medicine Inheritance and Innovation Science and Technology Project,No.MYSZYYXH-202426.
文摘BACKGROUND Stage IV pancreatic cancer(PC)has a poor prognosis and lacks individualized prognostic tools.Current survival prediction models are limited,and there is a need for more accurate,personalized methods.The Surveillance,Epidemiology,and End Results(SEER)database offers a valuable resource for studying large patient cohorts,yet machine learning-based nomograms for stage IV PC prognosis remain underexplored.This study hypothesizes that a machine learning-based nomogram can predict cancer-specific survival(CSS)and overall survival(OS)with high accuracy in stage IV PC patients.AIM To construct and validate a machine learning-based nomogram for predicting survival in stage IV PC patients using real-world data.METHODS Clinical data from stage IV PC patients diagnosed via pathology from 2000 to 2019 INTRODUCTION Pancreatic cancer(PC)is a significant human health issue and,by 2025,is projected to surpass breast cancer as the third leading cause of cancer-related deaths[1].In the United States,an estimated 66440 new cases and 51750 deaths due to PC were reported in 2024.PC is often asymptomatic in its early stages,with more than half of patients presenting with distant organ metastasis at the time of initial diagnosis[2].Consequently,the prognosis is very poor,with a 5-year relative survival rate of only 12.8%[2]In clinical practice,considerable heterogeneity in survival outcomes has been observed among patients with stage IV PC,highlighting the need for an individualized survival prediction tool for this population.Nomograms,which are visual tools incorporating multiple prognostic factors to predict patient survival,aid in person-alized treatment planning and clinical decision-making and are widely used in cancer prognosis evaluation[3-6].Machine learning,a core technique within artificial intelligence,employs algorithms to analyze data,learn from patterns,and predict real-world events with high accuracy,and is increasingly applied in health assessment,medical decision-making,prognosis,and personalized treatment[7-9].This study leverages the large sample size and comprehensive clinical data from the United State Surveillance,Epidemiology,and End Results(SEER)database to develop a prognostic nomogram for stage IV PC patients using machine learning,with the aim of providing individualized prognostic assessments to improve clinical decision-making.
