Biometric characteristics are playing a vital role in security for the last few years.Human gait classification in video sequences is an important biometrics attribute and is used for security purposes.A new framework...Biometric characteristics are playing a vital role in security for the last few years.Human gait classification in video sequences is an important biometrics attribute and is used for security purposes.A new framework for human gait classification in video sequences using deep learning(DL)fusion assisted and posterior probability-based moth flames optimization(MFO)is proposed.In the first step,the video frames are resized and finetuned by two pre-trained lightweight DL models,EfficientNetB0 and MobileNetV2.Both models are selected based on the top-5 accuracy and less number of parameters.Later,both models are trained through deep transfer learning and extracted deep features fused using a voting scheme.In the last step,the authors develop a posterior probabilitybased MFO feature selection algorithm to select the best features.The selected features are classified using several supervised learning methods.The CASIA-B publicly available dataset has been employed for the experimental process.On this dataset,the authors selected six angles such as 0°,18°,90°,108°,162°,and 180°and obtained an average accuracy of 96.9%,95.7%,86.8%,90.0%,95.1%,and 99.7%.Results demonstrate comparable improvement in accuracy and significantly minimize the computational time with recent state-of-the-art techniques.展开更多
This study aimed to address the challenge of accurately and reliably detecting tomatoes in dense planting environments,a critical prerequisite for the automation implementation of robotic harvesting.However,the heavy ...This study aimed to address the challenge of accurately and reliably detecting tomatoes in dense planting environments,a critical prerequisite for the automation implementation of robotic harvesting.However,the heavy reliance on extensive manually annotated datasets for training deep learning models still poses significant limitations to their application in real-world agricultural production environments.To overcome these limitations,we employed domain adaptive learning approach combined with the YOLOv5 model to develop a novel tomato detection model called as TDA-YOLO(tomato detection domain adaptation).We designated the normal illumination scenes in dense planting environments as the source domain and utilized various other illumination scenes as the target domain.To construct bridge mechanism between source and target domains,neural preset for color style transfer is introduced to generate a pseudo-dataset,which served to deal with domain discrepancy.Furthermore,this study combines the semi-supervised learning method to enable the model to extract domain-invariant features more fully,and uses knowledge distillation to improve the model's ability to adapt to the target domain.Additionally,for purpose of promoting inference speed and low computational demand,the lightweight FasterNet network was integrated into the YOLOv5's C3 module,creating a modified C3_Faster module.The experimental results demonstrated that the proposed TDA-YOLO model significantly outperformed original YOLOv5s model,achieving a mAP(mean average precision)of 96.80%for tomato detection across diverse scenarios in dense planting environments,increasing by 7.19 percentage points;Compared with the latest YOLOv8 and YOLOv9,it is also 2.17 and 1.19 percentage points higher,respectively.The model's average detection time per image was an impressive 15 milliseconds,with a FLOPs(floating point operations per second)count of 13.8 G.After acceleration processing,the detection accuracy of the TDA-YOLO model on the Jetson Xavier NX development board is 90.95%,the mAP value is 91.35%,and the detection time of each image is 21 ms,which can still meet the requirements of real-time detection of tomatoes in dense planting environment.The experimental results show that the proposed TDA-YOLO model can accurately and quickly detect tomatoes in dense planting environment,and at the same time avoid the use of a large number of annotated data,which provides technical support for the development of automatic harvesting systems for tomatoes and other fruits.展开更多
Hybrid precoding is considered as a promising low-cost technique for millimeter wave(mm-wave)massive Multi-Input Multi-Output(MIMO)systems.In this work,referring to the time-varying propagation circumstances,with semi...Hybrid precoding is considered as a promising low-cost technique for millimeter wave(mm-wave)massive Multi-Input Multi-Output(MIMO)systems.In this work,referring to the time-varying propagation circumstances,with semi-supervised Incremental Learning(IL),we propose an online hybrid beamforming scheme.Firstly,given the constraint of constant modulus on analog beamformer and combiner,we propose a new broadnetwork-based structure for the design model of hybrid beamforming.Compared with the existing network structure,the proposed network structure can achieve better transmission performance and lower complexity.Moreover,to enhance the efficiency of IL further,by combining the semi-supervised graph with IL,we propose a hybrid beamforming scheme based on chunk-by-chunk semi-supervised learning,where only few transmissions are required to calculate the label and all other unlabelled transmissions would also be put into a training data chunk.Unlike the existing single-by-single approach where transmissions during the model update are not taken into the consideration of model update,all transmissions,even the ones during the model update,would make contributions to model update in the proposed method.During the model update,the amount of unlabelled transmissions is very large and they also carry some information,the prediction performance can be enhanced to some extent by these unlabelled channel data.Simulation results demonstrate the spectral efficiency of the proposed method outperforms that of the existing single-by-single approach.Besides,we prove the general complexity of the proposed method is lower than that of the existing approach and give the condition under which its absolute complexity outperforms that of the existing approach.展开更多
This paper investigates the challenges associated with Unmanned Aerial Vehicle (UAV) collaborative search and target tracking in dynamic and unknown environments characterized by limited field of view. The primary obj...This paper investigates the challenges associated with Unmanned Aerial Vehicle (UAV) collaborative search and target tracking in dynamic and unknown environments characterized by limited field of view. The primary objective is to explore the unknown environments to locate and track targets effectively. To address this problem, we propose a novel Multi-Agent Reinforcement Learning (MARL) method based on Graph Neural Network (GNN). Firstly, a method is introduced for encoding continuous-space multi-UAV problem data into spatial graphs which establish essential relationships among agents, obstacles, and targets. Secondly, a Graph AttenTion network (GAT) model is presented, which focuses exclusively on adjacent nodes, learns attention weights adaptively and allows agents to better process information in dynamic environments. Reward functions are specifically designed to tackle exploration challenges in environments with sparse rewards. By introducing a framework that integrates centralized training and distributed execution, the advancement of models is facilitated. Simulation results show that the proposed method outperforms the existing MARL method in search rate and tracking performance with less collisions. The experiments show that the proposed method can be extended to applications with a larger number of agents, which provides a potential solution to the challenging problem of multi-UAV autonomous tracking in dynamic unknown environments.展开更多
Previous studies have extensively explored the critical influence of the built environment on land values,but the non-linear relationship has yet to be fully revealed.This study aims to uncover the non-linear relation...Previous studies have extensively explored the critical influence of the built environment on land values,but the non-linear relationship has yet to be fully revealed.This study aims to uncover the non-linear relationship between land values and the five built environment dimensions using machine learning algorithms and Shapley Additive ex Planation(SHAP).The results highlight that the Gradient Boost Decision Tree (GBDT) outperforms e Xtreme Gradient Boosting (XGBoost),Ordinary Least Squares (OLS),and Multiscale Geographically Weighted Regression (MGWR) in land value estimation,exhibiting higher R^(2) and lower Root Mean Square Error (RMSE) and Mean Absolute Error (MAE).The results illustrate that density and destination accessibility are the dominant factors,contributing 32.48%and37.38%to land value variation,respectively.We observed that the top three factors affecting land values are the built-floor area ratio,the number of floors and the number of restaurants.Additionally,the results revealed the non-linear relationship between the built environment and land values,suggesting that maintaining built environment features at optimal thresholds may increase land values.Neglecting interaction effects may lead to bias in determining relationships between land values and the built environment.This study contributes to the literature by providing non-linear and threshold identification evidence in land value determinants,offering valuable insights for urban planners and real estate managers.展开更多
The challenge of enhancing the generalization capacity of reinforcement learning(RL)agents remains a formidable obstacle.Existing RL methods,despite achieving superhuman performance on certain benchmarks,often struggl...The challenge of enhancing the generalization capacity of reinforcement learning(RL)agents remains a formidable obstacle.Existing RL methods,despite achieving superhuman performance on certain benchmarks,often struggle with this aspect.A potential reason is that the benchmarks used for training and evaluation may not adequately offer a diverse set of transferable tasks.Although recent studies have developed bench-marking environments to address this shortcoming,they typically fall short in providing tasks that both ensure a solid foundation for generalization and exhibit significant variability.To overcome these limitations,this work introduces the concept that‘objects are composed of more fundamental components’in environment design,as implemented in the proposed environment called summon the magic(StM).This environment generates tasks where objects are derived from extensible and shareable basic components,facilitating strategy reuse and enhancing generalization.Furthermore,two new metrics,adaptation sensitivity range(ASR)and parameter correlation coefficient(PCC),are proposed to better capture and evaluate the generalization process of RL agents.Experimental results show that increasing the number of basic components of the object reduces the proximal policy optimization(PPO)agent’s training-testing gap by 60.9%(in episode reward),significantly alleviating overfitting.Additionally,linear variations in other environmental factors,such as the training monster set proportion and the total number of basic components,uniformly decrease the gap by at least 32.1%.These results highlight StM’s effectiveness in benchmarking and probing the generalization capabilities of RL algorithms.展开更多
In this paper,we focus on the channel estimation for multi-user MIMO-OFDM systems in rich scattering environments.We find that channel sparsity in the delay-angle domain is severely compromised in rich scattering envi...In this paper,we focus on the channel estimation for multi-user MIMO-OFDM systems in rich scattering environments.We find that channel sparsity in the delay-angle domain is severely compromised in rich scattering environments,so that most existing compressed sensing(CS)based techniques can harvest a very limited gain(if any)in reducing the channel estimation overhead.To address the problem,we propose the learning-based turbo message passing(LTMP)algorithm.Instead of exploiting the channel sparsity,LTMP is able to efficiently extract the channel feature via deep learning as well as to exploit the channel continuity in the frequency domain via block-wise linear modelling.More specifically,as a component of LTMP,we develop a multi-scale parallel dilated convolutional neural network(MPDCNN),which leverages frequency-space channel correlation in different scales for channel denoising.We evaluate the LTMP’s performance in MIMO-OFDM channels using the 3rd generation partnership project(3GPP)clustered delay line(CDL)channel models.Simulation results show that the proposed channel estimation method has more than 5 dB power gain than the existing algorithms when the normalized mean-square error of the channel estimation is-20 dB.The proposed algorithm also exhibits strong robustness in various environments.展开更多
Blended learning(BL)has been widely adopted to improve students’academic achievements in higher education.However,its success relies mainly on student engagement,which plays an essential role in active learning and p...Blended learning(BL)has been widely adopted to improve students’academic achievements in higher education.However,its success relies mainly on student engagement,which plays an essential role in active learning and provides a rich understanding of students’experiences.The study utilized three self-designed scales-the Teacher Support Scale,Student Engagement Scale,and Student Learning Experience Scale-to gauge and examine the impact and relationship between perceived teacher support,student behavioral engagement,and the intermediary role of learning experiences.A cohort of 899 college students undertaking the obligatory College English course through BL modes across five Chinese universities actively participated by completing a comprehensive questionnaire.The results showed significant correlations between perceived teacher support,learning experience,and behavioral engagement.Perceived teacher support significantly predicted students’behavioral engagement,with socio-affective support exerting the most substantial predictive effects.All predictive effects were partially mediated by learning experience(learning mode,online resources,overall LMS-based learning,interaction with their instructor and peers,and learning outcome).The influence of perceived teacher support on behavioral engagement differed between students who reported the most positive(vs.negative)learning experiences.Suggestions for further research are offered for consideration.展开更多
The traditional concept of ”one-size-fits-all” educational and training programmes is no more fully adequate to meet the increasing demand worldwide. E-learning, as an alternative approach to traditional face-to-fac...The traditional concept of ”one-size-fits-all” educational and training programmes is no more fully adequate to meet the increasing demand worldwide. E-learning, as an alternative approach to traditional face-to-face education, is creating immense challenges for educational institutions to develop new approaches for the production and delivery of cost effective and efficient e-contents. Although, there have been many developments in web-based programmes, they have not fully attained their potential due to a variety of factors. These include: 1) lack of exchangeability between learning materials, 2) delivery mechanisms incompatible with the pedagogical design, 3) low student interaction and insensitive learning processes, 4) absence of intelligent online programme advice and guidance, 5) inflexibility in meeting diverse needs, and 6) institutionally centred ineffective implementation strategies. This paper addresses the critical elements for successful delivery of e-learning environments and then focuses on proposing a framework for the development of an integrated knowledge-based learning environment which has the potential to producer cost effective and personalised training programmes.展开更多
The paper analyzes the current condition of the use of virtual learning environment(VLE) in Zhejiang University of Chinese Medicine. It is indicated that students show a positive attitude toward this technology, but t...The paper analyzes the current condition of the use of virtual learning environment(VLE) in Zhejiang University of Chinese Medicine. It is indicated that students show a positive attitude toward this technology, but the use of it fails to meet students' perception. In light of this, recommendations are made with a view to enhance the use of VLE.展开更多
This paper, firstly, acknowledges the importance of classroom environment and the problems existing in the college English classroom. And then, it offers some ways of improving the classroom environment which is very ...This paper, firstly, acknowledges the importance of classroom environment and the problems existing in the college English classroom. And then, it offers some ways of improving the classroom environment which is very critical to evaluate educational programs and curriculum and provides guidance to teachers who are eager to boost their classroom teaching.展开更多
With the increasing development of economy and society,the 21st century will surely become an era of rapid development of information technology.Based on the macro and micro levels of education in China,this paper int...With the increasing development of economy and society,the 21st century will surely become an era of rapid development of information technology.Based on the macro and micro levels of education in China,this paper introduces the"affordance theory"to analyze and discuss the current situation of College English learning environment in China,and puts forward new goals and principles to promote the future development of College English learning environment in order to better promote its effective transformation.展开更多
Ubiquitous learning is a new type of learning method with rich learning concepts and educational significance. The study of ubiquitous learning began in 1991, and has experienced three stages of gestation, start-up an...Ubiquitous learning is a new type of learning method with rich learning concepts and educational significance. The study of ubiquitous learning began in 1991, and has experienced three stages of gestation, start-up and formation and development.After entering the 21 st century, new technologies and new ideas have emerged endlessly. The change in learning methods has led to the flip of classroom teaching, and ubiquitous learning has become more known as the pace of social development. The current higher vocational education presents the characteristics of disjointed education content, misaligned learning roles, and single teaching form. The integration of ubiquitous learning environment into vocational education teaching is a new direction for the development of vocational education.展开更多
Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and ...Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.展开更多
Multi-agent reinforcement learning(MARL)has been a rapidly evolving field.This paper presents a comprehensive survey of MARL and its applications.We trace the historical evolution of MARL,highlight its progress,and di...Multi-agent reinforcement learning(MARL)has been a rapidly evolving field.This paper presents a comprehensive survey of MARL and its applications.We trace the historical evolution of MARL,highlight its progress,and discuss related survey works.Then,we review the existing works addressing inherent challenges and those focusing on diverse applications.Some representative stochastic games,MARL means,spatial forms of MARL,and task classification are revisited.We then conduct an in-depth exploration of a variety of challenges encountered in MARL applications.We also address critical operational aspects,such as hyperparameter tuning and computational complexity,which are pivotal in practical implementations of MARL.Afterward,we make a thorough overview of the applications of MARL to intelligent machines and devices,chemical engineering,biotechnology,healthcare,and societal issues,which highlights the extensive potential and relevance of MARL within both current and future technological contexts.Our survey also encompasses a detailed examination of benchmark environments used in MARL research,which are instrumental in evaluating MARL algorithms and demonstrate the adaptability of MARL to diverse application scenarios.In the end,we give our prospect for MARL and discuss their related techniques and potential future applications.展开更多
基金King Saud University,Grant/Award Number:RSP2024R157。
文摘Biometric characteristics are playing a vital role in security for the last few years.Human gait classification in video sequences is an important biometrics attribute and is used for security purposes.A new framework for human gait classification in video sequences using deep learning(DL)fusion assisted and posterior probability-based moth flames optimization(MFO)is proposed.In the first step,the video frames are resized and finetuned by two pre-trained lightweight DL models,EfficientNetB0 and MobileNetV2.Both models are selected based on the top-5 accuracy and less number of parameters.Later,both models are trained through deep transfer learning and extracted deep features fused using a voting scheme.In the last step,the authors develop a posterior probabilitybased MFO feature selection algorithm to select the best features.The selected features are classified using several supervised learning methods.The CASIA-B publicly available dataset has been employed for the experimental process.On this dataset,the authors selected six angles such as 0°,18°,90°,108°,162°,and 180°and obtained an average accuracy of 96.9%,95.7%,86.8%,90.0%,95.1%,and 99.7%.Results demonstrate comparable improvement in accuracy and significantly minimize the computational time with recent state-of-the-art techniques.
基金The National Natural Science Foundation of China (32371993)The Natural Science Research Key Project of Anhui Provincial University(2022AH040125&2023AH040135)The Key Research and Development Plan of Anhui Province (202204c06020022&2023n06020057)。
文摘This study aimed to address the challenge of accurately and reliably detecting tomatoes in dense planting environments,a critical prerequisite for the automation implementation of robotic harvesting.However,the heavy reliance on extensive manually annotated datasets for training deep learning models still poses significant limitations to their application in real-world agricultural production environments.To overcome these limitations,we employed domain adaptive learning approach combined with the YOLOv5 model to develop a novel tomato detection model called as TDA-YOLO(tomato detection domain adaptation).We designated the normal illumination scenes in dense planting environments as the source domain and utilized various other illumination scenes as the target domain.To construct bridge mechanism between source and target domains,neural preset for color style transfer is introduced to generate a pseudo-dataset,which served to deal with domain discrepancy.Furthermore,this study combines the semi-supervised learning method to enable the model to extract domain-invariant features more fully,and uses knowledge distillation to improve the model's ability to adapt to the target domain.Additionally,for purpose of promoting inference speed and low computational demand,the lightweight FasterNet network was integrated into the YOLOv5's C3 module,creating a modified C3_Faster module.The experimental results demonstrated that the proposed TDA-YOLO model significantly outperformed original YOLOv5s model,achieving a mAP(mean average precision)of 96.80%for tomato detection across diverse scenarios in dense planting environments,increasing by 7.19 percentage points;Compared with the latest YOLOv8 and YOLOv9,it is also 2.17 and 1.19 percentage points higher,respectively.The model's average detection time per image was an impressive 15 milliseconds,with a FLOPs(floating point operations per second)count of 13.8 G.After acceleration processing,the detection accuracy of the TDA-YOLO model on the Jetson Xavier NX development board is 90.95%,the mAP value is 91.35%,and the detection time of each image is 21 ms,which can still meet the requirements of real-time detection of tomatoes in dense planting environment.The experimental results show that the proposed TDA-YOLO model can accurately and quickly detect tomatoes in dense planting environment,and at the same time avoid the use of a large number of annotated data,which provides technical support for the development of automatic harvesting systems for tomatoes and other fruits.
基金supported by the National Science Foundation of China under Grant No.62101467.
文摘Hybrid precoding is considered as a promising low-cost technique for millimeter wave(mm-wave)massive Multi-Input Multi-Output(MIMO)systems.In this work,referring to the time-varying propagation circumstances,with semi-supervised Incremental Learning(IL),we propose an online hybrid beamforming scheme.Firstly,given the constraint of constant modulus on analog beamformer and combiner,we propose a new broadnetwork-based structure for the design model of hybrid beamforming.Compared with the existing network structure,the proposed network structure can achieve better transmission performance and lower complexity.Moreover,to enhance the efficiency of IL further,by combining the semi-supervised graph with IL,we propose a hybrid beamforming scheme based on chunk-by-chunk semi-supervised learning,where only few transmissions are required to calculate the label and all other unlabelled transmissions would also be put into a training data chunk.Unlike the existing single-by-single approach where transmissions during the model update are not taken into the consideration of model update,all transmissions,even the ones during the model update,would make contributions to model update in the proposed method.During the model update,the amount of unlabelled transmissions is very large and they also carry some information,the prediction performance can be enhanced to some extent by these unlabelled channel data.Simulation results demonstrate the spectral efficiency of the proposed method outperforms that of the existing single-by-single approach.Besides,we prove the general complexity of the proposed method is lower than that of the existing approach and give the condition under which its absolute complexity outperforms that of the existing approach.
基金supported by the National Natural Science Foundation of China(Nos.12272104,U22B2013).
文摘This paper investigates the challenges associated with Unmanned Aerial Vehicle (UAV) collaborative search and target tracking in dynamic and unknown environments characterized by limited field of view. The primary objective is to explore the unknown environments to locate and track targets effectively. To address this problem, we propose a novel Multi-Agent Reinforcement Learning (MARL) method based on Graph Neural Network (GNN). Firstly, a method is introduced for encoding continuous-space multi-UAV problem data into spatial graphs which establish essential relationships among agents, obstacles, and targets. Secondly, a Graph AttenTion network (GAT) model is presented, which focuses exclusively on adjacent nodes, learns attention weights adaptively and allows agents to better process information in dynamic environments. Reward functions are specifically designed to tackle exploration challenges in environments with sparse rewards. By introducing a framework that integrates centralized training and distributed execution, the advancement of models is facilitated. Simulation results show that the proposed method outperforms the existing MARL method in search rate and tracking performance with less collisions. The experiments show that the proposed method can be extended to applications with a larger number of agents, which provides a potential solution to the challenging problem of multi-UAV autonomous tracking in dynamic unknown environments.
文摘Previous studies have extensively explored the critical influence of the built environment on land values,but the non-linear relationship has yet to be fully revealed.This study aims to uncover the non-linear relationship between land values and the five built environment dimensions using machine learning algorithms and Shapley Additive ex Planation(SHAP).The results highlight that the Gradient Boost Decision Tree (GBDT) outperforms e Xtreme Gradient Boosting (XGBoost),Ordinary Least Squares (OLS),and Multiscale Geographically Weighted Regression (MGWR) in land value estimation,exhibiting higher R^(2) and lower Root Mean Square Error (RMSE) and Mean Absolute Error (MAE).The results illustrate that density and destination accessibility are the dominant factors,contributing 32.48%and37.38%to land value variation,respectively.We observed that the top three factors affecting land values are the built-floor area ratio,the number of floors and the number of restaurants.Additionally,the results revealed the non-linear relationship between the built environment and land values,suggesting that maintaining built environment features at optimal thresholds may increase land values.Neglecting interaction effects may lead to bias in determining relationships between land values and the built environment.This study contributes to the literature by providing non-linear and threshold identification evidence in land value determinants,offering valuable insights for urban planners and real estate managers.
基金Supported by the National Key R&D Program of China(No.2023YFB4502200)the National Natural Science Foundation of China(No.U22A2028,61925208,62222214,62341411,62102398,62102399,U20A20227,62302478,62302482,62302483,62302480,62302481)+2 种基金the Strategic Priority Research Program of the Chinese Academy of Sciences(No.XDB0660300,XDB0660301,XDB0660302)the Chinese Academy of Sciences Project for Young Scientists in Basic Research(No.YSBR-029)the Youth Innovation Promotion Association of Chinese Academy of Sciences and Xplore Prize.
文摘The challenge of enhancing the generalization capacity of reinforcement learning(RL)agents remains a formidable obstacle.Existing RL methods,despite achieving superhuman performance on certain benchmarks,often struggle with this aspect.A potential reason is that the benchmarks used for training and evaluation may not adequately offer a diverse set of transferable tasks.Although recent studies have developed bench-marking environments to address this shortcoming,they typically fall short in providing tasks that both ensure a solid foundation for generalization and exhibit significant variability.To overcome these limitations,this work introduces the concept that‘objects are composed of more fundamental components’in environment design,as implemented in the proposed environment called summon the magic(StM).This environment generates tasks where objects are derived from extensible and shareable basic components,facilitating strategy reuse and enhancing generalization.Furthermore,two new metrics,adaptation sensitivity range(ASR)and parameter correlation coefficient(PCC),are proposed to better capture and evaluate the generalization process of RL agents.Experimental results show that increasing the number of basic components of the object reduces the proximal policy optimization(PPO)agent’s training-testing gap by 60.9%(in episode reward),significantly alleviating overfitting.Additionally,linear variations in other environmental factors,such as the training monster set proportion and the total number of basic components,uniformly decrease the gap by at least 32.1%.These results highlight StM’s effectiveness in benchmarking and probing the generalization capabilities of RL algorithms.
基金supported in part by the National Key Research and Development Program of China under Grant 2020YFB1804800.
文摘In this paper,we focus on the channel estimation for multi-user MIMO-OFDM systems in rich scattering environments.We find that channel sparsity in the delay-angle domain is severely compromised in rich scattering environments,so that most existing compressed sensing(CS)based techniques can harvest a very limited gain(if any)in reducing the channel estimation overhead.To address the problem,we propose the learning-based turbo message passing(LTMP)algorithm.Instead of exploiting the channel sparsity,LTMP is able to efficiently extract the channel feature via deep learning as well as to exploit the channel continuity in the frequency domain via block-wise linear modelling.More specifically,as a component of LTMP,we develop a multi-scale parallel dilated convolutional neural network(MPDCNN),which leverages frequency-space channel correlation in different scales for channel denoising.We evaluate the LTMP’s performance in MIMO-OFDM channels using the 3rd generation partnership project(3GPP)clustered delay line(CDL)channel models.Simulation results show that the proposed channel estimation method has more than 5 dB power gain than the existing algorithms when the normalized mean-square error of the channel estimation is-20 dB.The proposed algorithm also exhibits strong robustness in various environments.
基金Zhejiang Provincial Philosophy and Social Sciences Planning Project from Zhejiang Office of Philosophy and Social Science(21NDJC092YB)Zhejiang Provincial Educational Science Plan Project(2021SCG166)。
文摘Blended learning(BL)has been widely adopted to improve students’academic achievements in higher education.However,its success relies mainly on student engagement,which plays an essential role in active learning and provides a rich understanding of students’experiences.The study utilized three self-designed scales-the Teacher Support Scale,Student Engagement Scale,and Student Learning Experience Scale-to gauge and examine the impact and relationship between perceived teacher support,student behavioral engagement,and the intermediary role of learning experiences.A cohort of 899 college students undertaking the obligatory College English course through BL modes across five Chinese universities actively participated by completing a comprehensive questionnaire.The results showed significant correlations between perceived teacher support,learning experience,and behavioral engagement.Perceived teacher support significantly predicted students’behavioral engagement,with socio-affective support exerting the most substantial predictive effects.All predictive effects were partially mediated by learning experience(learning mode,online resources,overall LMS-based learning,interaction with their instructor and peers,and learning outcome).The influence of perceived teacher support on behavioral engagement differed between students who reported the most positive(vs.negative)learning experiences.Suggestions for further research are offered for consideration.
文摘The traditional concept of ”one-size-fits-all” educational and training programmes is no more fully adequate to meet the increasing demand worldwide. E-learning, as an alternative approach to traditional face-to-face education, is creating immense challenges for educational institutions to develop new approaches for the production and delivery of cost effective and efficient e-contents. Although, there have been many developments in web-based programmes, they have not fully attained their potential due to a variety of factors. These include: 1) lack of exchangeability between learning materials, 2) delivery mechanisms incompatible with the pedagogical design, 3) low student interaction and insensitive learning processes, 4) absence of intelligent online programme advice and guidance, 5) inflexibility in meeting diverse needs, and 6) institutionally centred ineffective implementation strategies. This paper addresses the critical elements for successful delivery of e-learning environments and then focuses on proposing a framework for the development of an integrated knowledge-based learning environment which has the potential to producer cost effective and personalised training programmes.
文摘The paper analyzes the current condition of the use of virtual learning environment(VLE) in Zhejiang University of Chinese Medicine. It is indicated that students show a positive attitude toward this technology, but the use of it fails to meet students' perception. In light of this, recommendations are made with a view to enhance the use of VLE.
文摘This paper, firstly, acknowledges the importance of classroom environment and the problems existing in the college English classroom. And then, it offers some ways of improving the classroom environment which is very critical to evaluate educational programs and curriculum and provides guidance to teachers who are eager to boost their classroom teaching.
文摘With the increasing development of economy and society,the 21st century will surely become an era of rapid development of information technology.Based on the macro and micro levels of education in China,this paper introduces the"affordance theory"to analyze and discuss the current situation of College English learning environment in China,and puts forward new goals and principles to promote the future development of College English learning environment in order to better promote its effective transformation.
文摘Ubiquitous learning is a new type of learning method with rich learning concepts and educational significance. The study of ubiquitous learning began in 1991, and has experienced three stages of gestation, start-up and formation and development.After entering the 21 st century, new technologies and new ideas have emerged endlessly. The change in learning methods has led to the flip of classroom teaching, and ubiquitous learning has become more known as the pace of social development. The current higher vocational education presents the characteristics of disjointed education content, misaligned learning roles, and single teaching form. The integration of ubiquitous learning environment into vocational education teaching is a new direction for the development of vocational education.
基金supported in part by the National Natural Science Foundation of China(62222301, 62073085, 62073158, 61890930-5, 62021003)the National Key Research and Development Program of China (2021ZD0112302, 2021ZD0112301, 2018YFC1900800-5)Beijing Natural Science Foundation (JQ19013)。
文摘Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.
基金Ministry of Education,Singapore,under AcRF TIER 1 Grant RG64/23the Eric and Wendy Schmidt AI in Science Postdoctoral Fellowship,a Schmidt Futures program,USA.
文摘Multi-agent reinforcement learning(MARL)has been a rapidly evolving field.This paper presents a comprehensive survey of MARL and its applications.We trace the historical evolution of MARL,highlight its progress,and discuss related survey works.Then,we review the existing works addressing inherent challenges and those focusing on diverse applications.Some representative stochastic games,MARL means,spatial forms of MARL,and task classification are revisited.We then conduct an in-depth exploration of a variety of challenges encountered in MARL applications.We also address critical operational aspects,such as hyperparameter tuning and computational complexity,which are pivotal in practical implementations of MARL.Afterward,we make a thorough overview of the applications of MARL to intelligent machines and devices,chemical engineering,biotechnology,healthcare,and societal issues,which highlights the extensive potential and relevance of MARL within both current and future technological contexts.Our survey also encompasses a detailed examination of benchmark environments used in MARL research,which are instrumental in evaluating MARL algorithms and demonstrate the adaptability of MARL to diverse application scenarios.In the end,we give our prospect for MARL and discuss their related techniques and potential future applications.
