With the advent of sixth-generation mobile communications(6G),space-air-ground integrated networks have become mainstream.This paper focuses on collaborative scheduling for mobile edge computing(MEC)under a three-tier...With the advent of sixth-generation mobile communications(6G),space-air-ground integrated networks have become mainstream.This paper focuses on collaborative scheduling for mobile edge computing(MEC)under a three-tier heterogeneous architecture composed of mobile devices,unmanned aerial vehicles(UAVs),and macro base stations(BSs).This scenario typically faces fast channel fading,dynamic computational loads,and energy constraints,whereas classical queuing-theoretic or convex-optimization approaches struggle to yield robust solutions in highly dynamic settings.To address this issue,we formulate a multi-agent Markov decision process(MDP)for an air-ground-fused MEC system,unify link selection,bandwidth/power allocation,and task offloading into a continuous action space and propose a joint scheduling strategy that is based on an improved MATD3 algorithm.The improvements include Alternating Layer Normalization(ALN)in the actor to suppress gradient variance,Residual Orthogonalization(RO)in the critic to reduce the correlation between the twin Q-value estimates,and a dynamic-temperature reward to enable adaptive trade-offs during training.On a multi-user,dual-link simulation platform,we conduct ablation and baseline comparisons.The results reveal that the proposed method has better convergence and stability.Compared with MADDPG,TD3,and DSAC,our algorithm achieves more robust performance across key metrics.展开更多
This paper focuses on the problem of multi-station multi-robot spot welding task assignment,and proposes a deep reinforcement learning(DRL)framework,which is made up of a public graph attention network and independent...This paper focuses on the problem of multi-station multi-robot spot welding task assignment,and proposes a deep reinforcement learning(DRL)framework,which is made up of a public graph attention network and independent policy networks.The graph of welding spots distribution is encoded using the graph attention network.Independent policy networks with attention mechanism as a decoder can handle the encoded graph and decide to assign robots to different tasks.The policy network is used to convert the large scale welding spots allocation problem to multiple small scale singlerobot welding path planning problems,and the path planning problem is quickly solved through existing methods.Then,the model is trained through reinforcement learning.In addition,the task balancing method is used to allocate tasks to multiple stations.The proposed algorithm is compared with classical algorithms,and the results show that the algorithm based on DRL can produce higher quality solutions.展开更多
Cybertwin-enabled 6th Generation(6G)network is envisioned to support artificial intelligence-native management to meet changing demands of 6G applications.Multi-Agent Deep Reinforcement Learning(MADRL)technologies dri...Cybertwin-enabled 6th Generation(6G)network is envisioned to support artificial intelligence-native management to meet changing demands of 6G applications.Multi-Agent Deep Reinforcement Learning(MADRL)technologies driven by Cybertwins have been proposed for adaptive task offloading strategies.However,the existence of random transmission delay between Cybertwin-driven agents and underlying networks is not considered in related works,which destroys the standard Markov property and increases the decision reaction time to reduce the task offloading strategy performance.In order to address this problem,we propose a pipelining task offloading method to lower the decision reaction time and model it as a delay-aware Markov Decision Process(MDP).Then,we design a delay-aware MADRL algorithm to minimize the weighted sum of task execution latency and energy consumption.Firstly,the state space is augmented using the lastly-received state and historical actions to rebuild the Markov property.Secondly,Gate Transformer-XL is introduced to capture historical actions'importance and maintain the consistent input dimension dynamically changed due to random transmission delays.Thirdly,a sampling method and a new loss function with the difference between the current and target state value and the difference between real state-action value and augmented state-action value are designed to obtain state transition trajectories close to the real ones.Numerical results demonstrate that the proposed methods are effective in reducing reaction time and improving the task offloading performance in the random-delay Cybertwin-enabled 6G networks.展开更多
The advent of the internet-of-everything era has led to the increased use of mobile edge computing.The rise of artificial intelligence has provided many possibilities for the low-latency task-offloading demands of use...The advent of the internet-of-everything era has led to the increased use of mobile edge computing.The rise of artificial intelligence has provided many possibilities for the low-latency task-offloading demands of users,but existing technologies rigidly assume that there is only one task to be offloaded in each time slot at the terminal.In practical scenarios,there are often numerous computing tasks to be executed at the terminal,leading to a cumulative delay for subsequent task offloading.Therefore,the efficient processing of multiple computing tasks on the terminal has become highly challenging.To address the lowlatency offloading requirements for multiple computational tasks on terminal devices,we propose a terminal multitask parallel offloading algorithm based on deep reinforcement learning.Specifically,we first establish a mobile edge computing system model consisting of a single edge server and multiple terminal users.We then model the task offloading decision problem as a Markov decision process,and solve this problem using the Dueling Deep-Q Network algorithm to obtain the optimal offloading strategy.Experimental results demonstrate that,under the same constraints,our proposed algorithm reduces the average system latency.展开更多
The rapid advancement of Industry 4.0 has revolutionized manufacturing,shifting production from centralized control to decentralized,intelligent systems.Smart factories are now expected to achieve high adaptability an...The rapid advancement of Industry 4.0 has revolutionized manufacturing,shifting production from centralized control to decentralized,intelligent systems.Smart factories are now expected to achieve high adaptability and resource efficiency,particularly in mass customization scenarios where production schedules must accommodate dynamic and personalized demands.To address the challenges of dynamic task allocation,uncertainty,and realtime decision-making,this paper proposes Pathfinder,a deep reinforcement learning-based scheduling framework.Pathfinder models scheduling data through three key matrices:execution time(the time required for a job to complete),completion time(the actual time at which a job is finished),and efficiency(the performance of executing a single job).By leveraging neural networks,Pathfinder extracts essential features from these matrices,enabling intelligent decision-making in dynamic production environments.Unlike traditional approaches with fixed scheduling rules,Pathfinder dynamically selects from ten diverse scheduling rules,optimizing decisions based on real-time environmental conditions.To further enhance scheduling efficiency,a specialized reward function is designed to support dynamic task allocation and real-time adjustments.This function helps Pathfinder continuously refine its scheduling strategy,improving machine utilization and minimizing job completion times.Through reinforcement learning,Pathfinder adapts to evolving production demands,ensuring robust performance in real-world applications.Experimental results demonstrate that Pathfinder outperforms traditional scheduling approaches,offering improved coordination and efficiency in smart factories.By integrating deep reinforcement learning,adaptable scheduling strategies,and an innovative reward function,Pathfinder provides an effective solution to the growing challenges of multi-robot job scheduling in mass customization environments.展开更多
With focus now placed on the learner, more attention is given to his learning style, multiple intelligence and developing learning strategies to enable him to make sense of and use of the target language appropriately...With focus now placed on the learner, more attention is given to his learning style, multiple intelligence and developing learning strategies to enable him to make sense of and use of the target language appropriately in varied contexts and with different uses of the language. To attain this, the teacher is tasked with designing, monitoring and processing language learning activities for students to carry out and in the process learn by doing and reflecting on the learning process they went through as they interacted socially with each other. This paper describes a task named"The Fishbowl Technique"and found to be effective in large ESL classes in the secondary level in the Philippines.展开更多
Traditionally, heuristic re-planning algorithms are used to tackle the problem of dynamic task planning for multiple satellites. However, the traditional heuristic strategies depend on the concrete tasks, which often ...Traditionally, heuristic re-planning algorithms are used to tackle the problem of dynamic task planning for multiple satellites. However, the traditional heuristic strategies depend on the concrete tasks, which often affect the result’s optimality. Noticing that the historical information of cooperative task planning will impact the latter planning results, we propose a hybrid learning algorithm for dynamic multi-satellite task planning, which is based on the multi-agent reinforcement learning of policy iteration and the transfer learning. The reinforcement learning strategy of each satellite is described with neural networks. The policy neural network individuals with the best topological structure and weights are found by applying co-evolutionary search iteratively. To avoid the failure of the historical learning caused by the randomly occurring observation requests, a novel approach is proposed to balance the quality and efficiency of the task planning, which converts the historical learning strategy to the current initial learning strategy by applying the transfer learning algorithm. The simulations and analysis show the feasibility and adaptability of the proposed approach especially for the situation with randomly occurring observation requests.展开更多
Blockchain technology,based on decentralized data storage and distributed consensus design,has become a promising solution to address data security risks and provide privacy protection in the Internet-of-Things(IoT)du...Blockchain technology,based on decentralized data storage and distributed consensus design,has become a promising solution to address data security risks and provide privacy protection in the Internet-of-Things(IoT)due to its tamper-proof and non-repudiation features.Although blockchain typically does not require the endorsement of third-party trust organizations,it mostly needs to perform necessary mathematical calculations to prevent malicious attacks,which results in stricter requirements for computation resources on the participating devices.By offloading the computation tasks required to support blockchain consensus to edge service nodes or the cloud,while providing data privacy protection for IoT applications,it can effectively address the limitations of computation and energy resources in IoT devices.However,how to make reasonable offloading decisions for IoT devices remains an open issue.Due to the excellent self-learning ability of Reinforcement Learning(RL),this paper proposes a RL enabled Swarm Intelligence Optimization Algorithm(RLSIOA)that aims to improve the quality of initial solutions and achieve efficient optimization of computation task offloading decisions.The algorithm considers various factors that may affect the revenue obtained by IoT devices executing consensus algorithms(e.g.,Proof-of-Work),it optimizes the proportion of sub-tasks to be offloaded and the scale of computing resources to be rented from the edge and cloud to maximize the revenue of devices.Experimental results show that RLSIOA can obtain higher-quality offloading decision-making schemes at lower latency costs compared to representative benchmark algorithms.展开更多
Porters play a crucial role in hospitals because they ensure the efficient transportation of patients,medical equipment,and vital documents.Despite its importance,there is a lack of research addressing the prediction ...Porters play a crucial role in hospitals because they ensure the efficient transportation of patients,medical equipment,and vital documents.Despite its importance,there is a lack of research addressing the prediction of completion times for porter tasks.To address this gap,we utilized real-world porter delivery data from Taiwan University Hospital,China,Yunlin Branch,Taiwan Region of China.We first identified key features that can influence the duration of porter tasks.We then employed three widely-used machine learning algorithms:decision tree,random forest,and gradient boosting.To leverage the strengths of each algorithm,we finally adopted an ensemble modeling approach that aggregates their individual predictions.Our experimental results show that the proposed ensemble model can achieve a mean absolute error of 3 min in predicting task response time and 4.42 min in task completion time.The prediction error is around 50%lower compared to using only the historical average.These results demonstrate that our method significantly improves the accuracy of porter task time prediction,supporting better resource planning and patient care.It helps ward staff streamline workflows by reducing delays,enables porter managers to allocate resources more effectively,and shortens patient waiting times,contributing to a better care experience.展开更多
The authors regret that there were errors in the affiliations and the funding declaration in the original published version.The affiliations a and b of the original manuscript are"School of Information Engineerin...The authors regret that there were errors in the affiliations and the funding declaration in the original published version.The affiliations a and b of the original manuscript are"School of Information Engineering,Jiangxi Provincial Key Laboratory of Advanced Signal Processing and Intelligent Communications,Nanchang University,Nanchang 330031,China",and"School of Internet of Things Engineering,Jiangnan University,Wuxi 214122,China",respectively.The order of the two affiliations are not correct.展开更多
Cooperative task assignment is one of the key research focuses in the field of unmanned aerial vehicles(UAVs). In this paper, an energy learning hyper-heuristic(EL-HH) algorithm is proposed to address the cooperative ...Cooperative task assignment is one of the key research focuses in the field of unmanned aerial vehicles(UAVs). In this paper, an energy learning hyper-heuristic(EL-HH) algorithm is proposed to address the cooperative task assignment problem of heterogeneous UAVs under complex constraints. First, a mathematical model is designed to define the scenario, complex constraints, and objective function of the problem. Then, the scheme encoding, the EL-HH strategy, multiple optimization operators, and the task sequence and time adjustment strategies are designed in the EL-HH algorithm. The scheme encoding is designed with three layers: task sequence, UAV sequence, and waiting time. The EL-HH strategy applies an energy learning method to adaptively adjust the energies of operators, thereby facilitating the selection and application of operators. Multiple optimization operators can update schemes in different ways, enabling the algorithm to fully explore the solution space. Afterward, the task order and time adjustment strategies are designed to adjust task order and insert waiting time. Through the iterative optimization process, a satisfactory assignment scheme is ultimately produced. Finally, simulation and experiment verify the effectiveness of the proposed algorithm.展开更多
Chronic uncontrolled inflammation is a major risk factor driving the occurrence of hepatocellular carcinoma(HCC),with over half of global cases attributed to hepatitis B virus(HBV)infection.Persistent inflammation fre...Chronic uncontrolled inflammation is a major risk factor driving the occurrence of hepatocellular carcinoma(HCC),with over half of global cases attributed to hepatitis B virus(HBV)infection.Persistent inflammation frequently progresses to cirrhosis and,ultimately,malignancy[1].Monitoring the key risk factors involved in the inflammatory-to-cancerous transformation in HCC is crucial for enabling timely intervention and improving patient survival rates.To address this challenge,we analyzed plasma samples collected from healthy volunteers and patients at various stages of HCC progression.展开更多
Dear Editor,Through distributed machine learning,multi-UAV systems can achieve global optimization goals without a centralized server,such as optimal target tracking,by leveraging local calculation and communication w...Dear Editor,Through distributed machine learning,multi-UAV systems can achieve global optimization goals without a centralized server,such as optimal target tracking,by leveraging local calculation and communication with neighbors.In this work,we implement the stochastic gradient descent algorithm(SGD)distributedly to optimize tracking errors based on local state and aggregation of the neighbors'estimation.However,Byzantine agents can mislead neighbors,causing deviations from optimal tracking.We prove that the swarm achieves resilient convergence if aggregated results lie within the normal neighbors'convex hull,which can be guaranteed by the introduced centerpoint-based aggregation rule.In the given simulated scenarios,distributed learning using average,geometric median(GM),and coordinate-wise median(CM)based aggregation rules fail to track the target.Compared to solely using the centerpoint aggregation method,our approach,which combines a pre-filter with the centroid aggregation rule,significantly enhances resilience against Byzantine attacks,achieving faster convergence and smaller tracking errors.展开更多
Introduction Human papillomavirus(HPV)vaccination is a cornerstone of cervical cancer prevention,particularly in low-and middle-income countries(LMICs),where the burden of disease remains high~1.The World Health Organ...Introduction Human papillomavirus(HPV)vaccination is a cornerstone of cervical cancer prevention,particularly in low-and middle-income countries(LMICs),where the burden of disease remains high~1.The World Health Organization(WHO)HPV Vaccine Introduction Clearing House reported that 147 countries(of 194 reporting)had fully introduced the HPV vaccine into their national schedules as of 20242.After COVID-19 pandemic disruptions,global coverage is again increasing.展开更多
The overall performance of multi-robot collaborative systems is significantly affected by the multi-robot task allocation.To improve the effectiveness,robustness,and safety of multi-robot collaborative systems,a multi...The overall performance of multi-robot collaborative systems is significantly affected by the multi-robot task allocation.To improve the effectiveness,robustness,and safety of multi-robot collaborative systems,a multimodal multi-objective evolutionary algorithm based on deep reinforcement learning is proposed in this paper.The improved multimodal multi-objective evolutionary algorithm is used to solve multi-robot task allo-cation problems.Moreover,a deep reinforcement learning strategy is used in the last generation to provide a high-quality path for each assigned robot via an end-to-end manner.Comparisons with three popular multimodal multi-objective evolutionary algorithms on three different scenarios of multi-robot task allocation problems are carried out to verify the performance of the proposed algorithm.The experimental test results show that the proposed algorithm can generate sufficient equivalent schemes to improve the availability and robustness of multi-robot collaborative systems in uncertain environments,and also produce the best scheme to improve the overall task execution efficiency of multi-robot collaborative systems.展开更多
The scale of ground-to-air confrontation task assignments is large and needs to deal with many concurrent task assignments and random events.Aiming at the problems where existing task assignment methods are applied to...The scale of ground-to-air confrontation task assignments is large and needs to deal with many concurrent task assignments and random events.Aiming at the problems where existing task assignment methods are applied to ground-to-air confrontation,there is low efficiency in dealing with complex tasks,and there are interactive conflicts in multiagent systems.This study proposes a multiagent architecture based on a one-general agent with multiple narrow agents(OGMN)to reduce task assignment conflicts.Considering the slow speed of traditional dynamic task assignment algorithms,this paper proposes the proximal policy optimization for task assignment of general and narrow agents(PPOTAGNA)algorithm.The algorithm based on the idea of the optimal assignment strategy algorithm and combined with the training framework of deep reinforcement learning(DRL)adds a multihead attention mechanism and a stage reward mechanism to the bilateral band clipping PPO algorithm to solve the problem of low training efficiency.Finally,simulation experiments are carried out in the digital battlefield.The multiagent architecture based on OGMN combined with the PPO-TAGNA algorithm can obtain higher rewards faster and has a higher win ratio.By analyzing agent behavior,the efficiency,superiority and rationality of resource utilization of this method are verified.展开更多
The Multi-access Edge Cloud(MEC) networks extend cloud computing services and capabilities to the edge of the networks. By bringing computation and storage capabilities closer to end-users and connected devices, MEC n...The Multi-access Edge Cloud(MEC) networks extend cloud computing services and capabilities to the edge of the networks. By bringing computation and storage capabilities closer to end-users and connected devices, MEC networks can support a wide range of applications. MEC networks can also leverage various types of resources, including computation resources, network resources, radio resources,and location-based resources, to provide multidimensional resources for intelligent applications in 5/6G.However, tasks generated by users often consist of multiple subtasks that require different types of resources. It is a challenging problem to offload multiresource task requests to the edge cloud aiming at maximizing benefits due to the heterogeneity of resources provided by devices. To address this issue,we mathematically model the task requests with multiple subtasks. Then, the problem of task offloading of multi-resource task requests is proved to be NP-hard. Furthermore, we propose a novel Dual-Agent Deep Reinforcement Learning algorithm with Node First and Link features(NF_L_DA_DRL) based on the policy network, to optimize the benefits generated by offloading multi-resource task requests in MEC networks. Finally, simulation results show that the proposed algorithm can effectively improve the benefit of task offloading with higher resource utilization compared with baseline algorithms.展开更多
To improve the quality of computation experience for mobile devices,mobile edge computing(MEC)is a promising paradigm by providing computing capabilities in close proximity within a sliced radio access network,which s...To improve the quality of computation experience for mobile devices,mobile edge computing(MEC)is a promising paradigm by providing computing capabilities in close proximity within a sliced radio access network,which supports both traditional communication and MEC services.However,this kind of intensive computing problem is a high dimensional NP hard problem,and some machine learning methods do not have a good effect on solving this problem.In this paper,the Markov decision process model is established to find the excellent task offloading scheme,which maximizes the long-term utility performance,so as to make the best offloading decision according to the queue state,energy queue state and channel quality between mobile users and BS.In order to explore the curse of high dimension in state space,a candidate network is proposed based on edge computing optimize offloading(ECOO)algorithm with the application of deep deterministic policy gradient algorithm.Through simulation experiments,it is proved that the ECOO algorithm is superior to some deep reinforcement learning algorithms in terms of energy consumption and time delay.So the ECOO is good at dealing with high dimensional problems.展开更多
文摘With the advent of sixth-generation mobile communications(6G),space-air-ground integrated networks have become mainstream.This paper focuses on collaborative scheduling for mobile edge computing(MEC)under a three-tier heterogeneous architecture composed of mobile devices,unmanned aerial vehicles(UAVs),and macro base stations(BSs).This scenario typically faces fast channel fading,dynamic computational loads,and energy constraints,whereas classical queuing-theoretic or convex-optimization approaches struggle to yield robust solutions in highly dynamic settings.To address this issue,we formulate a multi-agent Markov decision process(MDP)for an air-ground-fused MEC system,unify link selection,bandwidth/power allocation,and task offloading into a continuous action space and propose a joint scheduling strategy that is based on an improved MATD3 algorithm.The improvements include Alternating Layer Normalization(ALN)in the actor to suppress gradient variance,Residual Orthogonalization(RO)in the critic to reduce the correlation between the twin Q-value estimates,and a dynamic-temperature reward to enable adaptive trade-offs during training.On a multi-user,dual-link simulation platform,we conduct ablation and baseline comparisons.The results reveal that the proposed method has better convergence and stability.Compared with MADDPG,TD3,and DSAC,our algorithm achieves more robust performance across key metrics.
基金National Key Research and Development Program of China,Grant/Award Number:2021YFB1714700Postdoctoral Research Foundation of China,Grant/Award Number:2024M752364Postdoctoral Fellowship Program of CPSF,Grant/Award Number:GZB20240525。
文摘This paper focuses on the problem of multi-station multi-robot spot welding task assignment,and proposes a deep reinforcement learning(DRL)framework,which is made up of a public graph attention network and independent policy networks.The graph of welding spots distribution is encoded using the graph attention network.Independent policy networks with attention mechanism as a decoder can handle the encoded graph and decide to assign robots to different tasks.The policy network is used to convert the large scale welding spots allocation problem to multiple small scale singlerobot welding path planning problems,and the path planning problem is quickly solved through existing methods.Then,the model is trained through reinforcement learning.In addition,the task balancing method is used to allocate tasks to multiple stations.The proposed algorithm is compared with classical algorithms,and the results show that the algorithm based on DRL can produce higher quality solutions.
基金funded by the National Key Research and Development Program of China under Grant 2019YFB1803301Beijing Natural Science Foundation (L202002)。
文摘Cybertwin-enabled 6th Generation(6G)network is envisioned to support artificial intelligence-native management to meet changing demands of 6G applications.Multi-Agent Deep Reinforcement Learning(MADRL)technologies driven by Cybertwins have been proposed for adaptive task offloading strategies.However,the existence of random transmission delay between Cybertwin-driven agents and underlying networks is not considered in related works,which destroys the standard Markov property and increases the decision reaction time to reduce the task offloading strategy performance.In order to address this problem,we propose a pipelining task offloading method to lower the decision reaction time and model it as a delay-aware Markov Decision Process(MDP).Then,we design a delay-aware MADRL algorithm to minimize the weighted sum of task execution latency and energy consumption.Firstly,the state space is augmented using the lastly-received state and historical actions to rebuild the Markov property.Secondly,Gate Transformer-XL is introduced to capture historical actions'importance and maintain the consistent input dimension dynamically changed due to random transmission delays.Thirdly,a sampling method and a new loss function with the difference between the current and target state value and the difference between real state-action value and augmented state-action value are designed to obtain state transition trajectories close to the real ones.Numerical results demonstrate that the proposed methods are effective in reducing reaction time and improving the task offloading performance in the random-delay Cybertwin-enabled 6G networks.
基金supported by the National Natural Science Foundation of China(62202215)Liaoning Province Applied Basic Research Program(Youth Special Project,2023JH2/101600038)+2 种基金Shenyang Youth Science and Technology Innovation Talent Support Program(RC220458)Guangxuan Program of Shenyang Ligong University(SYLUGXRC202216)the Basic Research Special Funds for Undergraduate Universities in Liaoning Province(LJ212410144067).
文摘The advent of the internet-of-everything era has led to the increased use of mobile edge computing.The rise of artificial intelligence has provided many possibilities for the low-latency task-offloading demands of users,but existing technologies rigidly assume that there is only one task to be offloaded in each time slot at the terminal.In practical scenarios,there are often numerous computing tasks to be executed at the terminal,leading to a cumulative delay for subsequent task offloading.Therefore,the efficient processing of multiple computing tasks on the terminal has become highly challenging.To address the lowlatency offloading requirements for multiple computational tasks on terminal devices,we propose a terminal multitask parallel offloading algorithm based on deep reinforcement learning.Specifically,we first establish a mobile edge computing system model consisting of a single edge server and multiple terminal users.We then model the task offloading decision problem as a Markov decision process,and solve this problem using the Dueling Deep-Q Network algorithm to obtain the optimal offloading strategy.Experimental results demonstrate that,under the same constraints,our proposed algorithm reduces the average system latency.
基金supported by National Natural Science Foundation of China under Grant No.62372110Fujian Provincial Natural Science of Foundation under Grants 2023J02008,2024H0009.
文摘The rapid advancement of Industry 4.0 has revolutionized manufacturing,shifting production from centralized control to decentralized,intelligent systems.Smart factories are now expected to achieve high adaptability and resource efficiency,particularly in mass customization scenarios where production schedules must accommodate dynamic and personalized demands.To address the challenges of dynamic task allocation,uncertainty,and realtime decision-making,this paper proposes Pathfinder,a deep reinforcement learning-based scheduling framework.Pathfinder models scheduling data through three key matrices:execution time(the time required for a job to complete),completion time(the actual time at which a job is finished),and efficiency(the performance of executing a single job).By leveraging neural networks,Pathfinder extracts essential features from these matrices,enabling intelligent decision-making in dynamic production environments.Unlike traditional approaches with fixed scheduling rules,Pathfinder dynamically selects from ten diverse scheduling rules,optimizing decisions based on real-time environmental conditions.To further enhance scheduling efficiency,a specialized reward function is designed to support dynamic task allocation and real-time adjustments.This function helps Pathfinder continuously refine its scheduling strategy,improving machine utilization and minimizing job completion times.Through reinforcement learning,Pathfinder adapts to evolving production demands,ensuring robust performance in real-world applications.Experimental results demonstrate that Pathfinder outperforms traditional scheduling approaches,offering improved coordination and efficiency in smart factories.By integrating deep reinforcement learning,adaptable scheduling strategies,and an innovative reward function,Pathfinder provides an effective solution to the growing challenges of multi-robot job scheduling in mass customization environments.
文摘With focus now placed on the learner, more attention is given to his learning style, multiple intelligence and developing learning strategies to enable him to make sense of and use of the target language appropriately in varied contexts and with different uses of the language. To attain this, the teacher is tasked with designing, monitoring and processing language learning activities for students to carry out and in the process learn by doing and reflecting on the learning process they went through as they interacted socially with each other. This paper describes a task named"The Fishbowl Technique"and found to be effective in large ESL classes in the secondary level in the Philippines.
文摘Traditionally, heuristic re-planning algorithms are used to tackle the problem of dynamic task planning for multiple satellites. However, the traditional heuristic strategies depend on the concrete tasks, which often affect the result’s optimality. Noticing that the historical information of cooperative task planning will impact the latter planning results, we propose a hybrid learning algorithm for dynamic multi-satellite task planning, which is based on the multi-agent reinforcement learning of policy iteration and the transfer learning. The reinforcement learning strategy of each satellite is described with neural networks. The policy neural network individuals with the best topological structure and weights are found by applying co-evolutionary search iteratively. To avoid the failure of the historical learning caused by the randomly occurring observation requests, a novel approach is proposed to balance the quality and efficiency of the task planning, which converts the historical learning strategy to the current initial learning strategy by applying the transfer learning algorithm. The simulations and analysis show the feasibility and adaptability of the proposed approach especially for the situation with randomly occurring observation requests.
基金supported by the Project of Science and Technology Research Program of Chongqing Education Commission of China(No.KJZD-K202401105)High-Quality Development Action Plan for Graduate Education at Chongqing University of Technology(No.gzljg2023308,No.gzljd2024204)+1 种基金the Graduate Innovation Program of Chongqing University of Technology(No.gzlcx20233197)Yunnan Provincial Key R&D Program(202203AA080006).
文摘Blockchain technology,based on decentralized data storage and distributed consensus design,has become a promising solution to address data security risks and provide privacy protection in the Internet-of-Things(IoT)due to its tamper-proof and non-repudiation features.Although blockchain typically does not require the endorsement of third-party trust organizations,it mostly needs to perform necessary mathematical calculations to prevent malicious attacks,which results in stricter requirements for computation resources on the participating devices.By offloading the computation tasks required to support blockchain consensus to edge service nodes or the cloud,while providing data privacy protection for IoT applications,it can effectively address the limitations of computation and energy resources in IoT devices.However,how to make reasonable offloading decisions for IoT devices remains an open issue.Due to the excellent self-learning ability of Reinforcement Learning(RL),this paper proposes a RL enabled Swarm Intelligence Optimization Algorithm(RLSIOA)that aims to improve the quality of initial solutions and achieve efficient optimization of computation task offloading decisions.The algorithm considers various factors that may affect the revenue obtained by IoT devices executing consensus algorithms(e.g.,Proof-of-Work),it optimizes the proportion of sub-tasks to be offloaded and the scale of computing resources to be rented from the edge and cloud to maximize the revenue of devices.Experimental results show that RLSIOA can obtain higher-quality offloading decision-making schemes at lower latency costs compared to representative benchmark algorithms.
基金supported by National Taiwan University Hospital Yunlin Branch Project NTUHYL 110.C018National Science and Technology Council,Taiwan.
文摘Porters play a crucial role in hospitals because they ensure the efficient transportation of patients,medical equipment,and vital documents.Despite its importance,there is a lack of research addressing the prediction of completion times for porter tasks.To address this gap,we utilized real-world porter delivery data from Taiwan University Hospital,China,Yunlin Branch,Taiwan Region of China.We first identified key features that can influence the duration of porter tasks.We then employed three widely-used machine learning algorithms:decision tree,random forest,and gradient boosting.To leverage the strengths of each algorithm,we finally adopted an ensemble modeling approach that aggregates their individual predictions.Our experimental results show that the proposed ensemble model can achieve a mean absolute error of 3 min in predicting task response time and 4.42 min in task completion time.The prediction error is around 50%lower compared to using only the historical average.These results demonstrate that our method significantly improves the accuracy of porter task time prediction,supporting better resource planning and patient care.It helps ward staff streamline workflows by reducing delays,enables porter managers to allocate resources more effectively,and shortens patient waiting times,contributing to a better care experience.
文摘The authors regret that there were errors in the affiliations and the funding declaration in the original published version.The affiliations a and b of the original manuscript are"School of Information Engineering,Jiangxi Provincial Key Laboratory of Advanced Signal Processing and Intelligent Communications,Nanchang University,Nanchang 330031,China",and"School of Internet of Things Engineering,Jiangnan University,Wuxi 214122,China",respectively.The order of the two affiliations are not correct.
基金funded by the National Natural Science Foundation of China (Grant No.62203217)the Jiangsu Province Basic Research Program Natural Science Foundation (Grant No.BK20220885)+3 种基金the Hong Kong,Macao and Taiwan Science and Technology Cooperation Project of Special Foundation in Jiangsu Science and Technology Plan (Grant No.BZ2023057)the Fundamental Research Funds for the Central Universities (Grant No.NJ2024012)the China Postdoctoral Science Foundation (Grant No.GZC20242230)the Postgraduate Research & Practice Innovation Program of Jiangsu Province (Grant No.KYCX24_0586)。
文摘Cooperative task assignment is one of the key research focuses in the field of unmanned aerial vehicles(UAVs). In this paper, an energy learning hyper-heuristic(EL-HH) algorithm is proposed to address the cooperative task assignment problem of heterogeneous UAVs under complex constraints. First, a mathematical model is designed to define the scenario, complex constraints, and objective function of the problem. Then, the scheme encoding, the EL-HH strategy, multiple optimization operators, and the task sequence and time adjustment strategies are designed in the EL-HH algorithm. The scheme encoding is designed with three layers: task sequence, UAV sequence, and waiting time. The EL-HH strategy applies an energy learning method to adaptively adjust the energies of operators, thereby facilitating the selection and application of operators. Multiple optimization operators can update schemes in different ways, enabling the algorithm to fully explore the solution space. Afterward, the task order and time adjustment strategies are designed to adjust task order and insert waiting time. Through the iterative optimization process, a satisfactory assignment scheme is ultimately produced. Finally, simulation and experiment verify the effectiveness of the proposed algorithm.
基金supported by Liaoning Province Science and Technology Plan Project,China(Grant No.:2022JH2/101300038)Liaoning Provincial Applied Basic Research Project,China(Grant No.:2022020255-JH2/1013)+5 种基金the National Natural Science Foundation of China of China(Grant Nos.:82104379/H3203 and 82104126/H3410)Liaoning Distinguished Professor Project(2017)Liaoning BaiQianWan Talents Program in 2019(A-37)China,Liaoning key Research and Development Program(2018),China,the Natural Science Funds of Liaoning Province,China(Grant No.:2021JH2/10300068)the Basic Scientific Research Project of Higher Education Department of Liaoning Province,China(Grant No.:LJ212410163036)Educational Commission of Liaoning Province of China,China(Grant No.:LJ212410163026).
文摘Chronic uncontrolled inflammation is a major risk factor driving the occurrence of hepatocellular carcinoma(HCC),with over half of global cases attributed to hepatitis B virus(HBV)infection.Persistent inflammation frequently progresses to cirrhosis and,ultimately,malignancy[1].Monitoring the key risk factors involved in the inflammatory-to-cancerous transformation in HCC is crucial for enabling timely intervention and improving patient survival rates.To address this challenge,we analyzed plasma samples collected from healthy volunteers and patients at various stages of HCC progression.
基金supported By Guangdong Major Project of Basic and Applied Basic Research(2023B0303000009)Guangdong Basic and Applied Basic Research Foundation(2024A1515030153,2025A1515011587)+1 种基金Project of Department of Education of Guangdong Province(2023ZDZX4046)Shen-zhen Natural Science Fund(Stable Support Plan Program 20231122121608001),Ningbo Municipal Science and Technology Bureau(ZX2024000604).
文摘Dear Editor,Through distributed machine learning,multi-UAV systems can achieve global optimization goals without a centralized server,such as optimal target tracking,by leveraging local calculation and communication with neighbors.In this work,we implement the stochastic gradient descent algorithm(SGD)distributedly to optimize tracking errors based on local state and aggregation of the neighbors'estimation.However,Byzantine agents can mislead neighbors,causing deviations from optimal tracking.We prove that the swarm achieves resilient convergence if aggregated results lie within the normal neighbors'convex hull,which can be guaranteed by the introduced centerpoint-based aggregation rule.In the given simulated scenarios,distributed learning using average,geometric median(GM),and coordinate-wise median(CM)based aggregation rules fail to track the target.Compared to solely using the centerpoint aggregation method,our approach,which combines a pre-filter with the centroid aggregation rule,significantly enhances resilience against Byzantine attacks,achieving faster convergence and smaller tracking errors.
文摘Introduction Human papillomavirus(HPV)vaccination is a cornerstone of cervical cancer prevention,particularly in low-and middle-income countries(LMICs),where the burden of disease remains high~1.The World Health Organization(WHO)HPV Vaccine Introduction Clearing House reported that 147 countries(of 194 reporting)had fully introduced the HPV vaccine into their national schedules as of 20242.After COVID-19 pandemic disruptions,global coverage is again increasing.
基金the Shanghai Pujiang Program (No.22PJD030),the National Natural Science Foundation of China (Nos.61603244 and 71904116)the National Natural Science Foundation of China-Shandong Joint Fund (No.U2006228)。
文摘The overall performance of multi-robot collaborative systems is significantly affected by the multi-robot task allocation.To improve the effectiveness,robustness,and safety of multi-robot collaborative systems,a multimodal multi-objective evolutionary algorithm based on deep reinforcement learning is proposed in this paper.The improved multimodal multi-objective evolutionary algorithm is used to solve multi-robot task allo-cation problems.Moreover,a deep reinforcement learning strategy is used in the last generation to provide a high-quality path for each assigned robot via an end-to-end manner.Comparisons with three popular multimodal multi-objective evolutionary algorithms on three different scenarios of multi-robot task allocation problems are carried out to verify the performance of the proposed algorithm.The experimental test results show that the proposed algorithm can generate sufficient equivalent schemes to improve the availability and robustness of multi-robot collaborative systems in uncertain environments,and also produce the best scheme to improve the overall task execution efficiency of multi-robot collaborative systems.
基金the Project of National Natural Science Foundation of China(Grant No.62106283)the Project of National Natural Science Foundation of China(Grant No.72001214)to provide fund for conducting experimentsthe Project of Natural Science Foundation of Shaanxi Province(Grant No.2020JQ-484)。
文摘The scale of ground-to-air confrontation task assignments is large and needs to deal with many concurrent task assignments and random events.Aiming at the problems where existing task assignment methods are applied to ground-to-air confrontation,there is low efficiency in dealing with complex tasks,and there are interactive conflicts in multiagent systems.This study proposes a multiagent architecture based on a one-general agent with multiple narrow agents(OGMN)to reduce task assignment conflicts.Considering the slow speed of traditional dynamic task assignment algorithms,this paper proposes the proximal policy optimization for task assignment of general and narrow agents(PPOTAGNA)algorithm.The algorithm based on the idea of the optimal assignment strategy algorithm and combined with the training framework of deep reinforcement learning(DRL)adds a multihead attention mechanism and a stage reward mechanism to the bilateral band clipping PPO algorithm to solve the problem of low training efficiency.Finally,simulation experiments are carried out in the digital battlefield.The multiagent architecture based on OGMN combined with the PPO-TAGNA algorithm can obtain higher rewards faster and has a higher win ratio.By analyzing agent behavior,the efficiency,superiority and rationality of resource utilization of this method are verified.
基金Supported by National Natural Science Foundation of China（60474035）,National Research Foundation for the Doctoral Program of Higher Education of China（20050359004）,Natural Science Foundation of Anhui Province（070412035）
基金Manuscript received March 5, 2010 accepted March 2, 2011 Supported by National Natural Science Foundation of China （61004103）, National Research Foundation for the Doctoral Program of Higher Education of China （20100111110005）, China Postdoctoral Science Foundation （20090460742）, and Natural Science Foundation of Anhui Province of China （090412058, 11040606Q44）
基金supported in part by the National Natural Science Foundation of China under Grants 62201105,62331017,and 62075024in part by the Natural Science Foundation of Chongqing under Grant cstc2021jcyj-msxmX0404+1 种基金in part by the Chongqing Municipal Education Commission under Grant KJQN202100643in part by Guangdong Basic and Applied Basic Research Foundation under Grant 2022A1515110056.
文摘The Multi-access Edge Cloud(MEC) networks extend cloud computing services and capabilities to the edge of the networks. By bringing computation and storage capabilities closer to end-users and connected devices, MEC networks can support a wide range of applications. MEC networks can also leverage various types of resources, including computation resources, network resources, radio resources,and location-based resources, to provide multidimensional resources for intelligent applications in 5/6G.However, tasks generated by users often consist of multiple subtasks that require different types of resources. It is a challenging problem to offload multiresource task requests to the edge cloud aiming at maximizing benefits due to the heterogeneity of resources provided by devices. To address this issue,we mathematically model the task requests with multiple subtasks. Then, the problem of task offloading of multi-resource task requests is proved to be NP-hard. Furthermore, we propose a novel Dual-Agent Deep Reinforcement Learning algorithm with Node First and Link features(NF_L_DA_DRL) based on the policy network, to optimize the benefits generated by offloading multi-resource task requests in MEC networks. Finally, simulation results show that the proposed algorithm can effectively improve the benefit of task offloading with higher resource utilization compared with baseline algorithms.
基金National Natural Science Foundation of China(No.11461038)Science and Technology Support Program of Gansu Province(No.144NKCA040)。
文摘To improve the quality of computation experience for mobile devices,mobile edge computing(MEC)is a promising paradigm by providing computing capabilities in close proximity within a sliced radio access network,which supports both traditional communication and MEC services.However,this kind of intensive computing problem is a high dimensional NP hard problem,and some machine learning methods do not have a good effect on solving this problem.In this paper,the Markov decision process model is established to find the excellent task offloading scheme,which maximizes the long-term utility performance,so as to make the best offloading decision according to the queue state,energy queue state and channel quality between mobile users and BS.In order to explore the curse of high dimension in state space,a candidate network is proposed based on edge computing optimize offloading(ECOO)algorithm with the application of deep deterministic policy gradient algorithm.Through simulation experiments,it is proved that the ECOO algorithm is superior to some deep reinforcement learning algorithms in terms of energy consumption and time delay.So the ECOO is good at dealing with high dimensional problems.
