Container is an emerging virtualization technology and widely adopted in the cloud to provide services because of its lightweight,flexible,isolated and highly portable properties.Cloud services are often instantiated ...Container is an emerging virtualization technology and widely adopted in the cloud to provide services because of its lightweight,flexible,isolated and highly portable properties.Cloud services are often instantiated as clusters of interconnected containers.Due to the stochastic service arrival and complicated cloud environment,it is challenging to achieve an optimal container placement(CP)scheme.We propose to leverage Deep Reinforcement Learning(DRL)for solving CP problem,which is able to learn from experience interacting with the environment and does not rely on mathematical model or prior knowledge.However,applying DRL method directly dose not lead to a satisfying result because of sophisticated environment states and huge action spaces.In this paper,we propose UNREAL-CP,a DRL-based method to place container instances on servers while considering end to end delay and resource utilization cost.The proposed method is an actor-critic-based approach,which has advantages in dealing with the huge action space.Moreover,the idea of auxiliary learning is also included in our architecture.We design two auxiliary learning tasks about load balancing to improve algorithm performance.Compared to other DRL methods,extensive simulation results show that UNREAL-CP performs better up to 28.6%in terms of reducing delay and deployment cost with high training efficiency and responding speed.展开更多
The historical interaction sequences of users play a crucial role in training recommender systems that can accurately predict user preferences.However,due to the arbitrariness of user behaviors,the presence of noise i...The historical interaction sequences of users play a crucial role in training recommender systems that can accurately predict user preferences.However,due to the arbitrariness of user behaviors,the presence of noise in these sequences poses a challenge to predicting their next actions in recommender systems.To address this issue,our motivation is based on the observation that training noisy sequences and clean sequences(sequences without noise)with equal weights can impact the performance of the model.We propose the novel self-supervised Auxiliary Task Joint Training(ATJT)method aimed at more accurately reweighting noisy sequences in recommender systems.Specifically,we strategically select subsets from users’original sequences and perform random replacements to generate artificially replaced noisy sequences.Subsequently,we perform joint training on these artificially replaced noisy sequences and the original sequences.Through effective reweighting,we incorporate the training results of the noise recognition model into the recommender model.We evaluate our method on three datasets using a consistent base model.Experimental results demonstrate the effectiveness of introducing the self-supervised auxiliary task to enhance the base model’s performance.展开更多
基金This work is supported by the National Natural Science Foundation of China(61702048)the Public Support Platform Construction of Industrial Internet platform.
文摘Container is an emerging virtualization technology and widely adopted in the cloud to provide services because of its lightweight,flexible,isolated and highly portable properties.Cloud services are often instantiated as clusters of interconnected containers.Due to the stochastic service arrival and complicated cloud environment,it is challenging to achieve an optimal container placement(CP)scheme.We propose to leverage Deep Reinforcement Learning(DRL)for solving CP problem,which is able to learn from experience interacting with the environment and does not rely on mathematical model or prior knowledge.However,applying DRL method directly dose not lead to a satisfying result because of sophisticated environment states and huge action spaces.In this paper,we propose UNREAL-CP,a DRL-based method to place container instances on servers while considering end to end delay and resource utilization cost.The proposed method is an actor-critic-based approach,which has advantages in dealing with the huge action space.Moreover,the idea of auxiliary learning is also included in our architecture.We design two auxiliary learning tasks about load balancing to improve algorithm performance.Compared to other DRL methods,extensive simulation results show that UNREAL-CP performs better up to 28.6%in terms of reducing delay and deployment cost with high training efficiency and responding speed.
基金supported by the Program for Student Innovation Through Research and Training of Guizhou University under Grant No.2023SRT071.
文摘The historical interaction sequences of users play a crucial role in training recommender systems that can accurately predict user preferences.However,due to the arbitrariness of user behaviors,the presence of noise in these sequences poses a challenge to predicting their next actions in recommender systems.To address this issue,our motivation is based on the observation that training noisy sequences and clean sequences(sequences without noise)with equal weights can impact the performance of the model.We propose the novel self-supervised Auxiliary Task Joint Training(ATJT)method aimed at more accurately reweighting noisy sequences in recommender systems.Specifically,we strategically select subsets from users’original sequences and perform random replacements to generate artificially replaced noisy sequences.Subsequently,we perform joint training on these artificially replaced noisy sequences and the original sequences.Through effective reweighting,we incorporate the training results of the noise recognition model into the recommender model.We evaluate our method on three datasets using a consistent base model.Experimental results demonstrate the effectiveness of introducing the self-supervised auxiliary task to enhance the base model’s performance.
