Deep learning(DL)has been applied to the physical layer of wireless communication systems,which directly extracts environment knowledge from data and outperforms conventional methods either in accuracy or computation ...Deep learning(DL)has been applied to the physical layer of wireless communication systems,which directly extracts environment knowledge from data and outperforms conventional methods either in accuracy or computation complexity.However,most related research works employ centralized training that inevitably involves collecting training data from edge devices.The data uploading process usually results in excessive communication overhead and privacy disclosure.Alternatively,a distributed learning approach named federated edge learning(FEEL)is introduced to physical layer designs.In FEEL,all devices collaborate to train a global model only by exchanging parameters with a nearby access point.Because all datasets are kept local,data privacy is better protected and data transmission overhead can be reduced.This paper reviews the studies on applying FEEL to the wireless physical layer including channel state information acquisition,transmitter,and receiver design,which represent a paradigm shift of the DL-based physical layer design.In the meantime they also reveal several limitations inherent in FEEL,particularly when applied to the wireless physical layer,thus motivating further research efforts in the field.展开更多
By periodically aggregating local learning updates from edge users, federated edge learning (FEEL) is envisioned as a promising means to reap the benefit of local rich da?ta and protect users'privacy. However, the...By periodically aggregating local learning updates from edge users, federated edge learning (FEEL) is envisioned as a promising means to reap the benefit of local rich da?ta and protect users'privacy. However, the scarce wireless communication resource greatly limits the number of participated users and is regarded as the main bottleneck which hin?ders the development of FEEL. To tackle this issue, we propose a user selection policy based on data importance for FEEL system. In order to quantify the data importance of each user, we first analyze the relationship between the loss decay and the squared norm of gradi?ent. Then, we formulate a combinatorial optimization problem to maximize the learning effi?ciency by jointly considering user selection and communication resource allocation. By problem transformation and relaxation, the optimal user selection policy and resource alloca?tion are derived, and a polynomial-time optimal algorithm is developed. Finally, we deploy two commonly used deep neural network (DNN) models for simulation. The results validate that our proposed algorithm has strong generalization ability and can attain higher learning efficiency compared with other traditional algorithms.展开更多
Federated Edge Learning(FEL),an emerging distributed Machine Learning(ML)paradigm,enables model training in a distributed environment while ensuring user privacy by using physical separation for each user’s data.Howe...Federated Edge Learning(FEL),an emerging distributed Machine Learning(ML)paradigm,enables model training in a distributed environment while ensuring user privacy by using physical separation for each user’s data.However,with the development of complex application scenarios such as the Internet of Things(IoT)and Smart Earth,the conventional resource allocation schemes can no longer effectively support these growing computational and communication demands.Therefore,joint resource optimization may be the key solution to the scaling problem.This paper simultaneously addresses the multifaceted challenges of computation and communication,with the growing multiple resource demands.We systematically review the joint allocation strategies for different resources(computation,data,communication,and network topology)in FEL,and summarize the advantages in improving system efficiency,reducing latency,enhancing resource utilization,and enhancing robustness.In addition,we present the potential ability of joint optimization to enhance privacy preservation by reducing communication requirements,indirectly.This work not only provides theoretical support for resource management in federated learning(FL)systems,but also provides ideas for potential optimal deployment in multiple real-world scenarios.By thoroughly discussing the current challenges and future research directions,it also provides some important insights into multi-resource optimization in complex application environments.展开更多
As a popular distributed machine learning framework,wireless federated edge learning(FEEL)can keep original data local,while uploading model training updates to protect privacy and prevent data silos.However,since wir...As a popular distributed machine learning framework,wireless federated edge learning(FEEL)can keep original data local,while uploading model training updates to protect privacy and prevent data silos.However,since wireless channels are usually unreliable,there is no guarantee that the model updates uploaded by local devices are correct,thus greatly degrading the performance of the wireless FEEL.Conventional retransmission schemes designed for wireless systems generally aim to maximize the system throughput or minimize the packet error rate,which is not suitable for the FEEL system.A novel retransmission scheme is proposed for the FEEL system to make a tradeoff between model training accuracy and retransmission latency.In the proposed scheme,a retransmission device selection criterion is first designed based on the channel condition,the number of local data,and the importance of model updates.In addition,we design the air interface signaling under this retransmission scheme to facilitate the implementation of the proposed scheme in practical scenarios.Finally,the effectiveness of the proposed retransmission scheme is validated through simulation experiments.展开更多
This paper studies a federated edge learning system,in which an edge server coordinates a set of edge devices to train a shared machine learning(ML)model based on their locally distributed data samples.During the dist...This paper studies a federated edge learning system,in which an edge server coordinates a set of edge devices to train a shared machine learning(ML)model based on their locally distributed data samples.During the distributed training,we exploit the joint communication and computation design for improving the system energy efficiency,in which both the communication resource allocation for global ML-parameters aggregation and the computation resource allocation for locally updating ML-parameters are jointly optimized.In particular,we consider two transmission protocols for edge devices to upload ML-parameters to edge server,based on the non-orthogonal multiple access(NOMA)and time division multiple access(TDMA),respectively.Under both protocols,we minimize the total energy consumption at all edge devices over a particular finite training duration subject to a given training accuracy,by jointly optimizing the transmission power and rates at edge devices for uploading ML-parameters and their central processing unit(CPU)frequencies for local update.We propose efficient algorithms to solve the formulated energy minimization problems by using the techniques from convex optimization.Numerical results show that as compared to other benchmark schemes,our proposed joint communication and computation design significantly can improve the energy efficiency of the federated edge learning system,by properly balancing the energy tradeoff between communication and computation.展开更多
Training a machine learning model with federated edge learning(FEEL)is typically time consuming due to the constrained computation power of edge devices and the limited wireless resources in edge networks.In this stud...Training a machine learning model with federated edge learning(FEEL)is typically time consuming due to the constrained computation power of edge devices and the limited wireless resources in edge networks.In this study,the training time minimization problem is investigated in a quantized FEEL system,where heterogeneous edge devices send quantized gradients to the edge server via orthogonal channels.In particular,a stochastic quantization scheme is adopted for compression of uploaded gradients,which can reduce the burden of per-round communication but may come at the cost of increasing the number of communication rounds.The training time is modeled by taking into account the communication time,computation time,and the number of communication rounds.Based on the proposed training time model,the intrinsic trade-off between the number of communication rounds and per-round latency is characterized.Specifically,we analyze the convergence behavior of the quantized FEEL in terms of the optimality gap.Furthermore,a joint data-and-model-driven fitting method is proposed to obtain the exact optimality gap,based on which the closed-form expressions for the number of communication rounds and the total training time are obtained.Constrained by the total bandwidth,the training time minimization problem is formulated as a joint quantization level and bandwidth allocation optimization problem.To this end,an algorithm based on alternating optimization is proposed,which alternatively solves the subproblem of quantization optimization through successive convex approximation and the subproblem of bandwidth allocation by bisection search.With different learning tasks and models,the validation of our analysis and the near-optimal performance of the proposed optimization algorithm are demonstrated by the simulation results.展开更多
基金supported by the National Natural Science Foundation of China (NSFC) under Grants 61941104,61921004the Key Research and Development Program of Shandong Province under Grant 2020CXGC010108+1 种基金the Fundamental Research Funds for the Central Universities 2242022k30005supported in part by the Research Fund of the National Mobile Communications Research Laboratory,Southeast University。
文摘Deep learning(DL)has been applied to the physical layer of wireless communication systems,which directly extracts environment knowledge from data and outperforms conventional methods either in accuracy or computation complexity.However,most related research works employ centralized training that inevitably involves collecting training data from edge devices.The data uploading process usually results in excessive communication overhead and privacy disclosure.Alternatively,a distributed learning approach named federated edge learning(FEEL)is introduced to physical layer designs.In FEEL,all devices collaborate to train a global model only by exchanging parameters with a nearby access point.Because all datasets are kept local,data privacy is better protected and data transmission overhead can be reduced.This paper reviews the studies on applying FEEL to the wireless physical layer including channel state information acquisition,transmitter,and receiver design,which represent a paradigm shift of the DL-based physical layer design.In the meantime they also reveal several limitations inherent in FEEL,particularly when applied to the wireless physical layer,thus motivating further research efforts in the field.
基金This work was supported in part by the National Natural Science Founda⁃tion of China under Grant No.61671407.
文摘By periodically aggregating local learning updates from edge users, federated edge learning (FEEL) is envisioned as a promising means to reap the benefit of local rich da?ta and protect users'privacy. However, the scarce wireless communication resource greatly limits the number of participated users and is regarded as the main bottleneck which hin?ders the development of FEEL. To tackle this issue, we propose a user selection policy based on data importance for FEEL system. In order to quantify the data importance of each user, we first analyze the relationship between the loss decay and the squared norm of gradi?ent. Then, we formulate a combinatorial optimization problem to maximize the learning effi?ciency by jointly considering user selection and communication resource allocation. By problem transformation and relaxation, the optimal user selection policy and resource alloca?tion are derived, and a polynomial-time optimal algorithm is developed. Finally, we deploy two commonly used deep neural network (DNN) models for simulation. The results validate that our proposed algorithm has strong generalization ability and can attain higher learning efficiency compared with other traditional algorithms.
基金supported in part by the National Natural Science Foundation of China under Grant No.61701197in part by the National Key Research and Development Program of China under Grant No.2021YFA1000500(4)in part by the 111 Project under Grant No.B23008.
文摘Federated Edge Learning(FEL),an emerging distributed Machine Learning(ML)paradigm,enables model training in a distributed environment while ensuring user privacy by using physical separation for each user’s data.However,with the development of complex application scenarios such as the Internet of Things(IoT)and Smart Earth,the conventional resource allocation schemes can no longer effectively support these growing computational and communication demands.Therefore,joint resource optimization may be the key solution to the scaling problem.This paper simultaneously addresses the multifaceted challenges of computation and communication,with the growing multiple resource demands.We systematically review the joint allocation strategies for different resources(computation,data,communication,and network topology)in FEL,and summarize the advantages in improving system efficiency,reducing latency,enhancing resource utilization,and enhancing robustness.In addition,we present the potential ability of joint optimization to enhance privacy preservation by reducing communication requirements,indirectly.This work not only provides theoretical support for resource management in federated learning(FL)systems,but also provides ideas for potential optimal deployment in multiple real-world scenarios.By thoroughly discussing the current challenges and future research directions,it also provides some important insights into multi-resource optimization in complex application environments.
文摘As a popular distributed machine learning framework,wireless federated edge learning(FEEL)can keep original data local,while uploading model training updates to protect privacy and prevent data silos.However,since wireless channels are usually unreliable,there is no guarantee that the model updates uploaded by local devices are correct,thus greatly degrading the performance of the wireless FEEL.Conventional retransmission schemes designed for wireless systems generally aim to maximize the system throughput or minimize the packet error rate,which is not suitable for the FEEL system.A novel retransmission scheme is proposed for the FEEL system to make a tradeoff between model training accuracy and retransmission latency.In the proposed scheme,a retransmission device selection criterion is first designed based on the channel condition,the number of local data,and the importance of model updates.In addition,we design the air interface signaling under this retransmission scheme to facilitate the implementation of the proposed scheme in practical scenarios.Finally,the effectiveness of the proposed retransmission scheme is validated through simulation experiments.
基金the National Key R&D Program of China under Grant 2018YFB1800800Guangdong Province Key Area R&D Program under Grant 2018B030338001the Natural Science Foundation of China under Grant U2001208。
文摘This paper studies a federated edge learning system,in which an edge server coordinates a set of edge devices to train a shared machine learning(ML)model based on their locally distributed data samples.During the distributed training,we exploit the joint communication and computation design for improving the system energy efficiency,in which both the communication resource allocation for global ML-parameters aggregation and the computation resource allocation for locally updating ML-parameters are jointly optimized.In particular,we consider two transmission protocols for edge devices to upload ML-parameters to edge server,based on the non-orthogonal multiple access(NOMA)and time division multiple access(TDMA),respectively.Under both protocols,we minimize the total energy consumption at all edge devices over a particular finite training duration subject to a given training accuracy,by jointly optimizing the transmission power and rates at edge devices for uploading ML-parameters and their central processing unit(CPU)frequencies for local update.We propose efficient algorithms to solve the formulated energy minimization problems by using the techniques from convex optimization.Numerical results show that as compared to other benchmark schemes,our proposed joint communication and computation design significantly can improve the energy efficiency of the federated edge learning system,by properly balancing the energy tradeoff between communication and computation.
基金supported by the National Key R&D Program of China(No.2020YFB1807100)the National Natural Science Foundation of China(No.62001310)the Guangdong Basic and Applied Basic Research Foundation,China(No.2022A1515010109)。
文摘Training a machine learning model with federated edge learning(FEEL)is typically time consuming due to the constrained computation power of edge devices and the limited wireless resources in edge networks.In this study,the training time minimization problem is investigated in a quantized FEEL system,where heterogeneous edge devices send quantized gradients to the edge server via orthogonal channels.In particular,a stochastic quantization scheme is adopted for compression of uploaded gradients,which can reduce the burden of per-round communication but may come at the cost of increasing the number of communication rounds.The training time is modeled by taking into account the communication time,computation time,and the number of communication rounds.Based on the proposed training time model,the intrinsic trade-off between the number of communication rounds and per-round latency is characterized.Specifically,we analyze the convergence behavior of the quantized FEEL in terms of the optimality gap.Furthermore,a joint data-and-model-driven fitting method is proposed to obtain the exact optimality gap,based on which the closed-form expressions for the number of communication rounds and the total training time are obtained.Constrained by the total bandwidth,the training time minimization problem is formulated as a joint quantization level and bandwidth allocation optimization problem.To this end,an algorithm based on alternating optimization is proposed,which alternatively solves the subproblem of quantization optimization through successive convex approximation and the subproblem of bandwidth allocation by bisection search.With different learning tasks and models,the validation of our analysis and the near-optimal performance of the proposed optimization algorithm are demonstrated by the simulation results.
