联邦学习因具有隐私保护的天然特性,已经逐渐成为一个被广泛认可的分布式机器学习框架。但由于参与方数据分布的差异性,特别是呈现非独立同分布(Non-Independent and Identically Distributed,Non-IID)时,其面临着泛化性能不足、收敛性...联邦学习因具有隐私保护的天然特性,已经逐渐成为一个被广泛认可的分布式机器学习框架。但由于参与方数据分布的差异性,特别是呈现非独立同分布(Non-Independent and Identically Distributed,Non-IID)时,其面临着泛化性能不足、收敛性能下降、数据倾斜等严峻挑战。用预训练基础模型缓解Non-IID问题作为一种新颖的方法,演变出了各种各样的解决方案。对此,从预训练基础模型的角度,对现有工作进行了综述。首先介绍了基础模型方法,对典型的基础模型编码结构进行对比分析。其次从修改输入、基础模型部分结构再训练,以及参数高效微调3个角度,提出了一种新的分类方法。最后探讨了该类工作的核心难题和未来研究方向。展开更多
作为一种分布式训练框架,联邦学习在无线通信领域有着广阔的应用前景,也面临着多方面的技术挑战,其中之一源于参与训练用户数据集的非独立同分布(Independent and identically distributed,IID)。不少文献提出了解决方法,以减轻户数据集...作为一种分布式训练框架,联邦学习在无线通信领域有着广阔的应用前景,也面临着多方面的技术挑战,其中之一源于参与训练用户数据集的非独立同分布(Independent and identically distributed,IID)。不少文献提出了解决方法,以减轻户数据集非IID造成的联邦学习性能损失。本文以平均信道增益预测、正交幅度调制信号的解调这两个无线任务以及两个图像分类任务为例,分析用户数据集非IID对联邦学习性能的影响,通过神经网络损失函数的可视化和对模型参数的偏移量进行分析,尝试解释非IID数据集对不同任务影响程度不同的原因。分析结果表明,用户数据集非IID未必导致联邦学习性能的下降。在不同数据集上通过联邦平均算法训练得到的模型参数偏移程度和损失函数形状有很大的差异,二者共同导致了不同任务受数据非IID影响程度的不同;在同一个回归问题中,数据集非IID是否影响联邦学习的性能与引起数据非IID的具体因素有关。展开更多
In non-independent and identically distributed(non-IID)data environments,model performance often degrades significantly.To address this issue,two improvement methods are proposed:FedReg and FedReg^(*).FedReg is a meth...In non-independent and identically distributed(non-IID)data environments,model performance often degrades significantly.To address this issue,two improvement methods are proposed:FedReg and FedReg^(*).FedReg is a method based on hybrid regularization aimed at enhancing federated learning in non-IID scenarios.It introduces hybrid regularization to replace traditional L2 regularization,combining the advantages of L1 and L2 regularization to enable feature selection while preventing overfitting.This method better adapts to the diverse data distributions of different clients,improving the overall model performance.FedReg^(*)combines hybrid regularization with weighted model aggregation.In addition to the benefits of hybrid regularization,FedReg^(*)applies a weighted averaging method in the model aggregation process,calculating weights based on the cosine similarity between each client gradient and the global gradient to more reasonably distribute client contributions.By considering variations in data quality and quantity among clients,FedReg^(*)highlights the importance of key clients and enhances the model’s generalization performance.These improvement methods enhance model accuracy and communication efficiency.展开更多
With the notion of independent identically distributed(IID) random variables under sublinear expectations introduced by Peng,we investigate moment bounds for IID sequences under sublinear expectations. We obtain a mom...With the notion of independent identically distributed(IID) random variables under sublinear expectations introduced by Peng,we investigate moment bounds for IID sequences under sublinear expectations. We obtain a moment inequality for a sequence of IID random variables under sublinear expectations. As an application of this inequality,we get the following result:For any continuous functionsatisfying the growth condition |(x) | C(1 + |x|p) for some C > 0,p 1 depending on ,the central limit theorem under sublinear expectations obtained by Peng still holds.展开更多
With Globally Important Agricultural Heritage Systems(GIAHS)increasing in number around the world,their conservation has become a new international research theme.From the perspective of combining theoretical analyses...With Globally Important Agricultural Heritage Systems(GIAHS)increasing in number around the world,their conservation has become a new international research theme.From the perspective of combining theoretical analyses and practical case applications,this study examines the Important Agricultural Heritage Systems(IAHS)conservation pathways and operation mechanisms through industrial integration development(IID).First,the theoretical framework of IID in IAHS sites was constructed according to the requirements of IAHS conservation,which include analyses of the connotation and basic principles of IID,the necessity of IID for IAHS sites,the resource conditions,and the IID pathways.And then based on the theoretical framework,the IID of Longji Terraces in Guangxi,Honghe Hani Rice Terraces System in Yunnan(HHRTS),Aohan Dryland Farming System in Inner Mongolia(ADFS),and Huzhou Mulberry-dyke&Fish-pond System(HMFS)in Zhejiang are analyzed systematically.The main finding is that IID is an effective pathway for IAHS conservation.However,the IID in IAHS sites must stress the ecological and cultural values of the resources;IID should be based on local resource advantages;and IID should attach importance to the combination of different policies and coordination between different stakeholders.展开更多
文摘联邦学习因具有隐私保护的天然特性,已经逐渐成为一个被广泛认可的分布式机器学习框架。但由于参与方数据分布的差异性,特别是呈现非独立同分布(Non-Independent and Identically Distributed,Non-IID)时,其面临着泛化性能不足、收敛性能下降、数据倾斜等严峻挑战。用预训练基础模型缓解Non-IID问题作为一种新颖的方法,演变出了各种各样的解决方案。对此,从预训练基础模型的角度,对现有工作进行了综述。首先介绍了基础模型方法,对典型的基础模型编码结构进行对比分析。其次从修改输入、基础模型部分结构再训练,以及参数高效微调3个角度,提出了一种新的分类方法。最后探讨了该类工作的核心难题和未来研究方向。
文摘作为一种分布式训练框架,联邦学习在无线通信领域有着广阔的应用前景,也面临着多方面的技术挑战,其中之一源于参与训练用户数据集的非独立同分布(Independent and identically distributed,IID)。不少文献提出了解决方法,以减轻户数据集非IID造成的联邦学习性能损失。本文以平均信道增益预测、正交幅度调制信号的解调这两个无线任务以及两个图像分类任务为例,分析用户数据集非IID对联邦学习性能的影响,通过神经网络损失函数的可视化和对模型参数的偏移量进行分析,尝试解释非IID数据集对不同任务影响程度不同的原因。分析结果表明,用户数据集非IID未必导致联邦学习性能的下降。在不同数据集上通过联邦平均算法训练得到的模型参数偏移程度和损失函数形状有很大的差异,二者共同导致了不同任务受数据非IID影响程度的不同;在同一个回归问题中,数据集非IID是否影响联邦学习的性能与引起数据非IID的具体因素有关。
文摘In non-independent and identically distributed(non-IID)data environments,model performance often degrades significantly.To address this issue,two improvement methods are proposed:FedReg and FedReg^(*).FedReg is a method based on hybrid regularization aimed at enhancing federated learning in non-IID scenarios.It introduces hybrid regularization to replace traditional L2 regularization,combining the advantages of L1 and L2 regularization to enable feature selection while preventing overfitting.This method better adapts to the diverse data distributions of different clients,improving the overall model performance.FedReg^(*)combines hybrid regularization with weighted model aggregation.In addition to the benefits of hybrid regularization,FedReg^(*)applies a weighted averaging method in the model aggregation process,calculating weights based on the cosine similarity between each client gradient and the global gradient to more reasonably distribute client contributions.By considering variations in data quality and quantity among clients,FedReg^(*)highlights the importance of key clients and enhances the model’s generalization performance.These improvement methods enhance model accuracy and communication efficiency.
基金supported in part by National Basic Research Program of China(973 Program)(Grant No.2007CB814901)the Natural Science Foundation of Shandong Province(Grant No.ZR2009AL015)
文摘With the notion of independent identically distributed(IID) random variables under sublinear expectations introduced by Peng,we investigate moment bounds for IID sequences under sublinear expectations. We obtain a moment inequality for a sequence of IID random variables under sublinear expectations. As an application of this inequality,we get the following result:For any continuous functionsatisfying the growth condition |(x) | C(1 + |x|p) for some C > 0,p 1 depending on ,the central limit theorem under sublinear expectations obtained by Peng still holds.
基金The Agricultural Science and Technology Innovation Program (ASTIP-IAED-2021-06, STIP-IAED-2021-ZD-02)。
文摘With Globally Important Agricultural Heritage Systems(GIAHS)increasing in number around the world,their conservation has become a new international research theme.From the perspective of combining theoretical analyses and practical case applications,this study examines the Important Agricultural Heritage Systems(IAHS)conservation pathways and operation mechanisms through industrial integration development(IID).First,the theoretical framework of IID in IAHS sites was constructed according to the requirements of IAHS conservation,which include analyses of the connotation and basic principles of IID,the necessity of IID for IAHS sites,the resource conditions,and the IID pathways.And then based on the theoretical framework,the IID of Longji Terraces in Guangxi,Honghe Hani Rice Terraces System in Yunnan(HHRTS),Aohan Dryland Farming System in Inner Mongolia(ADFS),and Huzhou Mulberry-dyke&Fish-pond System(HMFS)in Zhejiang are analyzed systematically.The main finding is that IID is an effective pathway for IAHS conservation.However,the IID in IAHS sites must stress the ecological and cultural values of the resources;IID should be based on local resource advantages;and IID should attach importance to the combination of different policies and coordination between different stakeholders.
