The distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of n local cost functions by using local information exchange is considered.This problem is an important component of...The distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of n local cost functions by using local information exchange is considered.This problem is an important component of many machine learning techniques with data parallelism,such as deep learning and federated learning.We propose a distributed primal-dual stochastic gradient descent(SGD)algorithm,suitable for arbitrarily connected communication networks and any smooth(possibly nonconvex)cost functions.We show that the proposed algorithm achieves the linear speedup convergence rate O(1/(√nT))for general nonconvex cost functions and the linear speedup convergence rate O(1/(nT)) when the global cost function satisfies the Polyak-Lojasiewicz(P-L)condition,where T is the total number of iterations.We also show that the output of the proposed algorithm with constant parameters linearly converges to a neighborhood of a global optimum.We demonstrate through numerical experiments the efficiency of our algorithm in comparison with the baseline centralized SGD and recently proposed distributed SGD algorithms.展开更多
develop a mentation This paper considers the priority facility primal-dual 3-approximation algorithm for procedure, the authors further improve the location problem with penalties： The authors this problem. Combining...develop a mentation This paper considers the priority facility primal-dual 3-approximation algorithm for procedure, the authors further improve the location problem with penalties： The authors this problem. Combining with the greedy aug- previous ratio 3 to 1.8526.展开更多
Image segmentation is a hot topic in image science. In this paper we present a new variational segmentation model based on the theory of Mumford-Shah model. The aim of our model is to divide noised image, according to...Image segmentation is a hot topic in image science. In this paper we present a new variational segmentation model based on the theory of Mumford-Shah model. The aim of our model is to divide noised image, according to a certain criterion, into homogeneous and smooth regions that should correspond to structural units in the scene or objects of interest. The proposed region-based model uses total variation as a regularization term, and different fidelity term can be used for image segmentation in the cases of physical noise, such as Gaussian, Poisson and multiplicative speckle noise. Our model consists of five weighted terms, two of them are responsible for image denoising based on fidelity term and total variation term, the others assure that the three conditions of adherence to the data, smoothing, and discontinuity detection are met at once. We also develop a primal-dual hybrid gradient algorithm for our model. Numerical results on various synthetic and real images are provided to compare our method with others, these results show that our proposed model and algorithms are effective.展开更多
基金supported by the Knut and Alice Wallenberg Foundationthe Swedish Foundation for Strategic Research+1 种基金the Swedish Research Councilthe National Natural Science Foundation of China(62133003,61991403,61991404,61991400)。
文摘The distributed nonconvex optimization problem of minimizing a global cost function formed by a sum of n local cost functions by using local information exchange is considered.This problem is an important component of many machine learning techniques with data parallelism,such as deep learning and federated learning.We propose a distributed primal-dual stochastic gradient descent(SGD)algorithm,suitable for arbitrarily connected communication networks and any smooth(possibly nonconvex)cost functions.We show that the proposed algorithm achieves the linear speedup convergence rate O(1/(√nT))for general nonconvex cost functions and the linear speedup convergence rate O(1/(nT)) when the global cost function satisfies the Polyak-Lojasiewicz(P-L)condition,where T is the total number of iterations.We also show that the output of the proposed algorithm with constant parameters linearly converges to a neighborhood of a global optimum.We demonstrate through numerical experiments the efficiency of our algorithm in comparison with the baseline centralized SGD and recently proposed distributed SGD algorithms.
基金supported by the National Natural Science Foundation of China under Grant No.11371001
文摘develop a mentation This paper considers the priority facility primal-dual 3-approximation algorithm for procedure, the authors further improve the location problem with penalties： The authors this problem. Combining with the greedy aug- previous ratio 3 to 1.8526.
基金Supported in part by the NNSF of China(11301129,11271323,91330105,11326033)the Zhejiang Provincial Natural Science Foundation of China(LQ13A010025,LZ13A010002)
文摘Image segmentation is a hot topic in image science. In this paper we present a new variational segmentation model based on the theory of Mumford-Shah model. The aim of our model is to divide noised image, according to a certain criterion, into homogeneous and smooth regions that should correspond to structural units in the scene or objects of interest. The proposed region-based model uses total variation as a regularization term, and different fidelity term can be used for image segmentation in the cases of physical noise, such as Gaussian, Poisson and multiplicative speckle noise. Our model consists of five weighted terms, two of them are responsible for image denoising based on fidelity term and total variation term, the others assure that the three conditions of adherence to the data, smoothing, and discontinuity detection are met at once. We also develop a primal-dual hybrid gradient algorithm for our model. Numerical results on various synthetic and real images are provided to compare our method with others, these results show that our proposed model and algorithms are effective.
基金Supported by the Shanghai Pujiang Program(21PJ1413100)the National Natural Science Foundation of China(62003243,62103305)+3 种基金the Shanghai Municipal Science and Technology Major Project(2021SHZDZX0100)the Shanghai Municipal Commission of Science and Technology(1951-1132101)the Young Elite Scientist Sponsorship Program by cast of China Association for Science and Technology(YESS20200136)the Fundamental Research Funds for the Central Universities(22120210096)。
