Cyber-Physical-Social Intelligence(CPSI)enhances the interpretability of Gradient Descent(GD)optimizers and improves image recognition models.To address the black-box nature of optimizers,we propose a new approach to ...Cyber-Physical-Social Intelligence(CPSI)enhances the interpretability of Gradient Descent(GD)optimizers and improves image recognition models.To address the black-box nature of optimizers,we propose a new approach to map the GD optimizer to the frequency domain,allowing analysis of gradient variations at different frequencies.This approach aids in selecting optimal training and optimization strategies,offering a novel solution to the challenge of optimizer interpretability.Specifically,firstly,based on the transfer function of the gradient descent optimizer,the real-frequency characteristic and imaginary-frequency characteristic functions of the optimizer are derived for the first time,which provide a new perspective for the selection of the optimizer and the interpretability of the tuning parameter.Then,based on these characteristic functions,the learning properties of the optimizer in the frequency domain are analyzed for the first time,which provides an important reference for improving the performance of the optimizer in real application problems.Finally,the effectiveness of the frequency domain modulation properties is verified through convex and non-convex optimization problems.Experimental results show that the proposed theory not only improves the recognition accuracy,convergence speed,and stability,but also extends its scope in practical applications.展开更多
基金supported by the Hainan Institute of Zhejiang University(No.6602-A12202)the National Natural Science Foundation of China(Nos.62262014 and 62566020)+1 种基金the Hainan Provincial Natural Science Foundation of China(No.625QN269)the Hainan Province Talent Innovation in Science and Technology Project of China(No.KJRC2023D36).
文摘Cyber-Physical-Social Intelligence(CPSI)enhances the interpretability of Gradient Descent(GD)optimizers and improves image recognition models.To address the black-box nature of optimizers,we propose a new approach to map the GD optimizer to the frequency domain,allowing analysis of gradient variations at different frequencies.This approach aids in selecting optimal training and optimization strategies,offering a novel solution to the challenge of optimizer interpretability.Specifically,firstly,based on the transfer function of the gradient descent optimizer,the real-frequency characteristic and imaginary-frequency characteristic functions of the optimizer are derived for the first time,which provide a new perspective for the selection of the optimizer and the interpretability of the tuning parameter.Then,based on these characteristic functions,the learning properties of the optimizer in the frequency domain are analyzed for the first time,which provides an important reference for improving the performance of the optimizer in real application problems.Finally,the effectiveness of the frequency domain modulation properties is verified through convex and non-convex optimization problems.Experimental results show that the proposed theory not only improves the recognition accuracy,convergence speed,and stability,but also extends its scope in practical applications.
