期刊文献+

一种非光滑损失坐标下降算法 被引量:2

New coordinate descent algorithm for non-smooth losses
在线阅读 下载PDF
导出
摘要 针对非光滑损失问题提出一种新的坐标下降算法,采用排序搜索的方式求解子问题解析解。分析了算法的时间复杂度,并给出了三种提高收敛速度的实用技巧。实验表明算法对正则化Hinge损失问题具有良好的性能,达到了预期的效果。 For non-smooth losses,this paper presented a new coordinate descent algorithm,to get the closed form solution of the single variable problem by using the sorting and searching method.It analyzed the time complexity of algorithm and gave three practical skills to improve the convergence rate.The experiments demonstrate the expected efficiency of the proposed algorithms in the regularized Hinge loss.
出处 《计算机应用研究》 CSCD 北大核心 2012年第10期3688-3692,3700,共6页 Application Research of Computers
基金 国家自然科学基金资助项目(60975040)
关键词 机器学习 优化 坐标下降 非光滑损失 HINGE machine learning optimization coordinate descent non-smooth loss Hinge
  • 相关文献

参考文献19

  • 1孙正雅,陶卿.统计机器学习--损失函数与优化求解[J].中国计算机学会通讯,2009,5(8):7-14.
  • 2CHANG Kai-wei, HSIEH C J, LINC J. Coordinate descent method for large-scale L2-1oss linear support vector machines[ J]. ,Journal of Machine Learning Research,2008,9 (7) : 1369-1398.
  • 3HSIEH C J, CHANG Kai-wei, LIN C J, et al. A dual coordinate de- scent method for large-scale linear SVM[ C]//Proc of the 25th Inter- national Conference on Machine Learning. New York: ACM Press, 2008:408-415.
  • 4SAHA A, TEWARI A, On the finite time convergence of cyclic coor- dinate descent methods [ EB/OL ]. ( 2010- 05- 12 ). http ://arxiv. o14 glabs/10052146.
  • 5NESTEROV Y. Efficiency of coordinate descent methods on huge-scale optimization problems [ R 1- Belgium : University Catholique de Lou- vain, Center for Operations Research and Econometrics ,2010:2-20.
  • 6LUO Zhi-quan, TSENG P. On the convergence of the coordinate de- scent method for convex differentiable minimization [ J ]. Journal of Optimization Theory and Applications, 1992,72 (1) :7-35.
  • 7LUO Zhi-quan, TSENG P. On the linear convergence of descent methods for convex essentially smooth minimization[ J]. SiAM Jour- nal Control Optimization, 1992,30(2) : 408-425.
  • 8FU Wen-jiang. Penalized regressions: the bridge versus the lasso[ J]. Journal of Computational and Graphical Statistics, 1998,7 ( 3 ) : 397-416.
  • 9FRIEDMAN J, HASTIE T, HOFLING H, et al. Pathwise coordinate optimization[J]. Annals of Applied Statistics,2007,1 (2) :302-332.
  • 10WU Tong-tong, LANGE K. Coordinate descent algorithms for lasso penalized regression[ J]. Annals of Applied Statistics,2008,2 ( 1 ) : 224-244.

同被引文献22

  • 1李欣,张晋国,张孟杰,赵丽.麦田杂草的图像识别技术的研究[J].农机化研究,2007,29(5):64-68. 被引量:2
  • 2孙正雅,陶卿.统计机器学习综述:损失函数与优化求解[J].中国计算机学会通讯,2009,5(8):7-14.
  • 3BOYD S, VANDENBERGHE L. Convex optimization [ M]. Cam- bridge: Cambridge University Press, 2004.
  • 4GABA D, MERCIER B. A dual algorithm for the solution of non- linear variational problems via finite element approximation [ J]. Computers and Mathematics with Applications, 1976, 2(1) : 17 - 40.
  • 5HE B S, YUAN X M. On the O(t/t) convergence rate of alterna- ting direction method [ EB/OL]. [2012 - 12 - 11]. http://www. optimization-online, org/DB-HTML/2011/09/3157, html.
  • 6BOYD S, PARIKH N, CHU E, et al. Distributed optimization and statistical learning via the ahernating direction method of multipliers [ J]. Foundations and Trends in Machine Learning, 2011, 3(1) : 1 - 122.
  • 7YANG A Y, SASTRY S S, GANESH A, et al. Fast ll-minimization algorithms and an application in robust face recognition: a review [ C]//2010 17th IEEE International Conference on Image Process- ing. New York: ACM Press, 2010: 1849-1852.
  • 8CHAN R H, YANG J F, YUAN X M. Alternating direction method for image inpainting in wavelet domain [ J]. SIAM Journal on Ima- ging Sciences, 2011, 4(3): 807-826.
  • 9NESTEROV Y. Efficiency of coordinate descent methods on huge- scale optimization problems [ R]. University catholique de Louvain, Center for Operations Research and Econometrics, 2010.
  • 10YUAN G X, CHANG K W, HSIEH C J, eta/. A comparison of op- timization methods and software for large-scale Ll-regularized linear classification [ J]. Journal of Machine Learning Research, 2010, 11 (3): 3183-3234.

引证文献2

二级引证文献9

相关作者

内容加载中请稍等...

相关机构

内容加载中请稍等...

相关主题

内容加载中请稍等...

浏览历史

内容加载中请稍等...
;
使用帮助 返回顶部