Application of Linearized Alternating Direction Multiplier Method in Dictionary Learning
Abstract: The Alternating Direction Multiplier Method (ADMM) is widely used in various fields, and different variables are customized in the literature for different application scenarios    . Among them, the linearized alternating direction multiplier method (LADMM) has received extensive attention because of its effectiveness and ease of implementation. This paper mainly discusses the application of ADMM in dictionary learning (non-convex problem). Many numerical experiments show that to achieve higher convergence accuracy, the convergence speed of ADMM is slower, especially near the optimal solution. Therefore, we introduce the linearized alternating direction multiplier method (LADMM) to accelerate the convergence speed of ADMM. Specifically, the problem is solved by linearizing the quadratic term of the subproblem, and the convergence of the algorithm is proved. Finally, there is a brief summary of the full text.
Cite this paper: Yu, X. (2019) Application of Linearized Alternating Direction Multiplier Method in Dictionary Learning. Journal of Applied Mathematics and Physics, 7, 138-147. doi: 10.4236/jamp.2019.71012.
References

   Boyd, S., Parikh, N., Chu, E., Peleato, B. and Eckstein, J. (2010) Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Foundations and Trends® in Machine Learning, 3, 1-122.
https://doi.org/10.1561/2200000016

   Nadai E. (2011) Statistics for High-Dimensional Data: Methods, Theory and Applicatios. Springer, Berlin.

   Wu, Z.M. (2016) Linearized Alternating Direction Method for Solving Separable Convex Optimization Problems.
https://wenku.baidu.com/view/7aff15227ed5360cba1aa8114431b90d6c8589e3.html

   Fu, X.L., He, B.H., Wang, X.F., et al. (2014) Block-Wise Alternating Direction Method of Multipliers with Gaussian Back Substitution for Multiple-Block Convex Programming. 1-37.
http://xueshu.baidu.com/usercenter/paper/show?paperid=4cb7be88f0f61a585690896f1da8bf31&site=xueshu_se

   Wang, H.F. and Kong, L.C. (2017) The Linearized Alternating Direction Method of Multipliers for Sparse Group LAD Model. Beijing Jiaotong University, Beijing.

   Eckstein, J. (2016) Approximate ADMM Algorithms Derived from Lagrangian Splitting. Computational Optimization and Applications, 68, 363-405.
https://doi.org/10.1007/s10589-017-9911-z

   Haubruge, S., Nguyen, V.H. and Strodiot, J.J. (1998) Convergence Analysis and Applications of the Glowinski—Le Tallec Splitting Method for Finding a Zero of the Sum of Two Maximal Monotone Operators. Journal of Optimization Theory and Applications, 97, 645-673.

   Glowinski, R. and Marroco, A. (1975) Sur l’approximation, par éléments finis d’ordre un, et la résolution, par Pénalisation-Dualité d’une classe de problèmes de Dirichlet non linéaires. Journal of Equine Veterinary Science, 2, 41-76.

   Wang, H.F. (2017) Linearized Multiplier Alternating Direction Method for Solving Least Degree Model of Sparse Group.
https://wenku.baidu.com/view/f25e8ff1d5d8d15abe23482fb4daa58da0111c9c.html

   Jiao, Y.L., Jin, Q.N., Lu, X.L., et al. (2016) Alternating Direction Method of Multipliers for Linear Inverse Problems. SIAM Journal on Numerical Analysis, 54, 2114-2137.
https://doi.org/10.1137/15M1029308

   Wang, Y.L., Yang, J.F., Yin, W.T., et al. (2016) A New Alternating Minimization Algorithm for Total Variation Image Reconstruction. SIAM Journal on Imaging Sciences, 1, 248–272.

Top