STOCHASTIC ALTERNATING DIRECTION METHOD OF MULTIPLIERS FOR STRUCTURED REGULARIZATION STOCHASTIC ALTERNATING DIRECTION METHOD OF MULTIPLIERS FOR STRUCTURED REGULARIZATION

Access this Article

Author(s)

    • Suzuki Taiji
    • Graduate School of Information Science and Engineering, Tokyo Institute of Technology|PRESTO, Japan Science and Technology Agency, JAPAN

Abstract

<p>In this paper, we present stochastic optimization variants of the alternating direction method of multipliers (ADMM). ADMM is a useful method to solve a regularized risk minimization problem where the regularization term is complicated and not easily dealt with in an ordinary manner. For example, structured regularization is one of the typical applications of such regularization in which ADMM is effective. It includes group lasso regularization, low rank tensor regularization, and fused lasso regularization. Since ADMM is a general method and has wide applications, it is intensively studied and refined these days. However, ADMM is not suited to optimization problems with huge data. To resolve this problem, online stochastic optimization variants and a batch stochastic optimization variant of ADMM are presented. All the presented methods can be easily implemented and have wide applications. Moreover, the theoretical guarantees of the methods are given.</p>

<p>In this paper, we present stochastic optimization variants of the alternating direction method of multipliers (ADMM). ADMM is a useful method to solve a regularized risk minimization problem where the regularization term is complicated and not easily dealt with in an ordinary manner. For example, structured regularization is one of the typical applications of such regularization in which ADMM is effective. It includes group lasso regularization, low rank tensor regularization, and fused lasso regularization. Since ADMM is a general method and has wide applications, it is intensively studied and refined these days. However, ADMM is not suited to optimization problems with huge data. To resolve this problem, online stochastic optimization variants and a batch stochastic optimization variant of ADMM are presented. All the presented methods can be easily implemented and have wide applications. Moreover, the theoretical guarantees of the methods are given.</p>

Journal

  • Journal of the Japanese Society of Computational Statistics

    Journal of the Japanese Society of Computational Statistics 28(1), 105-124, 2015

    Japanese Society of Computational Statistics

Codes

Page Top