57003

Автор(ы): 

Автор(ов): 

3

Параметры публикации

Тип публикации: 

Статья в журнале/сборнике

Название: 

Non-monotone Behavior of the Heavy Ball Method

ISBN/ISSN: 

ISBN 978-3-030-35502-9

DOI: 

DOI 978-3-030-35502-9_9

Наименование источника: 

  • Springer Proceedings in Mathematics & Statistics: Difference Equations and Discrete Dynamical Systems with Applications

Обозначение и номер тома: 

Vol. 312

Город: 

  • Cham, Switzerland

Издательство: 

  • Springer

Год издания: 

2020

Страницы: 

213-230
Аннотация
We focus on the solutions of second-order stable linear difference equations and demonstrate that their behavior can be non-monotone and exhibit peak effects depending on initial conditions. The results are applied to the analysis of the accelerated unconstrained optimization method - the Heavy Ball method. We explain non-standard behavior of the method discovered in practical applications. In addition, such non-monotonicity complicates the correct choice of the parameters in optimization methods. We propose to overcome this difficulty by introducing new Lyapunov function which should decrease monotonically. By use of this function convergence of the method is established under less restrictive assumptions (for instance, with the lack of convexity). We also suggest some restart techniques to speed up the method’s convergence.

Библиографическая ссылка: 

Поляк Б.Т., Кулакова А.Д., Данилова М.Ю. Non-monotone Behavior of the Heavy Ball Method / Springer Proceedings in Mathematics & Statistics: Difference Equations and Discrete Dynamical Systems with Applications. Cham, Switzerland: Springer, 2020. Vol. 312. С. 213-230.