Embedding a heavy-ball type of momentum into the estimating sequences
Loading...
Access rights
openAccess
CC BY
CC BY
publishedVersion
URL
Journal Title
Journal ISSN
Volume Title
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
Date
Major/Subject
Mcode
Degree programme
Language
en
Pages
14
Series
Signal Processing, Volume 233
Abstract
We present a new accelerated gradient-based method for solving smooth unconstrained optimization problems. The new method exploits additional information about the objective function and is built by embedding a heavy-ball type of momentum into the Fast Gradient Method (FGM). We devise a generalization of the estimating sequences, which allows for encoding any form of information about the objective function that can aid in further accelerating the minimization process. In the black box framework, we propose a construction for the generalized estimating sequences, which is obtained by exploiting the history of the previously constructed estimating functions. Moreover, we prove that the proposed method requires at most [Formula presented] iterations to find a point x with f(x)−f∗≤ϵ, where ϵ is the desired tolerance and κ is the condition number of the problem. Our theoretical results are corroborated by numerical experiments on various types of optimization problems, often dealt with in different areas of the information processing sciences. Both synthetic and real-world datasets are utilized to demonstrate the efficiency of our proposed method in terms of decreasing the distance to the optimal solution, the norm of the gradient and the function value.Description
Publisher Copyright: © 2025 The Author(s)
Other note
Citation
Dosti, E, Vorobyov, S A & Charalambous, T 2025, 'Embedding a heavy-ball type of momentum into the estimating sequences', Signal Processing, vol. 233, 109865. https://doi.org/10.1016/j.sigpro.2024.109865