Speaker
Description
We propose a generalized alternating nonlinear generalized minimal residual method (GA-NGMRES) for accelerating first-order optimization algorithms. The method is applied to preconditioned first-order schemes by interpreting their update rules as fixed-point iterations. GA-NGMRES introduces a periodic mixing strategy that alternates between NGMRES extrapolation and fixed-point updates, resulting in improved robustness and efficiency.
We demonstrate that the proposed approach reduces both iteration counts and overall runtime compared to state-of-the-art methods. Numerical comparisons are provided against preconditioned gradient descent and preconditioned, inexact Gauss–Newton–Krylov methods. Since GA-NGMRES relies only on first-order derivative information, it is straightforward to implement. Performance is evaluated with respect to algorithmic hyperparameters, mesh resolution, and regularization parameters. For the problems considered, GA-NGMRES consistently outperforms Anderson acceleration.