How Memory in Optimization Algorithms Implicitly Modifies the Loss
arXiv:2502.02132v2 Announce Type: replace Abstract: In modern optimization methods used in deep learning, each update depends on the history of previous iterations, often referred to as memory, and this dependence decays fast as the iterates go further into the past.…
