QHM (Quasi-Hyperbolic Momentum)

Published on: August 13, 2021

QHM (Quasi-Hyperbolic Momentum)

Table of Content

Quasi-Hyperbolic Momentum Algorithm (QHM) is a simple alteration of SGD with momentum, averaging a plain SGD step with a momentum step, thereby decoupling the momentum term from the current gradient when updating the weights.

The authors recommend  and  as a good starting point. For more information about QHM, check out the resources below.

Code

Resources

More stories

  • Activation Functions

  • RAdam - Rectified Adam

  • QHAdam (Quasi-Hyperbolic Adam)