The Machine Learning “Advent Calendar” Bonus 2: Gradient Descent Variants in Excel | Towards Data Science

Gradient Descent, Momentum, RMSProp, and Adam all aim for the same minimum. They do not change the destination, only the path. Each method adds a mechanism that fixes a limitation of the previous o...

By · · 1 min read
The Machine Learning “Advent Calendar” Bonus 2: Gradient Descent Variants in Excel | Towards Data Science

Source: Towards Data Science

Gradient Descent, Momentum, RMSProp, and Adam all aim for the same minimum. They do not change the destination, only the path. Each method adds a mechanism that fixes a limitation of the previous one, making the movement faster, more stable, or more adaptive. The goal stays the same. The update becomes smarter.