r/deeplearning • u/Mind_the_math • 2d ago

The idea behind Adam Optimizer

https://www.youtube.com/watch?v=lRu8_6cP3AQ

Hi all! I'm AI engineer. This is the first video on my channel. I tried to build Adam from scratch, motivating each step by a concrete problem of the previous one: plain GD, Nesterov, AdaGrad, RMSProp ...

All animations are made with Manim. I'd really appreciate feedback, especially on anything that's mathematically imprecise or unclear.

22 Upvotes

87% Upvoted

1 comment sorted by

1

u/Chopok 1d ago

/stəˈtʃæstɪk/??? Really?