@runevision look at ADAM for step sizes. You could also try L-BFGS (not certain how it will do on 106 params). Can try random restarts to avoid local minima.
Mastodon is the best way to keep up with what's happening.
Follow anyone across the fediverse and see it all in chronological order. No algorithms, ads, or clickbait in sight.