https://htn20190109.hatenablog.com/entry/2025/08/23/002523

https://qiita.com/omiita/items/1735c1d048fe5f611f80
https://qiita.com/msekino/items/0b522df43d447c01e3ed

Momentumの更新式
$\displaystyle \mathbf{v}_{t} = \alpha \mathbf{v}_{t-1} - \eta \, \nabla_{\boldsymbol{\theta}} L(\boldsymbol{\theta}_{t})$
$\displaystyle \boldsymbol{\theta}_{t+1} = \boldsymbol{\theta}_{t} + \mathbf{v}_{t}$

NAG(Nesterov Accelerated Gradient)の更新式
$\displaystyle \mathbf{v}_{t} = \alpha \mathbf{v}_{t-1} - \eta \, \nabla_{\boldsymbol{\theta}} L(\boldsymbol{\theta}_{t} + \alpha \mathbf{v}_{t-1})$
$\displaystyle \boldsymbol{\theta}_{t+1} = \boldsymbol{\theta}_{t} + \mathbf{v}_{t}$

Pathological Curvature
→ 最適化アルゴリズムにおいて、損失関数の勾配が急激に変化する領域が存在する状態