Optimization / terrain versus update rules
Optimizer Racetrack
Question: how do update rules react to the same geometry? Pick a terrain, keep the learning rate matched for a controlled comparison, or tune each optimizer openly.
Pick a terrain
These are analytical losses, so every height and gradient is exact and inspectable.
Choose the comparison rule
Both modes are useful when their purpose is explicit.
Every trace receives the same η. This isolates the update rules while still letting you adjust their own β, ρ, and ε settings.
Curved ravine
Pale center means lower loss. Click a trace or its legend to choose the optimizer whose settings you want to inspect.
Race controls
Change where every optimizer starts, then scrub or run the shared clock.
Adam settings
The learning rate is copied to every optimizer for a controlled baseline. The optimizer's own mechanics remain editable.
Adam combines bias-corrected momentum and adaptive scaling.
Live lap board
The highlighted row is the path drawn brightest on the terrain.
| optimizer | learning rate | current loss | best loss | status |
|---|---|---|---|---|
| 0.070 | 3.27820 | 3.27820 | paused | |
| 0.070 | 3.27820 | 3.27820 | paused | |
| 0.070 | 3.27820 | 3.27820 | paused | |
| 0.070 | 3.27820 | 3.27820 | paused |
What this terrain is testing
A steep, curved valley makes side-to-side gradients much larger than the useful downhill direction.
No optimizer is universally best. The map lets you separate a controlled same-rate comparison from a transparent tuning exercise, rather than hiding that choice.
This is a deterministic analytical loss, not a model trained in the browser. It isolates optimizer behavior without a dataset or network obscuring the geometry.