Cosine Annealing Calculator
Calculate the learning rate at a given step under a cosine annealing schedule.
About Cosine Annealing Calculator
Cosine annealing smoothly decays the learning rate following a cosine curve from lr_max down to lr_min over T steps: lr(t) = lr_min + 0.5 × (lr_max − lr_min) × (1 + cos(π·t/T)). At t=0, cos(0)=1, giving lr_max; at t=T, cos(π)=−1, giving lr_min. The curve decreases slowly at first, accelerates through the middle of training, and flattens out again near the end — this shape empirically tends to help models settle into a good minimum more smoothly than a linear or step decay, and is a core component of the popular SGDR (Stochastic Gradient Descent with Warm Restarts) and one-cycle training schedules.
Formula
lr(t) = lr_min + 0.5 × (lr_max - lr_min) × (1 + cos(pi × t / T))
- lr_min
- Minimum learning rate
- lr_max
- Maximum learning rate
- t
- Current step
- T
- Total steps in the schedule
Worked example
With the default values below, Cosine Annealing Calculator returns Learning Rate at Step t of 5.0500 × 10⁻⁴. Change any input above and every figure here is recalculated in your browser.
Inputs used
- Max Learning Rateminimum 0
- 0.001
- Min Learning Rateminimum 0
- 0.00001
- Current Step (t)accepts 0–1,000,000,000
- 5,000
- Total Steps (T)accepts 1–1,000,000,000
- 10,000
Results
- Learning Rate at Step t
- 5.0500 × 10⁻⁴
- Schedule Progress
- 50.0%
Step by step
- 1cos(π × t ÷ T)= 0.000000
- 2lr = lr_min + 0.5×(lr_max−lr_min)×(1+cos)= 5.0500e-4
Assumptions and limits
- The result is derived only from the 4 inputs listed above. Anything not among them is not part of the calculation.
- In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
- Figures are rounded for display to 4 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.
How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.
Frequently Asked Questions
Why is cosine annealing popular for training deep models?
Its smooth, non-linear decay spends more time at higher learning rates early (encouraging fast initial progress) and tapers gradually near the end (allowing fine-grained convergence), which empirically often outperforms step decay or pure exponential decay for many vision and NLP training regimes.
What happens after t exceeds T?
The plain cosine annealing formula is only defined for t in [0, T]; this calculator clamps t to T, effectively holding the learning rate at lr_min once the schedule's horizon has passed — some implementations instead restart the cosine cycle (SGDR) or switch to a different post-schedule strategy.
How does this combine with warmup?
It's common to apply a linear warmup for the first 5–10% of steps before switching to cosine annealing for the remainder — the warmup steps are typically excluded from T, i.e. T represents only the post-warmup decay phase.
How is this different from one-cycle LR?
One-cycle LR combines an explicit warmup phase (ramping up to lr_max) with a cosine (or linear) decay phase, all in a single schedule; this calculator computes only the decay portion, assuming the ramp-up has already happened or isn't used.