Calculate the learning rate at a given step under a cosine annealing schedule.
Cosine annealing smoothly decays the learning rate following a cosine curve from lr_max down to lr_min over T steps: lr(t) = lr_min + 0.5 × (lr_max − lr_min) × (1 + cos(π·t/T)). At t=0, cos(0)=1, giving lr_max; at t=T, cos(π)=−1, giving lr_min. The curve decreases slowly at first, accelerates through the middle of training, and flattens out again near the end — this shape empirically tends to help models settle into a good minimum more smoothly than a linear or step decay, and is a core component of the popular SGDR (Stochastic Gradient Descent with Warm Restarts) and one-cycle training schedules.
lr(t) = lr_min + 0.5 × (lr_max - lr_min) × (1 + cos(pi × t / T))
Its smooth, non-linear decay spends more time at higher learning rates early (encouraging fast initial progress) and tapers gradually near the end (allowing fine-grained convergence), which empirically often outperforms step decay or pure exponential decay for many vision and NLP training regimes.
The plain cosine annealing formula is only defined for t in [0, T]; this calculator clamps t to T, effectively holding the learning rate at lr_min once the schedule's horizon has passed — some implementations instead restart the cosine cycle (SGDR) or switch to a different post-schedule strategy.
It's common to apply a linear warmup for the first 5–10% of steps before switching to cosine annealing for the remainder — the warmup steps are typically excluded from T, i.e. T represents only the post-warmup decay phase.
One-cycle LR combines an explicit warmup phase (ramping up to lr_max) with a cosine (or linear) decay phase, all in a single schedule; this calculator computes only the decay portion, assuming the ramp-up has already happened or isn't used.