Optimization Calculators
10 free optimization calculators in AI & ML. The first 4 have a full write-up: the method, a worked example and questions specific to that calculator.
- Adam Optimizer Update CalculatorCalculate the parameter update step produced by the Adam optimizer.
- AdamW Update CalculatorCalculate the parameter update step produced by the AdamW optimizer with decoupled weight decay.
- Cyclic Learning Rate CalculatorCalculate the learning rate at a given step under a cyclic learning rate schedule.
- One Cycle LR CalculatorCalculate the learning rate at a given step under a one-cycle policy schedule.
- Cosine Annealing CalculatorCalculate the learning rate at a given step under a cosine annealing schedule.
- Exponential Decay CalculatorCalculate the learning rate at a given step under an exponential decay schedule.
- Momentum Update CalculatorCalculate the parameter update step produced by gradient descent with momentum.
- Polynomial Decay CalculatorCalculate the learning rate at a given step under a polynomial decay schedule.
- RMSProp Update CalculatorCalculate the parameter update step produced by the RMSProp optimizer.
- SGD Update CalculatorCalculate the parameter update step produced by stochastic gradient descent.