Work out exploration exploitation ratio instantly with clear inputs, formula shown and shareable results.
Epsilon-greedy picks a random action with probability epsilon and the current best action otherwise. Linear annealing interpolates epsilon from its start to its floor over a fixed number of steps, so early training explores almost entirely and late training exploits almost entirely. A non-zero floor is kept deliberately so the agent keeps sampling alternatives in a changing environment.
Linear epsilon decay
epsilon(t) = start + (end - start) x min(1, t / decay steps); greedy share = 1 - epsilon
So the agent can still discover changes. With epsilon exactly zero the policy freezes and a shift in dynamics is never detected.
Yes for many settings. Upper-confidence-bound and Thompson sampling explore in proportion to uncertainty rather than uniformly at random, which reduces regret substantially.