Work out reinforcement learning reward discount instantly with clear inputs, formula shown and shareable results.
The discounted return is a geometric series: constant reward r over T steps sums to r(1 - gamma^T)/(1 - gamma), approaching r/(1 - gamma) as T grows. The quantity 1/(1 - gamma) is the effective horizon, so gamma = 0.99 means the agent effectively plans about 100 steps ahead and rewards beyond that are heavily discounted regardless of the episode length.
Discounted return
G = r (1 - gamma^T) / (1 - gamma); infinite horizon = r / (1 - gamma); effective horizon = 1 / (1 - gamma)
Set it so the effective horizon covers the delay between action and consequence. Credit assignment over 1,000 steps needs gamma around 0.999; short episodic tasks work fine at 0.9.
Undiscounted returns diverge for continuing tasks and make value estimates unstable. Discounting also encodes a genuine preference for sooner rewards.