Reinforcement Learning Reward Discount Calculator

Work out reinforcement learning reward discount instantly with clear inputs, formula shown and shareable results.

About Reinforcement Learning Reward Discount Calculator

The discounted return is a geometric series: constant reward r over T steps sums to r(1 - gamma^T)/(1 - gamma), approaching r/(1 - gamma) as T grows. The quantity 1/(1 - gamma) is the effective horizon, so gamma = 0.99 means the agent effectively plans about 100 steps ahead and rewards beyond that are heavily discounted regardless of the episode length.

Formula

Discounted return

G = r (1 - gamma^T) / (1 - gamma); infinite horizon = r / (1 - gamma); effective horizon = 1 / (1 - gamma)

Worked example

With the default values below, Reinforcement Learning Reward Discount Calculator returns Discounted return over the horizon of 99.3430. Change any input above and every figure here is recalculated in your browser.

Inputs used

Reward per step: 1
Discount factor gammaaccepts 0.01–1: 0.99
Horizon in stepsminimum 1: 500

Results

Discounted return over the horizon: 99.3430
Infinite-horizon value: 100.0000
Effective horizon 1/(1-gamma): 100.00 steps
Weight given to the final step: 0.00657048

Step by step

1Values used
2Discounted return
3Discounted return over the horizon= 99.3430
4Infinite-horizon value= 100.0000
5Effective horizon 1/(1-gamma)= 100.00 steps
6Weight given to the final step= 0.00657048

Assumptions and limits

The result is derived only from the 3 inputs listed above. Anything not among them is not part of the calculation.
In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
Figures are rounded for display to 8 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.

How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.

Frequently Asked Questions

How should gamma be chosen?

Set it so the effective horizon covers the delay between action and consequence. Credit assignment over 1,000 steps needs gamma around 0.999; short episodic tasks work fine at 0.9.

Why not use gamma = 1?

Undiscounted returns diverge for continuing tasks and make value estimates unstable. Discounting also encodes a genuine preference for sooner rewards.