Information Gain Calculator
Calculate the information gain of a feature split used in decision tree models.
About Information Gain Calculator
Information gain measures how much a split reduces uncertainty about the class label: IG = H(parent) − Σ (|child_i|/|parent|)·H(child_i), where H is Shannon entropy and the sum is weighted by each child node's share of the parent's samples. Decision tree algorithms like ID3 and C4.5 evaluate information gain for every candidate feature/split and choose the one that maximizes it — a higher information gain means the split produces child nodes that are more 'pure' (dominated by a single class) than the parent was.
Formula
IG = H(parent) - sum((|child_i| / |parent|) × H(child_i))
- H(parent)
- Entropy of the parent node
- H(child_i)
- Entropy of child node i
- |child_i|
- Number of samples in child i
- |parent|
- Number of samples in parent
Worked example
With the default values below, Information Gain Calculator returns Information Gain of 0.256426. Change any input above and every figure here is recalculated in your browser.
Inputs used
- Parent Node Class Counts (comma-separated)
- 30, 20
- Child 1 Class Counts (comma-separated)
- 25, 5
- Child 2 Class Counts (comma-separated)
- 5, 15
Results
- Information Gain
- 0.256426 bits
- Parent Entropy
- 0.970951 bits
- Weighted Child Entropy
- 0.714525 bits
Step by step
- 1Parent entropy: H(parent)= 0.970951 bits
- 2Child entropies= 0.650022, 0.811278 bits
- 3Weighted child entropy: Σ (|child_i| / |parent|) × H(child_i)= 0.714525 bits
- 4Information gain: H(parent) − weighted child entropy= 0.256426 bits
Assumptions and limits
- The result is derived only from the 3 inputs listed above. Anything not among them is not part of the calculation.
- In particular, this calculator does not take inflation, tax, fees or charges or compounding frequency into account.
- Figures are rounded for display to 6 decimal places, so totals built from several rounded parts can differ slightly from the unrounded arithmetic.
How these calculators are built and tested is described on the Methodology page. If a figure looks wrong, please report it.
Frequently Asked Questions
What does an information gain of 0 mean?
It means the split provided no reduction in entropy at all — the child nodes have the same class-mix uncertainty as the parent, so the feature used for that split carries no useful information for classification at that node.
Why is information gain weighted by child size?
A split that creates one large, pure child and one tiny, impure child is different from one that creates two equally impure medium-sized children — weighting by |child|/|parent| ensures that entropy reduction is measured proportionally to how many samples end up in each branch.
Is information gain biased toward features with many categories?
Yes — features with many distinct values can artificially produce high information gain by creating many small, pure child nodes; the Gain Ratio (information gain normalized by split entropy) is often used instead to correct for this bias, as done in the C4.5 algorithm.