Calculate the information gain of a feature split used in decision tree models.
Information gain measures how much a split reduces uncertainty about the class label: IG = H(parent) − Σ (|child_i|/|parent|)·H(child_i), where H is Shannon entropy and the sum is weighted by each child node's share of the parent's samples. Decision tree algorithms like ID3 and C4.5 evaluate information gain for every candidate feature/split and choose the one that maximizes it — a higher information gain means the split produces child nodes that are more 'pure' (dominated by a single class) than the parent was.
IG = H(parent) - sum((|child_i| / |parent|) × H(child_i))
It means the split provided no reduction in entropy at all — the child nodes have the same class-mix uncertainty as the parent, so the feature used for that split carries no useful information for classification at that node.
A split that creates one large, pure child and one tiny, impure child is different from one that creates two equally impure medium-sized children — weighting by |child|/|parent| ensures that entropy reduction is measured proportionally to how many samples end up in each branch.
Yes — features with many distinct values can artificially produce high information gain by creating many small, pure child nodes; the Gain Ratio (information gain normalized by split entropy) is often used instead to correct for this bias, as done in the C4.5 algorithm.