Calculate the symmetric Jensen-Shannon divergence between two probability distributions.
Jensen-Shannon divergence fixes KL divergence's asymmetry by measuring both distributions against their midpoint mixture M = (P+Q)/2: JSD = 0.5·KL(P‖M) + 0.5·KL(Q‖M). Because it averages two KL terms computed against a shared reference, JSD(P,Q) = JSD(Q,P) always — it is symmetric — and it is also bounded (between 0 and ln(2) when using natural log), unlike KL divergence which can grow unboundedly large. Its square root, √JSD, is additionally a true mathematical distance metric, which makes it popular for comparing distributions in generative models like GANs.
JSD(P, Q) = 0.5 × KL(P||M) + 0.5 × KL(Q||M), where M = (P + Q) / 2
JSD is symmetric (JSD(P,Q) = JSD(Q,P)) and always finite/bounded, whereas KL divergence is asymmetric and can become infinite when Q assigns near-zero probability to an event P considers likely — JSD avoids this by comparing both distributions to their shared mixture instead of directly to each other.
Using natural log, JSD is bounded between 0 and ln(2) ≈ 0.693, reached when P and Q have completely disjoint support (no overlapping probability mass) — a value of 0 means the distributions are identical.
It's the theoretical basis for the original GAN (Generative Adversarial Network) loss function, and is also commonly used to measure semantic similarity between topic distributions in NLP and to detect data/label drift between two sampled datasets.