Calculate the likelihood of observed data given a set of model parameters.
The likelihood function measures how probable the observed data is under a given set of model parameters. For Gaussian-distributed data, the log-likelihood is the sum of log-probabilities of each observation under the assumed normal distribution N(μ, σ²). Maximum Likelihood Estimation (MLE) finds parameters that maximize this value.
Gaussian Log-Likelihood
LL = sum(-0.5*ln(2*pi*sigma^2) - (x_i - mu)^2 / (2*sigma^2))
Multiplying many small probabilities leads to numerical underflow. Taking logs converts products to sums, which is numerically stable and mathematically equivalent for optimization (maximizing log-likelihood maximizes likelihood).
Training with cross-entropy loss is equivalent to maximizing the log-likelihood of the training data under the model's predicted distribution — MLE is the statistical foundation of most supervised learning.