In logistic regression, ‘model fitting’ is the process of determining the best values for the model’s coefficients to accurately predict the probability of a binary outcome.
For example, in predicting whether a basketball team will win a game (binary outcome), we need to find the best coefficient values for factors like average points scored, rebounds, and shooting percentage. If a team scores an average of 110 points with a 48% shooting percentage, these values would be used with their respective coefficients to calculate the probability of winning.
Basically, our goal is to align the model’s predicted probabilities with the observed data as closely as possible.
This is typically achieved using Maximum Likelihood Estimation (MLE), which finds the parameters that maximise the likelihood of observing the given data. When MLE cannot be solved analytically, numerical optimisation methods like Gradient Descent or Iterative Reweighted Least Squares (IRLS) are used.
A well-fitted model balances accuracy and generalisability, avoiding overfitting to the training data. Techniques like L1 (Lasso) and L2 (Ridge) regularization further improve fitting by simplifying the model and addressing overfitting or multicollinearity. The gives us an interpretable model that reliably estimates the probability of an outcome based on predictor variables.
To learn more about model fitting, read each of the following sections: