“Gradient Descent” is an optimisation algorithm used to estimate the coefficients in logistic regression, particularly when Maximum Likelihood Estimation cannot be solved analytically.
The method works iteratively: it starts with initial guesses for the parameters and gradually updates them by moving in the direction that reduces the model’s error. The "gradient" indicates the slope or rate of change of the loss function, guiding how the parameters are adjusted.
Step size, or learning rate, is crucial to ensure the method converges efficiently to the optimal solution without overshooting or stalling.
Gradient Descent is especially useful when we’re dealing with large datasets or complex models.
Next reading: