<aside> 💡

As we discussed earlier in the module, feature scaling is the process of standardising or normalising numerical input variables to ensure that they are on a similar scale.

</aside>

While logistic regression does not require feature scaling for model accuracy (since it’s not distance-based like k-NN or Support Vector Machines), scaling can still improve optimisation and model performance in certain scenarios.

Why is Feature Scaling Important?

1. Improves Convergence Speed – logistic regression is trained using gradient descent, which can be slow when features have vastly different scales. Scaling helps gradient descent converge faster and more smoothly.
2. Prevents Numerical Instability – large feature values can cause computational issues when calculating the log odds in logistic regression. Scaling reduces this risk.
3. Enhances Interpretability with Regularisation – when using L1 (Lasso) or L2 (Ridge) regularisation, scaling ensures that all features contribute equally to the penalty term, preventing features with larger magnitudes from dominating the model.

Common Feature Scaling Methods

• Min-Max Scaling: Rescales values to a range of [0,1]:

  $$ x' = \frac{x - \min(x)}{\max(x) - \min(x)} $$

• Standardisation (Z-score normalisation): Transforms data to have a mean of 0 and standard deviation of 1:

  $$ x′=x−μσx' = \frac{x - \mu}{\sigma} $$

When to Scale Features in Logistic Regression

• If using gradient-based optimisation (e.g., SGD).
• When applying regularisation (L1 or L2 penalty).
• When feature magnitudes vary significantly.

While not always necessary, feature scaling can improve efficiency and stability, making it a useful pre-processing step in logistic regression.

Next reading:

Interaction terms