<aside> 💡

Polynomial features transform the original input variables into higher-degree terms, allowing logistic regression to capture nonlinear relationships between features and the target variable.

</aside>

While standard logistic regression models linear decision boundaries, adding polynomial terms can introduce curvature, making the model more flexible for complex datasets.

How Polynomial Features Work

Given an input feature $x$, polynomial transformation creates additional features:

For degree 2:

$$ (x) \quad \rightarrow \quad (x, x^2) $$

For degree 3:

$$ (x) \quad \rightarrow \quad (x, x^2, x^3) $$

For multiple features $(x_1, x_2),$ it also includes interaction terms:

$$ (x_1, x_2) \quad \rightarrow \quad (x_1, x_2, x_1^2, x_2^2, x_1 x_2) $$

Why Use Polynomial Features in Logistic Regression?

• Handles Nonlinearity – If the relationship between predictors and the outcome is not purely linear, polynomial features allow logistic regression to fit more complex decision boundaries.
• Enhances Model Flexibility – Captures interactions and curved relationships without needing a non-linear model like neural networks.
• Works with Regularization – Polynomial terms can be penalized with L1/L2 regularization to prevent overfitting.

Limitations

• Computational Complexity – Higher-degree polynomials create many new features, increasing model complexity and computation time.
• Overfitting Risk – Too many polynomial terms can cause the model to fit noise in the training data instead of generalising well.
• Feature Scaling Needed – Polynomial features often lead to large feature magnitudes, so scaling is recommended to improve optimization stability.

When to Use Polynomial Features