<aside> 💡
Logistic regression requires numerical input features, so categorical variables must be transformed into a suitable format before being used in the model.
</aside>
This process, known as feature engineering for categorical variables, ensures that the model can correctly interpret non-numeric data while preserving meaningful relationships.
One-hot encoding creates binary (0/1) variables for each category. For a categorical feature like Colour (Red, Blue, Green), it transforms into:
| Colour | Red | Blue | Green |
|---|---|---|---|
| Red | 1 | 0 | 0 |
| Blue | 0 | 1 | 0 |
| Green | 0 | 0 | 1 |
Pros: Works well for nominal (unordered) categories.
Cons: Increases dimensionality when there are many unique categories.
To avoid redundancy, one column is usually dropped (e.g., drop "Green") to prevent perfect multicollinearity.
For ordinal variables (where categories have a meaningful order), values are mapped to integers. Example for Size (Small, Medium, Large):
| Size | Encoded Value |
|---|---|
| Small | 1 |
| Medium | 2 |
| Large | 3 |
Pros: Retains the order of categories.
Cons: Assumes that the numeric distances between categories are meaningful, which may not always be the case.
Replaces each category with its frequency in the dataset. Example for Product Category:
| Product | Frequency |
|---|---|
| A | 0.4 |
| B | 0.3 |
| C | 0.3 |
Pros: Captures the importance of each category.