<aside> 💡

Standardisation transforms data to have a mean of 0 and a standard deviation of 1.

</aside>

Introduction

Standardisation allows us to create datasets with comparable feature distributions, while still maintaining the shape of the original data distribution.

Standardisation is commonly used in several scenarios:

• When working with machine learning algorithms that are sensitive to the scale of input features (e.g., gradient descent-based algorithms, neural networks)
• For comparing features that are measured in different units or scales (e.g., comparing age with income in a dataset)
• When using distance-based algorithms like k-nearest neighbors (KNN) or k-means clustering
• In principal component analysis (PCA) and other dimensionality reduction techniques
• When dealing with regularisation in regression models (e.g., Lasso, Ridge regression)

Standardisation vs. Scaling

graph LR;
    A["Original Data
    1, 5, 10, 100"] --> B["Standardisation"];
    A --> C["Scaling"];
    B --> D["Z-scores
    -0.9, -0.7, -0.5, 2.1"];
    C --> E["[0,1] Range
    0, 0.04, 0.09, 1.0"];
    
    style A fill:#f9f9f9
    style B fill:#e6f3ff
    style C fill:#fff0e6
    style D fill:#e6f3ff
    style E fill:#fff0e6
    
    %% Standardisation preserves outliers
    %% Scaling compresses the range

While standardisation and scaling are both data transformation techniques, they serve different purposes:

• Standardisation transforms data to have a mean of 0 and standard deviation of 1, preserving outliers and the shape of the distribution;
• Scaling transforms data to a fixed range (typically [0,1]), which can compress the effect of outliers but changes the data distribution.

Example

Content map

To learn more about standardisation, read each of the following sections:

Why is have a ‘zero mean’ useful?

What is ‘unit variance’?

Z-score standardisation

The effect of standardisation on feature distribution

Handling ‘unbounded’ data

Next element

Normalisation