Relevance

The sigmoid function is fundamental to logistic regression for several key reasons:

• It transforms linear predictions into probability values between 0 and 1, making it perfect for binary classification problems;
• The smooth, continuous nature of the function allows for effective gradient-based optimisation during model training; and
• Its derivative has a simple form, which makes it computationally efficient for calculating gradients in the learning process.

In logistic regression, the sigmoid function acts as the activation function that converts the linear combination of inputs ($wx + b$) into a probability estimate:

$$ P(y=1|x) = \frac{1}{1 + e^{-(wx + b)}} $$

This probability output can then be used to make binary predictions by applying a threshold (typically 0.5).

What is the Sigmoid Function?

The sigmoid function, also known as the logistic function, is an S-shaped curve that maps any real-valued number into a value between 0 and 1. It is defined by the formula:

$$ σ(x) = \frac{1}{1 + e^{-x}} $$

Applications

The sigmoid function is widely used in:

• Machine Learning, particularly in logistic regression and neural networks;
• Binary classification, for converting continuous values to binary outcomes; and
• Probability estimation, when modelling probability distributions.

graph LR
    subgraph Sigmoid Function
    Input["-∞ to +∞"] --> Process["1 / (1 + e^-x)"] --> Output["0 to 1"]
    end
    style Process fill:#f9f,stroke:#333,stroke-width:2px
    style Input fill:#bbf,stroke:#333,stroke-width:2px
    style Output fill:#bfb,stroke:#333,stroke-width:2px

Next reading: