Probability Estimation in Logistic Regression

In logistic regression, the goal is to estimate the probability of an event occurring based on predictor variables.

Since probabilities range between 0 and 1, logistic regression models the probability using the sigmoid function, which ensures the output stays within this range. The probability of success ($p$) given predictor variables is:

$$ p = \frac{1}{1 + e^{-(\beta_0 + \beta_1 x_1 + \beta_2 x_2 + \dots)}} $$

This function transforms a linear combination of predictors into a probability, making logistic regression ideal for binary classification problems (e.g., predicting whether an email is spam or not).

Probability and Odds

<aside> 💡

Instead of predicting probability directly, logistic regression first estimates the log odds and then converts them into probability using the formula above.

</aside>

The relationship between probability ($p$) and odds is:

$$ Odds = \frac{p}{1 - p} $$

Solving for $p$:

$$ p = \frac{e^{(\beta_0 + \beta_1 x_1 + \dots)}}{1 + e^{(\beta_0 + \beta_1 x_1 + \dots)}} $$

This equation shows that as predictor values change, so does the probability of the outcome occurring.

Interpretation in Logistic Regression

The estimated probability helps in decision-making.

• If $p$ is greater than a threshold (commonly 0.5), we classify the outcome as 1 (success); otherwise, it's 0 (failure).
• The logistic model coefficients ($β$) influence how much each predictor contributes to increasing or decreasing the probability.