Logistic Regression — Step-by-Step Visualization

Algorithm Pattern

Sigmoid + Binary Cross-Entropy + Gradient Descent

Key Idea

Logistic regression applies the sigmoid function to a linear combination of features, squashing predictions to (0,1) and using cross-entropy loss for gradient-based training.

Step-by-Step Approach

1. Compute z = w·x + b for each training example.
2. Apply sigmoid: ŷ = 1 / (1 + e^(−z)) to get a class probability.
3. Compute binary cross-entropy loss: L = −mean[y·log(ŷ) + (1−y)·log(1−ŷ)].
4. Compute gradients: dw = mean((ŷ−y)·x) and db = mean(ŷ−y).
5. Update weights: w −= lr·dw, b −= lr·db.
6. The decision boundary is where ŷ = 0.5, i.e. x = −b/w.

Common Gotchas

• The decision boundary is linear — logistic regression cannot learn non-linear boundaries.
• Cross-entropy penalizes confident wrong predictions severely (log(0) → −∞).
• Feature scaling (standardization) helps gradient descent converge much faster.