Support Vector Machine — Step-by-Step Visualization

Algorithm Pattern

Maximum Margin Hyperplane

Key Idea

An SVM finds the hyperplane that maximizes the margin between classes, making it robust to noise — only the support vectors (closest points) determine the boundary.

Step-by-Step Approach

1. Find all hyperplanes that correctly separate the two classes.
2. For each class, identify the support vectors — the closest points to the boundary.
3. The margin is 2 / ‖w‖ where w is the hyperplane's normal vector.
4. The optimal hyperplane maximizes the margin, equivalent to minimizing ‖w‖².
5. Only support vectors matter — removing any other point leaves the boundary unchanged.

Common Gotchas

• The kernel trick maps data to higher dimensions to handle non-linear boundaries.
• Soft-margin SVMs (C parameter) allow some misclassifications for non-separable data.
• SVMs scale poorly to very large datasets — use kernel approximations for scale.