Load dataset
↓
Understand the data
↓
Split into training and testing data
↓
Create SVM model
↓
Train model
↓
Make predictions
↓
Evaluate accuracy
↓
Confusion matrix / classification report
↓
Predict a new flowerSupport Vector Machine (SVM) is a supervised machine-learning algorithm mainly used for:
For today's hands-on, we will use SVM for classification.
Imagine we have two types of data:
○ ○ ○
○ ○ ○
-------------------- ← decision boundary
× × ×
× × × ×The SVM tries to find a boundary that separates the classes.
The important idea is:
SVM tries to find the best separating boundary with the largest possible margin between classes.
In two dimensions, the boundary is a line:
For more dimensions, we call the separating boundary a hyperplane.
For a beginner, remember:
Hyperplane = decision boundary used by SVM to separate classes.
Class A | Class B
○ ○ ○ | × × ×
○ ○ ○ | × × ×
○ ○ ○ | × × ×
↑
Decision boundarySuppose we have:
○ ○ ○ | × × ×
○ ○ ○ | × × ×
|SVM doesn't simply want any line that separates the classes.
It wants a boundary that gives the classes the largest possible margin.
Conceptually:
Class A Class B
○ ○ ○ × × ×
○ ○ ○ × × ×
○ ○ × × ×
| |
| |
| |
↑
marginThe data points closest to the decision boundary are called support vectors.
Suppose
○ ○ ○ ○ | × × × ×
|The points closest to the boundary are especially important. They are called: Support vectors
These points help determine where the decision boundary should be.
So remember:
SVM
↓
Find decision boundary
↓
Maximize margin
↓
Important boundary points = Support VectorsIt has four features:
And three classes:
0 → Setosa
1 → Versicolor
2 → Virginica
So our problem is: Given the measurements of an iris flower, predict which species it belongs to.
import pandas as pd
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC ### Support Vector Classifier. This is the built-in scikit-learn implementation we will use.
### Evaluation functions
from sklearn.metrics import accuracy_score
from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report
StandardScaler
from sklearn.preprocessing import StandardScaler
SVM is sensitive to the scale of features.
For example:
Age = 20–60
Salary = 20,000–100,000
The values are on very different scales.
Standardization puts features on a comparable scale.
iris = load_iris()
print(iris)
print(iris.feature_names)
print(iris.target_names)
X_features = iris.data
y_target = iris.target
df = pd.DataFrame(
iris.data,
columns=iris.feature_names
)
df["target"] = iris.target
print(df.head())
X_train, X_test, y_train, y_test = train_test_split(
X_features,
y_target,
test_size=0.2,
random_state=42
)
This step is especially important for SVM.
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
There are two operations here:
fit
↓
Learn scaling parameters
transform
↓
Apply scaling
fit_transform() performs both.X_test = scaler.transform(X_test)
Notice that we do not use fit_transform() on the test data.
Why?
Because the test set should remain independent.
We learn the scaling information from the training data and apply that same transformation to the test data.
So:
Training data
↓
fit + transform
↓
Scaled training data
Testing data
↓
transform only
↓
Scaled testing dataSuppose we have:
Feature A = 1–5 Feature B = 100–10000
Feature B has much larger numerical values.
Since SVM uses distances/margins, feature scale can affect the model.
Standardization generally converts a feature approximately to:
mean = 0 standard deviation = 1
For beginners, remember: Scaling makes the numerical ranges of features comparable, which is important for many SVM applications.
model = SVC(kernel="linear")
### Create an SVM classifier using a linear decision boundary.
This is an important SVM concept.
The kernel determines how SVM creates the decision boundary.
Common options include:
Linear kernel
↓
Find a linear decision boundary
↓
Separate classesmodel.fit(X_train, y_train)
y_pred = model.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)
print("Accuracy:", accuracy)
cm = confusion_matrix(y_test, y_pred)
print(cm)
| Predicted | ||||
|---|---|---|---|---|
| Setosa | Versicolor | Virginica | ||
| Actual | ||||
| 0 | Setosa | 10 | 0 | 0 |
| 1 | Versicolor | 0 | 8 | 1 |
| 2 | Virginica | 0 | 0 | 11 |
print("\nClassification Report:")
print(
classification_report(
y_test,
y_pred,
target_names=iris.target_names
)
)
Precision is the proportion of correct positive predictions among all the positive predictions made by the model.
For example: Model predicts 100 patients → Cancer
90 actually have cancer
10 do not have cancer
So, 90% of the patients predicted as cancer-positive actually have cancer.
Recall is the proportion of correct positive predictions among all the actual positive cases.
For example: 100 patients actually have cancer
Model correctly identifies 95
Model misses 5
So, the model detected 95% of the actual cancer cases.
In cancer screening/classification, a false negative means: The model predicts "no cancer" when cancer is actually present.
# =========================================================
# STEP 1: Import Libraries
# =========================================================
import pandas as pd
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
from sklearn.metrics import accuracy_score
from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report
# =========================================================
# STEP 2: Load Inbuilt Iris Dataset
# =========================================================
iris = load_iris()
# =========================================================
# STEP 3: Understand the Dataset
# =========================================================
print("Feature Names:")
print(iris.feature_names)
print("\nTarget Names:")
print(iris.target_names)
print("\nDataset Shape:")
print(iris.data.shape)
# =========================================================
# STEP 4: Create X and y
# =========================================================
X = iris.data
y = iris.target
print("\nFirst 5 rows:")
print(X[:5])
print("\nFirst 5 target values:")
print(y[:5])
# =========================================================
# STEP 5: Create DataFrame
# =========================================================
df = pd.DataFrame(
iris.data,
columns=iris.feature_names
)
df["target"] = iris.target
print("\nDataset:")
print(df.head())
# =========================================================
# STEP 6: Split Dataset
# =========================================================
X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2,
random_state=42
)
print("\nTraining Data Shape:")
print(X_train.shape)
print("\nTesting Data Shape:")
print(X_test.shape)
# =========================================================
# STEP 7: Standardize Features
# =========================================================
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
# =========================================================
# STEP 8: Create SVM Model
# =========================================================
model = SVC(
kernel="linear"
)
# =========================================================
# STEP 9: Train SVM Model
# =========================================================
model.fit(
X_train,
y_train
)
# =========================================================
# STEP 10: Make Predictions
# =========================================================
y_pred = model.predict(
X_test
)
# =========================================================
# STEP 11: Calculate Accuracy
# =========================================================
accuracy = accuracy_score(
y_test,
y_pred
)
print("\nAccuracy:")
print(f"{accuracy * 100:.2f}%")
# =========================================================
# STEP 12: Confusion Matrix
# =========================================================
cm = confusion_matrix(
y_test,
y_pred
)
print("\nConfusion Matrix:")
print(cm)
# =========================================================
# STEP 13: Classification Report
# =========================================================
print("\nClassification Report:")
print(
classification_report(
y_test,
y_pred,
target_names=iris.target_names
)
)
# =========================================================
# STEP 14: Predict a New Flower
# =========================================================
new_flower = [[
5.1, # Sepal length
3.5, # Sepal width
1.4, # Petal length
0.2 # Petal width
]]
# Scale the new flower
new_flower_scaled = scaler.transform(
new_flower
)
# Predict
prediction = model.predict(
new_flower_scaled
)
# Display prediction
print("\nNew Flower Prediction:")
print(
iris.target_names[prediction[0]]
)