Naive Bayes Classifier

Write a program to implement the naïve Bayesian classifier for a sample training data set stored as a CSV file. Compute the accuracy of the classifier, considering few test data sets.

We can use a simple Weather → Play Tennis dataset. The program will:

1. Create/read a CSV file.
2. Separate input features and target class.
3. Split the data into training and testing data.
4. Train a Naïve Bayes classifier.
5. Predict the class for test records.
6. Calculate accuracy.
7. Test with new data

Import Required Libraries

import pandas as pd

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
from sklearn.naive_bayes import GaussianNB
from sklearn.metrics import confusion_matrix
from sklearn.metrics import accuracy_score

Explanation

1. pandas → reads and handles CSV data.
2. train_test_split → divides data into training and testing data.
3. LabelEncoder → converts text into numbers.
4. GaussianNB → Naïve Bayes classifier.
5. accuracy_score → calculates classification accuracy.

import csv

data = [
    ["Outlook", "Temperature", "Humidity", "Windy", "Play"],
    ["Sunny", "Hot", "High", "False", "No"],
    ["Sunny", "Hot", "High", "True", "No"],
    ["Overcast", "Hot", "High", "False", "Yes"],
    ["Rain", "Mild", "High", "False", "Yes"],
    ["Rain", "Cool", "Normal", "False", "Yes"],
    ["Rain", "Cool", "Normal", "True", "No"],
    ["Overcast", "Cool", "Normal", "True", "Yes"],
    ["Sunny", "Mild", "High", "False", "No"],
    ["Sunny", "Cool", "Normal", "False", "Yes"],
    ["Rain", "Mild", "Normal", "False", "Yes"],
    ["Sunny", "Mild", "Normal", "True", "Yes"],
    ["Overcast", "Mild", "High", "True", "Yes"],
    ["Overcast", "Hot", "Normal", "False", "Yes"],
    ["Rain", "Mild", "High", "True", "No"]
]

with open("weather.csv", "w", newline="") as file:
    writer = csv.writer(file)
    writer.writerows(data)

print("weather.csv created successfully!")

weather.csv created successfully!

Read the CSV file

Reads the CSV file and stores the data in a Pandas DataFrame.

df means DataFrame.

df = pd.read_csv("weather.csv")

print(df)

     Outlook Temperature Humidity  Windy Play
0      Sunny         Hot     High  False   No
1      Sunny         Hot     High   True   No
2   Overcast         Hot     High  False  Yes
3       Rain        Mild     High  False  Yes
4       Rain        Cool   Normal  False  Yes
5       Rain        Cool   Normal   True   No
6   Overcast        Cool   Normal   True  Yes
7      Sunny        Mild     High  False   No
8      Sunny        Cool   Normal  False  Yes
9       Rain        Mild   Normal  False  Yes
10     Sunny        Mild   Normal   True  Yes
11  Overcast        Mild     High   True  Yes
12  Overcast         Hot   Normal  False  Yes
13      Rain        Mild     High   True   No

Separate input and output

Our dataset has four input attributes:

1. Outlook
2. Temperature
3. Humidity
4. Windy

and one output: Play

X_features = df.drop("Play", axis=1)
y_target = df["Play"]

X_features contains the input features:

1. Outlook
2. Temperature
3. Humidity
4. Windy

y_target contains the target/output:

Play

So:

X_features → Features/Input

y_target → Target/Output

Convert text into numbers

Naïve Bayes in this example needs numerical input.

We need to convert these categories into numbers.

label_encoders = {}

for column in X_features.columns:
    le = LabelEncoder()
    X_features[column] = le.fit_transform(X_features[column])
    label_encoders[column] = le

label_encoder_y = LabelEncoder()
y_target = label_encoder_y.fit_transform(y_target)

print("Features")
print(X_features)
print("target")
print(y_target)

Features
    Outlook  Temperature  Humidity  Windy
0         2            1         0      0
1         2            1         0      1
2         0            1         0      0
3         1            2         0      0
4         1            0         1      0
5         1            0         1      1
6         0            0         1      1
7         2            2         0      0
8         2            0         1      0
9         1            2         1      0
10        2            2         1      1
11        0            2         0      1
12        0            1         1      0
13        1            2         0      1
target
[0 0 1 1 1 0 1 0 1 1 1 1 1 0]

No → 0

Yes → 1

NOTE: LabelEncoder is simply assigning a number to each category.

For example:

Sunny → 2

Overcast → 0

Rain → 1

The exact numbers are not important; they are simply numerical representations of the categories.

Divide the dataset into training and testing data

X_train, X_test, y_train, y_test = train_test_split(
    X_features,
    y_target,
    test_size=0.30,
    random_state=42
)

# 30% → Testing data
# 70% → Training data

Why do we split the data?

The classifier needs to learn from one set of records and then be tested on records that it did not use for learning.

Create the Naïve Bayes classifier

We can create a Naïve Bayes classifier using GaussianNB.

GaussianNB is the Gaussian Naïve Bayes implementation available in scikit-learn.

model = GaussianNB()

Train the classifier

This is the most important step.

The classifier examines the training data and learns the relationship between:

Features → Class

For example:

Sunny + Hot + High + False → No

Rain + Mild + Normal + False → Yes

model.fit(X_train, y_train)

GaussianNB()

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Make predictions

Now give the classifier the testing data:

y_pred = model.predict(X_test)

print(y_test)
print(y_pred)

[1 1 0 1 0]
[1 0 0 1 1]

Calculate accuracy

accuracy = accuracy_score(y_test, y_pred)

print("Accuracy:", accuracy)
print("Accuracy percentage:", accuracy * 100, "%")

Accuracy: 0.6
Accuracy percentage: 60.0 %

Create Confusion Matrix

cm = confusion_matrix(
    y_test,
    y_pred
)

print(cm)

[[1 1]
 [1 2]]

	Predicted: No	Predicted: Yes
Actual: No	1 (True Negative - TN)	1 (False Positive - FP)
Actual: Yes	1 (False Negative - FN)	2 (True Positive - TP)

Test with new data

new_data = pd.DataFrame({
    "Outlook": ["Sunny", "Rain", "Overcast"],
    "Temperature": ["Cool", "Mild", "Hot"],
    "Humidity": ["High", "Normal", "High"],
    "Windy": [False, False, False]
})

print("\nNew Test Data:")
print(new_data)

New Test Data:
    Outlook Temperature Humidity  Windy
0     Sunny        Cool     High  False
1      Rain        Mild   Normal  False
2  Overcast         Hot     High  False

# --------------------------------------------------
# 10. Encode NEW test data
# --------------------------------------------------

new_data_encoded = new_data.copy()

for column in new_data_encoded.columns:

    new_data_encoded[column] = label_encoders[column].transform(
        new_data_encoded[column]
    )


print("\nEncoded New Test Data:")
print(new_data_encoded)

Encoded New Test Data:
   Outlook  Temperature  Humidity  Windy
0        2            0         0      0
1        1            2         1      0
2        0            1         0      0

# --------------------------------------------------
# Predict new data
# --------------------------------------------------

predictions = model.predict(new_data_encoded)


# --------------------------------------------------
# Convert 0/1 back to Yes/No
# --------------------------------------------------

predicted_classes = label_encoder_y.inverse_transform(predictions)


print("\nPredictions:")

for i in range(len(predicted_classes)):

    print(
        new_data.iloc[i].to_dict(),
        "=>",
        predicted_classes[i]
    )

Predictions:
{'Outlook': 'Sunny', 'Temperature': 'Cool', 'Humidity': 'High', 'Windy': False} => No
{'Outlook': 'Rain', 'Temperature': 'Mild', 'Humidity': 'Normal', 'Windy': False} => Yes
{'Outlook': 'Overcast', 'Temperature': 'Hot', 'Humidity': 'High', 'Windy': False} => No

Accuracy on manually supplied new test data

accuracy_score(actual_classes, predicted_classes)

This is possible only when you know the actual class labels for those new records.

If you don't know the actual answers, you can show the predictions, but you cannot calculate accuracy.

from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report

cm = confusion_matrix(y_test, y_pred)

print(cm)

[[1 1]
 [1 2]]