import pandas as pd
import numpy as np
import re
from [Link] import LabelEncoder
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.model_selection import train_test_split
from [Link] import pad_sequences
from [Link] import Sequential
from [Link] import Embedding, SimpleRNN, Dense
# Load Dataset
df = pd.read_csv("/content/mail_data.csv")
texts = df["Message"].astype(str).tolist()
labels = df["Category"].astype(str).tolist()
# Encode Labels: "ham" -> 0, "spam" -> 1
label_encoder = LabelEncoder()
labels_encoded = label_encoder.fit_transform(labels)
# Basic Text Preprocessing (lowercase + remove punctuation)
def preprocess(text):
text = [Link]()
text = [Link](r'[^a-zA-Z\s]', '', text) # Remove punctuation
return text
texts = [preprocess(text) for text in texts]
# Use CountVectorizer to tokenize and build vocabulary
vectorizer = CountVectorizer()
[Link](texts)
vocab = vectorizer.get_feature_names_out()
vocab_size = len(vocab) + 1 # +1 for padding
# Convert texts to sequences of word indices
word_to_index = {word: idx+1 for idx, word in enumerate(vocab)} # reserve
0 for padding
sequences = []
for text in texts:
tokens = [Link]()
sequence = [word_to_index.get(token, 0) for token in tokens]
[Link](sequence)
# Pad sequences
maxlen = 100 # choose an appropriate max length
X_padded = pad_sequences(sequences, maxlen=maxlen, padding='post')
y = [Link](labels_encoded)
# Train-Test Split
X_train, X_test, y_train, y_test = train_test_split(X_padded, y,
test_size=0.2, random_state=42)
# Build Simple RNN Model
embedding_dim = 50 # since we are using one-hot-like integer
representation, embedding can still be learned
model = Sequential([
Embedding(input_dim=vocab_size, output_dim=embedding_dim,
input_length=maxlen),
SimpleRNN(64, activation='tanh'),
Dense(1, activation='sigmoid')
])
[Link](loss='binary_crossentropy', optimizer='adam',
metrics=['accuracy'])
print([Link]())
# Train Model
history = [Link](
X_train, y_train,
epochs=3,
batch_size=32,
validation_data=(X_test, y_test)
)
# Evaluate
loss, accuracy = [Link](X_test, y_test)
print(f"Test Accuracy: {accuracy:.4f}")
1. Data Loading and Label Encoding
o Correctly loads and encodes "ham" as 0 and "spam" as 1 using LabelEncoder().
-----------------------
Note: 1. LabelEncoder (from [Link])
Converts text labels (like "ham" or "spam") into numeric form that the model can
understand.
Example:
from [Link] import LabelEncoder
encoder = LabelEncoder()
labels = ["spam", "ham", "spam", "ham"]
encoded = encoder.fit_transform(labels)
print(encoded)
# Output: [1, 0, 1, 0]
-----------------------
2. Preprocessing
o Converts text to lowercase and removes punctuation.
o Keeps only alphabetic characters (a-zA-Z) — suitable for simple spam filtering.
-----------------------
Note:
[Link]()
Converts all characters to lowercase.
Purpose: ensures consistency — "Spam", "SPAM", and "spam" are treated as the same
word.
[Link](r'[^a-zA-Z\s]', '', text)
The pattern [^a-zA-Z\s] means:
^ → not
a-zA-Z → letters
\s → spaces
So, it removes numbers, punctuation, special symbols, etc.
Example:
import re
text = "Hello!!! How are you? Call me @ 123."
clean_text = [Link](r'[^a-zA-Z\s]', '', text)
print(clean_text)
# Output: "Hello How are you Call me "
Return text
texts = ["Hey!!! Call me at 9am.", "WIN cash prize $$$ now!!!"]
texts = [preprocess(text) for text in texts]
print(texts)
Output: ['hey call me at am', 'win cash prize now']
3. Vectorization and Vocabulary Building
o Builds a vocabulary using CountVectorizer.
o Manually maps words to indices (word_to_index) for embedding input.
o This approach is clean and custom, avoiding dependency on Tokenizer from
Keras (though you could use it too).
-----
Note:
texts = ["I love Python", "Python loves me"]
vectorizer = CountVectorizer()
Tokenize the text (split into words)
Build a vocabulary (unique list of words)
Count how many times each word appears in each text
[Link](texts)
- Analyzes all the input texts and builds the vocabulary of all unique words found.
"i" → 0
"love" → 1
"python" → 2
"loves" → 3
"me" → 4
This retrieves the list of all words (features) learned from the dataset.
vocab = vectorizer.get_feature_names_out()
print(vocab)
# Output: ['i', 'love', 'loves', 'me', 'python']
4. Sequence Padding
o Pads all sequences to the same length (maxlen=100), preparing them for RNN
input.
-----
Note:
# Convert texts to sequences of word indices
word_to_index = {word: idx+1 for idx, word in enumerate(vocab)} # reserve
0 for padding
sequences = []
for text in texts:
tokens = [Link]()
sequence = [word_to_index.get(token, 0) for token in tokens]
[Link](sequence)
texts = ["i love python", "python loves me"]
vocab = ['i', 'love', 'loves', 'me', 'python']
word_to_index = {'i': 1, 'love': 2, 'loves': 3, 'me': 4, 'python': 5}
sequences = []
for text in texts:
tokens = [Link]()
sequence = [word_to_index.get(token, 0) for token in tokens]
[Link](sequence)
print(sequences)
# Output: [[1, 2, 5], [5, 3, 4]]
# Pad sequences maxlen = 100
# choose an appropriate max length
X_padded = pad_sequences(sequences, maxlen=maxlen, padding='post')
y = [Link](labels_encoded)
-----------
Note:
from [Link] import pad_sequences
sequences = [
[1, 2, 3],
[4, 5],
[6]
]
X_padded = pad_sequences(sequences, maxlen=5, padding='post')
print(X_padded)
Output:
[[1 2 3 0 0]
[4 5 0 0 0]
[6 0 0 0 0]]
---------------
# Build Simple RNN Model
embedding_dim = 50 # since we are using one-hot-like integer
representation, embedding can still be learned
Word Integer Index Embedding Vector (50 dimensions)
“free” 1234 [0.23, -0.45, 0.17, ..., 0.09]
“win” 5678 [0.25, -0.47, 0.19, ..., 0.12]
“offer” 322 [0.21, -0.40, 0.15, ..., 0.11]
More details on Embedding
Problem: Words Are Not Numbers
Neural networks can only work with numbers — they cannot directly process words like:
["I", "love", "pizza"]
So we first convert each word into an integer index, for example:
I→1
love → 2
pizza → 3
Now we have:
[1, 2, 3]
But there’s a big problem
Why Integers Alone Are Not Enough
Integers just represent IDs, not meanings.
Word Index Problem
“good” 45 Numbers have no relation — model can’t know that “good” and “great” are similar
“great” 128
“bad” 302
To the model:
45, 128, and 302 are just arbitrary numbers.
There’s no semantic relationship (i.e., no sense that good ≈ great and bad ≠ good).
If we use only integer encoding, the model might incorrectly think:
“great” (128) is closer to “bad” (302) than to “good” (45) — just because 302 - 128 < 128 - 45.
Solution: Word Embeddings
A word embedding turns each word into a dense vector of real numbers — typically 50, 100, or
300 dimensions.
For example:
Word Embedding Vector (5 dimensions for illustration)
good [0.8, 0.3, 0.1, 0.2, 0.9]
great [0.79, 0.32, 0.08, 0.25, 0.88]
bad [-0.7, -0.3, -0.2, -0.4, -0.8]
Now:
“good” and “great” → similar vectors (they point in similar directions)
“bad” → different vector (points in the opposite direction)
Why This Helps
The embedding layer captures meaning, context, and relationships between words.
For example, after training:
Similarity: cosine_similarity("king", "queen") > cosine_similarity("king", "car")
Analogy:
“king” - “man” + “woman” ≈ “queen”
So the model understands semantic relationships in a numeric form.
5. Model Architecture
o Uses:
Embedding Layer — learns dense vector representations for words.
SimpleRNN Layer (64 units) — captures sequential dependencies.
Dense Output Layer — sigmoid activation for binary classification.
o Compiled with binary crossentropy and Adam optimizer, which are ideal for this
problem.
6. Training & Evaluation
o Splits dataset into train/test (80/20).
o Trains and evaluates accuracy.
Suggested Improvements
These tweaks can make your model more robust and improve accuracy:
Use Keras Tokenizer (Optional Simplification)
Instead of CountVectorizer + manual mapping:
from [Link] import Tokenizer
tokenizer = Tokenizer(num_words=5000, oov_token="<OOV>")
tokenizer.fit_on_texts(texts)
sequences = tokenizer.texts_to_sequences(texts)
word_index = tokenizer.word_index
vocab_size = len(word_index) + 1
Use LSTM or GRU for Better Context Understanding
SimpleRNN struggles with long-term dependencies. Replace it with:
from [Link] import LSTM
model = Sequential([
Embedding(input_dim=vocab_size, output_dim=embedding_dim,
input_length=maxlen),
LSTM(64, dropout=0.2, recurrent_dropout=0.2),
Dense(1, activation='sigmoid')
])
Increase Epochs
Train longer (5–10 epochs) for better convergence:
history = [Link](X_train, y_train, epochs=10, batch_size=32,
validation_data=(X_test, y_test))
Add Early Stopping
Prevent overfitting:
from [Link] import EarlyStopping
early_stop = EarlyStopping(monitor='val_loss', patience=2,
restore_best_weights=True)
history = [Link](X_train, y_train, epochs=10, batch_size=32,
validation_data=(X_test, y_test),
callbacks=[early_stop])
Performance Visualization (Optional)
To visualize training progress:
import [Link] as plt
[Link]([Link]['accuracy'], label='train accuracy')
[Link]([Link]['val_accuracy'], label='val accuracy')
[Link]()
[Link]()
Expected Results
For the standard [Link] dataset (5,000 messages), you can expect:
SimpleRNN: ~93–95% accuracy
LSTM: ~97–99% accuracy
GRU: ~97–98% accuracy