Algorithm

Step 1: Import required libraries

Step 2: Download required NLP resources
Step 3: Input sample text
Step 4: Perform NLTK tokenization
Step 5: Perform regex-based tokenization
Step 6: Remove stopwords
Step 7: Apply stemming
Step 8: Apply lemmatization using NLTK
Step 9: Apply lemmatization using SpaCy
Step 10: Display outputs

Implementation (Code Used)

Output Explanation
Sample Text:
"The cats are running quickly through the beautiful gardens!"

1️⃣ NLTK Tokens

['The', 'cats', 'are', 'running', 'quickly', 'through', 'the', 'beautiful', 'gardens', '!']

2️⃣ Regex Tokens

['the', 'cats', 'are', 'running', 'quickly', 'through', 'the', 'beautiful', 'gardens']

3️⃣ After Stopword Removal

['cats', 'running', 'quickly', 'beautiful', 'gardens']

4️⃣ After Stemming

['cat', 'run', 'quickli', 'beauti', 'garden']

5️⃣ Lemmatization (NLTK)

['cat', 'running', 'quickly', 'beautiful', 'garden']

6️⃣ Lemmatization (SpaCy)

['the', 'cat', 'be', 'run', 'quickly', 'through', 'the', 'beautiful', 'garden']

Observation:
SpaCy provides more accurate lemmatization because it considers Part-of-Speech (POS)
tagging.

Comparison Table
Technique Output Quality Dictionary Words Accuracy
Stemming Moderate No Lower
Lemmatization (NLTK) Better Yes Medium
Lemmatization (SpaCy) Best Yes High

Applications of Preprocessing
 Sentiment Analysis
 Chatbots
 Machine Translation
 Spam Detection
 Toxic Comment Classification
 Information Retrieval

Conclusion
In this experiment, we successfully implemented various basic NLP preprocessing techniques
using both inbuilt libraries and regular expressions.

We observed that:

 Tokenization divides text into meaningful units.

 Stopword removal eliminates unnecessary words.
 Stemming reduces words to root form but may produce non-meaningful words.
 Lemmatization provides accurate base forms of words.
 SpaCy lemmatization is more accurate compared to NLTK.

Thus, preprocessing is an essential step before applying machine learning algorithms in NLP
tasks.

Code Explanation – NLP Preprocessing

Experiment
🔹 1 Importing Libraries
import nltk
import spacy
import re
from [Link] import word_tokenize, sent_tokenize
from [Link] import stopwords
from [Link] import PorterStemmer, WordNetLemmatizer

Explanation:

 nltk → Natural Language Toolkit for NLP operations

 spacy → Advanced NLP library for accurate linguistic processing
 re → Regular Expression module for pattern matching

Specific imports:

 word_tokenize() → splits text into words

 sent_tokenize() → splits text into sentences
 stopwords → contains common English stopwords
 PorterStemmer() → performs stemming
 WordNetLemmatizer() → performs lemmatization

🔹 2 Downloading Required NLTK Resources

[Link]('punkt')
[Link]('stopwords')
[Link]('wordnet')

Explanation:

NLTK requires additional datasets:

 punkt → used for tokenization

 stopwords → list of common English stopwords
 wordnet → lexical database used for lemmatization

These are downloaded once and stored locally.

🔹 3 Loading SpaCy Model

nlp = [Link]('en_core_web_sm')

Explanation:

 Loads English language model.

 Enables advanced NLP tasks like:
o Lemmatization
o POS tagging
 o Named Entity Recognition

🔹 4 Sample Text
sample_text = "The cats are running quickly through the beautiful gardens!"

Explanation:

This is the input sentence on which preprocessing techniques will be applied.

🔹 5Tokenization Using NLTK

words_nltk = word_tokenize(sample_text)
sentences = sent_tokenize(sample_text)
print("NLTK Tokens:", words_nltk)

Explanation:

 word_tokenize() splits sentence into individual words.

 sent_tokenize() splits paragraph into sentences.
 Output includes punctuation.

Example Output:

['The', 'cats', 'are', 'running', ..., '!']

🔹 6 Tokenization Using Regular Expressions

regex_tokens = [Link](r'\b\w+\b', sample_text.lower())
print("Regex Tokens:", regex_tokens)

Explanation:

 sample_text.lower() → converts text to lowercase.

 \b\w+\b → regex pattern:
o \b → word boundary
o \w+ → one or more word characters

This removes punctuation automatically.

Output:

['the', 'cats', 'are', 'running', ...]

🔹 7 Stopword Removal
stop_words = set([Link]('english'))
filtered_words = [w for w in regex_tokens if w not in stop_words]
print("After Stopword Removal:", filtered_words)
Explanation:

 Loads English stopword list.

 Removes common words like:
o the
o are
o through

Uses list comprehension to filter meaningful words.

Output:

['cats', 'running', 'quickly', 'beautiful', 'gardens']

🔹 8⃣ Stemming
stemmer = PorterStemmer()
stemmed = [[Link](word) for word in filtered_words]
print("Stemmed:", stemmed)

Explanation:

 Creates Porter Stemmer object.

 Reduces words to root form by removing suffixes.

Example:

 running → run
 beautiful → beauti
 quickly → quickli

Note:
Stemming may produce non-dictionary words.

🔹 9⃣ Lemmatization Using NLTK

lemmatizer = WordNetLemmatizer()
lemmatized_nltk = [[Link](word) for word in filtered_words]
print("Lemmatized (NLTK):", lemmatized_nltk)

Explanation:

 Uses WordNet database.

 Converts words into meaningful base form.

Example:

 cats → cat
 gardens → garden
More accurate than stemming.

🔹 🔟 Lemmatization Using SpaCy

doc = nlp(sample_text)
lemmatized_spacy = [token.lemma_ for token in doc if not token.is_punct]
print("Lemmatized (SpaCy):", lemmatized_spacy)

Explanation:

 nlp(sample_text) → processes text using SpaCy pipeline.

 token.lemma_ → extracts base form.
 token.is_punct → removes punctuation.

Example:

 are → be
 running → run
 cats → cat

SpaCy considers grammar (POS tagging), so it gives better results.

Why Both NLTK and SpaCy?

Feature NLTK SpaCy

Lightweight Yes No

Speed Moderate Fast

Accuracy Medium High

Uses POS tagging for lemma No (by default) Yes

Final Workflow Summary

Raw Text
↓
Tokenization
↓
Stopword Removal
↓
Stemming
↓
Lemmatization
This converts unstructured text into clean structured tokens ready for machine learning.
 ------------3-------Core Concept
Minimum Edit Distance (MED), also called Levenshtein Distance, calculates the minimum
number of operations required to convert one string into another.

Allowed Operations:

1. Insertion
2. Deletion
3. Substitution

Each operation has cost = 1.

Why Dynamic Programming?

Because:

 The problem has overlapping subproblems

 It follows optimal substructure
 We store intermediate results in a matrix to avoid recomputation

Time Complexity:

O(m×n)O(m \times n)O(m×n)

Where:

 m = length of word1
 n = length of word2

Code Explanation

1️⃣ Import Library

import numpy as np

This imports NumPy.

Note: In this implementation, NumPy is not actually used since we are using a list-based
matrix. It can be removed safely.
2️⃣ Define Function: min_edit_distance()
def min_edit_distance(word1, word2):

This function calculates the edit distance between two words.

3️⃣ Get Length of Both Words

m, n = len(word1), len(word2)

 m → length of first word

 n → length of second word

4️⃣ Create Distance Matrix

dp = [[0 for _ in range(n + 1)] for _ in range(m + 1)]

Creates a matrix of size:

(m+1)×(n+1)(m+1) \times (n+1)(m+1)×(n+1)

Why +1?

To include comparison with empty string.

Example:

For "cat" and "cats":

Matrix size = 4 × 5

5️⃣ Initialize Base Cases

First Column (Deletions)

for i in range(m + 1):
dp[i][0] = i

Represents converting word1 → empty string.

Example:
"cat" → ""
Requires 3 deletions.

First Row (Insertions)

for j in range(n + 1):
dp[0][j] = j

Represents converting empty string → word2.

Example:
"" → "cats"
Requires 4 insertions.

6️⃣ Fill Matrix Using Recurrence Formula

for i in range(1, m + 1):
for j in range(1, n + 1):

Traverse matrix row-wise.

If Characters Match
if word1[i - 1] == word2[j - 1]:
dp[i][j] = dp[i - 1][j - 1]

No cost added.

Take diagonal value.

If Characters Do Not Match

dp[i][j] = min(
dp[i - 1][j] + 1, # deletion
dp[i][j - 1] + 1, # insertion
dp[i - 1][j - 1] + 1 # substitution
)

Choose minimum among:

 Deletion
 Insertion
 Substitution

Add cost = 1.

7️⃣ Print DP Matrix

print("Edit Distance Matrix:")

Displays matrix for visualization.

This is useful for:

 Journal diagrams
 Understanding DP working
 Viva explanation

8️⃣ Return Final Answer

return dp[m][n]

Bottom-right cell contains final edit distance.

Example Execution
word_pairs = [("book", "back"), ("cat", "cats"), ("hello", "helo")]

The program calculates:

1. book → back → Distance = 2

2. cat → cats → Distance = 1
3. hello → helo → Distance = 1

Autocorrect Function
def autocorrect(misspelled, vocabulary):

Purpose:
Find closest word from vocabulary list.

Logic:

1. Initialize minimum distance as infinity:

min_dist = float('inf')

2. Compare misspelled word with every word in vocabulary:

for word in vocabulary:

3. Compute edit distance.

4. Keep word with smallest distance.
5. Return corrected word and distance.
 Example
Vocabulary:

["book", "back", "books", "look", "took", "back", "black"]

Input:

bok

Closest match:

book

Distance = 1

So output:

'bok' → 'book' (distance: 1)

Algorithm Summary
Step 1: Create DP matrix
Step 2: Initialize first row & column
Step 3: Fill matrix using recurrence formula
Step 4: Return bottom-right value
Step 5: Compare with vocabulary for correction
------5-------
Step 1: Install Python (Important)
Download Python 3.10 from:

👉 [Link]

⚠️During installation:

✔ Check "Add Python to PATH"

✔ Click Install

After installation, open CMD and check:

python --version

Should show:

Python 3.10.x

Step 2: Create Project Folder

mkdir Voice_Recommendation
cd Voice_Recommendation

Copy your file:

[Link]

into this folder.

Step 3: Create Virtual Environment

(Recommended)
python -m venv venv
venv\Scripts\activate

You should see:

(venv)
This avoids Python conflicts.

Step 4: Install Required Libraries

pip install streamlit speechrecognition pyaudio scikit-learn pandas numpy

If PyAudio Error Comes (Windows Fix)

Download correct PyAudio wheel from:

👉 [Link]

Then install:

pip install PyAudio-0.2.11-cp310-cp310-win_amd64.whl

Step 5: Run Application Properly

⚠️DO NOT run:

python [Link]

Instead run:

streamlit run [Link]

Browser will open:

[Link]

App will start.

PART 2 — Complete Code Explanation

🔹 1. Import Section
import streamlit as st
import speech_recognition as sr
import pandas as pd
import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from [Link] import cosine_similarity

Explanation:

 streamlit → Web interface

 speech_recognition → Voice → text conversion
 pandas → Dataset handling
 TfidfVectorizer → Convert text to numeric vectors
 cosine_similarity → Similarity measurement

2. Class Definition
class VoiceRecommendationSystem:

Encapsulates full system inside class for modularity.

3. Constructor
[Link] = [Link]()

Creates speech recognition object.

Dataset:

[Link] = [Link]({...})

Stores:

 Movie title
 Genre
 Description

This acts as knowledge base.

4. Speech Recognition Function

def listen_and_transcribe(self):

Steps:
 1. Open microphone
2. Adjust background noise
3. Listen to speech
4. Convert speech → text using Google API

Uses:

[Link].recognize_google(audio)

Requires internet.

5. Preference Extraction
def extract_preferences(self, text):

Detects genre keywords:

 sci-fi
 action
 comedy
 drama
 thriller
 crime

Returns:

{
'genres': [...],
'query': text
}

6. Recommendation Engine
def get_recommendations(self, preferences):

Step 1: Filter movies by genre

Step 2: Apply TF-IDF on movie descriptions
Step 3: Convert user query to vector
Step 4: Compute cosine similarity
Step 5: Sort by similarity score
Step 6: Return top 3 movies

7. Streamlit UI
[Link]()

Creates web UI.

Buttons:

[Link]()

Triggers microphone capture.

Manual input option also available.

Algorithm Flow
User Speech
↓
Google Speech API
↓
Text Extraction
↓
Genre Detection
↓
TF-IDF Vectorization
↓
Cosine Similarity
↓
Top Recommendations

What To Explain To Examiner

Q1: What type of recommendation system is this?

Content-based recommendation system.

Q2: Why use TF-IDF?

To convert textual descriptions into numerical vectors for similarity comparison.

Q3: Why cosine similarity?

To measure similarity between user query vector and movie description vectors.

Q4: Why virtual environment?

To avoid dependency conflicts between multiple Python installations.

Q5: Why did earlier error occur?

Because the Anaconda Python environment was missing the standard library module aifc,
required by SpeechRecognition.

Common Errors & Fixes

Error Reason Solution

No module named aifc Broken Anaconda Use normal Python

PyAudio error Missing wheel Install correct wheel

Microphone not detected Permission issue Allow mic access

Speech service error No internet Connect internet

Final Conclusion
In this experiment, a voice-based recommendation system was developed using Streamlit and
Google Speech Recognition API. The system captures user voice input, converts it into text,
extracts preferences, and recommends items using TF-IDF and cosine similarity. The
application demonstrates integration of speech processing, NLP, and machine learning
techniques.
-----------6-------------
PART 1 — How To Run This On Another
Computer

Step 1 — Install Python 3.10

Download from:

👉 [Link]

During installation:

✔ Check Add Python to PATH

Verify:

python --version

Should show:

Python 3.10.x

Step 2 — Create Project Folder

mkdir Oral_Exam_System
cd Oral_Exam_System

Save your file as:

[Link]

Step 3 — Create Virtual Environment

(Recommended)
python -m venv venv
venv\Scripts\activate

You should see:

(venv)

Step 4 — Install Required Libraries

pip install speechrecognition pyaudio scikit-learn numpy nltk

If PyAudio Error Occurs (Windows)

Download correct wheel:

👉 [Link]

Then install:

pip install PyAudio-0.2.11-cp310-cp310-win_amd64.whl

Step 5 — Run The Program

⚠️This is a normal Python script (not Streamlit).

Run:

python [Link]

What Will Happen

1. Program prints question
2. You press Enter
3. It listens to your answer
4. Converts speech → text (Google API)
5. Evaluates answer
6. Prints score and feedback

PART 2 — Complete Code Explanation

1. Import Libraries
import speech_recognition as sr
import difflib
import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from [Link] import cosine_similarity
import nltk
import re

Purpose:

 speech_recognition → Speech to text

 difflib → Sequence matching
 TfidfVectorizer → Convert text to numerical vectors
 cosine_similarity → Semantic similarity
 re → Text cleaning

2. Class Definition
class OralExaminationSystem:

Encapsulates entire oral exam logic.

3. Constructor (init)
[Link] = [Link]()
[Link] = TfidfVectorizer(stop_words='english')

Creates:

 Speech recognizer
 TF-IDF vectorizer

4. Question Bank Setup

[Link] = {
"q1": {...},
"q2": {...}
}

Each question contains:

 Question text
 Answer key
  Important keywords

This acts as evaluation database.

5. Speech Capture
def listen_to_answer(self):

Steps:

1. Activate microphone
2. Adjust noise
3. Listen
4. Convert speech → text using Google API

If error:
Returns None.

6. Text Preprocessing
def preprocess_text(self, text):

Removes:

 Punctuation
 Extra spaces
 Converts to lowercase

Improves comparison accuracy.

7. Keyword Matching Score

def keyword_match_score(self):

Checks how many important keywords appear in answer.

Formula:

matched_keywords / total_keywords × 100

 8. Semantic Similarity
def semantic_similarity(self):

Steps:

1. Convert student answer & answer key into TF-IDF vectors

2. Compute cosine similarity
3. Multiply by 100

This checks conceptual similarity.

9. Sequence Matching Score

[Link]()

Measures text similarity at character level.

Gives ratio between 0 and 1.

10. Final Score Calculation

final_score = (keyword * 0.4 + semantic * 0.4 + sequence * 0.2)

Weight distribution:

 Keyword: 40%
 Semantic: 40%
 Sequence: 20%

Balanced evaluation.

11. Feedback Generation

Based on:

 Keyword score
 Semantic score

Returns:
  Excellent
 Good
 Fair
 Needs Improvement

12. Main Function

Loops through all questions.

For each:

1. Display question
2. Capture answer
3. Evaluate
4. Print result

What To Explain To Examiner

Q1: What type of system is this?

Automated Oral Examination System using NLP.

Q2: What techniques are used?

 Speech Recognition
 Keyword Matching
 TF-IDF Vectorization
 Cosine Similarity
 Sequence Matching

Q3: Why combine three scoring methods?

To ensure fair evaluation:

 Keyword → Concept coverage

 Semantic → Meaning similarity
 Sequence → Structural similarity
Q4: Why TF-IDF?

To convert text into numeric form for similarity calculation.

Q5: Is this supervised learning?

No. It is similarity-based evaluation, not classification.

Algorithm Flow
Speech Input
↓
Google Speech API
↓
Text Preprocessing
↓
Keyword Matching
↓
TF-IDF Vectorization
↓
Cosine Similarity
↓
Sequence Matching
↓
Weighted Score
↓
Feedback

Common Errors & Fixes

Error Reason Fix

No module named aifc Wrong Python Use Python310

PyAudio error Missing wheel Install correct wheel

Speech service error No internet Connect internet

Could not capture answer Mic permission Enable mic

Final Conclusion (For Journal)
In this experiment, an automated oral examination system was implemented using speech
recognition and NLP techniques. The system evaluates student responses using keyword
matching, semantic similarity (TF-IDF & cosine similarity), and sequence matching. A
weighted scoring mechanism ensures fair evaluation and feedback generation.