Week 1: Introduction to NLP & The Pipeline
1. What is NLP?
* Definition: NLP (Natural Language Processing) is a field of AI that helps computers
understand, interpret, and generate human language.
* Goal: To make computers communicate like humans (e.g., Chatbots, Translation).
2. Key Applications:
* Sentiment Analysis: Detecting if a review is positive or negative.
* Machine Translation: Google Translate (English to Urdu).
* Spam Filtering: Catching junk emails.
* Chatbots: Siri, Alexa, customer service bots.
3. The NLP Pipeline (How it works step-by-step):
The pipeline is the "assembly line" for processing text.
* Data Acquisition: Collecting text (e.g., downloading tweets).
* Text Cleaning: Removing noise like emojis, URLs, and HTML tags.
* Preprocessing: Preparing text for the machine (Tokenization, Lowercasing, Stopwords
removal).
* Feature Engineering: Converting text into numbers (e.g., Bag of Words) because
computers only understand math.
* Model Training: Teaching the machine using an algorithm (e.g., Naive Bayes or Neural
Networks).
* Evaluation: Testing if the model is accurate.
* Deployment: Putting the model into a real app.
4. History of NLP:
* Rule-Based (1950s-60s): Used strict grammar rules (e.g., ELIZA chatbot). Failed because
language is too complex.
* Statistical (1980s-90s): Used probability and counting (e.g., n-grams). Better but
struggled with context.
* Deep Learning (2010s-Now): Uses Neural Networks and "Transformers" (like BERT/GPT)
to understand deep meaning.
Week 2: Text Preprocessing (NLTK)
1. What is Tokenization?
* Definition: Breaking raw text into smaller units called "tokens" (words or sentences).
* Word Tokenization: "I love NLP" \rightarrow ['I', 'love', 'NLP'].
* Sentence Tokenization: "Hi. How are you?" \rightarrow ['Hi.', 'How are you?'].
* Why use NLTK? Simple Python commands like .split() fail with punctuation (e.g., "word,").
NLTK handles punctuation correctly.
2. Edge Cases (Tricky Situations):
* Punctuation: "Don't forget to call, Ali." (Comma changes meaning). NLTK separates
commas safely.
* Abbreviations: "Dr. Smith" or "U.S.A." contain dots but aren't end of sentences. NLTK's
punkt tokenizer is smart enough to handle this.
* Social Media: Tweets have hashtags (#AI), mentions (@user), and emojis. Standard
tokenizers break these. Use TweetTokenizer for social media text.
3. Stopwords:
* Definition: Common words like is, the, a, on. They carry little meaning and clutter the
data.
* Why remove them? To make the model faster and focused on important words (like "win",
"lottery").
* Zipf's Law: A theory stating that a few words (like "the") appear very often, while most
words appear rarely. This explains why removing stopwords is helpful.
Week 3: Normalization & Features
1. Stemming vs. Lemmatization (Normalization):
Goal: To treat "run", "running", and "ran" as the same word.
* Stemming: Chops off the end of words. Fast but rough.
* Example: "Studies" \rightarrow "Studi" (Not a real word).
* Algorithm: Porter Stemmer (common), Lancaster (aggressive).
* Lemmatization: Uses a dictionary to find the actual root word. Slower but accurate.
* Example: "Studies" \rightarrow "Study" (Real word).
* Note: Works best with POS Tags (telling it if the word is a noun or verb).
2. N-Grams:
* Unigrams: 1 word ("Machine"). Good for frequency.
* Bigrams: 2 words ("Machine Learning"). Captures context.
* Trigrams: 3 words ("New York City").
* Collocations: Words that naturally go together (e.g., "Strong tea", not "Powerful tea").
3. Converting Text to Numbers (Feature Engineering):
* Bag of Words (BoW): Counts how many times a word appears. Discards order.
* Result: A matrix of counts.
* TF-IDF: A smarter way to count.
* TF (Term Frequency): How often a word appears in one document.
* IDF (Inverse Document Frequency): Lowers the score of words that appear everywhere
(like "the").
* Benefit: Highlights unique, important keywords.
Week 4: Classification & Sentiment Analysis
1. Supervised Learning:
* Concept: Like a teacher and student. The "Teacher" gives examples (Features + Labels),
and the "Student" (Model) learns patterns.
* Logistic Regression: A simple algorithm for "Yes/No" or "Positive/Negative" tasks using a
sigmoid function.
2. Sentiment Analysis:
* Task: Detecting emotion (Positive vs. Negative).
* Negation Handling: Crucial for accuracy.
* Problem: "Not good" contains "good", so a basic model might think it's positive.
* Solution: Add a tag like _NEG to words after "not" (e.g., "not good_NEG") so the
computer knows it's bad.
3. Evaluation Metrics (How to check performance):
* Accuracy: Overall correct guesses. (Can be misleading if data is imbalanced, e.g., 90%
positive reviews) .
* Precision: Out of all predicted positives, how many were actually positive?.
* Recall: Out of all actual positives, how many did we catch?.
* F1-Score: The balance between Precision and Recall. Best for imbalanced data.
* Confusion Matrix: A table showing True Positives, False Positives, etc..
Week 5 & 6: Deep Learning & Embeddings
1. Machine Learning vs. Deep Learning:
* Machine Learning: You manually tell it what features to look for.
* Deep Learning: It automatically figures out features using "Neural Networks" (layers of
artificial neurons mimicking the brain).
* Layers: Input Layer \rightarrow Hidden Layers (learn patterns) \rightarrow Output Layer.
2. Word Embeddings:
* Problem: Old methods (BoW) didn't know that "Good" and "Great" are similar.
* Solution (Embeddings): Converts words into lists of numbers (vectors) based on
meaning.
* Key Concept: Words with similar meanings are close together in math space.
* Math with words: King - Man + Woman = Queen.
* GloVe / Word2Vec: Popular pre-trained embedding models. GloVe looks at how often
words appear together globally.
Week 7: Transformers & Modern Models
1. Limitations of Old Models (RNNs):
* They read one word at a time (slow) and forgot the start of long sentences.
2. The Transformer (2017):
* The Breakthrough: Uses Self-Attention. It looks at all words in a sentence at once to
understand context.
* Analogy: Highlighting key words in a sentence to understand the meaning.
3. Popular Transformer Models:
* BERT (Encoder): Best for understanding text (Classification, QA). Reads left-to-right and
right-to-left.
* GPT (Decoder): Best for generating text (Chatbots, writing stories). Reads left-to-right.
* T5: Converts every task into text-to-text (e.g., "translate English to Urdu: ...").
* Multilingual Models (mBERT, XLM-R): Understand 100+ languages including Urdu.
4. Token-Level Tasks:
* POS Tagging: Labeling grammar (Noun, Verb).
* NER (Named Entity Recognition): Finding real-world objects like "Ali" (Person) or "Lahore"
(Location).
Week 8: Advanced NLP Applications
1. Summarization:
* Extractive: Highlights and copies key sentences (like using a marker).
* Abstractive: Writes a new summary in its own words (like a human).
2. Question Answering (QA):
* The model reads a paragraph (Context) and finds the answer to a specific Question. It
doesn't guess; it extracts.
3. Machine Translation:
* Converting text (English \rightarrow Urdu) while keeping context. Modern models use
Encoder-Decoder Transformers.
4. Knowledge Distillation:
* Teaching a small "Student" model to mimic a huge "Teacher" model. Makes models
faster and usable on phones.
5. Ethics & Bias:
* Models learn from internet data, which has bias.
* Example: "Doctor" might be associated with men and "Nurse" with women. We must fix
this to make AI fair.
Week 9: File Formats & Tagging
1. File Types:
* .TXT: Simple, raw text. No labels.
* .CSV: Tables (Excel-like). Good for simple classification (Text, Label).
* .JSON/JSONL: Nested structures. Best for complex tasks like QA or Dialogues.
* .XML/TEI: Tree structure. Used in linguistics and history projects.
2. Tagging Schemes (for NER):
* BIO:
* B: Beginning of an entity (e.g., B-PER for Ali).
* I: Inside an entity.
* O: Outside (not an entity).
* BILOU: More detailed (adds Last and Unit token tags).
3. Trees:
* Dependency Trees: Show which word depends on which (e.g., Subject \rightarrow Verb).
* Constituency Trees: Show the grammatical structure (Noun Phrases, Verb Phrases).