Lecture 1
Natural Language Processing
NLP Introduction
Natural Language Processing (NLP) is a subfield of artificial intelligence and computational linguistics
that focuses on enabling computers to understand, interpret, and generate human language. It sits
at the intersection of computer science, linguistics, and machine learning.
Core Components of NLP
Natural
Language
Understanding
(NLU)
Natural
Language
Generation
(NLG)
Natural Language Generation (NLG)
Goal: Extract meaning
Tasks:
from human language
Intent
Semantic Entity Coreference
Recognition: Identifying
Analysis: Understanding Recognition: Identifying Resolution: Determining
user's purpose (e.g., in
word/sentence meaning names, places, dates what pronouns refer to
chatbots)
Natural Language Generation (NLG)
Goal: Generate
human-like text from Tasks:
structured data
Text summarization Machine translation Dialogue generation Content creation
NLP Applications
Text Classification • (spam detection, sentiment analysis)
Named Entity Recognition • (identifying people, places, organizations)
Machine Translation • (Google Translate)
Speech Recognition • (Siri, Alexa)
Question Answering • (chatbots, virtual assistants)
Text Summarization
Language Modeling • (predicting next word in sequence)
NLP vs Data Mining
Natural Language Processing (NLP) Data Mining
Primary Focus: Understanding, interpreting, Primary Focus: Discovering patterns, insights,
and generating human language and knowledge from any type of data
Goal: Enable computers to process and Goal: Extract valuable information from large
"understand" text/speech datasets
Domain: Specifically deals with unstructured Domain: Works with structured, semi-
language data structured, and unstructured data
Output: Language understanding, Output: Patterns, associations, clusters,
translation, generation, analysis predictions, insights
Key Differences
Aspect NLP Data Mining
Structured tables, text, images, etc. (any
Primary Data Type Text, speech (unstructured language)
format)
Core Objective Language understanding and generation Pattern discovery and knowledge extraction
Tokenization, parsing, embeddings,
Key Techniques Clustering, classification, association rules
transformers
Domain Specificity Highly language/linguistics focused Domain-agnostic, applicable to any field
Sentences, translations, summaries, Patterns, rules, clusters, predictions,
Output Form
answers visualizations
BLEU, ROUGE, perplexity, accuracy on Precision, recall, F1-score, support, lift,
Evaluation Metrics
language tasks confidence
Linguistic normalization (stemming, Data cleaning, normalization, feature
Preprocessing Focus
lemmatization) engineering
Human Often requires interpretation of
Directly readable output (text)
Interpretation patterns/rules
NLP vs Data Mining
Typical NLP Applications Typical Data Mining Applications
• Machine Translation: Google Translate • Customer Segmentation: Grouping similar
customers
• Chatbots & Virtual Assistants: Siri, Alexa
• Market Basket Analysis: "Customers who bought X
• Text Summarization: Automatic article
also bought Y"
summarizers
• Fraud Detection: Finding anomalous patterns
• Speech Recognition: Transcription services
• Predictive Maintenance: When will equipment
• Grammar Checkers: Grammarly
fail?
• Question Answering: IBM Watson
• Recommendation Systems: Netflix, Amazon
recommendations
• Financial Forecasting: Stock price prediction
Hybrid Applications (Both Fields)
Social Media Analysis: • NLP for understanding posts + data mining for trend detection
Healthcare Analytics: • NLP for clinical notes + data mining for patient outcome prediction
E-commerce: • NLP for reviews + data mining for customer behavior patterns
Risk Assessment: • NLP for news/articles + data mining for risk modeling
NLP Libraries
• NLTK (Natural Language Toolkit):
This is a comprehensive and widely used library, particularly popular for academic and research
purposes. It provides tools for a broad range of NLP tasks such as tokenization, stemming, lemmatization,
part-of-speech tagging, parsing, and classification. NLTK also includes interfaces to over 50 corpora and
lexical resources.
• spaCy:
Designed for efficiency and production use, spaCy offers fast and accurate NLP processing. It
excels in tasks like named entity recognition, dependency parsing, and text classification, and provides pre-
trained models for various languages.
• Hugging Face Transformers:
This library has revolutionized NLP by providing easy access to state-of-the-art pre-trained
transformer models like BERT, GPT, and T5. It facilitates fine-tuning these models for specific NLP tasks and
is a go-to choice for advanced applications.
NLP Pre Trained Model
Pre-trained models have revolutionized natural language processing by providing powerful base
models that can be fine-tuned for specific tasks. These models are trained on massive amounts of
text data to learn general language representations before being adapted to particular applications.
Choosing a Pre-trained Model
• Task type (classification, generation, etc.)
• Language requirements (multilingual or specific language)
• Computational constraints (model size, inference speed)
• Domain specificity (general purpose vs domain-adapted models)
1. FastText (Facebook): Extends Word2Vec with subword information
2. BERT (Google): Bidirectional transformer trained on masked language modeling
3. GPT (OpenAI): Unidirectional transformer using autoregressive language modeling
Comparison
Model What It Learns / Predicts
Learns that "Apple" and "startup" are similar to
FastText
tech domain
Understands whether "Apple" is a company or
BERT
fruit based on full context
Can continue the sentence like: → "Apple is
looking at buying a startup that specializes in AI
GPT
software."
Comparison
• FastText: Fast, efficient, good for basic NLP.
• BERT: Deep understanding of text, best for comprehension tasks.
• GPT: Creative and fluent, best for generating human-like language.
Frameworks for Pre-trained Models
1. Hugging Face Transformers: Provides easy access to thousands of pre-trained models
2. TensorFlow Hub: Repository of pre-trained models including NLP
3. PyTorch Hub: Similar model repository for PyTorch
Advantages of Pre-trained Models
• Transfer Learning: Leverage knowledge from general language understanding
• Reduced Data Requirements: Need less labeled data for specific tasks
• State-of-the-Art Performance: Achieve better results than training from scratch
• Faster Development: Skip the expensive initial training phase
NLP Libraries
1. Spacy
2. NLTK (Natural Language Toolkit)
Feature NLTK spaCy
Release Year 2001 (Academic origins) 2015 (Industrial focus)
20+ languages (better
Language Support 50+ languages
quality)
Very fast (Cython-
Performance Slower (Python-based)
optimized)
Ease of Use Steeper learning curve Pythonic, intuitive API
Limited (mostly rule- Extensive (transformers,
Pretrained Models
based) word vectors)
NLP Libraries
Choose NLTK
• You need educational/research flexibility (NLTK is a "toolbox" approach)
• Working with linguistic theories (e.g., WordNet hierarchies)
• Processing rare languages (spaCy's models may not exist)
Choose spaCy
• You need production-ready pipelines
• Prioritize speed and accuracy
• Want pre-trained transformers
• Need easy integration with ML frameworks (PyTorch, TensorFlow)
Installation Commands
NLP Pipeline
NLP Pipeline
Any Questions ?