Title: Introduction to Natural Language Processing (NLP)

Subtitle: "Understanding how machines interact with human language"

[Link] Bharathi
"Today, we’re diving into the interesting world of Natural Language Processing
(NLP). We’ll see how machines can understand and generate human language."
Agenda: "We’ll cover basic concepts, how to preprocess text, and a practical
application like sentiment analysis."
What is NLP?
Overview of NLP (Natural Language Processing)
Natural Language Processing (NLP) is a field of artificial intelligence that focuses
on enabling computers to understand, interpret, and generate human language. It
involves the interaction between computers and natural human languages, allowing
machines to process and analyze large amounts of natural language data.
In Simple, It’s like teaching machines to read, listen, and talk."
History of NLP
NLP has evolved over several decades, with major milestones marking its progress:
1. In (1950s): Alan Turing proposed the idea of machines understanding human
language called Turing Test.
2. In 1960s-1970s: Development of symbolic AI, focusing on grammar-based
approaches.
 SHRDLU (1970): A system that could understand and respond to simple
commands in a simulated environment.
3. 1980s-1990s: Shift to statistical methods with the introduction of machine
learning.
 Probabilistic models like Hidden Markov Models (HMMs) were used
for tasks like speech recognition.
4. 2000s: Data-driven techniques emerged with access to larger datasets and
improved computing power.
5. 2010s-Present: Deep learning revolutionized NLP with models like
Transformers and BERT (Bidirectional Encoder Representations from
Transformers).
 These models enable advanced tasks such as machine translation, text
generation, and sentiment analysis at scale.
Applications of NLP
1. Chatbots:
o Chatbots use NLP to simulate human conversation. They can understand
user input and generate appropriate responses, providing assistance in a
variety of fields such as customer service, healthcare, and education.
o Examples:
 Siri, Alexa, Google Assistant: Virtual assistants that process voice
commands to perform tasks.
 Customer Service Bots: Automate responses to customer
inquiries, reducing the need for human support.
2. Language Translation:
o NLP enables real-time language translation between different languages,
breaking down language barriers for communication.
o Example:
 Google Translate: Uses advanced neural machine translation
techniques to provide accurate translations between languages.
3. Sentiment Analysis:
o Sentiment analysis involves determining the emotional tone of a piece of
text, classifying it as positive, negative, or neutral. It’s widely used in
customer feedback and social media analysis.
o Examples:
 Business Reviews: Companies analyze product reviews to
understand customer sentiment and make data-driven decisions.
 Social Media Monitoring: Tools track public opinion by
analyzing tweets, posts, and comments to gauge sentiment
towards brands or political events.
key challenges in NLP:
1. Ambiguity:
 Lexical Ambiguity: Words have multiple meanings (e.g., "bank" as a
riverbank or a financial institution).
 Syntactic Ambiguity: Sentence structure can have more than one
interpretation (e.g., "Visiting relatives can be boring").
 2. Context Understanding:
 Understanding language context is essential but difficult (e.g., "He banked
on the river" could refer to fishing or finance).
3. Multilingual NLP:
 Building models for multiple languages is tough, especially for low-
resource languages with limited training data.
4. Sarcasm and Irony Detection:
 Detecting sarcasm and irony is hard as the literal meaning differs from the
intended sentiment (e.g., "Great, another meeting!" often implies
frustration).
5. Bias in NLP Models:
 Models can inherit biases from training data, leading to unfair outcomes.
Reducing this bias is an ongoing issue.
6. Generalization Across Domains:
 NLP models often excel in specific domains but struggle in others. Transfer
learning helps, but more adaptability is needed.
Some of the Key Concepts of NLP are Tokenization, Part-of-Speech (POS)
Tagging, Named Entity Recognition (NER), Lemmatization "These are the
foundational blocks for any NLP task, like analysing text or generating language."
Which we are going to discuss further in detail in Text preprocessing.
Next moving to Text Preprocessing in NLP
1. Text Cleaning: Remove unwanted characters, extra spaces, and formatting
issues (e.g., HTML tags, special symbols).
o Example: "Hello, world! @2024" to get "Hello world 2024."
2. Tokenization: Split the text into smaller units (tokens), such as words or
phrases, for easier analysis.
o Example: Split the sentence "Quick brown fox" into tokens: ["quick",
"brown", "fox"].
3. Lowercasing: Convert all text to lowercase to ensure consistency and reduce
redundancy
o Example: Convert "Apple" and "apple" to "apple" to ensure uniformity.
4. Stop-word Removal: Eliminate common words that do not add significant
meaning to the analysis (e.g., "and," "the," "is").
 o Example: Remove stop words from "The quick brown fox jumps over
the lazy dog" to get "quick brown fox jumps lazy dog."
5. Stemming: Reduce words to their root form by removing suffixes
o Example: Reduce "running," "runner," and "ran" to the root form "run."
6. Lemmatization: Return the base form of a word based on its meaning and
context
o Example: Convert "better" to "good" using its context to return the base
form.
7. Part-of-Speech Tagging: Identify the grammatical parts of speech (e.g., noun,
verb) to understand word roles in context.
o Example: Tag "The cat sat on the mat" as:
 "The" (determiner),
 "cat" (noun),
 "sat" (verb),
 "on" (preposition),
 "the" (determiner),
 "mat" (noun).
8. Named Entity Recognition (NER): Detect and classify named entities in text,
such as people, organizations, and locations.
o Example: Identify entities in "Barack Obama was born in Hawaii" as
"Barack Obama" (Person) and "Hawaii" (Location).
9. N-grams: Create sequences of "n" words to capture context and relationships
(e.g., bigrams for two-word combinations).
o Example: Create bigrams from "I learn NLP" to get: ["I learn", "learn
NLP"].
10. Vectorization: Convert text into numerical representations suitable for machine
learning algorithms (e.g., Bag of Words, TF-IDF).
o Example: Convert the sentence "I love NLP" into a vector using Bag of
Words: [1, 1, 1, 0, 0], where each position represents the presence of a
specific word.
11. Feature Extraction:
o Example: Identify key features such as sentiment scores or frequency of
specific terms in a document for analysis.
 12. Text Representation:
o Example: Use word embeddings to represent "king" and "queen" in a
vector space where their relationships can be captured (e.g., the
difference between them may reflect gender).
These examples illustrate how each preprocessing step transforms raw text into a more
usable format for NLP tasks.

Text Representation - This will help you understand how the cleaned and pre-
processed text can be transformed into a format suitable for machine learning
models.
 Therefore we can define Text Representation as - Converting text data into a
numerical format that machine learning models can understand.
There are several ways to represent text in Natural Language Processing (NLP),
ranging from simple to advanced methods. Here iam going to give an overview of
the most commonly used text representation techniques: the first one is
1. Bag of Words(BoW) - Represents text as a set of word counts.
o How it Works: A vector is created where each position corresponds to a
word in the vocabulary.
 Example:
o For the sentences: "I love NLP" and "NLP is fun", BoW will represent
them as:
 ["I", "love", "NLP", "is", "fun"]
 Sentence 1: [1, 1, 1, 0, 0]
 Sentence 2: [0, 0, 1, 1, 1]

2. TF-IDF (Term Frequency-Inverse Document Frequency)

 What is TF-IDF?
o A statistical measure that highlights important words in a document
relative to a collection of documents.
 Formula:
o TF: How frequently a term appears in a document.
o IDF: Importance of a term based on how rare it is across all documents.
 Example:
 o In a corpus of documents, Common words like "the" will have a low
score, while more unique words will have higher values.
o TF-IDF value = TF * IDF
3. One-Hot Encoding
One-Hot Encoding is a method of representing categorical data (like words) in
a format that machine learning algorithms can process. Each word or category
is represented as a binary vector where one element is "hot" (1) and the rest are
"cold" (0).
How One-Hot Encoding Works (with Example)
Let’s use the same two sentences for demonstration:
 Sentence 1: "I love NLP"
 Sentence 2: "NLP is fun"
Step 1: Build the Vocabulary
Just like with Bag of Words, the first step is to identify all unique words across
both sentences. The vocabulary is:
 Vocabulary: ["I", "love", "NLP", "is", "fun"]
Each word in this vocabulary will be assigned its own binary vector.
Step 2: Assign One-Hot Vectors
For each word in the vocabulary, we create a binary vector that is the length of
the vocabulary. Each vector has one "1" in the position corresponding to that
word, and the rest are "0"s.
One-Hot Encoding for Vocabulary:
 "I": [1, 0, 0, 0, 0]
 "love": [0, 1, 0, 0, 0]
 "NLP": [0, 0, 1, 0, 0]
 "is": [0, 0, 0, 1, 0]
 "fun": [0, 0, 0, 0, 1]
Each word has a unique binary vector where only one position is 1 (indicating
the word’s presence), and the rest are 0.
Step 3: Represent Each Sentence
Now, we can represent each sentence by using the one-hot encoded vectors of
the words it contains.
 For Sentence 1: "I love NLP"
 "I" → [1, 0, 0, 0, 0]
 "love" → [0, 1, 0, 0, 0]
 "NLP" → [0, 0, 1, 0, 0]
The one-hot encoding for Sentence 1 is the collection of the vectors for each
word in the sentence:
For Sentence 2: "NLP is fun"
 "NLP" → [0, 0, 1, 0, 0]
 "is" → [0, 0, 0, 1, 0]
 "fun" → [0, 0, 0, 0, 1]
The one-hot encoding for Sentence 2 is:
Limitations: For large vocabularies, the vectors become very long

(Next video)

4. Word Embeddings
 Description: Dense vector representations that capture semantic meanings of
words by placing similar words closer in vector space.
 Methods:
o Word2Vec (Skip-gram, CBOW)
o GloVe (Global Vectors for Word Representation)
o FastText (captures subword information)
 Example:
o "king" and "queen" will have vectors that are close in space and capture
relational similarities (e.g., "man" is to "king" as "woman" is to
"queen").
 Advantage: Captures both syntactic and semantic relationships between words.
5. N-grams
 Description: Represents text as a sequence of adjacent word pairs or triplets,
capturing local word context.
 Method:
o Bigrams: Two-word sequences (e.g., "machine learning" → "machine",
"learning")
o Trigrams: Three-word sequences (e.g., "natural language processing"
→ "natural", "language", "processing")
 Advantage: Captures word order to some extent but increases complexity.
6. Character-level Embeddings
 Description: Represents text at the character level instead of the word level,
useful for handling rare or unknown words (e.g., spelling variations, typos).
 Example:
o Each character in a word is embedded, allowing the model to recognize
patterns in prefixes or suffixes.
 Common Usage: Often combined with word embeddings for improved
robustness in tasks like text generation.
7. Sentence or Document Embeddings
 Description: Represents entire sentences or documents as a single dense vector
that captures context and meaning.
 Methods:
o Doc2Vec: Extends Word2Vec to represent documents.
o Universal Sentence Encoder: Provides fixed-length embeddings for
sentences.
o BERT/Transformer-based models: Contextual embeddings that
capture sentence-level meaning.
 Example: Each sentence gets a vector that incorporates the semantics of the
entire sentence, including word interactions.
8. Contextual Word Embeddings
 Description: Advanced embeddings that provide different representations for
the same word based on its surrounding context. This overcomes the limitation
of static embeddings (e.g., Word2Vec).
 Methods:
 o ELMo (Embeddings from Language Models)
o BERT (Bidirectional Encoder Representations from Transformers)
o GPT (Generative Pretrained Transformer)
 Example:
o The word "bank" in "river bank" and "bank account" will have different
vector representations depending on the context.
 Advantage: Greatly improves tasks like language understanding and text
generation by considering the entire sentence.
9. Bag of Concepts
 Description: Extends the BoW model by grouping words into semantic
concepts or categories.
 Method: Uses a knowledge base (e.g., WordNet or Wikipedia) to map words to
broader concepts, helping improve generalization across similar terms.
 Advantage: Reduces dimensionality by focusing on concepts rather than
individual words.
10. Dependency-based Embeddings
 Description: Represents words based on their syntactic relationships in a
sentence, typically used with dependency parsing.
 Method: Focuses on word dependencies rather than proximity, emphasizing
the grammatical structure.
 Example: A verb might be closely embedded with its subject or object, based
on their syntactic roles.