NATURAL LANGUAGE PROCESSING
Natural Language
Natural Language is the language naturally spoken by humans for everyday
communication.
Examples include English, Hindi, Marathi, Tamil, Telugu, etc.
Natural languages evolve naturally over time and are rich in ambiguity, emotions, and
context.
Natural Language is different from:
Programming languages such as Python and Java
Constructed languages such as Esperanto and Interlingua
Non-human communication systems like animal sounds
Challenges in Multilingual NLP
Different grammar rules
Different word orders
Different scripts
India has 22 official languages belonging to four major families:
Indo-Aryan
Dravidian
Tibeto-Burman
Austro-Asiatic
What is Natural Language Processing (NLP)?
Natural Language Processing is a branch of Artificial Intelligence that enables
computers to understand, interpret, and generate human language in a meaningful way.
NLP combines:
1
Computational Linguistics
Page
Machine Learning
Deep Learning
The main goal of NLP is to make machines interact with humans in a natural and
intelligent manner.
Applications of NLP
NLP is widely used in real-world applications such as:
Chatbots and virtual assistants
Sentiment analysis
Machine translation
Speech recognition
Text summarization
Information retrieval
Question answering systems
NLP = NLU + NLG
Natural Language Processing is divided into two parts:
Natural Language Understanding focuses on extracting meaning from text or
speech.
Natural Language Generation focuses on producing meaningful text or speech from
data.
Both together enable intelligent communication between humans and machines.
Characteristics and Challenges of Language
Language has the following characteristics:
Highly ambiguous
Context-dependent
Dynamic and evolving
Contains emotions and intentions
2
Challenges include:
Page
Multiple meanings of words
Complex grammar rules
Different sentence structures
Cultural and contextual variations
Why NLP is Hard
NLP is difficult because of ambiguity at multiple levels.
Word ambiguity
Example: Bank can mean a financial institution or river side
Grammatical ambiguity
Example: Chair can be noun or verb
Structural ambiguity
Example: I saw a man with a telescope
Context ambiguity
Example: The chicken is ready to eat
Machines must analyze context to choose the correct interpretation.
Levels of Language Processing in NLP
1 Phonological Level
This level deals with sounds of language.
It focuses on:
Phonemes (smallest sound units)
Phonetic transcription
Prosody (stress, rhythm, intonation)
Significance:
Essential for speech recognition
Used in text-to-speech systems
Helps identify accents and pronunciation differences
3
2 Morphological Level
Page
This level studies the internal structure of words.
Key concepts:
Morphemes – smallest meaningful units
Inflection – tense, number, gender changes
Derivation – forming new words
Applications:
Stemming
Lemmatization
Part-of-speech tagging
Example:
Happy → Unhappy
Running → Run
3 Lexical Level
This level deals with words and vocabulary.
Key components:
Tokenization
Lexical semantics
Word relationships
Importance:
Understanding word meaning
Handling synonyms and homonyms
Example:
The word “bank” changes meaning based on context.
4 Syntactic Level
This level focuses on sentence structure and grammar.
Key components:
Parsing
Constituency parsing
Dependency parsing
Importance:
Understanding grammatical correctness
4
Page
Used in machine translation and information extraction
Example:
Subject–Verb–Object structure in English
5 Semantic Level
This level focuses on meaning of sentences.
Key components:
Word Sense Disambiguation
Semantic Role Labeling
Named Entity Recognition
Applications:
Sentiment analysis
Information retrieval
Machine translation
Example:
John gave Mary a book
Agent: John
Recipient: Mary
Object: Book
6 Discourse Level
This level deals with relationships between sentences.
Key components:
Coherence
Cohesion
Anaphora resolution
Importance:
Understanding long texts
Used in summarization and dialogue systems
Example:
John loves his dog. It follows him everywhere.
“It” refers to the dog.
7 Pragmatic Level
5
Page
This level considers context and speaker intention.
Key components:
Speech acts
Deixis
Contextual understanding
Example:
Can you pass the salt?
This is a request, not a question.
Text Processing in NLP
Text processing converts raw human language into structured machine-readable
format.
Example steps:
Remove punctuation
Remove emojis
Convert to lowercase
Split into words
Apply stemming or lemmatization
This makes text meaningful for computers.
Data Collection in NLP
Text data is collected from:
Websites
Books
Articles
Social media
Challenges:
Noisy data
Multilingual content
Spelling errors
Privacy concerns
Bias in data
6
Page
Text Cleaning
Text cleaning removes unwanted elements from text.
Includes:
Removing punctuation, symbols, numbers
Correcting spelling mistakes
Normalizing slang and abbreviations
Example:
OMG, this txt is gr8
Becomes: Oh my god this text is great
Text Preprocessing
Text preprocessing improves data quality before applying NLP models.
Benefits:
Improves accuracy
Reduces noise
Improves training efficiency
Ensures consistency
Reduces bias
Text Preprocessing Steps
Tokenization
Lowercasing
Stopword removal
Normalization
Stemming or lemmatization
Tokenization
Tokenization splits text into words or sentences.
Challenges:
7
Abbreviations
Page
URLs
Contractions
Punctuation
Lowercasing
Lowercasing converts all text into lowercase.
Benefits:
Uniform representation
Simplifies matching
Improves model efficiency
Stopword Removal
Stopwords are common words such as and, the, is.
Stopwords should not be removed in:
Sentiment analysis
Text generation
Machine translation
Normalization
Normalization converts text into a standard format.
Includes:
Spelling normalization
Unicode normalization
Abbreviation expansion
Stemming and Lemmatization
Stemming removes suffixes using rules.
It may not produce valid words.
Lemmatization produces valid dictionary words using context.
Example:
Running → Run
8
Better → Good
Page
Word Embeddings
Word embeddings represent words as numerical vectors.
Words with similar meanings have similar vectors.
They capture semantic relationships.
Text Representation Techniques
1 Zipf’s Law
Word frequency is inversely proportional to its rank.
Few words occur frequently, many occur rarely.
2 Bag of Words (BoW)
Represents text using word frequency.
Advantages:
Simple
Easy to implement
Disadvantages:
No semantics
No word order
Sparse vectors
3 TF-IDF
Improves BoW by reducing weight of frequent words.
Limitations:
Still no semantics
Ignores word relationships
9
Page
Word2Vec
Word2Vec is a neural network-based word embedding model.
It maps words to continuous vector space.
1 CBOW
Predicts target word from context words.
Advantages:
Fast
Efficient
2 Skip-Gram
Predicts context words from target word.
Advantages:
Better semantic relationships
Works well with rare words
Introduction to BERT
1 What is BERT
BERT stands for Bidirectional Encoder Representations from Transformers.
It is a pre-trained NLP model developed by Google.
BERT reads text from both left-to-right and right-to-left simultaneously.
2 Why BERT is Needed
Traditional models like Word2Vec and GloVe process text in one direction.
10
They miss full context of a sentence.
Page
BERT understands the complete meaning of words using surrounding context.
3 BERT and Transformers
BERT is based on Transformer encoder architecture.
Transformers use self-attention mechanism to capture relationships between
words.
This allows BERT to understand long-range dependencies.
Applications of BERT
Classification tasks such as sentiment analysis
Question answering systems
Named Entity Recognition
BERT uses the [CLS] token for classification tasks.
For question answering, BERT predicts start and end positions of answers.
## END ##
Reference:
Prompt Engineering Basics PPT
With help of:
ChatGPT
Disclaimer:
This media is prepared for learning purposes and may contain some mistakes. AI tools were
used to assist in content creation. The author holds no responsibility for any errors.
11
Page
Created by KUNAL NIMJE (EXTC)
Page
12