Natural Language Processing — Class Notes
Original educational study material prepared as structured class notes.
1. What is NLP?
Natural Language Processing (NLP) combines language, linguistics, statistics, and machine learning to
process human language. Applications include search, classification, summarization, translation,
question answering, and information extraction.
Text is difficult for computers because meaning depends on context, word order, ambiguity, and
background knowledge. Modern NLP therefore relies heavily on learned representations and contextual
models.
Key review points
• Define the main concepts in your own words.
• Identify one practical example where the concept would be useful.
• Consider common limitations, failure modes, and evaluation methods.
2. Text Preprocessing
Traditional pipelines may include sentence segmentation, tokenization, normalization, stop-word
handling, and stemming or lemmatization. The exact preprocessing depends on the task.
Aggressive preprocessing can remove information. For modern transformer models, tokenization is
usually performed with subword methods that can represent common words efficiently while still
handling uncommon words.
Key review points
• Define the main concepts in your own words.
• Identify one practical example where the concept would be useful.
• Consider common limitations, failure modes, and evaluation methods.
3. Text Representation
Bag-of-words represents a document using counts of terms and ignores most word order. TF-IDF
increases the weight of informative terms and decreases the influence of terms that occur in many
documents.
Word embeddings represent words as dense vectors. Contextual language models instead produce
representations that depend on surrounding tokens, making the same word capable of receiving
different representations in different contexts.
Key review points
• Define the main concepts in your own words.
• Identify one practical example where the concept would be useful.
• Consider common limitations, failure modes, and evaluation methods.
4. Text Classification
Text classification assigns labels to documents or messages. Examples include sentiment analysis,
topic classification, spam detection, and intent recognition.
A complete classifier needs a representative dataset, a defined label policy, a suitable
train-validation-test split, and metrics appropriate to the class distribution. Error analysis is often as
important as the headline metric.
Key review points
• Define the main concepts in your own words.
• Identify one practical example where the concept would be useful.
• Consider common limitations, failure modes, and evaluation methods.
5. Sequence-to-Sequence Tasks
Translation, summarization, and question answering can be formulated as sequence-to-sequence
problems. Encoder-decoder architectures learn to transform an input sequence into an output sequence.
Attention allows the decoder to focus on relevant parts of the input. Transformer encoder-decoder
models can learn complex relationships without processing tokens strictly one at a time during training.
Key review points
• Define the main concepts in your own words.
• Identify one practical example where the concept would be useful.
• Consider common limitations, failure modes, and evaluation methods.
6. Large Language Models
Large language models are trained on large text collections to predict tokens or related objectives.
During pretraining they acquire statistical representations of syntax, semantics, and many recurring
patterns in language.
Instruction tuning can make a pretrained model more useful for following user requests.
Retrieval-augmented generation can provide external information at inference time, while evaluation
remains necessary to detect incorrect or unsupported outputs.
Key review points
• Define the main concepts in your own words.
• Identify one practical example where the concept would be useful.
• Consider common limitations, failure modes, and evaluation methods.
7. Responsible NLP
NLP systems can reproduce biases present in training data. Privacy, security, misinformation, copyright,
and harmful content are important considerations.
Evaluation should include both overall quality and failure cases. Sensitive applications should use
human review, access controls, logging, and domain-specific validation where appropriate.
Key review points
• Define the main concepts in your own words.
• Identify one practical example where the concept would be useful.
• Consider common limitations, failure modes, and evaluation methods.