Scribd
Upload
26 views2 pages

Understanding Language Models in NLP

A language model assigns probabilities to sequences of words to indicate their likelihood of occurrence, ensuring that all possible sentence probabilities sum to one. Unlike formal grammars that classify sentences as valid or invalid, probabilistic models provide a soft decision-making approach, accommodating the ambiguity of real-world language. Applications of language models include speech recognition, machine translation, predictive text, and chatbots.

Uploaded by

Aniruddh Kumar
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
26 views2 pages

Understanding Language Models in NLP

A language model assigns probabilities to sequences of words to indicate their likelihood of occurrence, ensuring that all possible sentence probabilities sum to one. Unlike formal grammars that classify sentences as valid or invalid, probabilistic models provide a soft decision-making approach, accommodating the ambiguity of real-world language. Applications of language models include speech recognition, machine translation, predictive text, and chatbots.

Uploaded by

Aniruddh Kumar
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

LANGUAGE MODEL

A language model is designed to assign a probability to a sequence of words,


ensuring that it represents the likelihood of that sequence occurring in a language. (Language
model ka main goal yeh hota hai ki woh kisi sequence of words ko ek probability assign kare.
Yeh probability distribution ke zariye hota hai).
Here’s an explanation of the key concepts:

Goal of a Language Model


 What does it do?
A language model evaluates a sequence of words and assigns a probability to indicate
how likely it is to occur.
 Example: Sentence: "I am reading a book"
 Probability: P("Iamreadingabook")=0.003
 Probability Distribution:
A language model ensures that the probabilities of all possible sentences in the
language add up to 1:

Formal Grammars vs. Probabilistic Models


1. Formal Grammars:
o Hard Binary Decisions:

 Formal grammars decide whether a sentence is valid (legal) or not in a


language.
 Example: "I am going" (valid) vs. "I going am" (invalid).
o Types of Grammars:

 Regular Grammars: Handle simple structures and patterns, e.g.,


a*b+.
 Context-Free Grammars (CFG): Deal with more complex structures,
such as nested parentheses or recursion.
Example: S → aSb is valid, but aS → b is not.
o Limitation:

 These grammars do not assign probabilities; they only classify


sentences as valid or invalid.

1
2. Probabilistic Language Models (NLP):
o Soft, Probabilistic Decisions:

 Instead of strictly classifying sentences, NLP models assign


probabilities, indicating how likely a sentence is to belong to a
language.
 Example:
 "I am going" → P=0.7 (more likely).
 "I going am" → P=0.02 (less likely).
o Why Probabilistic Models?

 Real-world language is ambiguous and noisy. Probabilistic models can


handle these uncertainties better.

Probability Distribution in NLP


To ensure a correct probability distribution:
1. Valid Sentences: All sentences in the language must have a probability greater than
or equal to zero.
2. Normalization: The sum of probabilities for all possible sentences must equal 1.
This normalization makes the model’s probabilities meaningful and consistent with the rules
of probability.

Applications of Language Models


1. Speech Recognition: Converting spoken words to text based on likely word
sequences.
2. Machine Translation: Translating sentences while maintaining fluency and
grammatical correctness.
3. Predictive Text: Suggesting the next word or phrase based on the current context.
4. Chatbots: Generating contextually appropriate responses.

You might also like