44

Mod 4: Semantic
Analysis
Page No. Date

Lexical demantics. It is the

study of word meanings.
Lo.
& the relationships between
words.
→ It focuses on how words
represent meaning
language & how those meanings
can be understood & processed
by
machines.

Lexeme: A lexeme is the base form of a

word
Ex: Runs, Running →
Run.
Kan ↳ lexemed

Relationship betwen lexemes &

lenses..
Homonymy
: Means words.
are
in

same
pronouncation
are same, but diff
meaning.
EM: 16 Give me that bat
 24 Anushka saw a bat flying
in sky.:
Hypernymy &
Hyponymy:
ay Hyponymy: Basica
La
• Basically a subclass of
ally Ex.: Dog is a animal..
Dog,
Is hyponymy of
animal
For: Rose is a hyponymy
of
a

parent class
 flower.
by Hypernymy: Relation between sense &
a superdess.
Ex: Animal is a
Ex
fruit is

with the
9

hypanymy of Derg hypernymy

of Mango
AFT

at a some

⇒ A word com be hyponymy &

hypermymy at m
Jon Color

Purple!!! light
 purphy Dank
qumple
фигры
time
Date

Meronymy: Being of a part of

something.
If x is

of y
a part af y,
fx Coconut is a
$201

In
nevonymy
then 21 is

"Meronymy of
Coceanut Tree
 : words having similar
meaning Ex. Beautiful
Gorgeous
Purchase Brey
IV.

Synony
my?

Rich

Antonymy:
Meanings
are

vi
other.

Borase Ex: Hot →→Cold

 Good Bad
J
T

Wealthy.
A

Opposites /contrant
with each

Day Night !!
Polysemy. And loveme has
multiple related
Senses
meaning
Ex: iy. Write your name on
the paper..
• He published a
paper
600

*Wordnet:
•

Did you
a paper on AIGUN
read the
paper
today?
ay. He hit his head.
•She is the head of
dept.
‫الدین‬

→ Before this we will

Inning

will study what is

Semantic Analyss
 pixess to find
the
meaning
isaa

+ Semantic Analysis is
a
of
a text.

Now this meaning is not the actual

meaning which humans understand, instead its
the meaning which computer / machines
understand
Basically Semantic Analysis
gives the power to computer's to
understand the text meaning.
Page No.

Date
• Ambiguity? A word
having multiple
meaning.

во
is called ambiguity,
me a base bat
Ex. Give me o

suppose
wooden bat

flying bird
machine
 know there is a sentence & toy needs

to find it's meaning, it uses

semantic Analysis to find it's meaning.
- But if that sentence contains ambiguity
ic. One word having multiple meanings, so
here first its
sentence non-ambiguous..
will make the sentence non-

i.e. It will
one the
amonguin..
& then it will do the further processing; as
so when semantic Analysis goes into the
sentence
 "Wordnet
Ambiguity, it user
"Wordnet"
dictionary,
to remore

- So basically "Wodnet" is a
has all the words &
meanings. to
Semantic
Analysis can identify
meaning at
of that, worth for from
multiple according to its context.,
wi
the
which

prop
words
 hence serwis Ambiguity is
folved. So basically."
"Demantic Analysis & wordnet"
goes hand in
So wordnet is a
hand in hand

dictionary having
multiple words
with ith diff
meaning.
I By using wordnet we can
the words.

find diffi relations

betreen
Page No.

Date
Frame Netinu
fourses on Understanding
the meanings of words
in
content, based on the idea of
of
frament is a lexical database based on
Semantics:
grausbere
→ Each frame describes a scenario & the
rotes (frame elements involved
I

Shich
frame

a
 Ex: To understand the word "Buy", you
need a Commercial. Transaction frame, which
includes:
Buyer Seller, Goods,
money.
frame: Commercial Transaction. dol
-Lexical Units: buy, sell, purchase
pay.
• Frame Elements (semantic Roles))
Buyerah When
dom
Seller

rosele

Goods

Money
Sentence: "John bought

Frame Elements
 Вице
R

a cay from Saman for '5000$"

Sentence Penting
John

Goods Seller

Money
→ Farframe Net is a
Ca
Salah

$5000

powerful NIP resource

fur
Semantic Role labeling, helping
Machines. understand who did what to whom,
when, where
६
q why
Page No.

Date

* Robust word dense Disambiguation

(WSD) is to remove
Ambiguity

⇒ A word having multiple

meanings is called ambiguity
Ex: She killed
him with a

Mammal
wooden oby

dala

Robust WSD is a process of

accmately determining
given
which meaning of a word is intended in a
context, even when faced with
ambiguous noisy → RISD should
perform well across various
domains, languages &
usuage styles. &
a courd has multiple Lash
job is : If meanings, then from than multiple
words which wong is
suitable for that
particular sentence according
meaning
of
of the content is choosen.
to its

Different
 Approaches:
iy knowledge Bared
Appreach!
• Resources Required: y
Raw Corpora
26 Machine Readable Dictiona

Diction
ary
J

Walki
ng

eventual
ly
Ex: The bank can guarantee
deposits will
cover future tullen costs because it
invests in adjustable rate
mortgage
Bank → It is ambiguous, ws@
job is to remove
ambiguity by
using.
WSD"
Page No.

Date

Now is WSD we have 2

2 bags.
14 Sense Bag
My Context
Bag.
-Sense Bag: Bank is
ambi
ambiguous

dictionary
going
So how the no. of meanings of "Bank" in a

Ex: Bank 2 Sense: A financial Institution

that accepts
Ex: "That bank holds mortgage on
my
to be considered as 1 Sense B
Ba

accepts dis
Bank 2 Sense :
Sloping
land
 especially
home" beside a
body
'coater.

af
Ex: I sat and the bank.
Content

Except word "Bank" haviny

ambiguous
Bag: Except
Consider all other words in the sentence So
all these couds is refer as "Content Bag"

→ Now Content Bags is been

compared with
all the sense Bank
post if max words New
A sense Bags - &
वे
 & Context
Rags
I matching
then the meaning
is selected. 4 particuled sense
Bag is selected. → to the
example, is related
• Hence the meaning
=
во

⇒
A
is

words are
that

is related to Benk I Senu

choosen.
Supervised Machine
Learning
It's like teaching a child with
excamples. you give computer lots
of sentences where the
of the word is already
known
Edlike a labelled
workbook). correct
meaning of the word
ii)
29,
Page No.

Date

Computer learns pattern is

how each sense is used sees a new
sentence, it uses what
the conect meaning.
why the
computer it have learned
& guess
Ex

Training
Sentence
Data
‫او‬
He hit a ball with a bat
The bat flew out of care.
The computer sees
that.
Meaning of "batt Sports equipment
Flying animal

• "hit" "ball" likely "bat =

sports equipment
• "Flew" "Cave" — likely "bat" =
animal
Ꮧ

By

you give
"The bat flew over the bees'
8 if

It
 pudicts: "bat"
=

flying
scanimal.

SUM
used can be Decision Tree Nemal N/w
Accurate displ
-Adv: iy Very
Accurate
& Learns specific
meanings clearly
His adv: Needs lots of labeled
sentences
27 Doesnt do well if it sees new or lave
meanings..
 This method is

by Unsupervised Machine
Learning Approach!
like computer having to
learn on being told the
correct answer.

its own,
with he
How it works: s
• The
•It

computer collects many sentences

when word
It looks at
 nearby.
words (content) &
appears.
groups
similar ones ・gether (cluster
Page No.

Date

£x!
bat

My He hit the ball with a 24 The scourg

the bat at the ball. 3) A but flew in
the night
4) The bat hangs upside down,
It notices:
→ Sentences 1
& I have words like
"hit" "ball" "swung".

→ Sentences
‫ما‬

Group
 1

3 & 4 have "flow" "night" "hangs"

-

• Group 2
So it mates?

·Group 1: Group
Sports equipment
2

: Animal

- Algo used can be k-mears Clustering,

word Embeddings.
- Adr!

Disadvis
Aut

& No need of labelled data. a) can

discover new or have meaning,
 a

Less accuate ligan Supersed

Neural Models for dementias

i) Sequence to Sequence
Model:

the

seg of info is called

→ If my model takes
sequen seg.
te

info. & it outputs

seg model.
Ex : 7
If air
Page No.
Date
 I want to convert English to
French is
seg to leg learning
a) If I have given
given image than
than that image can
be converted into text, it is called.
Captioning
Amag
e
Capt. on Google Search I
write "Dog", automatic
2. If in
been shown so it is called
ded image
 на
seg to seg

• In
Ib seg te seg we have
2

main components
-Encodey 17 15
J

Decoders.
Encode
0/P
-vecter

[w]
A content rector.
य
។
Z
<EOS>

'ttime
ti
t2

dosto
A
B

<EOS > W
2
 Y
Z

1071

Na N/W
→ Now we have taken

Supposi

english
any
I nemal n/w : ORON
LSTMs
we look LSTM
2 LSTM

3 GRU-

Now in Encoder suppose

we passed.
sh besit & coanted to convert it in French
& it will
pass
through
the encoder & the
opp is desired in the last block at encoder when
the
english
text ends.

So basically we cam called

this alp Now this vector content
as Content Vector [w
is passed to the decoder.

6/
Date

1) Attention Mechanisms:
It is a

concept in NUP that allows model

to focus on specific parts of the
specific parts of the ilp
sequence
each part of the ofp.
 cohen
producing
When
processing
a
a word in a
sequence,
which
other words in the Sequence should
the
more attention to ?"..!
- So instead of treating every ilp word
equally attention lets the model
dynamically weigh the importance of diff
words based in content. → Importance:

• Traditional models (like RNNs)

 had difficulty remembering long
dequences.
Attention overcomes this by creating
direct connection between relevant words, no
matter how far apart, then are in the
sentence.
This leads to better content
understanding & more accurate
predictions, expecially in tasks lite
translation, summarization etc...
waking:
Givers a leg af words, the model learns three
vectors for each word:

५
‫هللا‬
 Query (8): what we're
looking for. Key (K): what use
have.
value (v) = what we pass on to
next layer
The attention sure is
computed ushy:
T

Attention (Q.,X, V) =
Seftinian (18 x x + Y √ dx
)xV
where, &xx" → computes
dimilarity
K
->>

Converts Scores
Page No.

Dato

behneen words.
 Inte probabilities
Sceftmax
The result is

weighted sum of values

(v).
a

where weights reflect how much

cettention each
wold should get.
Types of attention..
self.
Intra Attenting.
luft Attention: (most common): Differentiable &
trainab Each word in a sequence attends all other
words, includiy
itself.
Ex: "The cat sat on the mat".
"The word "sat"
may
 attend to "cat" .to
understand the action

Inter Attention/Cross Attention:

- One dequence attends to another
sequence.
-

useful
In tasks like translation. Common
in encoder, decoder
encoder decoder Architectures Ex:
In Machine
Translation (Eng→ french)
the decoder generating french word
attend to Melevant English words in
i/p.
in
Jeft Attention: Assigns
probability distribution over i/p
tokens.
- Most coidely used cx it can be trained and
‫هـ‬

Ex.:." The" 0·1

J

'cat'
04
"Sat
11

Hand
Attention :
Selects one a
few top 2 32
to end

to focus
→ Non differentiable (reg reinforcement
leaving
Sampling
Gr
en

Ex: The model selects only "Cat" & "sat"

ignores other
it is more efficient.
Date

Language
Generation Models
- Language Generation models
in NLP
to produce Human-like
Types
:
e Human-like tent...
ari

My GPT (Generative Pre-trained

Transformer)
Get
generates
o create text.
designe
d

GPT is not just reading or

understanding text, its actually writing
tent like completing a story, sentence,
answering questions on ever
Ex:

Codi
ng

You hype:
"Once
upon
9

time there was a

diag
on
who....

GPT might complete its : lived in a

snowy
mountain & guarded ancient
treasure?
 you
the
use it.
→ Pre-brained. The model is trained
before. fort. It learns by reading a
lot of text, from
<«internet (books, articles, websites) so it
understands.
how language
works.
Then it's fine-tuned
(optional) a used directly for
tasks like thatting writting,
translating, summarizing etc
Transform
er:
2
 - A type a nemal /w
architecture used to
text
proce
ss
- Transformers are powerful
models that use something called
attention mechanism to understand
relationships
between words.
T

Page No.

Date

Ex: "The Cat sat on the

nat because it was tired".
a bansforme knows "it" reperes to "the
Cat:
 Working :/
steps
1. You
"

you give it a prompt (sentence

on
few words)
CAPT uses its training to predict the next
word, one at a time.
It keeps adding words until it completes
a freel
Mesponse.

Rx: Prompt
AJ is Changing the world by GPt
completes = "imploring healthcare,
education & Communication
 technologies".
= key concepts
in GPT.
• Token: A word or part of a
coord
• Auto regressive: Generates one ward
at a time,
•
from left to right.
• Language Model: A system that
undestands &
generates human
language.
E.
 Attention ?
•Parameters
Focuses on
most

imp
words in
a sentence.

Trainin
g
;

"Memory" or "knowledge of the

model. Data: Books, articles, websites GPT
read
• Loss
funcion
languag
e

to learn my
patterns.
7. Tom

To measure how far CPT's

predictions
are

from
the actual next word

Ex: How GYPT Predicts Nent coords.

It

"/p: "I love eatly chocolate

chip"
 predicts nent word by surry possible
candidates
scoring
Page No.

Date

d.

3.
word
Cookies

pizza
pencils
Score
0.60

0.20

0.01
 16 picks "cookies"
Architecture:

Embedding Layer. Converts words into

numbers. Positional Brcoding:
Adds word order information Transform a Blocks
Apply self Ottenbin & denne layers.

ofp Layer Predicts the next

word
дру
• GPT

family
 /versions
- APT-2
•GPT-3 - GPT-4
→ 2018

→ 2019
→ 2020

2023

- GPT 4.5/ GPT.5

(coming (expected)
* Auto en coders:

They find efficient way to

compress large data
like
to much smaller

Auts encoders are powerful

neured in the developed
representation
 าก

Aubencoder consists of of
parts.
--

Encoder
Decode

Latent Space
/P

2
Encode]
<
222

Bottleneck

Z
Page No.

Date

>Reconstructed i/p

Duoda
x1

into a

Io an auti encoder efficiently

 compresses the ilp
latent (hidden)
representation & then
Meconstructing the original ilf from it.
- Encoder: Compresses
i/p into
Q

lafent vector

Latent Space: Compressed, dense

representation cz it
-folles n/w to kain meaningful features in
fewer dimensing: Decoder Reconstructs the
i/p from the latent
as close as it can.

Fx: Imagine you

representat

have
 big
box af toys, you
need to
need to

put there toys into small bere he you use

the most efficient way to fit toys in this
smaller
ben
980 similary aut encoders find
efficient ways Compresslenge
data like images into smaller
- It is Used for:
t

represent

unnecessary info
1. Noise Reduction: Used to filter out un
11. Feature Extraction: Focuses on
 Types:
~ Denoising
Autoenco de
dala

imp features in
dold

2) Variations Autoencoder (VAF)

3) deg to sequence Autrencoder-
Page No.

Date

* Vaviational Autoencoders [VAES)

VAE S
дне
a

learningque -
They
to the
262

(23)

χι

type of generative modd

in
deep
Care used to generate new data
that is dimila
orginal
Latent state distribuhny

Mean

M2

Variance

or new Samples Similar

 bill
sample from
↑

31.

distribution

205

decoder
encode

VAES learn

learn a probabiling
probabiling distribution
over the
latent space
A Difference behreen Auto encodes & Variational
Af's
+ Advanced NUP Techniques:
My Transfer
hearning
in NCP:

pre-trained
> It is the technique of meusing a
on a new task, Mather than
a model from a scratch.
model

training
Page No.

Date

Benefits
:
needed
Reduce time & Date for
A

new tasks

beverages knowledge learned from a

 huge corpus
By Few- shot
hearning
in NCP.

a model to
perform
Refers to training or
promptinged examples
using only a few
labeted
a task using
only
Workin
g:
 the model we
use in-context.

• Uses pretained large

language Models (GPT-3, ck)
Instead of fine-thening learning.
34 Lew. Shot
Learning
Zew that means
in NIP

the model solves a task without

Seeing any examples of
that task
For this

it uses semantic knowledge

 learned
during large scale
pretraining
Relives CA Natural
Language prompts to
descube the task

* Comparision between
Advanced Nep Techniques.