Parsing Tutorial - Questions
Introduction
The purpose of this practical session is to gain more insight into using parsers and grammars and to get
familiarized with the tools provided to this end by NLTK. Our goals are to:
• Understand the process of parsing better by doing some examples by hand.
• Gain insight into grammars by writing a grammar of our own.
• Experiment with training a grammar from a corpus and evaluating it.
Useful NLTK commands
Grammar creation from a given string:
g = nltk . CFG . fromstring (""" < grammar rules here >""")
Use PCFG instead of CFG if you write a probabilistic context-free grammar.
Tokenize a sentence:
tokens = nltk . word_tokenize (" < sentence >")
Create different types of parsers starting from a grammar:
chart_parser = nltk . ChartParser ( g )
viterbi_parser = nltk . ViterbiParser ( g )
Parse a tokenized text:
trees = viterbi_parser . parse ( tokens ) # for example , for the Viterbi parser
Display the parse trees:
[If not done yet, you first need to install this library: [Link]
install_Tkinter]
The command below will generate a popup window, containing the graphical representation of the
tree; while the parse tree drawing window is open, your script will be paused. The script will be resumed
when you close the window.
from nltk . tree import *
from nltk . draw import tree
tree . draw ()
Section 1: Warm-up
1. In general, given a raw text, would it be possible to directly use a parser in order to generate a
parse tree? If not, explain what pre-processing steps are necessary to perform on raw texts prior
to using a parser. If yes, explain why it is possible.
1
2. Have a look at the texts and at the grammar below.
Input texts:
• I saw an elephant
• I saw an elephant in my pajamas
Grammar:
S -> NP VP [1.0]
PP -> P NP [1.0]
NP -> Det N [0.4]
NP -> Det N PP [0.2]
VP -> V NP [0.5]
VP -> VP PP [0.5]
NP -> ’I ’ [0.4]
Det -> ’an ’ [0.5]
Det -> ’my ’ [0.5]
N -> ’ elephant ’ [0.5]
N -> ’ pajamas ’ [0.5]
V -> ’saw ’ [1.0]
P -> ’in ’ [1.0]
3. Using the grammar, manually construct the parse tree(s) for the two sentences above; apply
whichever method you find suitable.
4. Now, it is time to verify your intuition and compare your solutions to the solutions obtained using
existing NLTK parsers. Try parsing with the NLTK Chart Parser and then with the NLTK Viterbi
parser.
5. Examine the output of the parsers. Compare the output to your intuitive parse trees. Is there any
difference?
6. Using the NLTK GUI, draw the most probable parse trees found by the NLTK parsers and compare
them to the trees you have found manually. How are they different? Do they differ only in the
names of the nonterminals used, or is there a structural difference as well? Why do you think these
differences occur?
Section 2: Hand-written grammars
1. You are given the following sentences:
• Alice wondered with Bob in the empty city streets
• Bob offered Alice an iguana for her birthday
• Alice gave an inspiring speech at the conference on education
Design a non-probabilistic context-free grammar for parsing these sentences and then manually
provide the parse tree(s) for these sentences. Here is a list of lexical rules and syntactic categories
that you could use to devise the grammar:
Lexical rules:
V -> ’ wondered ’ | ’ offered ’ | ’ gave ’
NP -> ’ Alice ’ | ’Bob ’
N -> ’ city ’ | ’ streets ’ | ’ iguana ’ | ’ birthday ’ | ’ speech ’ | ’ conference ’ |
’ education ’
Adj -> ’ empty ’ | ’ inspiring ’
Det -> ’the ’ | ’an ’ | ’her ’
Prep -> ’ with ’ | ’in ’ | ’for ’ | ’at ’ | ’on ’
Suggested syntactic categories (non-terminals): S - Sentence, NP - Noun Phrase, VP - Verb
Phrase, PNP - Prepositional Noun Phrase.
2
2. With the grammar you designed in point (1.), parse the above sentences using NLTK’s Chart
parser.
3. Display and compare the parse trees obtained in points (1.) and (2.).
Section 3: Extracting a Grammar from a Corpus
1. Download the corpus first using [Link](’treebank’)
2. Split the corpus into two parts: one for learning the grammar (learning set) and one for evaluating
it (test set). Use 3% of the total number of sentences for the test set.
3. Extract the grammar out of the learning corpus. You will need the entire lexicon (i.e. rules of
the type PoS − > word) even though the syntactic rules are extracted from only a portion of the
treebank.
4. Try to have a look at the grammar. How many rules are there?
5. Evaluate the performance of the grammars on the test set. What proportion of sentences from the
test set have been parsed as in the reference (i.e. are correctly parsed) by the extracted grammar?
6. We now want to study the size of the grammar with respect to the size of the learning set. Split the
former learning set into two parts: one kept for actual learning and another part which is not used.
Do this for several percentages (10%, 50%, 70%, 80%, 90%, 100%). Have a look at the resulting
grammars and try to understand their differences.
Section 4 (Optional): Error Analysis, Improving the Grammar
Take the grammar that is learned based on 100% of the learning set and a subset of sentences from the
test set (∼10 sentences).
1. Is the reference (i.e. the parse in the corpus, for the chosen sentence in the test set) correct
according to you?
2. If the reference is correct and the parsing result is incorrect, add the corresponding correct parse
to the learning set and learn again as long as you do not get the correct parse. As you are doing
this, is the performance improving on the rest of the test set?
3. If the reference is incorrect, correct it in the test set and start the evaluation again. Iterate the
process and study the performance on the test set.