Module 2 - Compiler
Module 2 - Compiler
Role of parser
● Syntax analysis or parsing is the second phase of a compiler.
● A lexical analyzer can identify tokens with the help of regular expressions and pattern
rules.
● But a lexical analyzer cannot check the syntax of a given sentence due to the
limitations of the regular expressions. Regular expressions cannot check
balancing tokens, such as parenthesis.
● Therefore, this phase uses context-free grammar (CFG), which is recognized by
push-down automata.
● The output of a syntax analyzer is a parse tree.
● For performing the syntax analysis, the grammar of the language has to be specified.
● CFG is used to define the grammar of the language. This process of verifying whether
an input string matches the grammar of the language is called parsing.
Error-Recovery Strategies
When an error is detected, the parser must recover to continue parsing.
● If errors are too many → compiler stops after a limit to avoid spurious errors.
1. Panic-Mode Recovery
● With this method, on discovering an error, the parser discards input symbols
one at a time until one of a designated set of synchronizing tokens is found. The
synchronizing tokens are usually delimiters, such as semicolon or }
● Pros: Simple, avoids infinite loops.
● Cons: May skip large portions of code.
2. Phrase -level recovery
● On discovering an error, a parser may perform local correction on the remaining
input; that is, it may replace a prefix of the remaining input by some string that
allows the parser to continue.
● Atypical local correction is to replace a comma by a semicolon, delete an
extraneous semicolon, or insert a missing semicolon.
4. Global correction
Attempts to transform the erroneous input into a valid program with minimal changes (insertions,
deletions, substitutions).
Ensures the corrected string is as close as possible to the original input.
Drawback: Very costly in time and space, so mostly theoretical.
T = Terminal symbols
P = Set of productions
S = Start symbol
A grammar G = (V, T, P, S) is said to be context free, if all productions in P have the form α→β, where |α | <=
|β| and α is element of V. That is, left-hand side contains only non- terminals. Right hand side is string
of terminals and/or non-terminals
2. Non terminals are syntactic variables that denote sets of strings. The non-terminals
define sets of strings that help define the language generated by the grammar. They also
impose a hierarchical structure on the language that is useful for both syntax analysis and
translation.
● A nonterminal called the head or left side of the production; this production defines
some of the strings denoted by the head.
● The symbol →. Sometimes : : = has been used in place of the arrow.
● A body or right side consisting of zero or more terminals and non-terminals.
All the production rules are of the form X→Y. Production rules are the heart of the
grammar. Consider the production rules
S → aSB, S → aB ,B → b
Here, V= {S, B} , T={a, b} and Starting symbol is S. Using this production rule , we
can derive the string aabb by
S → aSB
→ aaBB
→ aabB
→aabb
Here all the individual steps are called sentential form or Sequential form. All
steps together called Derivation
Eg: Let V= {S, C} , T={a, b} P={S→aCa, C→aCa, C→b}. Generate the string
2 2
a ba from the grammar given above?
S → aCa
→aaCaa
→aabaa
2 2
→ a ba
EXAMPLE:
Beginning with the start symbol, each rewriting step replaces a nonterminal by the
body of one of its productions.
E → E + E | E * E | E- E | ( E ) | id
We can take a single E and repeatedly apply productions in any order to get a
sequence of replacements. For example, E => - E => - (E) => - (id)
Example
Parse Tree
Parse tree is a hierarchical structure which represents the derivation of the
grammar to yield input strings.
Simply it is the graphical representation of derivations.
Root node of parse tree has the start symbol of the given grammar from where
the derivation proceeds.
Leaves of parse tree are labelled by non-terminals or terminals.
Each interior node is labelled by some non -terminals
If A ->xyz is a production, then the parse tree will have A as interior node
whose children are x, y and z from its left to right.
Yield Of Parse Tree
The leaves of the parse tree are labelled by non-terminals or terminals and read
from left to right, they constitute a sentential form, called the yield or frontier of
the tree.
Figure above represents the parse tree for the string id+ id*id. The string id +
id * id, is the yield of parse tree depicted in Figure.
AMBIGUITY
An ambiguous grammar is one that produces more than one leftmost or more
than one rightmost derivation for the same sentence
For most parsers, it is desirable that the grammar be made unambiguous, for if
it is not, we cannot uniquely determine which parse tree to select for a
sentence.
In other cases, it is convenient to use carefully chosen ambiguous grammars,
together with disambiguating rules that "throw away" undesirable parse trees,
leaving only one tree for each sentence.
Associativity and precedence of operators
1. Operator Precedence
2. Operator Associativity
Left to
1 (), [], . , ->
Right
3 *, /, % Left to
Precedence (High → Associativi
Operators
Low) ty
Right
Left to
4 +, -
Right
Left to
5 <<, >>
Right
Left to
6 <, <=, >, >=
Right
Left to
7 ==, !=
Right
Left to
8 & (bitwise AND)
Right
Left to
9 ^ (bitwise XOR)
Right
` (bitwise
10 `
OR)
Left to
11 && (logical AND)
Right
12 `
Right to
13 ?: (ternary)
Left
Right to
14 =, +=, -=, *=, /=, etc.
Left
Left to
15 (Lowest) , (comma)
Right
PARSING
Definition: Parsing is the process of determining if a string of token can be generated
by a grammar.
1. Top-Down Parsing
2. Bottom-Up Parsing
In top-down parsing, parse tree is constructed from top (root) to the bottom
(leaves).
In bottom-up parsing, parse tree is constructed from bottom (leaves)) to the top
(root).
It may involve backtracking, that is making repeated scans of input, to obtain the correct expansion of
the leftmost non-terminal.
A recursive-descent parsing program consists of a set of procedures, one for each nonterminal.
Execution begins with the procedure for the start symbol, which halts and announces success if its
procedure body scans the entire input string.
General recursive-descent may require backtracking; that is, it may require repeated scans over
the input.
However, backtracking is rarely needed to parse programming language constructs, so
backtracking parsers are not seen frequently
To construct a parse tree for this string top down, we initially create a tree consisting
of a single node labelled S.
An input pointer points to c, the first symbol of w. S has only one production, so
we use it to expand S and obtain the tree as:
The leftmost leaf, labelled c, matches the first symbol of input w, so we advance
the input pointer to a, the second symbol of w, and consider the next leaf,
labelled A.
Now, we expand A using the first alternative A → ab to obtain the tree as:
We have a match for the second input symbol, a, so we advance the input pointer
to d,
the third input symbol, and compare d against the next leaf, labelled b.
Since b does not match d, we report failure and go back to A to see whether
there is another alternative for A that has not been tried, but that might
produce a match.
In going back to A, we must reset the input pointer to position 2 , the position
it had when we first came to A, which means that the procedure for A must
store the input pointer in a local variable.
The second alternative for A produces the tree as:
The leaf a match the second symbol of w and the leaf d match the third symbol.
Since we have produced a parse tree for w, we halt and announce successful
completion of parsing. (that is the string parsed completely and the parser stops).
If the grammar is left recursive, the parser falls into infinite recursion (A → A
α | β)
These rules eliminate most common causes for backtracking although they do not guarantee a
completely backtrack-free parsing (called LL(1).)
Left Recursion
A grammar is said to be left –recursive if it has a non-terminal A such that there is a derivation
A A, for some string .
EXAMPLE
It recognizes the regular expression *. The problem is that if we use the first
production for top-down derivation, we will fall into an infinite derivation chain.
This is called left recursion.
Top–down parsing methods cannot handle left recursive grammars, so a
transformation that eliminates left-recursion is needed. The left-recursive pair of
productions A A| could be replaced by two non-recursive productions.
Q)
No matter how many A-productions there are, we can eliminate immediate left recursion from them by
the following technique. First, we group the A productions as
Left Factoring
are two A-productions, and the input begins with a non-empty string derived
from
we do not know whether to expand A to 1 or 2.
However, we may defer the decision by expanding A to B. Then, after seeing
the input derived from , we may expand B to 1 or 2 .
The construction of both top-down and bottom-up parsers is aided by two functions,
FIRST and FOLLOW, associated with a grammar G.
During top-down parsing, FIRST and FOLLOW allow us to choose which production to
apply, based on the next input symbol.
FIRST()
If 'α' is any string of grammar symbols, then FIRST(α) be the set of terminals that
begin the string derived from α .
If α==*>є then add є to FIRST(α). First is defined for both terminals and non-
terminals.
To Compute First Set
1. If X is a terminal , then FIRST(X) is {X}
2. If X є then add є to FIRST(X)
3. If X is a non-terminal and XY1Y2Y3...Yn , then put 'a' in
FIRST(X) if for some i, a is in FIRST(Yi) and є is in all of
FIRST(Y1),...FIRST(Yi-1).
FOLLOW()
FOLLOW is defined only for non-terminals of the grammar G.
It can be defined as the set of terminals of grammar G , which can
immediately follow the non-terminal in a production rule from start symbol.
In other words, if A is a nonterminal, then FOLLOW(A) is the set of terminals
'a' that can appear immediately to the right of A in some sentential form.
Rules to Compute Follow Set
1. If S is the start symbol, then add $ to the FOLLOW(S).
2. If there is a production rule A αBβ then everything in
FIRST(β) except for є is placed in FOLLOW(B).
3. If there is a production A αB , or a production AαBβ where
FIRST(β) contains
є then everything in FOLLOW(A) is in FOLLOW(B).
Example
LL(1) GRAMMAR
✔ Predictive Parsing
✔ Non-Recursive Descent Parsing
✔ Table-Driven Predictive Parsing
No Left Recursion
The grammar should not have left-recursive rules like −
A→Aα
The left recursion must be eliminated using left factoring or the idea called recursive
transformation.
No Ambiguity
Each nonterminal would have a unique production rule. That production rule can be
determined using a single token of lookahead.
That is
Q)
INPUT: Grammar G.
OUTPUT:Parsing table M.
Requirements
1. Stack
2. Parsing Table
3. Input Buffer
4. Parsing
Figure : Model of a non recursive predictive parser
Check whether the given input string is accepted by the parser or not
The parser is controlled by a program that behaves as follows. The program
considers X, the symbol on top of the stack, and a current input symbol. These
two symbols determine the action of the parser.
There are three possibilities,
1. If X = a = $ , the parser halts and announces successful
completion of parsing.
2. If X = a $ , the parser pops X off the stack and advances
the input pointer to the next input symbol,
3. If X is a nonterminal, the program consults entry M[X, a ] of
the parsing table M. The entry will be either an X-production
of the grammar or an error entry. If, for example, M [X, u ]=
{X UVW}, the parser replaces X on top of the stack by WVU
(with U on top). As output we shall assume that the parser
just prints the production used; any other code could be
executed here. If M[X, a] = error, the parser calls an error
recovery routine.
Consider grammar we have already seen it’s the parsing Table above. On input
id + id * id, the non -recursive predictive parser. These moves correspond to a
leftmost derivation (see Fig. 4.12 for the full derivation):
BOTTOM-UP PARSING
A bottom-up parse starts with the string of terminals itself and
builds from the leaves upward, working backwards to the
start symbol by applying the productions in reverse.
Along the way, a bottom-up parser searches for substrings of the
working string that match the right side of some production.
When it finds such a substring, it reduces it, i.e.,
substitutes the left side non-terminal for the
matching right side. The goal is to reduce all the
way up to the start symbol and report a successful
parse
The Theory behind bottom-up parsing involves
understanding the derivation process in reverse.
Reductions
Bottom up parsers begin with the input tokens and try to reduce
them into higher-level constructs.
bottom-up parsing use the process of reducing" a string w to the
start symbol of the grammar. At each reduction step, a
specific substring matching the body of a production is
replaced by the nonterminal at the head of that
production.
This continues until the entire input is reduced to the start
symbol.
Handle is a substring that matches with the right side of the production and
if the substring matches with the right side of the production, it is reduced
with the non- terminal on the left side of the production
EXAMPLE
E+E
E E*E E (E)
E id
E E + E
E + E * E
E + E * id3
E + id2 * id3
HANDLE PRUNING
STACK INPUT
$ w$
During a left-to-right scan of the input string, the parser shifts zero or
more input symbols onto the stack, until it is ready to reduce a string ‘β’
of grammar symbols on top of the stack.
It then reduces β to the head of the appropriate production. The parser
repeats this cycle until it has detected an error or until the stack contains the
start symbol and the input is empty as follows:
STACK INPUT
$S $
Upon entering this configuration, the parser halts and announces
successful completion of parsing.
There are actually four possible actions a shift-reduce parser can make:
Shift
Reduce
Accept
Error
Example:
$ w$
Example:
Consider the grammar E → E+E | E*E | id. Input string is id+id*id .The
implementation is as follows:
LR PARSING
LR parsing is most efficient method of bottom-up parsing which can
be used to parse large class of context free grammar.
The technique is called LR(k) parsing; the “L” is for left to right
scanning of input symbol, the “R” for constructing right most
derivation in reverse, and the k for the number of input symbols
of lookahead that are used in making parsing decision.
The cases k =0 or k = 1 are of practical interest, and we shall only
consider LR parsers with k =1 here. When (k) is omitted, k is assumed to
be 1.
LR(0) and SLR -parsing table is generated with canonical collection of LR(0)
item. For CLR and LALR by canonical collection of LR(1) item
The input buffer takes one symbol at a time from left to right a1, a2,..., am,$.
The stack contains the combination of the state symbol and the current input symbol.
The parsing table is a two-dimensional array containing the Action and Goto tables.
Augmented grammar
G’ is the augmented grammar of G with a new production rule S’→S where S’ is the new starting
symbol i.e,
This is done to signal to the parser when the parsing should stop to announce acceptance of input.
We will apply this rule until no more new LR(0) items can be added to closure(I).
EE’
GOTO Operation
The GOTO function is used to represent the transitions in the LR(0) automaton for a
grammar. The states of the automaton correspond to sets of items, and GOTO(I;X)
species the transition from the state for I under input X.
EXAMPLE
{ E→E+.T,
T→.T*F
T →.F
F →.(E)
F →.id }
LR(0) Automaton
Constructing SLR parsing table
Constructing SLR Parsing Table
INPUT: An augmented grammar G'.
OUTPUT: The SLR parsing table functions action and goto for G' .
METHOD:
If any conflicting actions are generated by the above rules, we say the
grammar is not SLR (1). The algorithm fails to produce a parser in this case.
3. The goto transitions for state i are constructed for all non-terminals A
using the rule: if GOTO(Ii , A)=Ij then GOTO[I, A]=j .
4. All entries not defined by rules (2) and (3) are made “error”.
5. The initial state of the parser is the one constructed from the set of items
containing [S’ S].
Shift/Reduce and Reduce/Reduce conflicts
If a state does not know whether it will make a shift operation or reduction
for a terminal, we say that there is a shift/reduce conflict.
If a state does not know whether it will make a reduction operation using the
production rule i or j for a terminal, we say that there is a reduce/reduce
conflict.
If the SLR parsing table of a grammar G has a conflict, we say that that
grammar is not SLR grammar.
Stack Implementation
Example:
Find the SLR or LR parsing table for
SAA FOLLOW(S)={$}
AaA/b FOLLOW(A)={a,b,$}