TOKENS
Smallest individual unit in a program is called a token.
They are:
K eywords
I dentifiers
L iterals
P unctuators
O perators
My funda of remembering Tokens: KILPO it’s just a shortcut given by me for remembrance ;)
TOKEN KEYWORDS
Keywords are the words that convey a special meaning to the
language compiler/interpreter. They are reserved for special
purpose and must not be used as normal identifier names.
and exec not
as finally or
assert for pass
break from print
class global raise
continue if return
def import try
del in while
elif is with
else lambda yield
except
TOKEN IDENTIFIERS
Identifiers are the fundamental building blocks of a program which
are used to identify a variable, function name, class name, module
name or any object.
RULES OF WRITING AN IDENTIFIER
An identifier starts with a letter A to Z, a to z or an underscore ( _ ) followed by
zero or more letters, underscores and digits (0 to 9).
Identifier name canNOT start with a digit.
Python does NOT allow special characters other than underscore ( _ ).
Identifier must NOT be a keyword of Python.
Python is a case sensitive programming language that is uppercase letters
(capital letters) and lowercase letters (small letters) are treated differently.
Eg. Marks and marks are two different identifiers in Python.
• Some valid identifiers : Mymarks , file12 , tv9r , Avg_2 , _no
• Some invalid identifier : 4Qtr , global , XI-CS , E mail , %age
Starts with digit keyword
Special characters - (hyphen) space %
TOKEN LITERALS / values
Literals are data items that have a fixed value.
Python allows several kind of literals:
Numeric Literals
• Int (signed integers) Positive or Negative
whole numbers with no decimal point Eg.
10, -96, 1234, 0
• Floating point/Real values float represent
real numbers and are written with a decimal
point dividing the integer and fractional part.
Eg 2.0,54.7,-12.0,-0.075, .4
• Complex (complex numbers) a+bj , where
a and b are floats and J(or j) represents √-1 ,
which is an imaginary number. a is the real
part and b is the imaginary part of the
number.
TOKEN LITERALS contd..
Boolean Literals
True and False are the only two Boolean
values
Special Literal None
None literal represents absence of a value.
That is why nothing is printed
for the value of b.
String Literal
String literal is a sequence of characters
surrounded by quotes (single or double or
triple)
String literals can be written in either single
quote ‘ ’ or double quote “ ”
TOKEN LITERALS contd..
ESCAPE SEQUENCES / Non-graphic characters
TOKEN LITERALS contd..
ESCAPE SEQUENCE Examples
The cursor moves to the next
line after printing ‘Hello’
The cursor keeps one tab
space after printing
‘wonderful’
The string when written
inside ‘ ‘ single quote
gives an error when we
try to print one ‘
(apostrophe symbol).
So, to remove the error
we use \ escape
sequence.
The same thing written
inside “ double quotes
gives no error.
TOKEN PUNCTUATORS
Punctuators are certain symbols which are used in
the programming languages to organize
programming sentence and structures, and indicate
the rhythm and emphasis of expressions, statements,
and program structure.
Most common punctuators of Python programming
language are:
‘ “ # \ ( ) [] {} @ , : . ` =
TOKEN OPERATORS
Operators are tokens that trigger some computation/ action when
applied to variables and other objects in an expression.
UNARY OPERATORS
Unary + , Unary - , ~ Bitwise complement , not logical negation
BINARY OPERATORS
ARITHMETIC OPERATORS: +,-,*,/,%,//,**
AUGMENTED ASSIGNMENT OPERATORS: = , +=,-=,*=,/=,%=,//=,**=
RELATIONAL OPERATORS: >,<,>=,<=,==,!=
LOGICAL OPERATORS: and , or, not
MEMBERSHIP OPERATORS: in, not in
IDENTITY OPERATORS: is, is not
BITWISE OPERATORS: &(Bitwise AND), ^(Bitwise XOR), |(Bitwise OR)
SHIFT OPERATORS: << (Shift Left) , >> (Shift Right)