Daa notes 1

Design and Analysis of
Algorithms
Prepared By:
Smruti Smaraki Sarangi
Asst. Professor
IMS Unison University, Dehradun
Module - 1

Algorithm:
 An algorithm is a process to solve a problem manually in
sequence with finite number of steps.
 It is a set of rules that must be followed when solving a
specific problem.
 It is a well-defined computational procedure which takes
some value or set of values as input and generates some
set of value as output.
 So, an algorithm is defined as a finite sequence of
computational steps, that transforms to given input into
the output for a given problem.

 An algorithm is considered to be correct, if for every input
instance, it generates the correct output and gets
terminated.
 So a correct algorithm solves a given computational
problem and gives the desired output.
 The main objectives of algorithm is
 To solve a problem manually in sequence with finite no.
of steps
 For designing an algorithm, we need to construct an
efficient solution for a problem.

Why we read Algorithm:
 We read the algorithm.
 For testing the good programmer in 2 levels. That is:
i. Macro-level (Implementation of algorithm like s/w)
ii. Micro-level (Algorithm part like h/w)
 For designing an algorithm, we have to learn to solve
computational problem in economics.
 By applying mathematical algebra, we get computational
logic.

Algorithm Paradigm:
 It includes 4 steps. That is:
 Design of algorithm
 Algorithm validation
 Analysis of algorithms
 Algorithm testing
1. Design of algorithm:
 Various designing techniques are available which
yield good and useful algorithm.
 These techniques are not only applicable to only
computer science, but also to other areas, such as
operation research and electrical engineering.

 The techniques are: divide and conquer, incremental
approach, dynamic programming etc. By studying this
we can formulate good algorithm.
2. Algorithm validation:
 Algorithm validation checks the algorithm result for
all legal set of input.
 After designing, it is necessary to check the
algorithm, whether it computes the correct and desired
result or not for all possible legal set of input.
 Here the algorithm is not converted into the program.
But after showing the validity of the method, a
program is written. This is known as “program
providing” or “program verification”.
 Here we check the program output for all possible set
of input.

 It requires that, each statement should be precisely
defined and all basic operations can be correctly
provided.
3. Analysis of algorithms:
 The analysis of algorithm focuses on time complexity
or space complexity.
 The amount of memory needed by program to run to
completion is referred to as space complexity.
 The amount of time needed by an algorithm to run to
completion is referred to as time complexity.
 For an algorithm time complexity depends upon the
size of the input, thus is a function of input size „n‟.
 Usually, we deal with the best case time, average case
time and worst case time for an algorithm.

 The minimum amount of time that an algorithm
requires for an input size „n‟, is referred to as Best
Case Time Complexity.
 Average Case Time Complexity is the execution of an
algorithm having typical input data of size „n‟.
 The maximum amount of time needed by an
algorithm for an input size „n‟ is referred to as Worst
Case Time Complexity.
4. Algorithm testing:
 This phase involves testing of a program. It consists
of two phases. That is: Debugging and Performance
Measurement.
 Debugging is the process of finding and correcting the
cause at variance with the desired and observed
behaviors.

 Debugging can only point to the presence of errors,
but not their absence.
 The Performance Measurement or Profiling precise
by described the correct program execution for all
possible data sets and it takes time and space to
compute results.
NOTES:
 While designing and analyzing an algorithm, two
fundamental issue to be considered. That is:
1. Correctness of the algorithm
2. Efficiency of the algorithm
 While designing the algorithm, it should be clear, simple
and should be unambiguous.
 The characteristics of algorithm is: finiteness,
definiteness, efficiency, input and output.

Analysis of Algorithms:
 Analysis of algorithms depend upon various factors, such
as memory, communication bandwidth or computer
hardware. But the most often used is the computational
time that an algorithm requires for completing the given
task.
 As algorithms are machine and language independent these
are the only important, durable and original parts of
computer science. Thus, we will do all our design and
implementation for the RAM model of computation.
 In RAM model, all instructions are executed sequentially
one after another with concurrent operations. In performing
simple operations like addition, subtraction, assignment etc.
model takes 1 step.

 A call to a subroutine and loops are not single step
operation. Instead each memory access takes exactly one
step. By counting the number of steps the running time of
an algorithm is reassured.
 The analysis of an algorithm focuses on the time and space
complexity. The space complexity refers to the amount of
memory required by an algorithm to run completion.
 Time complexity is a function of input size „n‟. It is
referred to as the amount of time required by an algorithm
to run to completion.
 Perhaps different time can arise for the same algorithm, we
usually refer best case, average case, worst case
complexity.

1. Worst-Case Time Complexity:
 The worst-case time complexity is a function defined
by the maximum amount of time needed by an
algorithm for an input size „n‟. Thus, it is the function
defined by the maximum no. of steps taken on any
instance of size „n‟.
 A worst case estimate is normally computed, because
it provides an upper bound for all inputs including
particularly the bad ones.
N
21
Worst Case
Average Case
Best Case
No. of
Steps

2. Average-Case Time Complexity:
 The average case time complexity is the execution of
an algorithm having typical input data of size „n‟, thus
if the function by the average no. of steps taken on
any instance of size „n‟.
 Average-case analysis does not provide the upper-
bound and it is difficult to compute.
3. Best-Case Time Complexity:
 The best-case time complexity is the maximum
amount of time that an algorithm requires for an input
of size „n‟, thus it is the function defined by the
minimum no. of steps taken on any instance of size
„n‟.
 All this time complexities define a numerical function
time ~ size.

Calculation of Running
Time:
 There are several ways to estimate the running time of a
program. If 2 programs are expected to take similar times,
probably the best way to decide which is faster is to code
them both up and run them.
 Generally there are several algorithmic ideas and we would
like to estimate the bad ones early.
 So, an analysis is usually required furthermore the ability to
do an analysis usually provides insight into designing
efficient algorithm.
 The analysis also generally pin points, which are worth
coding carefully.

 To simplify the analysis, we will adopt the conversion that
there are no particular units of time. Thus, we throw away
low ordered terms.
 So, what we are essentially doing is computing a big oh (O)
running time. Since big oh (O) is an upper bound, we must
be careful never to underestimate the running time of the
program.
 In effect, the answer provided is a guarantee that the
program will terminate within a certain time period. The
program may stop earlier than this, but never later.

Analyzing the control
structure:
 Sequencing:
 Let „P1‟ and „P2‟ be two fragments of an algorithm they
may be single instructions or complicated sub-
algorithms. Let „t1‟ and „t2‟ be the times taken by „P1‟ and
„P2‟ respectively. These times may depend on various
parameters such as the instance size.
 The sequencing root says that the time required
computing P1, P2. i.e.: 1st P1 then P2 is simply t1 + t2 by
maximum rule, the time will be O(max(t1, t2)).
 E.g.: i) t1 = θ(n), t2 = θ(n2). So, the computational time is:
t2 = θ(n2).
ii) if t1 = θ(n), t2 = θ(n2) => t2 = O(n2).
iii) if t1 = O(n), t2 = θ(n2) => t2 = θ(n2).

 Add the time individual statements. The maximum is the
one that count.
 If then else:
 Again consider P1 and P2 be the parts of an algorithm,
with computation time t1 and t2 respectively. Now „P1‟ is
compared only when the given condition is true.
Otherwise for the false condition „P2‟ is computed. Thus,
the total time is according to the conditional rule „if then
else‟.
 According to the maximum rule this computation time is:
max(t1, t2).
E.g.:
i) Suppose P1 = t1 = θ(n), P2 = t2 = θ(n2) => T(n) = θ(n2).
ii) t1 = O(n2), t2 = θ(n2) => T(n) = O(n2) or θ(n2) => O(n2)

 For Loop:
 It is to be noted that P(i) is computed for each iteration
from i ← 1 to m. If the value of „m‟ is zero, then we are
considering that „m‟ does not generate any error instead
the loop is terminated without doing anything.
E.g.: for i ← 1 to m
{
P(i)
}
 If P(i) takes any constant time „t‟, for its computation
then for „m‟ iterations, the total time for the loop is
simply „mt‟. Here, we are not considering the loop
control. As we know that for loop can be expressed as:

while(i ≤ m)
{
P(i)
i ← i + 1
}
 The test condition, the assignment instruction and
sequencing operation (goto: implicit in while loop) are
considered at unit cost for simplicity. Suppose if all these
operations are bounded by „c‟ then the computation time
for the loop is bundled above by:
T(n) ≤ c : for i ← 1
(m + 1)c: test condition i ≤ m.
mt: for execution of P(i).
mc: for execution of i ← i + 1.
mc: for the sequencing operation.

 T ≤ (t + 3c)m + 2c. If „c‟ is very small relative to „t‟, then
the computational time for the loop is bounded above by
T(n) ≤ mt.
 Now, if the computation time „ti‟ for P(i) varies as a
function of „i‟, then total computation time for the loop,
(after neglecting loop control) is given not by
multiplication, but by a sum.
for i ← 1 to m
{
P(i)
}
=> T(n) = 𝑡𝑖
𝑚
𝑖=1

E.g.: for i ← 1 to m
{
sum ← sum + t[i]
}
Total time = 𝑡𝑖
𝑚
𝑖=1 = θ(1)𝑚
𝑖=1 = θ( 1𝑚
𝑖=1 ) = θ(m)
 If the algorithm consists of nested for loop, then the total
time is:
for i ← 1 to m {
for j ← 1 to m {
P(i j)
}
} m
∑
i = 1
m
∑
j = 1
tij=> T(n) =

 While Loop:
 The while loops are difficult to analyze in comparison to
for loops, as in these there is no obvious method which
determines how many times we shall have to repeat the
loops.
 The simple technique for analyzing the loops is to firstly
determine functions of variables involve whose value
decreases each time. Secondly for determining the loop it
is necessary that this value must be a positive integer.
 By keeping the track of how many times the values of
function decreases, one can obtain the no. of repetition of
the loop. The other approach for analyzing while loops is
to treat them. E.g.:
while(m > 0)
{ m ← m – 1}
=> Time T(n) = θ(m)

 The rules are:
1. Sequencing: Add the time of the individual
statements. The maximum is the one that count.
2. Alternative Structures: Time for testing the
condition + the maximum time taken by any of the
alternative paths.
3. Loops: Execution time of a loop is at most the
execution time of the statements of the body
(including the condition tests).
4. Nested Loops: Analyze them as inside out.
5. Sub-programs: Analyze them as separate algorithms
and substitute the time whenever necessary.
6. Recursive Sub-programs: Generally the running
time can be expressed as a recurrence relation, with
solution growth rate of execution time.

Asymptotic Notation:
 The notation, which we use to describe the asymptotic
running time of an algorithm are defined in terms of
functions, whose domains are the set of natural numbers
and real numbers.
 The natural number set is denoted as: N = {0, 1, 2, …}
 The positive integer set is denoted as: N+ = {1, 2, 3, …}
 Real number set is denoted as R.
 Positive real number set is denoted as R+.
 Non-negative real number set is denoted as R*.
 Such notations are convenient for describing the worst
case running time function T(n), which is usually defined
only on integer input sizes.

 The different types of notations are:
 Big oh (O) notation
 Small oh (o) notation
 Theta (θ) notation
 Omega (Ω) notation
 Small omega (ω) notation
1. Big Oh (O) Notation:
 The upper bound for the function is provided by Big Oh
(O) notation. We can say, the running time of an
algorithm is O(g(n)), if whenever input size is equal to
or exceeds, some threshold „n0‟, its running time can be
bounded by some positive constant „c‟ time g(n).

 Let f(n) and g(n) are two functions from set of natural
numbers to set of non-negative real numbers and f(n) is
said to be O(g(n)).
 That is: f(n) = O(g(n)), iff there exist a natural number
„n0‟ and a positive constant c > 0, such that f(n) ≤
c(g(n)), for all n ≥ n0.
n0 Input size
Running
time
c(g(n))
f(n)

Examples:
1. f(n) = 2n2 + 7n – 10, n = 5, c = 3.
=> f(n) = O(g(n)), where g(n) = n2
f(n) ≤ c(g(n)) => 2n2 + 7n – 10 ≤ 3 x n2
=> 2 x 25 + 7 x 5 – 10 ≤ 3 x 25
=> 50 + 35 – 10 ≤ 75 => 75 ≤ 75.
So, it is in O(g(n)) = O(n2).
2. f(n) = 2n2 + 7n – 10, n = 4, c = 3, g(n) = n2
=> f(n) ≤ c(g(n)) => 2n2 + 7n – 10 ≤ 3 x n2
=> 2 x 16 + 7 x 4 – 10 ≤ 3 x 16
=> 32 + 28 – 10 ≤ 48 => 50 ≤ 48.
So, it is not in O(g(n)).

3. f(n) = 2n2 + 7n – 10, n = 6, c = 3, g(n) = n2
=> f(n) ≤ c(g(n)) => 2n2 + 7n – 10 ≤ 3 x n2
=> 2 x 36 + 7 x 6 – 10 ≤ 3 x 36
=> 72 + 42 – 10 ≤ 108 => 104 ≤ 108.
So, it is in O(g(n)) = O(n2).
2. Small Oh (o) Notation:
 The functions in small oh (o) notation are the smaller
function in Big oh (O) notation. We use small oh (o)
notation to denote an upper bound that is not
asymptotically tight.
 This notation defined as: f(n) = o(g(n)), iff there exist
any positive constant c > 0 and n0 > 0, such that f(n) <
c(g(n)), for all n > n0.
 The definition of O-notation and o-notation are similar.

 The main difference is that in f(n) = O(g(n)), the bound
f(n) ≤ c(g(n)) holds for some constant c > 0, but in f(n) =
o(g(n)), the bound f(n) < c(g(n)) hold for all constant
c > 0.
 In this notation, the function f(n) becomes insignificant
relative to g(n) as „n‟ approaches infinity. That is:
lim
𝑛→∞
𝑓 𝑛
𝑔 𝑛
= 0 .
3. Big omega (Ω) Notation:
 The lower bound for the function is provided by Big
Omega (Ω) Notation.
 We can say, the running time of an algorithm Ω(g(n)), if
whenever input size is equal to or exceeds some
thresholds value „n0‟, its running time can be denoted by
some positive constant „c‟ times g(n).

 Let f(n) and g(n) are 2 functions from set of natural
numbers to set of non-negative real numbers and f(n)
said to be Ω(g(n)).
 That is, f(n) = Ω(g(n)) iff there exist a natural number
„n0‟ and a constant c > 0, such that f(n) ≥ c(g(n)), for all
n ≥ n0.
n0 Input size
Running
time c(g(n))
f(n)

Example:
1. f(n) = n2 + 3n + 4, n = 1, c = 1.
=> f(n) = Ω(g(n)), where g(n) = n2
f(n) ≥ c(g(n)) => n2 + 3n + 4 ≥ cn2
=> 1 + 3 x 1 + 4 ≥ 1 x 1 => 8 ≥ 1
=> f(n) = Ω(g(n)) = Ω(n2). (proved)
4. Small omega (ω) Notation:
 For a given function g(n), we denoted it as ω(g(n)),
where „ω‟ notation are the larger functions of Big
omega (Ω) notation.
 We use a notation to denote a lower bound that is not
asymptotically tight.
 We define this notation as: f(n) = ω(g(n)), there exist
some positive constant c > 0 and n0 > 0, such that f(n)
> c(g(n)), for all n ≥ n0.
 The relation f(n) = ω(g(n)) implies that lim
𝑛→∞
𝑓 𝑛
𝑔 𝑛
= ∞ .

5. Theta (θ) Notation:
 For a given function g(n), we denoted it as θ(g(n)), the
set of functions f(n) = θ(g(n)), if there exist some
constant c1, c2 and n0, such that:
c1(g(n)) ≤ f(n) ≤ c2(g(n)), for all n ≥ n0.
 For all values of „n‟ to the right of „n0‟, the values of f(n)
lies at or above c1(g(n)) and at or below c2(g(n)).
 In other words, for all n ≥ n0, the function f(n) is equal to
g(n) within a constant factor.
 We say that g(n) is an asymptotically tight bound for f(n).

 The definition of θ(g(n)) requires that every member
f(n) є θ(g(n)) be asymptotically, non-negative. That is:
f(n) be non-negative whenever „n‟ is sufficiently large.
 So f(n) = θ(g(n)), which implies that:
lim
𝑛→∞
𝑓 𝑛
𝑔 𝑛
= 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡 𝑐 .
n0 Input size
Running
time c1(g(n))
f(n)
c2(g(n))

Asymptotic Notation
Properties:
1. Reflexivity:
i. f(n) = θ(f(n))
ii. f(n) = O(f(n))
iii. f(n) = Ω(f(n))
2. Symmetric:
i. f(n) = θ(g(n)) iff g(n) = θ(f(n))
3. Reflexivity:
i. f(n) = O(g(n)) iff g(n) = Ω(f(n))
ii. f(n) = o(g(n)) iff g(n) = ω(f(n))

4. Transitivity:
i. f(n) = θ(g(n)) and g(n) = θ(h(n)) => f(n) = θ(h(n))
ii. f(n) = O(g(n)) and g(n) = O(h(n)) => f(n) = O(h(n))
iii. f(n) = Ω(f(n)) and g(n) = Ω(h(n)) => f(n) = Ω(h(n))
iv. f(n) = o(g(n)) and g(n) = o(h(n)) => f(n) = o(h(n))
v. f(n) = ω(f(n)) and g(n) = ω(h(n)) => f(n) = ω(h(n))
5. Some Important Formula:
i. For any two functions f(n) and g(n), we have:
f(n) = θ(g(n) iff f(n) = O(g(n)) and f(n) = Ω(g(n))
ii. If f(n) = O(g(n)) and f(n) = Ω(g(n)) => f(n) = θ(g(n))
iii. If T1(n) = O(f(n)) and T2 = O(g(n)), then
a. T1(n) + T2(n) = O[max (f(n), g(n))]
b. T1(n) * T2(n) = o[f(n) * g(n)]
iv. f(n) and g(n) are two asymptotic non-negative
functions, then max(f(n), g(n)) = θ(f(n) + g(n)).

6. Some Important Formula:
i. lgn = log2n (binary)
ii. lnn = logen (natural)
iii. lgkn = (lgn)k (exponential)
iv. lglgn = lg(lgn) (composition)
v. a = blogba
vi. logc(ab) = logca + logcb
vii. logban = nlogba
viii.logca/b = logca/logcb
ix. logc(1/a) = -logba
x. logba = 1/logab
xi. alogbc = clogba
xii. logn < nlogn < n2 < 2n < n! < nn

7. Factorial Functions:
i. n! = {1, if n = 0 and n(n – 1)!, if n > 0.
So, n! = 1 * 2 * 3 * …. * n.
ii. n! = √2πn(n/e)n(1 + θ(1/n)) [Stirling‟s
Approximation]
iii. n! = O(nn)
iv. n! = ω(2n)
v. lg(n!) = θ(nlgn)
vi. lg*n = min{i > 0; lg(i) n < 1}
a. lg*2 = 1
b. lg*4 = 2
c. lg*16 = 3
d. lg*65536 = 4
e. lg*(265536) = 5

Problems:
1. Show that for any real constant ‘a’ and ‘b’,
where b > 0.
Ans: (n + a)b = θ(nb).
Here f(n) = (n + a)b and g(n) = nb.
We know that, f(n) = θ(g(n)), if lim
𝑛→∞
𝑓 𝑛
𝑔 𝑛
= 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡.
=> (n + a)b = θ(nb), if lim
𝑛→∞
(n + a)b
nb = 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡.
=> lim
𝑛→∞
*n(1 + a/n)}b
nb => lim
𝑛→∞
*nb(1 + a/n)}b
nb
=> lim
𝑛→∞
(1 + a/n)b => 1b = 1 = constant.
=> (n + a)b = θ(nb) (proved)

2. Prove that 2n + 1 = O(2n)
Ans: Here f(n) = 2n + 1 and g(n) = 2n.
lim
𝑛→∞
𝑓 𝑛
𝑔 𝑛
= lim
𝑛→∞
2n + 1
2n = lim
𝑛→∞
2n .2
2n = 2 = constant
=> f(n) = θ(g(n)).
We know that, f(n) = θ(g(n)) there exist f(n) < c(g(n)), for all
n ≥ n0. According to O-notation, f(n) = O(g(n)) iff there exist
positive constant „c‟ and „n0‟ and either f(n) ≤ c(g(n)) and
for all n ≥ n0.
=> 2n + 1 = O(2n) (proved)
3. Prove that 22n = ω(2n)
Ans: Here f(n) = 22n and g(n) = 2n.
lim
𝑛→∞
𝑓 𝑛
𝑔 𝑛
= lim
𝑛→∞
22n
2n = lim
𝑛→∞
22∗∞
2∞ = ∞ => f(n) = ω(g(n))
=> 22n = ω(2n), which is a ω-notation. (Proved)

4. Show that 5n2 = o(n3)
Ans: Here f(n) = 5n2 and g(n) = n3.
lim
𝑛→∞
𝑓 𝑛
𝑔 𝑛
= lim
𝑛→∞
5n2
n3 = lim
𝑛→∞
5
n
= 0. So, it is in small oh
notation. So => 5n2 = o(n3) (proved).
5. Show that 2n = o(n2)
Ans: Here f(n) = 2n and g(n) = n2.
lim
𝑛→∞
𝑓 𝑛
𝑔 𝑛
= lim
𝑛→∞
2n
n2 = 2/∞ = 0. So, f(n) = o(g(n)) => 2n =
o(n2) is proved. So, it is small oh notation.
6. Show that n2/2 = ω(n)
Ans: Here f(n) = n2/2 and g(n) = n.
lim
𝑛→∞
𝑓 𝑛
𝑔 𝑛
= lim
𝑛→∞
n2/2
n
= lim
𝑛→∞
𝑛2
2
∗
1
𝑛
= lim
𝑛→∞
𝑛
2
= ∞
So, f(n) = ω(g(n)) (proved).

7. Theorem:
Let f(n) = a0 + a1n + a2n2 + … + annm, then
prove that f(n) = θ(nm).
Ans: Here f(n) = a0 + a1n + a2n2 + … + annm and g(n) = nm.
According to θ-notation, lim
𝑛→∞
𝑓 𝑛
𝑔 𝑛
= 𝑐
=> f(n) = θ(g(n))
=> lim
𝑛→∞
a0 + a1n + a2n2 + … +annm
nm
=> lim
𝑛→∞
nm(a0/nm + a1n/nm+ a2n2/nm+ … + an)
nm
=> an = constant c => f(n) = θ(nm) (proved)

8. Let f(n) = 7n3 + 5n2 + 4n + 2. Prove f(n) = θ(n3).
Ans: Here f(n) = 7n3 + 5n2 + 4n + 2 and g(n) = n3.
According to θ-notation, lim
𝑛→∞
𝑓 𝑛
𝑔 𝑛
= 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡
=> f(n) = θ(g(n)) => lim
𝑛→∞
7n3 + 5n2 + 4n + 2
n3
=> lim
𝑛→∞
(7 + 5/n + 4/n2 + 2/n3) = 7 = co𝑛𝑠𝑡𝑎𝑛𝑡
=> f(n) = θ(n3) (proved)
9. Prove lg(n!) = θ(nlgn).
Ans: nn ≥ n!
=> lgnn ≥ lgn! => nlgn ≥ lgn! => lgn! ≤ nlgn
=> lgn! ≤ 1 – nlgn => lgn! = O(nlgn) --- (i)
Where c1 =1. Now to show that lgn! = Ω(nlgn). That is there
exist some constant „c‟ and „n0‟, such that 0 ≤ cnlgn ≤ lgn!

=> lgncn ≤ lgn! => ncn ≤ n!.
Taking c = 1/3, we get nn/3 ≤ n!, which is always true.
=> lgn! = Ω(nlgn) --- (ii)
From equation (i) and (ii), lgn! = θ(nlgn) (proved).
10. Prove n! = O(nn).
Ans: We have to show that, lim
𝑛→∞
n!
nn = 0
n! = √2πn(n/e)n(1 + θ(1/n)) (According to Stirling‟s
Approximation)
=> lim
𝑛→∞
√2πn(n/e)n(1 + θ(1/n))
en ,
Let g(n) = 1/n, f(n) = θ(g(n))
=> f(n) ≤ c1(g(n)) = c1.1/n => f(n) = θ(1/n) ≤ c1/n
But, θ(1/n) < c1.1/n.

=> lim
𝑛→∞
n!
nn ≤ lim
𝑛→∞
√2πn(1 + c1/n)
en
=> lim
𝑛→∞
√2π(√n/en + c1/√n.en)
en
=> √2π lim
𝑛→∞
(√n/en + lim
𝑛→∞
c1/√n.en)
=> √2π lim
𝑛→∞
(
1
2n1/2 − 1
+ 0)/nen – 1 (since lim
𝑛→∞
(
1
√n. en
= 0)
=> √2π lim
𝑛→∞
(
1
2n1/2 − 1
+ 0)
=> √2π/2 lim
𝑛→∞
1
n3/2en - 1 = √π/2 * 0 = 0.
Here, lim
𝑛→∞
n!
nn = 0 => n! = o(nn) proved.

Recurrence:
 A recurrence is an equation or inequality that describes a
function in terms of its value on small inputs.
 The running time of the recursive algorithm can be
obtained by a recurrence.
 To solve the recurrence relation means to obtain a function
defined on the natural numbers that satisfies the
recurrence.

Recurrence Relation:
 A recurrence relation (RR) is defined as for a sequence {an}
is an equation that expresses „an‟ in terms of one or more
previous elements a0, a1.. an-1 of sequence for all n ≥ n0
without any base cases.
 E.g.: i) for instance, consider a recurrence relation tn = 2tn –
1. if „c‟ is a constant, then any function of the form c2n is a
solution to the above recurrence relation.
 By considering, mathematical induction, we have induction
part, if tn – 1= c2n – 1.
 If we have the initial condition t0 = 5, then the only choice
for the constant „c‟ is to have value 5, so as to give the
correct initial value. Thus on the basis part of the proof, we
have tn = 5.2n.

 It does not matter in which order the basis and induction are
established, what matters is that both have been verified to
be correct. Hence the solution for the recurrence is tn = 5.2n.
 E.g.: ii) the worst-case running time T(n) of the MERGE-
SORT procedure could be described by the recurrence.
T(n) = θ(1), if n = 1
2T(n/2) + θ(n), if n > 1.
Whose solution was claimed to be T(n) = θ(nlgn)

Recurrence Equation
Method:
 We solve the recurrence equation by the following
methods. That is:
i. Substitution method
ii. Iterative method
iii. Master method
iv. Recurrence or Recursion Tree
1. Substitution Method:
 In this method, first we guess a solution and use
mathematical induction to find the constant and show
that the solution works.

 The substitution method can be used to establish either
upper or lower bounds on a recurrence.
 E.g.: T(n) = 2T(└n/2┘) + n --- (1) is a recurrence
relation. We guess the solution T(n) = O(nlgn), where
g(n) = nlgn is the solution. That is: f(n) = T(n) ≤ cnlgn ---
(2). Then we have to prove that, this solution is true, by
using mathematical induction.
 From equation-1, T(n) = 2T(└n/2┘) + n
=> T(n) ≤ 2c(└n/2┘) lg(└n/2┘) + n
=> T(n) ≤ 2c. n/2 lgn/2 + n = cn(lgn – lg2) + n
=> cnlgn – cn + n. So, T(n) ≤ cnlgn – n(c – 1)
=> T(n) ≤ cnlgn for c >1
 Now, using mathematical induction,
T(n) = T(n-1) + 1.

 For n = 1, if T(1) = 1, then 1 ≤ c => lg1 = 0
=> T(1) is not true for n = 1. But let T(1) is true.
 For n = 2, if T(2) = 2T(1) + 2 = 4, then
4 ≤ c . 2lg2 => c ≥ 2.
Hence, T(n) ≤ cnlgn, c > 1 is true for n = 2
=> T(2) is true.
 Similarly, T(3), T(4) … is true => T(k) is true.
=> T(k + 1) is true by using Tk.
=> T(n ) = T(n – 1) + 1, g(n) = n = O(n)
 T(n) = T(n – 1) + 1
= T(n – 2) + 1 + 1
= T(n + 3) + 1 + 1 + 1
 Let k = T(n – k) + k. Let n – k = 0 (base value).
=> n = k => T0 + n = n + 1

 Let T0 = 1 (base case)
T(1) = 1, n – k = 1 => k = n – 1 => T(n) = 1 + n – 1 = n
 So, it is true for └n/2┘and it is true for n. If it is true for
n, then it is true for 3. If it is true for 3, then it is true for 6
and 7 and if it is true for 6, then it is true for 12 and 13.
 Hence, we conclude that for n ≥ c, where c ≥ 2. So, T(n) =
O(nlgn) is a solution of T(n) = 2T(└n/2┘) + n.
 Substitution Method is of two types. That is :
i. Backward Substitution
ii. Forward Substitution

i. Backward Substitution Method:
 Question: T(n + 1) = 2T(n). Solve this recurrence
relation using substitution method, using backward
substitution.
 Ans: T(n + 1) = 2T(n), let base value T(0) = 1
=> T(n + 1) = 2(2T(n – 1)) = 22T(n – 1) (1st term)
= 22(2T(n – 3)) = 23T(n – 2) (2nd term)
For kth term = 2k + 1T(n – k)
 Let n – k = 0 => k = n.
So for kth term, it is 2n + 1 T(0) = 2n + 1.1 = 2n + 1
=> T(n + 1) = 2n + 1 => T(n) = 2n (Ans)

 Now to prove this backward substitution using
mathematical induction is: T(n) = 2n
Let for n = 1 is true => T(1) = 21 = 2
Let for n = n is true => T(n) = 2n
We have to prove, T(n + 1) = 2n + 1
=> T(n + 1) = 2.T(n) = 2.2n = 2n + 1
=> T(n + 1) = 2n + 1 (proved)
ii. Forward Substitution Method:
 Question: T(n + 1) = 2T(n). Solve this recurrence
relation using forward substitution method.
 Ans: T(n + 1) = 2T(n),
=> T(1) = 2T(0) = 2 (n = 0 and T(0) = 1)
=> T(2) = 2T(1) = 2 x 2 = 22 ( n = 1 and T(1) = 2)
=> T(3) = 2T(2) = 23

 For k, T(k) = 2k (n = k and T(k – 1) = 2k – 1
Putting n = k => T(n) = 2n => T(n + 1) = 2n + 1
 To prove this forward substitution method, we have to
solve by the mathematical induction process.
 That is: T(n) = 2n.
Let n = 1 is true => T(1) = 21 = 2
Let n = n is true => T(n) = 2n
 We have to prove, T(n + 1) = 2n + 1
=> T(n + 1) = 2.T(n) = 2. 2n = 2n + 1 [T(n) = 2n]
=> T(n +1) = 2n + 1 (proved)

Problems on Substitution
Method:
1. T(n) = 2T(n – 1) + 1, initial value/base value
T(0) = 1 using forward substitution.
 Ans: T(n) = 2T(n – 1) + 1, where T(0) = 1
=> T(1) = 2T(0) + 1 = 2 x 1 + 1 = 3 = 22 - 1
T(2) = 2T(1) + 1 = 2 x 3 + 1 = 7 = 23 - 1
T(3) = 2T(2) + 1 = 2 x 7 + 1 = 15 = 24 - 1
…
T(k) = 2k + 1 - 1
=> T(n) = 2n + 1 - 1
=> T(n + 1) = 2n + 2 – 1 = 2T(n) + 1

 Now, by using mathematical induction,
T(n) = 2n + 1 – 1. Let n = 1 => T(1) = 21 + 1 – 1.
 Similarly, let „n‟ is true => T(n) = 2n + 1 – 1
We have to prove, T(n + 1) = 2T(n) + 1 = 2(2n + 1 – 1) + 1
=> 2. 2n + 1 – 2 + 1 = 2n + 2 – 1
=> T(n + 1) = 2n + 2 – 1 (proved)
2. Consider the recurrence T(n) = 3T(n/2) + n,
n ≥ 1 with initial condition T(0) = 0, obtain the
solution for the above recurrence.
 Ans: T(n) = 3T(n/2) + n, n ≥ 1and T(0) = 0
Let for n = 1, T(1) = 3T(1/2) + 1 = 1
n = 2, T(2) = 3T(1) + 2 = 3 x 1 + 2 = 5
n = 22 = 4, T(4) = 3T(2) + 3 = 3(3 x 1 + 2) + 22
= 32 x 1 + 3 x 2 + 22 = 9 + 6 + 4 = 19

n = 23 = 8, T(8) = 65 = 33 x 1 + 32 x 2 + 3 x 22 + 23
n = 24 = 16, T(8) = 211 = 34 x 1 + 33 x 2 + 32 x 22 + 3 x 23 + 24
(We take the values represented in the powers of 2).
 From the above computation, we guess the solution is:
T(n) = 3n + 1 – 2n + 1
So, for n = 0, T(0) = 30 – 20 = 0, is satisfy the initial
condition. So the statement is correct6 for n = 1. Let us
assume for some k > 0 then:
T(2k) = 3k20 + 3k – 121 + 3k - 222 + … + 3 x 2k - 1 + 30 x 2k
= 3k (2/3)𝑖𝑘
𝑖=𝑜 = 3k + 1 – 2k + 1
 So it can be observed that, T(k) = 3k + 1 – 2k + 1 (if „n‟ is not
power of 2). By this, we guess as the solution is correct.
Thus, T(n) = 3n + 1 – 2n + 1

3. Consider the recurrence T(n) = 1, n = 1 and
2T(└n/2┘), n > 1. We have to find an asymptotic
bound on T(n).
 T(n) = 1, n = 1 and 2T(└n/2┘), n > 1
 For the above recurrence, we guess that it satisfies
O(nlgn). Thus, for this we have to show that there exists a
constant „c‟, such that T(m < n) ≤ cmlogm, which implies,
T(n) ≤ cnlogn
=> T(n) ≤ c2└n/2┘log└n/2┘+ n
=> T(n) ≤ c2logn – cnlog2 + n
=> T(n) ≤ cnlogn – (clog2 – 1)n
=> T(n) ≤ cnlogn, for all c > 1/log2
 By mathematical induction, we can check T(2) = 4 and
T(3) = 5.

 As for n = 1, T(n) ≤ cnlogn yields 0. Thus, the inductive
proof T(n) ≤ cnlogn for constant c ≥ 1 is completed by
choosing „c‟ large enough such that: T(2) ≤ c2log2 and
T(2) ≤ c3log3. Since, the above relation holds for c ≥ 2.
Thus, T(n) ≤ cnlogn holds true.
 Thus, our guess as the solution T(n) = O(nlogn) is correct.
4. Consider the recurrence T(n) = 2T(└n/2┘+ 16)
+ n. We have to show that it is asymptotically
bound on O(nlogn).
 T(n) = 2T(└n/2┘+ 16) + n. For, T(n) = O(nlogn), we have
to show that for some constant,
T(n) ≤ cnlogn
=> T(n) ≤ c2(└n/2┘+ 16) log(└n/2┘+ 16) + n
= cnlog(n/2) + 32 + n – cnlog2 + n
= cnlogn – cn + 32 + n

= cnlogn – (c – 1)n + 32
= cnlogn – b ≤ cnlogn (if c ≥ 1, b is constant)
Thus, T(n) = O(nlogn)
5. Consider the recurrence T(n) = 2T(└n/2┘) + n,
we have to show that it is asymptotically bound
on O(nlogn).
 T(n) = 2T(└n/2┘) + n. For, T(n) = Ω(nlogn), we have to
show that for some constant „c‟,
T(n) ≥ cnlogn
=> T(n) ≤ c2(└n/2┘)log(└n/2┘) + n
=> T(n) = cnlog(n/2) + n – cnlog2 + n
= cnlogn – cnlog2 + n
= cnlogn – cn + n
=> T(n) = cnlogn for c = 1. Thus T(n) = Ω(nlogn)

6. Consider the recurrence T(n) = 2T(└n/2┘) + 1,
we have to show that it is asymptotically bound
on O(logn).
 T(n) = 2T(└n/2┘) + 1. For, T(n) = O(logn), we have to
show that for some constant „c‟,
T(n) ≤ cnlogn
=> T(n) ≤ clog(└n/2┘) + 1 = clogn – clog2 + 1
=> T(n) ≤ clogn for c ≥ 1. Thus T(n) = O(logn)
2. Iterative Method:
 An iterative method is the method, where the recurrence
relation is solved by considering 3 steps. That is:
I. Step 1: expand the recurrence
II. Step 2: express is as a summation (∑) of terms,
dependent only on „n‟ and the initial condition.
III. Step 3: evaluate the summation (∑).

Problems on Iteration
Method:
1. T(n) = 0, n = 0 – (i) (initial condition) and
= c + T(n – 1), n > 0 – (ii)
=> T(n) = T(n – 1) + c
= T(n – 2) + c + c => T(n – 2) + 2c
= T(n – 3) + c + c + c => T(n – 3) + 3c
= T(n – 4) + c + c + c + c => T(n – 4) + 4c
For kth term, T(n – k) + c + c + c … k times
= T(n – k) + 𝑐𝑘
𝑖=1
From the base case, n – k = 0 => n = k
=> T(n) = T(0) + 𝑐𝑛
𝑖=1 = 0 + cn => T(n) = cn.

2. T(n) = 0, n = 0 – (i) (initial condition) and
= T(n – 1) + n, n > 0 – (ii)
=> T(n) = T(n – 1) + n
T(n – 1) = T(n – 2) + n – 1
=> T(n) = T(n – 2) + n – 1 + n
T(n – 2) = T(n – 3) + n – 2
=> T(n) = T(n – 3) + n – 2 + n – 1 + n
….
For kth term, T(n – k) + n – k – 1 + ….. + n – 1 + n
=> T(n) = T(n – k) + 𝑛 − 𝑖𝑘+1
𝑖=0
From the base case, n – k = 0 => n = k
=> T(n) = T(0) + 𝑛 − 𝑖𝑛+1
𝑖=0 = 0 + 𝑛 − 𝑖𝑛+1
𝑖=0
=> T(n) = 𝑛 − 𝑖𝑛+1
𝑖=0

We know that, 𝑖𝑛
𝑖=1 = n (n + 1) /2
T(n) = 𝑛 − 𝑖𝑛+1
𝑖=0 = n + n – 1 + n – 2 + … + 1 + 0 + (-1)
=> T(n) = n(n + 1)/2 – 1
3. T(n) = c, n = 1 – (i) (initial condition) and
= 2T(n/2) + c, n > 1 – (ii)
=> T(n) = 2T(n/2) + c
= 2T(2T(n/22) + c) + c = 22T(n/22) + 2c + c
= 22 (2T(n/23) + c) + 2c + c
= 23T(n/23) + 22c + 21c + 20c
….
For kth term, 2kT(n/2k) + 2k - 1c + … + 21c + 20c
=> n/2k = 1 => n = 2k => k = log2n

That is,
2logn + 2logn – 1/c + …. + 21c + 20c
= 2log2n = nlog22 = n1 = n (since, alogbc = clogba)
= 2kT(n/2k) + c 2𝑖 − 1𝑘
𝑖=1
(since, a + ar + ar2 + … + arn = a(rn+ 1 – 1)/r – 1)
= 2kT(n/2k) + c(2k – 1/2 – 1)
= 2kT(n/2k) + c(2k – 1)
Let k = logn,
= 2lognT(n/2logn) +c2logn – c
= nT(1) + cn – c
= 2cn – c
= c(2n – 1)

4. T(n) = 2T(└n/3┘) + n
Expanding the above terms, we get
T(n) = n + 2/3n + 4T(n/9)
= n + 2/3n + 4/9n + 8T(n/27)
It is to be noticed that, we can meet the boundary
condition, where (n/3i) ≤ 1. i.e.: after performing ┌log3n┐
expansions. Thus,
T(n) = n 2/3 𝑖 𝑛 + 2𝑙𝑜𝑔3 𝑛θ(1)
┌log3n┐
𝑖=0
≤ n 2/3 𝑖 𝑛 + 2𝑙𝑜𝑔3 𝑛θ(1)
log3n
𝑖=0
≤ n 2/3 𝑖 𝑛 + 𝑛𝑙𝑜𝑔32θ(1)∞
𝑖=0
= 3n + O(n) = O(n).
So, T(n) = O(n).

5. T(n) = T(n – 1) + 1, T(1) = θ(1
Expanding the above terms T(n – 1) = T(n – 2) + 1
So, T(n) = (T(n – 2) + 1) + 1 = T(n – 2) + 1
T(n – 2) = T(n – 3) + 1
=> T(n) = (T(n – 3) + 1) + 2 = T(n – 3) + 3
For kth term, T(n) = T(n – k) + k
When k = n – 1 => T(n – k) = T(1) = θ(1)
Thus, T(n) = θ(1) + (n – 1) = θ(n).
Hence, T(n) = θ(n)
6. T(n) = T(n/2) + n, T(1) = θ(1)
Expanding the above terms T(n/2) = T(n/4) + n/2
Thus, T(n) = T(n/4) + n/2 + n
T(n/4) = T(n/8) + n/4
=> T(n) = T(n/8) + n/4 + n/2 + n

T(n/8) = T(n/16) + n/8
=> T(n) = T(n/16) + n/8 + n/4 + n/2 + n
T(n/16) = T(n/32) + n/16
=> T(n) = T(n/32) + n/16 + n/8 + n/4 + n/2 + n
…..
For kth term, T(n) = T(n/2k) + (
𝑛
2 𝑗)𝑘 −1
𝑗=0
It can be observed that the recursion stops, when we get to
T(1). This happens when n/2k = 1. That is n = 2k => k = logn
Thus, T(n) = θ(1) + (
𝑛
2 𝑗)
𝑙𝑜𝑔𝑛 −1
𝑗=0
< θ(1) + (
𝑛
2 𝑗)∞
𝑗=0
< θ(1) + 2n = θ(n)
Hence, T(n) = θ(n)

7. T(n) = 3T(└n/4┘) + n
Expanding the above terms, we get
T(n) = 3(3T(└n/16┘)) + (└n/4┘) + n
= 3(3(3T(└n/164┘))) + (└n/16┘) + (└n/4┘) + n
= n + 3(└n/4┘) + 9(└n/16┘) + 2T(T(└n/164┘))
The recursion stops, when n/4i ≤ 1, which implies n ≤ 4i
=> log4n = i
Thus, T(n) = n + 3└n/4┘+ … + 3i└n/4i┘+ 3log4n.θ(1)
T(n) ≤ (n + 3n/4 + 9n/16 + … + 3log4n)θ(1)
≤ n (3/4) 𝑘∞
𝑘=0 + θ(nlog43)
{as 3log4n = nlog43}
≤ n(1/(1 – ¾)) + O(n)
= 4n + O(n) {as log43 < 1} = O(n)
Hence, T(n) = O(n) (proved)

3. Master Method:
 The master method is used for solving the following
types of recurrences, T(n) = aT(n/b) + f(n), where „a‟
and „b‟ are constants and a ≥ 1, b > 1.
 In the above recurrence, the problem of size „n‟ is
divided into „a‟ sub-problems each of size „n/b‟.
 Each sub-problem of size „n/b‟ can be solved recursively
in time T(n/b).
 The cost of dividing or splitting the problem and
combine the solutions or result is described by the
function f(n).
 Here the size is interpreted as └n/b┘or ┌n/b┐. The T(n)
can be bounded asymptotically by the following 3 cases.

1. CASE I: if f(n) = O(nlog
b
a – є) for some constant є > 0
then T(n) = θ(nlog
b
a)
2. CASE II: if f(n) = θ(nlog
b
a) then T(n) = θ(nlog
b
a . logn)
3. CASE III: if f(n) = Ω(nlog
b
a + є) for some constant є > 0
if af(n/b) ≤ cf(n), for some constant c > 0 and c < 1 and
n = sufficiently very large, then T(n) = θ(f(n))
NOTES:
 If the recurrence is of following form i.e.:
T(n) = aT(n/b) + cnd, n > n0
 Then the solution of the recurrence is,
T(n) = θ(nd), if a < (b)d
θ(ndlogn), if a = (b)d
θ(nlog
b
a), if a > (b)d

 E.g.: T(n) = 3T(n1/3) + log3n
Let us assume m = log3n => n = 3m. Thus, n1/3 = 3n/3
=> T(3m) = 3T(3m/3) + m. Again, consider s(m) = T(3m).
We have s(m) = 3s(m/3) + m.
Using master method,
s(m) є θ(mlogm) => T(n) є θ(log3n(log log3n)) (Ans)
 Let T(n) = 4T(n/2) + nlogn. If f(n) = logarithmic part or
polynomial part, then master method did not work or can‟t
be apply.
 So for the solution we have to apply substitution or
iterative method.

 Here a = 4, b = 2, f(n) = nlogn, nlog
b
a = n2. Here we can‟t
compare n2 with nlogn.
 So we can‟t say that those two terms are equal or greater
than or less than. So we go for iterative or substitution
method.
 T(n) = 2T(3n/2) + 3. Here a = 2, b = 2/3, f(n) = 3. Now
nlog
b
a = nlog
2/3
2.
 It can‟t be solved in master method, because logarithmic
function can‟t take fractional value.
 It always took integer form. So this type of problem is
solved by either iterative or substitution method.

Problems on Master
Method:
1. T(n) = 3T(n/2) + n2. Here a = 3, b = 2, f(n) = n2. Now,
nlog
b
a = nlog
2
3 = n1.585. But f(n) = n2, so it will be in case-3,
where є > 0.
=> є = .415 => f(n) = Ω(nlog
b
a + є) => T(n) = θ(f(n))
=> T(n) = θ(n2) and af(n/b) ≤ cf(n) => 3 * f(n/2) ≤ cn2.
=> 3n2/4 ≤ cn2 = 3/4 ≤ c. (Since c > 0 and c < 1).
So, af(n/b) ≤ cf(n) is satisfied.
nlog
b
a = nlog
2
4 = nlog
222 = n2log
2
2 = n2 x 1 = n2.
(since, log22 = 1). So it satisfied case – 2. i.e.:
f(n) = θ(nlog
b
a) => f(n) = θ(n2) => T(n) = θ(nlog
b
a . logn)
=> T(n) = θ(n2logn) (Ans)

3. T(n) = 2T(n/2) + n. Here a = 2, b = 2, f(n) = n. Now,
nlog
b
a = nlog
2
2 = n = f(n). So it satisfies case – 2. That is:
f(n) = θ(nlog
b
a) = n => T(n) = θ(nlog
b
a . logn) = θ(nlogn).
nlog
b
a = nlog
4
16 = nlog
442 = n2log
4
4 = n2 x 1 = n2. It satisfies
case – 1. That is f(n) = O(nlog
b
a – є)
=> n = O(n2 – є) = O(n2 – 1) => n = O(n)
(since, є = 1 and є > 0)
=> T(n) = θ(nlog
b
a) => T(n) = θ(n2) (Ans)
5. T(n) = 2T(n/2) + n – 1. Here a = 2, b = 2, f(n) = n – 1.
Now, nlog
b
a = nlog
2
2 = n. Since, f(n) = n – 1 that does not
belongs to O(nlog
b
a – є). So case – 1 does not apply. But as
f(n) є θ(n).
So according to case – 2, we have T(n) = O(nlogn) (Ans)

6. T(n) = T(3n/4) + 1 and T(1) = θ(1). We have to find its
asymptotic bound. Using the Master method we have,
a = 1, b = 4/3 and f(n) = 1. Now, nlog
b
a = nlog
4/3
1 = n0 = 1.
So case – 2 applies. Since 1 = θ(1). So T(n) = θ(logn)
nlog
b
a = nlog
2
4 = nlog
222 = n2log
2
2 = n2 . Since, f(n) = n. So it
satisfies case – 1. That is f(n) = O(nlog
b
a – є)
=> n = O(n2 – 1) => n = O(n). (since є > 0 and є = 1).
Thus T(n) = θ(nlog
b
a) => T(n) = θ(n2) (Ans).
nlog
b
a = nlog
2
4 = nlog
222 = n2log
2
2 = n2 . Since, f(n) = n3. So
it satisfies case – 3. That is f(n) = Ω(nlog
b
a + є) = Ω(n2 + 1)
=> n = Ω(n3). Thus, T(n) = θ(n3) and af(n/b) ≤ cf(n).
=> 4f(n/2) ≤ cn3 => 4f(n3/8) ≤ cn3 => 4/8n3 ≤ cn3
=> 1/2n3 = cn3 => c = 1/2. (since c > 0 and c < 1). So,
af(n/b) ≤ cf(n) is satisfied.

4. Recursion Tree Method:
 Recursion Tree Method is pictorial representation of an
iteration method, which is in the form of a tree, where at
each levels, nodes are expanded.
 It is used to keep track of the size of the remaining
arguments in the recurrence and the non-recursive costs.
In a recursion tree, each node represents the cost of a
single sub-problem.
 We add the cost within each level of the tree to obtain a
set of pre-level cost and then we add all the levels of
costs to determine the total cost of all levels of
recursion.
 In general, T(n) = aT(n/b) + f(n)

T(1)
f(n)
f(n/b)f(n/b)
f(n/b2)f(n/b2) f(n/b2)
f(n/b)
f(n/b2)f(n/b2) f(n/b2) f(n/b2)f(n/b2) f(n/b2)
af(n/b)
a2f(n/b2)
f(n/bn)f(n/bn) f(n/bn) anf(n/bn)
T(n) = aT(n/b) + f(n)
T(n/b) = aT(n/b2) + f(n/b)

Theorem:
1. Let a ≥ 1 and b > 1 be constants. Let f(n) be a non-
negative function defined on exact powers of „b‟. Define
T(n), be an exact power of „b‟, by the recurrence.
T(n) = θ(1) , n = 1
aT(n/b) + f(n), if n = bi, i = +ve integer
=> T(n) = θ(nlog
b
a) + 𝑎𝑖 𝑓(
𝑛
𝑏𝑖)
𝑙𝑜𝑔 𝑏
𝑛−1
𝑖=0
Since, alogbn = nlogba
θ(nlog
b
a) = Total cost for the leafs,
=
𝑙𝑜𝑔 𝑏
𝑛−1
𝑖=0 sum over of all levels
aif(n/bi) = cost per level

Problems on Recursion
Tree:
1. T(n) = T(n/3) + T(2n/3) + n. The recursion tree for this is:
=> T(n) = n + n + n + ….. log3/2n times = θ(nlogn).
T(n)/n
n/3 2n/3
n/9 2n/9 2n/9 4n/9
n
n
n
Total = θ(nlogn)
log3/2n

2. T(n) = 2T(n/2) + n2. The recursion tree for this is:
So, the above recurrence has the solution T(n) = θ(n2).
n2
(n/2)2 (n/2)2
(n/4)2 (n/4)2 (n/4)2
(n/4)2
n2/2
n2/4
n2
Total = θ(n2)
log2n

3. T(n) = 4T(n/2) + n. The recursion tree for this is:
We have, n + 2n + 4n + … + logn times
= n(1 + 2 + 4 ….. logn times)
= n(2log22 – 1)/(2 – 1) = n2 – n = θ(n2)
=> T(n) = θ(n2)
n
2(n/2) 2(n/2)
4(n/4) 4(n/4) 4(n/4) 4(n/4)
2n
4n
n
Total = θ(n2)
logn
1 1
logn
1 1 1

4. T(n) = 3T(n/4) + n. The recursion tree for this is:
=> T(n) = θ(n4log3) + (
3
4
)𝑖 𝑛
4𝑙𝑜𝑔𝑛−1
𝑖=0
=> T(n) < θ(n4log3) + (
3
4
)𝑖 𝑛∞
𝑖=0
=> T(n) < θ(n4log3) + (1/(1 – ¾))n
=> T(n) < θ(n4log3) + (1/4n)
=> T(n) є O(n).
Here, we have the linear worst case complexity.
n
n/4n/4
(n/16)(n/16) (n/16)
n/4
(n/16)(n/16) (n/16) (n/16)(n/16) (n/16)
3n/4
9n/16
T(1)T(1) T(1)
n4log3 leaves
n
4logn

5. Solve the factorial with recursion tree with recurrence
relation.
Factorial:
The term „n‟ factorial indicates the product of the
positive integers from 1 to n inclusive and is denoted
by n!. Factorial of number (n) is a recursive manner is
defined by the recurrence relation is:
Factorial(n) = T(n) = 1, n = 0
T(n – 1) * 1, n > 0
The algorithm for the factorial is:
fact(n)
{
if n = 0 then return 1
else if n > 0 return n * fact(n – 1)
}

From the recurrence relation,
T(n) = 1, n = 0
n * T(n – 1) * 1, n > 0
Put, n = 1, 2, …. N
T(1) = T(0) + 1, T(2) = T(1) + 1 …. T(n) = T(n – 1) + 1
Using recurrence tree, the solution is:
=> T(n) = T(n – 1) + 1
+ T(n – 1) = T(n – 2) + 1
+ T(n – 2) = T(n – 3) + 1 ….
+ T(2) = T(1) + 1
+ T(1) = T(0) + 1
---------------------------
T(n) = T(0) + 1 + 1 + 1 + …. + n => T(n) = T(0) + n
=> T(n) = n + 1

If, T(1) = 1 is given, then we have to calculate upto
T(2) = T(1) + 1
=> T(n) = T(1) + 1 + 1 + 1 + … + n – 1
=> T(n) = T(1) + n – 1 = 1 + n – 1 = n => T(n) = n
: Calculate if for „n‟ value n(1) = n
T(n)
T(n – 1)
T(n – 2)
T(n – 3)
T(0)
1
1
1
1

6. Solve the Fibonacci series with recursion tree with
recurrence relation.
Fibonacci Series:
Fibonacci Series is a series of positive integer in
manner that the next term of a series is the addition of
previous 2 terms. i.e.: 0, 1, 1, 2, 3, 5, 8, 13, 21.
The algorithm for Fibonacci series is:
Fibseq(n)
{
if n = 0
then return 0
else if n = 1
then return 1
else if n > 1
then return (Fibseq(n – 1) + Fibseq(n – 2))

The Fibonacci sequence in recursive manner is defined by
the recurrence relation is:
T(n) = 0, if n = 0
1, if n = 1
T(n – 1) * T(n – 2), if n > 1.
From the recurrence relation,
Put, n = 2, 3, …. n.
So, T(2) = T(0) + T(1),
T(3) = T(1) + T(2) ….
T(n) = T(n – 1) + T(n – 2)

T(n)
T(n – 2)
T(n – 2) T(n – 3)
T(n – 1)
T(n – 3) T(n – 4)
T(0) T(1) T(0) T(1) T(0) T(1) T(0) T(1)
=> T(n) = T(n – 1) + T(n – 2)
+ T(n – 1) = T(n – 2) + T(n – 3)
+ T(n – 2) = T(n – 3) + T(n – 4) ….
+ T(3) = T(1) + T(2)
+ T(2) = T(0) + T(1)
---------------------------
T(n) = T(0) + T(n – 1) = T(n – 1)
=> T(n) = T(n – 1)

Binary Search Algorithm:
 Binary_search(a[], n, x)
begin
low ← 1, high ← n, j =0;
while (low ≤ high AND j =0)
begin
mid ← └(low + high)/2┘;
if (x = a[mid])
j ← mid;
else if (x < a[mid])
high ← mid – 1;
else low ← mid + 1;
end while
return j; end

Binary Search Analysis:
 For 1st iteration – n
2nd iteration - └n/2┘
3rd iteration - └└n/2┘/2┘ = └n/4┘ ….
For jth iteration - └n/2j - 1┘= 1
 The floor definition, 1 ≤ n/2j – 1 ≤ 2 => 2j – 1 ≤ n ≤ 2j
=> log2j – 1 ≤ log2n ≤ log2j (Taking logarithmic value)
=> j – 1 ≤ n < j
=> j ≤ logn + 1 => j = └logn┘ + 1
So, the time complexity is: O(logn)

Insertion Sort Algorithm:
 Insertion_sort(a[], n)
begin
for (i = 2 to n)
begin j ← i;
temp ← a[i];
while(j > 1 AND a[j – 1] > temp)
begin
a[j] ← a[j – 1];
j ← j – 1;
end while
a[j] ← temp;
end for
end

Insertion sort Analysis:
 Insertion_sort(a[], n)
Sl.no Code Cost times
1 begin 0 1
2 for (i = 2 to n) C1 n
3 begin 0 n – 1
4 j ← i C2 n – 1
5 temp ← a[i] C3 n – 1
6 while(j > 1 AND a[j – 1] > temp) begin C4 𝑡𝑗
𝑛
j=2
7 a[j] ← a[j – 1] C5 𝑡𝑗
𝑛
𝑗=2 - 1
8 j ← j – 1 C6 𝑡𝑗
𝑛
𝑗=2 - 1
9 end while 0 𝑡𝑗
𝑛
𝑗=2 - 1
10 a[j] ← temp C7 n – 1
11 end for 0 n – 1
12 end 0 1

 Total time = ∑ cost x time
= 0 + C1n + 0 + C2(n – 1) + C3(n – 1) + C4 𝑡𝑗
𝑛
j=2 +
C5 𝑡𝑗 − 1𝑛
j=2 + C6 𝑡𝑗 − 1𝑛
j=2 + 0 + C7(n – 1) +
0 + 0
= C1n + C2(n – 1) + C3(n – 1) + C7(n – 1) +
C4 𝑡𝑗
𝑛
j=2 + C5 𝑡𝑗 − 1𝑛
j=2 + C6 𝑡𝑗 − 1𝑛
j=2
= (C1 + C2 + C3 + C7)n – (C2 + C3 + C7) + C4 𝑡𝑗
𝑛
j=2
+ C5 (𝑡𝑗 − 1)𝑛
j=2 + C6 (𝑡𝑗 − 1)𝑛
j=2
= C8n – C9 + C4 𝑡𝑗
𝑛
j=2 + C5 𝑡𝑗
𝑛
j=2 − C5 1𝑛
j=2 +
C6 𝑡𝑗
𝑛
j=2 − C6 1𝑛
j=2
= C8n – C9 + (C4 + C5 + C6) 𝑡𝑗
𝑛
j=2 − C5 1𝑛
j=2 −
C6 1𝑛
j=2
= C8n – C9 + C10 𝑡𝑗
𝑛
j=2 − C5n + C5 − C6n + C6

= C8n − C5n − C6n – C9 + C5 + C6 + C10 𝑡𝑗
𝑛
j=2
= n(C8 − C5 − C6) + (C5 + C6 – C9)+ C10 𝑡𝑗
𝑛
j=2
= C11 + C12n + C10 𝑡𝑗
𝑛
j=2
= C11 + C12n + C10n(n + 1)/2 − C10
= C11 − C10 + C12n + C10n2/2 + C10n/2
= C13 + C12n + C10n2/2 + C10n/2
= C13 + C12n + C10n/2 + C10n2/2
= C10n2/2 + C14n/2 + C13 ≈ An2 + Bn + C ≈ O(n2)
 So, the best case run time of insertion sort is:
T(n) = C1n + C2(n – 1) + C3(n – 1) + C7(n – 1)
= (C1 + C2 + C3 + C7)n – (C2 + C3 + C7)
 The running time, can be expressed as „an + b‟ for constant
„a‟ and „b‟ that depend on the statement costs „Ci‟, it is thus
a linear function of „n‟.

 The worst case run time of insertion sort:
T(n) = C1n + C2(n – 1) + C3(n – 1) + C4((n(n + 1) – 1/2) +
C5((n(n – 1)/2) + C6((n(n – 1)/6) + C7(n – 1)
= (C4/2 + C5/2 + C6/2)n2 + (C1 + C2 + C3 + C4/2 – C5/2
+ C6/2 + C7)
 The worst case running time can be expressed as:
an2 + bn + c for constants a, b, and c and again depend on
the statement costs „Ci‟, it is thus a quadratic function of
„n‟.

Bubble Sort Algorithm:
 This algorithm sorts the element of an array „A‟, having
elements in ascending or increasing order.
 Step 1: Initialization (p = pass counter, E = count the no. of
exchanges, l = unsorted element)
 Step 2: loop,
Repeat through step 4, while (p ≤ n – 1)
Set E ← 0 : initializing exchange variables
 Step 3: Comparison loop
Repeat for i ← 1, … l – 1
if (A[i] > A[i + 1]) then
set A[i] ← A[i + 1] : Exchanging values
set E ← E + 1

 Step 4: Finish or reduce the size
if (E – 0), then
exit
else
set l ← l - 1
 Here, pass refers to the search for the element with next
smallest key.
 At a time, each pass places one element in its proper
position. Thus, for performing the above sort „n – 1‟ passes
are required.
 In pass 1 the adjacent elements are compared such as A[1]
and A[2] and the elements are arranged in proper order like
A[2], A[1] (if A[2] < A[1]). After that A[1] and A[3] are
compared.

 The process continues until the greatest element is placed at
the last position. Thus A[n] contains the largest element. In
this pass (n – 1) comparisons are required.
 In pass 2, the second largest element is placed at A[n – 1]
by performing (n – 2) comparisons. After (n – 1) passes, we
get the final sorted list as: A1 ≤ A2 ≤ A3 ≤ … An - 1 ≤ An.
 The whole list of „n‟ elements of an array „A‟ is sorted after
(n – 1) passes. So the time complexity of Bubble sort is:
i. For pass 1: n elements
ii. For pass 2: n elements
iii. i.e.: n x n = n2 => O(n2)

Daa notes 1

Daa notes 1

More Related Content

What's hot

Viewers also liked

Similar to Daa notes 1

More from smruti sarangi

Recently uploaded

Daa notes 1