Lectures 1-2.
Introduction - Insertion Sort - Merge Sort
Algorithm
¢ A well-defined computational procedure that
transforms the input to the output
¢ Describes a specific computational procedure for
achieving the desired input/output relationship
¢ An instance of a problem is all the inputs needed
to compute a solution to the problem
¢ A correct algorithm
u halts with the correct output for every input instance
u is said to solve the problem
Algorithm
¢ Example: Sorting
u Input: A sequence of n numbers <a1, a2, ..., an>
u Output: A permutation (reordering) <b1, b2, ..., bn> of the
input sequence such that 𝑏! ≤ 𝑏" ≤ ⋯ ≤ 𝑏#
u Instance: <6, 4, 3, 7, 1, 4>
Algorithm
¢ Insertion Sort
u In-place sorting: Uses only a fixed amount of storage
beyond that needed for the data
¢ Example:
u 64371 4 46 3714
^* ^ *
34671 4 34 6714
* ^ *
13467 4 13 4467
^ *
Algorithm
¢ Example: 6 4 3 7 1 4
6 4 3 7 1 4
Algorithm
¢ Pseudocode:
INSERTION-SORT(A) /* A is an array of numbers */
1 for j ¬ 2 to length[A]
2 do key ¬ A[j]
3 /* insert A[j] into the sorted sequence A[1 .. j-1] */
4 i¬j-1
5 while i > 0 and A[i] > key
6 do A[i+1] ¬ A[i]
7 i¬i-1
8 A[i+1] ¬ key
Algorithm
Algorithm
¢ Example:
u 5 2 4 6 1 3
Analyzing Algorithm
¢ Predicting the resources, such as memory, bandwidth,
logic gates, or running time
¢ Assumed implementation model
u Random-access machine (RAM)
¢ Running time: f(input size)
¢ Input size:
u Sorting: number of items to be sorted.
u Multiplication: number of bits.
u Graphs: numbers of vertices and edges.
Analyzing Algorithm
¢ Running time for a particular input is the number of
primitive operations executed
¢ Assumption: Constant time ci for the execution of the
ith line (of pseudocode)
Note: tj is the number of times the while loop test in line 5 is executed for the value of j.
Analyzing Algorithm
Analyzing Algorithm
¢ Best case
u Array is already sorted, so tj = 1 for j = 2, 3, ..., n.
u T(n) = c1n + c2(n - 1) + c4(n - 1) + c5(n - 1) + c8(n - 1)
= (c1 + c2 + c4 + c5 + c8)n - (c2 + c4 + c5 + c8)
= an + b (linear in n)
¢ Worst case
u Array is sorted in reverse, so tj = j
= an2 + bn + c (quadratic in n).
Analyzing Algorithm
¢ Average Case?
¢ Concentrate on worst-case running time
u Provides the upper bound
u Average case is often as bad as the worst case
¢ Order of Growth
u The order of a running-time function is the fastest growing
term, discarding constant factors
u Insertion sort
ê Best case: an + b ® Q(n)
ê Worst case: an2 + bn + c ® Q(n2)
Designing Algorithms
¢ Incremental design
u Iterative
u Example: Insertion sort
¢ Divide-and-conquer algorithm
u Recursive
u Example: Merge sort
¢ Three steps in the divide-and-conquer paradigm
u Divide the problem into smaller subproblems
u Conquer subproblems by solving them recursively
u Combine solutions of subproblems
Designing Algorithms
¢ Merge Sort
u Divide the n-element sequence into two subsequences of n/2
elements each
u Conquer (sort) the two subsequences recursively using merge
sort
u Combine (merge) the two sorted subsequences to produce
the sorted sequence
¢ Note: Recursion bottoms out when only one element
to be sorted
Divide ...
5 1 4 2 10 3 9 15
5 1 4 2 10 3 9 15
5 1 4 2 10 3 9 15
5 1 4 2 10 3 9 15
And Conquer
1 2 3 4 5 9 10 15
1 2 4 5 3 9 10 15
1 5 2 4 3 10 9 15
5 1 4 2 10 3 9 15
Merge Sort
¢ For MERGE_SORT, an initial array is repeatedly divided
into halves (usually each is a separate array), until
arrays of just one element remain
¢ At each level of recombination, two sorted arrays are
merged into one
¢ This is done by copying the smaller of the two elements
from the sorted arrays into the new array, and then
moving along the arrays
1 13 24 26 2 15 27 38
Aptr Bptr Cptr
Merging
1 13 24 26 2 15 27 38 1
Aptr Bptr Cptr
1 13 24 26 2 15 27 38 1 2
Aptr Bptr Cptr
1 13 24 26 2 15 27 38 1 2 13
Aptr Bptr Cptr
etc.
Merge Sort Algorithm
¢ MERGE_SORT(A, p, r)
1 if p < r
2 then q ¬ ë ( p + r ) / 2û
3 MERGE_SORT(A, p, q)
4 MERGE_SORT(A, q+1, r)
5 MERGE(A, p, q, r)
Merge Sort Algorithm
Merge Sort Algorithm
¢ Note
u The MERGE_SORT(A, p, r) sorts the elements in the subarray
A[p .. r]
u If p >= r, the subarray has at most one element and is therefore
already sorted
u The procedure MERGE(A, p, q, r), where p <= q < r, merges two
already sorted subarrays A[p ..q] and A[q+1 .. r]. It takes Q(n)
time
u To sort an array A[1 .. n], we call MERGE_SORT(A, 1, n)
Analyzing Divide-And-Conquer Algorithms
¢ Running time is described by a recurrence equation or
recurrence
¢ Assume:
u A problem is divided into a subproblems, each of which is 1/b
the size of the original
u Dividing the problem takes D(n) time
u Combining the solutions to subproblems into the solution to
the original problem takes C(n) time
¢ T(n) = Q(1) if n <= c,
= aT(n/b) + D(n) + C(n) otherwise
Analyzing Divide-And-Conquer Algorithms
¢ Analysis of Merge Sort
u Divide: Computes the middle of the subarray D(n) = Q(1)
u Conquer: We recursively solve two subproblems, each of size
n/2, contributing 2T(n/2)
u Combine: The MERGE procedure takes Q(n),
so, C(n) = Q(n)
¢ The worst-case running time of merge sort is:
u T(n) = Q(1) if n = 1,
= 2T(n/2) + Q(n) if n > 1