Mastering Algorithm Design and Analysis
Mastering Algorithm Design and Analysis
Analysis
A comprehensive guide to understanding computational complexity, algorithmic
strategies, and data structure optimization. This presentation covers essential
topics from time complexity analysis to advanced graph algorithms, equipping you
with the theoretical foundations and practical techniques needed to design efficient
solutions for complex computational problems.
Time Complexity Analysis
Asymptotic Notation Common Complexity Classes
Big O (O) - Upper bound representing worst-case complexity.
# O(1) - Constant Time
Describes the maximum time an algorithm could take.
def get_first_element(arr):
"""Accesses an element at a fixed index."""
# Example: Linear Search (Worst Case: item at end or not found)
return arr[0] if arr else None
def linear_search_O(arr, target):
"""
Searches for a target in an array.
Time Complexity: O(n) - In the worst case, we check every # O(log n) - Logarithmic Time
element. def binary_search(arr, target):
""" """Searches a sorted array by repeatedly dividing the search
for i in range(len(arr)): interval in half."""
if arr[i] == target: left, right = 0, len(arr) - 1
return i while left <= right:
return -1 mid = (left + right) // 2
if arr[mid] == target:
# arr = [10, 20, 30, 40, 50], target = 50 (last element) or 60 (not return mid
found) elif arr[mid] < target:
# In both cases, the loop runs 'n' times. left = mid + 1
else:
right = mid - 1
Theta () - Tight bound representing average-case complexity. return -1
Describes both upper and lower bounds when they coincide.
left_half = merge_sort(left_half)
# Example: Linear Search (Best Case: item at beginning)
right_half = merge_sort(right_half)
def linear_search_omega(arr, target):
"""
return merge(left_half, right_half)
Searches for a target in an array.
Time Complexity: «(1) - In the best case, the target is the first
def merge(left, right):
element,
"""Merges two sorted arrays."""
and we find it in a single step.
result = []
"""
i=j=0
for i in range(len(arr)):
while i < len(left) and j < len(right):
if arr[i] == target:
if left[i] < right[j]:
return i
[Link](left[i])
return -1
i += 1
else:
# arr = [10, 20, 30, 40, 50], target = 10 (first element)
[Link](right[j])
# The loop runs only once.
j += 1
[Link](left[i:])
[Link](right[j:])
Understanding these notations allows precise communication about
return result
algorithm performance across different input scenarios.
def generate_permutations(elements):
"""
Generates all possible permutations of a list of elements.
The number of permutations for 'n' elements is n!.
"""
return list(permutations(elements))
Distributes elements into buckets, sorts each bucket. Works well for
uniformly distributed data.
Heaps & Priority Queues
Array Representation
Parent of i: (i-1)/2
Left child of i: 2i+1
Right child of i: 2i+2
Min heap: parent f children
Max heap: parent g children
Greedy Algorithms
Greedy algorithms make locally optimal choices at each step, hoping to find a global optimum. They work when problems exhibit optimal
substructure and the greedy choice property.
HuffmanCoding(frequencies): TSP_Greedy(graph):
Create leaf nodes for each char visited = {start}
Build min heap of nodes current = start
while [Link] > 1: tour = [start]
left = ExtractMin(heap) while [Link] < n:
right = ExtractMin(heap) nearest = FindNearest(current, unvisited)
parent = Node([Link] + [Link]) [Link](nearest)
[Link] = left [Link](nearest)
[Link] = right current = nearest
Insert(heap, parent) [Link](start)
return heap[0] return tour
Optimal prefix-free encoding. Build tree bottom-up by merging two Nearest neighbor heuristic. Not optimal but fast approximation. At
lowest frequency nodes. Left=0, right=1 for encoding. Frequently each city, visit nearest unvisited city. Returns to start.
used characters get shorter codes.
Order Statistics & Dynamic Programming
i-th Order Statistics Dynamic Programming Techniques
Finding the i-th smallest element in an unsorted array. DP solves problems by breaking them into overlapping subproblems, storing
solutions to avoid recomputation.
Max/Min: O(n) - Single pass comparison
Memoization (Top-Down): Recursive with caching
FindMax(arr):
max = arr[0] Fib_Memo(n, memo):
for i = 1 to n: if n in memo: return memo[n]
if arr[i] > max: if n <= 1: return n
max = arr[i] memo[n] = Fib_Memo(n-1, memo) + Fib_Memo(n-2, memo)
return max return memo[n]
Min-Max Pairwise: O(3n/2) - Compare pairs first Tabulation (Bottom-Up): Iterative table filling
FindMinMax(arr): Fib_Tab(n):
Compare pairs, track min/max dp[0] = 0, dp[1] = 1
Comparisons: 3+n/2+ - 2 for i = 2 to n:
dp[i] = dp[i-1] + dp[i-2]
Selection via Sorting: O(n log n) return dp[n]
Select(arr, i):
Sort(arr)
return arr[i]
RandomSelect(arr, l, r, i):
if l == r: return arr[l]
pi = RandomPartition(arr, l, r)
k = pi - l + 1
if i == k: return arr[pi]
else if i < k:
return RandomSelect(arr, l, pi-1, i)
else:
return RandomSelect(arr, pi+1, r, i-k)
Key DP Properties:
DFS(graph, u):
visited[u] = true
time++
discovery[u] = time
for v in adj[u]:
if not visited[v]:
parent[v] = u
DFS(graph, v)
time++
finish[u] = time
01 02 03
Topological Sort (Kahn's Algorithm) Topological Sort (DFS) Strongly Connected Components
Time: O(V+E) Time: O(V+E)
(Kosaraju)
Time: O(V+E)
TopSort_Kahn(graph): TopSort_DFS(graph):
Compute in-degree for all vertices for each vertex u: Kosaraju(graph):
queue = vertices with in-degree 0 if not visited[u]: // First DFS
while queue not empty: DFS_TopSort(u) for each vertex:
u = [Link]() return reversed(finish_order) if not visited:
[Link](u) DFS(graph, vertex)
for v in adj[u]: DFS_TopSort(u): // Transpose graph
in-degree[v]-- visited[u] = true graph_T = Transpose(graph)
if in-degree[v] == 0: for v in adj[u]: // Second DFS in reverse finish order
[Link](v) if not visited[v]: for vertex in reversed(finish_order):
DFS_TopSort(v) if not visited_T[vertex]:
[Link](u) DFS(graph_T, vertex)
// Each DFS tree is an SCC
Advanced Graph Algorithms & Beyond
Minimum Spanning Tree Shortest Paths Maximum Flow
Kruskal: O(E log E) - Sort edges, use Dijkstra (SSSP): O((V+E) log V) - Ford-Fulkerson/Edmonds-Karp: O(VE²)
Union-Find Non-negative weights
MaxFlow(graph, s, t):
Kruskal(graph): Dijkstra(graph, start): flow = 0
Sort edges by weight dist[start] = 0 while exists augmenting path
MST = [] pq = [(0, start)] (BFS):
for edge (u,v,w) in sorted_edges: while pq not empty: bottleneck = min capacity on
if Find(u) != Find(v): (d, u) = [Link]() path
Union(u, v) for (v, w) in adj[u]: flow += bottleneck
[Link]((u,v,w)) if dist[u] + w < dist[v]: Update residual graph
dist[v] = dist[u] + w return flow
Prim: O(E log V) - Grow tree from start parent[v] = u
vertex [Link]((dist[v], v)) Dinic's Algorithm: O(V²E)
FloydWarshall(graph):
for k = 1 to V:
for i = 1 to V:
for j = 1 to V:
dist[i][j] = min(dist[i][j],
dist[i][k] +
dist[k][j])