0% found this document useful (0 votes)
7 views3 pages

Algorithm Complexity Reference Guide

Uploaded by

wiremu casey
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
7 views3 pages

Algorithm Complexity Reference Guide

Uploaded by

wiremu casey
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Algorithms & Analysis - Quick Reference Guide

Complexity Hierarchy
O(1) < O(log n) < O(n) < O(n log n) < O(n²) < O(n³) < O(2ⁿ) < O(n!)

Asymptotic Notation
Big O (Upper Bound): f(n) ≤ c·g(n) for large n
Big Ω (Lower Bound): f(n) ≥ c·g(n) for large n
Big Θ (Tight Bound): c₁·g(n) ≤ f(n) ≤ c₂·g(n) for large n

Algorithm Complexity Table

Algorithm Best Average Worst Space Notes


Searching
Linear Search O(1) O(n) O(n) O(1) Unsorted data
Binary Search O(1) O(log n) O(log n) O(1) Sorted data required
Sorting
Selection Sort O(n²) O(n²) O(n²) O(1) Min swaps, unstable
Bubble Sort O(n) O(n²) O(n²) O(1) Stable, adaptive
Insertion Sort O(n) O(n²) O(n²) O(1) Stable, online
Merge Sort O(n log n) O(n log n) O(n log n) O(n) Stable, predictable
Quick Sort O(n log n) O(n log n) O(n²) O(log n) In-place, unstable
Heap Sort O(n log n) O(n log n) O(n log n) O(1) In-place, unstable
Graph Traversal
DFS O(V+E) O(V+E) O(V+E) O(V) Uses stack/recursion
BFS O(V+E) O(V+E) O(V+E) O(V) Uses queue, shortest path

Master Theorem
For T(n) = aT(n/b) + f(n):

Case 1: f(n) = O(n^(log_b(a) - ε)), ε > 0


→ T(n) = Θ(n^log_b(a))

Case 2: f(n) = Θ(n^log_b(a))


→ T(n) = Θ(n^log_b(a) · log n)

Case 3: f(n) = Ω(n^(log_b(a) + ε)), ε > 0, regularity condition


→ T(n) = Θ(f(n))

Common Recurrence Patterns


T(n) = T(n-1) + c → O(n) [Linear recursion]
T(n) = T(n/2) + c → O(log n) [Binary search]
T(n) = 2T(n/2) + c → O(n) [Simple divide & conquer]
T(n) = 2T(n/2) + n → O(n log n) [Merge sort]
T(n) = T(n-1) + n → O(n²) [Selection sort recursion]
T(n) = 2T(n-1) + c → O(2ⁿ) [Fibonacci recursion]

Summation Formulas
Σ(i=1 to n) 1 = n
Σ(i=1 to n) i = n(n+1)/2 ≈ n²/2
Σ(i=1 to n) i² = n(n+1)(2n+1)/6 ≈ n³/3
Σ(i=1 to n) i³ = [n(n+1)/2]² ≈ n⁴/4
Σ(i=0 to n) 2ⁱ = 2^(n+1) - 1 ≈ 2ⁿ
Σ(i=1 to n) 1/i ≈ ln(n)

Logarithm Properties
log(1) = 0
log(a) = 1 (base a)
log(xy) = log(x) + log(y)
log(x/y) = log(x) - log(y)
log(xʸ) = y·log(x)
log_a(x) = log_b(x) / log_b(a)
log₂(n) ≈ 3.32 · log₁₀(n)

Graph Representations
Adjacency Matrix
Space: O(V²)
Edge lookup: O(1)
Add vertex: O(V²)
Add edge: O(1)
Best for: Dense graphs, frequent edge queries

Adjacency List

Space: O(V + E)
Edge lookup: O(degree)
Add vertex: O(1)
Add edge: O(1)
Best for: Sparse graphs, traversal algorithms

Algorithm Design Paradigms


Brute Force
Strategy: Try all possibilities
Complexity: Often exponential
Examples: TSP, subset sum, naive string matching
When to use: Small inputs, need optimal solution

Decrease and Conquer


Strategy: Reduce to smaller subproblem
Variants: By constant (insertion sort), by factor (binary search), variable (GCD)
Complexity: Often O(n) or O(log n)
Examples: Binary search, insertion sort, Euclidean algorithm

Divide and Conquer


Strategy: Split into multiple subproblems
Complexity: Often O(n log n)
Examples: Merge sort, quick sort, binary tree operations
Recurrence: T(n) = aT(n/b) + f(n)

Data Structure Operations

Structure Access Search Insert Delete Space


Array O(1) O(n) O(n) O(n) O(n)
Sorted Array O(1) O(log n) O(n) O(n) O(n)
Linked List O(n) O(n) O(1) O(1) O(n)
Stack O(n) O(n) O(1) O(1) O(n)
Queue O(n) O(n) O(1) O(1) O(n)
Hash Table N/A O(1)* O(1)* O(1)* O(n)
Binary Tree O(n) O(n) O(n) O(n) O(n)
BST (balanced) O(log n) O(log n) O(log n) O(log n) O(n)
Heap N/A O(n) O(log n) O(log n) O(n)

*Average case for hash table

Problem-Solving Checklist
Algorithm Analysis
Identify input size parameter
Choose basic operation (innermost, most frequent)
Consider best/average/worst cases
Set up summation or recurrence
Solve using standard techniques
Express in asymptotic notation

Graph Problems
Understand graph type (directed/undirected, weighted/unweighted)
Choose representation (matrix vs list)
Select traversal method (DFS vs BFS)
Consider connectivity and path requirements
Analyze space and time complexity

Sorting Problems
Consider input characteristics (size, order, duplicates)
Determine stability requirements
Evaluate space constraints (in-place vs extra space)
Choose appropriate algorithm
Analyze complexity for given scenario
Common Pitfalls
Asymptotic notation: Confusing O, Ω, Θ meanings
Summation setup: Incorrect loop bounds or nesting
Recurrence solving: Wrong base case or substitution errors
Graph traversal: Using wrong data structure (stack vs queue)
Complexity analysis: Choosing wrong basic operation

Exam Strategy
1. Identify problem type: Analysis, design, tracing, comparison
2. Use systematic approach: Follow standard methods
3. Show all work: Partial credit for correct process
4. Verify with examples: Check answers on small inputs
5. State assumptions: Clarify ambiguous problems
6. Manage time: Don't get stuck on single question

Memory Aids
DFS = Depth First = Stack (LIFO) = Goes Deep
BFS = Breadth First = Queue (FIFO) = Level by Level

Stable sorts: Bubble, Insertion, Merge (BIM)


In-place sorts: Selection, Bubble, Insertion, Heap, Quick

O(log n): Binary operations (search, tree height)


O(n): Single pass through data
O(n log n): Optimal comparison sorting
O(n²): Nested loops, simple sorting
O(2ⁿ): Exponential, subset generation

Quick Complexity Estimates


n = 10³: log n ≈ 10, n log n ≈ 10⁴, n² ≈ 10⁶
n = 10⁶: log n ≈ 20, n log n ≈ 2×10⁷, n² ≈ 10¹²
n = 10⁹: log n ≈ 30, n log n ≈ 3×10¹⁰, n² ≈ 10¹⁸

Practical limits (1 second):


O(1), O(log n): Any reasonable n
O(n): n ≤ 10⁸
O(n log n): n ≤ 10⁷
O(n²): n ≤ 10⁴
O(n³): n ≤ 500
O(2ⁿ): n ≤ 25
O(n!): n ≤ 10

Common questions

Powered by AI

An adjacency matrix provides constant-time complexity for edge lookups and addition (O(1)), making it suitable for dense graphs with frequent edge queries. However, it has a higher space complexity of O(V²), where V is the number of vertices, which makes it less efficient for sparse graphs. Conversely, an adjacency list uses space proportional to the number of edges O(V + E), making it preferable for sparse graphs, but edge lookup incurs a time cost proportional to the degree of the vertex, and vertex additions take O(1) time. Thus, the choice depends on the graph density and specific operations required .

A balanced binary search tree (BST) provides significant performance benefits over a standard binary tree in scenarios requiring frequent operations like search, insert, and delete, which are optimized to O(log n) in a balanced BST compared to the potential O(n) in a standard binary tree. This efficiency arises because a balanced BST maintains its height close to the minimal log(n), ensuring optimal structural behavior for dynamic data sets. This is particularly beneficial in applications with high-frequency modifications or queries where performance consistency and speed are critical .

Sorting algorithm performance is influenced by whether an algorithm is stable or in-place. Stable sorting algorithms like Bubble Sort, Insertion Sort, and Merge Sort maintain the relative order of equal elements, which can be important for certain data types. In-place algorithms, such as Selection Sort, Bubble Sort, Insertion Sort, Heap Sort, and Quick Sort, do not require additional space proportional to the input size, making them more space-efficient. The choice between stability and in-place operations depends on specific requirements, such as the need for maintaining order in datasets with equal values versus minimizing space usage .

The Master Theorem provides a way to solve recurrence relations of the form T(n) = aT(n/b) + f(n) common in divide and conquer algorithms. It determines the time complexity based on the function f(n). Case 1 (f(n) = O(n^(log_b(a) - ε))) implies the direct cost of combining results is small, leading to T(n) = Θ(n^log_b(a)). Case 2 (f(n) = Θ(n^log_b(a))) means that the cost of combining results dominates or is equivalent to the subproblems, hence T(n) = Θ(n^log_b(a)·log n). Case 3 (f(n) = Ω(n^(log_b(a) + ε))) indicates the combination cost is significant, giving T(n) = Θ(f(n)) if a regularity condition is met. These cases guide the selection of appropriate algorithms based on problem characteristics .

Asymptotic notation helps in understanding the efficiency of algorithms by providing a high-level approximation of their behavior as input size grows, abstracting away lower-order terms and constant factors to focus on growth patterns. Key symbols include Big O (O) for the upper bound, indicating the worst-case scenario; Big Omega (Ω) for the lower bound, representing the best-case scenario; and Big Theta (Θ) for the tight bound, signifying both upper and lower bounds. These notations allow for a comparative analysis of algorithm efficiency, essential for performance prediction and improvement .

The space complexity of graph traversal methods DFS and BFS differs primarily in their stack versus queue data structures. DFS utilizes a stack or recursion, leading to a space complexity of O(V) in the worst-case scenario of deep recursion, suitable for exploring the depths of sparse or tree-like graphs. BFS, on the other hand, uses a queue, maintaining a broader frontier at each level, also with a space complexity of O(V) but can grow significantly in width in dense graphs. This wider breadth in BFS can lead to greater space consumption in large, dense graphs. Therefore, the choice between DFS and BFS depends on graph density and the desired exploration pattern .

One would choose Binary Search over Linear Search due to its much faster average and worst-case time complexity of O(log n) compared to O(n) of Linear Search. However, Binary Search requires the data to be sorted beforehand, which can be a limitation if sorting overhead is substantial or if the dataset is frequently updated. Therefore, the implications of using Binary Search include the additional cost of maintaining a sorted dataset, whereas Linear Search requires no such precondition but is less efficient for large datasets .

Bubble Sort and Insertion Sort are both stable sorting algorithms but differ in adaptability. Bubble Sort is adaptive, performing optimally with a time complexity of O(n) when the data is nearly sorted, making minimal swaps. In contrast, Insertion Sort is inherently adaptive and online, which means it can handle streams of data efficiently as items are inserted into the list. Insertion Sort performs optimally with O(n) complexity on nearly sorted data or small data sets due to linear comparisons and swaps. Its online nature gives a performance edge in dynamic data environments where elements are regularly added .

The adjacency matrix provides a space complexity of O(V²) and an edge lookup complexity of O(1), making it well-suited for dense graphs where many edges exist, facilitating rapid edge queries. In contrast, an adjacency list's space complexity is O(V + E), aligned with the number of edges, and edge lookup requires O(degree) time, making it more efficient for sparse graphs where space efficiency and traversal operations are a priority. Each representation benefits distinctively based on the graph's density and the frequency of operations like edge queries or traversals .

The time complexity of Quick Sort degrades to O(n²) in the worst-case scenario where the pivot chosen consistently etches out an unbalanced partition, such as choosing the smallest or largest element repeatedly in an already sorted or nearly sorted array (ascending or descending). This unbalance means that one of the partitions contains n-1 elements, leading to reduced efficiency of the divide-and-conquer mechanism. To avoid this degradation, randomized pivot selection or median-of-three strategies are often employed to ensure more balanced partitions on average .

You might also like