0% found this document useful (0 votes)
9 views3 pages

Greedy Algorithms & Dynamic Programming Guide

The document outlines Greedy Algorithms and Dynamic Programming, highlighting their definitions, key properties, and examples. Greedy Algorithms make locally optimal choices for a global optimum, while Dynamic Programming solves problems by breaking them into overlapping subproblems and storing solutions. Both approaches have their advantages and disadvantages, with Dynamic Programming guaranteeing optimal solutions but often requiring more memory.

Uploaded by

robatakele2024
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
9 views3 pages

Greedy Algorithms & Dynamic Programming Guide

The document outlines Greedy Algorithms and Dynamic Programming, highlighting their definitions, key properties, and examples. Greedy Algorithms make locally optimal choices for a global optimum, while Dynamic Programming solves problems by breaking them into overlapping subproblems and storing solutions. Both approaches have their advantages and disadvantages, with Dynamic Programming guaranteeing optimal solutions but often requiring more memory.

Uploaded by

robatakele2024
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Short Notes C3: on Greedy Algorithms & Dynamic Programming

1. Greedy Algorithms
Definition: Makes locally optimal choices at each step to find a global optimum.

Key Properties:

 Greedy Choice Property: Best local choice leads to global optimum.

 Optimal Substructure: Optimal solution includes optimal subproblem solutions.


Examples: Fractional Knapsack, Dijkstra’s Algorithm, Kruskal’s Algorithm, Huffman Coding.

How It Works
1. Start: Initial problem state.
2. Evaluate: All possible choices at current state.
3. Choose: Best local option (no backtracking).
4. Repeat: Until goal is reached or no further progress.
Example: Coin Change Problem
Goal: Minimum coins for change (e.g., 39¢ using [1, 2, 5, 10]).

Steps:
1. Pick largest coin ≤ remaining amount (10¢ → 39 - 10 = 29¢).
2. Repeat until amount = 0.
Solution: 3 × 10¢ + 1 × 5¢ + 2 × 2¢ = 4 coins.

Minimum Spanning Tree (MST)


Kruskal’s Algorithm:
Steps:
1. Sort edges by weight.
2. Add edges to MST if no cycle forms.
Time Complexity: O(E log V) (using Union-Find).
Example: Sorted edges: B→D (5), A→B (6), C→F (9), etc.
Total cost: 53.
Shortest Path (Dijkstra’s Algorithm)
Steps:
1. Initialize distances (source = 0, others = ∞).
2. Visit nearest unvisited node, update neighbors.
3. Repeat until all nodes visited.

Example:
Source (S): S→A (6), S→E (7), S→D (8).
Final shortest paths: A (6), E (7), D (8), C (11), B (15).

Pros & Cons


Advantages:

 Simple, efficient (often O(n log n)).

 No re-evaluation of past choices.

Disadvantages:

 May not yield global optimum (e.g., TSP).

 Limited to problems with greedy-choice property.

2. Dynamic Programming (DP)


Definition: Solves problems by breaking into overlapping subproblems, storing solutions to
avoid recomputation.

Key Properties:

 Overlapping Subproblems: Reuses solutions (memoization).

 Optimal Substructure: Combines subproblem solutions for global optimum.


Examples: Fibonacci, Knapsack, Floyd-Warshall, LCS.
DP vs. Greedy vs. Divide & Conquer

Feature DP Greedy Divide & Conquer

Subproblems Overlapping Independent Independent

Approach Bottom-up/Top-down Local optimum Recursive splitting

Storage Memoization No storage No storage

Steps to Solve DP Problems

 Characterize Structure: Define subproblems.

 Recursive Solution: Express problem in terms of subproblems.

 Compute Bottom-Up: Fill DP table iteratively.

 Construct Solution: Derive answer from table.


Example: Fibonacci
Recursive: O(2ⁿ) → Inefficient.
DP: Store computed values (O(n)).

Applications

 Matrix Chain Multiplication: Minimize scalar multiplications.

 0/1 Knapsack: Maximize value without exceeding weight.

 TSP: Shortest route visiting all cities (DP + bitmasking).

Pros & Cons


Advantages:

 Guarantees optimal solution.

 Avoids recomputation (efficient for overlapping problems).

Disadvantages:

 High memory usage (tables).

 Complex to implement.

Common questions

Powered by AI

Memoization is critical in dynamic programming because it enables the reuse of solutions to subproblems, reducing computation time by avoiding redundant calculations. It records already computed values, which solves overlapping subproblems efficiently, a key aspect of problems suited for dynamic programming. This differs from greedy algorithms, which do not store any previously computed values as they operate based on making a single local optimum choice per step without revisiting past decisions .

A greedy algorithm might fail to find a global optimum in problems that do not have the greedy-choice property or optimal substructure. For example, the Traveling Salesman Problem (TSP) can mislead a greedy algorithm into a suboptimal solution since the best immediate next choice can prevent achieving a minimum total route cost. Problems lacking the property where local decisions lead to optimal global outcomes are not suitable for greedy solutions .

The greedy choice property ensures that by making the best local choice at each step, a global optimum can be achieved. This property is essential because it allows greedy algorithms to construct a solution incrementally, making a choice that looks the best at that moment without considering the consequences. It assumes that by choosing a local optimum, one can build toward a global solution. Examples such as Dijkstra’s and Kruskal’s algorithms rely on this property to efficiently solve shortest paths and minimum spanning tree problems, respectively .

Optimal substructure is a shared characteristic of greedy algorithms and dynamic programming approaches, which means that an optimal solution to a problem can be constructed efficiently from optimal solutions to its subproblems. In greedy algorithms, this property allows for the global optimum by choosing the best local option iteratively, as seen in the fractional knapsack or Kruskal’s algorithm. In dynamic programming, this property permits combining previously solved subproblems systematically to build a global solution, such as in the Fibonacci sequence or matrix chain multiplication, making efficient use of stored results from subproblems .

Kruskal’s algorithm has a time complexity of O(E log V), efficient for finding a minimum spanning tree in graphs, where E is the number of edges and V is the number of vertices. Dijkstra’s algorithm, primarily used for finding the shortest path from a source node, can have a time complexity of O(V^2) with a simple implementation or O((V + E) log V) with a priority queue. Kruskal’s would be chosen when the specific problem is finding an MST due to its effective use of sorted edge weights, avoiding cycle formation. Dijkstra’s, being tailored for pathfinding, is the choice for shortest path problems .

Appropriate problem selection is critical for implementing greedy algorithms because these approaches are only effective when the problem inherently exhibits the greedy-choice property and optimal substructure. Problems like minimum spanning trees (MST) and shortest path problems are suitable because local decisions lead to global optimization without the need for backtracking. Incorrect problem selection, like complex combinatorial tasks without these properties, can result in suboptimal solutions or failures because greedy algorithms lack the ability to look ahead or revise past decisions. Ensuring a problem’s characteristics match the strengths of greedy methods is essential for their success .

Dynamic programming is preferred for the 0/1 Knapsack problem because it guarantees an optimal solution by considering every combination of items to maximize value within given constraints. This problem involves optimal substructure and overlapping subproblems, which suit dynamic programming’s systematic evaluation and storage of subproblem solutions in a table. A greedy algorithm might provide a quick but suboptimal solution because it could select items based on initial criteria (like highest value-to-weight ratio) that do not result in an overall optimal combination of items .

Dynamic programming generally has higher memory usage than greedy algorithms due to its reliance on storing and accessing a table of solutions for overlapping subproblems. This requirement can lead to substantial memory consumption, especially in cases with large subproblem spaces, offsetting its computational efficiency gains. In contrast, greedy algorithms typically use minimal memory by making immediate decisions based on current state variables without storing previously calculated data. This minimalistic approach reduces memory overhead, which can be advantageous in memory-constrained environments, albeit at potential cost to solution optimality if the problem doesn't suit a greedy approach .

Overlapping subproblems in dynamic programming enhance computational efficiency by avoiding redundant calculations of the same subproblems multiple times. Instead of recomputing the solution for these subproblems, dynamic programming stores previously computed results in a data structure (usually a table) and reuses them as needed. This reduces the overall number of computations and makes it feasible to solve complex problems that would otherwise have an exponential running time if recalculating each subproblem independently .

Dynamic programming and divide and conquer both solve problems by breaking them into smaller subproblems, but their approach and use of subproblem results differ. Dynamic programming deals with overlapping subproblems and constructs solutions using stored subproblem results (memoization) to ensure no recomputation, following a bottom-up or top-down approach. Divide and conquer splits problems into independent subproblems, solves each recursively, and combines results without overlapping subproblems or storing past solutions. This distinction makes dynamic programming more suited for problems like the Fibonacci sequence, whereas divide and conquer excels in tasks like merge sort .

You might also like