0% found this document useful (0 votes)
17 views6 pages

Dynamic Programming Explained: Concepts & Applications

Dynamic programming

Uploaded by

munasinmunna19
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
17 views6 pages

Dynamic Programming Explained: Concepts & Applications

Dynamic programming

Uploaded by

munasinmunna19
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd

Dynamic Programming

Dynamic programming is based on the principle of optimality (also coined by


Bellman)

The steps in a dynamic programming solution are:

 Verify that the principle of optimality holds

 Set up the dynamic-programming recurrence equations

 Solve the dynamic-programming recurrence equations for the value of the optimal
solution.

 Perform a trace back step in which the solution itself is constructed.

General method
• Works the same way as divide-and-conquer, by combining solutions to sub problems
– Divide-and-conquer partitions a problem into independent sub problems
– Greedy method only works with the local information

• Dynamic programming is required to take into account the fact that the problems may
not be partitioned into independent sub problems
– The sub problem is solved only once and the answer saved in a table
∗ Applicable when the sub problems overlap, or sub problems share
sub sub problems
– Solution to a problem can be viewed as the result of a sequence of decisions
∗ Knapsack problem solution
· Decide the values of xi, 0 ≤ i < n
· Make a decision on x0, then x1, and so on
· An optimal sequence maximizes the objective function ∑pixi under the
constraints ∑wi xi ≤ m and 0 ≤ xi ≤ 1

∗ Shortest path problem

Determine the shortest path from vertex i to vertex j by finding the second vertex in
the path, then the third, and so on, until vertex j is reached

· Optimal sequence of decisions is one with a path of least length


– Optimal sequence of decisions can be found by making the decisions one at a time and never
making an erroneous decision
∗ This is the idea followed in greedy algorithms
∗ Can be guaranteed by trying all possible decision sequences but the time and space
costs will be prohibitive

∗ Shortest path problem

· Find shortest path from vertex i to vertex j


· Let Ai be the set of vertices adjacent from i
· The selection of vertex from Ai cannot be finalized without looking further
ahead
· If we have to find a shortest path from a single source to all vertices in G, then at
each step, a correct decision can be made

– Dynamic programming drastically reduces the amount of enumeration by avoiding the


enumeration of some decision sequences that cannot possibly be optimal; an optimal
sequence of decisions is obtained by using the principle of optimality

∗ Applicable when the sub problems are not entirely independent, they may have
common sub subproblems

∗ Each sub subproblem is solved only once and the results used repeatedly

• The word programming in dynamic programming does not refer to coding but refers to
building tables of intermediate results

• Typically used for optimization problems that may have many possible solutions
– An optimal solution vs the optimum solution

Definition 1: The principle of optimality states that an optimal sequence of decisions


has the property that whatever the initial state and decision are, the remaining states must
constitute an optimal decision sequence with regard to the state resulting from the first
decision.

• Greedy method vs Dynamic programming


– In greedy method, only one decision sequence is ever generated
– In dynamic programming, many decision sequences may be generated
– Sequences containing suboptimal sequences cannot be optimal because of principle of
optimality, and so, will not be generated

greedy method dynamic programming


greedy method produces only one feasible dynamic programming produces all
solution, which may or may not be optimal, possible sub-problems at most once, one of
which guaranteed to be optimal.
– Shortest path problem

∗ Assume that i, i1, i2, . . . , ik, j is a shortest path from i to j


∗ Starting with vertex i, a decision is made to go to i1
∗ Following this decision, the problem state is defined by vertex i1, and we need to find
a path from i1 to j
∗ The sequence i1, i2, . . . , ik, j must be a shortest i1 to j path
∗ If not, let i1, r1, r2, . . . , rq, j be a shortest i1 to j path
∗ Then, i, i1, r1, r2, . . . , rq, j is an i to j path that is shorter than the path i, i1, i2, . . . , ik,
j
∗ Hence, principle of optimality is applicable

• Development of solution broken into four steps:

1. Characterize the structure of an optimal solution


2. Recursively define the value of an optimal solution
3. Compute the value of an optimal solution in bottom-up fashion
4. Construct an optimal solution from computed information

• Computing the nth Fibonacci number


int fib ( const int n )
{
if ( n <= 0 ) return ( 0 );
if ( n == 1 ) return ( 1 );
return ( fib ( n - 1 ) + fib ( n - 2 ) );
}
– This code is extremely inefficient; why?

• An efficient code using an array of size n


int fib ( const int n )
{
if ( n <= 0 ) return ( 0 );
if ( n == 1 ) return ( 1 );
int * a = new int[n+1];
a[0] = 0;
a[1] = 1;
for ( int i ( 2 ); i <= n; i++ )
a[i] = a[i-1] + a[i-2];
int tmp ( a[n] );
delete[] a;
return tmp;
}
• If we want to save space as well, the following code is more appropriate

int fib ( const int n )


{
if ( n <= 0 ) return ( 0 );
if ( n == 1 ) return ( 1 );
int f0 ( 0 ), f1 ( 1 ), f;
for ( int i ( 2 ); i <= n; i++ )
{
f = f1 + f0;
f0 = f1;
f1 = f;
}
return f;
}
The above solution is known as bottom-up dynamic programming – we compute the smallest
values first and build the solution using the solution to smaller problems; most of the real
dynamic programming situations refer to top-down dynamic programming (also known as
memoization) as you will see next in knapsack problem

• Knapsack problem
– Recursive solution
∗ Each time you choose an item, you assume that you can optimally find a
solution to pack the rest of the knapsack
struct item
{
int size;
int val;
};
int knapsack ( const int capacity )
{
int t;
// N is the number of item types
for ( int i ( 0 ), int max ( 0 ); i < N; i++ )
if ( ( int space = capacity - items[i].size ) >= 0 )
{
Remove items[i] from items;
if ( ( t = knapsack ( space ) + items[i].val ) > max )
max = t;
}
return ( max );
}
– Dynamic programming solution
∗ Decide the values of xi, 1 ≤ i ≤ n
∗ Make a decision on x1, then on x2, and so on
∗ An optimal sequence of decisionsmaximizes the objective function∑pixi, under the
Constraints ∑wixi ≤ m and 0 ≤ xi ≤ 1

int knapsack ( const int capacity )


{
int maxi, t;
if ( maxknown[capacity] )
return ( maxknown[capacity] );
for ( int i ( 0 ), int max ( 0 ); i < N; i++ )
if ( ( int space = capacity - items[i].size ) >= 0 )
if ( ( t = knapsack ( space ) + items[i].val ) > max )
{
max = t; maxi = i;
}
maxknown[capacity] = max;
itemknown[capacity] = items[maxi];
return ( max );
}
• 0/1 knapsack
– Same as regular knapsack problem except that the xi’s are restricted to a value of 0 or 1
(take nothing or everything for an item)
– Formally, an instance of knapsack problem knap ( l, j, y ) is
Maximize ∑l≤i≤j pixi
subject to ∑l≤i≤j wixi ≤ y
xi = 0 or 1, l ≤ i ≤ j

and the knapsack problem is represented by knap ( 1, n, m )


– Let y1, y2, . . . , yn be an optimal sequence of 0/1 values for x1, x2, . . . , xn
respectively
∗ If y1 = 0, then y2, y3, . . . , yn must constitute an optimal sequence for the
problem knap ( 2, n, m )
∗ If it is not an optimal subsequence, then y1, y2, . . . , yn is not an optimal
sequence for knap ( 1, n, m )
∗ If y1 = 1, then y2, y3, . . . , yn must constitute an optimal sequence for the
problemknap ( 2, n, m - w1 )
∗ If it is not an optimal subsequence, then · there is another 0/1 sequence z2, z3, . .
. , zn such that
n n n

∑ wizi ≤m−1∧∑ pizi>∑ piyi


i=2 i=2 i=2

∗ Hence, the sequence y1, z2, z3, . . . , zn gives a sequence with greater value
∗ Not possible due to principle of optimality QED

Common questions

Powered by AI

Recursive solutions for the knapsack problem directly reflect the decision process of including or excluding items, with each decision spawning new subproblems, leading to exponential time complexity due to repeated calculations. Conversely, dynamic programming transforms these overlapping subproblems into a systematic approach that calculates an optimal solution once and stores it for future use. Thus, dynamic programming reduces time complexity to polynomial, as it efficiently tabulates subproblem solutions and builds the optimal solution iteratively .

Table-building is significant in dynamic programming as it ensures each subproblem is solved only once and avoids repeated computation. In the 0/1 knapsack problem, the table stores the results of subproblems representing various capacities with available items, thus allowing the algorithm to systematically build the optimal solution based on these precomputed values. This approach efficiently reduces redundant calculations, hence optimizing the use of computational resources and ensuring timely decisions for larger capacities .

The principle of optimality is essential for dynamic programming because it dictates that any possible solution can only be optimal if every subproblem within that solution is also optimal. This principle means partial solutions need not be verified in isolation; instead, the global solution’s integrity verifies the constituent subproblems' optimality. Consequently, this principle allows dynamic programming to efficiently traverse and check fewer pathways by eliminating non-viable subproblem configurations early in the process .

The principle of optimality is critical in dynamic programming for distinguishing acceptable sequences from suboptimal ones by ensuring that any segment of an optimal decision sequence remains optimal. In shortest path problems, it dictates that the path constructed from any vertex must itself be the shortest path to subsequent vertices. This prevents consideration of suboptimal paths that might initially seem promising but contain subpaths that are not shortest. The principle allows dynamic programming to prune decision sequences that cannot be part of an optimal solution, enhancing efficiency .

Dynamic programming optimizes computational efforts in the knapsack problem by breaking it into subproblems, each representing a decision about an item. Unlike recursive strategies, dynamic programming avoids recalculating solutions by storing results in a table, thus ensuring each subproblem is solved only once. By applying the principle of optimality, it constructs an optimal solution incrementally, ensuring that the current choice does not negate previously optimal decisions, reducing redundant computations, and minimizing time complexity .

The 0/1 knapsack problem is more complex because it requires making binary decisions—take or leave each item—resulting in a combinatorial explosion of possible solutions. The principle of optimality simplifies its computation by ensuring that once the optimal choice is made for one item, the remaining subproblem must also be optimally solved. This allows dynamic programming to build a solution incrementally, using earlier computed optimal subproblems to construct solutions for larger problems without redundantly recalculating them .

The iterative solution of the Fibonacci sequence improves efficiency by using a bottom-up dynamic programming approach, storing and updating only two previous numbers at a time rather than recalculating every subproblem repeatedly as in the recursive approach. This method significantly reduces time complexity from exponential to linear (O(n)) and decreases space complexity, which can be controlled by using a constant amount of space. This efficiency stems from avoiding redundant calculations and minimizing memory usage .

Dynamic programming ensures the optimality of solutions by adhering to the principle of optimality, which states that an optimal sequence of decisions has the property that whatever the initial state and decision are, the remaining states must constitute an optimal decision sequence. In the shortest path problem, this principle dictates that for any path i, i1, i2, ..., ik, j, each subpath must also be optimal. If not, replacing a subpath with a shorter path results in a shorter overall path, contradicting the original's optimality. Dynamic programming reduces unnecessary enumeration by avoiding non-optimal decision sequences and storing intermediate results for repeated use .

The greedy method generates only one decision sequence, potentially non-optimal, as it makes decisions based purely on local information. In contrast, dynamic programming explores multiple decision sequences and utilizes the principle of optimality to ensure optimal solutions by considering overlapping subproblems, which are stored and reused. Dynamic programming is preferred for problems with overlapping subproblems like the knapsack problem because it systematically solves each subproblem once and uses previously computed results, thereby improving efficiency and ensuring global optimality .

Bottom-up dynamic programming computes solutions from the smallest subproblems upward, building on previously computed results, as seen in calculating Fibonacci numbers iteratively, which saves space and reduces time complexity. Top-down, or memoization, recursively breaks the problem into subproblems but stores results to avoid duplication, providing the same efficiency as bottom-up by caching. While both achieve the same result, bottom-up is iterative and suits problems where all subproblems are needed; top-down is more natural for recursive problem decomposition, exemplifying flexibility in approach application .

You might also like