NP-Complete Problems and Backtracking
NP-Complete Problems and Backtracking
1. NP COMPLETE PROBLEMS:
The computing times of algorithms fall into two groups.
● Group2 – contains problems whose best known algorithms are non polynomial.
Another useful property of the class NP is that all NP problems can be solved in
exponential time (EXP). This is because we can always list out all short certificates in
exponential time and check all O (2nk) of them. Thus, P is in NP, and NP is in EXP.
Although we know that P is not equal to EXP, it is possible that NP = P, or EXP, or
neither. Frustrating!
● NP-hardness
As we will see, some problems are at least as hard to solve as any problem in NP.
We call such problems NP-hard. How might we argue that problem X is at least as
hard (to within a polynomial factor) as problem Y? If X is at least as hard as Y, how
would we expect an algorithm that is able to solve X to behave?
1.1.1 NP –HARD and NP – Complete Problems Basic Concepts
If an NP-hard problem can be solved in polynomial time, then all NP-complete
problems can be solved in polynomial time.
All NP-complete problems are NP-hard, but all NP- hard problems are not NP-complete.
The class of NP-hard problems is very rich in the sense that it contains many problems
from a wide variety of disciplines.
● NP-complete (NPC): the class of problems which are NP-hard and belong to NP.
1.1.2 NP-Competence
• How we would you define NP-Complete
• They are the “hardest” problems in NP
NP-complete problems are a subset of the larger class of NP (nondeterministic
polynomial time) problems. NP problems are a class of computational problems that can be
solved in polynomial time by a non-deterministic machine and can be verified in
polynomial time by a deterministic Machine. A problem L in NP is NP-complete if all
other problems in NP can be reduced to L in polynomial time. If any NP-complete problem
can be solved in polynomial time, then every problem in NP can be solved in polynomial
time. NP-complete problems are the hardest problems in the NP set.
● A problem is NP-Hard if it obeys Property 2 above and need not obey Property 1.
Therefore, a problem is NP-complete if it is both NP and NP-hard.
Figure 3: Reduction
Learning reduction, in general, is very important. For example, if we have library functions
to solve certain problems and if we can reduce a new problem to one of the solved
problems, we save a lot of time. Consider the example of a problem where we have to find
the minimum product path in a given directed graph where the product of the path is the
multiplication of weights of edges along the path. If we have code for Dijkstra’s algorithm
to find the shortest path, we can take the log of all weights and use Dijkstra’s algorithm to
find the minimum product path rather than writing a fresh code for this new problem.
1.1.5 How to prove that a given problem is NP-complete?
From the definition of NP-complete, it appears impossible to prove that a problem L is NP-
Complete. By definition, it requires us to that show every problem in NP is polynomial
time reducible to L. Fortunately, there is an alternate way to prove it. The idea is to take a
known NP-Complete problem and reduce it to L. If a polynomial-time reduction is
possible, we can prove that L is NP-Complete by transitivity of reduction (If an NP-
Complete problem is reducible to L in polynomial time, then all problems are reducible to
L in polynomial time).
1.1.6 What was the first problem proved as NP-Complete?
There must be some first NP-Complete problem proved by the definition of NP-Complete
problems. SAT (Boolean satisfiability problem) is the first NP-Complete problem proved
by Cook (See CLRS book for proof).
It is always useful to know about NP-Completeness even for engineers. Suppose you are
asked to write an efficient algorithm to solve an extremely important problem for your
company. After a lot of thinking, you can only come up exponential time approach which
is impractical. If you don’t know about NP-Completeness, you can only say that I could
not come up with an efficient algorithm. If you know about NP-Completeness and prove
that the problem is NP-complete, you can proudly say that the polynomial-time solution is
unlikely to exist. If there is a polynomial-time solution possible, then that solution solves a
big problem of computer science many scientists have been trying for years.
1.1.7 Example of NP-Complete problem
NP problem: - Suppose a DECISION-BASED problem is provided in which a set of
inputs/high inputs you can get high output.
Criteria to come either in NP-hard or NP-complete.
The point to be noted here, the output is already given, and you can verify the
output/solution within the polynomial time but can't produce an output/solution in
polynomial time.
Here we need the concept of reduction because when you can't produce an output of the
problem according to the given input then in case you have to use an emphasis on the
concept of reduction in which you can convert one problem into another problem.
So according to the given decision-based NP problem, you can decide in the form of yes or
no. If, yes then you have to do verify and convert into another problem via reduction
concept. If you are being performed, both then decision-based NP problems are in NP
compete.
Here we will emphasize NPC.
Figure 4: NP-Complete
2. BACKTRACKING
Backtracking, which is a very important skill set to solve recursive solutions. Recursive
functions are those that calls itself more than once. Consider an example of Palindrome:
Initially, the function isPalindrome(S, 0, 8) is called once with the parameters
isPalindrome(S, 1, 7). The recursive call isPalindrome(S, 1, 7) is called once with the
parameters isPalindrome(S, 2, 6).
Backtracking is one of the techniques that can be used to solve the problem. We can
write the algorithm using this strategy. It uses the Brute force search to solve the problem,
and the brute force search says that for the given problem, we try to make all the possible
solutions and pick out the best solution from all the desired solutions. This rule is also
followed in dynamic programming, but dynamic programming is used for solving
optimization problems. In contrast, backtracking is not used in solving optimization
problems. Backtracking is used when we have multiple solutions, and we require all those
solutions.
Backtracking name itself suggests that we are going back and coming forward; if it
satisfies the condition, then return success, else we go back again. It is used to solve a
problem in which a sequence of objects is chosen from a specified set so that the sequence
satisfies some criteria.
● A piece of sufficient information is not available to make the best choice, so we use
the backtracking strategy to try out all the possible solutions.
● Each decision leads to a new set of choices. Then again, we backtrack to make new
decisions. In this case, we need to use the backtracking strategy.
We start with a start node. First, we move to node A. Since it is not a feasible solution
so we move to the next node, i.e., B. B is also not a feasible solution, and it is a dead-end
so we backtrack from node B to node A.
Suppose another path exists from node A to node C. So, we move from node A to
node C. It is also a dead-end, so again backtrack from node C to node A. We move from
node A to the starting node.
Now we will check any other path exists from the starting node. So, we move from
start node to the node D. Since it is not a feasible solution so we move from node D to
node E. The node E is also not a feasible solution. It is a dead end so we backtrack from
node E to node D.
Suppose another path exists from node D to node F. So, we move from node D to
node F. Since it is not a feasible solution and it's a dead-end, we check for another path
from node F.
Suppose another path exists from node D to node F. So, we move from node D to
node F. Since it is not a feasible solution and it's a dead-end, we check for another path
from node F.
Suppose there is another path exists from the node F to node G so move from node
F to node G. The node G is a success node.
The terms related to the backtracking are:
● Live node: The nodes that can be further generated are known as live nodes.
● E node: The nodes whose children are being generated and become a success node.
● Dead node: The node which cannot be further generated and also does not provide
a feasible solution is known as a dead node.
Many problems can be solved by backtracking strategy, and that problems satisfy complex
set of constraints, and these constraints are of two types:
● Explicit constraint: The rules that restrict each element to be chosen from the
given set.
● N-queen problem
● Sum of subset problem
● Graph coloring
● Hamiliton cycle
3. N QUEENS PROBLEM:
N - Queens problem is to place n - queens in such a manner on an n x n chessboard that
no queens attack each other by being in the same row, column or diagonal.
It can be seen that for n =1, the problem has a trivial solution, and no solution exists for
n =2 and n =3. So first we will consider the 4 queens problem and then generate it to n -
queens problem.
● The algorithm starts by placing a queen on the first column, then it proceeds to
the next column and places a queen in the first safe row of that column.
● If the algorithm reaches the 8th column and all queens are placed in a safe
position, it prints the board and returns true.
● The “isSafe” function checks if it is safe to place a queen on a certain row and
column by checking if there are any queens in the same row, diagonal or anti-
diagonal.
● It’s worth to notice that this is just a high-level pseudocode and it might need to
be adapted depending on the specific implementation and language you are
using.
Figure 9: 8 Queens Problem
● Therefore, two queens lie on the duplicate diagonal if and only if |j-l|=|i-k|
Place (k, i) returns a Boolean value that is true if the kth queen can be placed in column i. It
tests both whether i is distinct from all previous costs x1, x2,....xk-1 and whether there is
no other queen on the same diagonal.
Using place, we give a precise solution to then n- queens problem.
Place (k, i)
{
For j 1 to k - 1
do if (x [j] = i)
or (Abs x [j]) - i) = (Abs (j - k))
then return false;
return true;
}
Place (k, i) return true if a queen can be placed in the kth row and ith column otherwise
return is false.
x [] is a global array whose final k - 1 values have been set. Abs (r) returns the absolute
value of r.
N - Queens (k, n)
{
For i 1 to n
do if Place (k, i) then
{
x [k] i;
if (k ==n) then
write (x [1....n));
else
N - Queens (k + 1, n);
}
}
4. Sum of Subsets problem
It is one of the most important problems in complexity theory. The problem is given an
A set of integers a1, a2,…., an upto n integers. The question arises that is there a non-
empty subset such that the sum of the subset is given as M integer?. For example, the set is
given as [5, 2, 1, 3, 9], and the sum of the subset is 9; the answer is YES as the sum of the
subset [5, 3, 1] is equal to 9. This is an NP-complete problem again. It is the special case of
knapsack
Let's understand this problem through an example.
problem.
We have a set of 5 integers given below:
N = 4, -2, 2, 3, 1
We want to find out the subset whose sum is equal to 5. There are many solutions to this
problem.
The naïve approach, i.e., brute-force search generates all the possible subsets of the
original array, i.e., there are 2n possible states. Here the running time complexity would be
exponential. Then, we consider all these subsets in O(N) linear running time and checks
whether the sum of the items is M or not.
The dynamic programming has pseudo-polynomial running time.
Statement: Given a set of positive integers, and a value sum, determine that the sum of the
subset of a given set is equal to the given sum.
Or
Given an array of integers and a sum, the task is to have all subsets of given array with sum
equal to the given sum.
● Recursion
● Dynamic programming
● Include: Here include means that we are selecting the element from the array.
● Exclude: Here, exclude means that we are rejecting the element from the array.
● Second scenario is reject. The array arr contains the elements 4, 5, 2, i.e., arr = [4,
5, 2] and sum would be same as 9 as we are rejecting the element 3.
The result[] array would remain empty.
Now we perform the same select and reject operation on element 4 as it is the first element
of the array now.
● Select the element 4 from the array. Since we are selecting 4 from the array so array
arr would contain the elements 5, 2, i.e., arr = [5, 2]. The sum is equal to the 6-4 = 2
and the element 4 gets stored in the result arr. The result[] = {3, 4}.
● Reject the element 4 from the array. Since we are rejecting the 4 from the array so
array arr would contain the elements 5, 2, i.e., arr = [5, 2]. The sum would remain
same as 6 and the result array would be same as previous, i.e., {3}.
Now we perform the select and reject operation on element 5.
● Select the element 5 from the array. Since we are selecting 5 from the array so array
arr would contain the elements 2, i.e., arr = [2]. The sum is equal to the 2 - 5 equals
to -3 and the element 5 gets stored in the result arr. The result[] = {3, 4, 5}.
● Reject the element 5 from the array. Since we are rejecting 5 from the array so
array arr would contain the element 2, i.e., arr = [2]. The sum would remain same
as previous, i.e., 6 and the result array would be same as previous, i.e., {3, 4}.
If we observe S-5, we can see that the sum is negative that returns false. It means that there
is no further subset available in the set.
● Select the element 2 from the array. Once the element 2 gets selected, the array
becomes empty, i.e., arr[] = " ". The sum would be 2-2 equals to 0 and the
element 2 gets stored in the result array. The result[] = [3, 4, 2].
● Reject the element 2 from the array. Once the element 2 gets rejected, the array
becomes empty, i.e., arr[] = " ". The sum would be same as previous, i.e., 2 and
the result array would also be same as previous, i.e., [3, 4].
● Select the element 2 from the array. Since we are selecting 2 from the array so array arr
would be empty, i.e., arr = " ". The sum would be 1-2 equals to -1 and the element 2
gets stored in the result array. The result[] = [3, 5, 2].
● Reject the element 2 from the array. Since we are rejecting 2 from the array so array arr
would become empty. The sum would remain same as previous, i.e., 1 and the result
array would be same as previous, i.e., {3, 5}.
Consider R-5. It has two scenarios:
● Select the element 2 from the array. Since we are selecting 2 from the array so array
arr would be empty, i.e., arr = " ". The sum would be 6-2 equals to 4 and the
element 2 gets stored in the result array. The result[] = [3, 2].
● Reject the element 2 from the array. Since we are rejecting 2 from the array so
array arr would become empty. The sum would remain same as previous, i.e., 6 and
the result array would be same as previous, i.e., {3}.
Similarly, we get the reject case, i.e., R-3 as shown as below:
4.2 Method 2: Dynamic Programming
Let A be an array or set which contains 'n' non-negative integers. Find a subset 'x'
of set 'A' such that the sum of all the elements of x is equal to w where x is another input
(sum).
For example:
A = {1, 2, 5, 9, 4}
Sum(w) = 18
Now we have to find out the subset from the given set whose sum is equal to 18. Here we
will use the dynamic programming approach to solve the subset sum problem.
Example:
A = [2, 3, 5, 7, 10]
Sum(w) = 14
First, we create a table. The column contains the values from 0 to 14 while row contains
the elements of the given set shown as below:
In the below table:
Consider the element 2. We will use 1 as a true value and 0 as a false value. The value 1
comes under 0 and 2 columns shown as below:
When j= 1
Required sum = 1 - 2 = -1; Since the sum is negative so put 0 under the column 1 as
shown in the above table.
When j= 2
Required sum = 2 - 2 = 0; Since the value of sum is zero so we put 1 under the column 2
as shown in the above table.
We put 0 under the columns whose sum is greater than 2 as we cannot make sum more
than 2 from the element 2.
Here i = 2, a[i] = 3
The columns whose sum is less than 3 will have the same values as the previous columns.
When j = 3, sum[j] = 3
Required sum = 3 -3 = 0; since the sum is zero so we put 1 under the column 3 as shown
in the above table.
When j = 4; sum[j] = 4
Required sum = 4 - 3 = 1; Since the sum is 1 so we move to the previous row, i.e., i=1 and
j=1. The value at a[1][1] is 0 so we put 0 at a[2][4].
When j = 5, sum[j] = 5
When j = 6, sum[j] = 6
When j = 7, sum[7] = 7
Required sum = 5-5 = 0; Since the value of sum is 0; therefore, the value at a[2][5] equals
to 1.
When j = 6, sum[j] = 6
Required sum = 6-5 = 1; the value of sum is 1 so the value at a[2][1] equals to 0; therefore,
the value at a[3][6] equals to 0.
Required sum = 7-5 = 2; the value of sum is 2 so the value at a[2][2] equals to 1; therefore,
the value at a[3][7] equals to 1.
Required sum = 8-5 = 3; the value of sum is 3 so the value at a[2][3] equals to 1; therefore,
the value at a[3][8] equals to 1.
Required sum= 9-5 = 4; the value of sum is 4 so the value at a[2][4] equals to 0; therefore
the value at a[3][9] equals to 0.
Required sum = 9 - 7 = 2; the value of sum is 2 so the value at a[3][2] equals to 1; therefore
the value at a[4][9] equals to 1.
Required sum = 11-7 = 4; the value of sum is 4 so the value at a[3][4] equals to 0;
therefore, the value at a[4][11] equals to 0.
Required sum = 12-7 = 5; the value of sum is 5 so the value at a[3][5] equals to 1;
therefore, the value at a[4][12] equals to 1.
Required sum = 12-10 = 2; the value of sum is 2 so the value at a[4][2] equals to 1;
therefore, the value at a[5][12] equals to 1.
To determine whether the above given problem contains the subset or not, we need to
check the last row and the last column. If the value is 1 which means that there would exist
atleast one subset.
We have basically followed three conditions where we write 1 in the cell of the table:
● A[i] = j
● A[i-1][j] = 1
● A[i-1][j-A[i]] = 1