0% found this document useful (0 votes)
13 views6 pages

Understanding Algorithms and Their Design

An algorithm is a finite sequence of instructions designed to solve a specific problem, characterized by properties such as finiteness, definiteness, completeness, effectiveness, input, output, correctness, and efficiency. Various algorithm design techniques include Divide and Conquer, Greedy Technique, Dynamic Programming, Branch and Bound, Randomized Algorithms, and Backtracking, each suited for different types of problems. The analysis of algorithms focuses on measuring their efficiency through time and space complexity, evaluating worst, best, and average case scenarios.
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
13 views6 pages

Understanding Algorithms and Their Design

An algorithm is a finite sequence of instructions designed to solve a specific problem, characterized by properties such as finiteness, definiteness, completeness, effectiveness, input, output, correctness, and efficiency. Various algorithm design techniques include Divide and Conquer, Greedy Technique, Dynamic Programming, Branch and Bound, Randomized Algorithms, and Backtracking, each suited for different types of problems. The analysis of algorithms focuses on measuring their efficiency through time and space complexity, evaluating worst, best, and average case scenarios.
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

What is an Algorithm?

Definition:

1. An algorithm is a step-by-step finite sequence of instructions to solve a


particular problem.
2. It is the sequence of instructions that specify a sequence of steps to be
followed to solve a specific problem.

So, to solve any given problem, we first have to define the problem and then
design and analyse the algorithm required to solve that problem.

Example of an Algorithm:

Algorithm to add two numbers and store the result in a third variable.

Step1: Start

Step2: Read the first number in variable “num1”.

Step3: Read the second number in variable “num2”.

Step4: Perform the addition of num1 and num2 and store the result in variable
“sum”.

Step5: Print the value of “sum”.

Step6: Stop
An algorithm must have the following properties:

1. Finiteness: It means that the algorithm must complete its execution after
some finite number of steps.
2. Definiteness: Each steps of the algorithm must be precisely defined and
unambiguous.
3. Completeness or Generality: Algorithm should be complete so that it can
solve all the problems of the same type for which it is being designed.
4. Effectiveness: All the instructions and operations used in the algorithm
should be simple and feasible so that the algorithm can be implemented in
any programming language on the computer system.
5. Input: An algorithm must have 0 or well defined input for execution.
6. Output: An algorithm must give at least one output.
7. Correctness: The algorithm should produce the output based on the
requirement of the algorithm.
8. Efficient – Efficiency of an algorithm is measured in terms of time and
space complexity for implementing the algorithm. The algorithm should take
less running time and memory space for its execution.

Why study Algorithm?


As the speed of processor increases, performance is frequently said to be less
central than other software quality characteristics (e.g. security, extensibility,
reusability etc.). However, large problem sizes are commonplace in the area of
computational science, which makes performance a very important factor. This
is because longer computation time, to name a few mean slower results, less
through research and higher cost of computation (if buying CPU Hours from an
external party). The study of Algorithm, therefore, gives us a language to
express performance as a function of problem size.
Need of Algorithm
1. To understand the basic idea of the problem.
2. To find an approach to solve the problem.
3. To improve the efficiency of existing techniques.
4. To understand the basic principles of designing the algorithms.
5. To compare the performance of the algorithm with respect to other
techniques.
6. It is the best method of description without describing the implementation
detail.
7. The Algorithm gives a clear description of requirements and goal of the
problem to the designer.
8. A good design can produce a good solution.
9. To understand the flow of the problem.
10. To measure the behavior (or performance) of the methods in all cases (best
cases, worst cases, average cases)
11. With the help of an algorithm, we can also identify the resources (memory,
input-output) cycles required by the algorithm.
12. With the help of algorithm, we convert art into a science.
13. To understand the principle of designing.
14. We can measure and analyze the complexity (time and space) of the
problems concerning input size without implementing and running it; it will
reduce the cost of design.
Algorithm Design Techniques

The following is a list of several popular design approaches:


1. Divide and Conquer Approach: It is a top-down approach. The algorithms
which follow the divide & conquer techniques involve three steps:
o Divide the original problem into a set of subproblems.
o Solve every subproblem individually, recursively.
o Combine the solution of the subproblems (top level) into a solution of the
whole original problem.
Examples: Binary Search, Quick Sort, Merge Sort, Closest Pair of Points,
Strassen’s matrix multiplication algorithm.

2. Greedy Technique: Greedy method is used to solve the optimization


problem. An optimization problem is one in which we are given a set of
input values, which are required either to be maximized or minimized
(known as objective), i.e. some constraints or conditions.
o Greedy Algorithm always makes the choice (greedy criteria) looks best at
the moment, to optimize a given objective.
o The greedy algorithm doesn't always guarantee the optimal solution however
it generally produces a solution that is very close in value to the optimal.
Examples: Knapsack Problem, job Sequencing with deadline, Kruskal's
Minimal Spanning Tree Algorithm, Prim's Minimal Spanning
Tree Algorithm, Dijkstra's Minimal Spanning Tree Algorithm, Optimal
Merge Pattern, Huffman Coding.

3. Dynamic Programming: Dynamic Programming is a bottom-up approach


we solve all possible small problems and then combine them to obtain
solutions for bigger problems.
This is particularly helpful when the number of copying subproblems is
exponentially large. Dynamic Programming is frequently related
to Optimization Problems.
Examples: Matrix Chain Multiplication, Longest Common Subsequence,
Travelling Salesman Problem, 0/1 knapsack problem, All pair Shortest path
problem, Sum of subsets problem, Optimal Cost Binary Search Tree, Multi-
stage graph, Fibonacci Series.

4. Branch and Bound: In Branch & Bound algorithm a given subproblem,


which cannot be bounded, has to be divided into at least two new restricted
subproblems. Branch and Bound algorithm are methods for global
optimization in non-convex problems. Branch and Bound algorithms can be
slow, however in the worst case they require effort that grows exponentially
with problem size, but in some cases we are lucky, and the method coverage
with much less effort.
Examples: This approach is used for a number of NP-Hard problems, such
as Travelling Salesman Problem, Knapsack Problem, Assignment Problem,
Integer Programming, Non-Linear Programming.

5. Randomized Algorithms: A randomized algorithm is defined as an


algorithm that is allowed to access a source of independent, unbiased
random bits, and it is then allowed to use these random bits to influence its
computation.
Example: Randomized Quick Sort Algorithm, Randomized Minimum-cut
Algorithm, randomized Algorithm for the N-Queens Problem.
A randomized algorithm uses a random number at least once during the
computation make a decision.
Example 1: In Quick Sort, using a random number to choose a pivot.
Example 2: Trying to factor a large number by choosing a random number
as possible divisors.

6. Backtracking Algorithm: Backtracking Algorithm tries each possibility


until they find the right one. It is a depth-first search of the set of possible
solution. During the search, if an alternative doesn't work, then backtrack to
the choice point, the place which presented different alternatives, and tries
the next alternative.
Example: Recursive Maze Algorithm, Hamiltonian Circuit Problem,
Subset-sum Problem, N-Queens Problems, Graph Coloring Problem.
Analysis of Algorithm

The analysis of algorithm is performed to calculate the efficiency of an


algorithm. The efficiency of an algorithm is measured on the basis of the
performance of the algorithm.
We can measure the performance of an algorithm by computing two factors:
1. Amount of time required by an algorithm to execute (Time
Complexity).
2. Amount of space required by an algorithm (Space Complexity).
Types:
1. Worst Case Efficiency: It is the maximum number of steps that an
algorithm can take for any input size n.
2. Best Case Efficiency: It is the minimum number of steps that an algorithm
can take for any input size n.
3. Average Case Efficiency: It is the average number of steps that an
algorithm can take for any input size n.

Complexity of Algorithm
It is very convenient to classify algorithm based on the relative amount of time
or relative amount of space they required and specify the growth of time/space
requirement as a function of input size.
1. Time Complexity: Running time of a program as a function of the size of
the input.
2. Space Complexity: Some forms of analysis could be done based on how
much space an algorithm needs to complete its task. This space complexity
analysis was critical in the early days of computing when storage space on
the computer was limited. When considering this algorithm are divided into
those that need extra space to do their work and those that work in place.
But now a day's problem of space rarely occurs because space on the computer
(internal or external) is enough.

Common questions

Powered by AI

Algorithm analysis helps in reducing computational costs by determining the most efficient algorithm concerning time and space, which is essential for handling large-scale computations . By choosing algorithms with better time complexity, tasks are completed faster, and costs related to computational resources are minimized . This is particularly evident in cloud computing where computational time directly translates to costs. For example, using optimized algorithms in cloud-processing tasks minimizes data center usage time and resulting expenses .

The properties of an algorithm ensure its effectiveness in problem-solving by providing a structured approach for execution. Finiteness guarantees completion, while definiteness ensures each step is clear and unambiguous, preventing errors during execution . Completeness allows the algorithm to solve all problems of a similar type, ensuring its applicability across different scenarios . Effectiveness involves feasibility, ensuring that all operations can be practically implemented on computer systems . For example, in algorithms like binary search, all these properties are evident: the algorithm is finite, definite, complete, and effective, thus effectively finding an item in a sorted array .

Understanding time and space complexity is crucial in algorithm analysis as it measures efficiency. Time complexity reflects the algorithm's running time as a function of input size, which is vital for predicting performance on large inputs . Space complexity indicates the memory requirement, important for systems with limited memory capacity . Both measures allow developers to choose or design algorithms that balance speed and memory usage, critical for large-scale computing and optimization problems. For example, quick sort benefits from low space complexity due to its in-place sorting, whereas merge sort offers better time complexity in certain scenarios despite using extra space .

The study of algorithms is fundamental in advancing computer science as it provides a systematic approach to problem-solving that elevates software design and optimization . By understanding algorithms, computer scientists gain insights into designing more efficient systems, enabling innovations in areas from artificial intelligence to computational biology. The efficiency gained through algorithmic advancements translates into faster, more capable software that can handle increasingly complex tasks . Furthermore, algorithms form the backbone of emerging technologies, driving the development of more sophisticated applications, optimizing resource usage, and pushing the boundaries of existing computational limits . This comprehensive understanding not only contributes to the theoretical aspects of computing but also to practical applications that drive technological evolution.

Different algorithm design techniques tackle complex problems by using unique approaches suited to the problem type. The Divide and Conquer approach breaks the problem into subproblems, solves each recursively, and then combines the solutions, as seen in merge sort . The Greedy Technique makes decisions that seem optimal at each step to achieve a global optimum, like in Kruskal's Algorithm . Dynamic Programming, in contrast, solves smaller subproblems once and stores their results, which is useful for problems with overlapping subproblems, such as the Fibonacci series . Branch and Bound explores solutions in a tree structure, reducing search space based on bounds, common in NP-hard problems like the Traveling Salesman Problem . These differences highlight their applicability based on problem characteristics.

Algorithms provide significant benefits in computational science by enabling efficient problem-solving as problem sizes increase . They allow performance to be expressed as a function of problem size, facilitating improvements in scalability and feasibility of solutions for large datasets . This efficiency is critical in fields like data analysis, simulations, and modeling, where the ability to handle massive datasets swiftly enables more thorough research and quicker results, reducing costs like CPU hours . Moreover, the proper design and analysis of algorithms support advancements in areas requiring large-scale computation, ensuring longer computations remain feasible and affordable .

The divide and conquer approach solves computational problems by breaking down a problem into smaller, more manageable subproblems, each of which is solved independently . This is effective because it simplifies complex problems and allows recursive solutions that fit well in cases like sorting (e.g., merge sort) or searching (e.g., binary search). By addressing each subproblem as a stand-alone issue, it reduces the complexity of large problems, allowing for more efficient problem-solving as the solutions to subproblems are combined to solve the original problem .

A greedy algorithm makes the locally optimal choice at each step with the hope of finding the global optimum, but it doesn't guarantee the optimal solution. It often provides a feasible solution quickly, such as in the Knapsack Problem . In contrast, dynamic programming considers all possible solutions by solving each subproblem just once and storing the results, ensuring the optimal solution is found. This can lead to higher computational overhead compared to the greedy approach but guarantees optimal outcomes, as in the case of the 0/1 Knapsack Problem . The implications are significant; in time-critical applications, a greedy algorithm may suffice with acceptable results, whereas dynamic programming is preferable where accuracy is paramount.

Backtracking differs from other algorithm design techniques by systematically searching through all possible configuration spaces to find solutions. Unlike divide and conquer or greedy techniques which might not explore all possibilities, backtracking explores each branch of a solution space fully but backs out of branches that don’t lead to solutions . It is particularly suited for problems with constraints where exploring all configurations is necessary, such as the N-Queens Problem or Sudoku, where partial solutions can be tested and discarded when failure is detected . This makes it effective in search-based problem-solving scenarios where feasible solutions must adhere to defined constraints.

Randomized algorithms utilize randomness to make decisions within their execution, sometimes simplifying algorithms or speeding up performance . For instance, randomized quicksort uses a random pivot to improve expected performance over deterministic pivot selection . The advantages include simpler implementation and often an improvement in average performance time, especially useful in dealing with large data sets or uncertain data distributions. Despite their potential to perform worse in rare cases, they are effective for problems like the randomized min-cut algorithm, where the randomization can help avoid worst-case scenarios .

You might also like