Algorithms and Complexity - Detailed Notes
What is an Algorithm?
An algorithm is a step-by-step procedure or a finite sequence of instructions designed to perform a
specific task or solve a particular problem. Algorithms are fundamental to computer science and are
used in data processing, calculations, and automated reasoning. They are language-independent
and can be implemented using any programming language. The purpose of an algorithm is to
transform input into the desired output through a clearly defined process. For example, a sorting
algorithm takes an unsorted list of numbers as input and returns the list sorted in ascending or
descending order.
Characteristics of an Algorithm
To qualify as an algorithm, a set of instructions must have the following characteristics:
1. Input: An algorithm should have zero or more well-defined inputs.
2. Output: It must produce at least one output that is the solution to the problem.
3. Definiteness: Each instruction must be clear, unambiguous, and precisely defined.
4. Finiteness: The algorithm must terminate after a finite number of steps.
5. Effectiveness: The operations must be simple enough to be performed exactly and within a
reasonable amount of time.
What is Time Complexity?
Time complexity is a measure that indicates the amount of time an algorithm takes to complete as a
function of the input size (usually denoted by 'n'). It helps in evaluating the efficiency of an algorithm
by estimating the number of basic operations or steps executed. Time complexity is expressed
using asymptotic notations such as Big O (O), Omega (Ω), and Theta (Θ). For instance, an
algorithm with time complexity O(n^2) will take time proportional to the square of the input size.
Analyzing time complexity helps in choosing the best algorithm when multiple solutions are
available.
What is Asymptotic Notation?
Asymptotic notations are mathematical tools used to describe the behavior of an algorithm as the
input size becomes very large. They help in analyzing and comparing the efficiency of algorithms
independently of hardware or programming language. The three main types of asymptotic notations
are:
1. Big O Notation (O): Represents the worst-case time complexity. It defines the upper bound of an
algorithm's running time.
2. Omega Notation (Ω): Represents the best-case time complexity. It defines the lower bound of an
algorithm's running time.
3. Theta Notation (Θ): Represents the average-case or exact bound. It gives a tight bound on the
running time.