UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
Upload
Sign in
Edward Blurock, profile picture
Uploaded byEdward Blurock
PPTX, PDF2,733 views

Computability and Complexity

The document provides an overview of algorithms, computability, and complexity, focusing on the definitions and foundational concepts of algorithms and their limitations. It elaborates on the three fundamental questions about algorithms: their definition, their computability concerning specific tasks, and their complexity in terms of time and resources. Notably, it discusses Turing machines and the implications of the Entscheidungsproblem, emphasizing the existence of unsolvable problems in computation.

Education◦
Downloaded 101 times
1 / 104
2 / 104
Most read
3 / 104
4 / 104
5 / 104
6 / 104
7 / 104
8 / 104
9 / 104
10 / 104
11 / 104
Most read
12 / 104
13 / 104
Most read
14 / 104
15 / 104
16 / 104
17 / 104
18 / 104
19 / 104
20 / 104
Algorithms, Computability and Complexity Overview of mathematical concepts Review of concepts already discussed
Algorithm • You have a task • You want to formulate an algorithm to perform the task • Three basic questions: – What Is an algorithm? – Will an algorithm be able to compute the task? – How long will the algorithm take? • how complex is the algorithm?
Three questions about algorithms • These are actually non trivial questions – What Is an algorithm? – Will an algorithm be able to compute the task? – How long will the algorithm take? • how complex is the algorithm? • Important to mathematically formalize these questions – to create better algorithms – To know where the limitations of algorithms lie
Questions • What Is an algorithm? – To make an algorithm, you have to know “exactly” what it is.. Its components and how to put these components together. – To ask questions, you have to know what you are asking questions about • Will an algorithm be able to compute the task? – Is the task I want impossible? – Is it possible to formulate an impossible task – What is the consequences for other algorithms • How complex is the algorithm? • Will it take “too much” time? (for cryptography, this is a good thing) • Can I make a faster algorithm for a given type of task?
DEFINITION What Is an algorithm? To make an algorithm, you have to know “exactly” what it is.. Its components and how to put these components together. To ask questions, you have to know what you are asking questions about
Algorithm Definition • Part of the research is to formulate an exact workable definition of an algorithm • There are several/many definitions – The do have common characteristics
Algorithm Definition • A logical sequence of steps for solving a problem, … » From http://Dictionary.msn.com • Dale and Lewis: » a plan of solution for a problem » Algorithm – An unambiguous (and precise) set of steps for solving a problem (or sub-problem) in a finite amount of time using a finite amount of data.
Algorithm Definition • Shackelford, Russell L. in Introduction to Computing and Algorithms – – “An algorithm is a specification of a behavioral process. It consists of a finite set of instructions that govern behavior step-by-step.”
Definition • An algorithm is a finite sequence of step by step, discrete, unambiguous instructions for solving a particular problem – has input data, and is expected to produce output data – each instruction can be carried out in a finite amount of time in a deterministic way
Keywords • Notice the term finite. – Algorithms should lead to an eventual solution. – The algorithm should stop (halt). • Sequence of steps – Each step should do one logical action.
COMPUTABILITY Will an algorithm be able to compute the task? Is the task I want impossible? Is it possible to formulate an impossible task What is the consequences for other algorithms
Computability vs Complexity • Computability refers to whether of not in principle it is possible to evaluate f(n) by following a set of instructions. • We are not for the moment worried if this computation requires 1,000,000 or more consecutive steps. • The latter refers to complexity which we will return to later.
Ensheidungsproblem Decision problem Hilbert 1928 The Entscheidungsproblem asks for an algorithm that takes as input a statement of a first-order logic and answers "Yes" or "No" according to whether the statement is universally valid (valid in every structure satisfying the axioms). An algorithm to ask whether something is true Is it computable
Ensheidungsproblem Decision problem Hilbert 1928 • Before the question could be answered, the notion of "algorithm" had to be formally defined. – This was done by Alonzo Church in 1936 with the concept of "effective calculability" based on his λ calculus – and by Alan Turing in the same year with his concept of Turing machines. – It was recognized immediately by Turing that these are equivalent models of computation.
15 Alan Turing (1912 – 1954) Alonzo Church (1903-1995) Turing Machine Lambda calculus Two mathematical ways to ask questions about “computability”
Equivalent Computers z z zz z z z 1 Start HALT ), X, L 2: look for ( #, 1, - ), #, R (, #, L (, X, R #, 0, - Finite State Machine ... Turing Machine  term = variable | term term | (term) |  variable . term y. M  v. (M [y v]) where v does not occur in M. (x. M)N   M [ x N ]   Lambda Calculus
Alan Turing, 1936 • "On computable numbers, with an application to the Entscheidungsproblem [decision problem]” • addressed a previously unsolved mathematical problem posed by the German mathematician David Hilbert in 1928:
Turing, 1936 Is there, in principal, any definite mechanical method or process by which all mathematical questions could be decided? The Decision Problem
Turing, 1936 • To answer the question (in the negative), • Turing proposed a simple abstract computing machine, • modeled after a mathematician with a pencil, an eraser, and a stack of pieces of paper • He asserted that any function is a computable function if it is computable by one of these abstract machines.
Turing, 1936 • He then investigated whether there were some mathematical problems which could not be solved by any such abstract machine. • Such abstract computing machines are now called "Turing machines". • One particular Turing machine, called a "Universal Turing Machine", served as the model for designing actual programmable computers.
Why a Turing Machine Because Turing machines are very simple compared with computers in practical use, conceptually easier to prove impossibility results for Turing machines. These impossibility results apply as well to all known computers
Turing, 1936 • Startling result of Turing's 1936 paper • assertion that there are well-defined problems that cannot be solved by any computational procedure.
What is Computable? Computation is usually modelled as a mapping from inputs to outputs, carried out by a formal “machine,” or program, which processes its input in a sequence of steps. An “effectively computable” function is one that can be computed in a finite amount of time using finite resources. ………… ………… … input yes no output Effectively computable function
Unsolvable • problems are formulated as functions, we call unsolvable functions are • noncomputable; • Problems formulated as predicates, • they are called • undecidable. • Using Turing's concept of the abstract machine, we would say that a function is noncomputable if there exists no Turing machine that could compute it.
25 ............ Tape Read-Write head Control Unit
-Dept. of CSIE, CCU, Taiwan- 26 ............ Read-Write head No boundaries -- infinite length The head moves Left or Right The tape OR
-Dept. of CSIE, CCU, Taiwan- 27 ............ Read-Write head The head at each time step: 1. Reads a symbol 2. Writes a symbol 3. Moves Left or Right
-Dept. of CSIE, CCU, Taiwan- 28 ............ Example: Time 0 ............ Time 1 a a cb a b k c 1. Reads a 2. Writes k 3. Moves Left a
-Dept. of CSIE, CCU, Taiwan- 29 ............ Time 1 a b k c ............ Time 2 a k cf 1. Reads b 2. Writes f 3. Moves Right b
Add 1 to a sequence of ones If 1, then move right If 1, then move left Loop If 0, replace with 1 Start: 0 0 0 1 1 1 1 0 0 0 4 ones in a row End: 0 0 0 1 1 1 1 1 0 0 5 ones in a row
So what? You can add a 1 to a sequence of ones From a set of a few (a manageable number) of primitive operations If a number is represented by a sequence of ones then you have the x+1 operator, i.e addition This is a/(the most) basic mathematical operator Powerful enough to represent complex mathematics You can derive other (all) mathematic operations and objects
Powerful “language” From a set of a few (a manageable number) of primitive operations Easier enough To formulate Proofs about algorithms Powerful enough To represent complex mathematics All the ingredients are there Only have to deal with a few basic operations Not worried about complexity…. Worried about, for example, “can” we compute?
-Dept. of CSIE, CCU, Taiwan- 33 The Input String ............     Blank symbol head a b ca Head starts at the leftmost position of the input string Input string
-Dept. of CSIE, CCU, Taiwan- 34 States & Transitions 1q 2qLba , Read Write Move Left 1q 2qRba , Move Right
-Dept. of CSIE, CCU, Taiwan- 35 Turing machine for the language }{ nn ba 0q 1q 2q3q Rxa , Raa , Ryy , Lyb , Laa , Lyy , Rxx , Ryy , Ryy , 4q L,
-Dept. of CSIE, CCU, Taiwan- 36 0q 1q 2q3q Rxa , Raa , Ryy , Lyb , Laa , Lyy , Rxx , Ryy , Ryy , 4q L,  ba 0q a bTime 0 
-Dept. of CSIE, CCU, Taiwan- 37 0q 1q 2q3q Rxa , Raa , Ryy , Lyb , Laa , Lyy , Rxx , Ryy , Ryy , 4q L,  bx 1q a b Time 1
-Dept. of CSIE, CCU, Taiwan- 38 0q 1q 2q3q Rxa , Raa , Ryy , Lyb , Laa , Lyy , Rxx , Ryy , Ryy , 4q L,  x 2q a b Time 2 b
-Dept. of CSIE, CCU, Taiwan- 39 0q 1q 2q3q Rxa , Raa , Ryy , Lyb , Laa , Lyy , Rxx , Ryy , Ryy , 4q L,  yx 2q a b Time 3
-Dept. of CSIE, CCU, Taiwan- 40 0q 1q 2q3q Rxa , Raa , Ryy , Lyb , Laa , Lyy , Rxx , Ryy , Ryy , 4q L,  yx 2q a b Time 4
-Dept. of CSIE, CCU, Taiwan- 41 0q 1q 2q3q Rxa , Raa , Ryy , Lyb , Laa , Lyy , Rxx , Ryy , Ryy , 4q L,  yx 0q a b Time 5
-Dept. of CSIE, CCU, Taiwan- 42 0q 1q 2q3q Rxa , Raa , Ryy , Lyb , Laa , Lyy , Rxx , Ryy , Ryy , 4q L,  yx 1q x b Time 6
-Dept. of CSIE, CCU, Taiwan- 43 0q 1q 2q3q Rxa , Raa , Ryy , Lyb , Laa , Lyy , Rxx , Ryy , Ryy , 4q L,  yx 1q x b Time 7
-Dept. of CSIE, CCU, Taiwan- 44 0q 1q 2q3q Rxa , Raa , Ryy , Lyb , Laa , Lyy , Rxx , Ryy , Ryy , 4q L,  yx x y 2q Time 8
-Dept. of CSIE, CCU, Taiwan- 45 0q 1q 2q3q Rxa , Raa , Ryy , Lyb , Laa , Lyy , Rxx , Ryy , Ryy , 4q L,  yx x y 2q Time 9
-Dept. of CSIE, CCU, Taiwan- 46 0q 1q 2q3q Rxa , Raa , Ryy , Lyb , Laa , Lyy , Rxx , Ryy , Ryy , 4q L,  yx 0q x y Time 10
-Dept. of CSIE, CCU, Taiwan- 47 0q 1q 2q3q Rxa , Raa , Ryy , Lyb , Laa , Lyy , Rxx , Ryy , Ryy , 4q L,  yx 3q x y Time 11
-Dept. of CSIE, CCU, Taiwan- 48 0q 1q 2q3q Rxa , Raa , Ryy , Lyb , Laa , Lyy , Rxx , Ryy , Ryy , 4q L,  yx 3q x y Time 12
-Dept. of CSIE, CCU, Taiwan- 49 0q 1q 2q3q Rxa , Raa , Ryy , Lyb , Laa , Lyy , Rxx , Ryy , Ryy , 4q L,  yx 4q x y  Halt & Accept Time 13
What is Computable? Computation is usually modelled as a mapping from inputs to outputs, carried out by a formal “machine,” or program, which processes its input in a sequence of steps. An “effectively computable” function is one that can be computed in a finite amount of time using finite resources. ………… ………… … input yes no output Effectively computable function
Uncomputable Using Turing's concept of the abstract machine, we would say that a function is noncomputable if there exists no Turing machine that could compute it.
Computability Function f is computable if some program P computes it: For any input x, the computation P(x) halts with output f(x) Finite number of steps
Algorithm for 𝚷 • To calculate the area of a circle, • we need the value of 𝚷 • Can we find an algorithm to compute 𝚷?
𝚷 • 3.14159265358979323846264338327950288419716939937510 • 58209749445923078164062862089986280348253421170679 • 82148086513282306647093844609550582231725359408128 • 48111745028410270193852110555964462294895493038196 • 44288109756659334461284756482337867831652712019091 • 45648566923460348610454326648213393607260249141273 • 72458700660631558817488152092096282925409171536436 • 78925903600113305305488204665213841469519415116094 • 33057270365759591953092186117381932611793105118548 • 07446237996274956735188575272489122793818301194912 • 98336733624406566430860213949463952247371907021798 • 60943702770539217176293176752384674818467669405132 • 00056812714526356082778577134275778960917363717872 • 14684409012249534301465495853710507922796892589235 • 42019956112129021960864034418159813629774771309960 • 51870721134999999837297804995105973173281609631859 • 50244594553469083026425223082533446850352619311881 • 71010003137838752886587533208381420617177669147303 • 59825349042875546873115956286388235378759375195778 • 18577805321712268066130019278766111959092164201989……. The first 1000 digits of 𝚷 Goes on as long as you want… Π 10,000 digits Π 100,000 digits
Algorithm for 𝚷 An algorithm (in general) is a finite sequence of step by step, discrete, unambiguous instructions for solving a particular problem Problem 𝚷 is a infinite sequence of numbers… cannot calculate with a finite number of steps But there are algorithms????
Calculating 𝚷 • We can’t – Calculate the EXACT value of 𝚷 • However, we can – Calculate 𝚷 to whatever accuracy we desire – Finite number of digits, finite number of steps
Probabilistic Algorithms Calculate pi with a dart board Area of square d2 Area of Circle: P d 2 æ è ç ö ø ÷ 2 prob = circle square = p d 2 æ è ç ö ø ÷ 2 d2 = p 4 Probability dart will be in circle d number darts in circle divided by number of darts in total times Is π Monte Carlo Method Always Gives an answer But not necessarily Correct The probability of correctness goes up with time
Calculate All do not stop (halt)….. They can be an algorithm if only an approximation is needed
Halting function • Decide whether program halts on input – Given program P and input x to P, Halt (P,x) = Fact: There is no algorithm for Halt yes if P(x) halts no otherwise Clarifications Assume program P requires one string input x Write P(x) for output of P when run in input x Program P is string input to Halt
Unsolvability of the halting problem • Suppose P solves variant of halting problem – On input Q, assume P(Q) = • Build program D – D(Q) = • Does this make sense? What can D(D) do? – If D(D) halts, then D(D) runs forever. – If D(D) runs forever, then D(D) halts. – CONTRADICTION: program P must not exist. yes if Q(Q) halts no otherwise run forever if Q(Q) halts halt if Q(Q) runs forever
Lambda Calculus Alonzo Church (1903-1995) Another formulation of computability
Equivalent Computers z z zz z z z 1 Start HALT ), X, L 2: look for ( #, 1, - ), #, R (, #, L (, X, R #, 0, - Finite State Machine ... Turing Machine  term = variable | term term | (term) |  variable . term y. M  v. (M [y v]) where v does not occur in M. (x. M)N   M [ x N ]   Lambda Calculus
What is Calculus? • In High School: d/dx xn = nxn-1 [Power Rule] d/dx (f + g) = d/dx f + d/dx g [Sum Rule] Calculus is a branch of mathematics that deals with limits and the differentiation and integration of functions of one or more variables...
Real Definition • A calculus is just a bunch of rules for manipulating symbols. • People can give meaning to those symbols, but that’s not part of the calculus. • Differential calculus is a bunch of rules for manipulating symbols. There is an interpretation of those symbols corresponds with physics, slopes, etc.
-calculus Alonzo Church, 1940 term = variable | term term | (term) |  variable . term
-calculus  variable . term F(variable) = term
Why? • Once we have precise and formal rules for manipulating symbols, we can use it to reason with. • Since we can interpret the symbols as representing computations, we can use it to reason about programs.
Universal Computer • Lambda Calculus can simulate a Turing Machine – Everything a Turing Machine can compute, Lambda Calculus can compute also • Turing Machine can simulate Lambda Calculus – Everything Lambda Calculus can compute, a Turing Machine can compute also • Church-Turing Thesis: this is true for any other mechanical computer also
Normal Steps • Turing machine: – Read one square on tape, follow one FSM transition rule, write one square on tape, move tape head one square • Lambda calculus: – One beta reduction • Your PC: – Execute one instruction (?) • What one instruction does varies
-Reduction (the source of all computation) (x. M)N This is the function definition M is an expression with the variable x N is used at the value of x N substitutes x in the expression M
(x. x)a = a -Reduction (the source of all computation) Identity operation Function gives the argument as output (x. xx)a = aa (y.(x. xy)a)b =(y.ay)b=ab (y.(x. xy) z.zy)b =(y.(z.zy)y)b = (y. yy)b =bb
-calculus Useful and expression enough that a programming language was modeled after it LISP was developed from -calculus not the other way round.) This was the “first” general language of Artificial intelligence Also The “first” functional programming language
Complexity Algorithm: An algorithm is a finite sequence of step by step, discrete, unambiguous instructions for solving a particular problem Computable: Does an algorithm exist?? Complexity: The algorithm exists How “complex” is it? How many “descrete steps” does it have? how much does each step cost? How does complexity depend on size of input?
Vector/Array of size n Efficient for the nth element A single arithmetic calculation Complexity does not increase as the vector gets bigger O(1) 76 8 9 10 11 120 1 2 3 4 5 Pos(v[0]) Pos(v[0]) + 5 Find ith element of vector This is exactly an example for what a vector is designed for
Cost/Complexity Compute: Pos(v[0]) + n Find the nth element Cost: C1: Cost of determining the address of the first element Pos(v[0]) Does not depend on the size of the array of numbers C2: Cost of adding n to this address… cost of addition operation Does not depend on the size of the array of numbers Total cost: C1 + C2
Cost/Complexity Compute: Pos(v[0]) + nFind the nth element Cost: Does not depend on the size of the array of numbers Total cost = T = C1 + C2 N: size of array T:TotalCost Cost(n) = T a constant amount In the O notation O(1) The constant factor is not so important It is only important that there in no dependence on the size of the array
Vector/Array of size n Insert before ith element in vector 76 8 9 10 11 120 1 2 3 4 5 1. Allocate vector of size n+1 2. Copy element 0 to i-1 to places 0 to i-1 and copy elements i to n-1 to places i+1 to n 76 8 9 10 11 120 1 2 3 4 5 1 operation n operations 3. Set in element 76 8 9 10 11 120 1 2 3 4 5 13 1 operation Vectors are not designed to be used with insertion operations As the vector gets larger, the insertion takes more time/operations O(n)
Cost of insertion • C1: Allocate array size n+1 • C2: Copy I elements from old to new array • C2=(i)*c (c: the cost to copy an element) • C3: Copy the element to insert • C3= c (c: the cost to copy an element) • C4: Copy n-I elements from old to new array • C4=(n-i)*c Insert a new element at position I in array of length n Total Cost = T = C1 + C2 + C3 + C4 = C1 + i*c + c + (n-i)*c = C1 + (n+1)*c Total Cost = T = C1 + (n+1)*c
Cost of insertion Insert a new element at position I in array of length n Total Cost = T(n) = C1 + (n+1)*c Cost: Depends on the size of the array of numbers N: size of array T:TotalCost Cost(n) = C1 + (n+1)*c In the O notation O(n) The constant factor is not so important It is only important that there in no dependence on the size of the array
Complexity O(1) Time/operation complexity does not increase with the size of the problem O(n) Time/operation complexity does increases linearly with the size of the problem Find ith element of vector Insert before ith element in vector N: size of array T:TotalCost N: size of array T:TotalCost
Complexity of Bubble Sort
Bubble Sort: Idea • Idea: bubble in water. – Bubble in water moves upward. Why? • How? – When a bubble moves upward, the water from above will move downward to fill in the space left by the bubble.
Bubble Sort Example 9, 6, 2, 12, 11, 9, 3, 7 6, 9, 2, 12, 11, 9, 3, 7 6, 2, 9, 12, 11, 9, 3, 7 6, 2, 9, 12, 11, 9, 3, 7 6, 2, 9, 11, 12, 9, 3, 7 6, 2, 9, 11, 9, 12, 3, 7 6, 2, 9, 11, 9, 3, 12, 7 6, 2, 9, 11, 9, 3, 7, 12The 12 is greater than the 7 so they are exchanged. The 12 is greater than the 3 so they are exchanged. The twelve is greater than the 9 so they are exchanged The 12 is larger than the 11 so they are exchanged. In the third comparison, the 9 is not larger than the 12 so no exchange is made. We move on to compare the next pair without any change to the list. Now the next pair of numbers are compared. Again the 9 is the larger and so this pair is also exchanged. Bubblesort compares the numbers in pairs from left to right exchanging when necessary. Here the first number is compared to the second and as it is larger they are exchanged. The end of the list has been reached so this is the end of the first pass. The twelve at the end of the list must be largest number in the list and so is now in the correct position. We now start a new pass from left to right.
Bubble Sort Example 6, 2, 9, 11, 9, 3, 7, 122, 6, 9, 11, 9, 3, 7, 122, 6, 9, 9, 11, 3, 7, 122, 6, 9, 9, 3, 11, 7, 122, 6, 9, 9, 3, 7, 11, 12 6, 2, 9, 11, 9, 3, 7, 12 Notice that this time we do not have to compare the last two numbers as we know the 12 is in position. This pass therefore only requires 6 comparisons. First Pass Second Pass
Bubble Sort Example 2, 6, 9, 9, 3, 7, 11, 122, 6, 9, 3, 9, 7, 11, 122, 6, 9, 3, 7, 9, 11, 12 6, 2, 9, 11, 9, 3, 7, 12 2, 6, 9, 9, 3, 7, 11, 12 Second Pass First Pass Third Pass This time the 11 and 12 are in position. This pass therefore only requires 5 comparisons.
Bubble Sort Example 2, 6, 9, 3, 7, 9, 11, 122, 6, 3, 9, 7, 9, 11, 122, 6, 3, 7, 9, 9, 11, 12 6, 2, 9, 11, 9, 3, 7, 12 2, 6, 9, 9, 3, 7, 11, 12 Second Pass First Pass Third Pass Each pass requires fewer comparisons. This time only 4 are needed. 2, 6, 9, 3, 7, 9, 11, 12Fourth Pass
Bubble Sort Example 2, 6, 3, 7, 9, 9, 11, 122, 3, 6, 7, 9, 9, 11, 12 6, 2, 9, 11, 9, 3, 7, 12 2, 6, 9, 9, 3, 7, 11, 12 Second Pass First Pass Third Pass The list is now sorted but the algorithm does not know this until it completes a pass with no exchanges. 2, 6, 9, 3, 7, 9, 11, 12Fourth Pass 2, 6, 3, 7, 9, 9, 11, 12Fifth Pass
Bubble Sort Example 2, 3, 6, 7, 9, 9, 11, 12 6, 2, 9, 11, 9, 3, 7, 12 2, 6, 9, 9, 3, 7, 11, 12 Second Pass First Pass Third Pass 2, 6, 9, 3, 7, 9, 11, 12Fourth Pass 2, 6, 3, 7, 9, 9, 11, 12Fifth Pass Sixth Pass 2, 3, 6, 7, 9, 9, 11, 12 This pass no exchanges are made so the algorithm knows the list is sorted. It can therefore save time by not doing the final pass. With other lists this check could save much more work.
Bubble Sort: Example • Notice that at least one element will be in the correct position each iteration. 40 2 1 43 3 65 0 -1 58 3 42 4 652 1 40 3 43 0 -1 58 3 42 4 65581 2 3 40 0 -1 43 3 42 4 1 2 3 400 65-1 43 583 42 4 1 2 3 4
1 0 -1 32 653 43 5842404 Bubble Sort: Example 0 -1 1 2 653 43 58424043 -1 0 1 2 653 43 58424043 6 7 8 1 2 0 3-1 3 40 6543 584245
Bubble Sort Complexity 1 2 3 4 5 n-1 comparisons Total Cost= (n-1) * c Even the lowest cost is a function of n Cost = (n-1)*c = -c + n*c O(n) Remember: not so interested in the constant factors N: size of array T:TotalCost
Bubble Sort Complexity 5 4 3 2 1 Worse case: exactly opposite 4 5 3 2 1 Compare (c ) and swap (s) 4 3 5 2 1 Compare (c ) and swap (s) 4 3 2 5 1 Compare (c ) and swap (s) 4 3 2 1 5 Compare (c ) and swap (s) 3 2 1 4 5 3 4 2 1 5 Compare (c ) and swap (s) 3 2 4 1 5 Compare (c ) and swap (s) Compare (c ) and swap (s) 1 2 3 4 5 Compare (c ) and swap (s) Move 5 to place 4* (c+s) Move 5 to place 3* (c+s) Move 5 to place 1* (c+s) 2 3 1 4 5 Compare (c ) and swap (s) 2 1 3 4 5 Compare (c ) and swap (s) Move 5 to place 2* (c+s)
Bubble Sort Complexity Total Cost: 5*(c+s) + 4*(c+s) + 3*(c+s) + 2*(c+s) 1*(c+s) (5+4+3+2+1)*(c+s) 15*(c+s) 3*5*(c+s) In general the sum of numbers from 1 to n is: If n odd: n*(n+1)/2 If n even n*n/2
Bubble Sort Complexity Size of array to sort Cost: In general the sum of numbers from 1 to n is: If n odd: n*(n+1)/2= n2/2 + n/2 If n even n*n/2 = n2/2 Approx: n2/2 For a large n…. n2 is much much larger than n
Complexities Size of object for algorithm Cost: O(n) O(1) O(n2)
Tractability • Some problems are intractable: as they grow large, we are unable to solve them in reasonable time • What constitutes reasonable time? – Standard working definition: polynomial time – On an input of size n the worst-case running time is O(nk) for some constant k – O(n2), O(n3), O(1), O(n lg n), O(2n), O(nn), O(n!) – Polynomial time: O(n2), O(n3), O(1), O(n lg n) – Not in polynomial time: O(2n), O(nn), O(n!)
Polynomial-Time Algorithms • Are some problems solvable in polynomial time? – Of course: many algorithms we’ve studied provide polynomial-time solutions to some problems • Are all problems solvable in polynomial time? – No: Turing’s “Halting Problem” is not solvable by any computer, no matter how much time is given • Most problems that do not yield polynomial-time algorithms are either optimization or decision problems.
Optimization/Decision Problems • An optimization problem tries to find an optimal solution • A decision problem tries to answer a yes/no question • Many problems will have decision and optimization versions – Eg: Traveling salesman problem • optimization: find hamiltonian cycle of minimum weight • decision: is there a hamiltonian cycle of weight  k • Some problems are decidable, but intractable: as they grow large, we are unable to solve them in reasonable time – Is there a polynomial-time algorithm that solves the problem?
The Class P P: the class of decision problems that have polynomial-time deterministic algorithms. – That is, they are solvable in O(p(n)), where p(n) is a polynomial on n – A deterministic algorithm is (essentially) one that always computes the correct answer Why polynomial? – if not, very inefficient – nice closure properties • the sum and composition of two polynomials are always polynomials too
Sample Problems in P • Fractional Knapsack • MST • Sorting • Others?