Church's Thesis

(Church-Turing Thesis)
An algorithm is a formal procedure that halts.

The Thesis: Anything that can be computed by any

algorithm can be computed by a Turing machine.

Another way to state it: All "reasonable" formal

models of computation are equivalent to the Turing
machine.

This isn't a formal statement, so we can't prove it.

But many different computational models have been
proposed and they all turn out to be equivalent.

Examples:
 unrestricted grammars
 lambda calculus
 cellular automata
 DNA computing
 quantum computing (?)
The Unsolvability of the Halting Problem
Suppose we could implement

HALTS(M,x)
M: string representing a Turing Machine
x: string representing the input for M
If M(x) halts then True
else False

Then we could define

TROUBLE(x)
x: string
If HALTS(x,x) then loop forever
else halt

So now what happens if we invoke

TROUBLE(“TROUBLE”), which invokes
HALTS(“TROUBLE”,“TROUBLE”)

If HALTS says that TROUBLE halts on itself then

TROUBLE loops. IF HALTS says that TROUBLE
loops, then TROUBLE halts.
Contradiction. Therefore HALTS(M,w) cannot be a
decision procedure (recursive).
 Another View
The Problem View: The halting problem is
undecidable.

The Language View: Let H =

{"M" "w" : TM M halts on input string w}
H is recursively enumerable but not recursive.

Why?

H is recursively enumerable because it can be

semidecided by U.

But H cannot be recursive. If it were, then it would

be decided by some TM MH. But MH("M" "w")
would have to be:
If M is not a syntactically valid TM, then False.
Else HALTS("M" "w")

But we know cannot that HALTS cannot exist.

 If H were Recursive
H = {"M" "w" : TM M halts on input string w}

Theorem: If H were also recursive, then every

recursively enumerable language would be recursive.

Proof: Let L be any RE language. Since L is RE,

there exists a TM M that semidecides it.

Suppose H is recursive and thus is decided by some

TM O (oracle).

We can build a TM M' from M that decides L:

1. M' transforms its input tape from w to
"M""w".
2. M' invokes O on its tape and returns whatever
answer O returns.

So, if H were recursive, all RE languages would be.

But it isn't.
 Undecidable Problems,
Languages that Are Not Recursive, and
Partial Functions
The Problem View: The halting problem is
undecidable.

The Language View: Let H =

{"M" "w" : TM M halts on input string w}
H is recursively enumerable but not recursive.

The Functional View: Let f (w) = M(w)

f is a partial function on *

"M""w" pairs
 Other Undecidable Problems About
Turing Machines

 Given a Turing machine M, does M halt on the

empty tape?

 Given a Turing machine M, is there any string on

which M halts?

 Given a Turing machine M, does M halt on every

input string?

 Given two Turing machines M1 and M2, do they halt

on the same input strings?

 Given a Turing machine M, is the language that M

semidecides regular? Is it context-free? Is it
recursive?
 Post Correspondence Problem

Consider two lists of strings over some alphabet .

The lists must be finite and of equal length.

A = x1, x2, x3, …, xn

B = y1, y2, y3, …, yn

Question: Does there exist some finite sequence of

integers that can be viewed as indexes of A and B
such that, when elements of A are selected as
specified and concatenated together, we get the same
string we get when elements of B are selected also as
specified?

For example, if we assert that 1, 3, 4 is such a

sequence, we’re asserting that x1x3x4 = y1y3y4

Any problem of this form is an instance of the Post

Correspondence Problem.

Is the Post Correspondence Problem decidable?

Post Correspondence Problem Examples
i A B
1 1 111
2 10111 10
3 10 0

i A B
1 10 101
2 011 11
3 101 011
Some Languages Aren't Even Recursively
Enumerable
A pragmatically non RE language:

L1={ (i, j) : i, j are integers where the low order five

digits of i are a street address number and j is the
number of houses with that number on which it
rained on November 13, 1946 }

An analytically non RE language:

L2={x : x = "M" of a Turing machine M and M("M")

does not halt}

Why isn't L2 RE? Suppose it were. Then there would

be a TM M* that semidecides L2. Is "M*" in L2?
 If it is, then M*("M*") halts (by the definition of
M* as a semideciding machine for L2)
 But, by the definition of L2, if "M*"  L2, then
M*("M*") does not halt.
Contradiction.
So L2 is not RE.
 Another Non RE Language
H

Why not?
 Reduction

Let L1, L2  * be languages. A reduction from L1 to

L2 is a recursive function : *  * such that
x  L1 iff (x)  L2.

Example:

L1 = {a, b : a,b  N : b = a + 1}

  = Succ

 a, b becomes Succ(a), b

L2 = {a, b : a,b  N : a = b}

If there is a Turing machine M2 to decide L2, then I

can build a Turing machine M1 to decide L1:

1. Take the input and apply Succ to the first number.

2. Invoke M2 on the result.
3. Return whatever answer M2 returns.
 Reductions and Recursive Languages
Theorem: If there is a reduction from L1 to L2 and L2
is recursive, then L1 is recursive.

x
M1 x  L1?

y = M2 yes yes
 y  L2? no
(x) no

Theorem: If there is a reduction from L1 to L2 and L1

is not recursive, then L2 is not recursive.
 Reductions and RE Languages
Theorem: If there is a reduction from L1 to L2 and L2
is RE, then L1 is RE.
x
M1 x  L1?

y = M2 halt halt
 y  L2?
(x)

Theorem: If there is a reduction from L1 to L2 and L1

is not RE, then L2 is not RE.
 Is it Decidable Whether
M Halts on the Empty Tape?
This is equivalent to, "Is the language L2 =
{"M" : Turing machine M halts on the empty tape}
recursive?"

L1 = H = {s = "M" "w" : Turing machine M

halts on input string w}

 

(?M2) L2 = {s = "M" : Turing machine M halts

on the empty tape}

Let  be the function that, from "M" and "w",

constructs "M*", which operates as follows on an
empty input tape:
1. Write w on the tape.
2. Operate as M would have.

If M2 exists, then M1 = M2(M(s)) decides L1.

 A Formal Reduction Proof

Prove that L2 = {M: Turing machine M halts on the

empty tape} is not recursive.

Proof that L2 is not recursive via a reduction from H =

{M, w: Turing machine M halts on input string w},
a non-recursive language. Suppose that there exists a
TM, M2 that decides L2. Construct a machine to
decide H as M1(M, w) = M2((M, w)). The 
function creates from M and w a new machine
M*. M* ignores its input and runs M on w, halting
exactly when M halts on w.

 M, w  H  M halts on w  M* always halts 

  L(M*)  M*  L2  M2 accepts  M1
accepts.

 M, w  H  M does not halt on w    L(M*)

 M*  L2  M2 rejects  M1 rejects.

Thus, if there is a machine M2 that decides L2, we

could use it to build a machine that decides H.
Contradiction. L2 is not recursive.
Important Elements in a Reduction Proof

 A clear declaration of the reduction “from” and

“to” languages and what you’re trying to prove
with the reduction.
 A description of how a machine is being
constructed for the “from” language based on an
assumed machine for the “to” language and a
recursive  function.
 A description of the  function’s inputs and
outputs. If  is doing anything nontrivial, it is a
good idea to argue that it is recursive.
 Note that machine diagrams are not necessary or
even sufficient in these proofs. Use them as
thought devices, where needed.
 Run through the logic that demonstrates how the
“from” language is being decided by your
reduction. You must do both accepting and
rejecting cases.
 Declare that the reduction proves that your “to”
language is not recursive.
 The Most Common Mistake:
Doing the Reduction Backwards
The right way to use reduction to show that L2 is not
recursive:
[Link] that L1 is not recursive, L1
[Link] L1 to L2, i.e. show how to solve L1 (the
known one) in terms of L2 (the unknown one) L2

Example: If there exists a machine M2 that solves L2, the

problem of deciding whether a Turing machine halts on a
blank tape, then we could do H (deciding whether M halts
on w) as follows:
[Link] M* from M such that M*, given a blank tape,
first writes w on its tape, then simulates the behavior of
M.
[Link] M2("M*").

Doing it wrong by reducing L2 (the unknown one to L1):

If there exists a machine M1 that solves H, then we could
build a machine that solves L2 as follows:
[Link] (M1("M", "")).
 Why Backwards Doesn't Work
Suppose that we have proved that the following
problem L1 is unsolvable:

Determine the number of days that have elapsed since

the beginning of the universe.

Now consider the following problem L2:

Determine the number of days that had elapsed

between the beginning of the universe and the
assassination of Abraham Lincoln.

Reduce L1 to L2: L1
L1 = L2 + (now - 4/9/1865)
L2

Reduce L2 to L1: L2
L2 = L1 - (now - 4/9/1865)
L1
Why Backwards Doesn't Work, Continued
L1 = days since beginning of universe
L2 = elapsed days between the beginning of the
universe and the assassination of Abraham Lincoln.
L3 = days between the assassination of Abraham
Lincoln and now.

Considering L2: L1
Reduce L1 to L2:
L1 = L2 + (now - 4/9/1865) L2

Reduce L2 to L1: L2
L2 = L1 - (now - 4/9/1865)
L1

Considering L3: L1
Reduce L1 to L3:
L1 = oops L3

Reduce L3 to L1: L3
L3 = L1 - 365 - (now - 4/9/1866)
L1
 Is There Any String on Which M Halts?
L1 = H = {s = "M" "w" : Turing machine M
halts on input string w}

 

(?M2) L2 = {s = "M" : there exists a string on

which Turing machine M
halts}

Let  be the function that, from "M" and "w",

constructs "M*", which operates as follows:
1. M* examines its input tape.
2. If it is equal to w, then it simulates M.
3. If not, it loops.

Clearly the only input on which M* has a chance of

halting is w, which it does iff M would halt on w.

If M2 exists, then M1 = M2(M(s)) decides L1.

 Does M Halt on All Inputs?
L1 = {s = "M" : Turing machine M
halts on the empty tape}

 

(?M2) L2 = {s = "M" : Turing machine M

halts on all inputs}

Let  be the function that, from "M", constructs "M*",

which operates as follows:
1. Erase the input tape.
2. Simulate M.

Clearly M* either halts on all inputs or on none, since

it ignores its input.

If M2 exists, then M1 = M2(M(s)) decides L1.

L = { M: M is a TM and |L(M)|  2 }
 Rice's Theorem
Theorem: No nontrivial property of the recursively
enumerable languages is decidable.

Alternate statement: Let P: 2*{true, false} be a

nontrivial property of the recursively enumerable
languages. The language {“M”: P(L(M)) = True} is
not recursive.

By "nontrivial" we mean a property that is not simply

T for all languages or F for all languages.

Examples:
 L contains only even length strings.
 L contains an odd number of strings.
 L contains all strings that start with "a"
 L is infinite
 L is regular
Note:
Rice's theorem applies to languages, not machines.
So, for example, the following properties of machines
are decidable:
 M contains an even number of states
 M has an odd number of symbols in its tape
alphabet
Of course, we need a way to define a language. We'll
use machines to do that, but the properties we'll deal
with are properties of L(M), not of M itself.
 Proof of Rice's Theorem
Proof: Let P be any nontrivial property of the RE
languages.
L1 = H = {s = "M" "w" : Turing machine M
halts on input string w}

 

(?M2) L2 = {s = "M" : P(L(M)) = true}

Either P() = true or P() = false. Assume it is false

(a matching proof exists if it is true). Since P is
nontrivial, there is some language LP such that P(LP)
is true. Let MP be any Turing machine that
semidecides LP.

Let  construct "M*", which operates as follows:

1. Copy its input y to another track for later.
2. Write w on its input tape and execute M on w.
3. If M halts, put y back on the tape and execute MP.
4. If MP halts on y, accept.

Claim: If M2 exists, then M1 = M2(M(s)) decides L1.

 Why?

Let  construct "M*", which operates as follows:

1. Copy its input y to another track for later.
2. Write w on its input tape and execute M on w.
3. If M halts, put y back on the tape and execute MP.
4. If MP halts on y, accept.

Two cases to consider:

 "M" "w"  H  M halts on w  M* will halt on
all strings that are accepted by MP  L(M*) =
L(MP) = LP  P(L(M*)) = P(LP) = true  M2
decides P, so M2 accepts "M*"  M1 accepts.

 "M" "w"  H  M doesn’t halt on w  M* will

halt on nothing  L(M*) =   P(L(M*)) = P()
= false  M2 decides P, so M2 rejects "M*"  M1
rejects.
 Using Rice’s Theorem
Theorem: No nontrivial property of the recursively
enumerable languages is decidable.

To use Rice’s Theorem to show that a language L is

not recursive we must:

 Specify a language property, P(L)

 Show that the domain of P is the set of recursively

enumerable languages.

 Show that P is nontrivial:

 P is true of at least one language
 P is false of at least one language
 Using Rice’s Theorem: An Example
L = {s = "M" : there exists a string on which Turing
machine M halts}.
= {s = "M" : L(M)   }

 Specify a language property, P(L)

P(L) = true iff L  

 Show that the domain of P is the set of recursively

enumerable languages.
The domain of P is the set of languages
semidecided by some TM. This is exactly the
set of RE languages.

 Show that P is nontrivial:

 P is true of at least one language
P({}) = True
 P is false of at least one language
P() = False
 Inappropriate Uses of Rice’s Theorem
Example 1:
L = {s = "M" : M writes a 1 within three moves}.

 Specify a language property, P(L)

P(M?) = True if M writes a 1 within three moves,
False otherwise
 Show that the domain of P is the set of recursively
enumerable languages.
??? The domain of P is the set of all TMs, not
their languages

Example 2:
L = {s = "M1" "M2": L(M1) = L(M2)}.

 Specify a language property, P(L)

P(M1?, M2?) = True if L(M1) = L(M2)
False otherwise
 Show that the domain of P is the set of recursively
enumerable languages.
??? The domain of P is RE  RE
 Given a Turing Machine M, is L(M) Regular
(or Context Free or Recursive)?

Is this problem decidable?

No, by Rice’s Theorem, since being regular (or

context free or recursive) is a nontrivial property of
the recursively enumerable languages.

We can also show this directly (via the same

technique we used to prove the more general claim
contained in Rice’s Theorem):
 Given a Turing Machine M, is L(M) Regular
(or Context Free or Recursive)?

L1 = H = {s = "M" "w" : Turing machine M

halts on input string w}
 
(?M2) L2 = {s = "M" : L(M) is regular}

Let  be the function that, from "M" and "w",

constructs "M*", whose own input is a string
t = "M*" "w*"
M*("M*" "w*") operates as follows:
1. Copy its input to another track for later.
2. Write w on its input tape and execute M on w.
3. If M halts, invoke U on "M*" "w*".
4. If U halts, halt and accept.
If M2 exists, then M2(M*(s)) decides L1 (H).

Why?
If M does not halt on w, then M* accepts  (which is
regular).
If M does halt on w, then M* accepts H (which is not
regular).
 Undecidable Problems About
Unrestricted Grammars

 Given a grammar G and a string w, is w  L(G)?

 Given a grammar G, is   L(G)?

 Given two grammars G1 and G2, is L(G1) = L(G2)?

 Given a grammar G, is L(G) = ?

 Given a Grammar G and a String w,
Is w  L(G)?
L1 = H = {s = "M" "w" : Turing machine M
halts on input string w}

 

(?M2) L2 = {s = "G" "w" : w  L(G)}

Let  be the construction that builds a grammar G for

the language L that is semidecided by M. Thus
w  L(G) iff M(w) halts.

Then
("M" "w") = "G" "w"

If M2 exists, then M1 = M2(M(s)) decides L1.

 Undecidable Problems About
Context-Free Grammars

 Given a context-free grammar G, is L(G) = *?

 Given two context-free grammars G1 and G2,

is L(G1) = L(G2)?

 Given two context-free grammars G1 and G2,

is L(G1)  L(G2) = ?

 Given a context-free grammar G, is it ambiguous?

 Given two pushdown automata M1 and M2, do they

accept precisely the same language?

 Given a pushdown automaton M, find an equivalent

pushdown automaton with as few states as possible.
 Given Two Context-Free Grammars G1
and G2, Is L(G1) = L(G2)?

L1 = {s = "G" a CFG G and L(G) = *}

 

(?M2) L2 = {s = "G1" "G2" : G1 and G2 are CFGs

and L(G1) = L(G2)}

Let  append the description of a context free

grammar G* that generates *.

Then
("G") = "G" "G*"

If M2 exists, then M1 = M2(M(s)) decides L1.

 Non-RE Languages

There are an uncountable number of non-RE

languages, but only a countably infinite number of
TM’s (hence RE languages). The class of non-RE
languages is much bigger than RE languages!

Intuition: Non-RE languages usually involve either

infinite search or knowing a TM will infinite loop to
accept a string.

{M: M is a TM that does not halt the empty tape}

{M: M is a TM and L(M) = *}
{M: M is a TM and there does not exist a string on
which M halts}

Proving Languages are not RE

 Diagonalization
 Complement RE, not recursive
 Reduction from a non-RE language
 Rice’s theorem for non-RE languages
(not covered)
 Diagonalization

L={M: M is a TM and M(M) does not halt}

is not RE

Suppose L is RE. There is a TM M* that semidecides

L. Is M* in L?
 If it is, then M*(M*) halts (by the definition of
M* as a semideciding machine for L)
 But, by the definition of L, if M*  L, then
M*(M*) does not halt.
Contradiction. So L is not RE.

(This is a very “bare-bones” diagonalization proof.)

Diagonalization can only be easily applied to a few

non-RE languages.
 Complement of an RE, but not Recursive
Language

Theorem: If L is recursively enumerable, but not

recursive then L is not recursively enumerable.

Example: H = {M, w: M does not accept w}

Consider H = {M, w: M is a TM that accepts w}:
 H is RE—it is semidecided by U, the Universal
Turing Machine.
 H is not recursive—it is equivalent to the halting
problem, which is undecidable.
From the theorem, H is not RE.
 Reductions and RE Languages
Theorem: If there is a reduction from L1 to L2 and L2
is RE, then L1 is RE.
x
M1 x  L1?

y = M2 halt halt
 y  L2?
(x)

Theorem: If there is a reduction from L1 to L2 and L1

is not RE, then L2 is not RE.
 Reduction from a known Not-RE
Language
Using a reduction from a non-RE language:

L1 = H= {M, w: Turing machine M does

not halt on input string w}

 

(?M2) L2 = {M: there does not exist a string

on which Turing machine M halts}

Let  be the function that, from M and w,

constructs M*, which operates as follows:
1. Erase the input tape (M* ignores its input).
2. Write w on the tape
3. Run M on w.
 M, w

M1
M* halt halt
 M2

M*
x w halt halt
M

M, w  H  M does not halt on w  M* does not

halt on any input  M* halts on nothing  M2
accepts (halts).

M, w  H  M halts on w  M* halts on

everything  M2 loops.

If M2 exists, then M1(M, w) = M2(M(M, w)) and

M1 semidecides L1. Contradiction. L1 is non-RE.
 L2 is not RE.
 Language
Summary
IN OUT

Semidecidable Recursively
Enumerable Enumerable
Unrest. grammar

Decision proc. Recursive Diagonalize

Lexic. enum. Reduction
Complement RE

CF grammar Context Free Pumping

PDA Closure
Closure

Regular exp Regular Pumping

FSM Closure
Closure