Universal Turing Machines
& Undecidability

CS305 — Formal Language Theory

Use arrow keys or buttons to navigate

1 / 21

The Big Picture

We've climbed the Chomsky hierarchy. Now we hit the ceiling.

The Chomsky Hierarchy of Power: Regular --> Context-Free --> Context-Sensitive --> Recursively Enumerable (DFA) (PDA) (LBA) (TM) | | | | v v v v Least More Even more MOST POWERFUL powerful powerful powerful machine possible | v But is it ALL-powerful? Can it solve EVERYTHING?

Key Question

Turing machines are the most powerful computational model we know. But are there problems that even TMs cannot solve? The shocking answer is yes — and in fact, most problems are unsolvable.

2 / 21

Encoding Turing Machines

Any TM can be written down as a finite binary string.

The Idea

A Turing machine M is defined by a finite set of components:

States: q0, q1, ..., qn
Tape alphabet: 0, 1, B
Transitions: (state, symbol) -> (state, symbol, direction)
Start, accept, reject states

All of this is finite, so we can encode it as a binary string <M>.

Click Step to walk through encoding

Analogy

Think of source code. A Python program is just a text file — a string of characters. Similarly, a Turing machine is just a string of bits. Programs are data. This is the insight that changes everything.

3 / 21

The Universal Turing Machine

One machine to simulate them all.

Watch the UTM process an input

Key Idea

The UTM U takes two inputs: the code of a TM <M> and an input string w. It then simulates M running on w. Whatever M would do, U does the same thing. It is a universal simulator.

Analogy

The UTM is an interpreter. Just like the Python interpreter reads a .py file and executes it, the UTM reads the encoding of a TM and executes it. Or think of it as an emulator — like running a Game Boy game on your PC.

4 / 21

How the UTM Works

Three-tape simulation, step by step.

Step 0 / 4

Ready to simulate M on "011"

Simulation Loop

Read current state from Tape 3
Read current symbol from Tape 1 (at head position)
Look up (state, symbol) in Tape 2 to find the transition
Write new symbol on Tape 1
Move head on Tape 1 (L or R)
Update state on Tape 3
If new state is accept/reject, halt accordingly. Otherwise, repeat.

Important Detail

The UTM uses 3 tapes for clarity, but we know any multi-tape TM can be converted to a single-tape TM. So a UTM can be a single-tape machine. It's just slower.

Key Idea

The UTM is itself a Turing machine — it has a finite number of states and a finite transition function. The "program" it runs is on its tape, not in its states.

5 / 21

UTM = Programmable Computer

The birth of the stored-program concept.

Historical Significance

Alan Turing described the UTM in 1936, over a decade before real computers existed.

The UTM is the theoretical foundation of:

Stored-program computers (von Neumann architecture)
The software/hardware distinction
Interpreters and compilers
Operating systems (they load and run arbitrary programs)

Key Idea

The UTM tells us: you only need one machine. You don't need different hardware for every task. One universal machine + different programs = all computable functions.

6 / 21

Diagonalization: The Counting Argument

There are more problems than programs. Way more.

Turing Machines are Countable

Every TM is a finite binary string. We can list all binary strings in order:

String #1: 0 String #2: 1 String #3: 00 String #4: 01 String #5: 10 String #6: 11 String #7: 000 ... Some are valid TM encodings, some are garbage. Either way: COUNTABLY many TMs. |TMs| = |N| = aleph-0

Languages are Uncountable

A language over {0,1} is any subset of {0,1}*. Each language corresponds to a function f: {0,1}* -> {yes, no}.

For each string, independently choose IN or OUT: String: 0 1 00 01 10 11 ... Lang A: Y N Y N Y N ... Lang B: N N N Y Y Y ... Lang C: Y Y Y Y N N ... ... Each row is an infinite binary sequence. By Cantor's argument: UNCOUNTABLY many languages. |Languages| = |R| = 2^(aleph-0)

Explore the Counting Argument

TMs (Countable)

20 TMs listed

Languages (Uncountable)

The Devastating Conclusion

Countably many TMs, uncountably many languages. So most languages have no TM that recognizes them. Almost all problems are unsolvable! The solvable problems are a vanishingly tiny fraction of all problems.

7 / 21

The Diagonalization Language L_d

A specific language that NO Turing machine can recognize.

Enumerate all TMs as M1, M2, M3, ... and all strings as w1, w2, w3, ...

Build a table: entry (i, j) = 1 if Mi accepts wj, and 0 otherwise.

w1

w2

w3

w4

w5

w6

Key Idea

L_d is defined by looking at the diagonal of the table and flipping every bit. String w_i is in L_d if and only if machine M_i does not accept w_i. This is the same trick Cantor used to prove the reals are uncountable!

8 / 21

L_d is Not Recursively Enumerable

No TM can even recognize L_d — not even if we allow it to loop forever on non-members.

Proof by Contradiction

Assume some TM M_j recognizes L_d.
Then M_j appears somewhere in our enumeration — it's the j-th machine.
Ask: Is w_j in L_d?
Case 1: w_j is in L_d
- By definition of L_d: M_j does NOT accept w_j
- But M_j recognizes L_d, so it should accept w_j
- Contradiction!
Case 2: w_j is NOT in L_d
- By definition of L_d: M_j DOES accept w_j
- But M_j recognizes L_d, so it should not accept w_j
- Contradiction!

If Mj recognizes L_d: wj in L_d? | +-- YES --> Mj does NOT accept wj | (by def of L_d) | But Mj recognizes L_d, | so Mj SHOULD accept wj | CONTRADICTION! | +-- NO --> Mj DOES accept wj (by def of L_d) But Mj recognizes L_d, so Mj should NOT accept wj CONTRADICTION! Both cases lead to contradiction. Therefore, no such Mj exists.

What This Means

L_d is not just undecidable — it is not even recognizable. No TM of any kind can recognize it, even one that is allowed to loop forever on strings not in the language.

9 / 21

The Halting Problem

The most famous undecidable problem in computer science.

Definition

The Halting Problem is the language:
HALT = { <M, w> | M is a TM and M halts on input w }
That is: given a program and an input, does the program eventually stop?

Can YOU Be the Halt Decider?

Decide: does this program halt or loop forever?

Why It Matters

If we could solve the Halting Problem, we could:

Build a perfect debugger that catches all infinite loops
Verify that any program terminates
Solve Goldbach's conjecture and many open math problems
Build a perfect virus scanner

But we cannot. And the proof is elegant.

Analogy

Imagine you're a fortune teller who claims to predict the future of any machine. Someone hands you a description of a machine designed specifically to do the opposite of whatever you predict. You're stuck — you can't be right no matter what you say.

10 / 21

Proof: Halting is Undecidable

A careful, step-by-step proof by contradiction.

Step 1: Assume HALT is decidable

Suppose there exists a TM H that decides HALT. That is, H(<M, w>) = accept if M halts on w, and H(<M, w>) = reject if M loops on w. H always halts with the correct answer.

<M, w> ----> +-----+ | H |----> "accept" (M halts on w) | |----> "reject" (M loops on w) +-----+ H always halts. H is never wrong. (assumption)

Step 2: Build a new machine D (the "Diagonal" machine)

Using H as a subroutine, construct D that takes input <M> (the code of a TM) and does:

D(<M>): 1. Run H(<M, <M>>) -- ask "does M halt when given its own code as input?" 2. If H says "accept": -- M halts on <M> D enters an infinite loop (D does NOT halt) 3. If H says "reject": -- M loops on <M> D halts and accepts

Key Insight

D does the opposite of what M does on its own code. If M halts on <M>, then D loops. If M loops on <M>, then D halts. This is the setup for the contradiction on the next slide.

Step 0 / 3

Click Step to build the proof

11 / 21

The Paradox: D on Its Own Code

What happens when we feed D its own encoding?

Step 3: Run D(<D>) — feed D to itself

D(<D>): 1. Run H(<D, <D>>) -- "Does D halt on <D>?" 2. If H says "accept": -- H claims D halts on <D> D loops forever -- but then D does NOT halt! H was WRONG! 3. If H says "reject": -- H claims D loops on <D> D halts and accepts -- but then D DOES halt! H was WRONG! +==========================================================+ | | | CASE A: D halts on <D> | | --> H says "accept" | | --> D enters infinite loop | | --> D does NOT halt on <D> CONTRADICTION! | | | | CASE B: D does not halt on <D> | | --> H says "reject" | | --> D halts and accepts | | --> D DOES halt on <D> CONTRADICTION! | | | | Both cases are impossible. | | Therefore our assumption that H exists must be FALSE. | | | +==========================================================+

The Conclusion

No TM H can decide the Halting Problem. It is undecidable. This is not a limitation of our current technology — it is a fundamental mathematical impossibility.

Analogy: The Barber Paradox

"A barber shaves everyone who doesn't shave themselves." Who shaves the barber? If he does, he doesn't. If he doesn't, he does. Same structure: self-reference + negation = paradox.

12 / 21

Consequences of Undecidability

What we can NEVER build, no matter how smart we get.

Impossible Programs

Perfect Debugger: No tool can detect all infinite loops in arbitrary programs
Perfect Virus Scanner: Cannot determine if arbitrary code is malicious
Perfect Optimizer: Cannot determine if two programs compute the same function
Perfect Verifier: Cannot determine if a program meets an arbitrary specification
Perfect Type Checker: Cannot determine all type errors in a sufficiently expressive language

Real-world impact: Compiler: "Will this loop terminate?" Answer: "Undecidable." Antivirus: "Is this program malicious?" Answer: "Undecidable in general." Verifier: "Does this code match the spec?" Answer: "Undecidable in general."

But We Still Write Debuggers!

We can't solve these problems in general, but we CAN solve them for specific cases. Real tools use heuristics, approximations, and restricted inputs. They work most of the time but can never be perfect.

13 / 21

Reductions

Proving new problems undecidable by "reducing" from known undecidable problems.

THE IDEA: "If you could solve Problem B, you could solve Problem A." If A is known to be undecidable, then B must be undecidable too! +-------------------+ +-------------------+ | Problem A | reduce | Problem B | | (known | -------> | (want to prove | | undecidable) | | undecidable) | +-------------------+ +-------------------+ HALT New Problem Reduction = a computable transformation: instance of A ---f---> instance of B such that: A says "yes" iff B says "yes"

Proof Pattern

Want to prove problem B is undecidable
Assume (for contradiction) B is decidable
Show how to use a decider for B to decide HALT
But HALT is undecidable — contradiction!
Therefore B is undecidable

E_TM: "Is L(M) empty?" Build M' that accepts everything iff M halts on w.

14 / 21

Reduction Example: Is L(M) Empty?

E_TM = { <M> | L(M) = empty set }. Is this decidable?

Proof: E_TM is Undecidable

Assume we have a decider R for E_TM.
Given <M, w>, build a new TM M':
M'(x): 1. Ignore x entirely 2. Run M on w 3. If M accepts w, accept x (otherwise M' loops/rejects)
Observe:
- If M accepts w: M' accepts everything, so L(M') = Sigma* (not empty)
- If M does not accept w: M' accepts nothing, so L(M') = empty
Run R(<M'>):
- R says "L(M') is empty" iff M does not accept w
- R says "L(M') is not empty" iff M accepts w
This decides whether M accepts w — that's the Halting Problem!

Reduction Flow (animated)

Step 0 / 6

Click Step to walk through the reduction proof.

15 / 21

Rice's Theorem

The nuclear weapon of undecidability results.

Rice's Theorem (1953)

Every non-trivial property of the language recognized by a Turing machine is undecidable.

What does "non-trivial" mean?

A property P of languages is non-trivial if:

At least one TM has a language satisfying P
At least one TM has a language NOT satisfying P

In other words, the property is not always-true or always-false.

What does "property of the language" mean?

It depends only on what the TM computes (its language), not how it computes it (its structure).

"Is L(M) ____?" +---------------------------+--------+ | Property | Decide | | | -able? | +---------------------------+--------+ | L(M) is empty? | NO | | L(M) is finite? | NO | | L(M) is regular? | NO | | L(M) is context-free? | NO | | L(M) = Sigma*? | NO | | L(M) contains "hello"? | NO | | L(M) = some specific L? | NO | +---------------------------+--------+ | L(M) is RE? | YES* | | (*trivial: always true!) | | +---------------------------+--------+ ANY interesting question about what a TM computes is undecidable.

Rice's Theorem Quick Checker

Is L(M) empty?

Is L(M) finite?

Does M have exactly 5 states?

Is L(M) = {0ⁿ1ⁿ}?

Does M ever move left?

Is L(M) context-free?

Analogy

Imagine you're told: "You can never determine ANY interesting fact about what a program outputs, just by looking at the program." That's Rice's Theorem. You can ask about the code's structure (how many lines? what variables?), but you can't determine anything about its behavior.

16 / 21

Rice's Theorem: Examples

All of these questions are undecidable. All of them.

Undecidable (by Rice's)

Question	Property of L(M)?	Non-trivial?
Is L(M) = empty set?	Yes	Yes
Is L(M) finite?	Yes	Yes
Is L(M) regular?	Yes	Yes
Is L(M) context-free?	Yes	Yes
Is L(M) = {0^n 1^n}?	Yes	Yes
Is L(M) = Sigma*?	Yes	Yes
Does L(M) contain ""?	Yes	Yes

NOT covered by Rice's

Rice's does NOT apply to:

Properties of the TM itself (not its language):
- "Does M have exactly 5 states?" (decidable)
- "Does M ever move left?" (decidable)
Trivial properties:
- "Is L(M) RE?" (trivially true for all TMs)
- "Is L(M) a set of strings?" (trivially true)

Common Mistake

Rice's theorem says nothing about properties of the machine. "Does M have 100 states?" is about the machine's structure, not about L(M). That IS decidable — just look at the encoding!

17 / 21

The Hierarchy of Problems

Not all unsolvable problems are equally unsolvable.

aⁿbⁿ All CFLs HALT Compl. of HALT L_d {ww} E_TM EQ_TM

Click a language to place it in the hierarchy

Key Insight

A language is decidable iff it is both RE and co-RE. The decidable languages sit at the intersection. HALT is RE (we can simulate and say yes if it halts) but not co-RE (we can't detect non-halting). L_d is neither RE nor co-RE.

18 / 21

Post's Correspondence Problem

A surprisingly simple-looking problem that is undecidable.

The Problem

Click dominoes to build a sequence. Do the top and bottom strings match?

b

ca

a

ab

ca

a

abc

c

Your sequence:

Why It Matters

PCP looks nothing like the Halting Problem
Yet it is undecidable (proven by reduction from HALT)
It is used as an intermediate step to prove many other problems undecidable
Shows undecidability isn't limited to questions about TMs — it appears in "ordinary" combinatorial problems too

Reduction chain: HALT -----> PCP -----> CFG ambiguity \----> other problems PCP is a useful "bridge" between TM problems and grammar/language problems.

Key Idea

Undecidability is everywhere. It's not just about Turing machines — it infects combinatorics, logic, algebra, and even simple-looking puzzle problems.

19 / 21

Summary & Cheat Sheet

Everything you need to know, on one slide.

Big Results

Concept	Key Point
TM Encoding	Every TM is a finite binary string
UTM	One TM simulates all others (= interpreter)
Counting	Countable TMs vs uncountable languages
L_d	Diag. language, not even RE
HALT	RE but not decidable
Reductions	Prove B undecidable via A ≤ B
Rice's Thm	All non-trivial properties of L(M) undecidable
PCP	Undecidability beyond TM questions

The Halting Proof in 4 Lines

1. Assume decider H for HALT exists
2. Build D: run H(<M,<M>>); do opposite
3. Run D(<D>): contradiction either way
4. Therefore H cannot exist

Exam Tip

When asked "is X decidable?", first check: is it a non-trivial property of L(M)? If yes, cite Rice's Theorem and you're done. Only do a full reduction proof if Rice's doesn't apply (e.g., the property involves the machine itself, or you need to show it's not RE).

20 / 21

Challenge Quiz

Test your understanding of UTMs and undecidability.

21 / 21

Universal Turing Machines& Undecidability

The Big Picture

Key Question

Encoding Turing Machines

The Idea

Analogy

The Universal Turing Machine

Key Idea

Analogy

How the UTM Works

Simulation Loop

Important Detail

Key Idea

UTM = Programmable Computer

Historical Significance

Key Idea

Diagonalization: The Counting Argument

Turing Machines are Countable

Languages are Uncountable

Explore the Counting Argument

TMs (Countable)

Languages (Uncountable)

The Devastating Conclusion

The Diagonalization Language Ld

Key Idea

Ld is Not Recursively Enumerable

Proof by Contradiction

What This Means

The Halting Problem

Definition

Can YOU Be the Halt Decider?

Why It Matters

Analogy

Proof: Halting is Undecidable

Step 1: Assume HALT is decidable

Step 2: Build a new machine D (the "Diagonal" machine)

Key Insight

The Paradox: D on Its Own Code

Step 3: Run D(<D>) — feed D to itself

The Conclusion

Analogy: The Barber Paradox

Consequences of Undecidability

Impossible Programs

But We Still Write Debuggers!

Reductions

Proof Pattern

Reduction Example: Is L(M) Empty?

Proof: ETM is Undecidable

Reduction Flow (animated)

Rice's Theorem

Rice's Theorem (1953)

What does "non-trivial" mean?

What does "property of the language" mean?

Rice's Theorem Quick Checker

Analogy

Rice's Theorem: Examples

Undecidable (by Rice's)

NOT covered by Rice's

Common Mistake

The Hierarchy of Problems

Key Insight

Post's Correspondence Problem

The Problem

Why It Matters

Key Idea

Summary & Cheat Sheet

Big Results

The Halting Proof in 4 Lines

Exam Tip

Challenge Quiz

Universal Turing Machines
& Undecidability

The Diagonalization Language L_d

L_d is Not Recursively Enumerable

Proof: E_TM is Undecidable