Theoretical Computer Science

What can a computer do at all, and what can it do efficiently? This page walks through Turing machines, the halting problem, the complexity zoo, and the most famous open question in CS — P vs NP. The answers turn out to be "less than you'd expect," in both senses.

Prereq: discrete math, algorithmic complexity Read time: ~35 min Interactive figures: 2 Code: Python

1. Why theoretical CS

Most of computer science asks "how do I build this?" Theoretical CS asks two stranger questions:

  1. Can it be built at all? Some problems have no algorithm. Not "no efficient algorithm" — no algorithm. We can prove this. The halting problem is the textbook example.
  2. If it can be built, how cheap can we make it? Some problems clearly have algorithms but every known algorithm runs in time that grows wildly with input size. We strongly suspect — but cannot prove — that no algorithm can do better.

Both answers are deflating. There exist well-defined questions that are mathematically immune to computation. And among the questions that can be computed, there's a huge class — the kind that show up in scheduling, logistics, cryptanalysis, and protein folding — for which the only known approach is, essentially, brute force. This page is about making both of those statements precise.

The two questions correspond to the two pillars of theoretical CS: computability theory (what's solvable in principle) and complexity theory (what's solvable in practice). They share a common toolkit — formal models of computation, reductions, diagonal arguments — and they share a common ancestor: a 1936 paper by a 23-year-old named Alan Turing.

Before going further, you'll want to be loosely comfortable with the language of discrete math (sets, functions, relations, proof by contradiction) and the basics of algorithmic complexity (big-O notation, the difference between $O(n)$ and $O(n^2)$). Nothing exotic. If "$\{0,1\}^*$ means the set of all finite binary strings" doesn't scare you, you're ready.

2. The Turing machine

In 1936 Turing wanted to formalize what a "computation" actually is. He picked the most stripped-down model he could imagine, then proved every reasonable model is equivalent to it. The model is:

Formally a Turing machine is a 7-tuple. Don't be alarmed by the notation; it's just a list:

$$M = (Q, \Sigma, \Gamma, \delta, q_0, q_{\text{accept}}, q_{\text{reject}})$$

Turing machine, formally

$Q$
The finite set of states. Think "boxes in a flowchart." Maybe 5 of them, maybe 5000 — but finite.
$\Sigma$
The input alphabet — the symbols allowed in the original input string. Often $\{0,1\}$.
$\Gamma$
The tape alphabet, which contains $\Sigma$ plus a special blank symbol $\sqcup$ and any extra working symbols you want.
$\delta$
The transition function: $\delta : Q \times \Gamma \to Q \times \Gamma \times \{L, R\}$. Read "given state $q$ and symbol $a$, write some symbol $b$, move left or right, and go to state $q'$."
$q_0$
The start state — where the machine begins before reading anything.
$q_{\text{accept}}$
If the machine ever lands here, it halts and says "yes."
$q_{\text{reject}}$
If it lands here, it halts and says "no."

Analogy. Imagine a person with a long roll of toilet paper, a pencil, an eraser, and a one-page checklist of rules. They look at the square in front of them, find the matching row in the checklist, do what it says (write, erase, scoot left, scoot right, jump to a new row), and repeat. That's it. Everything your laptop can do, this person can do — slower, sure, but the set of computable things is exactly the same.

The machine takes an input string written on the tape, runs $\delta$ over and over, and either reaches $q_{\text{accept}}$, reaches $q_{\text{reject}}$, or runs forever. A language is the set of all strings the machine accepts. A machine decides a language if it always halts (accept or reject) on every input. It recognizes a language if it accepts every string in the language but may loop forever on strings outside it. That little distinction will turn out to matter a lot.

A "configuration" of a Turing machine is the complete snapshot you'd need to resume it: (current state, position of the head, contents of the tape). Two TMs are equivalent if they go through the same sequence of configurations on every input.

Why this minimal model and not, say, a Python interpreter? Because Turing wanted to argue mathematically about all possible computations, and you can't reason cleanly about "all programming languages." A TM is small enough to fit on a chalkboard but expressive enough that nothing has ever been shown to outdo it.

Source — Turing Machine Simulator
-- Turing Machine that accepts {0^n 1^n | n >= 1}
-- Tape alphabet: {0, 1, X, Y, _}  (X = marked 0, Y = marked 1)

TM for {0^n 1^n}:
  States: q0, q1, q2, q3, q4, q_accept, q_reject
  Start:  q0

  q0: scan right looking for the leftmost 0
    read 0 -> write X, move R, go to q1   (mark a 0, now hunt for a 1)
    read Y -> move R, go to q3            (all 0s matched, sweep past Ys)
    read _ -> go to q_reject              (empty or no 0)
    read X -> move R, stay q0             (skip already-marked)

  q1: move right past 0s and Ys, looking for a 1
    read 0 -> move R, stay q1
    read Y -> move R, stay q1
    read 1 -> write Y, move L, go to q2   (mark matching 1, go back left)
    read _ -> go to q_reject              (ran out of 1s)

  q2: move left back to the leftmost X or start
    read 0 -> move L, stay q2
    read Y -> move L, stay q2
    read X -> move R, go to q0            (found the boundary, loop)
    read _ -> move R, go to q0

  q3: finishing sweep — only Ys and _ allowed
    read Y -> move R, stay q3
    read _ -> go to q_accept
    read anything else -> go to q_reject

  q_accept: HALT accept
  q_reject: HALT reject

run_tm(tape):
  head = 0, state = q0
  loop:
    (new_state, new_sym, dir) = delta[state][tape[head]]
    tape[head] = new_sym
    head += (dir == R) ? 1 : -1
    state = new_state
    if state in {q_accept, q_reject}: break
  return state == q_accept
def run_tm(tape, transitions, start='q0', accept='q_accept', reject='q_reject', blank='_'):
    """
    Run a Turing machine.
    tape:        list of symbols (will be modified in place)
    transitions: dict mapping (state, symbol) -> (new_state, new_symbol, direction)
                 direction is 'L' or 'R'
    Returns True if accepted, False if rejected.
    """
    tape = list(tape) + [blank] * 20
    head = 0
    state = start

    for _ in range(10_000):  # step limit to catch infinite loops
        if state == accept:
            return True
        if state == reject:
            return False
        sym = tape[head]
        key = (state, sym)
        if key not in transitions:
            return False  # implicit reject
        new_state, new_sym, direction = transitions[key]
        tape[head] = new_sym
        state = new_state
        if direction == 'R':
            head += 1
            if head >= len(tape):
                tape.append(blank)
        else:
            head = max(0, head - 1)

    return False  # ran out of steps

# TM for {0^n 1^n}: mark a 0 with X, find matching 1, mark it Y, repeat
delta_0n1n = {
    ('q0', '0'): ('q1', 'X', 'R'),
    ('q0', 'Y'): ('q3', 'Y', 'R'),
    ('q0', '_'): ('q_reject', '_', 'R'),
    ('q1', '0'): ('q1', '0', 'R'),
    ('q1', 'Y'): ('q1', 'Y', 'R'),
    ('q1', '1'): ('q2', 'Y', 'L'),
    ('q1', '_'): ('q_reject', '_', 'R'),
    ('q2', '0'): ('q2', '0', 'L'),
    ('q2', 'Y'): ('q2', 'Y', 'L'),
    ('q2', 'X'): ('q0', 'X', 'R'),
    ('q2', '_'): ('q0', '_', 'R'),
    ('q3', 'Y'): ('q3', 'Y', 'R'),
    ('q3', '_'): ('q_accept', '_', 'R'),
}

tests = [('01', True), ('0011', True), ('001', False), ('0001111', False), ('', False)]
for inp, expected in tests:
    result = run_tm(list(inp), delta_0n1n)
    status = 'OK' if result == expected else 'FAIL'
    print(f"{status}  '{inp}' -> {result}  (expected {expected})")
function runTM(tape, transitions, start = 'q0', accept = 'q_accept', reject = 'q_reject', blank = '_') {
    tape = [...tape, ...Array(20).fill(blank)];
    let head = 0;
    let state = start;

    for (let step = 0; step < 10000; step++) {
        if (state === accept) return true;
        if (state === reject) return false;
        const sym = tape[head];
        const key = `${state},${sym}`;
        if (!(key in transitions)) return false;
        const [newState, newSym, dir] = transitions[key];
        tape[head] = newSym;
        state = newState;
        if (dir === 'R') {
            head++;
            if (head >= tape.length) tape.push(blank);
        } else {
            head = Math.max(0, head - 1);
        }
    }
    return false;
}

// TM for {0^n 1^n}
const delta = {
    'q0,0': ['q1', 'X', 'R'],
    'q0,Y': ['q3', 'Y', 'R'],
    'q0,_': ['q_reject', '_', 'R'],
    'q1,0': ['q1', '0', 'R'],
    'q1,Y': ['q1', 'Y', 'R'],
    'q1,1': ['q2', 'Y', 'L'],
    'q1,_': ['q_reject', '_', 'R'],
    'q2,0': ['q2', '0', 'L'],
    'q2,Y': ['q2', 'Y', 'L'],
    'q2,X': ['q0', 'X', 'R'],
    'q2,_': ['q0', '_', 'R'],
    'q3,Y': ['q3', 'Y', 'R'],
    'q3,_': ['q_accept', '_', 'R'],
};

const tests = [['01', true], ['0011', true], ['001', false], ['0001111', false]];
for (const [inp, expected] of tests) {
    const result = runTM(inp.split(''), delta);
    console.log(`${result === expected ? 'OK' : 'FAIL'}  '${inp}' -> ${result}`);
}
#include <stdio.h>
#include <string.h>
#include <stdlib.h>

#define TAPE_SIZE 200
#define MAX_STEPS 10000

typedef struct { const char *state; char read; const char *new_state; char write; int dir; } Rule;

// dir: +1 = R, -1 = L
static Rule rules[] = {
    {"q0",'0',"q1",'X',+1}, {"q0",'Y',"q3",'Y',+1}, {"q0",'_',"qR",'_',+1},
    {"q1",'0',"q1",'0',+1}, {"q1",'Y',"q1",'Y',+1}, {"q1",'1',"q2",'Y',-1},
    {"q1",'_',"qR",'_',+1},
    {"q2",'0',"q2",'0',-1}, {"q2",'Y',"q2",'Y',-1}, {"q2",'X',"q0",'X',+1},
    {"q2",'_',"q0",'_',+1},
    {"q3",'Y',"q3",'Y',+1}, {"q3",'_',"qA",'_',+1},
};
static int nrules = sizeof(rules)/sizeof(rules[0]);

int run_tm(const char *input) {
    char tape[TAPE_SIZE];
    memset(tape, '_', TAPE_SIZE);
    int n = strlen(input);
    memcpy(tape, input, n);
    int head = 0;
    const char *state = "q0";
    for (int s = 0; s < MAX_STEPS; s++) {
        if (strcmp(state,"qA")==0) return 1;
        if (strcmp(state,"qR")==0) return 0;
        char sym = tape[head];
        int matched = 0;
        for (int i = 0; i < nrules; i++) {
            if (strcmp(rules[i].state, state)==0 && rules[i].read == sym) {
                tape[head] = rules[i].write;
                state = rules[i].new_state;
                head += rules[i].dir;
                if (head < 0) head = 0;
                matched = 1; break;
            }
        }
        if (!matched) return 0;
    }
    return 0;
}

int main(void) {
    const char *tests[] = {"01","0011","001","0001111",""};
    int expected[]       = {1,   1,     0,    0,         0};
    for (int i = 0; i < 5; i++) {
        int r = run_tm(tests[i]);
        printf("%s  '%s' -> %d\n", r==expected[i]?"OK":"FAIL", tests[i], r);
    }
    return 0;
}
#include <iostream>
#include <string>
#include <unordered_map>
#include <tuple>

using Transition = std::tuple<std::string, char, int>;  // (new_state, write, dir)

bool runTM(std::string tape,
           const std::unordered_map<std::string, Transition>& delta,
           const std::string& start = "q0",
           const std::string& accept = "qA",
           const std::string& reject = "qR") {
    tape += std::string(20, '_');
    int head = 0;
    std::string state = start;
    for (int step = 0; step < 10000; step++) {
        if (state == accept) return true;
        if (state == reject) return false;
        std::string key = state + "," + tape[head];
        auto it = delta.find(key);
        if (it == delta.end()) return false;
        auto [ns, w, dir] = it->second;
        tape[head] = w;
        state = ns;
        head = std::max(0, head + dir);
        if (head >= (int)tape.size()) tape += '_';
    }
    return false;
}

int main() {
    std::unordered_map<std::string, Transition> delta = {
        {"q0,0",{"q1",'X',+1}}, {"q0,Y",{"q3",'Y',+1}}, {"q0,_",{"qR",'_',+1}},
        {"q1,0",{"q1",'0',+1}}, {"q1,Y",{"q1",'Y',+1}}, {"q1,1",{"q2",'Y',-1}},
        {"q1,_",{"qR",'_',+1}},
        {"q2,0",{"q2",'0',-1}}, {"q2,Y",{"q2",'Y',-1}}, {"q2,X",{"q0",'X',+1}},
        {"q2,_",{"q0",'_',+1}},
        {"q3,Y",{"q3",'Y',+1}}, {"q3,_",{"qA",'_',+1}},
    };
    std::pair<std::string,bool> tests[] = {{"01",true},{"0011",true},{"001",false},{"",false}};
    for (auto& [inp, exp] : tests) {
        bool r = runTM(inp, delta);
        std::cout << (r==exp?"OK":"FAIL") << "  '" << inp << "' -> " << r << "\n";
    }
}
import java.util.*;

public class TuringMachine {
    record Rule(String newState, char write, int dir) {}

    static boolean runTM(String input, Map<String, Rule> delta,
                         String start, String accept, String reject) {
        StringBuilder tape = new StringBuilder(input);
        for (int i = 0; i < 20; i++) tape.append('_');
        int head = 0;
        String state = start;
        for (int step = 0; step < 10000; step++) {
            if (state.equals(accept)) return true;
            if (state.equals(reject)) return false;
            char sym = tape.charAt(head);
            String key = state + "," + sym;
            Rule r = delta.get(key);
            if (r == null) return false;
            tape.setCharAt(head, r.write());
            state = r.newState();
            head = Math.max(0, head + r.dir());
            if (head >= tape.length()) tape.append('_');
        }
        return false;
    }

    public static void main(String[] args) {
        Map<String, Rule> delta = new HashMap<>();
        delta.put("q0,0", new Rule("q1", 'X', +1));
        delta.put("q0,Y", new Rule("q3", 'Y', +1));
        delta.put("q0,_", new Rule("qR", '_', +1));
        delta.put("q1,0", new Rule("q1", '0', +1));
        delta.put("q1,Y", new Rule("q1", 'Y', +1));
        delta.put("q1,1", new Rule("q2", 'Y', -1));
        delta.put("q1,_", new Rule("qR", '_', +1));
        delta.put("q2,0", new Rule("q2", '0', -1));
        delta.put("q2,Y", new Rule("q2", 'Y', -1));
        delta.put("q2,X", new Rule("q0", 'X', +1));
        delta.put("q2,_", new Rule("q0", '_', +1));
        delta.put("q3,Y", new Rule("q3", 'Y', +1));
        delta.put("q3,_", new Rule("qA", '_', +1));

        String[][] tests = {{"01","true"},{"0011","true"},{"001","false"},{"","false"}};
        for (String[] t : tests) {
            boolean r = runTM(t[0], delta, "q0", "qA", "qR");
            System.out.println((r == Boolean.parseBoolean(t[1]) ? "OK" : "FAIL") +
                "  '" + t[0] + "' -> " + r);
        }
    }
}
package main

import (
	"fmt"
	"strings"
)

type Rule struct {
	newState string
	write    byte
	dir      int
}

func runTM(input string, delta map[string]Rule, start, accept, reject string) bool {
	tape := []byte(input + strings.Repeat("_", 20))
	head := 0
	state := start
	for step := 0; step < 10000; step++ {
		if state == accept { return true }
		if state == reject { return false }
		sym := tape[head]
		key := state + "," + string(sym)
		r, ok := delta[key]
		if !ok { return false }
		tape[head] = r.write
		state = r.newState
		head += r.dir
		if head < 0 { head = 0 }
		if head >= len(tape) { tape = append(tape, '_') }
	}
	return false
}

func main() {
	delta := map[string]Rule{
		"q0,0": {"q1", 'X', +1}, "q0,Y": {"q3", 'Y', +1}, "q0,_": {"qR", '_', +1},
		"q1,0": {"q1", '0', +1}, "q1,Y": {"q1", 'Y', +1}, "q1,1": {"q2", 'Y', -1},
		"q1,_": {"qR", '_', +1},
		"q2,0": {"q2", '0', -1}, "q2,Y": {"q2", 'Y', -1}, "q2,X": {"q0", 'X', +1},
		"q2,_": {"q0", '_', +1},
		"q3,Y": {"q3", 'Y', +1}, "q3,_": {"qA", '_', +1},
	}
	type tc struct { inp string; exp bool }
	tests := []tc{{"01", true}, {"0011", true}, {"001", false}, {"0001111", false}, {"", false}}
	for _, t := range tests {
		r := runTM(t.inp, delta, "q0", "qA", "qR")
		status := "OK"
		if r != t.exp { status = "FAIL" }
		fmt.Printf("%s  '%s' -> %v\n", status, t.inp, r)
	}
}

3. The Church-Turing thesis

In 1936, two answers to "what is a computation?" appeared at the same time. Turing's tape machine was one. Alonzo Church's lambda calculus — a tiny system of function abstraction and application, no tape at all — was the other. Within months, the two were shown to compute exactly the same set of functions. So were Kurt Gödel's recursive functions, Emil Post's rewriting systems, register machines, your laptop, the cellular automaton Rule 110, Conway's Game of Life, and Magic: The Gathering combo decks.

The Church-Turing thesis is the informal claim that anything you'd intuitively call a computation can be done by a Turing machine. It is not a theorem — you can't prove "intuitively" — but in the 90 years since 1936, every formal model anyone has invented either turns out to be Turing-equivalent or strictly weaker. No exceptions, despite many attempts.

What this buys us. It means we get to use the words "computable," "algorithm," and "decidable" without committing to a specific model. When we prove "no algorithm can do X," we mean "no Turing machine," and via Church-Turing that means no thing at all that we'd recognize as a computer. (Quantum computers complicate the efficiency story, but not the computability story — they compute the same set of functions, just sometimes faster.)

Heads-up on a common confusion. The thesis is about what is computable, not what is computable in reasonable time. A Turing machine can simulate your laptop, but it might take $10^{30}$ steps to do what your laptop does in a microsecond. Computability and efficiency are different floors. The next several sections are about computability; we get to efficiency around section 8.

4. Interactive: a tiny Turing machine simulator

Below is a 5-state Turing machine that, given a unary input like 1111 (the number 4), increments it by 1 to 11111. It works by walking right to find the end of the input and writing one more 1. Click Step to advance one transition, or Run to play it through. Watch the head, the tape, and the current state change.

Input: state: q0

A unary increment machine. State q0 = scan right; q1 = write a 1 and finish; q_accept = done.

The transition table is just five rules. In words: "in q0, if you see a 1, leave it alone and move right; if you see a blank, switch to q1." "In q1, write a 1, move right, switch to q_accept." The whole machine fits in your head. Yet by composing more rules, you can get a Turing machine that adds, multiplies, computes square roots, runs a Python interpreter, or simulates your phone.

Below the simulator is a second interactive figure: the complexity-class hierarchy. We'll get to that in section 8 — once we have the language to describe what it means.

5. Decidability — what computers can answer at all

Now that we have a formal model, we can sort problems into three buckets.

Two big surprises live in this taxonomy. The first is that the decidable problems are the rare ones in some precise sense: there are only countably many Turing machines (each is a finite description) but uncountably many languages over $\{0,1\}^*$, so by Cantor's diagonal argument most languages are not even recognizable. The second is that famous, natural problems live in the "not decidable" bucket. The most famous of all is the halting problem.

Counting argument

The set of Turing machines is countable: each TM is a finite string over a finite alphabet, and finite strings can be enumerated $1, 2, 3, \dots$ The set of all subsets of $\{0,1\}^*$ — that is, the set of all languages — has cardinality $2^{\aleph_0}$, strictly larger by Cantor's theorem. So the function "which language does this TM decide?" cannot possibly be onto. There must exist languages no TM decides. We don't even need to look at a specific one to know they exist.

Source — Decidability Examples
-- Two examples of DECIDABLE problems.
-- "Decidable" means: a program that ALWAYS halts and outputs yes or no.

-- (a) Palindrome checker — O(n)
is_palindrome(s):
  left = 0, right = len(s) - 1
  while left < right:
    if s[left] != s[right]: return NO
    left++, right--
  return YES

-- (b) Even-length string checker — O(n)
is_even_length(s):
  count = 0
  for each symbol in s: count++
  if count mod 2 == 0: return YES
  else: return NO

-- Both always halt in linear time. No infinite loops possible.
-- This is exactly what "decidable" means in computability theory.
def is_palindrome(s: str) -> bool:
    """
    Decidable in O(n). Always halts and answers yes/no.
    Illustrates: a Turing machine that accepts exactly the palindromes
    over some alphabet is a *decider* for that language.
    """
    left, right = 0, len(s) - 1
    while left < right:
        if s[left] != s[right]:
            return False
        left += 1
        right -= 1
    return True


def is_even_length(s: str) -> bool:
    """
    Decidable in O(n). Accepts strings whose length is even.
    The simplest nontrivial regular language — even a 2-state DFA decides it.
    """
    return len(s) % 2 == 0


# Demonstrate both
for word in ["racecar", "hello", "abba", "ab", "a", ""]:
    print(f"  '{word}'  palindrome={is_palindrome(word)}  even_len={is_even_length(word)}")

# Key point: neither function can loop forever. Every string gets a definitive
# answer in O(n) steps. That's the hallmark of a decidable language.
function isPalindrome(s) {
    // Always halts. Decides the palindrome language in O(n).
    let left = 0, right = s.length - 1;
    while (left < right) {
        if (s[left] !== s[right]) return false;
        left++; right--;
    }
    return true;
}

function isEvenLength(s) {
    // Always halts. Decides {strings of even length} in O(n).
    return s.length % 2 === 0;
}

const words = ['racecar', 'hello', 'abba', 'ab', 'a', ''];
for (const w of words) {
    console.log(`'${w}'  palindrome=${isPalindrome(w)}  evenLen=${isEvenLength(w)}`);
}
#include <stdio.h>
#include <string.h>

int is_palindrome(const char *s) {
    int left = 0, right = (int)strlen(s) - 1;
    while (left < right) {
        if (s[left] != s[right]) return 0;
        left++; right--;
    }
    return 1;
}

int is_even_length(const char *s) {
    return strlen(s) % 2 == 0;
}

int main(void) {
    const char *words[] = {"racecar", "hello", "abba", "ab", "a", ""};
    for (int i = 0; i < 6; i++) {
        printf("'%s'  palindrome=%d  even_len=%d\n",
               words[i], is_palindrome(words[i]), is_even_length(words[i]));
    }
    return 0;
}
#include <iostream>
#include <string>
#include <algorithm>

bool isPalindrome(const std::string& s) {
    int left = 0, right = (int)s.size() - 1;
    while (left < right) {
        if (s[left] != s[right]) return false;
        left++; right--;
    }
    return true;
}

bool isEvenLength(const std::string& s) {
    return s.size() % 2 == 0;
}

int main() {
    for (auto& w : {"racecar", "hello", "abba", "ab", "a", ""}) {
        std::string s(w);
        std::cout << "'" << s << "'  palindrome=" << isPalindrome(s)
                  << "  even_len=" << isEvenLength(s) << "\n";
    }
}
public class Decidability {
    static boolean isPalindrome(String s) {
        int left = 0, right = s.length() - 1;
        while (left < right) {
            if (s.charAt(left) != s.charAt(right)) return false;
            left++; right--;
        }
        return true;
    }

    static boolean isEvenLength(String s) {
        return s.length() % 2 == 0;
    }

    public static void main(String[] args) {
        String[] words = {"racecar", "hello", "abba", "ab", "a", ""};
        for (String w : words) {
            System.out.printf("'%s'  palindrome=%b  evenLen=%b%n",
                w, isPalindrome(w), isEvenLength(w));
        }
    }
}
package main

import "fmt"

func isPalindrome(s string) bool {
	runes := []rune(s)
	left, right := 0, len(runes)-1
	for left < right {
		if runes[left] != runes[right] {
			return false
		}
		left++
		right--
	}
	return true
}

func isEvenLength(s string) bool {
	return len([]rune(s))%2 == 0
}

func main() {
	words := []string{"racecar", "hello", "abba", "ab", "a", ""}
	for _, w := range words {
		fmt.Printf("'%s'  palindrome=%v  evenLen=%v\n", w, isPalindrome(w), isEvenLength(w))
	}
}

6. The halting problem

The halting problem asks: given the description of a Turing machine $M$ and an input $w$, will $M$ eventually halt on $w$, or will it run forever? Concretely: is there a single algorithm that, given any program and any input, can tell you whether the program will eventually finish?

The answer is no, and Turing proved it by contradiction in 1936. The proof is short and worth following carefully. It uses a self-reference trick — a program that is given its own source code as input. This is the trickiest paragraph on the page; read it twice if you need to.

Step 1. Suppose for contradiction that there is a TM called $H$ that decides halting. That is, $H$ takes as input the description of any machine $M$ and any input $w$, always halts, and outputs:

$$H(\langle M\rangle, w) = \begin{cases} \text{accept} & \text{if } M \text{ halts on } w \\ \text{reject} & \text{if } M \text{ loops forever on } w \end{cases}$$

The hypothetical halting decider

$H$
A Turing machine we're imagining exists. The proof will show it can't.
$M$
An arbitrary Turing machine — the one we're asking about.
$\langle M\rangle$
The "encoding" of $M$ as a string. Since $M$ is a finite object, you can write its transition table as a string of 0s and 1s and feed it as input. This is the idea that makes self-reference possible — programs can take other programs as input, including themselves.
$w$
The input we want to know whether $M$ halts on.
"accept" / "reject"
$H$'s yes-or-no output. Crucially, $H$ always reaches one of them — that's what "decides" means.

Why feed a program its own source code? Because we can. Compilers read source code. A halting checker would have to read source code. There's no rule against feeding it the source code of itself, just like there's no rule against pointing a video camera at its own monitor. The interesting question is what happens when you do.

Step 2. Now build a new machine $D$ that takes the description of a TM $M$ as input. $D$ does the following:

  1. Run $H$ on input $(\langle M\rangle, \langle M\rangle)$. (That is, ask "does $M$ halt when given itself as input?")
  2. If $H$ says "accept" — meaning $M$ does halt on its own description — then $D$ deliberately enters an infinite loop.
  3. If $H$ says "reject" — meaning $M$ loops forever on its own description — then $D$ halts immediately.

$D$ is perfectly constructible. It's just a wrapper around $H$ with a small if-else. Since $H$ is total (always halts), $D$ either halts or loops forever, depending on what $H$ said.

Step 3. Now run $D$ on its own description: ask "what does $D(\langle D\rangle)$ do?"

$$D(\langle D\rangle) \text{ halts} \iff H(\langle D\rangle, \langle D\rangle) = \text{reject} \iff D \text{ loops forever on } \langle D\rangle$$

The contradiction

$D(\langle D\rangle) \text{ halts}$
The literal claim "this run of $D$ on input $\langle D\rangle$ eventually finishes."
$\iff$
"If and only if." The two sides are logically equivalent.
$H(\langle D\rangle, \langle D\rangle) = \text{reject}$
By construction of $D$, the only way $D$ halts on input $\langle D\rangle$ is if its internal call to $H$ returned "reject."
$D \text{ loops forever on } \langle D\rangle$
By the assumed correctness of $H$, $H$ rejects $(\langle D\rangle, \langle D\rangle)$ exactly when $D$ does not halt on input $\langle D\rangle$. Which is the opposite of what we started with.

The squeeze. "$D$ halts on its own description" turns out to be equivalent to "$D$ does not halt on its own description." That is a contradiction in two short steps. The only assumption we made was that $H$ exists. So $H$ doesn't exist. The halting problem is undecidable.

If this feels like a magic trick, it's because it is one — the same magic trick Cantor used to show real numbers outnumber integers, and the same trick Gödel used to show some true statements about arithmetic can't be proved. The recipe: assume a complete-and-perfect oracle exists, build an object that asks the oracle about itself, then arrange that the object disagrees with whatever the oracle says. The oracle can't survive that.

What this does and doesn't say. It does not say "we can never tell whether any specific program halts." For most actual programs you encounter, you can tell. Compilers do termination analysis on loops every day. What it says is that no single algorithm works for every program. For any halting checker you write, there is some adversarial program designed to break it. The undecidability is about the impossibility of a uniform solution, not about your specific Friday-afternoon code review.

Source — Halting Problem (Self-Reference Paradox)
-- The diagonalization / self-reference argument.
-- We show that a hypothetical halts(f, x) oracle leads to a contradiction.

-- Step 1: Assume an oracle exists
oracle halts(f, x):
  -- Returns TRUE  if f(x) terminates
  -- Returns FALSE if f(x) runs forever
  -- This oracle ALWAYS terminates. That's its defining property.

-- Step 2: Build confuse() using the oracle
function confuse(f):
  if halts(f, f):      -- ask "does f halt when given itself as input?"
    loop forever       -- if yes, we loop
  else:
    return             -- if no, we halt

-- Step 3: Run confuse on itself: confuse(confuse)
-- Case A: confuse(confuse) halts
--   => halts(confuse, confuse) returned TRUE
--   => the if-branch ran => we looped forever => CONTRADICTION
-- Case B: confuse(confuse) loops forever
--   => halts(confuse, confuse) returned FALSE
--   => the else-branch ran => we returned => CONTRADICTION
-- Both cases are impossible. Therefore the oracle cannot exist.
# Python: the self-reference argument in runnable (but broken-by-design) form.
# We cannot actually write halts() — that's the whole point.
# But we can show the logical structure clearly.

# --- Hypothetical oracle (does NOT exist) ---
def halts(f, x) -> bool:
    """
    HYPOTHETICAL: returns True if f(x) terminates, False if it loops forever.
    No real implementation is possible for all f and x.
    If you inspect f's source with `import inspect; src = inspect.getsource(f)`
    you still cannot decide in general — this is what Turing proved.
    """
    raise NotImplementedError("This oracle cannot exist!")

# --- The confuse function that breaks the oracle ---
def confuse(f):
    """
    If halts(f, f) says 'yes, f halts on itself':  we loop forever.
    If halts(f, f) says 'no, f loops on itself':   we return immediately.
    Running confuse(confuse) produces a contradiction either way.
    """
    if halts(f, f):      # oracle says f(f) halts
        while True:      # so we deliberately loop — contradiction!
            pass
    else:                # oracle says f(f) loops
        return           # so we deliberately halt — contradiction!

# --- What happens at confuse(confuse)? ---
# confuse(confuse) halts  =>  halts(confuse, confuse) was True
#                         =>  the while True ran  =>  it loops  =>  CONTRADICTION
# confuse(confuse) loops  =>  halts(confuse, confuse) was False
#                         =>  the return ran       =>  it halts =>  CONTRADICTION
#
# Conclusion: halts() cannot exist as a total computable function.
print("The contradiction shows no total halts() oracle can exist.")
print("The halting problem is undecidable.")
// JavaScript: same logical argument.

// Hypothetical oracle — cannot be implemented for all programs
function halts(f, x) {
    // HYPOTHETICAL: returns true if f(x) terminates.
    // No universal implementation exists. This is the statement of undecidability.
    throw new Error("This oracle cannot exist!");
}

// The confuse function
function confuse(f) {
    if (halts(f, f)) {
        // Oracle says f halts on itself — so we loop forever (contradiction)
        while (true) { /* infinite loop */ }
    } else {
        // Oracle says f loops on itself — so we return immediately (contradiction)
        return;
    }
}

// Trace the contradiction:
// confuse(confuse) halts?
//   TRUE  => halts(confuse,confuse) returned true
//         => while(true) executed => it loops => CONTRADICTION
//   FALSE => halts(confuse,confuse) returned false
//         => return executed => it halts => CONTRADICTION
//
// Therefore no such oracle can exist.
console.log("No total halts() oracle can exist — the halting problem is undecidable.");
/* C: the self-reference argument.
   We cannot write a real halts() for arbitrary programs in C either.
   We show the logical structure with function pointers. */

#include <stdio.h>
#include <stdnoreturn.h>

typedef void (*Fn)(void*);

/* Hypothetical oracle — cannot be implemented */
int halts(Fn f, void *x) {
    /* If this existed and were total, confuse() below produces a contradiction. */
    (void)f; (void)x;
    return 0; /* placeholder — any return value leads to contradiction */
}

noreturn void loop_forever(void) { for (;;) {} }

/* confuse: if halts says f(f) halts, we loop; if it says loop, we halt */
void confuse(Fn f) {
    if (halts(f, (void*)f))
        loop_forever();   /* contradiction path A */
    else
        return;           /* contradiction path B */
}

int main(void) {
    /* confuse((Fn)confuse) leads to contradiction either way. */
    printf("Logical structure: halts() cannot be a total computable function.\n");
    printf("The halting problem is undecidable.\n");
    return 0;
}
// C++: same argument with std::function for clarity.
#include <iostream>
#include <functional>
#include <stdexcept>

using AnyFn = std::function<void(std::any)>;

// Hypothetical oracle
bool halts(AnyFn f, std::any x) {
    // Cannot be implemented for all f, x.
    // Proof by contradiction: if it existed, confuse() below is a paradox.
    throw std::logic_error("This oracle cannot exist!");
}

// The contradiction machine
void confuse(AnyFn f) {
    if (halts(f, f)) {
        // Oracle says f(f) halts -> we loop forever
        while (true) {}  // contradiction
    } else {
        // Oracle says f(f) loops -> we return
        return;          // contradiction
    }
}

// Calling confuse(confuse) produces a paradox:
//   If it halts:  halts() said true, so we loop  => does NOT halt  => contradiction
//   If it loops:  halts() said false, so we halt => DOES halt      => contradiction
// Therefore halts() cannot exist as a total function.

int main() {
    std::cout << "No total halts() oracle can exist.\n";
    std::cout << "The halting problem is undecidable.\n";
}
import java.util.function.Consumer;

/**
 * Java: the self-reference / diagonalization argument.
 * We model programs as Consumer<Object> and show halts() leads to paradox.
 */
public class HaltingProblem {

    /** Hypothetical oracle — cannot be implemented for all programs. */
    static boolean halts(Consumer<Object> f, Object x) {
        // If this were a real total function, confuse() below is a paradox.
        throw new UnsupportedOperationException("This oracle cannot exist!");
    }

    /**
     * confuse: designed to do the opposite of what halts() predicts.
     * Running confuse(confuse) is the contradiction.
     */
    static void confuse(Consumer<Object> f) {
        if (halts(f, f)) {
            // Oracle says f halts on itself — we loop forever
            //noinspection InfiniteLoopStatement
            while (true) { /* contradiction path A */ }
        } else {
            // Oracle says f loops on itself — we halt immediately
            return; // contradiction path B
        }
    }

    public static void main(String[] args) {
        // confuse(confuse):
        //   Halts?  => halts() returned true  => while(true) runs  => loops  => CONTRADICTION
        //   Loops?  => halts() returned false => return runs       => halts  => CONTRADICTION
        System.out.println("halts() cannot be a total computable function.");
        System.out.println("The halting problem is undecidable.");
    }
}
package main

import "fmt"

// halts is a hypothetical oracle.
// It cannot be implemented for all programs — that is exactly the theorem.
func halts(f func(interface{}), x interface{}) bool {
	// Any implementation leads to contradiction via confuse below.
	panic("This oracle cannot exist!")
}

// confuse does the opposite of what halts predicts.
func confuse(f func(interface{})) {
	if halts(f, f) {
		// Oracle says f(f) halts — we loop forever (contradiction)
		for { /* infinite loop */ }
	} else {
		// Oracle says f(f) loops — we return immediately (contradiction)
		return
	}
}

// Running confuse(confuse) is the paradox:
//   If confuse(confuse) halts:
//     halts(confuse,confuse) returned true => for{} executed => loops => CONTRADICTION
//   If confuse(confuse) loops:
//     halts(confuse,confuse) returned false => return executed => halts => CONTRADICTION

func main() {
	fmt.Println("No total halts() oracle can exist.")
	fmt.Println("The halting problem is undecidable.")
}

7. Reductions — the tool for comparing problems

We just showed one problem (halting) is undecidable. How do we show other problems are also undecidable, without redoing the diagonal argument every time? The answer is reductions, which are also the central tool of complexity theory. They're how problems get compared.

The slogan: "if I could solve B, I could solve A." A reduction from A to B is a way of converting any instance of A into an instance of B such that the answer to B tells you the answer to A. If reductions exist in both directions, A and B are essentially the same problem in different costumes.

Many-one reduction (computability flavor). Problem $A$ reduces to problem $B$ — written $A \le_m B$ — if there's a computable function $f$ such that for every input $x$, $x \in A$ if and only if $f(x) \in B$. So you take an instance of $A$, run $f$ on it to get an instance of $B$, and you've translated the question. If you have a solver for $B$, the composition $\text{Solver}_B \circ f$ solves $A$.

The contrapositive is the workhorse. If $A \le_m B$ and $A$ is undecidable, then $B$ is undecidable too — because if you could decide $B$, you'd decide $A$, which you can't. So if you want to show some new problem $B$ is undecidable, you reduce halting (or some other known undecidable problem) to $B$. This is how every other undecidability proof in the field works.

Example: "does this program ever print the word 'hello'?" is undecidable. To show this, take any halting question "does $M$ halt on $w$?" and transform it into "does the modified program $M'$ ever print 'hello'?" — where $M'$ is $M$ with a print('hello') statement added at the very end. $M'$ prints 'hello' if and only if $M$ halts. If we could decide the print question, we could decide halting, which we can't. So the print question is undecidable too.

For complexity theory, we use a stricter version:

Polynomial-time reduction. $A \le_p B$ if the function $f$ above can be computed in time polynomial in the size of $x$. The "polynomial" matters because reductions are how we compare problems for efficiency. A reduction that takes exponential time is too expensive to use as evidence that two problems are the same difficulty.

A polynomial-time reduction lets you say: if $B$ has a polynomial-time algorithm, then so does $A$ (just compose the reduction with $B$'s solver — polynomial composed with polynomial is still polynomial). And contrapositively, if $A$ is hard, then so is $B$. This is the engine that powers the next several sections.

Source — Polynomial Reduction (SAT → 3-SAT)
-- Polynomial reduction: convert a general SAT formula to 3-SAT.
-- Every clause must have exactly 3 literals.

-- Case 1: clause has 1 literal (a)
--   Add two fresh variables z1, z2.
--   Replace with: (a v z1 v z2) ^ (a v z1 v ~z2) ^ (a v ~z1 v z2) ^ (a v ~z1 v ~z2)
--   Satisfiable iff a is true (the z variables can always be set to make 3 of 4 clauses happy,
--   but all 4 are satisfiable simultaneously only when a is true).

-- Case 2: clause has 2 literals (a v b)
--   Add one fresh variable z.
--   Replace with: (a v b v z) ^ (a v b v ~z)
--   Satisfiable iff at least one of a,b is true.
--   Proof: if a or b is true, set z freely. If both false, no z works.

-- Case 3: clause already has 3 literals — keep as-is.

-- Case 4: clause has k > 3 literals (a1 v a2 v ... v ak)
--   Introduce fresh variables z1 ... z_{k-3}.
--   (a1 v a2 v z1) ^ (~z1 v a3 v z2) ^ (~z2 v a4 v z3) ^ ... ^ (~z_{k-3} v a_{k-1} v ak)
--   This chain is satisfiable iff at least one ai is true.

sat_to_3sat(formula):
  new_formula = []
  fresh_counter = 0
  for clause in formula:
    new_formula += expand_clause(clause, fresh_counter)
  return new_formula
from itertools import count as _count

def sat_to_3sat(formula: list[list[int]]) -> list[list[int]]:
    """
    Reduce a SAT formula (list of clauses, each a list of ints) to 3-SAT.
    Positive int k  = variable x_k.
    Negative int -k = negation ~x_k.
    Fresh variables start at max_var + 1.
    """
    if not formula:
        return formula
    max_var = max(abs(lit) for clause in formula for lit in clause)
    fresh = _count(max_var + 1)

    result = []
    for clause in formula:
        k = len(clause)
        if k == 1:
            # (a) -> (a v z v ~z) style — use 4 clauses with 2 fresh vars
            z1, z2 = next(fresh), next(fresh)
            result.extend([
                [clause[0],  z1,  z2],
                [clause[0],  z1, -z2],
                [clause[0], -z1,  z2],
                [clause[0], -z1, -z2],
            ])
        elif k == 2:
            # (a v b) -> (a v b v z) ^ (a v b v ~z)
            z = next(fresh)
            result.append([clause[0], clause[1],  z])
            result.append([clause[0], clause[1], -z])
        elif k == 3:
            result.append(list(clause))
        else:
            # k > 3: chain of 3-literal clauses
            # (a1 v a2 v z1) ^ (~z1 v a3 v z2) ^ ... ^ (~z_{k-3} v a_{k-1} v ak)
            zs = [next(fresh) for _ in range(k - 3)]
            # First clause
            result.append([clause[0], clause[1], zs[0]])
            # Middle clauses
            for i in range(1, k - 3):
                result.append([-zs[i-1], clause[i+1], zs[i]])
            # Last clause
            result.append([-zs[-1], clause[-2], clause[-1]])

    return result


# Demo: (x1) ^ (x2 v x3) ^ (x1 v ~x2 v x3 v x4)
formula = [[1], [2, 3], [1, -2, 3, 4]]
three_sat = sat_to_3sat(formula)
print(f"Original: {formula}")
print(f"3-SAT:    {three_sat}")
print(f"Clause count: {len(formula)} -> {len(three_sat)}")
/**
 * Reduce a SAT formula to 3-SAT.
 * Formula: array of clauses, each clause an array of signed integers.
 * Positive k = x_k, negative -k = NOT x_k.
 */
function satTo3Sat(formula) {
    if (formula.length === 0) return formula;
    let maxVar = 0;
    for (const clause of formula)
        for (const lit of clause)
            maxVar = Math.max(maxVar, Math.abs(lit));
    let fresh = maxVar + 1;

    const result = [];
    for (const clause of formula) {
        const k = clause.length;
        if (k === 1) {
            const [z1, z2] = [fresh++, fresh++];
            result.push([clause[0],  z1,  z2]);
            result.push([clause[0],  z1, -z2]);
            result.push([clause[0], -z1,  z2]);
            result.push([clause[0], -z1, -z2]);
        } else if (k === 2) {
            const z = fresh++;
            result.push([clause[0], clause[1],  z]);
            result.push([clause[0], clause[1], -z]);
        } else if (k === 3) {
            result.push([...clause]);
        } else {
            // Chain reduction for k > 3
            const zs = Array.from({length: k - 3}, () => fresh++);
            result.push([clause[0], clause[1], zs[0]]);
            for (let i = 1; i < k - 3; i++)
                result.push([-zs[i-1], clause[i+1], zs[i]]);
            result.push([-zs[zs.length-1], clause[k-2], clause[k-1]]);
        }
    }
    return result;
}

const formula = [[1], [2, 3], [1, -2, 3, 4]];
console.log("Original:", JSON.stringify(formula));
console.log("3-SAT:   ", JSON.stringify(satTo3Sat(formula)));
#include <stdio.h>
#include <stdlib.h>

/* Represent a formula as a flat array of clauses; each clause is
   a length-3 array of signed ints (the 3-SAT output). */

typedef struct { int lits[3]; } Clause3;

static int fresh_var;

static void emit(Clause3 **out, int *n, int a, int b, int c) {
    *out = realloc(*out, (*n + 1) * sizeof(Clause3));
    (*out)[*n].lits[0] = a;
    (*out)[*n].lits[1] = b;
    (*out)[*n].lits[2] = c;
    (*n)++;
}

/* Expand a single clause of arbitrary length into 3-SAT clauses. */
void expand_clause(int *lits, int k, Clause3 **out, int *n) {
    if (k == 1) {
        int z1 = fresh_var++, z2 = fresh_var++;
        emit(out, n, lits[0],  z1,  z2);
        emit(out, n, lits[0],  z1, -z2);
        emit(out, n, lits[0], -z1,  z2);
        emit(out, n, lits[0], -z1, -z2);
    } else if (k == 2) {
        int z = fresh_var++;
        emit(out, n, lits[0], lits[1],  z);
        emit(out, n, lits[0], lits[1], -z);
    } else if (k == 3) {
        emit(out, n, lits[0], lits[1], lits[2]);
    } else {
        /* Chain for k > 3 */
        int *zs = malloc((k - 3) * sizeof(int));
        for (int i = 0; i < k - 3; i++) zs[i] = fresh_var++;
        emit(out, n, lits[0], lits[1], zs[0]);
        for (int i = 1; i < k - 3; i++)
            emit(out, n, -zs[i-1], lits[i+1], zs[i]);
        emit(out, n, -zs[k-4], lits[k-2], lits[k-1]);
        free(zs);
    }
}

int main(void) {
    fresh_var = 5; /* variables x1..x4 already used */
    int c1[] = {1};          /* (x1) */
    int c2[] = {2, 3};       /* (x2 v x3) */
    int c3[] = {1,-2,3,4};   /* (x1 v ~x2 v x3 v x4) */

    Clause3 *out = NULL; int n = 0;
    expand_clause(c1, 1, &out, &n);
    expand_clause(c2, 2, &out, &n);
    expand_clause(c3, 4, &out, &n);

    printf("3-SAT has %d clauses:\n", n);
    for (int i = 0; i < n; i++)
        printf("  (%d v %d v %d)\n", out[i].lits[0], out[i].lits[1], out[i].lits[2]);
    free(out);
    return 0;
}
#include <iostream>
#include <vector>
#include <algorithm>

using Clause = std::vector<int>;
using Formula = std::vector<Clause>;

Formula satTo3Sat(const Formula& f) {
    int maxVar = 0;
    for (auto& c : f) for (int l : c) maxVar = std::max(maxVar, std::abs(l));
    int fresh = maxVar + 1;
    Formula result;

    for (auto& clause : f) {
        int k = clause.size();
        if (k == 1) {
            int z1 = fresh++, z2 = fresh++;
            result.push_back({ clause[0],  z1,  z2});
            result.push_back({ clause[0],  z1, -z2});
            result.push_back({ clause[0], -z1,  z2});
            result.push_back({ clause[0], -z1, -z2});
        } else if (k == 2) {
            int z = fresh++;
            result.push_back({clause[0], clause[1],  z});
            result.push_back({clause[0], clause[1], -z});
        } else if (k == 3) {
            result.push_back(clause);
        } else {
            std::vector<int> zs(k - 3);
            for (auto& z : zs) z = fresh++;
            result.push_back({clause[0], clause[1], zs[0]});
            for (int i = 1; i < k - 3; i++)
                result.push_back({-zs[i-1], clause[i+1], zs[i]});
            result.push_back({-zs.back(), clause[k-2], clause[k-1]});
        }
    }
    return result;
}

int main() {
    Formula f = {{1}, {2,3}, {1,-2,3,4}};
    auto f3 = satTo3Sat(f);
    std::cout << "3-SAT clauses: " << f3.size() << "\n";
    for (auto& c : f3)
        std::cout << "  (" << c[0] << " v " << c[1] << " v " << c[2] << ")\n";
}
import java.util.*;

public class SatTo3Sat {
    static int fresh;

    static List<int[]> satTo3Sat(List<int[]> formula) {
        fresh = formula.stream()
            .flatMapToInt(c -> Arrays.stream(c).map(Math::abs))
            .max().orElse(0) + 1;

        List<int[]> result = new ArrayList<>();
        for (int[] clause : formula) {
            int k = clause.length;
            if (k == 1) {
                int z1 = fresh++, z2 = fresh++;
                result.add(new int[]{clause[0],  z1,  z2});
                result.add(new int[]{clause[0],  z1, -z2});
                result.add(new int[]{clause[0], -z1,  z2});
                result.add(new int[]{clause[0], -z1, -z2});
            } else if (k == 2) {
                int z = fresh++;
                result.add(new int[]{clause[0], clause[1],  z});
                result.add(new int[]{clause[0], clause[1], -z});
            } else if (k == 3) {
                result.add(clause.clone());
            } else {
                int[] zs = new int[k - 3];
                for (int i = 0; i < zs.length; i++) zs[i] = fresh++;
                result.add(new int[]{clause[0], clause[1], zs[0]});
                for (int i = 1; i < k - 3; i++)
                    result.add(new int[]{-zs[i-1], clause[i+1], zs[i]});
                result.add(new int[]{-zs[zs.length-1], clause[k-2], clause[k-1]});
            }
        }
        return result;
    }

    public static void main(String[] args) {
        List<int[]> f = Arrays.asList(new int[]{1}, new int[]{2,3}, new int[]{1,-2,3,4});
        List<int[]> f3 = satTo3Sat(f);
        System.out.println("3-SAT clauses: " + f3.size());
        for (int[] c : f3)
            System.out.printf("  (%d v %d v %d)%n", c[0], c[1], c[2]);
    }
}
package main

import (
	"fmt"
)

func satTo3Sat(formula [][]int) [][]int {
	maxVar := 0
	for _, clause := range formula {
		for _, lit := range clause {
			if v := lit; v < 0 { v = -v }; maxVar = func(a,b int) int { if a>b {return a}; return b }(maxVar, func() int { if lit < 0 { return -lit }; return lit }())
		}
	}
	fresh := maxVar + 1
	newFresh := func() int { v := fresh; fresh++; return v }

	var result [][]int
	for _, clause := range formula {
		k := len(clause)
		switch {
		case k == 1:
			z1, z2 := newFresh(), newFresh()
			result = append(result,
				[]int{clause[0],  z1,  z2},
				[]int{clause[0],  z1, -z2},
				[]int{clause[0], -z1,  z2},
				[]int{clause[0], -z1, -z2})
		case k == 2:
			z := newFresh()
			result = append(result,
				[]int{clause[0], clause[1],  z},
				[]int{clause[0], clause[1], -z})
		case k == 3:
			result = append(result, append([]int{}, clause...))
		default:
			zs := make([]int, k-3)
			for i := range zs { zs[i] = newFresh() }
			result = append(result, []int{clause[0], clause[1], zs[0]})
			for i := 1; i < k-3; i++ {
				result = append(result, []int{-zs[i-1], clause[i+1], zs[i]})
			}
			result = append(result, []int{-zs[len(zs)-1], clause[k-2], clause[k-1]})
		}
	}
	return result
}

func main() {
	f := [][]int{{1}, {2, 3}, {1, -2, 3, 4}}
	f3 := satTo3Sat(f)
	fmt.Printf("3-SAT clauses: %d\n", len(f3))
	for _, c := range f3 {
		fmt.Printf("  (%d v %d v %d)\n", c[0], c[1], c[2])
	}
}

8. Complexity classes — sorting problems by difficulty

Inside the world of decidable problems, we sort by how much resource (time or space) they need. A complexity class is just a name for a set of problems that share a resource bound. The famous ones, in roughly increasing order:

What we know about how they fit together:

$$\text{P} \subseteq \text{NP} \subseteq \text{PSPACE} \subseteq \text{EXPTIME} \subseteq \text{NEXPTIME} \subseteq \text{EXPSPACE}$$

The complexity tower

$\subseteq$
"Is a subset of." Every problem in the smaller class is also in the larger one. Reading left to right: a polynomial-time algorithm is a special case of an NP algorithm (just ignore the hint), which is a special case of a PSPACE algorithm, and so on.
P
The class we believe corresponds to "can be solved in practice." Polynomial time covers $n$, $n^2$, $n^{100}$ — the constant in the exponent doesn't matter for the definition.
NP
"Nondeterministic Polynomial." Not "non-polynomial." A common confusion. NP is about being checkable in polynomial time, not about being unsolvable in polynomial time.
PSPACE
Anything you can do with a polynomial-sized scratchpad, given unbounded time. Surprisingly powerful — chess endgame analysis lives here.
EXPTIME
Exponential time: $2^n$, $2^{n^2}$, etc. Generalized chess on an $n \times n$ board lives here.

Strict or not? Most of those $\subseteq$s could in principle be $=$. We know for sure that $\text{P} \subsetneq \text{EXPTIME}$ (by the time hierarchy theorem in section 16) and $\text{NP} \subsetneq \text{NEXPTIME}$. But P vs NP, NP vs PSPACE, PSPACE vs EXPTIME — every adjacent pair you'd actually want to compare — are open. Almost everyone believes the inclusions are strict, but no one can prove it.

Below is an interactive view of the inclusion chain. Click any class to see canonical problems known to live in it (and not, as far as we know, in any smaller class).

Click a class: P — polynomial time

P is the smallest, EXPSPACE the largest. Each ring contains everything inside it. Hover or click to see which class a problem belongs to.

9. P vs NP

This is the centerpiece. Two equivalent definitions of NP, both worth memorizing.

Definition 1 (verifier model). A language $L$ is in NP if there exists a polynomial-time verifier $V$ and a polynomial $p$ such that for every input $x$:

$$x \in L \iff \exists\, c \in \{0,1\}^{p(|x|)} \text{ such that } V(x, c) = \text{accept}$$

NP via verifiers

$L$
The language (set of yes-instances) we're trying to recognize.
$x$
An input string. $|x|$ is its length in symbols.
$c$
The "certificate" or "witness" — a hint that proves $x$ is a yes-instance. Crucially, $c$ has length bounded by some polynomial $p$ in $|x|$, so it's not too big to read in polynomial time.
$V(x, c)$
The verifier — a polynomial-time deterministic algorithm that checks whether $c$ really proves $x \in L$.
$\exists$
"There exists." We're saying $x$ is a yes-instance exactly when some short certificate exists that the verifier accepts.

Plain English. A problem is in NP if, when the answer is "yes," somebody can hand you a short proof and you can quickly check it. Sudoku: a filled-in grid is the certificate, and "is this grid valid?" is a fast check. Factoring: the prime factors are the certificate, and multiplying them back together is the check. Graph 3-coloring: the coloring is the certificate, and "does any edge have two same-colored endpoints?" is the check.

Definition 2 (nondeterministic TM). Equivalently, NP is the set of languages decided in polynomial time by a nondeterministic Turing machine — a TM whose transition function $\delta$ can return a set of next moves rather than a single one. The machine accepts if any sequence of choices leads to an accept state. Think of it as a TM that magically guesses the right move at every fork.

The two definitions are equivalent because the nondeterministic guesses are exactly the certificate $c$ — you can write down the sequence of guesses as a binary string of polynomial length, and a deterministic verifier can replay them.

P is the much smaller-feeling cousin: a problem is in P if you can find the answer in polynomial time, not just check it. Multiplying two numbers, sorting a list, deciding whether a graph is connected — all in P.

The P vs NP question. Is every problem whose solution can be quickly verified also one whose solution can be quickly found? Does $\text{P} = \text{NP}$?

Almost everyone in the field believes the answer is no — that there are problems for which checking is fundamentally easier than finding. But after fifty years of trying, no one has a proof in either direction. The question is one of the seven Clay Millennium Problems; a correct proof is worth a million dollars and a Fields Medal in spirit if not in fact (the medal has an age cutoff).

Why everyone thinks P ≠ NP. If P = NP, then SAT, traveling salesman, graph coloring, integer programming, protein folding, and most of cryptography would all collapse to polynomial time. We'd have spent decades not finding fast algorithms for any of them. You can read that as either "we haven't tried hard enough" or "they really don't exist." Most theorists go with the second.

10. NP-completeness

Some problems in NP are at least as hard as every other problem in NP. They are called NP-complete. Formally:

A problem $B$ is NP-hard if every problem $A \in \text{NP}$ reduces to it in polynomial time: $A \le_p B$. It is NP-complete if it is both NP-hard and itself in NP. NP-complete problems are the hardest problems in NP — if any one of them is in P, then all of NP is in P, so P = NP.

For this definition to be useful, we need at least one problem to be NP-complete, so that future proofs can reduce from it. The first one was found by Stephen Cook in 1971 (and independently by Leonid Levin). Their result is the Cook-Levin theorem:

Cook-Levin (1971)

The Boolean satisfiability problem SAT is NP-complete. SAT asks: given a Boolean formula like $(x_1 \lor \neg x_2 \lor x_3) \land (\neg x_1 \lor x_2)$, is there an assignment of true/false values to the variables that makes the formula evaluate to true?

The proof of Cook-Levin is a tour de force: take any nondeterministic TM that runs in polynomial time, and encode its entire computation tableau (every cell of the tape at every time step) as a giant Boolean formula whose satisfying assignments are exactly the accepting computation paths. Since every NP problem can be decided by such a TM, every NP problem reduces to SAT. SAT is at least as hard as anything in NP.

Once you have one NP-complete problem, you get more by reduction. If you reduce SAT to your favorite problem $X$ in polynomial time, then $X$ is NP-hard too. And if $X$ is also in NP, it's NP-complete. Within a year of Cook-Levin, Richard Karp published a paper showing 21 problems are NP-complete by chains of reductions from SAT — the famous "Karp's 21." That paper is the reason most computer scientists believe NP-completeness is everywhere.

Source — SAT Solver (DPLL Brute Force)
-- DPLL-style SAT solver with unit propagation.
-- formula: list of clauses (each clause is a set of literals)
-- assignment: dict var -> True/False

dpll(formula, assignment):
  -- Unit propagation: if a clause has one unassigned literal, force it
  changed = True
  while changed:
    changed = False
    for each clause in formula:
      unassigned = [lit for lit in clause if var(lit) not in assignment]
      if len(unassigned) == 0:
        if no literal in clause is satisfied: return UNSAT
      elif len(unassigned) == 1:
        force assignment[var(unassigned[0])] = polarity(unassigned[0])
        changed = True

  -- Check if all clauses are satisfied
  if all clauses have at least one satisfied literal: return SAT, assignment

  -- Choose an unassigned variable and branch
  v = pick any unassigned variable
  for value in [True, False]:
    result = dpll(formula, assignment + {v: value})
    if result == SAT: return SAT, result.assignment
  return UNSAT
from typing import Optional

def dpll(clauses: list[list[int]], assignment: dict[int, bool] = None) -> Optional[dict[int, bool]]:
    """
    DPLL SAT solver with unit propagation.
    Literals: positive int k = x_k, negative int -k = NOT x_k.
    Returns a satisfying assignment dict or None if UNSAT.
    """
    if assignment is None:
        assignment = {}

    # Unit propagation loop
    changed = True
    while changed:
        changed = False
        for clause in clauses:
            true_lits  = [l for l in clause if  (l > 0) == assignment.get(abs(l), None if True else False)
                          if abs(l) in assignment]
            # Simpler approach: evaluate each literal
            sat_lits   = [l for l in clause if _lit_val(l, assignment) is True]
            undef_lits = [l for l in clause if _lit_val(l, assignment) is None]
            false_lits = [l for l in clause if _lit_val(l, assignment) is False]
            if sat_lits:
                continue  # clause already satisfied
            if not undef_lits:
                return None  # all literals false -> UNSAT
            if len(undef_lits) == 1:
                # Unit clause: force this literal
                lit = undef_lits[0]
                assignment = dict(assignment)  # copy before mutating
                assignment[abs(lit)] = (lit > 0)
                changed = True
                break  # restart loop with updated assignment

    # Check if all clauses satisfied
    if all(any(_lit_val(l, assignment) is True for l in c) for c in clauses):
        return assignment

    # Find an unassigned variable
    all_vars = {abs(l) for c in clauses for l in c}
    unassigned = [v for v in all_vars if v not in assignment]
    if not unassigned:
        return None  # no solution

    v = unassigned[0]
    for val in (True, False):
        new_assign = dict(assignment)
        new_assign[v] = val
        result = dpll(clauses, new_assign)
        if result is not None:
            return result
    return None


def _lit_val(lit: int, assignment: dict) -> Optional[bool]:
    v = abs(lit)
    if v not in assignment:
        return None
    return assignment[v] if lit > 0 else not assignment[v]


# Example: (x1 v ~x2) ^ (~x1 v x2 v x3) ^ (~x3)
formula = [[1, -2], [-1, 2, 3], [-3]]
result = dpll(formula)
if result:
    print("SAT:", {f"x{k}": v for k, v in sorted(result.items())})
else:
    print("UNSAT")
# Expected: SAT with some assignment like x1=False, x2=False, x3=False
/**
 * Brute-force SAT solver: enumerate all 2^n assignments.
 * Simple but exponential — the point is to show the baseline.
 */
function bruteForce3Sat(clauses, nVars) {
    // Try all 2^nVars assignments using a bitmask
    for (let mask = 0; mask < (1 << nVars); mask++) {
        const assignment = {};
        for (let i = 0; i < nVars; i++)
            assignment[i + 1] = !!(mask & (1 << i));

        let allSat = true;
        for (const clause of clauses) {
            const clauseSat = clause.some(lit =>
                lit > 0 ? assignment[lit] : !assignment[-lit]);
            if (!clauseSat) { allSat = false; break; }
        }
        if (allSat) return assignment;
    }
    return null;
}

const formula = [[1, -2], [-1, 2, 3], [-3]];
const result = bruteForce3Sat(formula, 3);
if (result) {
    console.log("SAT:", Object.entries(result).map(([k,v]) => `x${k}=${v}`).join(", "));
} else {
    console.log("UNSAT");
}
#include <stdio.h>
#include <string.h>

/* Brute-force SAT via bitmask over 2^n assignments. */
#define MAX_VARS 20

typedef struct { int lits[3]; int len; } Clause;

int eval_lit(int lit, int mask) {
    int var = (lit > 0 ? lit : -lit) - 1;   /* 0-indexed */
    int val = (mask >> var) & 1;             /* 1 = true */
    return lit > 0 ? val : !val;
}

int solve(Clause *clauses, int nclauses, int nvars, int *out_mask) {
    for (int mask = 0; mask < (1 << nvars); mask++) {
        int ok = 1;
        for (int i = 0; i < nclauses; i++) {
            int sat = 0;
            for (int j = 0; j < clauses[i].len; j++)
                if (eval_lit(clauses[i].lits[j], mask)) { sat = 1; break; }
            if (!sat) { ok = 0; break; }
        }
        if (ok) { *out_mask = mask; return 1; }
    }
    return 0;
}

int main(void) {
    /* (x1 v ~x2) ^ (~x1 v x2 v x3) ^ (~x3) */
    Clause f[] = {
        {{1,-2,0},2}, {{-1,2,3},3}, {{-3,0,0},1}
    };
    int mask;
    if (solve(f, 3, 3, &mask)) {
        printf("SAT: x1=%d x2=%d x3=%d\n",
               (mask>>0)&1, (mask>>1)&1, (mask>>2)&1);
    } else {
        printf("UNSAT\n");
    }
    return 0;
}
#include <iostream>
#include <vector>

using Clause = std::vector<int>;

bool evalLit(int lit, int mask) {
    int v = std::abs(lit) - 1;
    bool val = (mask >> v) & 1;
    return lit > 0 ? val : !val;
}

std::pair<bool,int> bruteForce(const std::vector<Clause>& clauses, int nVars) {
    for (int mask = 0; mask < (1 << nVars); mask++) {
        bool ok = true;
        for (auto& c : clauses) {
            bool sat = false;
            for (int l : c) if (evalLit(l, mask)) { sat = true; break; }
            if (!sat) { ok = false; break; }
        }
        if (ok) return {true, mask};
    }
    return {false, -1};
}

int main() {
    std::vector<Clause> f = {{1,-2},{-1,2,3},{-3}};
    auto [sat, mask] = bruteForce(f, 3);
    if (sat)
        std::cout << "SAT: x1=" << (mask&1) << " x2=" << ((mask>>1)&1)
                  << " x3=" << ((mask>>2)&1) << "\n";
    else
        std::cout << "UNSAT\n";
}
import java.util.*;

public class BruteForceSAT {
    static boolean evalLit(int lit, int mask) {
        int v = Math.abs(lit) - 1;
        boolean val = ((mask >> v) & 1) == 1;
        return lit > 0 ? val : !val;
    }

    static int bruteForce(int[][] clauses, int nVars) {
        for (int mask = 0; mask < (1 << nVars); mask++) {
            boolean ok = true;
            for (int[] clause : clauses) {
                boolean sat = false;
                for (int lit : clause) if (evalLit(lit, mask)) { sat = true; break; }
                if (!sat) { ok = false; break; }
            }
            if (ok) return mask;
        }
        return -1;
    }

    public static void main(String[] args) {
        int[][] f = {{1, -2}, {-1, 2, 3}, {-3}};
        int mask = bruteForce(f, 3);
        if (mask >= 0)
            System.out.printf("SAT: x1=%d x2=%d x3=%d%n",
                mask & 1, (mask >> 1) & 1, (mask >> 2) & 1);
        else
            System.out.println("UNSAT");
    }
}
package main

import (
	"fmt"
)

func evalLit(lit, mask int) bool {
	v := lit
	if v < 0 { v = -v }
	v-- // 0-indexed
	val := (mask>>v)&1 == 1
	if lit > 0 { return val }
	return !val
}

func bruteForce(clauses [][]int, nVars int) int {
	for mask := 0; mask < (1 << nVars); mask++ {
		ok := true
		for _, clause := range clauses {
			sat := false
			for _, lit := range clause {
				if evalLit(lit, mask) { sat = true; break }
			}
			if !sat { ok = false; break }
		}
		if ok { return mask }
	}
	return -1
}

func main() {
	// (x1 v ~x2) ^ (~x1 v x2 v x3) ^ (~x3)
	f := [][]int{{1, -2}, {-1, 2, 3}, {-3}}
	mask := bruteForce(f, 3)
	if mask >= 0 {
		fmt.Printf("SAT: x1=%d x2=%d x3=%d\n", mask&1, (mask>>1)&1, (mask>>2)&1)
	} else {
		fmt.Println("UNSAT")
	}
}

11. Canonical NP-complete problems

The standard list. You will recognize most of these from real life.

Let's slow down on one reduction so you can see how the trick works. We'll show 3-SAT $\le_p$ 3-COLOR: if you had a fast 3-COLOR solver, you could solve 3-SAT fast, by encoding any 3-SAT formula as a graph that is 3-colorable iff the formula is satisfiable.

The reduction (sketch). Use three special "color" vertices $T$, $F$, $X$ joined as a triangle, so they must take all three different colors in any valid 3-coloring. Call those colors True, False, and Other. For each variable $v_i$, add two vertices $v_i$ and $\neg v_i$ connected to each other and to $X$. The triangle $\{v_i, \neg v_i, X\}$ forces $v_i$ and $\neg v_i$ to take the True/False colors in opposite assignments — exactly the "a variable is true or false but not both" structure. For each 3-SAT clause $(\ell_1 \lor \ell_2 \lor \ell_3)$, attach a small "OR gadget" of six extra vertices that is 3-colorable if and only if at least one of $\ell_1, \ell_2, \ell_3$ has color True. The whole graph is 3-colorable exactly when the formula is satisfiable.

That's a lot to take in the first time. The key intuition: the gadget for each clause acts like a tiny puzzle that can only be solved if at least one literal in the clause is true. The variable gadget guarantees consistent truth assignments. Stack everything together and you get a graph whose colorability mirrors the formula's satisfiability.

Reductions like this are the everyday work of complexity theory. They're confusing the first time. They're confusing the fifth time. The skill develops from doing them by hand. Sipser's textbook (linked at the bottom) walks through several in detail.

Source — NP-Hard Problems (Exact Solvers)
-- (a) Hamiltonian Path (brute force) — NP-complete
-- Check every permutation of vertices; see if any is a valid path.

hamiltonian_path(graph, n):
  for each permutation P of vertices [0..n-1]:
    valid = True
    for i in 0..n-2:
      if edge(P[i], P[i+1]) not in graph:
        valid = False; break
    if valid: return P   -- found a Hamiltonian path
  return NONE            -- no path exists

-- Time: O(n!) — infeasible for n > ~12 without pruning.

-- (b) 0/1 Knapsack (dynamic programming) — pseudo-polynomial O(n * W)
-- items: list of (weight, value) pairs
-- W: maximum weight capacity

knapsack_dp(items, W):
  n = len(items)
  dp[0..n][0..W] = 0   -- dp[i][w] = best value using first i items with weight <= w

  for i in 1..n:
    for w in 0..W:
      dp[i][w] = dp[i-1][w]                            -- skip item i
      if items[i-1].weight <= w:
        take = dp[i-1][w - items[i-1].weight] + items[i-1].value
        dp[i][w] = max(dp[i][w], take)                 -- take item i

  return dp[n][W]   -- maximum achievable value
from itertools import permutations

# ---- (a) Hamiltonian Path — brute force ----
def hamiltonian_path(adj: list[list[int]]) -> list[int] | None:
    """
    adj: adjacency matrix (adj[i][j] = 1 if edge exists).
    Returns a Hamiltonian path as a list of vertices, or None.
    Time: O(n!) — NP-complete, no known polynomial algorithm.
    """
    n = len(adj)
    for perm in permutations(range(n)):
        if all(adj[perm[i]][perm[i+1]] for i in range(n - 1)):
            return list(perm)
    return None

# Example: a path graph 0-1-2-3
adj4 = [
    [0, 1, 0, 0],
    [1, 0, 1, 0],
    [0, 1, 0, 1],
    [0, 0, 1, 0],
]
print("Hamiltonian path:", hamiltonian_path(adj4))  # e.g. [0, 1, 2, 3]

# No Hamiltonian path: disconnected
adj_bad = [[0,1,0,0],[1,0,0,0],[0,0,0,1],[0,0,1,0]]
print("Hamiltonian path:", hamiltonian_path(adj_bad))  # None


# ---- (b) 0/1 Knapsack — O(n * W) DP ----
def knapsack(items: list[tuple[int,int]], W: int) -> tuple[int, list[int]]:
    """
    items: list of (weight, value) pairs.
    W: maximum weight.
    Returns (best_value, list of chosen item indices).
    Pseudo-polynomial: O(n * W). Not polynomial in input *bit-length*.
    """
    n = len(items)
    # dp[i][w] = max value using first i items with total weight <= w
    dp = [[0] * (W + 1) for _ in range(n + 1)]
    for i in range(1, n + 1):
        wt, val = items[i - 1]
        for w in range(W + 1):
            dp[i][w] = dp[i-1][w]
            if wt <= w:
                dp[i][w] = max(dp[i][w], dp[i-1][w - wt] + val)

    # Backtrack to find chosen items
    chosen, w = [], W
    for i in range(n, 0, -1):
        if dp[i][w] != dp[i-1][w]:
            chosen.append(i - 1)
            w -= items[i-1][0]

    return dp[n][W], chosen[::-1]

items = [(2, 6), (2, 10), (3, 12), (1, 3)]  # (weight, value)
W = 5
value, chosen = knapsack(items, W)
print(f"Knapsack: best value = {value}, items = {chosen}")
# Expected: value=22, items=[1,2] (weights 2+3=5, values 10+12=22)
// (a) Hamiltonian path — brute force
function hamiltonianPath(adj) {
    const n = adj.length;
    const perm = Array.from({length: n}, (_, i) => i);

    function* permutations(arr) {
        if (arr.length <= 1) { yield arr; return; }
        for (let i = 0; i < arr.length; i++) {
            const rest = [...arr.slice(0,i), ...arr.slice(i+1)];
            for (const p of permutations(rest)) yield [arr[i], ...p];
        }
    }

    for (const p of permutations(perm)) {
        let ok = true;
        for (let i = 0; i < n - 1; i++)
            if (!adj[p[i]][p[i+1]]) { ok = false; break; }
        if (ok) return p;
    }
    return null;
}

// (b) 0/1 knapsack DP
function knapsack(items, W) {
    const n = items.length;
    const dp = Array.from({length: n+1}, () => new Array(W+1).fill(0));
    for (let i = 1; i <= n; i++) {
        const [wt, val] = items[i-1];
        for (let w = 0; w <= W; w++) {
            dp[i][w] = dp[i-1][w];
            if (wt <= w) dp[i][w] = Math.max(dp[i][w], dp[i-1][w-wt] + val);
        }
    }
    return dp[n][W];
}

const adj = [[0,1,0,0],[1,0,1,0],[0,1,0,1],[0,0,1,0]];
console.log("Ham. path:", hamiltonianPath(adj));

const items = [[2,6],[2,10],[3,12],[1,3]];
console.log("Knapsack:", knapsack(items, 5));  // 22
#include <stdio.h>
#include <string.h>

/* (a) Hamiltonian path — brute force with Heap's permutation algorithm */
#define MAXN 8
int adj[MAXN][MAXN], n_nodes;
int perm[MAXN];

int check_ham(void) {
    for (int i = 0; i < n_nodes - 1; i++)
        if (!adj[perm[i]][perm[i+1]]) return 0;
    return 1;
}

void swap(int *a, int *b) { int t=*a; *a=*b; *b=t; }

int heap_perm(int k) {
    if (k == 1) return check_ham();
    for (int i = 0; i < k; i++) {
        if (heap_perm(k - 1)) return 1;
        if (k % 2 == 0) swap(&perm[i], &perm[k-1]);
        else             swap(&perm[0], &perm[k-1]);
    }
    return 0;
}

/* (b) 0/1 knapsack DP */
int knapsack(int *wts, int *vals, int n, int W) {
    static int dp[100][1001];
    memset(dp, 0, sizeof(dp));
    for (int i = 1; i <= n; i++)
        for (int w = 0; w <= W; w++) {
            dp[i][w] = dp[i-1][w];
            if (wts[i-1] <= w) {
                int take = dp[i-1][w-wts[i-1]] + vals[i-1];
                if (take > dp[i][w]) dp[i][w] = take;
            }
        }
    return dp[n][W];
}

int main(void) {
    n_nodes = 4;
    int a[4][4] = {{0,1,0,0},{1,0,1,0},{0,1,0,1},{0,0,1,0}};
    for (int i=0;i<4;i++) for(int j=0;j<4;j++) adj[i][j]=a[i][j];
    for (int i=0;i<4;i++) perm[i]=i;
    printf("Hamiltonian path found: %s\n", heap_perm(n_nodes)?"yes":"no");

    int wts[]={2,2,3,1}, vals[]={6,10,12,3};
    printf("Knapsack best value: %d\n", knapsack(wts,vals,4,5));  /* 22 */
    return 0;
}
#include <iostream>
#include <vector>
#include <algorithm>
#include <numeric>

// (a) Hamiltonian path
bool hamiltonianPath(const std::vector<std::vector<int>>& adj) {
    int n = adj.size();
    std::vector<int> perm(n);
    std::iota(perm.begin(), perm.end(), 0);
    do {
        bool ok = true;
        for (int i = 0; i < n - 1; i++)
            if (!adj[perm[i]][perm[i+1]]) { ok = false; break; }
        if (ok) return true;
    } while (std::next_permutation(perm.begin(), perm.end()));
    return false;
}

// (b) 0/1 Knapsack DP
int knapsack(const std::vector<std::pair<int,int>>& items, int W) {
    int n = items.size();
    std::vector<std::vector<int>> dp(n+1, std::vector<int>(W+1, 0));
    for (int i = 1; i <= n; i++) {
        auto [wt, val] = items[i-1];
        for (int w = 0; w <= W; w++) {
            dp[i][w] = dp[i-1][w];
            if (wt <= w) dp[i][w] = std::max(dp[i][w], dp[i-1][w-wt] + val);
        }
    }
    return dp[n][W];
}

int main() {
    std::vector<std::vector<int>> adj = {{0,1,0,0},{1,0,1,0},{0,1,0,1},{0,0,1,0}};
    std::cout << "Ham. path: " << (hamiltonianPath(adj) ? "yes" : "no") << "\n";

    std::vector<std::pair<int,int>> items = {{2,6},{2,10},{3,12},{1,3}};
    std::cout << "Knapsack best: " << knapsack(items, 5) << "\n";  // 22
}
import java.util.*;

public class NpHardSolvers {

    // (a) Hamiltonian path — brute force
    static boolean hamiltonianPath(int[][] adj) {
        int n = adj.length;
        int[] perm = new int[n];
        for (int i = 0; i < n; i++) perm[i] = i;
        do {
            boolean ok = true;
            for (int i = 0; i < n - 1; i++)
                if (adj[perm[i]][perm[i+1]] == 0) { ok = false; break; }
            if (ok) return true;
        } while (nextPermutation(perm));
        return false;
    }

    static boolean nextPermutation(int[] arr) {
        int n = arr.length, i = n - 2;
        while (i >= 0 && arr[i] >= arr[i+1]) i--;
        if (i < 0) return false;
        int j = n - 1;
        while (arr[j] <= arr[i]) j--;
        int tmp = arr[i]; arr[i] = arr[j]; arr[j] = tmp;
        for (int l=i+1, r=n-1; l < r; l++, r--) { tmp=arr[l]; arr[l]=arr[r]; arr[r]=tmp; }
        return true;
    }

    // (b) 0/1 Knapsack DP
    static int knapsack(int[] wts, int[] vals, int W) {
        int n = wts.length;
        int[][] dp = new int[n+1][W+1];
        for (int i = 1; i <= n; i++)
            for (int w = 0; w <= W; w++) {
                dp[i][w] = dp[i-1][w];
                if (wts[i-1] <= w)
                    dp[i][w] = Math.max(dp[i][w], dp[i-1][w-wts[i-1]] + vals[i-1]);
            }
        return dp[n][W];
    }

    public static void main(String[] args) {
        int[][] adj = {{0,1,0,0},{1,0,1,0},{0,1,0,1},{0,0,1,0}};
        System.out.println("Ham. path: " + hamiltonianPath(adj));

        int[] wts = {2,2,3,1}, vals = {6,10,12,3};
        System.out.println("Knapsack best: " + knapsack(wts, vals, 5));  // 22
    }
}
package main

import (
	"fmt"
)

// (a) Hamiltonian path — brute force permutations
func hamiltonianPath(adj [][]int) bool {
	n := len(adj)
	perm := make([]int, n)
	for i := range perm { perm[i] = i }

	var check func([]int) bool
	check = func(p []int) bool {
		for i := 0; i < len(p)-1; i++ {
			if adj[p[i]][p[i+1]] == 0 { return false }
		}
		return true
	}

	var permute func([]int, int) bool
	permute = func(p []int, k int) bool {
		if k == len(p) { return check(p) }
		for i := k; i < len(p); i++ {
			p[k], p[i] = p[i], p[k]
			if permute(p, k+1) { return true }
			p[k], p[i] = p[i], p[k]
		}
		return false
	}
	return permute(perm, 0)
}

// (b) 0/1 Knapsack DP
func knapsack(wts, vals []int, W int) int {
	n := len(wts)
	dp := make([][]int, n+1)
	for i := range dp { dp[i] = make([]int, W+1) }
	for i := 1; i <= n; i++ {
		for w := 0; w <= W; w++ {
			dp[i][w] = dp[i-1][w]
			if wts[i-1] <= w {
				if v := dp[i-1][w-wts[i-1]] + vals[i-1]; v > dp[i][w] {
					dp[i][w] = v
				}
			}
		}
	}
	return dp[n][W]
}

func main() {
	adj := [][]int{{0,1,0,0},{1,0,1,0},{0,1,0,1},{0,0,1,0}}
	fmt.Println("Ham. path:", hamiltonianPath(adj))

	wts := []int{2, 2, 3, 1}
	vals := []int{6, 10, 12, 3}
	fmt.Println("Knapsack best:", knapsack(wts, vals, 5))  // 22
}

12. Why P vs NP matters

The question matters because the NP-complete problems are everywhere in real applications and we strongly suspect they're irreducibly hard. If P = NP, the world looks one way; if P ≠ NP, it looks very different.

If P = NP:

If P ≠ NP (the assumed-true case):

Subtlety. Even if P = NP, the polynomial in question might be $n^{100}$ with a constant of $10^{50}$. Theoretically efficient does not mean practically efficient. Conversely, even if P ≠ NP, many NP-complete problems have algorithms that are fast enough in practice on the instances people care about — modern SAT solvers handle problems with millions of variables. The theoretical worst case is not the same as the typical case.

13. Approximation algorithms

For NP-hard optimization problems (find the minimum-cost tour, the largest independent set, the minimum vertex cover), we often settle for an approximate answer: not the optimum, but provably close to it. The quality is measured by the approximation ratio:

$$\rho = \max_x \frac{\text{ALG}(x)}{\text{OPT}(x)}$$

Approximation ratio

$\rho$
The approximation ratio. For minimization problems, $\rho \ge 1$; smaller is better. $\rho = 1.5$ means the algorithm's answer is at most 50% worse than optimal.
$\text{ALG}(x)$
The cost of the solution returned by your algorithm on input $x$.
$\text{OPT}(x)$
The cost of the true optimal solution on input $x$ — what you'd find with unlimited time.
$\max_x$
The maximum over all inputs. Approximation ratios are worst-case guarantees.

The point. Even if you can't find the best answer in polynomial time, you can sometimes find an answer that's provably within a factor of 2 or 1.5 or even $1 + \epsilon$ of optimal, in polynomial time. That's often enough for practice.

Example: 2-approximation for Vertex Cover. Vertex Cover is NP-hard, but here is a stupidly simple 2-approximation:

  1. Repeatedly pick any uncovered edge $(u, v)$.
  2. Add both $u$ and $v$ to your cover.
  3. Remove all edges touching $u$ or $v$.
  4. Repeat until no edges remain.

Why is this within a factor of 2? Every edge you picked must have at least one endpoint in the optimal cover (since the optimum covers all edges). You added both endpoints, so for each "essential" optimum vertex you spent at most 2 of your own. The worst you can do is 2× optimal. That's a worst-case guarantee with no tuning, no randomness, no parameters.

Different problems have wildly different approximation behavior. TSP on a metric (where distances satisfy the triangle inequality) has a $\rho = 3/2$ algorithm by Christofides. General TSP can't be approximated within any constant unless P = NP. MAX-3-SAT has a $\rho = 8/7$ algorithm. The PCP theorem (a deep result from 1992) says many problems can't be approximated within better than some constant, ever.

Source — 2-Approx Vertex Cover
-- 2-APPROXIMATION FOR VERTEX COVER via maximal matching
-- A vertex cover is a set S such that every edge has at least one endpoint in S.
-- OPT = minimum vertex cover size. This gives |cover| <= 2 * OPT.

vertex_cover_2approx(graph):
  cover = empty set
  edges = copy of all edges
  while edges is not empty:
    pick any edge (u, v) from edges
    add u and v to cover         -- always take BOTH endpoints
    remove all edges touching u or v from edges
  return cover

-- Why ratio = 2: the matched edges form a matching M.
-- OPT must include at least one endpoint per matched edge, so |OPT| >= |M|.
-- Our cover has exactly 2|M| vertices, so |cover| = 2|M| <= 2*|OPT|.

-- GREEDY SET COVER (ln n approximation)
-- Universe U of n elements, collection of sets. Cover U with fewest sets.

greedy_set_cover(U, sets):
  covered = empty
  chosen = []
  while covered != U:
    S = argmax_{S in sets} |S \ covered|   -- set that covers most new elements
    chosen.append(S)
    covered = covered union S
  return chosen
-- Approximation ratio: H(max_set_size) ~= ln(n) where H is the harmonic number.
def vertex_cover_2approx(n: int, edges: list[tuple[int,int]]) -> set[int]:
    """
    2-approximation for minimum vertex cover via maximal matching.
    n: number of vertices (0-indexed).
    edges: list of (u, v) pairs.
    Returns a vertex cover S with |S| <= 2 * OPT.
    """
    cover = set()
    remaining = list(edges)
    while remaining:
        u, v = remaining.pop()      # pick any uncovered edge
        cover.add(u)
        cover.add(v)
        # remove all edges touching u or v
        remaining = [(a, b) for a, b in remaining if a != u and a != v and b != u and b != v]
    return cover


def greedy_set_cover(universe: set, sets: list[set]) -> list[int]:
    """
    Greedy set cover: always pick the set that covers the most uncovered elements.
    Approximation ratio: H(|max_set|) ≈ ln(n).
    Returns list of chosen set indices.
    """
    covered = set()
    chosen = []
    remaining = list(range(len(sets)))
    while covered != universe:
        # find the set that covers the most new elements
        best = max(remaining, key=lambda i: len(sets[i] - covered))
        chosen.append(best)
        covered |= sets[best]
        remaining.remove(best)
    return chosen


# Demo: vertex cover on a small graph
edges = [(0,1),(0,2),(1,3),(2,3),(3,4)]
cover = vertex_cover_2approx(5, edges)
print("Vertex cover:", cover)
# Verify every edge is covered
assert all(u in cover or v in cover for u,v in edges), "Not a valid cover!"
print("All edges covered: OK")

# Demo: set cover
U = {1,2,3,4,5,6,7,8,9,10}
sets = [{1,2,3,4},{3,4,5},{5,6,7},{7,8,9,10},{1,5,9}]
chosen = greedy_set_cover(U, sets)
print("Set cover indices:", chosen, "-> sets:", [sets[i] for i in chosen])
function vertexCover2Approx(n, edges) {
    const cover = new Set();
    let remaining = edges.map(e => [...e]);  // copy
    while (remaining.length > 0) {
        const [u, v] = remaining.pop();
        cover.add(u);
        cover.add(v);
        remaining = remaining.filter(([a, b]) => a !== u && a !== v && b !== u && b !== v);
    }
    return cover;
}

function greedySetCover(universe, sets) {
    const covered = new Set();
    const chosen = [];
    const remaining = new Set(sets.keys());
    while (covered.size < universe.size) {
        let best = -1, bestCount = -1;
        for (const i of remaining) {
            const count = [...sets[i]].filter(x => !covered.has(x)).length;
            if (count > bestCount) { bestCount = count; best = i; }
        }
        chosen.push(best);
        for (const x of sets[best]) covered.add(x);
        remaining.delete(best);
    }
    return chosen;
}

const edges = [[0,1],[0,2],[1,3],[2,3],[3,4]];
const cover = vertexCover2Approx(5, edges);
console.log("Vertex cover:", [...cover]);

const U = new Set([1,2,3,4,5,6,7,8,9,10]);
const sets = [[1,2,3,4],[3,4,5],[5,6,7],[7,8,9,10],[1,5,9]].map(a => new Set(a));
console.log("Set cover:", greedySetCover(U, sets));
#include <stdio.h>
#include <string.h>

#define MAXV 100
#define MAXE 1000

/* 2-approximation vertex cover */
void vertex_cover_2approx(int eu[], int ev[], int ne, int cover[], int *nc) {
    int used[MAXE] = {0};
    *nc = 0;
    int in_cover[MAXV] = {0};
    for (int i = 0; i < ne; i++) {
        if (used[i]) continue;
        int u = eu[i], v = ev[i];
        if (!in_cover[u]) { cover[(*nc)++] = u; in_cover[u] = 1; }
        if (!in_cover[v]) { cover[(*nc)++] = v; in_cover[v] = 1; }
        /* mark all edges touching u or v */
        for (int j = i; j < ne; j++)
            if (eu[j]==u||eu[j]==v||ev[j]==u||ev[j]==v) used[j]=1;
    }
}

int main(void) {
    int eu[] = {0,0,1,2,3};
    int ev[] = {1,2,3,3,4};
    int ne = 5;
    int cover[MAXV], nc = 0;
    vertex_cover_2approx(eu, ev, ne, cover, &nc);
    printf("Vertex cover (%d vertices):", nc);
    for (int i = 0; i < nc; i++) printf(" %d", cover[i]);
    printf("\n");

    /* Verify every edge is covered */
    for (int i = 0; i < ne; i++) {
        int ok = 0;
        for (int j = 0; j < nc; j++)
            if (cover[j]==eu[i]||cover[j]==ev[i]) { ok=1; break; }
        if (!ok) { printf("ERROR: edge (%d,%d) not covered!\n",eu[i],ev[i]); return 1; }
    }
    printf("All edges covered: OK\n");
    return 0;
}
#include <iostream>
#include <vector>
#include <set>
#include <algorithm>

using Edge = std::pair<int,int>;

std::set<int> vertexCover2Approx(const std::vector<Edge>& edges) {
    std::set<int> cover;
    std::vector<Edge> rem = edges;
    while (!rem.empty()) {
        auto [u, v] = rem.back(); rem.pop_back();
        cover.insert(u); cover.insert(v);
        rem.erase(std::remove_if(rem.begin(), rem.end(),
            [&](const Edge& e){ return e.first==u||e.first==v||e.second==u||e.second==v; }),
            rem.end());
    }
    return cover;
}

int main() {
    std::vector<Edge> edges = {{0,1},{0,2},{1,3},{2,3},{3,4}};
    auto cover = vertexCover2Approx(edges);
    std::cout << "Vertex cover:";
    for (int v : cover) std::cout << " " << v;
    std::cout << "\n";

    for (auto& [u,v] : edges)
        if (!cover.count(u) && !cover.count(v)) { std::cout << "ERROR\n"; return 1; }
    std::cout << "All edges covered: OK\n";
}
import java.util.*;

public class VertexCover {
    static Set<Integer> vertexCover2Approx(int n, int[][] edges) {
        Set<Integer> cover = new HashSet<>();
        List<int[]> rem = new ArrayList<>(Arrays.asList(edges));
        while (!rem.isEmpty()) {
            int[] e = rem.remove(rem.size() - 1);
            int u = e[0], v = e[1];
            cover.add(u); cover.add(v);
            rem.removeIf(x -> x[0]==u || x[0]==v || x[1]==u || x[1]==v);
        }
        return cover;
    }

    public static void main(String[] args) {
        int[][] edges = {{0,1},{0,2},{1,3},{2,3},{3,4}};
        Set<Integer> cover = vertexCover2Approx(5, edges);
        System.out.println("Vertex cover: " + cover);

        for (int[] e : edges)
            if (!cover.contains(e[0]) && !cover.contains(e[1]))
                throw new RuntimeException("Edge not covered: " + Arrays.toString(e));
        System.out.println("All edges covered: OK");
    }
}
package main

import "fmt"

type Edge struct{ u, v int }

func vertexCover2Approx(edges []Edge) map[int]bool {
	cover := map[int]bool{}
	rem := append([]Edge{}, edges...)
	for len(rem) > 0 {
		e := rem[len(rem)-1]
		rem = rem[:len(rem)-1]
		u, v := e.u, e.v
		cover[u] = true
		cover[v] = true
		filtered := rem[:0]
		for _, x := range rem {
			if x.u != u && x.u != v && x.v != u && x.v != v {
				filtered = append(filtered, x)
			}
		}
		rem = filtered
	}
	return cover
}

func main() {
	edges := []Edge{{0,1},{0,2},{1,3},{2,3},{3,4}}
	cover := vertexCover2Approx(edges)
	fmt.Print("Vertex cover:")
	for v := range cover { fmt.Print(" ", v) }
	fmt.Println()

	for _, e := range edges {
		if !cover[e.u] && !cover[e.v] {
			fmt.Printf("ERROR: edge (%d,%d) not covered\n", e.u, e.v)
			return
		}
	}
	fmt.Println("All edges covered: OK")
}

14. Randomized complexity

What if your algorithm is allowed to flip coins? Sometimes a random choice works dramatically better than any deterministic strategy. This gives rise to randomized complexity classes:

Examples to anchor these:

The big question in randomized complexity is whether randomness actually helps. For decades it looked like BPP might be strictly larger than P. But starting in the 1990s, derandomization results piled up: under reasonable hardness assumptions, $\text{P} = \text{BPP}$. Most theorists now believe the equality holds, even though it's not proven. The intuitive picture: pseudo-random generators are good enough that "real" randomness rarely buys you more than constant-factor speedup.

Source — Miller-Rabin Primality Test
-- Miller-Rabin probabilistic primality test.
-- An example of an RP (Randomized Polynomial) algorithm.
-- If n is composite, each random witness a catches it with probability >= 3/4.
-- k rounds -> error probability <= (1/4)^k.

miller_rabin(n, k=20):
  if n < 2: return COMPOSITE
  if n == 2 or n == 3: return PRIME
  if n is even: return COMPOSITE

  -- Write n-1 as 2^r * d, where d is odd
  r = 0, d = n - 1
  while d is even: r++, d /= 2

  -- k rounds with random witnesses
  repeat k times:
    a = random integer in [2, n-2]
    x = a^d mod n

    if x == 1 or x == n-1: continue  -- probably prime for this witness

    -- Square x up to r-1 times
    repeat r-1 times:
      x = x^2 mod n
      if x == n-1: break (go to next witness)
    else:
      return COMPOSITE  -- n is definitely composite

  return PROBABLY_PRIME  -- no witness found it composite
import random

def miller_rabin(n: int, k: int = 20) -> bool:
    """
    Miller-Rabin probabilistic primality test.
    Returns True if n is probably prime, False if definitely composite.
    Error probability: at most (1/4)^k per call.
    This is an RP-class algorithm: no false negatives, possible false positives.
    """
    if n < 2:   return False
    if n == 2:  return True
    if n == 3:  return True
    if n % 2 == 0: return False

    # Write n-1 as 2^r * d with d odd
    r, d = 0, n - 1
    while d % 2 == 0:
        r += 1
        d //= 2

    for _ in range(k):
        a = random.randrange(2, n - 1)
        x = pow(a, d, n)          # Python's built-in modular exponentiation

        if x == 1 or x == n - 1:
            continue               # probably prime for this witness

        for _ in range(r - 1):
            x = pow(x, 2, n)
            if x == n - 1:
                break
        else:
            return False           # definitely composite

    return True                    # probably prime


# Test on some known primes and composites
primes    = [2, 3, 5, 7, 11, 13, 17, 104729, 15485863, 32452843]
composites = [4, 9, 25, 100, 104730, 15485864]

for p in primes:
    assert miller_rabin(p), f"False negative: {p} should be prime"
for c in composites:
    assert not miller_rabin(c), f"False positive: {c} should be composite"

print("All tests passed. Miller-Rabin error rate per round: <= 1/4")
/**
 * Miller-Rabin primality test in JavaScript.
 * Note: JavaScript's Number is a 64-bit float; for large primes use BigInt.
 */
function millerRabin(n, k = 20) {
    if (n < 2n) return false;
    if (n === 2n || n === 3n) return true;
    if (n % 2n === 0n) return false;

    // Write n-1 as 2^r * d
    let r = 0n, d = n - 1n;
    while (d % 2n === 0n) { r++; d /= 2n; }

    function modpow(base, exp, mod) {
        let result = 1n;
        base %= mod;
        while (exp > 0n) {
            if (exp % 2n === 1n) result = result * base % mod;
            exp /= 2n;
            base = base * base % mod;
        }
        return result;
    }

    for (let i = 0; i < k; i++) {
        // Random witness in [2, n-2]
        const a = 2n + BigInt(Math.floor(Math.random() * Number(n - 4n)));
        let x = modpow(a, d, n);
        if (x === 1n || x === n - 1n) continue;

        let composite = true;
        for (let j = 0n; j < r - 1n; j++) {
            x = modpow(x, 2n, n);
            if (x === n - 1n) { composite = false; break; }
        }
        if (composite) return false;
    }
    return true;
}

const tests = [2n, 3n, 7n, 104729n, 15485863n, 4n, 9n, 104730n];
for (const n of tests)
    console.log(`${n}: ${millerRabin(n) ? "probably prime" : "composite"}`);
#include <stdio.h>
#include <stdlib.h>
#include <time.h>
#include <stdint.h>

/* 64-bit modular multiplication to avoid overflow */
typedef unsigned long long ull;

ull mulmod(ull a, ull b, ull m) {
    return (__uint128_t)a * b % m;
}

ull powmod(ull base, ull exp, ull m) {
    ull result = 1;
    base %= m;
    while (exp > 0) {
        if (exp & 1) result = mulmod(result, base, m);
        exp >>= 1;
        base = mulmod(base, base, m);
    }
    return result;
}

int miller_rabin(ull n, int k) {
    if (n < 2) return 0;
    if (n == 2 || n == 3) return 1;
    if (n % 2 == 0) return 0;

    ull d = n - 1; int r = 0;
    while (d % 2 == 0) { d /= 2; r++; }

    for (int i = 0; i < k; i++) {
        ull a = 2 + rand() % (n - 4);
        ull x = powmod(a, d, n);
        if (x == 1 || x == n - 1) continue;
        int maybe_prime = 0;
        for (int j = 0; j < r - 1; j++) {
            x = mulmod(x, x, n);
            if (x == n - 1) { maybe_prime = 1; break; }
        }
        if (!maybe_prime) return 0;  /* composite */
    }
    return 1;  /* probably prime */
}

int main(void) {
    srand((unsigned)time(NULL));
    ull tests[] = {2,3,7,104729,15485863,4,9,104730};
    for (int i = 0; i < 8; i++)
        printf("%llu: %s\n", tests[i], miller_rabin(tests[i],20)?"probably prime":"composite");
    return 0;
}
#include <iostream>
#include <cstdint>
#include <random>

using ull = unsigned long long;
using u128 = __uint128_t;

ull mulmod(ull a, ull b, ull m) { return (u128)a * b % m; }

ull powmod(ull b, ull e, ull m) {
    ull r = 1; b %= m;
    for (; e; e >>= 1, b = mulmod(b,b,m))
        if (e & 1) r = mulmod(r, b, m);
    return r;
}

bool millerRabin(ull n, int k = 20) {
    if (n < 2) return false;
    if (n == 2 || n == 3) return true;
    if (n % 2 == 0) return false;

    ull d = n - 1; int r = 0;
    while (!(d & 1)) { d >>= 1; r++; }

    std::mt19937_64 rng(std::random_device{}());
    std::uniform_int_distribution<ull> dist(2, n - 2);

    for (int i = 0; i < k; i++) {
        ull a = dist(rng);
        ull x = powmod(a, d, n);
        if (x == 1 || x == n - 1) continue;
        bool ok = false;
        for (int j = 0; j < r - 1; j++) {
            x = mulmod(x, x, n);
            if (x == n - 1) { ok = true; break; }
        }
        if (!ok) return false;
    }
    return true;
}

int main() {
    for (ull n : {2ULL,3ULL,7ULL,104729ULL,15485863ULL,4ULL,9ULL,104730ULL})
        std::cout << n << ": " << (millerRabin(n) ? "probably prime" : "composite") << "\n";
}
import java.math.BigInteger;
import java.util.Random;

public class MillerRabin {
    static Random rng = new Random();

    static boolean millerRabin(long n, int k) {
        if (n < 2) return false;
        if (n == 2 || n == 3) return true;
        if (n % 2 == 0) return false;

        long d = n - 1; int r = 0;
        while (d % 2 == 0) { d /= 2; r++; }

        BigInteger N = BigInteger.valueOf(n);
        BigInteger D = BigInteger.valueOf(d);
        BigInteger N1 = N.subtract(BigInteger.ONE);

        for (int i = 0; i < k; i++) {
            long aLong = 2 + Math.abs(rng.nextLong()) % (n - 3);
            BigInteger a = BigInteger.valueOf(aLong);
            BigInteger x = a.modPow(D, N);
            if (x.equals(BigInteger.ONE) || x.equals(N1)) continue;
            boolean ok = false;
            for (int j = 0; j < r - 1; j++) {
                x = x.multiply(x).mod(N);
                if (x.equals(N1)) { ok = true; break; }
            }
            if (!ok) return false;
        }
        return true;
    }

    public static void main(String[] args) {
        long[] tests = {2,3,7,104729L,15485863L,4,9,104730L};
        for (long n : tests)
            System.out.println(n + ": " + (millerRabin(n, 20) ? "probably prime" : "composite"));
    }
}
package main

import (
	"fmt"
	"math/big"
	"math/rand"
)

func millerRabin(n int64, k int) bool {
	if n < 2 { return false }
	if n == 2 || n == 3 { return true }
	if n%2 == 0 { return false }

	d, r := n-1, 0
	for d%2 == 0 { d /= 2; r++ }

	N := big.NewInt(n)
	D := big.NewInt(d)
	N1 := new(big.Int).Sub(N, big.NewInt(1))
	one := big.NewInt(1)

	for i := 0; i < k; i++ {
		a := big.NewInt(2 + rand.Int63n(n-3))
		x := new(big.Int).Exp(a, D, N)
		if x.Cmp(one) == 0 || x.Cmp(N1) == 0 { continue }
		ok := false
		for j := 0; j < r-1; j++ {
			x.Mul(x, x).Mod(x, N)
			if x.Cmp(N1) == 0 { ok = true; break }
		}
		if !ok { return false }
	}
	return true
}

func main() {
	tests := []int64{2, 3, 7, 104729, 15485863, 4, 9, 104730}
	for _, n := range tests {
		s := "composite"
		if millerRabin(n, 20) { s = "probably prime" }
		fmt.Printf("%d: %s\n", n, s)
	}
}

15. Quantum, briefly

What if your algorithm can manipulate quantum amplitudes? You get a new class.

BQP contains P, and is contained in PSPACE. Whether it's strictly larger than P is open in the same way P vs NP is open. BQP also has an unclear relationship to NP — it is widely believed that BQP and NP are incomparable, neither contained in the other.

Two famous problems where quantum gives an exponential or nearly-exponential speedup:

What quantum doesn't do. Quantum computers are not magic NP solvers. The general consensus is BQP does not contain NP — Grover is the best general quantum speedup we expect for NP problems, and it's quadratic, not exponential. Quantum specifically helps with structured problems like factoring, discrete log, and certain simulation tasks. It does not give you a free pass on the entire NP-completeness wall.

Source — Quantum Gate Simulation
-- Simulate quantum gates on a classical computer using complex-number vectors.
-- A 1-qubit state is a 2-component complex vector [alpha, beta]
--   where |alpha|^2 + |beta|^2 = 1.
-- |0> = [1, 0],  |1> = [0, 1]

-- Hadamard gate H: creates superposition from a basis state
H = (1/sqrt(2)) * [[1,  1],
                    [1, -1]]

H|0> = (1/sqrt(2))[1, 1]   -- equal superposition of |0> and |1>
H|1> = (1/sqrt(2))[1,-1]

-- CNOT gate (2-qubit): flips the target qubit if the control is |1>
-- Basis order: |00>, |01>, |10>, |11>
CNOT = [[1,0,0,0],
        [0,1,0,0],
        [0,0,0,1],
        [0,0,1,0]]

-- Bell state |Phi+> = (|00> + |11>) / sqrt(2)
-- Circuit: start with |00>, apply H to qubit 0, then CNOT

|00> --[H on q0]--> (1/sqrt(2))(|00> + |10>) --[CNOT]--> (1/sqrt(2))(|00> + |11>)

result = CNOT * (H tensor I) * |00>
       = [1/sqrt(2), 0, 0, 1/sqrt(2)]   -- Bell state |Phi+>
import numpy as np

# Single-qubit states
ket0 = np.array([1.0+0j, 0.0+0j])  # |0>
ket1 = np.array([0.0+0j, 1.0+0j])  # |1>

# Hadamard gate
H = np.array([[1, 1],
              [1,-1]], dtype=complex) / np.sqrt(2)

# Pauli-X (NOT) gate
X = np.array([[0,1],[1,0]], dtype=complex)

# CNOT gate (4x4, acts on 2-qubit state space)
CNOT = np.array([[1,0,0,0],
                 [0,1,0,0],
                 [0,0,0,1],
                 [0,0,1,0]], dtype=complex)

def tensor(A, B):
    """Kronecker product — combines two gate matrices."""
    return np.kron(A, B)

def apply(gate, state):
    """Apply a gate (matrix) to a state vector."""
    return gate @ state

# --- Demo 1: Hadamard on |0> ---
h0 = apply(H, ket0)
print("H|0> =", h0)                    # [0.707, 0.707] — equal superposition

# --- Demo 2: Bell state |Phi+> = (|00> + |11>) / sqrt(2) ---
# Start from |00> = tensor product of |0> and |0>
state_00 = np.kron(ket0, ket0)          # [1,0,0,0]

# Apply H to qubit 0 and identity I to qubit 1
I = np.eye(2, dtype=complex)
H_I = tensor(H, I)                      # 4x4 gate

after_H = apply(H_I, state_00)          # (|00> + |10>) / sqrt(2)
bell = apply(CNOT, after_H)             # (|00> + |11>) / sqrt(2)

print("Bell |Phi+>:", np.round(bell, 4))
# Expected: [0.7071, 0, 0, 0.7071]

# Verify normalization
print("Norm:", np.round(np.linalg.norm(bell)**2, 10))  # should be 1.0

# --- Demo 3: measure probabilities ---
probs = np.abs(bell)**2
labels = ["|00>", "|01>", "|10>", "|11>"]
for label, p in zip(labels, probs):
    print(f"  P({label}) = {p:.4f}")
# P(|00>) = 0.5000, P(|11>) = 0.5000 — the signature of entanglement
/**
 * Quantum gate simulation in JavaScript.
 * States and gates are represented as flat arrays of [real, imag] pairs.
 * We use a simple 2x2 and 4x4 matrix multiply — no external library.
 */

// Complex number helpers
const mul = ([ar,ai],[br,bi]) => [ar*br-ai*bi, ar*bi+ai*br];
const add = ([ar,ai],[br,bi]) => [ar+br, ai+bi];

// Matrix-vector multiply (n x n matrix, n-vector; entries are [re,im] pairs)
function matvec(M, v) {
    const n = v.length;
    return Array.from({length: n}, (_, i) =>
        v.reduce((acc, vj, j) => add(acc, mul(M[i][j], vj)), [0,0]));
}

// Kronecker product of two matrices
function kron(A, B) {
    const na = A.length, nb = B.length, n = na * nb;
    return Array.from({length: n}, (_, i) =>
        Array.from({length: n}, (_, j) =>
            mul(A[Math.floor(i/nb)][Math.floor(j/nb)], B[i%nb][j%nb])));
}

const s = 1/Math.sqrt(2);
const H    = [[[s,0],[s,0]], [[s,0],[-s,0]]];
const I2   = [[[1,0],[0,0]], [[0,0],[1,0]]];
const CNOT = [[[1,0],[0,0],[0,0],[0,0]],
              [[0,0],[1,0],[0,0],[0,0]],
              [[0,0],[0,0],[0,0],[1,0]],
              [[0,0],[0,0],[1,0],[0,0]]];

// |00> state
const ket00 = [[1,0],[0,0],[0,0],[0,0]];

// Apply H tensor I, then CNOT
const H_I      = kron(H, I2);
const afterH   = matvec(H_I, ket00);
const bell     = matvec(CNOT, afterH);

console.log("Bell |Phi+>:");
["|00>","|01>","|10>","|11>"].forEach((label,i) => {
    const [re,im] = bell[i];
    const prob = re*re + im*im;
    console.log(`  ${label}: re=${re.toFixed(4)} im=${im.toFixed(4)} P=${prob.toFixed(4)}`);
});
#include <stdio.h>
#include <math.h>

/* Complex number as (re, im) pair */
typedef struct { double re, im; } C;
static C cadd(C a,C b){return (C){a.re+b.re,a.im+b.im};}
static C cmul(C a,C b){return (C){a.re*b.re-a.im*b.im,a.re*b.im+a.im*b.re};}
static double cabs2(C a){return a.re*a.re+a.im*a.im;}

/* 4-vector state: |00>, |01>, |10>, |11> */
static void matvec4(C M[4][4], C v[4], C out[4]) {
    for (int i=0;i<4;i++) {
        out[i]=(C){0,0};
        for (int j=0;j<4;j++) out[i]=cadd(out[i],cmul(M[i][j],v[j]));
    }
}

int main(void) {
    double s = 1.0/sqrt(2.0);
    /* H tensor I — 4x4 */
    C HI[4][4] = {
        {{ s,0},{ 0,0},{ s,0},{ 0,0}},
        {{ 0,0},{ s,0},{ 0,0},{ s,0}},
        {{ s,0},{ 0,0},{-s,0},{ 0,0}},
        {{ 0,0},{ s,0},{ 0,0},{-s,0}},
    };
    /* CNOT */
    C CNOT[4][4] = {
        {{1,0},{0,0},{0,0},{0,0}},
        {{0,0},{1,0},{0,0},{0,0}},
        {{0,0},{0,0},{0,0},{1,0}},
        {{0,0},{0,0},{1,0},{0,0}},
    };
    /* |00> */
    C state[4] = {{1,0},{0,0},{0,0},{0,0}};
    C tmp[4], bell[4];

    matvec4(HI,   state, tmp);
    matvec4(CNOT, tmp,   bell);

    const char *labels[] = {"|00>","|01>","|10>","|11>"};
    printf("Bell |Phi+>:\n");
    for (int i=0;i<4;i++)
        printf("  %s re=%.4f im=%.4f P=%.4f\n",
               labels[i], bell[i].re, bell[i].im, cabs2(bell[i]));
    return 0;
}
#include <iostream>
#include <complex>
#include <array>
#include <cmath>

using C = std::complex<double>;
using State4 = std::array<C,4>;
using Gate4  = std::array<std::array<C,4>,4>;

State4 apply(const Gate4& M, const State4& v) {
    State4 out{};
    for (int i=0;i<4;i++) for (int j=0;j<4;j++) out[i]+=M[i][j]*v[j];
    return out;
}

int main() {
    const double s = 1.0/std::sqrt(2.0);
    // H tensor I
    Gate4 HI = {{
        {{{ s,0},{ 0,0},{ s,0},{ 0,0}}},
        {{{ 0,0},{ s,0},{ 0,0},{ s,0}}},
        {{{ s,0},{ 0,0},{-s,0},{ 0,0}}},
        {{{ 0,0},{ s,0},{ 0,0},{-s,0}}},
    }};
    // CNOT
    Gate4 CNOT = {{
        {{{1,0},{0,0},{0,0},{0,0}}},
        {{{0,0},{1,0},{0,0},{0,0}}},
        {{{0,0},{0,0},{0,0},{1,0}}},
        {{{0,0},{0,0},{1,0},{0,0}}},
    }};

    State4 state = {C(1,0),C(0,0),C(0,0),C(0,0)};  // |00>
    state = apply(CNOT, apply(HI, state));

    const char* labels[] = {"|00>","|01>","|10>","|11>"};
    for (int i=0;i<4;i++)
        std::cout << labels[i] << " re=" << state[i].real()
                  << " P=" << std::norm(state[i]) << "\n";
}
public class QuantumGates {
    // Complex multiply and add
    static double[] cmul(double[] a, double[] b) {
        return new double[]{a[0]*b[0]-a[1]*b[1], a[0]*b[1]+a[1]*b[0]};
    }
    static double[] cadd(double[] a, double[] b) {
        return new double[]{a[0]+b[0], a[1]+b[1]};
    }

    static double[][] matvec(double[][][] M, double[][] v) {
        int n = v.length;
        double[][] out = new double[n][2];
        for (int i = 0; i < n; i++)
            for (int j = 0; j < n; j++)
                out[i] = cadd(out[i], cmul(M[i][j], v[j]));
        return out;
    }

    public static void main(String[] args) {
        double s = 1.0 / Math.sqrt(2.0);
        // H tensor I (4x4)
        double[][][] HI = {
            {{s,0},{0,0},{s,0},{0,0}},
            {{0,0},{s,0},{0,0},{s,0}},
            {{s,0},{0,0},{-s,0},{0,0}},
            {{0,0},{s,0},{0,0},{-s,0}},
        };
        // CNOT
        double[][][] CNOT = {
            {{1,0},{0,0},{0,0},{0,0}},
            {{0,0},{1,0},{0,0},{0,0}},
            {{0,0},{0,0},{0,0},{1,0}},
            {{0,0},{0,0},{1,0},{0,0}},
        };

        // |00> state
        double[][] state = {{1,0},{0,0},{0,0},{0,0}};
        state = matvec(HI,   state);
        state = matvec(CNOT, state);

        String[] labels = {"|00>","|01>","|10>","|11>"};
        System.out.println("Bell |Phi+>:");
        for (int i = 0; i < 4; i++) {
            double p = state[i][0]*state[i][0] + state[i][1]*state[i][1];
            System.out.printf("  %s re=%.4f im=%.4f P=%.4f%n",
                labels[i], state[i][0], state[i][1], p);
        }
    }
}
package main

import (
	"fmt"
	"math"
	"math/cmplx"
)

type State4 [4]complex128
type Gate4  [4][4]complex128

func apply(M Gate4, v State4) State4 {
	var out State4
	for i := 0; i < 4; i++ {
		for j := 0; j < 4; j++ {
			out[i] += M[i][j] * v[j]
		}
	}
	return out
}

func main() {
	s := complex(1.0/math.Sqrt(2), 0)
	// H tensor I (4x4)
	HI := Gate4{
		{s, 0, s, 0},
		{0, s, 0, s},
		{s, 0, -s, 0},
		{0, s, 0, -s},
	}
	// CNOT
	CNOT := Gate4{
		{1, 0, 0, 0},
		{0, 1, 0, 0},
		{0, 0, 0, 1},
		{0, 0, 1, 0},
	}

	state := State4{1, 0, 0, 0} // |00>
	state = apply(HI, state)
	state = apply(CNOT, state)

	labels := []string{"|00>", "|01>", "|10>", "|11>"}
	fmt.Println("Bell |Phi+>:")
	for i, amp := range state {
		fmt.Printf("  %s re=%.4f im=%.4f P=%.4f\n",
			labels[i], real(amp), imag(amp), cmplx.Abs(amp)*cmplx.Abs(amp))
	}
}

16. Other interesting classes — and a real theorem

Some classes worth knowing the names of, even if you never use them:

One thing we can prove cleanly: more time really does let you solve more problems.

Time hierarchy theorem (Hartmanis-Stearns, 1965)

If $f$ and $g$ are time-constructible and $f(n) \log f(n) = o(g(n))$, then $\text{DTIME}(f(n)) \subsetneq \text{DTIME}(g(n))$. In English: there are problems that can be solved in $O(g(n))$ time that genuinely cannot be solved in $O(f(n))$ time. As a corollary, $\text{P} \subsetneq \text{EXPTIME}$ — proven, no asterisks.

This is one of the few "X is strictly harder than Y" statements we have a real proof of. The proof also uses diagonalization — building a problem that explicitly defeats every $f(n)$-time machine. It does not extend to P vs NP because nondeterminism breaks the diagonalization step in subtle ways. Which is why P vs NP is so much harder than it looks.

17. Oracles and relativization

There's a deep methodological point worth knowing. In 1975 Baker, Gill, and Solovay proved a result that explains why P vs NP has resisted attack for so long. They constructed two oracles — imaginary devices a Turing machine can consult for free — such that:

Why does this matter? Because most proof techniques in complexity theory relativize — they go through unchanged when you add an oracle to both sides. Diagonalization, in particular, relativizes. Baker-Gill-Solovay tells us: any proof of P vs NP that relativizes must give different answers depending on the oracle, which is inconsistent with proving P = NP or P ≠ NP unconditionally. So any actual proof of P vs NP cannot relativize — it has to be sensitive to the internal structure of computations in a way diagonalization isn't.

The current state of barriers. Beyond relativization, two more barriers have been identified: the "natural proofs" barrier (Razborov-Rudich, 1994) and the "algebrization" barrier (Aaronson-Wigderson, 2008). Together they rule out almost every proof technique we know how to use. A successful resolution of P vs NP will need to invent something genuinely new.

18. Source code

Two pieces of working Python: a Turing machine simulator (the same one as the SVG above, but in code), and a small recursive SAT solver demonstrating the brute-force baseline that polynomial-time algorithms try to beat.

Source — Compiled Examples
Python · pedagogical, not optimized
# A Turing machine that increments a unary number.
# Input: '1111' (= 4 in unary). Output: '11111'.

class TM:
    def __init__(self, transitions, start, accept, reject, blank='_'):
        self.delta  = transitions   # dict (state, symbol) -> (state, symbol, dir)
        self.state  = start
        self.accept = accept
        self.reject = reject
        self.blank  = blank
        self.tape   = []
        self.head   = 0

    def load(self, input_string):
        self.tape  = list(input_string) + [self.blank] * 10
        self.head  = 0

    def step(self):
        if self.state in (self.accept, self.reject):
            return False
        symbol = self.tape[self.head]
        key = (self.state, symbol)
        if key not in self.delta:
            self.state = self.reject
            return False
        new_state, new_symbol, direction = self.delta[key]
        self.tape[self.head] = new_symbol
        self.state = new_state
        if direction == 'R':
            self.head += 1
            if self.head >= len(self.tape):
                self.tape.append(self.blank)
        elif direction == 'L':
            self.head = max(0, self.head - 1)
        return True

    def run(self, max_steps=1000):
        for _ in range(max_steps):
            if not self.step():
                break
        return self.state == self.accept, ''.join(self.tape).rstrip(self.blank)

# Unary increment: walk right past all 1s, then write one more 1.
delta = {
    ('q0', '1'): ('q0', '1', 'R'),   # keep scanning right
    ('q0', '_'): ('q1', '_', 'L'),   # found the end, prep to write
    ('q1', '1'): ('q2', '1', 'R'),   # step back onto last 1
    ('q2', '_'): ('qA', '1', 'R'),   # write the new 1, accept
}

machine = TM(delta, start='q0', accept='qA', reject='qR')
machine.load('1111')
ok, tape = machine.run()
print(f"accepted={ok}, tape={tape!r}")
# accepted=True, tape='11111'
# Brute-force SAT solver. Exponential in the number of variables.
# This is the dumbest possible algorithm; it's the baseline a polynomial-time
# SAT solver (which would prove P = NP) would have to beat in the worst case.

from itertools import product

def evaluate(formula, assignment):
    # formula is a list of clauses; each clause is a list of literals.
    # A literal is an int: 3 means x_3, -3 means NOT x_3.
    for clause in formula:
        satisfied = False
        for lit in clause:
            v = abs(lit)
            value = assignment[v - 1] if lit > 0 else not assignment[v - 1]
            if value:
                satisfied = True
                break
        if not satisfied:
            return False
    return True

def brute_force_sat(formula, n_vars):
    for assignment in product([False, True], repeat=n_vars):
        if evaluate(formula, assignment):
            return list(assignment)
    return None

# Example: (x1 OR NOT x2) AND (NOT x1 OR x2 OR x3) AND (NOT x3)
formula = [
    [1, -2],
    [-1, 2, 3],
    [-3],
]
print(brute_force_sat(formula, n_vars=3))
# [False, False, False]  -- one valid assignment

# For n_vars = 30, this loop is 2^30 = 1 billion iterations.
# For n_vars = 100, it's 2^100, which is hopeless. Modern SAT solvers use
# conflict-driven clause learning, watched literals, and a hundred other
# tricks to skip most of the search tree. But the worst case is still 2^n.
# A small reduction sketch: convert any 3-SAT instance into a graph
# for 3-COLOR. We build the gadgets described in section 11.
# This is illustrative; production reductions handle edge cases.

def three_sat_to_three_color(clauses, n_vars):
    # Returns a graph as an edge list, plus a label dict for inspection.
    edges = []
    labels = {}
    # Palette triangle: True, False, Other
    T, F, X = 'T', 'F', 'X'
    edges += [(T, F), (F, X), (X, T)]
    for name in (T, F, X): labels[name] = name

    # Variable gadgets: v_i and not v_i form a triangle with X
    for i in range(1, n_vars + 1):
        v  = f'v{i}'
        nv = f'~v{i}'
        edges += [(v, nv), (v, X), (nv, X)]
        labels[v]  = f'x_{i}'
        labels[nv] = f'¬x_{i}'

    # Clause gadgets: a 6-vertex OR gadget is connected to the three literal
    # vertices and to F so that 3-coloring exists iff at least one literal is T.
    for ci, clause in enumerate(clauses):
        assert len(clause) == 3
        a, b, c = (f'v{l}' if l > 0 else f'~v{-l}' for l in clause)
        # 6 helper nodes per clause
        h = [f'c{ci}_{j}' for j in range(6)]
        edges += [(a, h[0]), (b, h[1]), (h[0], h[1]),
                  (h[0], h[2]), (h[1], h[2]),
                  (h[2], h[3]), (c, h[3]), (h[3], h[4]),
                  (h[4], F), (h[4], h[5]), (h[5], X)]
    return edges, labels

# Try it on (x1 OR x2 OR x3) AND (NOT x1 OR NOT x2 OR x3)
clauses = [[1, 2, 3], [-1, -2, 3]]
edges, labels = three_sat_to_three_color(clauses, n_vars=3)
print(len(edges), "edges in the reduced graph")
# The graph is 3-colorable iff the formula is satisfiable.
package main

// Go: unary-increment TM + brute-force SAT, mirroring the Python above.

import (
	"fmt"
	"strings"
)

// --- Turing machine (unary increment) ---
type Rule struct {
	newState string
	write    byte
	dir      int
}

func runTM(input string, delta map[string]Rule) (bool, string) {
	tape := []byte(input + strings.Repeat("_", 10))
	head, state := 0, "q0"
	for i := 0; i < 1000; i++ {
		if state == "qA" {
			return true, strings.TrimRight(string(tape), "_")
		}
		if state == "qR" {
			return false, ""
		}
		sym := tape[head]
		key := state + "," + string(sym)
		r, ok := delta[key]
		if !ok {
			return false, ""
		}
		tape[head] = r.write
		state = r.newState
		head += r.dir
		if head < 0 {
			head = 0
		}
		if head >= len(tape) {
			tape = append(tape, '_')
		}
	}
	return false, ""
}

// --- Brute-force SAT (bitmask over 2^n) ---
func evalClause(clause []int, mask int) bool {
	for _, lit := range clause {
		v := lit
		if v < 0 {
			v = -v
		}
		v-- // 0-indexed
		val := (mask>>v)&1 == 1
		if lit > 0 {
			if val {
				return true
			}
		} else {
			if !val {
				return true
			}
		}
	}
	return false
}

func bruteForceSAT(clauses [][]int, nVars int) (bool, int) {
	for mask := 0; mask < (1 << nVars); mask++ {
		ok := true
		for _, c := range clauses {
			if !evalClause(c, mask) {
				ok = false
				break
			}
		}
		if ok {
			return true, mask
		}
	}
	return false, -1
}

func main() {
	// TM: unary increment
	delta := map[string]Rule{
		"q0,1": {"q0", '1', +1},
		"q0,_": {"q1", '_', -1},
		"q1,1": {"q2", '1', +1},
		"q2,_": {"qA", '1', +1},
	}
	ok, tape := runTM("1111", delta)
	fmt.Printf("TM: accepted=%v tape=%q\n", ok, tape) // accepted=true tape="11111"

	// SAT: (x1 OR NOT x2) AND (NOT x1 OR x2 OR x3) AND (NOT x3)
	formula := [][]int{{1, -2}, {-1, 2, 3}, {-3}}
	sat, mask := bruteForceSAT(formula, 3)
	if sat {
		fmt.Printf("SAT: x1=%d x2=%d x3=%d\n", mask&1, (mask>>1)&1, (mask>>2)&1)
	} else {
		fmt.Println("UNSAT")
	}
}

19. Open problems

The famous unresolved questions in this corner of CS:

That's the field. Computability tells you what can ever be done; complexity tells you what can be done cheaply; reductions tell you which problems are secretly the same. The whole edifice is built on a 1936 paper, a few diagonal arguments, and the patient accumulation of results from there. Most of the deepest questions remain open. Welcome to theoretical CS.

See also

Discrete math

Sets, functions, relations, proof techniques. The grammar of everything on this page.

prereq

Algorithmic complexity

Big-O notation, time and space analysis, the complexity of standard algorithms. The "applied" side of section 8.

prereq

Optimization

What you do when the exact problem is NP-hard. LP relaxations, gradient methods, heuristics, metaheuristics.

next

Reasoning models

Modern LLMs that solve SAT-like problems via chain-of-thought search. The link between theoretical hardness and practical AI.

related

AGI

The Church-Turing thesis sets a floor on what any general intelligence — biological or artificial — can compute.

related

20. Further reading

Textbooks

  • Sipser, M. (2012) — Introduction to the Theory of Computation, 3rd ed. The standard undergraduate text. Clear, slow, exhaustive on Turing machines, decidability, and reductions. Read this first.
  • Arora, S. & Barak, B. (2009) — Computational Complexity: A Modern Approach. Graduate-level. Goes deep into PH, PCP, derandomization, circuit complexity. Free draft online.
  • Papadimitriou, C. (1993) — Computational Complexity. The classic complexity textbook, slightly older than Arora-Barak but still excellent for the foundations.
  • Hopcroft, Motwani & Ullman (2006) — Introduction to Automata Theory, Languages, and Computation. The other standard. Heavy on automata and formal languages.

Papers

  • Turing, A. (1936) — On Computable Numbers, with an Application to the Entscheidungsproblem. The original Turing machine paper. Surprisingly readable.
  • Cook, S. (1971) — The Complexity of Theorem-Proving Procedures. The Cook-Levin theorem.
  • Karp, R. (1972) — Reducibility Among Combinatorial Problems. The 21 NP-complete problems.
  • Baker, T., Gill, J. & Solovay, R. (1975) — Relativizations of the P =? NP question.
  • Razborov, A. & Rudich, S. (1997) — Natural Proofs. The second great barrier.
  • Aaronson, S. & Wigderson, A. (2008) — Algebrization: A New Barrier in Complexity Theory.

Surveys and lectures

  • Aaronson, S. — P =? NP survey (2017). A friendly modern overview of why we believe what we believe.
  • Wigderson, A. (2019) — Mathematics and Computation. A book-length view of complexity from a leading practitioner. Free online.
NEXT UP
→ Optimization

Now that you know NP-hard problems are everywhere and exact solutions are out of reach, the next question is how to actually solve them in practice. Linear programming, gradient methods, and the metaheuristic toolbox.