Neuro-Symbolic AI

Marrying the two oldest traditions in AI: neural networks that perceive and learn, and symbolic systems that reason and prove. Neural nets are fast, fuzzy, and statistical; symbolic systems are slow, precise, and logical. Neuro-symbolic AI is the bet that you need both.

Prereq: first-order logic, neural nets Time to read: ~20 min Interactive figures: 1 Code: Python, Prolog, PyTorch

1. Two schools of AI

AI has always had two traditions. The symbolic school, dominant from the 1950s through the 1990s, represents the world as facts and rules and does inference by manipulating discrete symbols. Expert systems, Prolog, Cyc, theorem provers, and chess engines like Deep Blue all belong here. Strengths: perfect logical consistency, interpretable reasoning, can handle problems that need proofs. Weaknesses: brittle, can't learn from raw data, every fact has to be hand-encoded, no graceful degradation under noise.

The connectionist school — neural networks — is the opposite. Strengths: learns from raw data, generalizes, handles noise and uncertainty, scales with compute. Weaknesses: can't reliably do long chains of precise reasoning, hallucinates, can't guarantee a proof is correct, nobody knows why it does what it does.

For sixty years these schools barely spoke. The deep learning revolution of the 2010s looked like connectionism had won outright. But by 2023 a few cracks had formed in that story: LLMs were great at language and terrible at arithmetic. They could cite a theorem and then misapply it. They would confidently output a math proof with a step that didn't follow. No amount of scale seemed to fix this class of error, because it's not a scale problem — it's a reasoning model problem.

Neuro-symbolic AI is the attempt to put the two schools back together: keep the neural nets for perception and pattern matching, keep the symbolic engines for verified reasoning, and find the right interface between them.

2. Why combine them?

Some tasks have a clean split between "a hard perception problem" and "a hard reasoning problem":

The bet is: neither pure system handles the whole thing, but a hybrid does. And the interface between them — how the neural net "talks to" the symbolic engine — becomes the central design problem.

3. Integration patterns

There are roughly four ways to stitch the two halves together:

  1. Neural front-end, symbolic back-end. The neural net converts raw input (pixels, text) into a structured representation — a parse tree, a scene graph, a set of facts. A symbolic engine then runs inference over that. Most "LLM + calculator" and "LLM + SQL" setups are this pattern.
  2. Symbolic loss / constraint layer. You add a logic-based penalty to the neural training loss: the network's outputs should satisfy first-order constraints (e.g., "the predicted parent-child relation graph must be a tree"). Differentiable relaxations of logic let you backprop through this.
  3. Neural guidance for symbolic search. A symbolic search algorithm (MCTS, SAT solver, theorem prover) explores an enormous space. A neural network prunes it, suggesting promising moves or subgoals. AlphaGo, AlphaZero, and AlphaGeometry all live here.
  4. Knowledge-graph-augmented neural. Link the neural model to an external KG and let it query at inference time. Similar to RAG but with structured facts instead of free text.

4. Differentiable logic

For patterns that train end-to-end, you need to make logic differentiable. The standard trick is to replace boolean operators with smooth real-valued relaxations. For truth values $a, b \in [0, 1]$:

$$\text{AND}(a, b) = a \cdot b$$
$$\text{OR}(a, b) = a + b - a \cdot b$$
$$\text{NOT}(a) = 1 - a$$
$$\text{IMPLIES}(a, b) = \min(1, 1 - a + b)$$

Fuzzy logic operators (Gödel / product t-norms)

$a, b$
Truth values in $[0, 1]$ instead of $\{0, 1\}$. Interpreted as the probability or the degree to which a proposition is true.
$\text{AND}(a, b) = ab$
If both $a$ and $b$ are near 1, the product is near 1. If either is near 0, the product is near 0. Reduces to classical AND when $a, b \in \{0, 1\}$.
$\text{OR}(a, b) = a + b - ab$
The "probabilistic OR" — the probability that at least one of two independent events is true. Reduces to classical OR on $\{0, 1\}$.
$\text{NOT}(a) = 1 - a$
Obvious. Keeps the range in $[0, 1]$.
$\text{IMPLIES}(a, b)$
Łukasiewicz implication, a standard choice. When $a = 1$ and $b = 0$ it gives 0 (implication fails); everywhere else it returns something $> 0$.

Why this matters All of these are differentiable w.r.t. $a$ and $b$. You can plug them into a neural network's loss function as "soft constraints" that the output should satisfy. Training pushes the network to not just minimize prediction error but also obey the logical rules. Examples: Semantic Loss (Xu et al. 2018), DeepProbLog (Manhaeve et al. 2018), and Logic Tensor Networks (Serafini & d'Avila Garcez 2016).

5. Interactive fuzzy logic

Drag the sliders for $a$ and $b$ to see each operator's output. The lines are the classical boolean values (0 or 1); the shaded curves are the fuzzy relaxations. Notice how fuzzy AND smoothly interpolates instead of switching at the thresholds.

a: 0.80 b: 0.40

Fuzzy logic operators. Each bar is one operator; the orange fill is the fuzzy value, the dashed line is the classical boolean value.

6. Case study: AlphaGeometry (2024)

The most striking neuro-symbolic system of the last few years is DeepMind's AlphaGeometry. It solves IMO (International Mathematical Olympiad) geometry problems at roughly silver-medal level. The architecture is a canonical example of pattern 3 — neural guidance for symbolic search:

Neither half works alone. The symbolic engine can't invent the creative extra constructions. The language model can't reliably execute the multi-step logical chain. Together they solved 25 out of 30 Olympiad problems — the previous best was 10 out of 30. This is the clearest evidence yet that for some classes of reasoning, hybrid is not just a nice-to-have but a necessary architecture.

7. Source code

Three examples: fuzzy logic in NumPy, an LLM + calculator pattern, and a Prolog-style rule in Python.

neuro-symbolic · integration patterns 
import torch

def fuzzy_and(a, b):      return a * b
def fuzzy_or(a, b):       return a + b - a * b
def fuzzy_not(a):         return 1 - a
def fuzzy_implies(a, b):  return torch.clamp(1 - a + b, 0, 1)

# Semantic loss: penalize predictions that violate a known constraint.
# E.g., a multi-label classifier where labels "cat" and "dog" are mutually exclusive.
def semantic_loss(probs):
    p_cat, p_dog = probs[..., 0], probs[..., 1]
    # NOT (cat AND dog) — the constraint as a soft truth value
    violation = fuzzy_not(fuzzy_and(p_cat, p_dog))
    return -torch.log(violation + 1e-9).mean()

# Total loss: cross-entropy for accuracy + λ * semantic loss for logical consistency

import re, sympy

def llm_plus_calculator(llm, question):
    # 1. Ask the LLM to produce a SymPy expression that solves the problem.
    prompt = f"""Problem: {question}
Write a one-line SymPy expression whose value is the answer.
Expression:"""
    expr_str = llm(prompt).strip()

    # 2. Evaluate it symbolically — this is the verified step.
    try:
        result = sympy.sympify(expr_str).evalf()
    except sympy.SympifyError as e:
        return f"Invalid expression: {e}"

    # 3. Feed the verified answer back to the LLM for natural-language explanation.
    explain = llm(f"The answer is {result}. Explain briefly.")
    return result, explain

# This pattern guarantees the numerical answer is correct even though the LLM
# did the hard perception of parsing the problem.

# Tiny forward-chaining rule engine. Not Prolog, but close in spirit.

class KB:
    def __init__(self):
        self.facts = set()
        self.rules = []                    # each rule: (premises, conclusion)

    def tell(self, fact):  self.facts.add(fact)
    def rule(self, premises, conclusion):
        self.rules.append((premises, conclusion))

    def forward_chain(self):
        changed = True
        while changed:
            changed = False
            for premises, conclusion in self.rules:
                if all(p in self.facts for p in premises):
                    if conclusion not in self.facts:
                        self.facts.add(conclusion)
                        changed = True

kb = KB()
kb.tell("human(socrates)")
kb.rule(["human(X)"], "mortal(X)")   # (pretend X unifies)
kb.forward_chain()
# Neural half: an LLM extracts facts ("Socrates is a human") → feeds them to kb.
# Symbolic half: kb chains rules forward to "mortal(socrates)".

8. Summary

Further reading

  • Garcez, Lamb, Gabbay (2009) — Neural-Symbolic Cognitive Reasoning. The classical reference.
  • Manhaeve et al. (2018) — DeepProbLog: Neural Probabilistic Logic Programming.
  • Xu et al. (2018) — A Semantic Loss Function for Deep Learning with Symbolic Knowledge.
  • Trinh et al. (2024) — Solving Olympiad Geometry without Human Demonstrations (AlphaGeometry).
  • Marcus (2020) — The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence. Polemical but widely cited.
NEXT UP
→ Edge & On-Device AI

Once a model is good, the next frontier is making it small enough to run anywhere. Quantization, distillation, pruning — the algorithms that put 7B models on your phone.