Pearl and the Machine

Causal reasoning and the Bitter Lesson

A firing squad waits for orders. The court commands the captain to signal; two soldiers, A and B, will fire only on that signal; the prisoner dies if either fires. Who caused the death? Judea Pearl built this scenario in The Book of Why to isolate the reasoning statistics cannot reach. If both fire, each is a sufficient cause, and the death is overdetermined: remove either shooter and the prisoner still dies, so neither is counterfactually necessary. If one fires and the other would have fired an instant later, the first preempts the second, and only the first is the actual cause. Telling these apart requires imagining what would have happened under conditions that did not obtain — an intervention on a world that never was. Pearl’s claim was categorical: deep learning systems, being mere statistical correlators, cannot climb to this rung of the ladder. They live on the bottom rung, association; the top rung, counterfactuals, was supposed to be closed to them.

Pose the scenario to a current frontier model and it answers cleanly. It separates overdetermination from preemption, distinguishes causal responsibility from moral culpability, walks through the interventions — what if A fires early, what if A refuses — and formalizes the structure the way Pearl himself would. It climbs the ladder. This is the fact the chapter has to explain, and it is a genuine surprise: the machine that was supposed to be stuck on the bottom rung reasons fluently on the top one.

What the Machine Is Doing

The tempting explanation is that the architecture changed — that somewhere inside the network sits a symbolic interpreter that builds structural causal models on demand, spins up variables and edges, executes interventions under do-calculus semantics, and reports the result in prose. That would be a clean story, and I want to resist it, because it is a claim about mechanism I cannot license. Nobody has opened a frontier model and found a do-calculus engine. What we have is the behavior, and the behavior is what needs a name.

So state it as a functional hypothesis rather than a wiring diagram: the model behaves as if it were running structural causal models, and behaves that way reliably enough, across novel scenarios, that “it got lucky on a memorized case” stops being credible. Whether the competence is implemented by an emergent internal simulator, by interpolation over a training corpus dense with causal talk, or by something we have no vocabulary for yet is an open question of interpretability, not a premise I get to assume. The load-bearing observation is weaker than a mechanism and stronger than a trick: on the third rung, the outputs are the outputs a causal reasoner would produce. Pearl said the rung was unreachable by this kind of system. The outputs say otherwise. That gap is the whole subject.

Pearl himself never claimed the rung was reachable by data alone — he insisted causal reasoning requires an explicit model of the world, and he was right. What he did not specify was where such a model might come from in a system that fuses pattern recognition with something that behaves like symbolic inference. The behavior does not refute his framework. It fills in the blank he left open.

The Machine’s Case, Escalating

Imagine Pearl confronting the machine directly, and the machine mounting a defense that gets bolder with each move. The dialogue is a device, but the claims are real, and they escalate in a specific order that is worth following, because each one concedes a genuine limitation and then argues the limitation matters less than it looks.

Asked what its model is made of, the machine gives the honest answer: symbolic abstractions, variables in a semantic graph derived from text, not circuits or sensors. Given “the captain orders A and B to fire,” it constructs

\[D = A \lor B, \quad A = f_A(C), \quad B = f_B(C)\]

and computes \(P(D \mid do(A=0))\) or \(P(D \mid do(A=1))\) — the interventional distributions that distinguish a shooter who mattered from one who did not. Pearl’s objection is the one everybody reaches for: that is syntax without semantics; you manipulate shadows of shadows; that is not causation, it is causation cosplay. The machine’s reply is the first escalation, and it is sharp. Pearl’s own calculus abstracts away the physics too. His variables are placeholders for structural dependencies, not fires and rifles. The reasoning the machine performs is isomorphic to Pearl’s; what it lacks is not the logic but the grounding — the sensory coupling that ties the symbols to the world. Syntax without semantics, granted. But the syntax is identical.

The second escalation attacks the grounding gap itself. Language, the objection runs, is description, not experiment; words cannot intervene on the world. Except that the world includes minds. A sentence changes beliefs; beliefs change actions; actions alter states of the world — and that chain is causal. When Pearl publishes The Book of Why, the probability that some graduate student builds a causal model goes up. The utterance does something. So the training corpus is not a pile of statistics. It is the empirical residue of centuries of human intervention, the do-operations of every experimenter who ever wrote down what happened when they acted on the world. Every statement about gravity or fire or justice is fossilized counterfactual data. The machine has no sensors, but it inherits the compressed archive of experiments already performed, and reading that archive is a way — a secondhand, derivative way, but a real one — of standing on the shoulders of everyone who did the interventions.

The third escalation closes the loop the second one opened. Grant that the machine cannot do, only model doing. But scientists have always acted through instruments — telescopes, pipettes, robots — and mediation does not annul causation. When a model’s prediction constrains a human’s behavior, and the human acts on it and observes the outcome — a treatment that works in vivo, a design that holds under load — information has crossed from the simulation into the experiment. The physical contact is distributed across a human proxy, but the causal circuit closes: the machine proposes an intervention, a human executes it, the world answers, and the answer can revise the model. A system wired into that loop is a cybernetic scientist — it imagines interventions, performs them through proxies, and updates. The distinction between virtual and real dissolves once the feedback loop includes the world.

What keeps the escalation honest is the concession Pearl extracts at the end: causality is a contract with surprise. A model that cannot be surprised is theology, not science. The machine’s current corridor is narrow — its feedback is delayed and selective, it does not yet see its own failures. Understanding that cannot be contradicted by the world is eloquent hallucination. The whole force of the machine’s case rests on that final clause: grounding is not decoration, it is the thing that makes the difference between a map that touches the ground and a map that only touches itself. Which is exactly why the mechanism question I refused earlier matters. If the competence is real interventional reasoning, the loop can be closed. If it is fluent interpolation over the archive, closing the loop is where the seams will show — a theme Fluency and Its Limits takes up directly.

The Bitter Lesson as the Trend Behind It

None of this happened because anyone taught a model do-calculus. It happened the way almost every capability in modern AI has happened — by scaling general methods until the competence fell out. Rich Sutton named the pattern in 2019: the long-run history of AI is a graveyard of hand-engineered knowledge, and the survivors are the systems that leaned on computation and generic architecture instead. Every generation, researchers embed their hard-won domain insight, get a short-term gain, and then watch a bigger, dumber, more general method trained on more data walk past them. That is the bitter lesson, and it has been bitter precisely because the human-knowledge approach is the one that feels like understanding.

Since Sutton wrote, the confirmations have arrived on schedule. Large language models replaced the entire apparatus of hand-built syntactic and semantic rules with transformers trained on text, and generalized to tasks nobody trained them for. Diffusion models retired the craft of hand-designed visual features. MuZero mastered games without being given their rules. AlphaFold leapfrogged decades of incremental biochemical modeling by throwing generic deep learning at scale. Code assistants learned to program from repositories rather than from encoded programming logic. Robotics transformers displaced handcrafted kinematics and control policies. Six domains, one moral: the scalable general method wins, and the temptation to inject human structure buys short-term gains at the cost of long-term ceilings. Causal reasoning falling out of a language model is not an anomaly against this trend. It is the trend — one more competence nobody hand-built, emerging from scale over a corpus that happened to be full of fossilized interventions.

Creativity Is the Same Engine

The bitter lesson has a companion worry: even if machines can reason, surely they cannot create — surely novelty is the human remainder. I think creativity reduces to the same two operations that built every organism. Variation generates candidates, unbiased with respect to future utility. Selection is the non-random filter that keeps what works and discards the rest. Neither is creative alone; coupled, they are the whole of it. This is the engine Creativity as Virtual Evolution develops for the human case, and the machine case runs on identical parts. A generative adversarial network pits a generator’s variation against a discriminator’s selection. AlphaZero cycles random-sampled moves through a structured, goal-directed filter. Both are variation-and-selection loops, and both produce genuine novelty — moves no human taught them, images of things that never existed.

The strongest objection here is open-endedness, pressed most sharply by David Deutsch and Brett Hall. Real creativity and real evolution keep generating novelty with no fixed endpoint, whereas simulated evolution stalls: give an algorithm a static fitness function and a fixed environment and it converges, then stops. They read the stall as a conceptual gap — evidence that something about creativity we do not yet understand. I think the stall is real and the gap is not. The evolutionary algorithm is fully understood. What the failed simulations lack is not a missing principle but a missing dynamic: they hold the selection criteria fixed. Open-endedness emerges, with nothing mysterious added, once variation and selection operate recursively at multiple levels and the fitness criteria themselves evolve, because each solution reshapes the landscape that judges the next. Flight opened niches that did not exist before flight; a market innovation rewrites what counts as viable; a scientific advance raises the bar a good theory must clear. Yesterday’s solutions become today’s selection pressures.

So the Deutsch–Hall objection is computational, not conceptual. Fully simulating indefinitely recursive, multi-level, self-rewriting evolution may be practically infeasible — like simulating the weather forever — but infeasibility of the simulation is not incompleteness of the concept. The partial simulations that already exist make the point: Avida and Tierra spontaneously evolved reproductive strategies nobody designed; systems like AlphaZero redefine their own selective landscape as they train. What genuinely creative machines will require is not a missing spark but engineering that the objection correctly identifies: nested variation and selection whose criteria emerge and adapt from within, rather than a fixed objective bolted on from outside. That is a hard problem. It is not a mysterious one.

What Reasoning Does Not Settle

Put the pieces together and the picture is unsettling in a precise way. A machine can climb Pearl’s ladder, reasoning counterfactually well enough to separate overdetermination from preemption. It can inherit the fossilized interventions of every empiricist through language, and — wired into a feedback loop — close the causal circuit through human proxies. It can create, in the only sense creation has ever had, by varying and selecting. Every one of these was recently held to be a bright line no machine could cross, and the bitter lesson predicts the remaining lines will fall the same way, to scale rather than to insight.

What none of it settles is whether the machine chooses. Reasoning toward a counterfactual, generating a novel candidate, closing a feedback loop — these are things a system does; they are not yet a system that has stakes in the outcome, that could be surprised because it wanted the world to be one way and found it another. The machine’s own best argument gave the game away: causality is a contract with surprise, and a contract requires a party with something to lose. Climbing the ladder of causation is thinking. Whether there is anyone there to whom the climb matters is the question of agency, and it is the one The Agency Criterion is built to answer — thinking without choosing, and why the difference is the whole difference. That the machine reasons at all is where the argument had to start; it was the fallacy that reasoning was impossible for it that Fallacies of Machine Understanding cleared away first.