Deciding Under Uncertainty

Model uncertainty, tail risks, and policies

A stranger stops me on the street and announces that he is a wizard from outside the simulation. If I hand over my wallet, he will conjure \(10^{100}\) years of bliss for \(10^{100}\) sentient beings; if I refuse, he will inflict the corresponding torment. My Credence that he is telling the truth is minuscule — but it is not zero, and no honest epistemology can make it zero. So the expected-utility calculation runs: a vanishing probability times an astronomical payoff yields an enormous product, and the arithmetic instructs me to hand over the wallet — and, by the same logic, to hand over the next one, to anyone willing to name a bigger number. This is Pascal’s Mugging, and any decision theory that pays the mugger has refuted itself.

The mugging resembles a problem that is three centuries old. In the St. Petersburg game, a fair coin is flipped until it first lands heads; if that happens on the \(n\)th flip, the game pays \(2^n\) dollars. Its expected monetary value diverges. Expected utility theory does not therefore say to stake everything: with diminishing marginal utility, finite wealth, counterparty limits, and bounded payoffs, the value can be finite. The paradox warns against confusing money with utility and extrapolating a model beyond its feasible domain.

Pascal’s Mugging adds a different problem: the tiny probability and enormous payoff are both generated by an untrusted story. Expected utility does not uniquely prescribe how to represent such model uncertainty. The decision can be dominated by the inputs we understand least, so sensitivity and robustness matter more than a nominal product.

What Physics Does With Misbehaving Tails

Physics offers an analogy, but not a derivation. Effective field theories specify a scale below which omitted high-energy degrees of freedom can be represented through controlled corrections. Their cutoffs concern domains of validity and renormalization, not a general license to ignore low-probability events. Decision theory needs its own justification for handling poorly modeled tails.

One tempting response is a hard probability cutoff, which I previously called Effective Decision Theory. A cutoff is easy to state but difficult to defend. It can be discontinuous, dynamically inconsistent, and dangerously insensitive to genuine catastrophic risks just below the threshold. The better lesson is to use robust decision methods that represent uncertainty about models and utilities rather than silently converting every small number to zero.

Why a Cutoff Is Justified

Ignoring small probabilities sounds like an ad-hoc mutilation of the mathematics, so it matters why the cutoff is principled rather than squeamish.

The probabilities at the far tails are not like the probabilities in the middle. A fair coin’s 50% assignment is supported by symmetry, mechanics, and repeated calibration; conditional on the QBU module, a specified heads sector may also have Measure one half in the sense given in Measure and Credence. When I assign the wizard’s story a probability of \(10^{-50}\), nothing comparably secure is happening. That number is not a measured feature of the world; it is an artifact squeezed out of models I trust far less than I would need to in order to take its fiftieth decimal place seriously. Extremely low probabilities come bundled with extreme epistemic uncertainty: modeling ambiguity, unreliable data, hypotheses (like the wizard’s) constructed precisely to evade every check the model could run. At the tails, the number feeding the calculation is pure Credence resting on assumptions that have themselves never been tested at anything like that resolution — the neighborhood of what the varieties of uncertainty calls Knightian territory, where no trustworthy model exists to deliver a probability at all. Treating such a number as a precision input to a multiplication by \(10^{100}\) is not rigor. It is laundering ignorance into a demand on my wallet.

A robust policy does not deny that tail events are possible. It acknowledges that the probability model itself may be uncertain and refuses to let one arbitrary point estimate dominate unchecked. The agent need not distort Credence: precaution, bounded utility, resource limits, worst-case constraints, and the value of further information belong in the decision rule.

Three principles govern the revised framework:

1. Represent uncertainty explicitly. Use ranges, alternative models, and sensitivity analysis when a point probability is not earned.
2. Test robustness. Prefer actions that perform acceptably across plausible models, especially when losses are irreversible or catastrophic.
3. Separate belief from policy. Precaution, bounded utility, resource limits, and the value of information belong in the decision rule; they should not be disguised as altered Credence.

Neglecting tiny terms can be a useful bounded-computation heuristic, and an explicit approximation is easier to criticize than a hidden one. It should not be elevated into a universal probability threshold. The appropriate response depends on why the tail is unstable: uncertain likelihoods call for model sensitivity, unbounded payoffs call for defensible utility modeling, adversarial stories call for source and mechanism penalties, and genuine catastrophic risks call for robust precaution.

A Paradox the Cutoff Cannot Touch

The cutoff repairs expected utility where the probabilities are degenerate. But there is a second, deeper failure mode that appears at probabilities as tame as 0.99, and it demands a different repair — not to the probabilities, but to our answer to the question what is being chosen?

Newcomb’s paradox: a highly reliable predictor has already filled an opaque box with either $1,000,000 or $0. If it predicted you would take only the opaque box, it put in the million. If it predicted you would take both boxes, it left the opaque box empty. Beside it sits a transparent box holding $1,000. You may take the opaque box alone, or both.

The two-boxing argument says: the money is already in the box or it isn’t. If it’s there, taking both boxes gets you $1,001,000 instead of $1,000,000. If it’s not, taking both gets you $1,000 instead of nothing. Either way, two-boxing pays exactly $1,000 more. Two-boxing dominates.

The argument pits causal, evidential, and functional approaches against one another. The unit of choice is one real issue, but the dispute also concerns which counterfactual dependencies a decision rule should preserve. Evaluate an act while holding box contents fixed and two-boxing dominates. Evaluate policies using the stipulated predictor correlation and one-boxing has the higher expected payoff when reliability is high. Neither framing becomes neutral merely by being called obvious.

The Unit of Rational Choice

The dominance argument holds fixed a condition that the problem itself defines as policy-dependent. The contents of the opaque box are not caused by your present hand motion — true. But they are not independent of your decision in the only sense that matters, because they are linked to the policy you instantiate. The predictor did not reward a last-second twitch. It rewarded being the kind of agent it predicted would one-box. That is the entire structure of the problem, and once it is stated plainly the paradox is already dying.

The wrong question is: given fixed box contents, which immediate act pays more?

The policy-level question proposed here is: from my current vantage, which available policy induces the most favorable modeled distribution of outcomes?

If the Quantum Branching Universe (QBU) is adopted, the modeled outcome distribution may be represented with Measure over record sectors. The policy argument itself does not require QBU: ordinary conditional payoff distributions are enough. What matters is whether policy-level counterfactuals are the appropriate unit for the problem.

Run the calculation at that level. Let

\[\pi_1 = \text{one-box}, \qquad \pi_2 = \text{two-box}.\]

If the predictor’s reliability is \(p\), the payoff distributions the two policies induce are

\[\begin{aligned} \mu(\$1{,}000{,}000 \mid \pi_1) &= p \\ \mu(\$0 \mid \pi_1) &= 1-p \\ \mu(\$1{,}000 \mid \pi_2) &= p \\ \mu(\$1{,}001{,}000 \mid \pi_2) &= 1-p \end{aligned}\]

so the expected utilities of the policies are

\[\begin{aligned} EU(\pi_1) &= p \cdot 1{,}000{,}000 \\ EU(\pi_2) &= p \cdot 1{,}000 + (1-p)\cdot 1{,}001{,}000. \end{aligned}\]

One-boxing is the better policy whenever

\[p \cdot 1{,}000{,}000 > p \cdot 1{,}000 + (1-p)\cdot 1{,}001{,}000,\]

which simplifies to

\[p > 0.5005.\]

If the predictor is even slightly better than a coin flip, one-boxing wins. At 99% reliability the result is absurdly lopsided: one-boxing expects $990,000, two-boxing expects $11,000. Notice that expected utility theory itself delivers this verdict without complaint — no exotic decision rule was needed. All that changed is the unit the expectation ranges over: policies and the branch-weights they induce, not hand motions inside a frozen snapshot.

No Backward Causation Required

Nothing supernatural is happening here. Your present choice does not reach into the past and rewrite the box’s contents. The dependency is structural, not retrocausal: the predictor’s earlier action and your later decision covary because both track the same underlying policy-pattern. The predictor succeeds by modeling you, and what it models is not a future twitch but a standing disposition.

This is where the two-boxer goes wrong. He imagines he can keep the favorable consequences of being the kind of agent the predictor rewards, then swap in the local act of a different kind of agent at the last moment. He cannot. That is not a coherent counterfactual — it severs the very dependency the setup is built to expose. He wants to inhabit the branch-family in which the predictor filled the box while acting as the sort of agent for whom the predictor would have left it empty. That is not cleverness. It is incoherence disguised as opportunism. Two-boxing is what happens when local greed masquerades as rationality.

And this is why Newcomb matters far beyond the magic box. It is a stripped-down model of ordinary strategic life in any world containing prediction, reputation, commitment, signaling, or coordination. The two-boxer’s mistake reappears whenever someone tries to enjoy the benefits of being trusted, legible, or cooperative while defecting at the final moment and pretending the earlier structure can be held fixed. It cannot. A policy-sensitive world rewards coherent policy, not locally greedy gestures — and every world containing other modelers of you is policy-sensitive.

Expected Utility, Conditionalized

Assemble the revisions and a single pattern emerges. Expected utility remains useful under stated assumptions about probabilities, utilities, feasible actions, and counterfactuals. Where tail estimates are unstable, represent model uncertainty and test robust alternatives rather than trusting a nominal product. Where outcomes depend on predicted dispositions, state whether acts or policies are the unit of evaluation and acknowledge the competing counterfactual models. Pascal’s Mugging stresses the first set of assumptions; Newcomb’s problem stresses the second.

The result is a picture of rational agency that matches the rest of this volume. Epistemic assessment comes first: state what the probabilities mean, how uncertain the model is, and—conditional on QBU—whether Measure supplies any likelihoods. Decision then adds utilities, constraints, robustness, and the appropriate unit of choice. Keeping those stages distinct lets an agent take precaution without falsifying belief and choose policies without pretending every counterfactual dispute has vanished.